Innovation in project management education-let’s get serious!

More educators use serious games (e.g., games where the primary objective is learning rather than enjoyment) to enhance learning due to benefits such as improved understanding and engagement. However, using serious games within project management education is not well understood. The aim of this research is to investigate project management serious games in higher education: i) determine the extent of gamification in PMI-accredited project management programs, and ii) survey university students about their experiences playing project management serious games. Two separate mixed-method studies reveal insights about serious games in higher education and where innovations may be leveraged. Traditional statistics were used to analyze quantitative data, and coding was used to analyze the qualitative data. The results from a global survey of ten PMI-accredited university programs suggest that serious games are embryonic but promising. A case study at one university reveals that students enjoy learning through games but caution against using games to formally assess students’ learning. The paper concludes with recommendations for further research and development.


Introduction
Few will argue the importance of project management with an expected 87.7 million people working in projects by 2027 (PMI, 2017). The Project Management Institute (PMI, 2017) predicts a significant talent gap that could result in a potential loss in GDP of USD 207.9 billion through to 2027 for the 11 countries analyzed. The 2017 figures quoted above suggest that education providers are likely to see an increase in demand for project management services, so innovative approaches to training and upskilling new entrants seem essential.
Educators might add "serious games" to their innovative teaching techniques repertoire to meet learning objectives. Surprisingly, this term is relatively rare in project management research even though a considerable body of literature has emerged, illustrating the benefits of games and gamification for educational purposes (Mekler, et al., 2017). Simply, the focus of serious games is on learning rather than on having fun, enjoyment, or entertainment. For example, simulations, challenges, and role-playing are examples of serious games in game-based education. Our interest in game-based learning to potentially improve project management learning is the rationale for undertaking this study.

Literature review
We find in our literature review that serious games are widely used in higher education, but mostly silent about educators using serious games to teach project management. In their critique of project management education, Winter, et al. (2006) suggested that educators do more to prepare students. Córdoba and Piki (2012) found that a group-based approach with real-life components can motivate and engage students. Ramazani and Jergeas (2015) called for an educational environment that exposes students to project situations. Game-based learning has the benefit in that it can approximate these workplace challenges (Hung, 2017). We examine game-based learning: i) technical aspects, ii) gamification, and serious games as classified by Subhash and Cudney (2018). Gamification is adding game design elements in non-game contexts (Deterding, et al., 2011).

TECHNICAL ASPECTS
Project management serious games and gamification include technical aspects like game engines and mobile platform design that can impact learning and motivation. Özhan and Kocadere (2020) investigated the design process and found there were positive effects on undergraduate motivation in a gamified learning environment. Ortega-Arranz, et al. (2019) researched the efficacy of rewards in Massive Open Online Courses (MOOCs). Höllig, Tumasjan, and Welpe (2020) studied leaderboard design related to the role of competitiveness in learning. Further, Imran (2019) showed a positive relationship between digital badges and student engagement in gamified e-learning systems. Such technical aspects of gamification and serious games, although imperative, are beyond the scope of this study. (Mulcahy, Russell-Bennett and Iacobucci, 2020), smart cities to reduce traffic (Cellina, et al., 2020), marketing applications to increase customer commitment, willingness to pay, and to provide customer referrals (Wolf, Weiger and Hammerschmidt, 2020). Enterprise Resource Planning (ERP) systems are expensive and challenging to implement; however, gamification elements have been added to the ERP lifecycle to increase benefits with an earlier payback (El-Telbany and Elragal, 2017).
Educators use gamification in higher education in engineering programs (Alhammad and Moreno, 2018). Adding gamification elements to student quizzes has improved test scores (Lin and Windasari, 2019). Motivation and academic achievement are enhanced with Kahoot software in engineering subjects (Fuster-Guilló, et al., 2019). Game elements encouraged students to complete out-of-classroom activities in a flipped classroom model (Huang and Hew, 2018). Gamification elements were added to surveys to improve research response rates (Triantoro, et al., 2019). Such examples of gamified behavioral change are also beyond the scope of this study.

SERIOUS GAMES
Game-based-learning is playing games to achieve learning objectives (Gatti, Ulrich and Seele, 2019) and is the focus of this research. While playing these games might be fun, entertainment is not the primary purpose (Müller, Reise and Seliger, 2015). Serious games include i) puzzle games, ii) adventure games, iii) simulation games, iv) strategy games, and v) edutainment (Calderón, Ruiz and O'Connor, 2018). In this study, we review serious games used in higher education and specifically in science, technology, engineering, and mathematics (STEM) fields, and project management.
The literature reveals an increasing number of empirical studies about serious games in higher education reporting positive outcomes, including improved participation and engagement (Law, 2019), motivation (Alsawaier, 2018), cooperation and teamwork (Geithner, et al., 2016;), and decreased cognitive loading and learning anxiety (Alsawaier, 2018).
There are challenges, too; some have criticized games and gamification approaches that focus on badges, levels, or leaderboards rather than learning (Kapp, 2012). Robertson (2010) criticized gamification as 'taking the thing that is least essential to games and representing it as the core of the experience.' In her view, such approaches would be better described as 'pointsification' and distract from the primary purpose of learning through games. Koivisto and Hamari (2014) argue that gamification brings only short-term benefits. Christy and Fox (2014) cautioned that serious games could bring adverse effects from social comparison and competition. Dominguez, et al. (2013) found challenges in evaluating student performance in their gamified course. Rumeser and Elmsey (2018) cautioned that some students have learning styles that prefer other teaching methods (e.g., step-by-step instructions) rather than game-based learning.
Nevertheless, there is much evidence of benefits derived from using serious games in higher education. For example, first-year students learned about using library resources through games to develop critical mindsets (O'Brien and Pitera, 2019). Games were used to teach sustainability (Gatti, Ulrich and Seele, 2019), entrepreneurship (Aries, et al., 2020) and, leadership (Sousa and Rocha, 2019). Nursing students developed decision-making skills through games (García-Viola, et al., 2019), and pharmacy and osteopathic students developed stronger interprofessional relationships by playing games (Boylan, et al., 2020).
Serious games have a strong following in the STEM disciplines. Engineering students played games to learn about lean construction practices (Hamzeh, et al., 2017), sustainable leadership (Müller, Reise and Seliger, 2015), and manufacturing practices (Despeisse, 2018). Information technology students played serious games to learn Adobe Photoshop (Park, et al., 2019), and systems analysis skills (Su, 2016). Serious games in software engineering subjects are increasing in prevalence where students use games to learn project management tools and techniques (Lui, Lee and Ng, 2015;Maratou, Chatzidaki and Xenos, 2016;Calderón, Ruiz and O'Connor, 2017;Calderón, Ruiz and O'Connor, 2018). Calderon and Ruiz (2015) published a systematic literature review of how serious games are used to teach project management to software development students.
Project management programs in higher education have started to use serious games in their subjects. A project management simulation game to plan a summer party assesses 15 project management technical and behavioral competencies based on Prince2 TM (González-Marcos, Alba-Elías and Ordieres-Meré, 2016). Technology management students played a project simulation game to bring a high-tech product to market using project management tools and processes (Law, 2019). Construction and engineering students applied their project management skills in engineering, procurement, and construction management (EPCM) projects (Miettinen, et al., 2016). Construction and engineering students learned project management by planning their project, then "challenging" the robustness of other student project plans (Misfeldt, 2015). Some serious games focus on developing specific project management learning outcomes like requirements management (Seager, et al., 2011), project management decision-making (Rumeser and Emsley, 2019a), project scheduling (Rumeser and Emsley, 2019b), and program scheduling (Rumeser and Emsley, 2018). Table 1 lists some of the more recent project management simulation games cited in the literature.
This study focuses on simulation games since project management simulation games can approximate reality, with the absence of negative consequences like project failure (Calderón and Ruiz, 2015). In some studies, students' project management competence is assessed based on their game score. For example, there are many ways to measure game performance: i) stakeholder satisfaction, ii) time, iii) quality, iv) cost, and v) team morale (Lui, Lee and Ng, 2015). We can also measure performance by cost, time, human resource management, etc. based on the ISO 21500 Project Management framework (Calderón, Ruiz and O'Connor, 2018) or the Prince2 TM framework (González-Marcos, Alba-Elías and Ordieres-Meré, 2016). Other researchers assessed project management serious game elements such as game stability and usability (Maratou, Chatzidaki and Xenos, 2016), or student feedback in terms of i) team report, ii) peer evaluation, and iii) individual reflection (Law, 2019). Thus, game-based learning is becoming more common in higher education due to benefits like improved student engagement. While games are common in software project management education, we know little about serious games in project management education.

Method and results
The primary purpose of this research is to investigate project management serious games in higher education. This research is guided by two questions:

#1
To what extent are PMI-accredited programs in higher education using project management serious games?
#2 What is the learning experience evidenced by students who played project management serious games as part of their educational experience?
Our research proceeded in two phases to answer our two research questions: a global study of project management educators, followed by a case study of students' learning from a project management simulation game (see Table 2). The case study leveraged open-ended questions to understand the students' experiences with simulation. In Phase I, "gamification" was used as an umbrella term, and "game elements," "games," and "serious games" were used interchangeably. However, the focus in Phase II was on serious game simulation.

GAMIFICATION GLOBAL STUDY
In Phase I, 56 universities globally accredited by PMI were invited to participate in this research, and ten universities participated.
Participants were asked eight questions about serious games in project management, and the results are summarized in Table 3: 7. If not, why do you think using elements of gamification will not be useful in your program? 8. Would you be willing to participate in a subsequent semi-structured interview?
Each participant had a slightly different understanding of gamification. Five participants use gamification in their programs (3x America, 1x Australia, 1x Latin America). Our research participants also reported a lack of knowledge and implementation struggles with gamification as barriers to using games in the classroom. Table 3 Phase I metadata Two universities reported using digital badges to gamify their project management programs. One university awards performance badges within a single subject, whereas the other grants badges over the whole degree for each PMBOK ® Guide knowledge area. A second finding is that some instructors award badges immediately after the student performed the assignment task, while other badges were awarded upon graduation. Both instructors try to achieve similar goals with digital badges: motivate the students to participate and to achieve the learning outcomes. Our research participants reported that their students seem to be more motivated, with higher voluntary attendance rates in additional pop-up classes. However, one of the participants indicated that underperforming students lose interest in badges very quickly.
One participant allowed their students to resubmit assignments after receiving feedback. The participant described this opportunity as having multiple lives in a computer game where the player can retry a level if he failed. It has been his experience that most students take the opportunity to resubmit an assignment after they have received feedback.
Simulations are classified into two broad categories: i) computer-based (commercial offthe-shelf and custom, in-house developments) and ii) non-computer-based simulations (e.g., role-playing). Four research participants reported they used computer-based simulations, including SimulTrain® and Sim4Projects. Only one academic developed a custom simulation called Risky Business, where students proceed through a series of decision-making scenarios. SimulTrain® and Sim4Projects both emulate midsize projects where students plan, monitor, and control project activities. SimulTrain® is offered as an all-day extracurricular simulation challenge.
In other simulations, lecturers give a grade based on individual or group reflection. Students prepare an interim report summarizing project progress and their plans for project completion. The lecturer plays the project sponsor and debriefs the students on what went well and areas for improvement. At the end of the simulation, students give another presentation: the final status report, a project audit report, and their reflections on the game-based learning experience. One participant reported that students are very positive towards role-playing even though it provided both challenges and uncertainties. Another university uses role-playing to simulate a negotiation. Students worked in groups of four and prepared a negotiation scenario with opposing positions, including information not available to the other party. Students prepared then negotiated for ten minutes with each other in front of the whole class. The lecturer provided feedback about the experience.

SIMULATION CASE STUDY
SimulTrain® is a team-based interactive (serious) game designed to improve project management competency in the areas of cost management, schedule management, quality management, risk management, and team dynamics. It involves two hours and 45 minutes of game time (over two sessions) that represents 12 weeks of project activity in one day of class time. As the application is accessed online, teams in different geographical locations can compete together. A leaderboard updates team performance in real-time. Each team receives project feedback that highlights effective and weak practices.
SimulTrain® is produced by STS (Sauter Training & Simulation SA, Avenue de la Gare 10, CH-1003 Lausanne, Switzerland). According to their website, "SimulTrain® is a gamebased online project management simulator which allows learners to acquire core competencies while improving teamwork and leadership skills in a realistic & fast-paced environment." STS claim that the simulator has been used in more than fifty countries and has contributed to the training of more than 150,000 project managers 1 .
Bond University was the first university to use SimulTrain® in Australia. Students were able to volunteer to experience this simulation as part of a pop-up (extracurricular) class offered once per year for the last four years. The latest instance was held on 23 October 2019 and comprised 20 postgraduate project management students self-selected into five teams. The class comprised an introduction to the simulation and time to plan and develop a team strategy before the game is activated. Two game sessions run with a lunch break and replanning opportunity in-between and a reflective discussion at the end. The highest performing team on the day receives a prize.
Team 3 achieved the highest simulation score (see Figure 1). The following semester, these students were surveyed about their project management simulation experience. This survey involved ten questions seeking data via Likert scale responses, and one open question to provide detailed qualitative data. The responses to the questions in Table 4 were varied, but the responses were nonetheless strongly skewed towards the positive end of the Likert scales. Data revealed that all participants believed their game-based experience to be either effective or very effective. Two participants regarded their learning from game-based education to be about the same as from traditional education, while nine rate it as better or much better. One participant rated the authenticity of the game-based learning experience as somewhat authentic, two rated it neither significantly authentic or inauthentic, eight rated it as either quite authentic or very authentic.

Figure 1
SimulTrain® team performances (n=5)  In stark contradiction to the positive survey results, only one participant regarded a game-based assessment to be a valid testing strategy for all core subjects. In contrast, the remainder regarded it as appropriate for minor assessments or only one assessment task in each core subject. The role of game-based assessment is at odds with the high level of enthusiasm for game-based learning. We concluded that this probably reflected a reasonably widespread preference for individual rather than group assessment tasks among participants, despite students understanding that employers prefer graduates who can demonstrate proven teamwork experience.  The data in Table 5 are interesting for the fact that the majority of participants who had experienced game-based learning generally rated the factors often seen as critical characteristics of effective game-based learning (Bond, 2019) as important or extremely important.
Participants who had experienced game-based training were generally positive in their comments, in line with the nature of their responses to the main survey questions. However, they were not always in agreement about benefits and concerns. Following are extracts from the data that confirm the general positivity towards game-based education:

Thanks to the Faculty for arranging this opportunity for students to understand the real range of project management responsibilities and duties This game-based exercise was all about collaborative learning and motivated people to learn in a fun yet serious practical way
In a constructive context, participants did offer some criticism. Indeed, the data from this category probably provides some of the most valuable from the perspective of designing and using game-based project management education in the future. For that reason, several examples are provided:

With team games, some students have dominating personalities and may affect the collaboration process thus negatively influencing the final outcome. Probably that is why this system should not be used in grading unless it is an individual assessment
It would be even more useful if the games system could recommend areas or show problems at an individual level so each person who undertook the training could review and ensure a more helpful learning outcome. I found it difficult, partly because of time pressure, to understand where I/my team had problems and which solutions were the best to solve the issues we had.

If game-based learning is used for assessment, I think it would be a huge stress, but it is also big fun for students
An important aspect of the training exercise was the team feedback provided from within the simulation and presented hereunder:

Discussion
The literature review and data collected in this study show that gamification and serious games are infrequently used in higher education project management subjects. All participants (global and case study) described students' simulation experiences as positive and generally aligned with the literature reviewed. Students who experienced project management simulations reported they were more engaged, their teamwork improved, and they reflected on their learning experience. Furthermore, students mentioned they liked to be challenged with reallife project scenarios of uncertainty supporting Rumeser and Emsley (2018), who report that students prefer complex project games. Global participants mentioned their students felt more confident applying what they learned as a result of project management simulations. What became apparent is that students faced real project challenges in a safe environment where they could test their project decisions without harmful repercussions.
Digital badges are frequently used to gamify classrooms (Hanus and Fox, 2015). In this study, one university used digital badges to encourage participation in extra-curricular learning activities like project simulations. Given the research results of Abramovich, Schunn, and Higashi (2013), it might be questionable if the badges lead to an increase in student motivation and subsequent performance. Nevertheless, one research participant reported that digital badges led to a higher participation rate in optional pop-up classes. This viewpoint aligns with the results of Barata, et al. (2017), who achieved increased attendance and participation through gamification. However, it is possible digital badges do not motivate all students to engage in learning; therefore, digital badges may be combined with other modes of teaching like assigned reading, serious games, exams, reports, etc.
The idea of having multiple chances to learn (multiple lives) by redoing an assignment or task is based on the gamification design principle of freedom to fail (Dichev and Dicheva, 2017). The concept is like continuous improvement and may contribute to deeper learning. However, there is no agreement repeating an assignment is a good thing (Kapp, 2012), while others suggest freedom to fail encourages exploration (Oxford Analytica Ltd, 2016). Students undertaking SimulTrain® can participate again in the following year.
Perhaps the most significant finding reported in the simulations is in the level of importance given to debriefing sessions. Hertel and Millis (2011) state that reflection helps students to integrate learning into their lives as a foundation for future learning, thus increasing the likelihood of knowledge and skill transfer in a professional setting. Therefore, simulation debriefing appears sensible since students may not have incorporated all the learnings from the simulation. The "learning points" provided within the SimulTrain® game and the "coach feedback" that occurs live during game execution appear to help students understand their performance score and how they might improve.
There are teaching and learning implications derived from this research. An early decision for the educator is to develop game-based learning appropriate for the class size and learning objectives. Role-playing might not be appropriate for large classes, but online, individual experiences with gamification may be more appropriate. For the teacher, it may take longer to develop a gamification lesson than a traditional PowerPoint lecture. However, successful gamification can provide life-long lessons and increase engagement. Teachers need to allocate sufficient time to prepare gamification lessons. Additionally, there might be greater teacher satisfaction if administrators acknowledged the effort and innovation required to incorporate gamification within subjects. As noted in the literature, educators should be thoughtful about awarding points so that the focus is on learning rather than on pointsification.
Educators should add a debrief opportunity to gamification. Longer games may have a mid-point debrief, while shorter games might only have an end of game debrief. The key is to provide feedback about performance and learning outcomes attainment as close to the experience as possible. And where appropriate, link gamification learnings with the needs and expectations of the learners. There might be an additional step where the students update their resumes with these new skills and knowledge. Not all students learn the same way, which is why student-centered learning has been a pervasive construct in education. Therefore, there may be some students who do not like simulations, some who do not like teamwork, some who prefer lectures, some who like hands-on activities, etc. The point is there might be students who do not perform well in simulations. Therefore, effective teachers use multiple teaching modalities throughout the subject to optimize learning. The role of assessments in serious games is in its infancy and is a topic that requires more research.

Conclusion
Within this study, higher education providers who have PMI-accredited programs shared their gamification insights in project management education. There is potential for a broader application of gamification and game-based learning in project management education as few institutions claimed to use it. Simulations were the most employed tool, and various approaches were discussed. Participants reported mostly positive effects of using gamification in their classrooms. Benefits they have realized include an increased participation rate, higher student engagement, as well as experiencing the project context and complexity, resulting in more confidence in applying project management knowledge, tools, and processes. Students who participated in the follow-up case study both enjoyed and learned from the experience. They generally recommended more simulation games, not less in project management higher education.
Gamification can contribute to meeting the actual demands of industry in project management education. One benefit it can deliver is the possibility for students to experience real-world scenarios safely in simulations. Simulations, coupled with the freedom to fail, timely feedback opportunities, and formal reflection, can give students valuable insights and may prevent future mistakes. Furthermore, gamification can help students to understand the project context. Perhaps an added benefit for educators is that gamification can bring life-long learning lessons into the classroom, recognized by student smiles enjoying the simulation experience.

LIMITATIONS
As with most project management research, this research has limitations, such as the limited sample size; larger samples would allow for more detailed analyses. This shortcoming is a function of the embryonic nature of the topic. However, the results generally support earlier gamification research findings, albeit in areas other than project management. The sample was also limited to include only those institutions accredited by PMI; therefore, there are institutions and countries not affiliated with PMI and not contacted that may be using gamification in novel ways and would have like to participate in this research. Indeed, there may be other types of simulations and serious games that can advance our understanding of learning through gamification and serious games.