Development of generic key performance indicators for PMBOK® using a 3D project integration model

Since Martin Barnes’ so-called ‘iron triangle’ circa 1969, much debate has occurred over how best to describe the fundamental constraints that underpin project success. This paper develops a 3D project integration model for PMBOK® comprising core constraints of scope, cost, time and risk as a basis to propose six generic key performance indicators (KPIs) that articulate successful project delivery. These KPIs are defined as value, efficiency, speed, innovation, complexity and impact and can each be measured objectively as ratios of the core constraints. An overall KPI (denoted as s/ctr) is also derived. The aim in this paper is to set out the case for such a model and to demonstrate how it can be employed to assess the performance of project teams in delivering successful outcomes at various stages in the project life cycle. As part of the model’s development, a new PMBOK® knowledge area concerning environmental management is advanced.


Introduction
Project success has long been one of the most discussed issues in project management globally.In the process, conventional wisdom over the merit of the cost-time-quality relationship, or its derivatives, has been challenged.There are calls for a new paradigm (e.g.Weaver, 2012) and yet plenty of support for the old one.There is confusion between terms such as 'project success' and 'project management success', and between 'success factors' and 'success criteria'.International standards appear to increasingly avoid the issue and focus more on practice and procedure for the discipline.
Project Integration Management is a knowledge area within the Project Management Body of Knowledge (PMBOK®) as published by the Project Management Institute.It is described as the unification of the other knowledge areas, presumably with the view of ensuring successful outcomes are achieved.But the lack of a cohesive model is arguably an impediment to resolving what constitutes a successful project.
The aim in this paper is to set out the case for such a model and to demonstrate how it can be employed to assess the performance of project teams in delivering successful outcomes at various stages in the project life cycle.This includes a review of recent literature, development of a project integration model, derivation of generic KPIs, and a practical example of how it works, followed by discussion and conclusions.

Literature Review Existing PMBOK® Knowledge Areas
The PMBOK® Guide Fifth Edition (PMI, 2013) is founded on ten knowledge areas that each describe a set of concepts, terms and activities that contribute to the definition of the discipline of project management.These comprise:

1.
Project integration management.It identifies, defines, combines, unifies and coordinates the various processes and activities within the Project Management Process Groups, including decision choices about resource allocation, making trade-offs among competing objectives and alternatives, and managing the interdependencies among the other knowledge areas.

2.
Project scope management.It includes the processes required to ensure that the project output contains all the work required, and only the work required, to complete the project as designed.

3.
Project time management.It involves all the processes required to manage timely completion of the project as scheduled.

4.
Project cost management.It involves the processes of planning, estimating, budgeting, financing, funding, managing and controlling costs so that the project can be completed within the approved allocation.

5.
Project quality management.It includes the processes and activities that determine quality policies, objectives and responsibilities so that the project will satisfy the required needs to the agreed standard.

6.
Project human resource management.It includes the processes that organise, manage and lead the project team to ensure that expected performance is realised.

7.
Project communications management.It includes the processes that ensure timely and appropriate management of project information both internal and external to the performing organisation.

8.
Project risk management.It includes the processes of conducting risk management planning, identification, analysis, response planning and control for project delivery to maximise positive and minimise negative risk events.

9.
Project procurement management.It includes the processes necessary to purchase or acquire products, services or results needed from outside the resources of the project team.
10. Project stakeholder management.It includes the processes required to identify the people, groups or organisations that could impact or be impacted by the project, to analyse their expectations and potential impacts, and to develop appropriate strategies to ensure effective engagement.
Each of these knowledge areas contains overlaps that underscore the importance of the role of project integration as an opportunity for unification, optimisation and conflict resolution.In this context, project integration management is a higher-order process group than the remainder.Zwikael (2009), in a structured survey of 783 project managers from a wide range of countries and industries, found that Project Integration Management is the most important knowledge area in the PMBOK® Guide.Kirsilä et al. (2007) point out that analogous words in the literature to describe the concept of project integration include unite, combine, unify, consolidate, concentrate, organise and systematise.Put another way, integration is about bringing a number of distinct things together into a harmonious unit.This is what differentiates it from the other knowledge areas.

The 'Iron Triangle'
Martin Barnes is credited with the notion of core project constraints that underpin successful project delivery.Known as the 'iron triangle' (or triple constraint), it was initially defined, circa 1969, as the relationship between cost, time and output.He argued that making a change to one affects the other two.Many modifications ensued, including output being variously renamed as quality, scope or performance.Some preferred the terms of 'budget, schedule and scope', while others simplified it further as 'cheap, fast and good'.There have been numerous ways of illustrating these project constraints, as a search of images for 'iron triangle' in Google will testify.Jha and Iyer (2007) find that the commitment, coordination and competence of project players are key factors that underpin the iron triangle.These are argued as bearing on time, cost and quality/scope objectives respectively, and when nurtured, successful performance outcomes are more likely to occur.

Project Constraints
Today, Barnes' triangle has also lost favour due to the almost endless list of project objectives that have been discussed in the literature, including safety, defect minimisation, environmental impact, continuous process improvement, team development, conflict avoidance, and perhaps more significantly, client, user and stakeholder satisfaction (e.g.Wateridge, 1998;Fortune and White, 2006;Toor and Ogunlana, 2010).It has now been dropped completely from the PMBOK® Guide Fifth Edition in exchange for the following statement (PMI, 2013:6): … balancing the competing project constraints, which include, but are not limited to: scope, quality, schedule, budget, resources and risks [… where] the relationship among these factors is such that if any one factor changes, at least one other factor is likely to be affected.
Practitioners still refer to the concept and many defend it.Others have called for a new approach to convey the essential need to balance competing constraints.Yet some confusion exists between project delivery success and project success, where the latter is obviously influenced by factors beyond the control of the project team and sometimes outside the boundaries of the project itself (de Wit, 1988;Munns and Bjeirmi, 1996).

Project Environmental Management
In recent years the importance of environmental sustainability has emerged and captured the attention of project management teams (e.g.Fernández-Sánchez and Rodríguez-López, 2010; Ebbesen and Hope, 2013;Hwang and Ng, 2013).Not only are environmental controls likely to impact on project outcomes and choices, but the wider moral imperative of a sustainable future has led to concern that the balance between economic, social and environmental criteria (i.e.triple bottom line thinking) is not well served by the current PMBOK® framework.In much the same way that stakeholder management was separated from communications management and elevated to higher importance, so too will environmental management need to be extracted from scope, quality, procurement and risk management and given more prominence and coherence.Sustainability considerations are paramount to our collective future.Silvius et al. (2012) provide a robust case for the inclusion of sustainability in project management.They highlight its scant treatment to date in international standards like the PMBOK® Guide and the need for a new approach.
In the next edition of PMBOK®, formal consideration should be given to the inclusion of an eleventh knowledge area called Project Environmental Management.These changes will cause little alteration to existing knowledge areas other than suitable cross-referencing.Suggested generic processes for this knowledge area are: 1.
Plan environmental management.It includes the process of documenting project environmental decisions, establishing an appropriate strategy, and planning to ensure that valuable environmental assets are protected.

2.
Identify potential impacts.It includes the process of determining which project attributes may impact on the natural environment, listing issues of significance, and prioritising them so that subsequent analysis can occur.

3.
Assess environmental values.It includes the process of quantifying positive and negative influences that the project induces on its natural surroundings (i.e.environmental impact assessment) using appropriate multiple criteria.

4.
Control mitigation.It includes the process of implementing strategies that conserve and protect environmental assets, monitoring performance, identifying new risks, and evaluating effectiveness throughout the project.

Success Criteria for Project Delivery
Project delivery success (PDS) and its measurement feature in the literature over a long period.Generally the findings acknowledge the influence of scope, cost and time, but identify other success criteria that should be included.Cooke-Davies (2002) highlights the difference between success criteria (used for evaluating success) and success factors (inputs that lead to success).The list in both cases is long and criteria/factors are often specific to particular types of projects and sponsors (see Atkinson, 1999;Shenhar et al., 2001;Bryde and Robinson, 2005;Müller and Turner, 2007;Ika, 2009;Al-Tmeemy et al., 2011;McLeod et al., 2012;Tabish and Jha, 2012;Davis, 2013).Success criteria and success factors can be linked at the level of core project constraints (Westerveld, 2003).
In a study based on more than one hundred defence research and development projects, Dvir et al. (2003) found that project success is insensitive to the level of implementation of management processes and procedures, but positively correlated with the investment in requirements' definition and development of technical specifications.They viewed success as meeting design goals and the benefit to the customer.Clearly this underscores the mismatch of whether success relates to the project or the project's delivery.
PMBOK® Guide Fifth Edition (PMI, 2013:35) states that: 'Since projects are temporary in nature, the success of the project should be measured in terms of completing the project within the constraints of scope, time, cost, quality, resources, and risk as approved between the project managers and senior management … project success should be referred to the last baselines approved by the authorised stakeholders [… and] the project manager is responsible and accountable for setting realistic and achievable boundaries for the project and to accomplish the project within the approved baselines.' At the completion of a project, its successful delivery will directly influence the reputation of the performing organisation and its ability to grow and prosper.It is vital to ensure that the criteria upon which success will be judged is clear from the outset.Business generally uses KPIs for this purpose.Bryde (2005:119) argues that the absence of KPIs in project management should be redressed.He concludes that this is 'seen to contribute to a failure by organisations to manage necessary increases in their project management capability and to be acting as a possible barrier to long-term, sustainable improvements in performance'.
However, a detailed search of PMBOK® shows that discussion of KPIs is virtually absent.Therefore the question arises: what constitutes a successful project?A common answer from practitioners might be a successful project is one that is delivered within budget, on time and as specified, with no unwanted surprises.Saputra and Ladamay (2011) refer to the probability that this happened as 'project reliability'.

Project Integration Model
Core project constraints might therefore be distilled down to cost, time, scope and risk.Each of these is suitable for objective measurement.But rather than suggesting a triangular framework, four constraints are best illustrated in the form of a tetrahedron.The base of the tetrahedron reflects the traditional iron triangle, while the apex is risk.Other identified constraints of quality and human resources are subjective and hence difficult to quantify.
The PMBOK® knowledge areas can also be displayed using this framework.Key Performance Indicators Chan and Chan (2004) state that the purpose of KPIs is to enable measurement of project or organisational performance.Collin (2002, cited in Chan andChan, 2004) advocates that the following attributes should be kept in mind: 1.
KPIs are general indicators of performance that focus on critical aspects of outputs or outcomes.

2.
Only a limited, manageable number of KPIs is maintainable for regular use (too many or too complex KPIs can be time and resource consuming).

3.
The systematic use of KPIs is essential as their value is almost completely derived from their consistent use across projects.

4.
Data collection must be made as simple as possible.

5.
KPIs should be generic and able to be used on every project.

6.
For performance measurement to be effective, the KPIs must be widely accepted, understood and owned.Bryde (2003) proposes six criteria for measuring project management performance.These comprise leadership, staff, policy and strategy, partnerships and resources, project life cycle management processes, and use of KPIs.The two most important KPIs are client/customer perception and meeting specified project objectives.Perception is qualitative, but meeting objectives has the potential to be assessed quantitatively.
In this research, six generic KPIs capable of quantitative measurement flow directly from the development of the project integration model.They relate to project delivery and comprise: 1.
Value.Defined as the ratio of scope over cost, this KPI is one that should be maximised.Value is a function of Project Stakeholder Management, namely meeting expectations and fostering engagement.Scope is treated as an output and cost is treated as an input, so the more utility per unit of cost the greater is the value for money.

2.
Efficiency.Defined as the ratio of cost over time, this KPI is also one that should be maximised.Efficiency is a function of Project Human Resource Management, namely team performance and leadership.Cost in this case is treated as an output (value of work completed) and time as an input, so the more money spent per unit of time the more efficient is the delivery process.

3.
Speed.Defined as the ratio of scope over time, this KPI is another that should be maximised.Speed is a function of Project Procurement Management, namely outsourcing strategies and parallel supply chains.Scope is treated as an output and time as an input, so the more utility provided per unit of time the faster is the delivery process.

4.
Innovation.Defined as the ratio of risk over cost, this KPI should be maximised too.
Innovation is a function of Project Communications Management, namely knowledge management and research-informed learning.Risk is treated as an output (innovation leads to development risks) and cost as an input, so a higher level of risk per unit of cost reflects the search for better ways of doing things.

5.
Complexity.Defined as the ratio of risk over time, this KPI is one that should be minimised.Complexity is a function of Project Quality Management, namely excessive quality assurance paperwork and engineering over design.Risk is treated as an output and time as an input, so a higher level of risk per unit of time is a sign of project difficulty that should be avoided.

6.
Impact.Defined as the ratio of risk over scope, this KPI is also one that should be minimised.Impact is a function of Project Environmental Management, namely adverse sustainability outcomes and unnecessary resource consumption.Risk is treated as an output and scope as an input, so a higher risk level per unit of utility reflects unwanted environmental disruption.Baccarini (1996) defines managing project complexity as within the purview of integration.Perhaps the remaining KPIs of value, efficiency, speed, innovation and impact should be treated similarly.
The relationships between the project constraints of cost, time, scope and risk and the KPIs of value, efficiency, speed, innovation, complexity and impact are illustrated in Figure 2. A 2D version of the model is presented here for ease of comprehension, but it turns into a 3D tetrahedron by folding along the dotted lines.Core project constraints, which are equally weighted, are shown in upper case.Clearly it is impossible to optimise all KPIs given that most constraints act as outputs in some cases and inputs in other cases.

Figure 2 Project constraints and key performance indicators
A balanced approach is therefore sought.The six KPIs can be merged into one overall KPI as shown in Equation 1.In other words, maximising scope and minimising cost, time and risk will lead to higher project delivery success.These constraints should only be varied with sponsor approval.Note that a common percentage increase in scope, cost, time and risk has no effect on the outcome and would be judged as equally successful.There are other potential factors at play, such as stakeholder satisfaction, creativity, beauty, legacy and personal security, but those embedded in the model reflect the core targets to maximise PDS.Nevertheless, there is no reason against a few more KPIs being introduced in particular circumstances.
PDS is therefore defined as s 3 /ctr.It is recommended that definitions for each variable be established during the project planning stage.Their values are recorded prior to project commencement and KPIs are then each set to a base of 1. Upon completion, actual KPIs are calculated proportionally to their base to highlight the change.An answer less than 1 indicates poor performance, while an answer greater than 1 indicates the project exceeds expectations.Obviously the higher the result, the better is the outcome.In the case of the overall KPI, its value before and after is computed to determine the percentage change from the original core constraint baselines.Any increase indicates successful project delivery.The higher the percent increase, the more successful is the project compared with expectations.
Scope may be measured as the magnitude (or size) of the main deliverable as designed and as delivered, such as m 2 of usable floor area, number of products sold, value-add of finished project or benefit-cost ratio, perhaps weighted for quality standard.Cost may be measured as the contracted amount for the project and its final account after reconciliation.Time may be measured in working hours or days, or calendar weeks or months from contract commencement to completion as planned or achieved.Risk may be measured as the sum of probability x impact (using a 1-3 scale) for all identified risks, both before commencement and upon completion.In the latter case, probability=3 (i.e.very likely) is assumed for all identified risks and actual impact is re-evaluated with the benefit of hindsight.

Practical Example Overview
The application of the new model is best illustrated by example.A hypothetical project presented to students at Bond University, Australia, is used for this purpose.Students are given a contemporary problem to address.The number of asylum seekers risking their lives to jump the formal UN refugee process and reach Australia by boat from Indonesia is rising exponentially.Temporary detention and refugee processing is budgeted to cost AUD$3 billion in 2013/14.There is a need to stop the boats and resettle refugees within the region in a safe and self-sustaining environment.The United Nations High Commissioner for Refugees (UNHCR) is identified as the project sponsor.
It is assumed that UNHCR has support from Australia, New Zealand, Papua New Guinea and other Pacific countries to create an independent island nation for bona fide refugees.This new principality, to be named Utopia, will be governed by UNHCR.The island is in a remote and undisclosed location in the South Pacific.Detailed project planning for Stage 1 is the assigned task.While an overall project plan is provided, students need to select one aspect (procurement logistics, energy infrastructure, shared facilities, accommodation construction or environmental sustainability) and develop it further using the PMBOK® Guide Fifth Edition as the framework.Collaboration occurs over the semester as a mix of face-to-face discussions and online interaction using a project-based blended learning strategy and access to shared project resources.

Scope
Accommodation is required for 5,000 households each comprising an average of 3.5 people (i.e.capacity is defined as 17,500 refugees).An administration building, small airstrip, base hospital, communal/catering areas, school, water, power and sewage infrastructure and a deep water jetty are also required.All facilities are to be eco-friendly and as self-sustaining as possible.Land will be designated for growing food, and a local fishing industry will be encouraged.Citizens will be employed in managing the island and will receive food, health and education services in return.This is not a detention camp or processing facility, but a permanent home for genuine refugees fleeing persecution from their home country.Utopia is to be an independent island nation, based on a barter economy, where refugees are citizens and provided with land and housing on a 10-year renewable lease.
A key part of the project is the construction of accommodation, which must be prefabricated in Australia and transported by sea to Utopia.The standard of accommodation is specified as part of the scope and embodies issues of sustainability and value for money.Figure 3 provides insight into the design, which reflects prefinished modular units that are stackable in a variety of arrangements that minimise onsite construction works.In total 3,000 single 'crates' (sleeps 2) and 2,000 double 'crates' (sleeps 6) are anticipated, equating to 7,000 individual units or 18,000 refugee beds.guaranteed maximum price contract is anticipated.Figure 4 shows the distribution of cost by sub-project.

Risk
Utopia is presently an uninhabited tropical island of undisclosed location, with a large fresh water lake, fertile farming land and abundant natural fishery.Key risks for the project concern its location, which is in a cyclone zone in an active volcanic/earthquake region and with limited access to the outside world.Construction of an eco-community in such a remote setting with few natural resources to support construction creates additional challenges.An assessment of the key risks for Stage 1 is included in Figure 6.Using a 3x3 risk matrix, the mean of probability x impact across each identified risk event is 4.03.

Key Performance Indicators
Figure 7 provides the calculation for each KPI.These are then set to 1 for the purpose of later comparison against actual performance.The PDS score (overall KPI) is 2.66.Assuming that 18,000 refugee beds are achieved, any additional costs are absorbed with the contingency, final handover is delayed 20 days and not all of the identified risks impacted the project as anticipated, then the actual PDS score is 3.02, or 13.44% better than forecast.Actual risk is computed by assuming that each identified risk event now has a probability of 3 (i.e.complete certainty).If a risk event had little or no impact on the project it is scored as 1, while 2 indicates that a moderate impact occurred and 3 indicates that a significant impact occurred.All identified risks are scored, even if they never eventuated.In this example the actual mean risk score is computed as 3.79 and suggests that the project went well.Any unforeseen risks must be added to the calculation.

Discussion
The Utopia example demonstrates the process of applying the project integration model and its associated KPIs to a project, albeit hypothetical.It is useful to crystallise success into a measurable outcome that enables objective assessment.Based on assumed actual performance, expected KPIs for value, speed, complexity and impact were surpassed, while KPIs for efficiency and innovation were not.It is impossible for all six KPIs to be exceeded.Hence the PDS score is critical to the assessment of project delivery success.Presumably in this case the UNHCR would have been pleased with an outcome that provided additional scope within budget and with no unexpected surprises, even though a slight delay in handover could not be avoided.
Scope, cost, time and risk are shown to be the foundations of six generic KPIs.Scope is not just the work that is required, but reflects the standard (i.e.quality) that is expected and specified in contractual documentation.Cost is not just whether the project is delivered within budget, but reflects the resources needed to undertake the work.Time is not just whether the project is completed on time, but reflects the procurement decisions necessary to support the process.Risk is not just exposure to unexpected surprises, but reflects the ability of the project management team to deal with issues when they arise, and resolve any arguments over compensation.Project integration, therefore, ensures that the right balance between the core project constraints is achieved.Stakeholder satisfaction is likely to be realised if the overall KPI is better than forecast.
Satisfaction, however, is complicated.It is conceivable that KPIs are delivered and one or more stakeholder groups are dissatisfied.Therefore the question arises as to whether stakeholder satisfaction is a criterion or a phenomenon.The latter is suspected.
The use of KPIs can obviously occur at the completion of a project to judge overall success.However, they can also apply at any stage during the execution of a project to assess current performance of the project management team.Trend analysis could be used to see whether the PDS score is rising or falling, and which KPIs might benefit from some direct intervention.
More detailed KPIs may be employed for particular projects and used as secondary success criteria.Yet incorporating them into an overall KPI may prove counterproductive.The benefit of the original 'iron triangle' was its simplicity to explain generic core project constraints, and this advantage is retained and enhanced in the model presented here.Perhaps one day, if incorporated into international standards like the PMBOK® Guide, core project constraints of scope, cost, time and risk may be widely referred to as the 'iron pyramid'.

Conclusion
A model for describing project integration is presented in the form of a tetrahedron containing all existing PMBOK® knowledge areas plus a new one of environmental management to elevate the importance of sustainability in modern projects.This paper sets out the case for such a model and demonstrates how it can be employed to assess the performance of project teams in delivering successful outcomes at various stages in the project life cycle.This is achieved through the identification of core project constraints, occupying the four vertices of the model, and six KPIs, represented by the edges of the model.KPIs express the relationships between constraints, are relevant to any type of project, and are capable of objective measurement.
Project delivery success is a result of the balance between the six KPIs, recognising that collectively they are mutually exclusive.If the actual PDS score is higher than the expected score, then the project is considered successful at a macro level and the project management team should be congratulated.Emphasis can then be placed on other indicators to fine tune project performance expectations and to ensure continuous process improvement (or learning outcomes) are pursued.These secondary KPIs are unlikely to be generic, but lend themselves to a hierarchical framework of classification below the six KPIs identified in this paper.
The four vertices are denoted as Project Cost Management, Project Time Management, Project Scope Management and Project Risk Management.The six edges are occupied by the remaining knowledge areas: Project Human Resource Management, Project Procurement Management, Project Stakeholder Management, Project Communications Management, Project Quality Management and the new Project Environmental Management.The tetrahedron itself reflects the higher-order knowledge area of Project Integration Management, and is shown in Figure 1.

Figure 1
Figure 1 Project integration model

Figure 3
Figure 3 Utopia accommodation design

Figure 4
Figure 4 Utopia cost baseline

Figure 5
Figure 5 Utopia time baseline

Figure 6
Figure 6 Utopia risk baseline

Figure 7
Figure 7 Utopia key performance indicators and PDS outcome