Establishment of Performance Scales for Team Integration Assessment

The increased use of the alliance model in recent years highlights the importance of integration practice among multi-disciplinary teams, as it promotes a collaborative culture and the continuity of equitable relationships to improve project performance. If continuous improvement in project alliances is to be achieved through the use of integrated teams, then a means of assessing how well teams integrate and how team integration changes over time, needs to be introduced. In response to that need, an Alliance Team Integration Performance Index (ATIPI) has been developed as part of a wider study to develop an assessment tool for team integration in road construction alliance projects in New Zealand. In this paper, a set of Key Indicators (KIs) of team integration practice and the Quantitative Measures (QMs) for each KI are first introduced and then, in order to enhance the associated ATIPI model, the establishment of scales for the performance levels, namely ‘poor’, ‘average’, ‘good’, ‘very good’ and ‘excellent’, for each KI are described. The establishment of performance scales will help ensure that the assessment of team integration practice is quantified in a consistent and objective manner.


Introduction
A key driver of project success, specifically in project alliances, is the ability of alliance teams to promote a collaborative culture and embrace a high level of integration practice.Despite agreement among scholars and practitioners on the importance of the alliance model in influencing the integration practice, the ability of project teams to sustain and consistently drive the collaborative culture still remains a concern (Rooney, 2009;Ross, 2009).Laan et al. (2011), in their study, found that individuals who are versed in more traditional types of contracts may not be able to adopt an attitude of cooperative relationship automatically in order to ensure the success of the project alliance.Furthermore, individuals involved in alliance contracts may import working methods from non-collaborative types of arrangement, and these can be detrimental to the.In addition, Bresnen (2007) argued that it is questionable if a collaborative and cooperative philosophy, which normally requires a longer period of time to develop, can be shaped and sustained in a project-based setting.
Therefore, it is necessary to develop and consistently sustain integration practice within alliance teams over the lifecycle of the project.If continuous improvement in project alliances is to be achieved through the use of integrated teams, then a means of assessing how well teams integrate and how team integration changes over time, needs to be introduced.By measuring team integration with such an assessment tool, the performance of the integration activities can be continuously monitored and managed at an earlier stage (Baiden et al., 2006), which is crucial to ensure the success of alliancing (Hauck et al., 2004).
To address this need, Ibrahim et al. (2013a) first identified the key indicators (KIs) as part of a conceptual index, the Alliance Team Integration Performance Index (ATIPI) for assessing team integration in alliance projects in New Zealand.Then, they determined the assessment attributes for each indicator, the Quantitative Measures (QMs) to help quantify the KIs objectively (see Ibrahim et al., 2013b).Although the identification of KIs and QMs form the basis of the ATIPI, the need to develop the associated scales for the performance levels, namely 'poor ', 'average', 'good', 'very good' and 'excellent', is critical to completing the ATIPI and ensuring it will result into a fully working assessment model.This paper, therefore, focuses on establishing the performance scales for team integration assessment in alliance projects.By means of a questionnaire survey with 17 experienced alliance practitioners, the views of alliance practitioners on the performance scales are incorporated.The establishment of the performance scales will enhance the current ATIPI and ensure that the assessment of team integration practice is quantified in a consistent and objective manner.

KIs and QMs for Measuring Team Integration Practice
Based on a comprehensive review of the available literature, the authors identified KIs of successful team integration in construction projects through observation of previous studies (see Ibrahim et al., 2013c).The identification of the indicators was followed by validation through a survey conducted with a construction peer group from New Zealand, the Construction Clients' Group (CCG) (see Ibrahim et al., 2013d).This resulted in a total of 17 KIs being identified, which together form the basis for transforming disparate project teams into a highly integrated team in various types of procurement approach.
Subsequently, four rounds of Delphi questionnaire survey was undertaken with a panel of 17 experienced alliance practitioners to establish the most significant key indicators (KIs), among the identified 17 indicators, to help measure the success of team integration, specifically in alliancing road construction projects (See Ibrahim et al., 2013a).The resulting seven team integration practice KIs were: team leadership; trust & respect; a single team focus on project objectives and KRAs; collective understanding; commitment from project alliance board; creation of single & co-located alliance team; and free flow communication.A conceptual Alliance Team Integration Performance Index (ATIPI) was then developed based on the identified KIs and their weightings (see Table 1).
Then, a semi-structured interview with five experienced alliance practitioners was conducted to recognise suitable, practical and objective measures to help evaluate the seven selected weighted KIs.Finally, a total of 29 quantitative measures (QMs) were proposed and recommended by the five interviewees (See Ibrahim et al., 2013b).Two rounds of Delphi questionnaire survey were then undertaken with the same 17 Delphi experts to identify the most appropriate QMs for each KI based on their levels of importance, measurability and obtainability in order to provide objective assessment based on quantitative evidence (See Table 1).
In the ATIPI, the degree of performance for team integration practice is defined by two elements, the weighting coefficient and the rating score of the indicator.The weighting coefficients define the contribution of the measures for the ATIPI, this has been achieved in Ibrahim et al. (2013a).The rating scores, however, have to be measured for each individual alliance project being assessed.Before such a rating can be measured, the attributes for each indicator need to be determined (this has been achieved in Ibrahim et al. (2013b)) and associated scales of performance need to be established.By establishing such scales of performance, each KI can be defined and quantified objectively.The significance of defining appropriate scales of performance is of paramount importance to the success of the ATIPI, as the interpretation and perception of different evaluators as to what constitutes very good or excellent could be quite different (Ng and Chow, 2004).In addition, as demonstrated by Chow and Ng (2007) and Yeung et al., (2008;2012), should there be more than one evaluator, a clearly defined scale could help eliminate discrepancies, manipulations and subjective judgement for evaluation as different evaluators could perceive the same performance level with different numerical figures.For instance, a value of 5, on a scale of 1 to 10, in a survey of wider alliance teams' satisfaction on the level of trust and respect may indicate 'average performance' to individuals who are not familiar with the importance of trust in collaborative contracting; whereas for individuals who are well-versed in the alliancing environment it may indicate 'poor performance' as trust and respect is one of the principle tenets in building alliancing capability.Consequently, in order to remove this deficiency, it is necessary to define clearly the scales of performance for each KI.In this study, the establishment of performance scales for the KIs is described as the final step in developing the ATIPI into a working model for assessing the integration performance in on-going alliance projects in New Zealand.

Research Method -Questionnaire Survey
In this research, a questionnaire survey was developed to establish the performance scales for team integration performance assessment in alliance projects.Initially, a discussion with three experienced alliance practitioners in road infrastructure projects in New Zealand was conducted to help define typical ranges for each of the QMs associated with the seven selected KIs of team integration practice in alliance projects.The selection of these three practitioners was based on their experience in alliance projects (an average of 6 years) and their direct involvement in Alliance Management Teams (AMT) in their respective projects.
The discussion focused on the clarity of the information presented in the questionnaire, appropriate terminology used in the questions and suggestions on the appropriate range of scales reflected in measuring the KIs.Finally, the seven weighted KIs and their QMs, together with the typical ranges for each of the performance scales were finalised.

Participants
A panel of 17 recognised experts in the field of project alliancing were contacted, based on pre-defined selection criteria outlined in Ibrahim et al. (2013a), to help define the five different performance levels namely 'poor', 'average', 'good', 'very good' and 'excellent'.The same experts had previously participated in four rounds of the first stage and two rounds of the second stage of Delphi survey to identify KIs and QMs respectively.Five of the experts (29%) were from owner representative, eleven experts (65%) were from NOPs representative and one expert (6%), the alliance culture manager, was an independent consultant.
A questionnaire survey stating the objective of the study and guidance on completing the survey was emailed to the 17 alliance experts.A two week period in which to respond to the questionnaire was given.After that, a reminder email to all the experts who did not return the questionnaire in time was issued followed up by a phone call.Finally, all the 17 experts managed to complete and return the questionnaire, representing a response rate of 100%.

Results: Mean Value of the Performance Scales Against the Five Different Performance Levels
Table 2 summarises the results of the survey in terms of the mean value and the standard deviation of each KI against the five performance levels.Although the mean values represent each of the performance levels, the results indicate that there are differences in opinion between experts' perception of the performance scales of each KI.Based on the standard deviation value, it can be seen that there is slight to moderate deviation from the mean value in most of the KI's performance levels.Nevertheless, the deviations are not particularly high except for some of the KIs, namely, 'variation of actual time against programme expressed as a percentage of the project's progress' (SD for the excellent performance = 7.02); 'variation of actual cost against budget expressed as a percentage of the project's progress' (SD for the poor performance = 7.66); 'percentage of PAB members (original) attendance in PAB meetings' (SD for the poor performance = 22.81); 'number of staff allocated on-site against the overall number of staff expressed as a percentage of the single and co-located alliance team' (SD for the poor performance = 18.11); and 'percentage of alliance team attendance in weekly project briefing' (SD for the poor performance = 19.16).
Therefore, in response to these deviations, it is appropriate to establish a suitable range of performance scales for each KI.A similar approach to that adopted by Chow and Ng (2007) and Yeung et al. (2008) was followed in establishing scales for the performance levels.
Accordingly, the boundaries between the five performance levels from poor to excellent have been defined by taking the average value of each two consecutive performance levels.For example, to establish a range of performance scales for team leadership (measured using the QM time performance), a lower boundary for the 'excellent' performance was taken as the average of the mean value 'very good' (MVG = 9.53) and 'excellent' (ME = 18.53) performance levels.Consequently, for team leadership to be regarded as 'excellent', the project should be running ahead of schedule by more than 14.03% variation of actual time against programme, as shown in Figure 1.A variation between 14.03% and 5.94% will result in 'very good' performance, while a variation between 5.94% and -3.74% will indicate 'good' performance.Considering project alliances are normally established to construct complex and challenging infrastructure projects, having a variation between -3.74% and -10.86% indicates 'average' performance.However, if the variation is more than -10.86%, the performance can be regarded as 'poor'.Using the same approach when assessing the commitment of the PAB, experts suggested that 'excellent' performance is represented by at least 93.39% attendance at PAB meetings.Should the percentage of attendance result in greater than or equal to 85.15% or greater than or equal to 75.89%, their performance can be regarded as 'very good' or 'good', respectively.An 'average' performance refers to attendance greater than or equal to 64.71%, while if the attendance is less than 64.71%, their performance can be considered as 'poor' in reflecting their commitment to influencing the integration practice.The proposed ranges of performance scales for each KI are presented in Table 3.

Discussion
The results of the research, included in Table 3, represent the views of the experts on the performance scales of each KI against five levels of performance; 'poor', 'average', 'good', 'very good' and 'excellent' for measuring team integration practice in alliance projects.The establishment of the performance scales serves as a quick 'rule-of-thumb' for evaluators to distinguish between the five levels of performance in measuring the team integration.In addition, the establishment of performance scales in this study exhibit some important comparisons to other types of procurement.For example, when compared with a study on partnering performance, Yeung et al. (2008) identified that to achieve an 'excellent' level of trust and respect in partnering, the survey result must indicate at least a score of 8.06.While in this study, a score of 9.00 is required to achieve 'excellent' performance.This is a reflection of the significant influence of trust and respect in fostering true integration in a project alliance.Salicru (2010) emphasized that trust and respect are vital for alliance teams to improve their relationships significantly, as this indicator is the most common source of conflict in alliances.Love et al. (2010) added that when there is a lack of trust in an alliance, confidence in best-for-project unanimous decision-making may be risked and opportunities for innovation can be missed.Another example based on the Yeung et al. (2008) study is that in partnering, the commitment of top management is regarded as 'excellent' when the percentage of attendance is at 84.09% and above, while in this study, only when the percentage of PAB attendance at 93.39% and above is performance regarded as 'excellent'.This higher degree of attendance required in alliancing shows that support from the PAB is essential to ensure the existence of an effective and consistent corporate commitment in achieving the alliance objectives.Overall, it is believed that the high scores required to achieve 'excellent' performance is appropriate for alliances as a successful alliance is engendered by a commitment that creates cogency that enables the team to develop and sustain a high intensity of integrated performance.

Conclusions and Future Research
This research study has established a range of performance scales for team integration performance assessment in alliance projects by conducting a questionnaire survey with 17 alliance experts resulting in a simple and practical way of defining performance levels for the seven KIs.The establishment of the scales for the performance levels will help senior levels of alliance management ensure that the assessment of team integration practice is quantified in a consistent and objective manner.
As part of a wider assessment tool currently being designed for practitioners to monitor, measure and improve team integration within alliance projects, the inclusion of the range of performance scales in the ATIPI will provide the ability to consistently assess the performance.This reduces the reliance on subjective judgement of evaluators while still maintaining the simplicity of the assessment process based on tangible evidence.Such an enhancement to the ATIPI will therefore allow team integration performance to be compared over the lifecycle of a project, as well as allowing benchmarking with other alliance projects.
Finally, this study focused on the opinions of alliance practitioners experienced in project alliancing in New Zealand.Consequently, further research should also be conducted in other countries for comparative purposes.

Figure 1 :
Figure 1: Example of scale boundaries for the team leadership performance measure (time performance)

Table 1 :
KIs and corresponding QMs for measuring team integration practice in alliance projects

Table 2 :
Mean Value and Standard Deviation for the Five Performance Levels of each KI Ibrahim, K.I, Costello, S.B & Wilkinson, S. 2014, 'Establishment of Performance Scales for Team Integration Assessment', Australasian Journal of Construction Economics and Building Conference Series, 2(2), 36-43.40

Table 3 :
Proposed Range of Performance Scales for each KI