Institutional Repository Modelling client satisfaction levels : the impact of contractor performance

The performance of contractors is known to be a key determinant of client satisfaction.Here, using factor analysis, clients’ satisfaction is defined in several dimensions. Based onclients’ assessment of contractor performance, a number of satisfaction models developedusing the multiple regression (MR) technique are presented. The models identify arange of variables encompassing contractor performance, project performance and respondent(i.e. client) attributes as useful predictors of satisfaction levels. Contractor performanceattributes were found to be of utmost importance indicating that clientsatisfaction levels are mainly dependent on the performance of the contractor. Furthermore,findings suggest that subjectivity is to some extent prevalent in clients’ performanceassessment. The models demonstrate accurate and reliable predictive power as confirmedby validation tests. Contractors could use the models to help improve their performanceleading to more satisfied clients. This would also promote the development ofharmonious working relationships within the construction project coalition.


INTRODUCTION
Traditionally, the main participants of the construction project coalition (PC) are the client, the architect and the contractor.The interactions and interrelationships between these participants largely determine the overall performance of a construction project (Smith and Wilkins, 1996;Egan, 1998).The performance of these participants is also interdependent (Higgin and Jessop, 1965;Mohsini, 1989).Hence, in order to perform effectively, a reciprocal requirement exists, whereby each participant requires the other participants to perform their duties effectively and in harmony with others.Notwithstanding this mutual dependency, the performance of individual participants remains important because overall project performance is a function of the performance of each participant (Liu and Walker, 1998).
UK contractors have long been criticised for their failure to fulfil the needs of their clients (Latham, 1994;Egan, 1998).In a broader sense, contractors should also perform to the satisfaction of other PC participants (e.g.architects) to maintain harmonious working relationships.This is because harmonious working relationships are essential if projects are to be successful (Baker et al., 1988;Smith and Wilkins, 1996;Egan, 1998).There is a need therefore, to investigate contractor performance from the viewpoint of other PC participants (especially clients), from which models for predicting levels of (client) satisfaction can be developed.The objective of this paper is to present and describe the development of such models which were developed using the multiple regression (MR) technique.The models could be used to identify attributes influencing satisfactory contractor performance assessment.This would ultimately help to improve performance and enhance satisfaction for the betterment of overall project performance.

CONCEPTUAL MODEL OF PERFORMANCE ASSESSMENT
Satisfaction is regarded as an internal frame of mind, tied only to mental interpretations of performance levels (Oliver, 1997).That is, a performance assessor (e.g.client or architect) will have their own psychological interpretation of the performance of others (e.g.contractors).This psychological process is subjective and difficult to interpret.Based on this theorem, a conceptual model of performance assessment has been developed (refer to Figure 1).Conceptually, the outcomes of performance assessment (in terms of satisfaction levels) can be influenced by two major attributes; those of the performer (i.e.performance attributes) and those of the assessor (i.e.satisfaction attributes).Satisfaction attributes are differentiable from performance attributes mainly due to their unique nature, being inherent within an individual (i.e.assessor).That is, performance attributes may reflect on both participants and projects, and will influence both participant and project performance.In contrast, satisfaction attributes reflect solely on the assessor and influence their performance assessment and as such are beyond the control of the performer.
Performance attributes consist of participant attributes and project attributes.Participant attributes represent the characteristics or nature of a particular participant or their organisation, such as company age and turnover.Project attributes represent the characteristics/nature of a project, comprising attributes which may be outside the control of the participants.Controllable attributes are, for example, forms of contract, procurement route, and extent of design completed prior to work on site.Uncontrollable attributes include type of project, ground and weather conditions.Satisfaction attributes include the personal attributes of the individual assessor (e.g.experience, vocational background) and attributes of their employer (e.g. company assessor attributes).Company attributes are characteristics of the assessor's company, which may influence their assessment (e.g. company age, turnover, number of employees).
Figure 1 demonstrates the relationships between these variables.The performance attributes of a participant have a direct influence on their own performance in the construction process.Project attributes indirectly influence the participant's performance since the attributes may enable/hamper the participant in executing their duties.Performance assessment in this respect is considered as 'objective' (i.e.tangible) in nature.For example, contractor performance may be assessed in terms of cost, time and quality performance (Holt, 1995).
However, performance assessment goes beyond the objective aspects outlined above since it involves the feelings of the assessor, which in turn are dependent on their background, i.e. frame of reference.This assessment is considered 'subjective' and at a higher level.This research embraces both 'objective' and 'subjective' (or higher level) performance assessment.In this case, satisfaction is measured using predetermined performance criteria, which are explained in research methodology section.
A list of all performance and satisfaction attributes (as independent variables) identified from the literature is presented in Table 1 (column 1).Using the correlation technique, possible significant variables for modelling were selected and are shown in column 2. Some degree of multicollinearity was found in several groups of variables.To rectify this problem, those variables which were highly correlated were combined into a single indicator as suggested by Lewis-Beck (1993).The variables used for modelling are presented in column 3.

RESEARCH METHODOLOGY
In the context of this paper, contractor performance criteria are defined as those used to measure the performance of contractors based on the views of clients.These criteria were determined through interviews with twelve experienced clients and supported by a literature review in the domain of (contractor) performance.These criteria were categorised under several main headings.A full list of the criteria identified is presented in Table 2.The questionnaire To provide the main modelling data, a questionnaire was developed based on the attributes and performance criteria identified.Respondents (clients) were asked to identify a recent (within 2 years) UK building project in which they were involved (referred to as the 'case project').Respondents were asked to relate all their answers to the questions contained in the questionnaire to this one case project.This strategy was designed in order to capture a true and realistic reflection of assessors' satisfaction/ dissatisfaction feelings.To protect the confidentiality of the other parties involved in these case projects, respondents were not asked to identify projects, nor to name other participants.

The survey
Following the development of the questionnaire and implementation of a pilot survey, a UK-wide questionnaire survey of clients was conducted.Distribution involved 536 experienced UK private and public clients, defined as those who regularly procure construction works from the industry.Private clients consisted of developers, retailers and financial institutions.Retailers and financial institutions were identified from the listing of Key British Enterprises (Dun and Bradstreet, 1998) representing the top UK retailers and financial institutions.Developers were identified from the Estates Gazette (1999).Public clients, i.e. local authorities or City Councils, were identified from the Municipal Year Book (1999).
Overall, 77 responses were received representing a 14.4% response rate.This relatively low response rate is about the 'norm' for construction management research and in many ways can be associated with the 'confidential' nature of the questions and the comprehensive nature of the research instrument.

DIMENSIONS OF CLIENT SATISFACTION
In this research, satisfaction is measured using an interval scale (i.e.scale 0-10) which assumes that satisfaction is a matter of degree, not an all or none property.To measure an abstract concept such as satisfaction, the concept should be defined at an operational (i.e.lower) level, which is observable and directly measurable (Johnson and Fornell, 1991).If the relationship between the abstract concept and the operational definition of satisfaction (i.e.performance criteria) is strong, the measurement instrument can be considered as valid and reliable to represent the abstract concept (Carmines and Zeller, 1979).
To derive the dimensions of client satisfaction the factor analysis technique was applied to the performance criteria of 50 responses (case projects).The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (0.673) confirmed that the factor analysis technique could be meaningfully applied (Norusis, 1994: 52-53).This was further confirmed by Bartlett's test of sphericity (chi-square = 3198.153,p < 0.0005).
This technique has been previously used in construction research.For example, Sawacha et al. (1999) utilised the factor analysis technique to determine the group of factors affecting site safety performance.Langford et al. (2000) used factor analysis to identify factors that prompted the strongest effect upon attitudes to safety management.Chan et al. (2001) used factor analysis to categorise project success factors into smaller number of groups.
The main purpose was to determine the number of common factors (i.e.satisfaction dimensions) that would satisfactorily produce the correlations among the observed variables (Kim and Mueller, 1978a).The method of extraction was principal components analysis.This method allows for data reduction and is considered as a means of exploring interdependence of variables.The number of factors determined was based on the criterion that the eigenvalue for each factor should be greater than 1 (i.e.Kaiser's criterion) (Torbica, 1997;Bryman and Cramer, 1999).This method is considered the most commonly used procedure to determine the number of initial factors to be extracted (Kim and Mueller, 1978b).To achieve the simplest possible factor structure in order to obtain more interpretable factors/dimensions, promax oblique rotation with the power (Kappa) valued at 4 was utilised.Oblique rotation (as opposed to orthogonal rotation) was utilised since it allows the presence of correlations between factors/dimensions.In fact, this assumption concurs with the real life situation since one aspect of performance should be, to some extent, related to other aspects.Further, Norusis (1994) claimed that oblique rotations have often been found to yield substantively meaningful factors since it is likely that influences in nature are correlated.
Promax has a reputation for demonstrable quality as evidenced in empirical studies (Gorsuch, 1983).Promax rotation raises the factor loading to a higher power in order that moderate and low loadings need to be lower while the high loadings remain relatively high (Gorsuch, 1983.).For example, the original loadings were 0.9 and 0.3.0.3 is one-third as large as 0.9, but the squared loading for the second variable is 0.09 which is one-ninth as large as the squared loading for the first variable (0.81).By raising the power of factor loadings, the factor structure becomes more interpretable.The power is known as the coefficient Kappa (k).
Gorsuch recommended that the proper power is that which gives the simplest structure with the least correlation among factors.Furthermore, he claimed that a good solution is generally achieved by raising the loadings to a power of four (SPSS default).In this research, Kappa = 2 and 6 were trialed, but these did not derive better solutions than Kappa = 4.
Five dimensions of client satisfaction were extracted and altogether represent 76% of the variations in the variables (refer to Table 3).The scores of the performance criteria under each dimension were then averaged to obtain the satisfaction measure (i.e.factor score).The factor score serves as an index of attitude towards a particular dimension of satisfaction under investigation (Torbica, 1997).From the original 48 performance criteria, 28 were included in one of the five factors.The validity and reliability of the satisfaction measures were confirmed.The validity assessment included the assessment of content, criterion-related and construct validity.The reliability of the measures (in terms of their internal consistency reliability) was assessed using coefficient Cronbach's alpha.For a full description of the validity and reliability of empirical measurement, readers may wish to consult Bohrnstedt (1970), Nunnally (1978), Carmines and Zeller (1979) and Litwin (1995).In construction, Torbica (1997) used a similar method for testing the validity and reliability of satisfaction measures of home buyers.

Multiple regression technique
As the purpose of the analysis was to develop models to predict levels of client satisfaction (a matter of degree, not an all or nothing property), the multiple regression (MR) technique was chosen as the modelling tool.Moreover, preliminary data examination showed a degree of linear relationship between dependent and independent variables.That is, MR represented an appropriate methodology for data of this nature (Lewis-Beck, 1993).The stepwise method for inclusion/exclusion of independent variables was utilised.Stepwise multiple regression is the most commonly used method for model building (Everitt and Dunn, 1991;Norusis, 1995;Bryman and Cramer, 1999).Draper and Smith (1981) and Kinnear and Gray (2000) regarded step-wise as one of the best variable selection procedures.The procedure selects the independent variables step by step.At each step variables already in the equation are evaluated according to the selection criteria for removal, and variables not in the equation are evaluated for entry.This process repeats until no variable in the block is eligible for entry or removal (Norusis, 1995).Fstatistics with probability of 5% and 10% were employed for entry and removal criteria as suggested by Draper and Smith (1981: 311).

CLIENT SATISFACTION MODELS
In total, seven models were developed to predict levels of client satisfaction based on contractor performance (refer to Table 4).

Summarisation of the MR models
In multiple regression, standardised coefficients (β ) can be used to assess the relative importance among the independent variables in determining the dependent variable within one model.Suppose, a simple model with two independent variables in standardised form (Lewis-Beck, 1993) is: , are partial regression coefficients.
Beta weight indicates the average standard deviation change in Y associated with a standard deviation change in X, when the other independent variables are held constant (Lewis-Beck, 1993).From the formula of beta weight, it is obvious that partial regression coefficients are corrected by the ratio of the standard deviation of the independent variable to the standard deviation of the dependent variable.
In comparing the importance of an independent variable across several models, beta weights are determined by the standard deviation of the variable in the models.Therefore, the standard deviation must be held constant in each model.In the case of comparisons across samples (e.g. a comparison of the importance of an independent variable in two models which were developed from two different samples), unstandardised partial regression coefficients are preferred to beta weights (Lewis-Beck, 1993: 57-58).In this research, the standard deviation of any independent variable is constant in several models since these models use similar independent variables (i.e. from the same sample).However, the standard deviation of dependent variables is not constant across several models due to the use of several satisfaction measures in the models.This means that beta weights of an independent variable are not comparable across several models.This problem can be overcome by multiplying the beta weight by the standard deviation of the dependent variable.
Based on this, importance weights (IWs) of the independent variables identified were established using the product of the standardised coefficient (beta weight, β) of the independent variables in absolute terms and the standard deviation of the dependent variable (S Y ).These weights were comparable across several models developed from the same sample.Then, the total importance weight (TIW) of the independent vari- ables was obtained by adding the importance weights (IWs) of the variable in each relevant model.Table 5 shows the calculation of TIWs for independent variables identified as useful predictors of the satisfaction measures.For each satisfaction measure, an IW for each variable was produced (Table 5, column 2 to 8).These weights were summed producing a TIW for each variable.These variables could then be ranked according to their TIWs in descending order (column 10).In order to ease discussion, based on their TIWs, the variables could be grouped into four categories, i.e. extremely important (TIW ≥ 2.0), highly important (1.0 ≤ TIW < 2.0), medium importance (0.1 ≤ TIW < 1.0) and some importance (TIW < 0.1) (last column).

Discussion of the models
The models identified seventeen independent variables as useful predictors.One variable was classified as 'extremely important', namely past performance of contractor in terms of cost, time and quality (CLATPPQU).This suggests that contractors whose past performance is good are more likely to satisfy their clients.Numerous scholars (e.g.Russell et al., 1992;Assaf and Jannadi, 1994;Holt et al., 1994;Tam and Harris, 1996;Hatush and Skitmore, 1997;Ng and Skitmore, 1999) have reported that past performance is one of the most important attributes influencing contractor performance.Therefore, this aspect should be carefully considered in the contractor selection process in order to achieve higher client satisfaction levels.
Three variables were classified as 'highly important': health and safety past performance and policy (COATTHS) quality control policy (COATTQC) the extent of variations caused by contractor (PRVARCO).
While COATTHS and COATTQC positively influence satisfaction, PRVARCO negatively influences satisfaction.This indicates that health and safety is a highly important factor for clients, even more so than quality.Variations often hamper project performance (Thomas and Napolitan, 1995;Ibbs, 1997) and hence will impact on satisfaction levels.Contractors should maintain high levels of safety and quality, and attempt to reduce variations if they are to satisfy their clients.
Variables classified as 'medium importance' comprised contractor, project and respondent attributes.Contractor attributes included: financial soundness and experience in type and size of project (COATFISI) qualification and experience of site personnel (COATTSP) formal training regime of site personnel (COATTTR) cost reimbursement method of contractor payment (COPAYCO2) contractor selected through negotiation (COSELCO2).
Financially sound contractors who have experience in similar projects are more likely to satisfy their clients.Such contractors are more likely to provide an effective level of service.COATTSP and COATTTR highlight the importance of site personnel to contractor performance and hence client satisfaction.That is, the site personnel represent a key resource in the production process.Contractors paid by cost reimbursement methods and selected through negotiation derive higher levels of client satisfaction.This suggests that less 'confrontational' methods of contractor procurement (rather than e.g.competitive tendering) are more likely to derive higher client satisfaction levels.
Project attributes classified as 'medium importance' were severity of variations (PRVARSE) project overbudget cost (PRBUDMO) planned project duration (PRDURPL) residential projects (PRTBD3) the extent to which the project is constrained by weather conditions (PRCONWE) public building projects (PRTBD0) number of storeys (PRSTO).
It is no real surprise that clients become dissatisfied when projects are completed overbudget and incur many variations.Interestingly, larger projects were found to raise satisfaction levels.This may be connected to the prestige associated with such projects, and the need to involve well resourced and experienced contractors whose performance may be superior to smaller firms.Clients were more satisfied on residential projects than public building projects.PRCONWE suggests adverse weather conditions may hamper contractor performance and hence negate client satisfaction.
One client attribute representing perceptions of the assessor was found to be of 'medium importance', namely those who perceive contractors to be claim conscious, to fail to deliver projects on time, and to be contractual (RSCO24).Clients who have such perceptions are likely to suffer lower satisfaction levels.This suggests that some degree of subjectivity is prevalent in the clients' assessment of contractor performance.

MODEL VALIDATION
To confirm the robustness (in terms of accuracy and consistency) of the models in predicting satisfaction levels, the models were validated using a hold-back sample of 27 case projects.
The predictive performance of the models was assessed by examining the residual (i.e. the difference between the actual and the models' predicted satisfaction levels).These were measured using two prediction performance measures: mean absolute deviation (MAD) and mean absolute percentage error (MAPE) (Kvanli et al., 1996).While MAD indicates the mean of absolute deviation of the predicted levels from the actual levels, MAPE indicates the mean of absolute percentage of that deviation from the actual levels.Using these measures, it could be concluded that a model yields predicted values with an average deviation of ± MAD, which is MAPE % from actual levels.For data of this nature, MAD of 1.5 to 2.0 and MAPE of 30 to 35% are considered acceptable.MAD of less than 1 and MAPE of less than 20% indicate good predictive performance.The performance of the models was also tested using chi-square (χ 2 ) analysis and Pearson's correlation coefficient (Edwards, 1999).
Results are summarised in Table 6.On average, the deviation of the predicted satisfaction levels is 1.12 (MAD), which is 22.22% from the actual levels (MAPE).This is quite good given the subjective nature of satisfaction/dissatisfaction judgements.Pearson's correlation tests confirmed that this level of accuracy is significant.Moreover, chisquare tests confirmed that the models have consistent predictive performance.These indicate that the MR models developed are valid and robust.The past performance of the contractor in terms of cost, time and quality was identified as the most important independent variable.This suggests that contractors whose past performance is good are more likely to satisfy their clients.Moreover, health and safety, quality control, and the variations caused by contractors were also found to be of importance.Health and safety is emerging as a significant determinant of client satisfaction.The most important variables indicate that client satisfaction levels are, to some extent, within the 'control' of contractor.The models also suggest that subjectivity is to some extent prevalent in clients' performance assessment.In sum, contractors should focus on those attributes found to be significant in order to continuously improve performance and enhance client satisfaction levels.
In summary, the models developed could be used, specifically by contractors, to improve performance and thereby improve levels of client satisfaction.This ultimately will help to create a performance-enhancing environment leading to harmonious working relationships between PC participants.This also ensures continuous performance improvement for the betterment of all involved.

Figure
Figure 1: Conceptual performance assessment model

Table 2 :
List of contractor performance criteria based on clients' management (sufficient quantity and quality of craftsmen) R2 ~ Equipment and plant management R3 ~ Management and co-ordination of subcontractors and suppliers R4 ~ Payment to subcontractors and suppliers (on time) R5 ~ Strength of contractor site team (i.e.quantity) R6 ~ Concern/awareness of environmental issues R7 Site personnel ~ Co-operation with client (i.e.client representative) E1 ~ Individual performance and ability E2 ~ Project manager performance and adequacy of authority E3 ~ Site manner (i.e.no loud noises and swearing) E4 Variations and drawings ~ Processing variations (e.g.speed, flexibility) V1 ~ Preparation of shop drawings and as-built drawings V2 ~ Contribution to development of design drawings V3 Completion stage and ease of delivery ~ Completion of defects C1 ~ Smoothness of operation and hand-over C2 ~ Quality of hand-over document (O&M manual, H&S) C3 ~ Ease / speed of settlement of final account C4 ~ Ease of delivery (general feeling on how things went) C5 Principal ~ Adherence to schedule (time performance) M1 ~ Adherence to budget (cost performance) M2 ~ Quality of construction and workmanship M3 Quality of service ~ Handling of complaints (effectiveness) Q1 ~ Telephone inquiries and correspondence handled courteously and adequately Q2 ~ Speed and reliability of service Q3 ~ Responsiveness to client's queries Q4 ~ Ability to make rapid decisions Q5 ~ Commitment of key person (active & continuous) spirit of co-operation / team work A2 ~ Customer focus / proactive to understand client A3 ~ Keep the client informed A4 ~ Communication (to coalition member & site person) A5 ~ Pro-active attitude toward problems A6 ~ Avoidance of claims (i.e.not claims conscious) A7 ~ Responsibility for their decision (understand the cost of their recommendations) A8

Y
is a dependent variable, X 1 and X 2 are independent variables,

Table 1 :
List of independent variables of clients' assessment of contractor performance

Table 3 :
Factor structure of contractor performance criteria based on clients' assessment

Table 4 :
MR models of clients' satisfaction

Table 5 :
Summary of independent variables' total importance weights (TIWs) derived from clients' assessment of contractor performance

Table 6 :
Summary of the validation of the MR modelsBased on a UK wide questionnaire survey of clients, multiple regression (MR) models have been developed and validated to predict several dimensions of client satisfaction resulting from the performance of contractors.For this research the MR technique was found to be appropriate, given the nature of the problem (i.e.satisfaction being a matter of degree) and results of preliminary data examination.