Policies to Avoid Cost Overruns in Infrastructure Projects : Critical Evaluation and Recommendations

Many infrastructure projects have cost overruns and there has been a lot of research both on why these cost overruns occur and what can be done to reduce hem. Bent Flyvbjerg is the leading researcher in the area and in this article his proposals are used as the starting point. Besides a literature review, a questionnaire was sent out to experienced Swedish project managers to find out what they thought could reduce cost overruns. The literature review and the questionnaire were the foundation for the proposals formulated in this article. Proposals concerned three areas: (1). Organisational macro-structure, e.g. using more PPP projects but also decentralisation of budgets where cost overruns in one project in a region lead to less alternative projects in the specific region. (2). Organisational quality: It should be easy to see when and where cost overruns occur and who was responsible. There should be a well-developed knowledge management system in the organisation and an organisation culture of openness with a focus on improvements. (3). Organisational processes, e.g. a systematic use of external reviewers in different stages of a project.


Introduction
Cost overruns have been a regular problem in large infrastructure projects.Flyvbjerg, Skamris Holm and Buhl (2003) presented results from a database with over 250 large projects from different countries and different time periods.The result can be summarized as follows:  86% of the projects had cost overruns compared to forecasted costs.The average overrun was 28%. The overruns were highest in rail projects with cost overruns of 45%.For roads it was 20%. Cost overruns were somewhat lower in Europe compared to North America and "Other geographical areas". There was no historical trend.Cost overruns in recent periods have the same magnitude as in earlier periods.A recent Swedish dissertation showed that there were large cost overruns in almost all major infrastructural projects in Sweden during the last decade (Lundman 2011).
This raises two main questions: Why is this the case and what can be done about it?The leading researcher in this area -Bent Flyvbjerg -presents three major explanations: Technological factors (unforeseen technical complications), psychological factors (optimism bias, "an inside view" of the project) and political factors (conscious underestimation in early stages in order to get the project started.The different explanations and possible policies against cost overruns suggested by Flyvbjerg can be found in e.g.Flyvbjerg, Skamris Holm and Buhl (2003), Flyvbjerg (2005), Flyvbjerg (2007), Flyvbjerg (2008), Flyvbjerg, Garbuio and Lovallo (2009) and Cantarelli, Flyvbjerg, Molin, and van Wee (2010).Brunes and Lind (2013) proposed a somewhat modified list that included Political/strategic factors, psychological factors, competence related factors and simply bad luck.
The focus in this paper is however on the second issue: Assuming that cost overruns depend on a combination of the factors mentioned above -what can be done to reduce cost overruns?In order to formulate a policy against cost overruns, a literature review was first carried out.As we were especially interested in policies that might be efficient in Sweden, a questionnaire was sent out to experienced project managers on both the client and contractor sides.This material was then used to formulate a credible policy package that should be able to lead to a reduction in cost overruns.This paper is therefore a combination of a review paper, an empirical study and a conceptual policy paper.

Research Method
The Literature Review The literature review started with the work of Bent Flyvbjerg and then focused on studies published in academic articles in the last 3-4 years.The choice of Flyvbjerg should be uncontroversial given his large number of publications in the area of cost overruns and how often he is referred to.As there is so much written on cost overruns, it was decided to limit the analysis of other literature to the most recent studies.The idea is of course that if new ideas have come up they should be found in the most recent literature.Literature searches were made primarily through the extensive data bases available through the library at KTH and references found in these articles.All articles were checked carefully for statements about ways to reduce cost overruns and arguments for the proposals made.

The Empirical Study about Sweden
Beside the view from researchers covered in the literature review, we also wanted to find out the view of experienced project managers concerning what could be done to reduce cost overruns.A questionnaire was therefore sent out to two groups: project managers working for the Swedish Traffic Administration (STA, Trafikverket) and to senior staff in three leading contractor companies.The first group was found by going through current projects on the STA website.The experienced project managers on the contractor side was found by asking leading staff in each company to recommend suitable persons that could participate in a questionnaire about cost overruns in infrastructure projects.It is therefore a strategic sample and not a stochastic sample, and this has limitations when it comes to generalizing.The advantage with this strategic sample is however that it was possible to choose competent and experienced project managers many of whom had worked on both contractor and client sides.Almost all of them had worked more than 10 years in the industry.
The number of questionnaires sent out was 230 in total and the completed questionnaires returned were 97, giving at total response rate of 42%.As presented in Table 1 the number of questionnaires started was 106 and some respondents did not answer every question.The difference between respondents and non-respondents was checked but no systematic difference could be found.The survey was constructed following some general principles.There were five possible answers concerning to what extent they agreed with the statements made.It was believed that it would be difficult for the respondents to distinguish between more alternatives.We used a neutral "maybe" and added a "don't know" option in order to have an exhaustive list of possible answers.

Limitations
As with all research methods there are limitations with questionnaires.A questionnaire is based on communication between the researcher and the observer.The first problem concerns definitions, as a questionnaire does not make it possible for the respondent to ask questions of the researcher on matters that seem unclear.One such matter in this paper might be the definition of cost overruns.To reduce this problem a definition of cost overruns was presented at the beginning of the questionnaire.Another problem that is discussed in the method literature is how respondents answer a question depending on which order it is given in.It could be the case that claims asked for first are given a higher positive response rate than claims asked for later in the questionnaire.
Another limitation is of course the external validity; can the results be applied to infrastructure projects in general?The questionnaire was sent to project managers found on the Swedish Traffic Authorities' website and even though it was not a random sample it should be representative of the population of project managers working for the STA.The project managers on the contractor side were chosen for their experience and even though they are not representative of project managers in general, they were considered to be the best informants concerning cost overruns.There might be some risk for bias on some questions and this issue will be returned to when analysing the results presented below.As the number of respondents is rather small, comparisons between the answers from the client and contractors should be seen more as indications than statistically established differences.

Measures Proposed in the Literature
The Measures Proposed by Flyvbjerg In the works of Flyvbjerg (see references below) two main ideas can be found about how the risk of cost overruns can be reduced.The first is reference class forecasting and the second is increased public sector accountability through more involvement by private parties.
In several of the articles (e.g.Flyvbjerg 2007Flyvbjerg , 2008;;Cantarelli et al. (2012)) "Reference class forecasting" is proposed as one way of curbing cost overruns.The method has also been endorsed by the American Planning Association: The new method achieves accuracy in projections by basing them on actual performance in a reference class of comparable actions (Flybjerg 2008, p 3).
The idea is that instead of (only) doing an "inside" view prediction of the cost of a project, one should also take an outside view and compare with earlier similar projects.
There seem to be two somewhat different interpretations of Reference class forecasting.
One is that cost from earlier projects (adjusted with a price index) should be used to estimate the cost for the current project.One can read: A reference class forecast of a given planned action is based on knowledge about actual performance in a reference class of comparable actions already carried out (p 4).
More specifically, reference class forecasting for a particular project requires the following three steps:  Identification of a relevant reference class of past, similar projects.The class must be broad enough to be statistically meaningful but narrow enough to be truly comparable with the specific project. Establishing a probability distribution for the selected reference class.This requires access to credible, empirical data for a sufficient number of projects within the reference class to make statistically meaningful conclusions. Comparing the specific project with the reference class distribution, in order to establish the most likely outcome for the specific project."(p 8) The other interpretation is that one should look at average cost overruns in earlier projects and then make adjustments of the initial predictions based on these historical figures.Flyvjerg (2008, p 11) quotes a British report: There is a demonstrated, systematic, tendency for project appraisers to be overly optimistic.To redress this tendency appraisers should make explicit, empirically based adjustments to the estimates of a project's costs, benefits, and duration . ..[I]t is recommended that these adjustments be based on data from past projects or similar projects elsewhere.(HM Treasury, 2003b, p 1) In this context Flyvbjerg talks about providing: Empirically based optimism bias uplifts for selected reference classes of transportation infrastructure projects, and secondly, providing guidance on using the established uplifts to produce more realistic forecasts of capital expenditures in individual projects.(p 11) Data about such uplifts are presented in the articles.
Our view is that this second form of reference class forecasting is very problematic as it would punish participants who try to improve the quality of the cost estimations.If these better estimations are also uplifted then these projects would seem to be especially costly and would then be ranked lower.In the rest of the article this form of reference class forecasting will therefore not be discussed.
The second proposed policy measure is increased transparency and use of public-private partnerships.
Flyvbjerg (2008, p 593) writes: Two basic types of accountability define liberal democracies: (1) public-sector accountability through transparency and public control, and (2) private-sector accountability via competition and market control.Both types of accountability may be effective tools to curb planners' misrepresentation in forecasting and to promote a culture which acknowledges and deals effectively with risk.
On the first point Flyvbjerg mentions for example that central governments should not use project specific grants, that they should demand that predictions of costs and benefits are peer-reviewed and that everything should be open to public debate before decisions are taken.
On the second point he mentions that economic risks should be shared between the public and the private sector and also with for example a local government if they have taken the project initiative even though most of the financing is approved at higher levels.Private investors should be willing to supply at least part of the capital and should receive a return that is related to the outcome of the predictions of costs and benefits.
In the rest of this article Public Private Partnerships will not be discussed, not because we have anything against them, but that there always will be a number of projects where this form is not suitable or not used for political reasons.Currently in Sweden, and many other countries, almost all projects are procured as individual construction projects with either Design-Bid-Build contracts or Design-Build contracts and that is the situation that this article focuses on.

Proposals in Recent Studies of Cost Overruns
There is a continuous flow of studies concerning cost overruns and in this section proposals by a number of recent studies are presented.Rahman et al. (2013) use a statistical method to identify causes and from that they conclude that site management factors are of major importance and therefore they suggest (p 1970) that "improved site management and supervision of contractors can…result in control of cost overruns."Most other studies however point out other factors as more important, and that most cost overruns occur in the design and planning stage (see Brunes and Lind (2013) for an overview of causes of cost overruns).Therefore measures concerning site management, though important, should not be expected to be efficient when considered against most cost overruns.
Doloi (2013) makes a similar statistical analysis and in the concluding section he writes: The findings of this research highlight a new emphasis on well-developed technical skills as the key in controlling cost overruns in modern projects.
Measures that focus on competence development will be returned to below.
Two recent studies focus more directly on methods for predicting costs and risks by statistical analysis of earlier projects.Berechman and Chen (2011) developed a combination of reference class forecasting and decision tree analysis, where more or less risky alternatives are included.The probabilities in the decision tree analysis are derived from a set of observed projects.They write (p 103): The main conclusion from this paper is that the risk of cost overruns can, and should be, incorporated in transportation project evaluation and decision making.
Love et al. ( 2013) argue for a similar approach and try to fit a statistical distribution to a set of actual projects with different characteristics that can then be used to predict cost overruns.
The results are however rather disappointing and they write: No significant differences for cost overruns were found among procurement method, project type and contract size.(p 329) Chevroulet, Giorgi & Reynaud (2012) analyse six projects in order to find a pattern that could be used to predict cost overruns.This is another version of reference class forecasting.They also find a large divergence and write (p 301): The study also highlights the lack of reliable data for carrying out useful reference class forecasting or scenario analysis as recommended by Flyvbjerg.
At the end they make a list of suggestions under three headings:  Before construction: Improvement in decision support. During construction: Management and monitoring. After construction: Feedback and consolidation of knowledge.
In a recent Swedish report (Nilsson 2013) the following conclusions and recommendations concerning the Swedish Transport Administration are presented.Data on budgets and costs are not collected in a systematic way and this has to be changed in order to make some method of reference class forecasting possible (p 26).He further recommends the use of the Norwegian system where independent external reviews are made for projects above a certain size (p 27).

What Swedish Project Managers Think would Reduce Cost Overruns Background Questions
As can be seen in Table 2, almost all project managers answered that cost overruns occur often or sometimes in infrastructure projects.More than half of the respondents answered that cost overruns occur often.Of the project managers from STA, 89% had experienced projects where there had been cost overruns and 96% of the project managers on the contractor side had similar experience of cost overruns in projects.The Role of Procurement Form The respondents were presented with a number of statements and asked to choose from a number of alternative answers.The first question related to a current debate in Sweden about the role of the procurement forms Design Bid Build versus Design Build.Among the STA-staff changing from Design Bid Build was not expected to reduce cost overruns, while the contractors had a more positive view of the effects of changing procurement type (see Table 3).There might however be some bias in the answers on this question as a number of contractor companies have argued strongly for increased use of Design Build contracts (see e.g.www.ncc.se).

The Role of External Reviewers
As was reported in the literature review above, several researchers recommend the use of external reviewers to reduce both psychological and political/strategic bias.This idea does not get strong general support, but there are large differences between the groups as 50% of the respondents from the contractor side support the statement while the share is 28% on the client side (see Table 4).Here there might also be some bias in the answers, especially from the client side, as the use of external reviewers could be seen as a criticism of their work.

Resources Spent in Early Stages and Risk Analysis
The technological explanation of cost overruns suggests that if more resources are spent on technical investigations in the early stages, then there would be fewer surprises and a

Incentives
The next question dealt with consequences for those responsible for the cost overruns; see Table 7.The idea behind the question is that if the persons directly involved in the design and the cost calculations would for example risk being fired if there were cost overruns that would affect the quality of their work.As can be seen below, this idea does not get much support and the difference between the groups is small.

Overview of the Policies
Simplifying the answers to "Yes", "No", "Maybe" and "Do not know" the result from the survey can be summarized as shown in Table 8.Project managers in Sweden regard better technical investigation and better risk analysis as good policies against cost overruns.The majority do not regard the use of Design Build procurements as a cure for cost overruns.
The opinions were divided concerning the effects on cost overruns of using external reviewers and introducing stronger consequences for those responsible.

Other Policy Measures
At the end of the questionnaire was an open question where the respondents could present other policies that they believed would be efficient against cost overruns: See Table 9.Some of the factors mentioned actually overlapped with the policies mentioned in the questions and have therefore been excluded in the list below.It is interesting to note that competence related factors are mentioned by both sides.

Higher client competence concerning calculation
More contacts between client and contractor in the early stages More continuous monitoring of projects, more centrally controller internal reviews, faster reaction when signs of cost overruns More feedback, less prestige More cooperation between client and contractor (partnering) Bonus systems for those involved Change the procurement process, less on lowest price, more on documented competence

A Policy Package against Cost Overruns
The aim of this section is to give a systematic overview of what could be recommended as a structured policy package against cost overruns from a public sector perspective.The list is based on both the measures recommended in the earlier literature and on the results from the questionnaire, but also on more general views about what drives an organization.The measures will be grouped in three broad categories: Organizational macro-structure, Organizational quality and Organizational processes.
It should however be stressed that reducing cost overruns is not the only important target.Mandell and Brunes (2014) show that there might be a rational and efficient procurement process that cause side-effects in which cost overruns might occur.As argued in Warsame, Lind & Borg (2013) the quality of the final product is of course also important.

Organizational Macro-Structure
One example of measures on this level is Flyvbjerg´s proposal to carry out more projects as PPP-projects.A more radical solution would be to go in the direction of more complete privatization of infrastructure facilities, where the private sector both invest and are free to charge the customer as in any other market.
Organizational changes within the public sector can also affect the incentives to reduce cost overruns.Hasselgren (2013) discusses different ways to approach decision making and budgeting in the public sector and contrasts two alternatives.The first is based on welfare economics where projects are evaluated with a social cost benefit analysis and then paid with general taxes.The second is based on Coase´s criticism of the welfare economics model and proposes that each sector should pay its own costs.Roads are paid with car taxes or example.This means that if there are cost overruns in one road project, these taxes have to be raised or other road projects will be postponed, while in the welfare economics based model the cost is spread over the whole population.The incentives to reduce cost overruns would therefore be weaker in the welfare economics model.The same argument can be used for delegating resources for public investment budgets to regions, such as an infrastructure investment grant to a region on a per capita basis, and then let the regions decide on the projects.In that way a cost overrun in one project in the region means that other projects in the same region would have to be postponed.The common idea behind these two proposals is to move the consequences of the cost overruns closer to the decision maker and thereby strengthen incentives to avoid cost overruns and thus reduce the risk for primarily strategic/political manipulations.

Organizational Quality
What can be seen in both our questionnaire and in other studies is that procurement type does not seem to be a very important issue in the context of cost overruns: Both Design-Build and Design-Bid-Build can lead to cost overruns, and this is not surprising from the perspective of the next group of factors: Organizational quality.This can be subdivided into a number of interrelated aspects (see Warsame, Lind & Borg 2013 for a similar approach, but where the focus is on quality problem and not cost overruns): "Order in the bookkeeping".Several Swedish studies e.g.Riksrevisionen (2010Riksrevisionen ( , 2011 a,b) a,b) and Nilsson (2013) show that it is not easy to find out how much projects really cost and during what stage cost overruns occur.Without this basic data any kind of reference class forecasting would be impossible and a minimum demand of an organization is that it be able to present what various projects have cost in the past and why there have been cost overruns in earlier projects.Brunes and Lind (2013) present a framework that can be used for this purpose.
"Knowing who did what".If it is not possible to know who was responsible for carrying out certain tasks, or approving certain calculations, then it is also not possible to know who has a good track record and who has a bad track record.Even if the questionnaire did not give strong support for idea of increasing the individual consequences of cost overruns, this aspect seems important from a transaction cost theory perspective where individual utility maximizing based on expected consequences is a fundamental idea.
"Knowledge management".This is a growing area in many sectors of the economy, (see Ilina (2011) for an overview).If we look at knowledge management from the perspective of cost overruns it concerns:  Systematic evaluation of earlier projects and finding patterns that can make it easier to see what can lead to cost overruns in specific projects and what can be done to avoid this in coming projects.This is of course the core aim in reference class forecasting. Collecting information about other organizations experiences, e.g.traffic agencies in other countries. Recruitment of staff with the right combination of training and experience. Policies to develop the competence of the staff.Increased competence was also seen as important by many respondents in the questionnaire.
"Organization culture".Ravasi and Schultz (2006) define organizational culture as a set of shared mental assumptions that guide interpretation and action in organizations by defining appropriate behaviour for various situations.Without an organizational culture that promotes openness and a continuous focus on improvements, it will be difficult to implement the more specific measures that have cost overruns as their target.There have to be clear signals and behaviour consistent with these signals from the top management and -for government authorities -from leading politicians.

Organizational Processes
Here the focus is on processes directly aimed at reducing the risk of cost overruns in specific projects and one proposal discussed in the literature, that also seems logical given the different theories about the causes of cost overruns, is the use of external reviews.Some countries (see Nilsson 2013) already use this for large projects, but our view is that it can be used in all kinds of projects and in all stages.
In smaller projects the use of experienced persons within the organization in a more informal review process concerning both design, technical investigations and cost calculations is one way or reducing risks related to lack of competence, cognitive biases and strategic manipulation.One can think of a hierarchy of projects related to size and technical complexity and where different types of experts -from the consultancy sector or from the academic world both nationally and internationally -can be involved to provide second opinions in more or less formalized ways.More information in all stages of the projects can also be made publicly available on the organizations' websites, and thereby make feedback possible.

Concluding Comments
Policy proposals are based on predictions of the effects of various actions, and it is well known that predictions in complex environments are always uncertain.There can be complex interactions and unexpected consequences.Our belief, based on the arguments and studies presented, is that the proposed system of measures presented above would be efficient.But is important to continuously revaluate what really happens when a new policy is introduced.An experimental approach where different methods are tested in different projects can be a rational strategy when there is uncertainty about the effects of a policy.This is a part of the organizational culture and maybe that is the most important issue.
The proposals above are directed at different types of practitioners.Organizational macrostructure is decided by policy makers at the central government level, even if some of them can be introduced independently at regional levels.Organizational quality and organizational processes are, on the other hand, primarily issues for boards and top management in government authorities, but it could also be argued that it is the responsibility of everyone in an organization to contribute to transparency and efficiency.

Table 1 :
The number of survey participants and the response rate

Table 2 :
Question.What is your view on the prevalence of cost overruns in infrastructure projects?

Table 3 :
Statement.Cost overruns would be considerably less if Design Build would be used instead of Design Bid Build?

Table 6 :
Statement.Cost overruns would be considerably less if the client puts more resources into evaluation of risks early in the project

Table 7 :
Statement.Cost overruns would be considerably less if it hard larger consequences for those responsible

Table 8 :
Simplified overview of the results

Table 9 :
Other measures that can reduce cost overruns