ASSESSMENT OF PROFESSIONAL COMPETENCE IN A CONSTRUCTION MANAGEMENT PROBLEM-BASED LEARNING SETTING

INTRODUCTION The idea of a competency-based approach to higher education has generated a wide range of responses ranging from resentment of imposed changes to hopeful optimism about the opportunities that might exist. Early surveys conducted by Bowden and Masters (1992) indicated that the most prevalent response from academics towards a competency-based approach to higher education is negative, viewing the approach as “too narrow... mechanistic and prescriptive.” On the other hand, there was also a significantly positive response which saw competency based education and training as benefiting higher education in clarifying intended outcomes of undergraduate programmes, particularly in relation to workplace requirements.


INTRODUCTION
The idea of a competency-based approach to higher education has generated a wide range of responses ranging from resentment of imposed changes to hopeful optimism about the opportunities that might exist.Early surveys conducted by Bowden and Masters (1992) indicated that the most prevalent response from academics towards a competency-based approach to higher education is negative, viewing the approach as "too narrow... mechanistic and prescriptive."On the other hand, there was also a significantly positive response which saw competency based education and training as benefiting higher education in clarifying intended outcomes of undergraduate programmes, particularly in relation to workplace requirements.

Competency based assessment, higher education and problem based learning
Increasingly, professional courses are seeking to determine graduate outcomes through assessment within a competency framework.It is necessary to distinguish competency standards appropriate for higher education from the standards used at vocational education and training levels (Kingsland and Chen, 1993).
It is useful in defining competencies appropriate to the development of higher education curricula, or higher-order competencies, to contrast the attributes and objectives of these competencies with those appropriate to vocational training and those appropriate to professional practice.Where this approach to competency-based education and training is adopted, a broader view of the competencies assessed must be taken and this can only be achieved when performance is viewed holistically and in context.
In the context of professional courses, a competency oriented approach to assessment differs from the traditional behaviourist approach to task performance that has pervaded discussions of competency based assessment.Assessment of professional competence, especially in problem-based learning (PBL) programmes, is fundamentally different from competency based assessment described in vocational training assessment.To consider it to be the same negates the characteristics of PBL which, in addition to drawing from practice contexts and using "real life" situations, emphasises thinking skills (such as problem solving, critical thinking, decision making), requires an integration of knowledge, skills, attitudes and behaviours, promotes self direction and lifelong learning, and recognises that learning is shared and constructed in the context of relationships with others (PROBLARC, 1996).Indeed, these characteristics of PBL form the basis of the so-called "higher order" competencies.
When used in professional education courses, PBL claims to provide opportunities for the individualisation of the learning experiences in order to engage and ultimately empower learners.Use of PBL is claimed to result in empowered exploration, deep learning, creativity and critical thinking (Engel, 1991).The goals of higher education are to empower individuals to be "creators in the development of knowledge and skills, leaders of change, not just reproducers of existing knowledge and skills" (Chen et al., 1999).Therefore, assessment approaches in PBL programmes in higher education should be designed to support these outcomes.

Implications of competency based, problem based curricula for professional education in Australian universities
In Australia, it has been clearly expressed in government policy statements that the competency-based approach embodied in the plan for integrated national training and education would not tie universities to the competency standards established by training agencies in relation to vocational proficiencies.Indeed, relevant professional bodies and higher education institutions offering professional courses are expected to determine for themselves any relationship between competency standards for the professions and the education provided (Baldwin, 1992).
Debate about whether the competency standards determined by the professions are the appropriate ones to drive the development of curricula in professional courses offered by higher education institutions is tied to the contentious issue of what the role of higher education should be.It could be argued that higher education should adequately prepare graduates to successfully make the transition from a generalised role model to a diverse range of specialised roles possible in professional practice.
The development of curricula in professional courses is frequently based on a conception of assumed and notional roles carried out by practitioners.Cowdroy (1991) proposed that professional education is conditioned by the casting of the practitioner into idealised role models from which competencies that drive curriculum development are derived.However, in reality, practitioners specialise and develop skills profiles according to the particular roles they take on after graduation.This suggests that higher education should address a wide platform of competencies which prepare graduates for development into specialised roles with continuing professional development.The preparatory nature of these competencies distinguishes, where necessary, competencies appropriate for the higher education context from competencies appropriate to the practice context.These higher order competencies also embrace the important intellectual processes of critical reasoning and reflection (Barnett, 1990), conceptualisation, innovative thinking and the ability to transfer skills and knowledge to new tasks and situations (Barnett, 1994).Higher order competencies may be specific or complex combinations of lower order competencies or individual attributes.These combinatorial competencies are also synergistic in nature, representing more than just the aggregate of knowledge and skills required for the successful performance of the component attributes.In practice, even specific competencies require the professional to understand when and why these combinations of attributes are called for (Gonczi et al., 1990).The exercise of judgement, and all the attendant intellectual processes, is often called for.
Higher order competencies also reflect a creative dimension.They require more than the reproduction of specified performance to set standards.Performance processes associated with higher order competencies are flexible, open-ended and complex.Analysis, problem-identification and solution, ideas generation, and on-going learning are some of the skills that characterise higher-order competencies.It is also essential that higher-order competencies include the ability to generalise, extend and transfer skills that may have been developed in a specific learning situation.
Professional courses will never be able to expose students to all knowledge and skills that practice may demand.Professional education should enable students to "seek out, integrate and use knowledge" (Alverno College, 1985).The development of appropriate integrated research skills enables higher-order competencies to be maintained in situations subject to information obsolescence and expansion.Higher-order competencies should also require performance to strongly demonstrate the integration of attitudes with appropriate knowledge and skills in complex situations.This may be reflected in choices and actions which assess and determine priorities, recognise conflicts and propose compromises, and respond to wider societal demands, moral and ethical questions (Dall'Alba and Sandberg, 1993).Clearly, a problem based approach to curriculum and instruction is consistent with a desire to prepare graduates for professional practice contexts, and the competencies essential to lifelong learning and professional performance.

Assessment of professional competence: a goal of PBL curricula
Assessment is widely recognised as one of the most powerful influences on the learning approaches and behaviours adopted by students (Biggs, 1996;Feletti, 1997).While there may be debate about the extent to which assessment of professional competence is the sole outcome of the curriculum, it is clearly one of the goals of professional education.
While conventional assessment methods are often considered to be at odds with problem based learning approaches used in professional courses (Cowdroy et al., 1999), and even at odds with the goals of higher education, use of professional competence as a framework for both learning and assessment is implicit in any PBL curriculum.Assessment of professional competence focuses on outputs -what learners can do as result of learning process -rather than on inputs, in the sense of what subjects or courses have been completed (Harris et al., 1995).Thus, a PBL approach to curriculum and instruction accommodates the needs of higher education and professional performance and a competency oriented approach to assessment provides a framework in which to collect data about students' performance in a range of areas using a variety of assessment tools -conventional or otherwise.Figure 1 demonstrates the interrelationship between competency-oriented assessment and PBL as a curriculum design.

CASE STUDY
At the University of Newcastle, the Bachelor of Construction Management (Building) programme views curriculum design as a dynamic and interactive process.Consistent with intra-professional directions, it has incorporated the competency standards of each relevant profession into its programme.

Description of the programme and its implementation of assessment of professional competence
In the construction management programme, the use of a competency-based approach to assessment reflects changes in the practice of the discipline.Accrediting authorities for construction management and construction economics courses in Australia, the Australian Institute of Building (AIB) and the Australian Institute of Quantity Surveyors (AIQS), have decided to use competency-based criteria for future accreditation of undergraduate programmes in Australian universities (AIB, 1997;AIQS, 1998).The introduction of competencybased criteria for the accreditation of undergraduate courses in construction management provided the Department of Building (the discipline of 'Building' is now located within the 'School of Architecture and Built Environment' after a University wide restructuring process in 2002) at the University of Newcastle with an opportunity to review and introduce innovative processes into its problem-based learning course (Jefferies and Chen, 1999).
The trend in general education is to view knowledge as constructed within personal and social contexts (Biggs and Telfer, 1987;Leder, 1993) and the need for students to acquire lifelong learning skills (Bates et al., 1999).

Competency oriented assessment
The Bachelor of Construction Management (Building) programme has been using an integrated problem-based learning approach and an outcomes-based description of its curriculum since its inception in 1990.These outcomes are defined as objectives for each of the sub-disciplines and stages of the course.Using a hierarchical process these are then expressed as skills, knowledge and processes that can be demonstrated by students as evidence of competence.The assessment criteria are directly linked to the learning objectives at each level and to the overall course objectives.Appropriate learning activities, generated by projects, are designed to permit students to develop and demonstrate their competence as required by the programme.
A two-stage assessment process has been implemented with the continued use of integrated problem based learning.Whereas previous assessment processes involved the grading of formal submissions in accordance with the satisfaction of stated learning objectives linked to those submissions, the assignment of marks in this single stage assessment process is not consistent with the assessment of elements in a competency-based approach.The synergistic nature of higher-order competencies with regard to its component elements of competence also demands a more sophisticated assessment process to identify and recognise the higher-order attributes that characterise these competencies and distinguish them from lower-order competencies.
At the start of each problem phase (a phase being defined as a period of learning culminating in a formal submission), students are advised of the elements of competence to be demonstrated and the evidence required.Normally these competencies are demonstrated through the formal submission, which contains the required evidence.
During the phase students receive formative assessment in the form of feedback during tutorials, trial submissions and self or guided assessments.This feedback is intended to facilitate the development of the submission response and does not have a direct impact on the students' final grades.At the time of formal submission, students complete a checklist for self-assessment to be submitted with the assessment product.This checklist also identifies the evidence offered to satisfy the learning objectives and assessment criteria.The checklist is returned with the assessor's summative feedback.The feedback identifies whether the criteria have been met satisfactorily and provides qualitative feedback indicating the quality of the submitted work and, if appropriate, provides guidance on how the work may be improved.Should any student fail to adequately demonstrate any of the required elements of competence, they may continue to work at it until they satisfy the evidential requirements.This process is necessarily limited by the administrative requirements for an enrolment period (semester or year).The assessment of these elements of competence is simply on a 'satisfactory' or 'not satisfactory' basis, consistent with other forms of competency based assessment.
The second stage of assessment occurs at the end of the semester and is a holistic assessment of higher-order competencies.At this stage, students present evidence of performance demonstrating higher-order attributes to a panel of assessors.This evidence generally takes the form of a portfolio of work accompanied by a reflective journal, verbal presentation or any other relevant form of evidence decided and designed by the student.
The remainder of this paper explores the issues that have emerged in the context of the discipline and the process of implementation of assessment of competence.

METHOD Overview of case study approach and relationship to PBL principles
Case study is a practice based research method that involves analysis and description of a "real life" situation to promote understanding (Yin, 1989).According to Yin (1989) "as a research endeavour, the case study contributes uniquely to our knowledge of individual, organisational, social and political phenomena."The ability to appreciate such phenomena in relation to practice is one of the goals of PBL curricula.
In an educational context, using case study design requires the evaluator to note what happens to learners who have received the instructional treatment being evaluated (Popham, 1993).In this paper, the 'learners' are both students and staff of the of the Building discipline, at the University of Newcastle, Australia, and the 'treatment' is the introduction of competency based assessment into the Bachelor of Construction Management (Building) programme.
The authors of this paper believe that a case study approach is consistent with the philosophy of PBL as both value the examination of situations in context and are practice oriented.Similarly, the collection of a range of data from both staff and students reflects a commitment to examining multiple perspectives and viewing staff as learners in any implementation of an educational intervention.
While limitations of the case study approach are well documented and include numerous threats to internal validity (Popham, 1993) and limitation in generalisability, the advantages of a case study approach are that it enables comparison of different perspectives and illuminates views and experiences (Thomson, 1998).
Data collection methods used by the authors to inform the content of this paper included: interviews with both staff and students in the school about their experiences of assessment analysis of student reflective journals in relation to assessment analysis of staff workshop data in relation to assessment observation of the conduct of assessment of competence

RESULTS
The data revealed a number of recurring themes for both staff and students in assessment of professional competence.These were: assessment of professional competence makes what had previously been implicit quite explicit concern about a perceived lack of clarity, equity and objectivity in assessment of professional competence issues related to learning strategies and construction of knowledge.

Making the implicit explicit
Staff in the study revealed they had quite implicit ways of assessing students and invariably indicated that they were assessing students' ability to communicate clearly and professionally, and their enthusiasm, interest, confidence in and commitment to the learning experience.However, it seems it is often the students' responsibility to demonstrate these characteristics rather than the assessment criteria's purpose to elicit behaviours indicative of these.Assessment based on professional performance competencies demands that evidence to support judgements about performance be collected.The implicit nature of the assessment by educators who rely on their own expert judgement of performance is not adequate grounds for deeming a student's progress unsatisfactory.
The Construction Management Programme has sought to address this through presentation of a portfolio before a panel of experts.

Concern about a perceived lack of clarity, equity and objectivity in assessment of professional competence
Staff and students in the study expressed concern about these issues.There was a tendency among staff to use the holistic performance standards as a substitute for collection of evidence about student performance.While staff highlighted their own confusion about how the assessment related to the students' level, and tensions between use of professional competence as both a formative and summative assessment method, students expressed dissatisfaction with "putting in a lot of work to find out that you have got it wrong".Related to this, staff expressed concern about the tension between recognising that students in a PBL situation would approach things from a variety of perspectives and ensuring that there was some 'core knowledge' underpinning a student's learning experience.
Students reported that that they were unclear about the purpose and structure of assessment.Although students in first and second years indicated that the process of assessment became increasingly clear to them as they progressed throughout their course, students indicated that they sought guidance about strategies to meet the goals as well as more clear articulation of the goals.Despite this there was evidence to suggest that students did acquire the ability to define their progress and identify areas of improvement in their performance.
While staff expressed difficulty in determining and discriminating between the level of students' performance in terms of year expectations, students seemed to be able to discriminate between the level were at and their achievements, perhaps because they were in the process of developing professional competence.Staff, on the other hand, have expectations of performance based on their understanding of practice and their own professional experience.

Issues related to learning strategies and construction of knowledge
For both staff and students in this study there were difficulties in seeing how the professional competencies reflected their ability to conceptualise their discipline.While one should not be surprised as to students' difficulties with this (after all the goal of any programme is to teach them this), it is disturbing when staff do not seem to be able to demonstrate a relationship between professional performance criteria and their discipline.

DISCUSSION
This study raises many issues for the Construction Management programme and the nature of assessment of professional competence.

Implications for curriculum planning in non integrated PBL curricula
The AIB and AIQS use competency-based criteria accreditation of undergraduate programmes in Australian universities (AIB, 1997;AIQS, 1998).The Construction Management programme at Newcastle uses a curriculum mapping approach in order to demonstrate how it meets the accreditation requirements of the AIB and AIQS along with the needs of the industry, students and the community (Chen and Ostwald, 1995).
Education should be transformative, emancipatory, dialogic and reflective (Cranton, 1994;Friere, 1972;Mezirow, 1985).Thus, it would seem that assessment of professional competence is consistent with the direction of the discipline.
In this study there was evidence that students and staff reported being unable to see how traditional assessment tasks articulate with the profession's competency statements.As the competency based approach provides a common framework for developing and linking many aspects of work and education (Preston and Walker, 1993), it may be of assistance to students if each assessment item were identified in terms of how it related to the profile of the graduate and the professional competence of a construction manager.

Implications for staff development activity
There is substantial evidence that students view their educators, both on campus and in the industry-based environment, as role models as they seek to determine their professional roles (Chen et al., 1994).Evidence in this study suggests that those role models have difficulty themselves in seeing how the competency statements relate to 'everyday' practice.Additionally, some staff, particularly the industry-based educators who are isolated from on-campus activity, indicate that they need guidance and support in assessment procedures.The School has attempted to address these issues through staff development programmes that focus on assessment.Preliminary evaluations of these programmes suggests that these have resulted in increased awareness of the curriculum and an increased understanding of the use of the assessment tools and principles of assessment.

Implications for existing assessment strategies
Assessment in higher education is driven by institutional as well as educational demands and therefore needs to serve formative, summative and normative purposes.Table 1 differentiates between these.

Professional judgement
Formative assessment needs to occur during the learning process in order to assist and support continuous learning and needs to provide meaningful feedback to the student.Summative assessment should occur at the end of the learning process or at some predetermined stage (such as formal examinations) and be a simple judgement of whether the performance demonstrated by a student meets satisfactory standards.Where performance is not satisfactory, specific remedial education and training needs can be readily identified from the assessment reference criteria.Normative assessment is usually required by institutions for purposes such as honours and other awards.This requires performances to be judged on a relative basis and makes marking a necessary procedure.This is more difficult to reconcile with an assessment process which is based on professional judgement of broad and complex criteria deriving from higher order competencies rather than the more precise testing instruments based on clearly defined problems.
An implication communicated in Bowden and Masters' (1992) discussion of assessment of outcomes is that outcomes must be expressed as objectively observable and assessable items.This does not address the issue of synergy in integrated learning.It also would negate the possibility of trying to develop skills, attitudes and behaviours for which only subjective criteria, standards and assessment are possible.
The key point is, therefore, to shift the emphasis of assessment away from judgements about knowledge acquisition and towards the ability to perform specific workplace tasks and roles competently.It is where these tasks involve large, complex processes and require the integration of knowledge from a wide variety of sources that we can effectively encapsulate competency assessment of higher order skills.Harris et al. (1995) describe competencybased assessment as a process of risk management which requires valid and quality assessment processes.These authors note that assessment may be conducted: by the individual themselves (if they do not know how competent they are, how competent are they?) by the individual's peers ( this may include fellow learners, workmates, fellow practitioners) by 'experts' ( including teachers, lecturers, trainers, supervisors and a variety of other individuals).
If one is to ensure that students of PBL programmes graduate with the abilities necessary to work as members of teams and self evaluate performance (Engel, 1991), it is essential that assessment activity incorporate self and peer evaluation as well as 'expert' evaluation.This presents numerous challenges for staff who are reluctant to accept these as valid means of collecting data about performance.While staff are apparently suspicious and untrusting of self and peer assessment, learners are not likely to accept responsibility and accountability for their own performance and as such will continue to demand that staff are accountable for their learning.
According to Kennedy (1987), the technicalskills definition of expertise has been most criticised for its assumptions about how technical skills contribute to professional practice.What the technical skills orientation lacks is attention to the rest of professional practice: theory and principles, analytic capacity, and so forth.It overlooks the decisions that professionals make about whether and when to employ a particular skill.
During this study it became apparent that both staff and students struggle with the process of decision making in professional practice and how possible it is to explicate the tacit knowledge of the professional.Further complicating this is a need in higher education to discriminate not only between levels of performance for the purposes of grading students, but also between the level of performance on a year or semester basis.Despite claims that levels of competence and progression between levels required and assessment should be devised to infer an individual's current level of competence, to decide whether the level satisfies some predetermined standard, and to provide feedback to assist in the person's professional development (Masters and McCurry, 1990), there is evidence that the tensions between use of assessment as part of instruction and assessment as summative evaluation of performance continue.
Large student to staff ratios and assessment of students by individual members of staff may preclude individual learning and potentiate the use of normative assessment approaches under the guise of criterion referenced assessment.The Construction Management programme has addressed this through recognition that reducing assessment of professional competency to higher order skills checklists are of little value because they reinforce the notion that competence can be captured in a list of observable behaviours (Masters and McCurry, 1990).Hence, the Construction Management programme has sought to provide opportunity for integrated, global assessment that requires the person being assessed to justify his/her decision making in context.In the programme, this takes the form of panel assessment of a portfolio presentation that is providing meaningful indicators of students' professional performance.
The purposes and forms of assessment need to be clearly linked to the learning process and the outcomes of learning (Biggs, 1996).Assessment of complex multiple higher-order competencies presents difficulties in quantification of outcomes, because assessment of the resolution of complex issues is more than an assemblage of parts.Each part may be able to be assessed on an objective scale but the integration into a holistic product can be assessed only by qualitative judgement (Biggs, 1996).While "the characteristics of higher-order competencies described earlier make professional judgement a valid basis for assessment" (Chen et al., 1999), staff are attempting to design assessment items that are inclusive of both the product and process of learning, are conceptually and professionally relevant, and not prescriptive and constraining, yet clear enough to guide students' learning meaningfully.Assessment tools are the means to operationalise concepts in competence for both staff and students.

CONCLUSION
It has been noted that the educational system should prepare a person for a life of learning with emphasis on cognitive and affective processes, rather than on content and techniques.The industry-based environment is primarily concerned with technical and organisational competencies where technical competencies are still used as the measure for success.
Unless we are aware of essential tensions between the goals of educational and service sectors and acknowledge and address these in our curricula and assessment criteria, there will be potential for staff and students to dismiss professional competencies as vague, abstract and meaningless.Performance assessment may revert to a disintegrated and decontextualised task oriented approach to learning outcomes that is incongruent with the philosophy and principles of PBL.
The debate over competency-based approaches in higher education should not be isolated from the other major contemporaneous issues in higher education.Irrespective of the motives in setting the contemporary agenda for higher education, it should be recognised that some principles of the competency-based approach offer opportunities for improvements in higher education.These include the focus on the outcomes of the process of higher education.This is important for quality management, accountability to society for the public funding supporting higher education, clarification and communication of higher education objectives and functions to stakeholders including government, professions, industry, the community and students.
The competency-based approach can provide a framework for identifying, organising and implementing relevant curriculum, and clarify understanding of assessment objectives, procedures and strategies.It can synchronise the needs of industry, professions and the community with the preparatory role of higher education.This is particularly evident when this is embedded within a PBL approach to teaching and learning as PBL, both as a curriculum design and instructional method, provides the framework for "looking to identify key knowledge and skills in the context of realistic professional tasks" (Masters and McCurry, 1990).According to The University of Newcastle (1999), PBL, as delivered by the Building discipline at Newcastle, helps to develop graduates who integrate problem solving skills, encourage creative thinking and implement various examples of innovative practice into the construction industry.

Figure
Figure 1: PBL and competency oriented assessment