Development and Validation of Diagnostic Instruments for the Measurement of Teacher Competencies (Project 1)

Project 1 of the research program 1 "Professional Minds" deals with the development of competency profiles (link) for vocational school teachers (VET teachers) and of instruments for their measurement. For the competency measurement the Advocatory Approach (link) is applied. An expert profile forms the quality norm.

Project 1 of the research program 1 Professional Minds investigates the research question which competencies VET teachers need for a successful teaching. The aim is to develop a set of competency profiles and to measure their level of quality. A competency profile is "an impact teaching action in complex situations. This action must be definable, adequate, efficient and ethically justifiable and stimulates trainees in a differential manner" (Oser, 2007, S. 105). An example is: "The teacher is able to ask questions in the lessons and take up the answers of the learners. The teacher is able to estimate and prevent possible results of the Question-Answer-Teaching (link)." During the first phase of the project, in collaboration with experienced VET teachers and vocational specialists, a large number of teacher acting situations were collected. On that basis, in a quasi-Delphi-study, 45 competency profiles were developed and validated by a sample of 789 VET teachers. The criteria encompassed importance, frequency of use, difficulty and significance for training. During a second phase of the project, diagnostic instruments for the different competency profiles are being developed. Their core element are short film clips (film vignettes), showing a teacher in a real classroom from several perspectives. Teacher competence is measured by means of the so called Advocatory Approach (link). Teachers who wish to determine their level of teaching competence observe the film vignette and judge the teaching process of the shown teacher by means of a questionnaire on different quality dimensions. In order to determine the level of quality, the teacher judgment is related to an expert profile. The expert profile is set through an expert rating. Using a discursive procedure, a group of experts assess the shown teaching behavior on the different quality dimensions and give reasons for their assessment. So far, for several competency profiles diagnostic instruments have been developed and validated by a sample of over hundred persons. The results show that the theoretically developed quality dimensions of the developed competency profiles can be verified empirically. Furthermore, differences between evaluator groups could be found. On most quality dimensions, teachers approach the expert profile significantly better than non-teachers. This is an important indication of the professionalism of VET teachers. In the continuation of the research project, diagnostic instruments for additional competency profiles have to be developed. Further efforts must be undertaken to prove how they can optimally be implemented in the training and further training of VET teachers (link).

Oser. F. (2007). Willkür als Feind der Spontaneität. Aspekte der Standardisierung des Lehrerhandelns. In Benner, D. (Hrsg.), Bildungsstandards. Instrumente zur Qualitätssicherung im Bildungswesen. Chancen und Grenzen – Beispiele und Perspektiven (S.103-122). Paderborn; München; Wien; Zürich: Ferdinand Schöningh.