swiseSWISE 050310 015SWISE 050310 008

Beitrag 125

Psychological Assessment Tools as Instruments for the Development of Aptitude

 

There is a big controversy around selection strategies for study programmes for the teaching profession. One issue is the modest prognostic validity of indicators of aptitude for the different criteria for teachers’ occupational performance. As a consequence, many universities use the available forms of aptitude assessment as a stimulus for the individual career decisions only. Funded by the federal programme «Qualitätsoffensive Lehrerbildung», we at the University of Erfurt are developing and applying assessment instruments to provide students with basic information about their own habits and personality.

At the beginning of the first term of the teacher study programmes, students are offered multi-facetted personality scales and tests regarding the status of their occupational choice. The feedback sessions use a counseling framework which helps the students to reflect on their desired career. During the bachelor study phase, the students’ school internships are planned in relation to the individual profile of the student’s personality. The interns are required to reflect their job experiences with respect to their own strengths and weaknesses. 

The research team is furthermore developing so-called Multiple Mini-Interviews (MMIs) that have been used in Canada during the application procedures for the study of medicine, and which are now used in various countries and for different study programmes. In six to ten situations with a time span of just a few minutes each, the candidates are requested to demonstrate, among others, social competencies, teamwork, evaluation skills, and responsible reflection on ethical issues (Eva et al., 2004). MMIs are widely used now (Knorr & Hissbach, 2014) since the acceptance by the applicants is rather high (Humphreys et al., 2008).

At the University of Erfurt 98 students of different teacher study programmes were observed in six different performance situations. Participants were required to explain their own occupational choice, to plan a school excursion to an animals’ garden, to argue against critical remarks aiming at the teaching profession, to describe a geometrical pattern (so others were able to redraw it), and to deal with colleagues in two situations with conflicts. The participants’ behavior was recorded and later evaluated with the help of rubrics. In addition, the staff who administered the MMIs rated the behavior of the participants globally, and participants rated their own competencies.

Calculations of the psychometric quality showed that the raters of the MMI contributed much to the variance of the performances. Reliabilities were thus low and could be improved only by the careful development of rubrics, the training of the raters, and a high number of raters. Regarding the construct validity, the indicators correlated more within the situations but less within the theoretical constructs. Substantial correlations (r ~ .50) were found between the detailed behavioral judgments and the global ratings, thus corroborating the criterion validity of the measures.

In sum we have to invest a lot of effort to develop psychometrically acceptable performance assessment tasks. However, this effort is indispensable if the goal is to avoid subjective bias in those judgments which are of central relevance to the practices of career counseling.

 

Literatur

Eva, K. W., Rosenfeld, J., Reiter, H. I. & Norman, G. R. (2004). An admissions OSCE: the multiple mini-interview. Medical Education, 38(3), 314-326.

Humphrey, S., Dowson, S., Wall, D., Diwakar, V. & Goodyear, H. M. (2008). Multiple mini-interviews: Opinions of candidates and interviewers. Medical Education, 42, 207-213.

Knorr, M. & Hissbach, J. (2014). Multiple mini-interviews: Same concept, different approaches. Medical Education, 48(12), 1157-1175.