Reliability of Mechanical Maintenance Performance Measures

Abstract : A fundamental requirement in the development and administration of performance measuresis that such assessments should result in reliable scores that accurately indicate a person's level of proficiency. This research memorandumexamines the reliability of two performance measuresof mechanical maintenance developed for the Marine Corps Job PerformanceMeasurement Project: hands-on performancetests and job knowledge tests. Multiple estimates of reliability were computed, and the consistency of test administrators in scoring hands-on performancewas specifically examined. The hands-on performancetests and the job knowledge tests were found to result in very reliable measurements. Properly trained and monitored test administrators were able to score hands-on performanceconsistently across examinees, over time, and for different test content. Implications for subsequent performance measurementare presented, and possible training implications based on mechanics who were retested are noted.