Testing the Test: A Study of the Reliability and Validity of the Northern Ireland Transfer Procedure Test in Enabling the Selection of Pupils for Grammar School Places

Abstract

The Transfer Procedure Test is taken by children around 11years of age who wish to attend grammar schools in Northern Ireland. It is ahigh stakes test in that children are only allowed one attempt and theirperformance determines their future schooling in a manner that is not of theirchoice or of their parents. Candidates usually take two test forms, with asupplementary test available for those, who for one reason or another, do nothave both test scores. This report describes the largest independent study ofthe Transfer Procedure Test ever conducted. Samples of test scripts used aspractice tests in 52 primary schools were analyzed to see if the testfunctions effectively in enabling the selection of students for grammar schoolplaces. The samples used for the analyses consisted of 1,288 students (test1), 1,270 students (.test 2), and 623 students (supplementary test). The testis designed to measure mathematics, English, and science achievement, and theaddition of these scores in the Transfer Procedure Test is questionable.Because the test does not measure any single attribute of candidates, itcannot be used as a proxy for any particular attribute. The test would beperceived as "easy" by many students, since more than 65% answered more than70% of the questions correctly. The "easiness" is a serious design flaw, aschildren would have been awarded a "D" with a 70% score, and it is difficultto justify failing a student who has answered so many items correctly. Thetests were found to be highly reliable, but the published information on thetest does not meet the requirements of international standards on educationtesting, nor does it supply reliability and validity information. Informationis provided about the predictive ability of the test for later studentachievement. (Contains 22 tables,3 figures, and 26 references.)