Testing Instrument Development

In general terms, a blueprint document shapes the chosen testing instrument.

The content of the testing instrument must reflect the identification and weightings of the competencies found in the blueprint. Test questions (referred to as "items") are developed using content experts who are trained through item writing workshops and training sessions.

Item quality is a key determinant of the validity and reliability of the test and also influences candidate perceptions of the test. The items continue to be developed and banked until there are sufficient items to prepare a test form, as well as additional items for experimental questions, anchor questions, instruction guides, preparation guides, replacement questions and future tests. Item development is followed by an editorial review to ensure clarity. A cultural review can also be undertaken to ensure acceptability. Items, particularly for licensure tests, are then pilot tested, often as experimental items and statistically analyzed to determine suitability. Items with poor statistics are revised or deleted.

Once all items have passed the review and revision process and a preliminary form of the test has been produced, a passmark is established. This standard is set in reference to the content and level of difficulty of the test questions. The most common technique for establishing a passmark is known as the Angoff method. This process is important for proper standard setting and for equating test forms as the items change.