|
Some employers and institutions are
considering in-home (or internet) based computer administered tests as a
means to increase test-takers' comfort and convenience, and to increase
their participation rates in required screening and evaluation programs. One
impediment to in-home high-stakes testing involves the integrity of the
reported test performance. In the absence of an impartial proctor, how can
the institution verify that the answers attributed to one individual were
not in fact provided by another (presumably more able) test-taker?
Similarly, how can the institution verify that inappropriate resources
(e.g., dictionaries and encyclopedias) were not used as aids by the
test-taker in the unmonitored privacy of their home? In a typical
high-stakes setting, these assurances are provided by proctors who verify
the examinee's identity and adherence to test-security protocol. With
unproctored in-home exams, some test-takers may be tempted to enlist the aid
of others to achieve high scores and all the benefits derived from these
improved scores.
One solution to the verification dilemma requires the administration of a
short second exam given under secure proctored conditions. If performance
levels on the initial (unproctored/in-home) exam are consistent with the
short proctored verification exam, then the in-home scores become the
applicant's scores of record, otherwise the initial test-scores are
invalidated and the applicant is required to retake an alternate form of the
full-length exam under proctored conditions.
This paper presents a new statistical method for assessing consistency of
test performance across two occasions, where on one occasion the level of
performance may be enhanced or misrepresented, and on the second occasion it
is not. The new procedure is based on the application of Bayesian model
assessment techniques to item response theory. The performance of the
proposed method is evaluated through the use of simulated data based on a
high-stakes multiple aptitude test-battery.. |
