Lessons Learned From the First Year of Computer-Based Testing

Lessons Learned From the First Year of Computer-Based Testing

BY Jerry Carter, Chief Executive Officer, NCEES Posted: 9 Mar 2015

This article is adapted here with permission from the February 2015 issue of NCEES Licensure Exchange.

The National Council of Examiners for Engineering and Surveying (NCEES) has now completed a full year of administering the Fundamentals of Engineering (FE) and Fundamentals of Surveying (FS) exams via computer-based testing (CBT). NCEES staff, along with various committees and task forces, spent several years in planning for the transition of the FE and FS exams to CBT.

The CBT system has worked extremely well from the start, and licensure candidates, member boards, and NCEES staff have all realized significant efficiencies. A majority of state licensure and registration boards opted to use the automatic option, which allows a candidate to apply directly to NCEES to take the FE or FS exam. The candidate’s results are then provided to that board once the candidate seeks recognition as an intern, or applies to take the Principles and Practice of Engineering (PE) exam, or the Principles and Practice of Surveying (PS) exam. Results are now posted between five and seven days of a candidate completing an exam, along with diagnostics for any failing candidates. Security issues related to the administration of the FE and FS exams have been greatly reduced, and the decision to employ the linear-on-the-fly, or LOFT, method of CBT delivery ensures that each candidate’s exam is unique to him or her. Also, the validity of the exams and overall pass rates have remained essentially unchanged from the pencil-and-paper exams.

Just as NCEES was warned by counterparts in other professions that have been employing CBT for some time, the candidate population did decline. This appears to be a common phenomenon experienced when transitioning from pencil-and-paper to CBT exams. Much of the decline has to do with candidates' ability to put off taking their exams rather than having to adhere to a rigid schedule, as when the FE and FS were only offered twice annually. A piece of advice from other organizations that administer exams via CBT was to provide some type of penalty for candidates who wished to reschedule their exam to mitigate the procrastination effect. A surprising impact of NCEES installing a fee to reschedule an exam is that NCEES unintentionally created a new revenue stream, which for the first 10 months of 2014 exceeded $180,000. NCEES is working to improve its understanding of the behavior of candidates and how to motivate them to schedule and take their exam. (IEEE-USA members planning to take the FE exam should take note that there is a $50.00 fee if you need to reschedule.)

During the 2014 testing year, 29,710 FE exams and 744 FS exams were administered in CBT format. NCEES staff members are now working with the exam development committees for the principles and practice exams (i.e., PE) to transition these exams to CBT over the next few years. NCEES will use the lessons learned from this past year to continue to improve the testing products that are offered, while still ensuring that the exams continue to have a high degree of reliability and serve the state licensure and registration boards in determining minimum competence.

Recommended Content

Comments Comment Policy

No comments yet. Please sign in to add comment.