Getting Personal


The Use of Computer Adaptive Testing in Assessment

Part 5 of a 5 part series exploring the benefits of technology-enhanced items in education and assessment. Brought to you by Learnosity and A Pass Educational Group.


Technology-enhanced items have brought about immense changes in education—improving the learning experiences of students and giving teachers valuable insight into their classroom.

This paper explores adaptive assessment, a computerized testing methodology that adjusts testing based on individual achievement levels. With computer adaptive testing, an item creator builds a large pool of questions with levels of difficulty assigned to each one. As students answer questions, future questions are selected using an algorithm designed to adjust the level of difficulty according to each student’s ability. At a basic level, answering questions correctly leads to more difficult questions, while answering questions incorrectly leads to less difficult questions.

Benefits of Adaptive Assessments

Adaptive assessments are designed to challenge students. High-achieving students who take a test designed around an average are not challenged by questions below their individual achievement abilities. Likewise, lower-achieving students are not served by questions that are so far above their current abilities that they’re left guessing the answers instead of applying what they know. Adaptive testing addresses these issues by adjusting to the individual proficiencies of students. High-achieving students can be challenged by more difficult questions, while students who are slightly below the average are not overwhelmed but rather encouraged to continue moving forward by answering questions at or slightly above their current achievement level. This motivates students to reach slightly beyond their comfort zone—no matter where that zone might be—and prevents students from getting discouraged.

On a more practical level, adaptive testing is also more efficient than non-adaptive. High-achieving students can quickly move through lower-difficulty levels and on to the questions that more accurately reflect their current achievement levels. Students who are struggling will not waste time on questions that are ultimately beyond their current reach. Instead, every student spends time answering questions centered around their current ability level, rather than questions that do not provide them with a realistic challenge or the instructor with a meaningful unit of measure.

Adaptive assessments are also beneficial to teachers. While a test designed around an average can tell an instructor what a high-achieving student does know, it fails to tell the instructor what the student does not know.

Therefore, instructors are left with little guidance on how to challenge such students and continue instruction in a progressive way. Similarly, with adaptive testing, a teacher can get a more accurate picture of where lower-achieving students need help. Some students with lower overall scores may, in fact, have mastered certain areas of knowledge. This information helps teachers target further instruction. Adaptive models also provide teachers with a specific point where students have reached the boundary of their current achievement level; this knowledge helps teachers and school officials understand what is left for students to accomplish and helps them set realistic and concrete goals for future instruction.

Adaptive Assessment in Practice

Learnosity offers two types of adaptive assessment. The first type, item adaptive, adjusts each future item based on an average estimated ability, called the ability estimate. This type of adaptive testing has many possible uses. For example, an ideal use for this technology is vocabulary testing, where students are presented with words and asked to define them (or they are presented definitions and asked for the correct term). When a student answers incorrectly, the student is presented with lower-level vocabulary terms. As students get questions correct, the terms and/or definitions can become increasingly difficult and complex.

Item Adaptive Testing

 

Another potential use is in mathematics. An instructor can select a specific topic or standard, such as solving a system of linear equations. Questions at a lower level of difficulty might consist of equations already set up to be solved using substitution and make use of nice, evenly divisible integers. Questions at a mid-range level difficulty might require students to manipulate the given equations before they can be solved using substitution or another method. These questions might also include fractions or decimals. Finally, questions at a high level of difficulty might model the one below:

3x-2y=-15

ax+4y=24

If the solution to the system of equations above is (-1,6), what is the value of a?

This type of adaptive model can be used with any question type, across any discipline. Literature and social science questions can require an increasing level of understanding, starting with recite-level questions and progressing to analyze-level questions. Science questions can begin at the recall level and advance to complex application questions. Instructors can also make the tests broad in nature, covering a wide range of topics and varying levels of difficulty, or test specific areas of knowledge, like the first examples given. In either case, the applications of the adaptive assessment are both wide and far-reaching.

The second type of adaptive assessment offered by Learnosity is called branching. This method of adaptive testing allows item creators to form mini-tests, called testlets, which consist of groups of questions or activities to create larger chunks of questions, which are subject to the adaptive algorithm. In this scenario, each testlet is chosen based on the student’s ability estimate up to that point. This approach allows teachers to progressively test topics rather than individual questions.

Branching Diagram

For example, a teacher might begin with a question set on adding and subtracting integers. This could progress to a group of questions that assess adding and subtracting decimals. If answered correctly, the next testlet might consist of multiplying and dividing integers. Finally, the highest-level testlet might be made of questions related to multiplying and dividing decimals. In this way, teachers are assessing slightly different skill sets in a related umbrella subject matter.

In the humanities, testlets might begin at the recall, identify, and define levels. A student might move on to a testlet consisting of classify, categorize, and summarize-level questions. Then, the next testlet might be made up of compare and contrast, assess, and hypothesize questions. Finally, at the highest level, testlets might be composed of critique, analyze, and apply items.

These testlets can each test a specific standard or topic separately, or each testlet can include a variety of topics, set to increasing levels of difficulty. This gives teachers the complete flexibility to focus either on depth or breadth of knowledge, or a combination of both. Overall, the customization and individualization of the adaptive assessment paradigm allows instructors and school officials to administer more accurate and effective assessments that are both useful to the instructor and relevant to the student.

Summary

Computer adaptive testing (CAT) is a computer-based test which adapts to the student’s ability level in real time. By creating designated item pools, the CAT system can select questions based on the student’s response to previous questions. It offers a more personalised testing approach and one of the main benefits offered is a more accurate estimate of the student’s ability level. From a student perspective, motivation is naturally achieved from this form of testing, as the test is based around their own ability. From a teacher perspective, less time is spent on testing and the risk of cheating is reduced. CAT delivery options include: item adaptive where the delivery of each item is based on the student response to the previous item and; branching, a delivery method where items are presented in groups called “testlets”. The student’s performance in the first testlet determines the next testlet to be presented.

See the Learnosity demos site for illustrative, interactive examples of both item adaptive and branching.

Download Button for Adaptive Testing in Technology Enhanced Items Assessments

This brings to a close our series of articles on technology-enhanced items. The final piece of this series will be our interactive webinar, hosted by Learnosity CEO Gavin Cooney and Lynsey Peterson, Director of STEM Development at A Pass Educational Group, on November 19 at 1 p.m. (EST).
This hour-long webinar aims to explore the overall benefits of using technology-enhanced items in assessment, and will give a hands-on demonstration of the authoring experience for some of the more advanced technology-enhanced solutions, such as spoken response, rule-based grading for math and chemistry, creating worked solutions and hints for students, and adaptive testing. Join us November 19 for: “Don’t Just Check the Box: Technology-Enhanced Items in Assessment.

 

Button to Register for Technology Enhanced Items Webinar


About Learnosity:

Learnosity Assessment SoftwarePartnering with companies across a range of sectors—including K-12, Higher Education, and Corporate Education—Learnosity provides the technology framework for authoring, assessment delivery, and reporting for many of the world’s best assessment solutions. Leveraging the Learnosity offerings enables clients to enhance any digital product, new or existing. It also reduces the need to reinvent the wheel, significantly increases speed to market, and decreases the overall cost of ownership.

Learnosity offers one of the widest ranges of technology-enhanced items on the market, with over 55 distinct question types currently available, ranging from the more basic fill-in-the-blank question types to spoken-response capture, Cartesian graphing, handwriting-recognition technology as well as advanced Math and Chemistry formulas. The authoring experience is designed to allow any user—professional content author or the more casual teacher author—to create advanced technology-enhanced items in minutes. It’s as simple as using any word processor. A wide range of implementation and storage solutions are also available. Clients can choose to use the Learnosity item bank, which uses a flexible, tag-based system for organizing Items as well as providing test-construction facilities. Alternatively, clients can choose to simply add the Learnosity TEI editor to their existing Content Management System.

The Learnosity reporting solutions allow clients to easily embed HTML reports, with useful insights from individual and group analytics, on any webpage. Clients also have on-demand access to the raw data, providing the flexibility to use as needed at any time.


About A Pass Educational Group:

A Pass EducationA Pass Educational Group, LLC is very proud of its reputation for delivering high-quality work on time. We are also proud of our reputation as being responsive, friendly, and easy to work with. Finally, we cherish the loyalty of our staff and contract associates. We offer the following services: Instructional Design. A Pass recognizes that high-quality learning requires exemplary instructional design. Students and teachers must fully understand expectations and easily engage in content for optimum learning. Read More Assessment Design. A Pass develops many types of assessment questions, including multiple choice, true/false, constructed response, short essay, technology-enhanced items (TEIs) and more. We design assessments to accompany textbooks and study aids as well as high-stakes assessments for state and national standardized exams. Read More Course Development. The A Pass team can develop courses and /or lesson plans in any subject area. If we do not have the appropriate resources in house, we have the ability to find them. Read More Editing. A Pass has deep experience editing client content and resources—this includes developmental editing, copy editing, fact checking, and more. Read More

 


This post was posted in , , , by on