Auto Scoring for Open-Ended Math Questions
Part 3 of a 5 part series exploring the benefits of technology-enhanced items in education & assessment. Brought to you by Learnosity and A Pass Education Group.
Technology-enhanced items are continuing to grow in popularity.
Modern-day assessments now have the appropriate tools and features to allow for much more challenging and innovative questions than ever before. Learner engagement levels are significantly increased, and assessors have the ability to test a variety of skill sets that were previously untestable. One of the most beneficial features offered to assessors is the impressive auto-scoring capabilities of technology-enhanced items.
This paper explores the capabilities of the Learnosity math & chemistry scoring engine used in technology-enhanced items, demonstrating the many advantages of this feature from both an educator and student perspective. From basic formulas to the more complex mathematical expressions, the powerful auto-scoring math engine enables advanced, rule-based auto-grading.
Math Scoring Basics
The smart math scoring feature of the Learnosity math engine permits question creators to ask open-ended math questions. The key element of this feature is that these answers are auto-scorable; the author of the question can define a set of parameters for the correct answer and, as long as the student’s response fits these parameters, the question will be scored as correct. This means that students can enter their answer in a wide variety of forms and syntaxes and will not be penalized for using different variable ordering, bracketing, or spaces between elements. There is both an onscreen keyboard, which allows the input of symbols not readily available on standard QWERTY keyboards, as well as a handwriting feature, which enables students to utilize a touchscreen interface. Calculators and measuring tools such as rulers and protractors are also available.
The scoring feature can read numbers, letters, symbols, expressions, and equations.
The author of the question can define how strictly students must adhere to a specific form. For example, the question can be set so that “” scores as correct, but an unsimplified form like will score as incorrect. Similarly, if a question states that an answer must be written in expanded standard form, an author can set to score as correct, but (the factored form) or (an equivalent expression not in standard form) to score as incorrect. Conversely, the author can also allow flexibility in answering a question, such as allowing either or to validate correctly. There are a variety of other specific validation features that will be discussed in-depth later in this brief.
Advantages of Smart Math Scoring
There are several important advantages of this functionality. Most significantly, it allows for the assessment of higher levels of mathematical skills. Whenever multiple-choice or another structured response format is involved, there is always the possibility that students can simply guess the correct answer. Furthermore, although authors seek to avoid this pitfall, sometimes answer choices can give students hints that guide them to the correct answer without having to demonstrate that the skill being tested has been mastered. For example, if a student is asked to factor a quadratic and all of the answer choices contain radical expressions, this gives students a clue that they will need to use the quadratic formula to correctly factor the equation. Additionally, when students perform computations and then do not find their answer among the choices given, they know they made an error and can reevaluate the steps they took to reach their answer. Use of open-ended math questions effectively addresses each of these concerns.
A second advantage is higher author efficiency. When crafting questions in a structured-response format, such as multiple choice or drag-and-drop, the question author must craft distractors that do not provide clues that allow the student to easily guess the correct answer. This often involves creating many distractors, such as numerous possible factors, both positive and negative, between and . This process can be time-consuming. Additionally, the different validation features, such as allowing equivalent forms of an expression to validate as correct, reduce authoring time by allowing the question creator to simply adjust a setting, rather than having to manually enter all of the possible equivalent forms.
These features not only save time, they also reduce the risk of human error on the part of the author.
Smart Math Scoring in Action
To start with a basic example, students can be asked to add or subtract integers.
Using the “EquivLiteral” setting, only the correct answer, entered exactly as it was by the author, will score correctly. If the question asks for the value of , only will validate.
Similarly, students may be asked to add two fractions, such as . In this case, if the author uses the “EquivValue” scoring method, students will be able to enter either , the unsimplified form, or , the simplified form or, indeed, , the decimal form. If the author only wants to accept the simplified fraction, he or she may use the “EquivLiteral” setting.
At a higher level, students may be asked to create equivalent equations, such as manipulating a quadratic equation from standard form to vertex form. In this case, the author would use “EquivLiteral” because vertex form requires the answer to be in a specific order. On the other hand, if students are asked for the product of and , the author can use the “EquivSymbolic” setting, which will accept all equivalent forms of the expression product of . In order to make sure students cannot enter [math](x-5)[/math] as an answer (which “EquivSymbolic” would validate as correct), the author could use an additional setting, “IsExpanded,” which will only accept answers that are in the expanded, rather than factored, form. Similarly, there is an “IsFactorized” setting, which operates in the opposite way, only accepting factored, rather than expanded, forms. There is a similar “IsSimplified” setting, which would accept , but not .
The scoring feature can handle more complex expressions and symbols, such as radical expressions and (pi). Students can enter answers such as or . Question creators can give students a complex equation, such as , Newton’s universal law of gravitation, and ask students to write the equation in terms of . Students can also enter logarithm answers, such as .
Question creators can provide students with a variety of keyboards to allow them to enter nearly any symbol or mathematical operation needed, including Greek letters, absolute values, trigonometric functions, integrals, and matrices. Authors may also choose to create custom keyboards.
Using the alternate response feature, authors can ask more open-ended questions. For example, if an equation has three factors, , , and , the author can ask a question such as “What is one factor of ?” and allow any of the above factors to validate as correct. Question creators also have the flexibility to ask for any equivalent expression, and allow any equivalent expression to validate except the original expression given in the question stem by using the “inverse result” setting. There are numerous other options, including allowing an answer such as to validate correctly with or without the comma as the thousands separator, or allowing (or disallowing) the use of a decimal or a fraction (such as or ) as a correct answer, or even setting the number of decimal places required for a correct answer.
Sometimes, question creators will need to use a combination of settings, such as the “EquivSymbolic” and “inverse result” to allow any equivalent expressions except for the one given, or “EquivLiteral” and “ignore order” to allow certain terms to be entered, but in any order (such as or ). Learnosity has the flexibility to accommodate these combinations as needed, allowing for nearly endless possibilities. This level of customization makes the math-scoring feature a powerful tool that can be used in a variety of situations to suit the needs of the assessor.
Smart math scoring can be used in many different question scenarios. Rule-based, math grading can be used in multiple different math question types. Depending on the grade level of the student and the pedagogical intention of the question it may be appropriate to format the question differently. For example the student could be asked to fill in the blanks or label an image rather than being asked a completely open-ended question
Handwriting Recognition Technology
All Math Formula questions can be enabled with handwriting recognition functionality.
This makes it significantly easier for students using touchscreen devices such as iPad and Android tablets to enter complex Math Formulas.
Students “write” the required symbols, and the handwriting recognition technology will translate them into machine-readable digital information. All student responses are then scored using the smart math scoring detailed above.
This provides students with a more natural and accessible math learning experience while also giving them all the cognitive benefits of handwriting.
The smart math-scoring feature offers several significant advantages to students and educators. One of the most powerful advantages of the feature is that it allows for the assessment of higher levels of mathematical skills through the use of open-ended questions for complex expressions and symbols that are frequently used in advanced math and chemistry. It also enables higher author efficiency through the use of various validation features that save time and simultaneously reduce the risk of human error on the part of the author.
SAVE THE DATE: Interactive webinar, November 19th 1pm (EST)
Don’t Just Check the Box – Technology Enhanced Items in Assessment
Explore the overall benefits of using technology-enhanced items in assessment, and get interactive demo of the authoring experience for some of the more advanced technology-enhanced solutions such as spoken response, rule-based grading for math and chemistry, creating worked solutions and hints for students and adaptive testing.
Partnering with companies across a range of sectors—including K-12, Higher Education, and Corporate Education—Learnosity provides the technology framework for authoring, assessment delivery, and reporting for many of the world’s best assessment solutions. Leveraging the Learnosity offerings enables clients to enhance any digital product, new or existing. It also reduces the need to reinvent the wheel, significantly increases speed to market, and decreases the overall cost of ownership.
Learnosity offers one of the widest ranges of technology-enhanced items on the market, with over 55 distinct question types currently available, ranging from the more basic fill-in-the-blank question types to spoken-response capture, Cartesian graphing, handwriting-recognition technology as well as advanced Math and Chemistry formulas. The authoring experience is designed to allow any user—professional content author or the more casual teacher author—to create advanced technology-enhanced items in minutes. It’s as simple as using any word processor. A wide range of implementation and storage solutions are also available. Clients can choose to use the Learnosity item bank, which uses a flexible, tag-based system for organizing Items as well as providing test-construction facilities. Alternatively, clients can choose to simply add the Learnosity TEI editor to their existing Content Management System.
The Learnosity reporting solutions allow clients to easily embed HTML reports, with useful insights from individual and group analytics, on any webpage. Clients also have on-demand access to the raw data, providing the flexibility to use as needed at any time.
- Product demos are available at: http://www.learnosity.com/tour/ and http://www.learnosity.com/authoring/
- For further information email: firstname.lastname@example.org
- Blog: www.learnosity.com/blog/
- Facebook: www.facebook.com/learnosity
- LinkedIn: https://www.linkedin.com/company/learnosity
- Twitter: www.twitter.com/learnosity
About A Pass Educational Group:
A Pass Educational Group, LLC is very proud of its reputation for delivering high-quality work on time. We are also proud of our reputation as being responsive, friendly, and easy to work with. Finally, we cherish the loyalty of our staff and contract associates. We offer the following services: Instructional Design. A Pass recognizes that high-quality learning requires exemplary instructional design. Students and teachers must fully understand expectations and easily engage in content for optimum learning. Read More Assessment Design. A Pass develops many types of assessment questions, including multiple choice, true/false, constructed response, short essay, technology-enhanced items (TEIs) and more. We design assessments to accompany textbooks and study aids as well as high-stakes assessments for state and national standardized exams. Read More Course Development. The A Pass team can develop courses and /or lesson plans in any subject area. If we do not have the appropriate resources in house, we have the ability to find them. Read More Editing. A Pass has deep experience editing client content and resources—this includes developmental editing, copy editing, fact checking, and more. Read More
- Product demos are available at: http://www.apasseducation.com/see-what-we-do
- Blog: http://www.apasseducation.com/blog
- Facebook: https://www.facebook.com/APassEducationalGroup
- Twitter: https://twitter.com/APassEducation
- More Content: http://www.apasseducation.com/more-a-pass-content-gateway
- Contact Us: http://www.apasseducation.com/connect-with-us or info@APASSeducation.com