Object-Match Question Type

Abstract

The Object-Match Question Type is a proposed online question type that tasks the learner to build an object by choosing from various properties, which may either be static (constant list) or dynamic (one property choice may affect other choices). When the learner submits their answer, their object is compared against the "correct answer" object for marking.

Introduction

There are already many online question types defined by standard or by common usage. One such family of types is the set of multiple choice question types.

Multiple choice question types have the advantage of being easy to mark in code, allowing instant feedback, and perhaps tailored responses to each correct and incorrect response. However, they also have weaknesses, such as being guessable (in a choice of four, the learner has a 25% chance of guessing correctly at random) and might involve recognition instead of recall.

The Object-Match Question Type (hereafter OMQT) is a proposed alternative to the multiple-choice single-response question type where:

  1. the answers come from a set of objects which belong to a class with a finite (small) number of finite (values come from a small, discrete list) properties;
  2. there is only one correct answer, and that answer has only one representation (distinct and unambiguous from other instances);
  3. optionally, there is value for the learner in understanding how the objects in their class are subdivided by their properties.

Example question sets

The example I will use throughout this discussion is a quiz about road signs, as you might find in a motor vehicle driving test.

A typical question might be:

What is the road sign for give way to traffic on major road?

Here is a very small subset of UK Department for Transport road traffic signs, with their signficant properties listed:

Small set of UK road signs giving orders
Sign Description Shape Background Border Symbol Text
Road sign: maximum speed 40. Maximum speed (40) circle white red none 40
Road sign: stop and give way. Stop and give way octagon red none none STOP
Road sign: no overtaking. No overtaking circle white red black car, red car none
Road sign: keep left. Keep left circle blue none down+left arrow none
Road sign: give way. Give way to traffic on major road inverted triangle white red none GIVE WAY

The artwork of traffic signs is covered by a waiver and may therefore be reproduced free of charge, without the need to obtain permission, subject to it being reproduced accurately and not used in a misleading context. The copyright source of the material must also be acknowledged.
© Crown copyright 2007

Each of the properties could be selected from a list, as follows (with some extra values from other road signs thrown in, to make a manageable number of four choices for each property):

  • shape:
    • circle
    • octagon
    • triangle
    • inverted triangle
  • background (colour):
    • white
    • blue
    • red
    • green
  • border (colour):
    • none
    • red
    • black
    • white
  • symbol:
    • none
    • black car+red car
    • down+left arrow
    • bicycle
  • text:
    • none
    • 40
    • GIVE WAY
    • STOP
Screenshot of road sign question.
The learner designs a roadsign to answer the question, submits the answer, which is marked and feedback supplied.

Notes on road sign examples

  1. Some simplifications have been used here. For example, a border may be thin or thick, not just coloured; and some signs may be qualified by plates beneath the main sign. However, these should not substantially affect the value of these examples.
  2. It would be possible to make the text property input a free-text entry rather than a choice.
  3. Since "signs with red circles are mostly prohibitive", this is a component of the sign that has a meaning that the learner should come to recognize.
  4. Some property value lists, such as colours, could be taken from a master list and reused again and again.
  5. Some property values are essentially optional (those which have "none" as a value). The treatment of optional values will need to be consistent.
  6. Apart from the text, the other property values are identifiers which will be used to look up the actual definitions used to render the road sign (shapes will have a geometry, colours a RGB value, symbols an image of some kind).

Design pattern

What follows is a discussion about a design pattern to implement the requirement for OMQT, covering the principles that might inform the approach, and building from a simple case to more advanced cases.

Principles

  1. An implementation of OMQT should work alongside existing online question types, in the same environments, interfacing with assessment frameworks.
  2. The technology should work in modern web browsers without needing any kind of server-side support or plug-ins. More specifically, the technologies would include HTML5, CSS, JavaScript/ECMAScript, XML, SVG and closely-related, standardized technologies.
  3. In regard to the above, it may be possible for the learner to access the underlying answer database through code, just as the browser does. Therefore the following design patterns are only suitable for formative rather than summative assessments, and revision.

Simplest case

Our simplest case goes something like this.

We have a single web page holding a single question, which is displayed to the learner. We will use the question:

What is the road sign for "give way to traffic on major road"?

The rest of the page is split into:

  • a set of input options which choose the various properties of the learner answer road sign;
  • a graphical display of the learner answer road sign as it takes place;
  • a button for the learner to submit the learner answer road sign for marking;
  • an output display, which will contain feedback, such as the results of marking, and possibly a rendering of the correct answer road sign.

Hidden from the learner but available to the web browser is the correct answer road sign. In this case, the object will have the following properties:

shape
inverted triangle
background
white
border
red
symbol
none
text
GIVE WAY

Interactions take place as follows:

  1. The learner selects (or inputs) a property value using the input options.
  2. Immediately, the graphical display updates to render the effect of the input.
  3. This repeats until the learner submits the answer for marking.
  4. The page then compares the learner answer road sign with the correct answer road sign, and provides appropriate feedback (marks the answer). In this simplest case, the answer will be correct if the two objects are identical, incorrect if there are any differences.

Try the "What is road sign?" question. The source code will be provided under an open licence once in a reasonably complete state.

Even in this simple case, there were some adaptive considerations:

  • when a light or dark background was chosen, any symbol or text applied should be displayed in contrasting colours; and if text/symbol was chosen first, its colour should not provide a clue to the background colour;
  • different shapes had different sizes and centres, so symbols/text should be resized and located to appear naturally inside any sign shape;

In other words, to minimize guessability, dependencies between different property choices should be removed as far as possible.

A more complex case

More complex cases could involve:

  • sets of questions, possibly displaying all answers on screen as the assessment progresses;
  • partial scoring (points for some of the properties being correct);
  • feedback on each choice;
  • dynamically changing the inputs depending on choices (cascading selection);
  • a different representation format than SVG, perhaps something like X3D using x3dom, such as this "which aircraft markings?" question example (browsers: Safari, Chrome, Firefox…).

Generating questions from a knowledge base

I have elsewhere covered the possibility of generating a set of questions based on a knowledge base (or ontology). In our example, we could have a knowledge base of road signs, and generate a random set of questions directly from it.

Technical considerations

Web technologies

My samples generally use XML for modelling objects and their rendering (like SVG).

Alternative technology would include JavaScript objects.

With HTML5 being increasingly supported by the latest web browsers, advantage can be taken of features such as inline SVG for object representation, although you could also use the canvas element or a variety of other approaches.

Styling can be ably handled by CSS, which can be used for both HTML and SVG, and set by a variety of means including script.

The intention was to produce working prototypes that need no browser plug-ins, possibly no JavaScript libraries, and work entirely in the client (browser) without needing server-side scripting. This makes it straightforward to incorporate them within learning objects and distribute them on any web server.

Whilst no particular effort was made to optimize file sizes in the sample code, it is expected that this method could be very efficient in terms of small, quick downloads.

Compatibility

Mobile device compatibility is highly desirable; however, not all mobile browsers have the same capability (for example, for XSLT transforms) as desktop browsers.

Internet Explorer 9 and 10 should be supported, but my primary web browser targets were the latest couple of desktop versions of Chrome, Safari, Firefox and Opera.

Accessibility

Accessible web design here needs to go beyond labelling form controls and providing text descriptions of the representation format. Probably a full WAI-ARIA treatment is needed.

Tools

The approach should be aimed at eventual incorporation within authoring tools. The XML schemas that can model the object classes could, in principle, be authored by non-technical staff in tools such as Microsoft InfoPath.

Ideally, this question type could be incorporated into a standard like QTI. At any rate, the scoring of the question should be passable into a host environment like a VLE.

Technical design principles

A major consideration was separation of concerns.

In the examples, the definition of the object (say, by XML Schema) is separate from its representation in the browser (say, by SVG graphic). This allows the rendering engine which creates the representation to be upgraded or replaced while not needing to change the answering system.

Styles could also easily be abstracted into external stylesheets and even switched between alternates by users (perhaps to get a high-contrast style, although that wouldn't be appropriate in all cases, like road signs). There's an example of this method in HeDGe, although at the time of writing it only works properly in Firefox: create your design then View > Page Style > choose between alternate styles.

Ideally, the whole user interface should be generated by declarative underlying data (an ontology, a question format standard, the representation rendering engine, device-responsive transformations and so on).

Summary

Key features of the OMQT are:

  1. A question which generally has one correct (canonical) answer.
  2. The answer is an object with multiple properties, which belongs to a recognizable class of objects.
  3. Each property can be treated as a more basic question type, typically multiple-choice single-response.
  4. The class of objects should have a representation (typically 2D graphical, but maybe 3D graphical, purely textual, or a mixture).
  5. Simple controls should allow the learner to build an answer object, with immediate updating of a displayed representation providing feedback. Typically, the learner should be able to make as many adjustments and changes as they like, before submitting their answer.
  6. On submission, the learner answer is compared with the "correct" answer, a scoring system applied, and feedback given.

It should be possible to encode OMQT design patterns so that they can be standardized and authored in specialist tools by non-developers.

Created: . Last updated: .

Creative Commons License
Object-Match Question Type by Tavis Reddick is licensed under a Creative Commons Attribution 3.0 Unported License.