About LORI
The Learning Object Review Instrument (LORI) is used to evaluate the quality of e-learning resources. LORI is an online form consisting of rubrics, rating scales and comment fields. The current version of LORI available from eLera is version 1.5.
How can I obtain LORI?
After joining eLera you can use LORI to evaluate learning objects on this website. Or, to obtain a printable version, download the LORI Manual.
Why do we need a review instrument like LORI?
Reviews help users to select learning resources for quality and fit. Review instruments like LORI make it easier to compare resources by providing a structured evaluation format.
What does LORI cover?
Using LORI, reviewers can evaluate nine dimensions of quality.
Content Quality
Veracity, accuracy, balanced presentation of ideas, and appropriate level of detail
Learning Goal Alignment
Alignment among learning goals, activities, assessments, and learner characteristics
Feedback and Adaptation
Adaptive content or feedback driven by differential learner input or learner modeling
Motivation
Ability to motivate, and stimulate the interest or curiosity of, an identified population of learners
Presentation Design
Design of visual and auditory information for enhanced learning and efficient mental processing
Interaction Usability
Ease of navigation, predictability of the user interface, and the quality of UI help features
Accessibility
Support for learners with disabilities
Reusability
Ability to port between different courses or learning contexts without modification
Standards Compliance
Adherence to international standards and specifications
What type of rating scale does LORI use?
LORI reviewers rate quality on each item using a five-point scale. Rubrics are provided that describe objects scoring one, three, or five points. Reviewers can opt out of an item if they are unable to evaluate the object on that item.
How should LORI be used?
LORI may be used for individual or panel reviews. When a panel is available to evaluate a set of learning objects, we advocate the use of the convergent participation model described by Nesbit and Belfer (in press).
How reliable is LORI?
In educational settings, evaluation reliability is never solely dependent on the instrument used. Rather, reliability may vary widely as a function of the context, the subjects being evaluated, who is doing the evaluation, and their level of training. A formative reliability analysis was conducted with the convergent participation model and an early version of LORI that consisted of ten items (Vargo, Nesbit, Belfer & Archambault, 2003). When the ratings of 10 evaluators were averaged, three items showed reliability above .90, three items showed reliability between .80 and .90, two items showed reliability between .70 and .80, and the reliability of two items could not be measured. LORI 1.5 is the result of a revision process that used findings from the reliability study to simply rubrics and improve reliability of all items. The study also indicated that reviewers' ratings converge after discussing individual evalutions, and that collaborative evaluation is an effective way to learn about design principles for learning resources.