Senior Educational Consultant
August 5, 2013
Student Ratings of Instruction System
Adjusted Scores - makes classes more comparable by considering factors that influence student ratings, yet are beyond the instructor’s control. Statistical procedure changes raw scores by adjusting them either up or down based on student average responses to questions about their work habits, effort in in the course, desire to take the course, course difficulty, and instructor-provided enrollment.
Aggregated Data - summary statistics (e.g., means, frequencies) of student ratings stored at the class level. IDEA can create a spreadsheet of aggregated data across multiple courses that enable institutions to conduct their own statistical analyses for accreditation and research purposes.
Benchmarking - IDEA Benchmarking for Learning “allows campuses to compare their student ratings results to a group of 6 to 10 peers, selected by the institution, who also use the IDEA Student Ratings System. There are also Benchmarking Reports for various disciplines.
Comparative Information - information contained on pages 1 and 2 of the individual class report; three types of comparative information can be provided: for the overall IDEA database, the discipline, and the institution. This tells the instructor how she/he compares to those groups. Comparative information for the discipline and institution is provided only if there are at least 400 courses in the IDEA database.
Converted Scores - scores that statistically transform mean raw scores from the five-point scale to a standard scale with a mean of 50 and standard deviation of 10 (T-scores). Enables comparisons between instructors and across courses. Converted scores are provided for progress on relevant objectives, excellence of teacher, excellence of course, and summary evaluation.
Course Characteristics - on the Diagnostic Form, three questions pertaining to amount of reading, amount of work in other (non-reading) assignments, and course difficulty.
Diagnostic Form - one of two Student Rating Forms available. Students complete this form, which is a 47-item instrument that contains the 12 learning objectives, 20 teaching methods, three overall measures, student and course characteristics, and a few “experimental” items.
Discipline Codes - the four digit code the instructor uses to indicate the academic discipline. These codes are slightly modified from the list of the Classification of Instructional Programs (CIP) of the National Center for Education Statistics. See full list here. Note: Institutions must use codes from the IDEA list (not from the national CIP list)
Faculty Information Form (also called the FIF) - instructor must complete this form for each class taught, including for each section of a multiple section course or for team-taught classes where the instructors are rated separately. The instructor must select at least one objective as relevant for each class, otherwise all objectives are counted as “important.”
Formative Information - information provided on page 3 of the Diagnostic Report; pertains to feedback about how frequently the students perceived use of each teaching method relative to relevant objectives; also makes recommendations for suggested actions.
Group Summary Reports (also called GSRs) - the individual class report tells the instructor “How am I doing?” The GSR informs the unit (e.g., department, college, general education curriculum) “How are we doing?” The 10-page report provides summary statistics on aggregated ratings across courses in the unit.
IDEAblog - written by IDEA staff and invited guests, this is a weekly discussion/information site published on The IDEA Center Web site; pertains to topics related to teaching, learning, student ratings, evaluation, and so forth.
IDEA Papers - papers written by professionals in the field on various topics related to teaching, learning, and evaluation.
Individual Class Report - for the Diagnostic, this is the 4-page report the instructor receives for each class rated. The first page contains summative information about student progress on relevant objectives and overall ratings of the course and instructor. The second page provides more detailed information about student progress on each objective and student/course characteristics. Page 3 provides formative information based on student ratings of how frequently the instructor uses each of 20 teaching methods. Page 4 provides statistical detail about student ratings on each item.
For the Short Form, only the information on Pages 1, 2, and 4 of the Diagnostic Report are included.
Learning Objectives - 12 student outcomes the instructor rates for relevance to the course (M=of minor or no importance, I=important, E=essential); students rate how much progress they have made on each objective, using a five-point scale. These objectives are listed both the Faculty Information Form and Studetnt Rating Forms
Local Codes - optional codes instructors and units can use to designate various characteristics of a course (e.g., general education, evening classes, day classes, taught by adjunct or full-time instructors, etc.).
Norming Groups - in the IDEA system there are three norm groups: IDEA database, discipline, and institution. However, users can also request a Benchmarking Report that provides information about how the unit compares to relevant peers, Carnegie classification, and the overall IDEA database.
Norms - a statistical concept that refers to average scores within a representative group to which an individual is compared.
Progress on Relevant Learning Objectives (PRO) - a weighted average of student ratings of progress on relevant learning objectives; “essential” objectives are double weighted and “important” objectives are single weighted.
Raw Data - individual student ratings within classes. Data are stored at the student level. IDEA can create a spreadsheet of raw data across multiple courses to enable institutions to conduct their own statistical analyses for accreditation and research purposes. Student anonymity is preserved.
Raw Scores - mean class ratings on progress on progress on relevant objectives, excellence of teacher, excellence of course, and summary evaluation.
Relevant Learning Objectives - from the 12 Learning Objectives on the Faculty Information Form, these are the objectives the instructor rates as either “important” or “essential” for the course; IDEA recommends 3-5 per course.
Short Form - One of two Student Rating Forms available. Students complete this form, which is an alternative to the Diagnostic. It is an 18-item instrument that includes the 12 learning objectives, three overall measures, and three student characteristics. Unlike the Diagnostic, it provides no formative feedback about how to improve.
Standardized - meaning in a consistent manner, this pertains to the converted scores described previously; that is, the raw scores are converted or “standardized” to the same scale for consistency and comparability. Standardization also is involved in the fact that students answer the same questions, using the same scales, across courses. In addition, scores are interpreted consistently across courses. The IDEA Student Rating System uses standardized forms, meaning the items are set and cannot be modified or deleted (although additional questions can be added). This allows for comparisons to be made to other norming groups.
Student Characteristics - on both student ratings forms questions pertaining to desire to take course, effort in the course, typical work habits, and background preparation.
Summative Information - information provided on pages 1 and 2 of the individual class report; pertains to average raw, adjusted, and converted scores on overall summary measures.
Teaching Method - these are principles used for instruction. In the IDEA SRI there are 20 teaching methods students rate on a five-point scale to indicate how frequently the instructor practices each one.