The quickstart guide
If you are like most instructors, you continuously monitor the success of your courses. Consciously or unconsciously, you probably devote a great deal of energy to identifying course content and pedagogical techniques that do not achieve the intended results; you then make appropriate corrections.
A Course Improvement Plan (CIP) is a tool for formalizing and documenting this process.
There are additional benefits to such a tool, especially one that resides on the Internet:
A Course Improvement Plan is to be completed once a year for every unique course that you teach during the year; you do not need to complete a separate plan for unique sections of a course. The general process is as follows. (Keep in mind that the goal is to evaluate the course as a whole, not the progress of individual students.)
Before the semester:
Identify several learning outcomes (at least three) that you expect your students to achieve by the end of the course.
Construct a tool for assessing the extent to which each outcome is achieved.
Identify specific criteria that define successful achievement.
Throughout the semester:
In an informal manner, track your general sense of your students' progress toward achieving the outcome.
After the semester:
Collate the results of your assessment.
Interpret the results.
Based on your interpetation of the results, describe a plan for improving the course.
Each aspect of the process is described in greater depth below.
To be completed before the semester.
A learning outcome identifies something that a student will know (a cognitive outcome), think or feel (an attitudinal outcome), or be able to do (a behavioral outcome) after completing a course of study.
A learning outcome may derive from the mission of Snow College, the General Education Outcomes, outcomes defined in the course syllabus, or your professional understanding of what a student should learn.
The kind of learning outcome to be addressed in this document is one that can be assessed in an objective and consistent manner. This is important, because the result of your assessment will help you to plan improvements to the course.
A Course Improvement Plan should address at least three clearly defined learning outcomes. They do not have to be the same outcomes from semester to semester. A college course is a complex organism, and breadth can be a virtue. On the other hand, you can learn a lot about your course by tracking a single outcome for several consecutive semesters or years.
Examples:
Each student will be able to write a short essay that explains three characteristics of a constitutional monarchy.
Each student will be able to install and finish wallboard in a manner consistent with the Uniform Building Code and generally accepted professional standards.
Each student will feel confident that he or she can be successful in college.
Note that a grade is not a learning outcome. It is one possible result of assessing an outcome.
To be completed before the semester.
To assess a learning outcome is to determine the extent to which students in your course are achieving the outcome. Some students will achieve it completely, others will fail, and most will achieve it partially. The assessment process will help you understand the proportion of students who belong to each category. In turn, this information will help you identify areas of your course that would benefit from improvement.
As noted elsewhere, an ideal means of assessment is as objective and consistent as possible. Idiosyncratic tastes or biases should be set aside whenever possible. Different instructors who assess the same essay or performance should be able to assign similar grades and make comments that attend to similar issues.
A rigorous evaluation of a learning outcome will use 2-3 distinct means of assessing it.
The two main kinds of assessment are qualitative and quantitative.
Qualitative assessment tends to be holistic in nature, as when an instructor assesses the overall quality of a portfolio or performance. The fine arts and performing arts make extensive use of qualitative assessment, as do many of the humanities areas. The biggest danger to qualitative assessment is subjectivity. Not everyone, for example, will agree that a dance is aesthetically pleasing. One way of avoiding this danger is to have more than one instructor assess the project. You may be aware of other techniques specific to your discipline.
Quantitative assessment tends to be detail-oriented, examining a student's mastery of individual facts or skills. The social and physical sciences make extensive use of quantitative assessment, as do foreign language programs and, to a lesser extent, writing programs. The advantage to quantitative assessment is that it tends to generate data that are consistent and less susceptible to taste or bias. Such data are also easy to compare from one semester to the next. The biggest danger to quantitative assessment is that it can tempt instructors to emphasize small details at the expense of the bigger picture. For example, a student may grasp the overall significance of the United States Constitution while forgetting the name of the principal author. An ideal means of assessment would attend to both kinds of knowledge and assign them different values.
Examples:
I will document the way 2-3 art instructors respond to a painting project completed by each student in the class. [A qualitative assessment of a behavioral outcome.]
I will document the number of students who have learned to enjoy an unfamiliar genre of music. [A quantitative assessment of an attitudinal outcome.]
I will document the number of students who can write clearly focused paragraphs. [A quantitative assessment of a behavioral outcome.]
I will document the number of students who can correctly identify three central features of behaviorism. [A quantitative assessment of cognitive outcome.]
Note that in most circumstances grades are the result of an assessment, not a means of assessment.
To be completed before the semester.
Assessment only makes sense if student performance is measured against some sort of criteria. This will be most easily understood by turning directly to examples. The following examples build on those presented in the previous section.
Examples:
1. If the means of assessment is this:
document the way 2-3 art instructors respond to a painting project completed by each student in an art class
then a criterion for assessing the effectiveness of the course might be something like this:
50% of the class will produce a painting judged very good or excellent by the panel.
2. If the means of assessment is this:
document the number of students who have learned to enjoy an unfamiliar genre of music
then a criterion for assessment might look like this:
80% of the class will indicate that they have learned to enjoy an unfamiliar genre of music.
3. If the means of assessment is this:
document the number of students who can write clearly focused paragraphs
then a criterion for assessment might look like this:
80% of the class will write an essay in which every paragraph is mostly or entirely focused.
4. If the means of assessment is this:
document the number of students who can identify the central features of respondent conditioning
then a criterion for assessment might look like this:
75% of the class will write an examination response in which they correctly identify the central features of respondent conditioning.
Clearly the criteria for assessment are closely tied to the means of assessment. What the criteria add to the process are the answers to questions like "How many?" and "How much?"
At first, the numbers you assign to these benchmarks may seem arbitrary. Given time and experience, you should expect to arrive at a set of criteria that represent a standard of excellence appropriate to your discipline and reflect the abilities of the students who normally take your course.
In an advanced course, for example, it may be perfectly reasonable to expect near-perfect performance from 80% of the students. In an introductory class that satisfies a GE requirement, it might be reasonable to expect that only 30% of the students will generate near-perfect work.
Note that grades are generally a normalized representation of student performance. To simply write 80% of the students will score a B or higher on the widget examination will not illuminate very much. The results will be much more instructive if the criteria look more like this: 80% of the students will be able to operate a widget for 30 minutes with 80% efficiency or better.
To be completed during the semester.
Assessment is an ongoing process that instructors use to sustain and improve the quality of their courses. The Midterm Assessment exists to encourage instructors to assess the course at least once, but ideally several times, during the semester, not merely at the end.
Things change. The unexpected occurs. Learning outcomes prove more or less achievable than they were originally imagined. Pedagogical strategies prove more or less effective. Means of assessment prove more or less practical.
Instructors usually discover such discrepancies as they arise. Often, they make adjustments on the spot.
The Midterm Assessment is a space in which instructors can document such discoveries and adjustments.
Example:
As a group, my students are coming nowhere close to achieving this outcome. I will take two class periods from the next unit and use them to re-teach the central concepts of this unit. Small group discussion might help.
That was just an example. No particular format is suggested. Informal comments or narratives are completely acceptable; a table or list would also be acceptable.
An instructor who thinks it might be helpful to date individual comments should feel free to do so.
To be completed after the semester.
Asessment of a learning outcome yields results. As with any scientific experiment, results are neither favorable nor unfavorable. They simply indicate whether the outcome was achieved. There are many possible reasons why an outcome may or may not have been achieved, and several of them suggest nothing about the quality of instruction. The interpretation section of this document discusses these reasons in greater depth.
Results are intimately connected to the means of assessment and the criteria for assessment. As a consequence, a presentation of results will naturally draw its language from these two sections of the Course Improvement Plan. It is important to remember that results are measurements; as such, a large proportion of this section should be described numerically.
Examples:
40% of the class produced a painting judged very good or excellent by the panel. The result is 10% lower than anticipated. The outcome was not achieved.
85% of the class indicated that they have learned to enjoy an unfamiliar genre of music. The result is 5% higher than anticipated. The outcome was achieved.
40% of the class consistently wrote an essay in which every paragraph was mostly or entirely focused. The result is 40% lower than anticipated. The outcome was not achieved by an extraordinary margin.
50% of the class wrote an examination response in which they correctly identified the central features of respondent conditioning. The result is 25% lower than anticipated. The outcome was not achieved.
For the sake of clarity, commentary should be kept to a minimum in this section. The interpretation section is the appropriate place for a discussion of the results.
To be completed after the semester.
Results are one thing. An appropriate interpretation of those results is another, and instructors are cautioned not to jump to hasty conclusions. The results of learning-outcome assessment can lead to many kinds of interpretation, of which the following is an incomplete list:
The course content was overwhelming, insufficient, just plain wrong, or just about right.
The textbook, supplementary materials, and class presentations matched or did not match the quizzes and exams. (Whether the reports are true or not, students regularly complain that they have been tested on content that was not made available during the course.)
Individual pedagogical techniques were effective, ineffective, or somewhere in between.
The laboratory apparatus is appropriate and effective, hopelessly damaged, or no longer matches the course content.
The learning outcome itself is appropriate or needs to be restructured.
The means of assessment was effective, ineffective, or somewhere in between.
The criteria for assessment matched or did not match the student population and/or the standards of this discipline.
In short, a careful interpretation of the assessment results is prerequisite to making appropriate improvements to the course. The assessment technique itself is as likely to be flawed as any pedagogical strategy.
The following examples are more specific:
Examples:
Paragraph focus is explicitly discussed near the beginning of the semester. The concept is reinforced throughout the semester, but this reinforcement mostly occurs during small-group discussions of student essays. Small groups are notorious for allowing some students to opt out of discussion, while only a few students genuinely participate. My application of this pedagogical strategy may be flawed.
30% of the class confused the features of operant conditioning with those of respondent conditioning. Examination of the textbook reveals that these concepts are presented on sequential pages, and that the sections devoted to each are quite short. I therefore suspect that the textbook is part of the problem.
To be completed after the semester.
The planning section of a Course Improvement Plan can be long or short, depending on the results of the assessment and the way the results have been interpeted.
If an outcome has been achieved by a large majority of the student population, for example, it may be sufficient to write the following:
There is no need to change the course with respect to this outcome. My next CIP will not address this outcome.
Because instructors are more likely to assess troublesome outcomes, it is more likely that the data will suggest a course of action, as in the following:
I will spend an additional class period discussing this material.
I will introduce a hands-on activity to reinforce this material.
Occasionally, the learning outcome itself may come into question:
As important as it is for students to write an argumentative essay on a controversial topic, the student population does not seem mature enough to do so in an objective fashion. I will adjust my outcome such that students will write essays on less polarizing topics.
It is equally likely that the means of assessment, or the criteria for assessment, come into question:
After four semesters pursuing this outcome, using a variety of pedagogical strategies, it is clear that the number of students who succeed will never reach 80%. The average during this time has been between 35% and 50%. I will adjust the learning outcome to a 50% rate of success.
For as long as I can remember, my own qualitative evaluation of the students' capstone performance has been consistent with the panel's. I will not convene a panel next semester, and will trust my own evaluations. I will, however, survey my students to learn if they believe that my personal evaluation is fair.
It really is your document.
No one wants to be burdened with needless paperwork. Few things discourage instructors more than the sense that they are filling out forms that never again see the light of day.
The Course Improvement Plan is designed foremost to help you become a better instructor. Use it to study the effectiveness of your course content and pedagogical strategies. Use it to reaffirm the strengths of your courses, and also to make necessary corrections.
Collectively, CIPs are also used by deans and other administrators to maintain and improve the quality of our academic programs as a whole. They are a repository of experience that all instructors can benefit from. And they are one more piece of evidence that demonstrates to accreditors that Snow College reviews itself on a continuous basis.
If you have any suggestions for improving the structure of the document, please contact your dean.
Much of this document has been derived from Nichols and Nichols (2000), The Departmental Guide and Record Book for Student Outcomes Assessment and Institutional Effectiveness.
