Assessment Corner - October 2010

Assessment plans are like property taxes: they are due twice a year and no one really looks forward to them.” I heard this from an unnamed faculty member the other week. Now, it was said in jest, but there often is an uncomfortable truth behind humor. This was one of those things that made me go hmmmm. Why is it the case that assessment is viewed as something negative or “something I have to do” when the potential for course and program improvement is so obvious? Let’s speculate, shall we…

After our last student learning committee meeting, a point or a potential cause became evident to me- the issue could simply be either misunderstanding or not understanding the last two columns of our six-column form, which call for an analysis of the results (column five) and recommendations (column six). Thus, I will try to explain these last two columns in a cogent and lucid manner.

Column Five:

This column calls for you to analyze your results. For example, let’s look at an Assessment plan from our Aviation Technician program (housed in Building F on the northwest end of the campus)…

Here is a recap of columns one through four for your viewing pleasure.

• Column 1: PCC- Diagnose and repair airplane engines.
• Column 2: Expectation- National Aviation Technician Certification Computerized Exam
• Column 3: Implementation- Capstone course in May 2010
• Column 4: Results- 57% of SCC Students passed (57 out of 100).

And now, the good stuff!

Column 5: Analysis- Why did 43% (43 out of 100) of our Aviation students fail this standardized test? Analysis shows that 38 of these students had minor to significant attendance issues. Analysis also showed a lower than average program test score over the last series of classes for this group of students. Analysis also revealed that 31 of the 43 students who did not pass experienced significant difficulties in hands-on in-class group assignments.

Further analysis: 73% of SCC Aviation Technology students scored above the national average for their normed group on the NOCTI exam. This is good, but nonetheless 43% of our students did not pass this examination.

And now… column 6!

Column 6:

Column six calls for you to make recommendations as to how you will change your program content, instruction style, et cetera to address the negative or positive outcomes analyzed in column five. These recommended changes should then be incorporated into your next year’s assessment plan and/or programmatic content to help address the deficits or to accentuate the positives.

Column 6: Recommendations- first of all, mandatory attendance guidelines will be written into all syllabi, clearly documenting the consequences of excessive absences. We feel this will help improve pass rates on this certification examination.

I will also look at my curriculum to ensure it is current and up-to-date. I’ll also look at practice questions to see if what I am teaching matches the focus of the certification examination.

Additionally, formative assessments will be better utilized and tracked to work on improving in-class test scores to see if this proves to be a predictor of pass/fail rates on this certification exam.

Next, instruction styles and techniques will be driven by our formative assessments. If students do not seem to be connecting the dots during in-class hands-on assignments, instruction will be modified and re-delivered to the students to ensure student learning is occurring.

Finally, while 73% of our students did indeed score above the national average, 43% of the total test-takers did not pass and will have to re-test to get certification to work in the industry. We will be setting a goal of having 95% of our students score above the national average and pass the certification exam.

So, what IS the deal with columns five and six?

Although this is a rather simplistic example, it does properly demonstrate the functions of columns five and six. Column five simply wants you to apply logic to your results to determine why something happened or did not happen.

• Was it a matter of attendance?
• Is your curriculum up-to-date, or have things recently changed that is not represented in your instruction materials?
• Are your students getting what they need from you, or do you need to modify your presentation of the materials?

For example, if your car will not start, you may apply logic to try and determine why the car will not start, eliminating simple things such as gas level and battery strength before working toward more complex causes. Sometimes, the simplest solution is best, such as looking at your gas gauge and thinking, “Gee, I’m outta gas!”

Column six, on the other hand, examines what you are going to do to try and improve your student’s learning.

• What are you going to do differently next year to assure that your student are getting what they should be getting.

You can see in the example above that I laid out clear and explicit responses to potential deficits addressed in column five. We will then know next year whether or not these changes were the right fix for the program’s relatively poor scores on the certification exam. Increasing attendance should help improve pass scores, as students are actually in the seats for instruction. Utilizing formative assessments will help me understand whether my students are “getting it” during class, and will also alert me to a potential need of changing my teaching style somewhat. These should all help me in working toward my targeted pass rate of 95%. Finally, is my curriculum good, or has a revolutionary new edition of my textbook come out of which I was not aware?

Can you see how column five naturally led to column six?

And finally… keep this in mind: Column five calls for an analysis of your results, and column six wants to know what you are going to do to try and correct the issues you are having.

Have a great month!