Student questionnaires have been an inescapable part of academic life for quite some time, probably longer in the USA than here. My first (temporary) teaching position was at the University of Michigan in 1973, and this is where I met questionnaires for the first time.
It was a learning experience for me. I read the comments on the first questionnaire and was delighted; this student praised me highly, said I was the best teacher he had ever had. So I turned to the second, and was brought down to earth by the mirror image; I was the worst teacher he had ever had, and he explained why in some detail. I was lucky; this experience inoculated me against the disease of taking questionnaire comments too seriously!
My present employer has just changed the questionnaire system. They have done an unbelievably bad job; everyone I tell the story to gets angry about a different part of it.
The background is that the department ran the questionnaires for many years. The questions had evolved over time to give a reasonably good assessment of what the students thought about all aspects of the course. A new question was added after some debate to test how new technology (especially course web pages) was being used.
Then, halfway through the academic year, the College decided that new questionnaires would be introduced, which would be identical in all subjects. Departments were given almost no say in the matter. If we wished to continue our existing questionnaires, they should be in parallel with the College ones. Correctly, we decided not to overburden the students with bureaucratic nonsense, and withdrew from the field.
The questionnaire consists of seven statements; students are asked to rate each on a five-point scale from “Definitely agree” to “Definitely disagree”. On the back, there is space for comments, segregated into several fields. So here are some of the things wrong with the system. I have put first the one that troubles me most; but you may have a different view.
1. One of the statements is
Through studying this module I understand what I need to do to be successful in the assessment
None of the statements says anything about understanding the material or learning anything. So I infer that the College’s official view of my job as a teacher is to ensure that the students to do well in their exams. If that is the case, this is not an institution I want to be part of! In any case, rather than soliciting opinions on this, they could wait a couple of months and look at exam results to obtain more objective data.
2. In the old system, questionnaires were done in week 6 of the semester, and the results processed quickly. If there were serious problems with a module, the management would become aware of this in time to do something about it. In the new system, the questionnaires are filled in in weeks 9-11, and because they are read by computer the processing takes much longer. It was only by taking a stand that we were able to get the first few results returned by the end of week 11. This is far too late for the feedback to have any effect.
3. The questionnaire design is bad. In both our old questionnaires and the National Student Survey, “good” answers are on the right; on the new ones, they are on the left. I know for a fact that some students simply got it wrong.
4. This is compounded by the way the data is processed. In the old system, after the answers had been tallied, the questionnaires went back to the lecturer for comment. There is some chance of catching problems like the above. If the numerical answers are all extremely bad, but the comments on the back are very positive, you might guess that this had happened. But now the comments are detached from the tickboxes and the different fields separated out by the computer that reads the forms, so this kind of deduction is no longer possible.
5. The most absurd thing about the data processing is this. With the old form, the summary statistic was the median answer to each question; the only reasonable thing to do with ordinal data. In the new system, the software that reads the boxes calculates the mean and standard deviation. Then it plots the seven means on a diagram, and runs a curve through the points(!). When I asked about this, I was told, “Not everybody understands the median.” Oh dear.
I believe this happened because they bought in the entire system, fancy paper for the questionnaires, software to process them, etc. The cost of all this would probably have paid for another teacher, which might have had more effect on the student experience.
Peter Cameron's Blog 
Peter I think you are missing the worst thing: some people believe the results actually reflect the quality of teaching. I am quite sure that this will eventualy be used as a way to measure the quality of teaching (or maybe this is already the case). So instead of questionnaries being used by the lecturer to improve their teaching it will become a tool to decide about promotion and hiring.
This is already happening. Ask some of my younger colleagues.
Of course, we have had questionnaires for a long time, so this is not so surprising. The new thing is that, as they are processed centrally, the data will be used centrally, and the department head’s ability to use it carefully will be eroded.
This is a real shame because what should have been a tool to improve teaching becomes a process that eventually will water down the level of material we teach. We will all be scared to challenge our students because when you challenge people some of them fail. It is particularly bad in the UK where tenure no longer exists and academics are very weak compared to administrators.
The only way to fight such things is if the academic community (e.g. the Royal Society, LMS, or just the whole department), will act together. Unfortunately, we academics do not like to be part of a crowd and we usually have very weak leadership. Prominent academics are not willing to get into dirty political fights. We expect to be appreciated because we are smart, working hard, and doing important stuff. But most of our contributions are long term and subtle (especially in math) so we are not noticed, especially by politicians who think short term.
I completely agree. We can’t have an effect unless we unite, and we can’t unite because our habit is to think things out for ourselves.
But here is a cheering story. One of my PhD students was nominated by his students for a teaching prize. When he told his girlfriend, her response was, “You’ve made a difference to somebody’s life.”
That’s why we do it.
We just had the feedback on the new questionnaire. It is judged to have been a great success because it was easy to administer. Among comments from departments, I was gratified to see that somebody had pointed out that it asked about doing well on assessment rather than learning and understanding anything; our Director of Undergraduate Studies said he’d included my comment on this in our return, and suspected that I was the only one to say this.
How depressing.