Is the Leaving Cert fit for purpose?

I came across a tweet yesterday in which an Irish academic based in England suggested that the Leaving Cert was “not fit for purpose”. That raises the question as to what the purpose of the Leaving actually is. In the past, a good Leaving Cert meant you could go to college or go straight to the workplace (in the banks, for example), or become a trainee in any number of professions.

These days, though, the Leaving Cert is largely an entry exam for higher and further education and this is unlikely to change soon despite the periodic talk about apprenticeships.  Is it fit for purpose as an entry exam? Well, we do know that high CAO points are associated with low non-progression rates although less well correlated with grades but that might be too much to expect given how much change students experience during the transition from secondary school to third level.

Against that background many academics still complain that incoming students lack key skills required to thrive at third level. However, I’m not aware of any data to suggest that this apparent lack of skills is leading to high failure rates or, indeed, a dumbing down of the curriculum. So, if students lack third level skills on entry, we must be doing a good job of developing those skills in our students. (My tongue is firmly in my cheek here.)

What this means is that the Leaving Cert is a decent signalling device – if you do well in the Leaving it signals that you have a work ethic, the intelligence and an ability to perform under pressure, all vital attributes if you are to succeed at third level.

That’s not to say that the Leaving does not need to be modified to include more diverse ways of assessment. But should the Leaving Cert be transformed?

We can be sure of one thing: if the Leaving is transformed it will be transformed in the same way that the Junior Cert has been transformed into the Junior Cycle. That process was driven by constructivist ideologues who believe that the job of education is to teach “skills”, who believe that process is more important that content, and who have a particular fondness for inquiry based approaches to learning.  Many teachers and lecturers, me included, have serious reservations about this approach.

We need to be clear about one aspect of the Junior Cycle and it is this: it’s an experiment. We have no idea if it will prepare students for the Senior Cycle with many people believing that it won’t. If we turn the Senior Cycle into a more advanced version of the Junior Cycle, these two cycles might be well-aligned but we will have no idea if a constructivist Senior Cycle will have any value as a signal for performance at third level.

We have a moral duty to proceed very carefully when making changes to our education system. International experience (e.g. Scotland’s Curriculum for Excellence or Canada’s fixation with discovery maths) suggests that transformation is a high risk strategy.

Ten Education Questions for 2019

Here are some questions I’d like to have answers to in 2019. Guest posts welcome as long as your argument is backed up with evidence.

1. Are graduates really better ‘critical thinkers’ than school-leavers?
2. Does industry really need more PhD graduates or is the PhD yet another example of ‘signalling’?
3. In a small country with an open economy, does basic research conducted in Irish institutions make any meaningful contribution to the economy?
4. Does the world of work really want creative problem solvers or does it want conscientious team players who’ll do things the company way?
5. Does higher education really make you wiser and more ‘cultured’?
6. While state funding per student has halved in the last decade, how has the total funding per student fared in that period?
7. Is there uniformity of academic standards across the institutions?
8. Just how much are students working part-time and how much are they studying?
9. Does spending time ‘learning’ with toys like Lego really have any benefit other than keeping students engaged for a class period?
10. Are so-called STEM activities (see #stem on Twitter) really an exercise in pulling the wool over youngsters’ eyes?

10a What is the distribution of teaching hours for permanent academic staff across the university sector?

Is the ISSE a wasted opportunity? Part III

The answer to the question is yes. First I should stress that a lot of people have put a lot of hard work into this survey and I don’t want to disparage anyone’s efforts. But I think the fundamental problem is that there is a  ‘political’ undercurrent at work here whereby the ISSE is designed to conceal as much as it reveals; or at least to be “economical with the truth”. I think there is also an ideological undercurrent whereby a lot of educators with a progressive outlook view learning as ‘natural’ and blame a lack of engagement on poor or (in their view) outdated teaching methods.

But we really need to look at the background to this survey. The background is one where our higher education participation rates are increasing every year and are likely to increase in the years ahead. At the same time, government funding per student has halved while the missions of higher education institutions have broadened. While these things have happened, higher education  has become a market, not just a domestic one, but an international one. The words “rankings” and “quality” are on everyone’s lips.

As Irish institutions have fallen in the international rankings they have pleaded increasingly loudly for more government funding. At the same time, however, they have claimed that real quality – as opposed to the arbitrary markers of quality used in the rankings – has remain unaffected. The claim however is that, just like Achilles will never quite catch the tortoise in Zeno’s paradox, the  tipping point when quality starts to decline is never quite reached. It’s always a few years ahead.

So, in that environment, there is a strong incentive for the ISSE not to contain any questions about what’s really happening at third level. Because some of it might turn out to be embarrassing.

It would raise serious questions, for example, if students were revealed to be getting high grades through last minute cramming. It would raise serious questions if students were getting high grades while working so many part-time hours that they were effectively part-time students. It would raise serious questions if students were getting through exams by doing no more than rote learning?

You’ll notice that despite the STEM hype, there are no questions in the survey that pertain to the experiences of STEM students. You could ask: to what extent do permanent academic staff engage with laboratory modules? You could ask whether lab manuals are provided to students in a timely manner? You could ask about students’ experience of the  equipment itself; are students given the opportunity to actually use up-to-date equipment or is much of the equipment run down and obsolete?  Do undergraduates every see the inside of research centres? Is the health and safety of students taken seriously? And what about the availability of computers; do students feel they have sufficient access to high-spec computers?

And what about student’s opportunities to take part in research projects  at a time when many academics claim that being a research intensive university provides for a better undergraduate experience? And what is the quality of those research projects; do they involve working with post grads and postdocs? Is the supervision of sufficient quality?

And what about the way students go about completing continuous assessment assignments? Do they use a last minute approach? Do students feel they have the necessary IT skills to produce professional-looking documents. Is their ability to make deadlines affected by their part-time work or their socializing? Do what extent does alcohol consumption affect students’ ability to concentrate in class?

There are so many interesting questions we could ask but don’t. So yes, the ISSE is a wasted opportunity. Ask Bertie Ahern once said, it’s all “smokes and daggers”.

And that concludes the series…

Is the ISSE a wasted opportunity? Part 2

I’m presuming you’ve read Part 1 so I’ll plough on with the next lot of questions.

Questions relating to student-faculty interaction

During the academic year about how often have you…

1 Talked about career plans with academic staff?

The fact that half of all students replied “never” was picked up by the media who made a big fuss about it. But really it’s hard to take any particular stance on this – there are too many variables including the quality of the careers office, the personality of the student, the personality of the lecturer (especially his/her approachability), the size of the class and the workload of the lecturer.

2 Worked with academic staff on activities other than coursework?

Surprise surprise , nearly two thirds of students answered “never”. What were the questioners expecting?

3 Discussed course topics, ideas or concepts with academic staff outside of class?

Over 40% of students replied “never” to this question but what does that convey? It’s interesting in a so-what kind of way. There are so many factors just like there were in the first question.

4 Discussed your performance with academic staff?

38% replied “never” to this question and 43% replied  “sometimes”. I’m not sure what to make of this question and the replies because it is not clear to me, and may not have been clear to the students, if short exchanges during oral feedback on a particular assignment counts as “discussing your performance”.

Questions relating to Quality of Interactions

At your institution, please indicate the quality of interactions with

1. Students
2. Academic advisors
3. Academic Staff
4. Support Staff
5. Other Admin staff

I’m not sure who academic advisors are but apart from that the information yielded from this line of questioning was pretty uninteresting but the replies were broadly positive. The best interactions students have are, unsurprisingly, with each other. Here are the results – the seven columns represent the % answers on a seven point scale from poor to excellent.

My main issue here is that this section doesn’t delve into the tutor-tutee system that most institutions operate. Rather asking questions related to that system (how often do you meet?, who initiates contact?, where does the interaction take place?, how long does a typical interaction last?) the survey just asks one vague question around ‘quality’. That’s puzzling.

Questions relating to a supportive environment

How much does your institution emphasise…

1 Providing support to help students succeed academically

2 Using learning support services (e.g. maths learning centre)

3 Contact with students from different backgrounds?

4 Providing opportunities to be involved socially

5 Providing support for you overall wellbeing

6 Helping you manage your non-academic responsibilities

7 Attending campus activities (sport, cultural)

8 Attending campus events that address important social, economic, or political issues

A summary of the replies to these questions can be found below: the columns span the range from “very little” to “very much”. Not much can be gleaned from the data other than that most institutions are pretty supportive but you would worry that the line of questioning in this section is tantamount to asking about the extent to which the institutions have become  “nanny institutions” or even a therapeutic ones.

The last section of the survey involves a series of seemingly random questions – see page 24 of the survey document itself. I won’t go through them here because I’m beginning to lose the will to live. The replies to these questions are summarised below. They’re not very revealing and it’s not surprising that the media didn’t pick up on much in this year’s survey. For me, this last series of questions is the equivalent of asking a youngster “how was school today?” only to be answered with “grand”.

So having gone through all the survey questions I’m none the wiser really. I don’t have any real sense of what the big issues are, both for students and for the institutions. I’ll sum up in Part III.

Is the ISSE a wasted opportunity? Part 1

The ISSE is the Irish Survey of Student Engagement. According to studentsurvey.ie the purpose of the survey is to:

“provide benefits to each institution and its students by helping to improve feedback and appropriate follow up action. Objectives identified for the taught survey include:

• To increase transparency in relation to the student experience in higher education institutions
• To enable direct student input on levels of engagement and satisfaction with their higher education institution
• To identify good practice that enhances the student experience
• To assist institutions to identify issues and challenges affecting the student experience
• To serve as a guide for continual enhancement of institutions’ teaching and learning and student engagement
• To document the experiences of the student population, thus enabling year on year comparisons of key performance indicators
• To provide insight into student opinion on important issues of higher education policy and practice
• To facilitate comparison with other higher education systems internationally

All of this fine. But  when you get down to the actual questions things get a little odd. Bear with me as I go through each question, one by one.

Questions related to higher-order learning

The first and most obvious point to be made here is that what constitutes “higher-order learning” is by no means clear. These days I suspect that many in the world of education associate higher-order learning with things like inquiry based learning, problem-solving, group work etc. because that’s the constructivist philosophy. On the other hand, many would believe (me included) that there is no hierarchy of learning but there is a progression from acquiring new knowledge (by being taught) to developing an understanding of that knowledge (where understanding is really just more knowledge) to being well practiced at using that knowledge. But anyway, let’s look at the questions.

During the current academic year, how much has your course work emphasised…

1.Applying facts, theories, or methods to practical problems or new situations

Now I’m sure an engineering student would reply “very often” but what about someone studying English literature or French or philosophy? There is an obvious bias in this question towards an instrumentalist view of education.

2. Analysing an idea, experience, or line of reasoning in depth by examining its parts

This seems like a reasonable but not very enlightening question because one would have thought that examining anything in depth by examining its parts was a given in higher education.

3. Evaluating a point of view, decision, or information source

This question seems to be aimed at students in the humanities apart, perhaps, for the “information” bit. Most students studying on STEM programmes spend most of their time learning the fundamentals of their discipline. “Points of view” don’t arise very often except perhaps in final year where students might be asked to evaluate an academic paper.

4.Forming an understanding or new idea from various bits of information

This just seems a bit wishy-washy and I think the average student is likely to go for a middle ground here. Indeed if you look at the breakdown of the responses to the  4 questions in this section (each column represents from “very little” to “very much”) nothing startling or even interesting emerges.  

 

Questions relating to reflective and integrative learning

Again you could argue about what exactly these words mean but let’s plough on anyway.

During the current academic year, how often have you…

1.Combined ideas from different subjects/modules when completing assignments

The implication here is that integrating ideas from modules is a good in itself – and it is.  I have to say that in my experience it’s a rare thing for students to integrate their learning, which is why I’m surprised that the data show that 39.2% of students combined ideas from different subjects “often”.

2.Connected your learning to problems or issues in society…

I’m not sure why this question is being asked.  There’s an ideology at work here and it’s the belief that education should be immediately relevant or, to use a buzz word of progressive education, “authentic”. But think of a person studying pure mathematics or English literature, or music or even physics. Is it not enough to simply study those disciplines without having to divert into discussing the topical issues of the day?

3.Included diverse perspectives (political, religious, racial/ethnic, gender) in discussions or assignments

Now we’re getting into dangerous territory. There is a growing tendency for almost every activity or body of knowledge to be seen through a the lens of internationalist. It’s a tendency that has had a crushing effect on many American Universities and it is beginning to infiltrate British and Irish ones. Sure, talk about these issues in sociology or history  but let’s keep the thought police away from other disciplines.

4.Examined the strengths and weaknesses of your own views on a topic or issue

Again, this is a question that will be fine for some disciplines but puzzling to others and if a question has meaning for some disciplines but not for others then should it really be in the survey?

5.Tried to better understand someone else’s view by imagining how an issue looks from their perspective

I’m now thinking what a chemistry student would make of this question. Of course, students will, from time to time, question each other about how to solve a problem in organic chemistry or how to conduct an experiment but I don’t think that’s what the question is really asking.

6.Learned something that changed the way you understand an issue or concept

I would have thought that most students would answer “often” or very “often” to this because surely changing how you understand things is the very purpose of higher education.

7.Connected ideas from your subjects/modules to your prior experience and knowledge

Hopefully a majority of students would reply often or very often to this question.

Looking at the replies to these questions, we find a bit more variability than before. The biggest outlier relates to the question on diversity (third set of columns) where there is a high “never” score. That’s good.

 

 

Questions relating to quantitative reasoning

In my view these questions are pointless since the answer is so  discipline-specific.

During the current academic year how often do you

1.Reached conclusions based on your analysis of information (numbers, graphs, statistics etc)

This is a reasonable question but the breakdown of the question replies will be little more than a breakdown of the disciplines within the survey sample.

2.Used numerical information to examine a real-world problem or issue (unemployment, climate change, public health etc.)

Again we see this term “real-world” crop up. What does it mean? If a chemical engineering student uses numerical information to evaluate the performance of reactor, is a reactor the “real world”. It’s interesting that the question gives examples of ‘topical’ issues which perhaps shows the mindset of the questioners.

3.Evaluated what others have concluded from numerical information

There’s nothing wrong with this question but I’m not sure what it adds to the previous one other than to hint at the idea of peer evaluation.

The breakdown of replies reveals a high proportion of “nevers” which makes sense consider the focus of the questions

Question Relating to Learning Strategies

During the current academic year how often have you

1.Identified key information from recommended reading materials

2.Reviewed notes after class

3.Summarised what you learned in class or from course materials

All of these questions are mildly interesting.

The key thing, though, about this section is that it is a massive lost opportunity. There is huge scope here to really dig deep and find out how and when students study. My guess is that most students don’t study effectively even if they don’t cram – many spend their time reading and re-reading notes and/or highlighting. These are notoriously poor strategies. Furthermore, previous student surveys, when they asked the question “how often do you study”, found that student study levels are nowhere near the levels they need to be at. Oddly enough, that question has been removed from the survey.

I won’t give the breakdown of replies to these questions other than to say that only 15.3% of students study their notes “very often” after class while 41.6% “sometimes” study their notes. It’s a pity students weren’t asked when they study their notes because there is an optimum time for revisiting your notes – neither too soon nor too late.

Questions Relating to Collaborative Learning

Collaborative learning is  fetish of progressive educators because they believe that learning is a social process, not a solitary one. They also see education in instrumentalist terms and deduce that since collaboration is often required in the workplace, then it makes sense for students to learn collaboratively (which is a bit like saying rugby players should train by playing full-contact 15-a-side matches). The very fact that the following questions are being asked is because the underlying philosophy of the ISSE is a constructivist one.

During the current academic year how often have you

1.Asked another student to help you understand course materials

2.Explained course material to one or more students

3.Prepared for exams by discussing or working through course material with other students

Taking these three questions together, what is really being asked is what the class ‘vibe’ is. Students who know each other and get on well together tend to collaborate – I see it all the time in my laboratory modules when individuals are struggling with calculations. As for studying together for exams, I’d be neutral on this one – it all depends on your personality.

4.Worked with other students on projects or assignments

The replies to this question are mildly interesting but not fascinating. Only 10% of students never work with students on projects or assignments. But it’s important to note that much of the collaborative work that goes on in universities is a result of capacity limitations: too many students and too little lab space. Dividing students into groups is often a pragmatic solution, not necessarily a pedagogical choice.

Questions Relating to Effective Teaching Practices

During the current academic year to what extent have lecturers…

1.Clearly explained course goals and requirements

2.Taught in an organised way

3.Used examples or illustrations to explain difficult points

4.Provided feedback on a draft or work in progress

5.Provided prompt and detailed feedback on tests or completed assignments

All reasonable questions but I’ll make one comment before showing the results: there is increased pressure on academics to provide students with rapid, personalized feedback, much of it coming from organisations like the National Forum for Teaching and Learning. Given that the role of academics is to do a lot more than teach, expectations around feedback especially around individualised feedback (and even more especially around feedback of drafts) and the rapidity of it need to be examined.  It is quite easy for students to fall into a ‘satnav’ approach to receiving feedback.

Questions 4 and 5 above, which relate to feedback, show that for half of the student body the level of feedback is probably not what they would want. The solution in my view is to adopt a more whole-class, feedforward approach. People can only work so hard.

So what are my conclusions so far? So far I’m very disappointed.  The questions are often vague and too discipline-specific. I’d like to see questions like

• How many hours of part-time work do you do each week?
• How long is your commute?
• What percentage of your lectures do you attend?
• For lab modules, how much time do you spend studying, in advance, the lab manual?
• Even though slides might be available on Moodle/Blackboard, do you take notes during lectures?
• During semester, how many hours per week do you spend on reviewing your notes
• How many hours do your per week do you spend on continuous assessment?
• Are your continuous assessments well spread out?
• How often do you glance at your phone during lectures?
• How often do you come to early morning lectures too tired to concentrate?
• Does your timetable encourage you to remain in campus all day?
• How happy are you with the study spaces in your institution?

Hopefully I’ll see some of these in Part II.

Alternatives to Lectures?

It’s probably fair to say that the act of lecturing has been around for hundreds if not thousands of years. The purpose of a lecture was to chart a course through a subject for students, transmitting key points of knowledge, explaining difficult concepts, getting students to question, schooling them in the ways of experts in the discipline in question. For lectures to be effective students had to listen hard and they had to take notes

Lectures existed because there was no other way that the learning process could happen. With no photocopiers and no computers the only way the interaction between expert and novice, teacher and student, could take place, was orally.

With the advent of photocopiers and eventually computers and printers, it became possible to give students “handouts” to supplement the lectures. Distant learning now became possible and institutions like the Open University adopted a model where most of the learning occurred independently through the study of high quality booklets. This learning was supplemented with a small number of tutorials.

As digital technologies evolved, it became possible to add video into the mix and so we now have the resources to establish the perfect learning environment: one where the student can learn by listening, writing, observing and even interacting with each other and with software. (And of course through work placements.)

So what’s the best way to teach? I think the best way, if there is one, will be discipline specific but an approach that is overly dependent on students attending lectures and taking down notes is not going to be very effective and not because I’ve anything against lectures per se. The recent ISSE report indicated that only 15% of students review their lecture notes “very often” while 34% review their notes “often”. No matter how good the lecture is and no matter how good students are at taking notes, failure to review those notes means that most of their learning is done through cramming. And material learned through cramming is rapidly forgotten.

Ultimately this whole debate is not really about whether lectures are obsolete or not; it’s about asking what approaches to teaching can we adopt that force/incentivise/encourage (choose your own word) students to study consistently throughout the semester.

But there is a background to this. Lectures require a lot of space and space is expensive. And, when you have too little space and too many students, timetables become sub-optimal and learning suffers. So reducing our reliance on lectures makes sense.