Why lack of knowledge is at the core of third level’s problems

I think it’s fairly obvious from my last post that I’ve found the last few weeks quite stressful. I’ve been really disappointed with the quality of work I’ve been getting from students and not just work that has been done under the pressure of the exam hall. The continuous assessment work has been the big disappointment . To some extent, though, many of the problems I encounter (shoddy graphs, shoddy formatting, meaningless sentences etc.) would be fixed by a simple change in students’ mindset – a little more conscientiousness would go a long way.

The real source of worry though is the fact that many students (not all, there are some good ones) seem to be unable to write coherent, well-argued answers to what are very straightforward questions. The typical response of many education commentators is that “we need to teach critical thinking”. But you can’t think critically unless you have knowledge in your long term memory to think with. And that’s the crux of the problem: many students simply don’t know very much at all. They will freely admit that most of them do not study during the semester but leave everything to the last minute when they rote learn in the hope that, whatever the question, they’ll be able to do a brain dump of information that has at least some connection to the question being asked. Talk to students about the 125-hour module and they’ll laugh and give you a look that says “you must be joking”.

The approach to learning that students are taking means that material studied at the last minute is rapidly forgotten so when the next semester of modules comes around they have very little knowledge in their heads that they can build on. In that sense they are always starting from scratch. And that’s why final year students frequently cannot make up solutions of a given molarity. And it’s why I’m frequently met with blank faces when I remind students that they have studied material before in an earlier module. They literally have no memory of it at all.

Advertisements

When enough is enough

Today (Friday) I came home from work at lunchtime. The reason: I was physically and emotionally drained. You see, this is the time of year when students who have sat exams in January want to review their scripts (“I can’t believe I got no attempt marks!”); students who submitted term papers in December and who I sent to the DCU Writing Centre for some remedial ‘treatment’ want feedback on their revised work; students who are “disappointed” with their mark in my lab module contact the programme chair to complain (despite not being able to draw a decent graph); final year project students seek daily advice on coding problems;  second year students email me to switch the date of an in-class exam because it doesn’t suit some of them for all kinds of reasons; and masters students want worked examples posted on Moodle so they can learn off the solutions to problems that they think will come up in next Monday’s exam.

I’m my own worst enemy because I promise a lot but sometimes I cannot deliver so I’ve cancelled my meetings for this afternoon. I’ve had enough for this week.

The Leaving Cert, signalling and third level non-completion rates

The fact that completion rates in university and IoT courses are closely correlated with CAO points has been known for some time. However, the fact that this is still newsworthy is interesting and probably has to do with the prevailing narrative, a narrative in which the Leaving Cert is deemed to be unfit for purpose and inadequate as a preparation for the rigours of 21st century higher education.

In his book, “The Case against Education”, Bryan Caplan argues that one of the main purposes of higher education is to “signal” to employers that a job applicant has a range of desirable attributes, including intelligence, work ethic and persistence. What they know is often not so important. It is hard to argue that there is not some truth in Caplan’s position.

For many years the HEA have been producing data that shows unequivocally that a good performance in the Leaving Cert sends out a strong signal that a student will have the intelligence, the work ethic and the resilience to succeed in higher education. Despite this, many third level academics and second level curriculum designers insist that the Leaving Cert does not prepare school-leavers for third level. Of course, the transition from second level to third level is difficult for all students but the idea that this is because of the nature of the Leaving Cert simply doesn’t stand up to scrutiny. Talk to third level students and you will discover that rote learning is alive and well in higher education and the difficulties that students encounter have more to do with motivation and time management than the so-called critical thinking culture of university-level learning.

Unfortunately, education in Ireland is increasingly dominated by ideologues who are in thrall to so-called student-centred methods that have failed everywhere they have been tried.  These ideologues will, no doubt, redesign the Senior Cycle in the manner of the Junior Cycle and it will be very interesting to observe what kind of signal, if any, performance in the revamped Senior Cycle sends out.

When lecturing becomes like teaching children

When students do badly in my modules, I feel frustrated, bewildered and annoyed, and all at the same time. I want to shake students,  to sit them down and ask them what the hell they are playing at. And when my emotions cool down, I ask myself if it’s really all my fault. Maybe I didn’t explain the material very well? Maybe I didn’t make clear what my expectations were? Maybe I’m just downright boring and failing miserably to inspire my students?

So I go through an annual academic cycle: it starts with hope and enthusiasm, it’s often followed by frustration and anger and it ends with doubt and guilt. And then it starts again: September approaches, the optimism returns and I look forward to getting stuck in again. There’s a touch of insanity about it all.

At the moment I’m bewildered. How is it that third year students cannot write a coherent sentence even when they’re not under time pressure? How can they persist in omitting units from their axes labels despite my saying, over and over, that they will lose marks if they do so? How can they put the ‘cause’ on the vertical axis and the ‘effect’ on the horizontal axis? How can they, the so-called Google generation, produce documents so badly formatted as to be almost unreadable?

It seems to me that we have an attention-to-detail problem and teaching college students is increasingly akin to teaching children. Yet the world of education is fixated with ‘skills’ like creativity and critical thinking and problem solving as if they were independent of the core attribute of conscientiousness.

The Leaving Cert and the Premier League

One of the purported aims of the new Leaving Cert grading and points-awarding system (in the news today) was to “take the heat out of the CAO system”. The argument was that by having fewer grade bands, students would be under less pressure to scramble for every point to get to the next grade and earn more points. But that neglected the likelihood that students would scramble for every point to ensure they don’t drop out of a grade band.

The same sort of thinking encouraged football leagues to move from a 2-points-for- a win-system to a 3- points-for-a- win system quite a few years ago now. The thinking was that having three points for a win would encourage teams to play more attacking football. But it was forgotten that opposing teams, especially those playing away from home, might well adopt a more defensive approach to stop the home team getting three rather than two points. The net effect? Nothing. Below is a plot that represents the current state of play in the Premier League. It’s a plot of the points each team would have if a two-point rule were adopted versus  current points  based on the three point rule.

premier

There are only four places where the rule has made a difference in ranking: three in which a team below has drawn level with the team above and only one where the team below leapfrogs the one above. (Chelsea would go ahead of  Spurs.)

Meddling with grades and bands was never going to change much – at least not for the better. The Leaving will always be a high-stakes, high-pressure exam as long as it is the de facto entrance exam for college and as long as we continue to push third level education.

Continuous Assessment, Grade Inflation and the Leaving Cert

The figure below shows the exam and CA marks for two of my third year modules in DCU for the last three years. The blue data is for a rather mathematical module for which the CA is worth 20% of the module marks. Two of us teach into the module and both of our CAs involve in-class tests. The red data is for a module for which the CA (also worth 20%) involves a short discussion-type assignment for my half of the module and a group presentation for my colleague’s half.

ca

For the ‘blue’ module (the mathematical one) the standard deviation of the CA marks is pretty much the same as that of the exam marks. However the average CA mark is 12% higher than the average exam mark.

For the ‘red’ module (the less mathematical one) the standard deviation of the CA marks is less than half that of the exam marks while the average CA mark is about 14% higher than the average exam mark. This bunching of CA marks is a common feature of non-mathematical subjects.

Two points can be taken from this. Firstly, the inclusion of continuous assessment elements in many modules these days is probably a factor in so-called “grade inflation”.

Secondly and given that the Leaving Cert is likely to remain the de facto entrance exam for college for the foreseeable future, great care will have to be taken if and when continuous assessment elements are incorporated into a revamped Senior Cycle. As it is, the grade distributions for Leaving Cert subjects vary quite a bit and unless there are well defined marking schemes for continuous assessment components, schemes that allow the examiners to award marks across the full 0-100% range, even for project work, those grade distributions could become even more disparate.

University ‘teaching’ then and now: Part 2

When I went to university in the US I was struck by how ‘managed’ student learning was. Each module had homeworks, quizzes, mid-terms and finals. It was common for lecturers to follow a single textbook and to assign readings for that textbook. It was a radically different approach to what I was used to in UCD. In Ireland we were expected to manage our own learning: to attend lectures, to take notes, to do problem sets in our own time, to revise, in a well-planned and effective way, a full academic year’s work in advance of the final exams in May and June. When I went to America it made sense to me that they referred to their universities as “schools” because they were actually a lot more school-like than our universities.

Over the years, our system here in Ireland has become a lot more school-like. We are modularised and semesterised, we have adopted a learning outcomes philosophy and it has become the norm for lecture notes and supporting materials to be provided online. Students are assessed more often and in shorter exams (down from three hours to two hours); continuous assessment is common and often takes the form of in-class tests not unlike the quizzes employed in the American system; and the entire environment is generally more supportive and compassionate than before; students are taught, not lectured at. External examiners tend to adopt a very student-centred approach and on more than one occasion I have been ‘encouraged’ by extern examiners to break down my questions into parts in order to make it easier for weak students to pass. The consequence in many cases is that the student is effectively told how to do the question. Our concept of fairness has definitely shifted. There have also been regulatory changes: to get a 2.1 these days you need to average at least 60% whereas before it was 63%.

What all this means is that the standard of the material that students learn is not necessarily any lower but the current methods of assessment make it easier for committed and astute students to get very high marks. These days, marks in the 80s and 90s are not uncommon in quantitative subjects whereas they would have been very rare a decade or two ago. Part of the reason for this is that the good students ‘ace’ the continuous assessment which are based on a very small amount of material.

Does any of this matter? Personally I don’t think so as longs as we all know where we’re coming from. Anyway, the 70% bar for a H1 is purely arbitrary and the reluctance in the past to award marks over 70% in many disciplines was little more than a fetish and not based on any clearly articulated criteria as to what a first class honours answer should look like. Indeed, one of the more recent trends in the preparation of exam papers is the need to provide the external examiner with either model answers or well-defined statements as to what a student has to do to achieve a certain grade. This increased transparency is bound to lead to an improvement in grades as it has done in the Leaving Cert where marking schemes are well known to the students.

A final point to note concerns the exam board system. In my experience these are conducted in a fair and, as far as possible, in a consistent way. Yes individual marks are ‘moved’, mainly for compassionate reasons (e.g so that a student doesn’t have to repeat the year because he/she is a few per cent short) but the type of incident that occurred in Tralee where an arbitrary number of marks are added to every students’ score is highly irregular. Mind you, if my entire class failed my module I’d be taking a long hard look at myself and not railing against the system.

University ‘teaching’ then and now. Part 1

The recent fuss about grade inflation has me wandering a little down memory lane. I have absolutely no doubt that the standard of ‘teaching’ in higher education is far ‘better’ now than it was back in my time. (I’ll explain what I mean by ‘better’ in my next post.)

Those of us who were lucky enough to go to university back in the 1980s had experiences of lecturers who were just so dire that being in class was almost like living through a real-life Monty Python sketch. When I studied chemical engineering in UCD, we had one lecturer who just told anecdotes about his career in the metallurgical business while leaning back in his chair, legs crossed, staring at the ceiling. He always wore a white lab coat during his lectures and for no apparent reason. It’s not as if he ever got up off his arse to use the blackboard or the overhead projector. Another, a well-known business guru at the time, simply used the business pages of The Irish Times as his lecture notes. Not much preparation required there. Then there was the bloke who used to put up an overhead of complicated organic chemistry reactions for us all to transcribe while he looked out the window in silence, interrupting the quiet every now and then with a comment or two about electrons jumping from one place on a benzene ring to another – or something. He religiously got through four overheads per lecture. I often used to think that he suffered from depression – he certainly didn’t seem very happy – but recently I heard that he was an avid sailor so maybe he was just dreaming about being out on his yacht, standing on deck, face to the wind.

One eminent professor mumbled his way through lectures on seemingly random topics, aided by overhead transparencies that he appeared to have written on a rapidly-moving DART. Another, a bow-tied engineering legend, literally scribbled on the overhead projector and, although entertaining, you learned absolutely nothing in his lectures. It was appalling stuff.

But there was unintended method in the madness. The poor quality of some lecturers was the perfect incentive for us to develop our own “independent learning skills” as they say these days. And so, lecture notes, researched and written by students, got passed down, modified and improved from year to year. This was collaborative learning long before that term became fashionable.

Some lecturers didn’t seem to quite know who they were teaching. As chemical engineering students we had to endure one module on electrical engineering – electrical machines and the like – riveting stuff. None of us knew what the hell the lecturer was talking about and he seemed to be under the impression that we were third year electrical engineering students. Despite being clueless, we all rote-learned our way through the course and I am only slightly embarrassed to say that I got 75% in that subject without having the faintest idea what I was doing. It’s one of the few marks I remember – because of the madness of it. Rote learning was alive and kicking in the 1980s.

Nonetheless, I did have some excellent lecturers who were able to make even the most technical of subjects (fluid mechanics, reactor design, mathematical physics etc.) really interesting and they did so because they simply had some sort of lecturing x-factor. They did nothing fancy or innovative – they lectured in a traditional way and they were brilliant. Most of our best lecturers were in the chemical engineering department itself where there was a wonderful family atmosphere and, for me at least, a real sense of wonder and discovery. Chemical engineering seemed to be such a marvellous and creative mix of physics, chemistry and technology. We were learning about phenomena that you’d never have imagined would be the subject of an academic discipline, like the coalescence and break-up of drops and bubbles! I loved it.

Things are very different now: lecturing has become teaching and the modern university is much closer in spirit to secondary school than it used to be. More on that, and grade inflation, next time.

Is the Leaving Cert fit for purpose?

I came across a tweet yesterday in which an Irish academic based in England suggested that the Leaving Cert was “not fit for purpose”. That raises the question as to what the purpose of the Leaving actually is. In the past, a good Leaving Cert meant you could go to college or go straight to the workplace (in the banks, for example), or become a trainee in any number of professions.

These days, though, the Leaving Cert is largely an entry exam for higher and further education and this is unlikely to change soon despite the periodic talk about apprenticeships.  Is it fit for purpose as an entry exam? Well, we do know that high CAO points are associated with low non-progression rates although less well correlated with grades but that might be too much to expect given how much change students experience during the transition from secondary school to third level.

Against that background many academics still complain that incoming students lack key skills required to thrive at third level. However, I’m not aware of any data to suggest that this apparent lack of skills is leading to high failure rates or, indeed, a dumbing down of the curriculum. So, if students lack third level skills on entry, we must be doing a good job of developing those skills in our students. (My tongue is firmly in my cheek here.)

What this means is that the Leaving Cert is a decent signalling device – if you do well in the Leaving it signals that you have a work ethic, the intelligence and an ability to perform under pressure, all vital attributes if you are to succeed at third level.

That’s not to say that the Leaving does not need to be modified to include more diverse ways of assessment. But should the Leaving Cert be transformed?

We can be sure of one thing: if the Leaving is transformed it will be transformed in the same way that the Junior Cert has been transformed into the Junior Cycle. That process was driven by constructivist ideologues who believe that the job of education is to teach “skills”, who believe that process is more important that content, and who have a particular fondness for inquiry based approaches to learning.  Many teachers and lecturers, me included, have serious reservations about this approach.

We need to be clear about one aspect of the Junior Cycle and it is this: it’s an experiment. We have no idea if it will prepare students for the Senior Cycle with many people believing that it won’t. If we turn the Senior Cycle into a more advanced version of the Junior Cycle, these two cycles might be well-aligned but we will have no idea if a constructivist Senior Cycle will have any value as a signal for performance at third level.

We have a moral duty to proceed very carefully when making changes to our education system. International experience (e.g. Scotland’s Curriculum for Excellence or Canada’s fixation with discovery maths) suggests that transformation is a high risk strategy.

Ten Education Questions for 2019

Here are some questions I’d like to have answers to in 2019. Guest posts welcome as long as your argument is backed up with evidence.

  1. Are graduates really better ‘critical thinkers’ than school-leavers?
  2. Does industry really need more PhD graduates or is the PhD yet another example of ‘signalling’?
  3. In a small country with an open economy, does basic research conducted in Irish institutions make any meaningful contribution to the economy?
  4. Does the world of work really want creative problem solvers or does it want conscientious team players who’ll do things the company way?
  5. Does higher education really make you wiser and more ‘cultured’?
  6. While state funding per student has halved in the last decade, how has the total funding per student fared in that period?
  7. Is there uniformity of academic standards across the institutions?
  8. Just how much are students working part-time and how much are they studying?
  9. Does spending time ‘learning’ with toys like Lego really have any benefit other than keeping students engaged for a class period?
  10. Are so-called STEM activities (see #stem on Twitter) really an exercise in pulling the wool over youngsters’ eyes?

10a What is the distribution of teaching hours for permanent academic staff across the university sector?