So, what exactly is the College Board’s agenda for AP Chemistry? Something’s up.
AP Chemistry scores will be released tomorrow, July 6th, and there are going to be more than a few, unpleasant results for many students and their teachers. Only 8.4% of kids will receive 5’s (12,600 kids worldwide); this is a decrease from approx. 15,000 (10.1%) of candidates in 2014.
Firstly some facts. Up until the summer of 2013 and before the new curriculum was introduced (what we call the ‘legacy exam’), an average of approx. 17.3%* of students received a 5 on the test – *data from 2006-13. Then we come to 2014 the first year of the new curriculum, and the the first year of the new exam. This is a really important watershed moment because the new exam was chosen to set a new norm for scores, and therefore a new distribution of grades. For whatever reason (and this is the crux of the matter), the College Board decided that an unusually high bar should be set in order to get a 5. They coupled that decision with some new policies regarding the way questions would be presented, and what would constitute a ‘correct’ answer. That of course, is their business and they can do whatever they choose to, but the question is; Why have they decided to effectively cut the number of 5’s in half? What’s Going On?
Once that new norm had been set in 2014, the distribution of scores for 2015 should not be that shocking – at least not in one way. By comparing the 2015 performance of students against their peers in 2014 (the new norm), and by looking at multiple choice items that appeared on both tests, the College Board did a statistical analysis and found that kids did not perform as well in 2015 as they had in 2014, especially at the higher end of the grading scale. As a result, smaller percentages of kids were awarded 4’s and 5’s in 2015.
Now, in isolation the 2015 numbers actually do ‘make sense’, since they are normed against the 2014 data. I FULLY understand that, but that is not the part of the shenanigans that I am questioning. The real question is, why was the new exam graded so dramatically differently, when compared to the legacy exam?
You might be asking, ‘why did I not write this post 12 months, when the first precipitous drop in scores was evident?’ Simple, I wanted to wait and see if the College Board were actually going to do what we feared (i.e., ‘norming’ to some new, arbitrary scale) rather than 2014 being just a ‘blip’ associated with the new curriculum – now we know.
So what are some possible answers to the question, ‘What’s going on?’
There are a number of potential answers that we might consider.
1. The AP exam was too easy up to 2013. Well, it’s certainly true that the AP exam became easier throughout the 00’s when compared to the 60’s, 70’s, 80’s and 90’s, but it is still considered by many as one of the harder exams.
2. There is pressure from colleges, that are finding that kids with high scores are still unprepared for college chemistry (see #1). Maybe, I have no idea, but read on to find out why I don’t think that #1 or #2 are likely to be legitimate answers.
3. The College Board wants to justify the massive expense of re-designing the new curriculum by pointing to statistics that prove there has been some dramatic, ‘change for the better’. This seems more likely, but the problem with that hypothesis is the same as the problem with suggested explanations #1 and #2, and that’s this;
The CHEMISTRY (yes the chemistry, you remember, that stuff that the exam is supposed to be all about), didn’t get any harder in 2014 or 2015 than it was on the legacy exams. Really it didn’t!
This is a BIG issue, and it is the stumbling block that I cannot get over.
Ask any experienced AP chemistry teacher and they will tell you the same. As teachers, we did not find either of the new exams any more difficult IN TERMS OF THE CHEMISTRY than any of the prior, legacy exams – this is the crucial point. As a result, three kids with the same intellectual capacity in terms of their chemistry, and the same performance on the exams in 2013, 2014 and 2015, could easily receive, in order, 5, 4 and 3 as their scores. That makes no logical sense to me. However, it does ‘make sense’ when you consider things were newly-normed in 2014. However, all that does is bring us back to questioning the logic of the new-norming decision, when the chemistry did not change to any, significant degree.
So what did change significantly? Answer, two things.
Firstly the questions are being asked in a much more wordy, esoteric and odd manner, with a leaning toward ‘real world’ applications and forced ‘context’. Ironically of course, forcing those things is exactly the opposite of the same. Secondly, the College Board is now requiring very specific, (in some cases) non-chemistry related jargon and language, in order to score credit in answers (see the justify versus explanation debacle).
So in a nutshell one’s ability to decipher the chemistry on the AP chemistry exam, appears to be of increasingly diminishing importance when it comes to scoring highly on the same. That’s simultaneously stunningly sad, and perhaps not that surprising. Large parts of modern pedagogy are systematically attempting to reduce the importance of factual knowledge and understanding of subject matter, and replace them with some bizarre mishmash of process skills and ‘innovation’ (whatever that means). Here’s the problem with that thinking; you can’t ‘innovate’ or process anything, if you know nothing.
When people collaborate (another massively overused and overrated word in 21st Century edubabble), they do so because they need expert subject knowledge. Successful collaboration is not a 60’s style, Google/Apple love in, where the answers magically appear via holding hands and divine intervention, rather it’s a question of leaning on the deep SILO-ED knowledge of people in a different field – that’s the whole point of it!
The change we have seen in the AP Chemistry results does I fear, reflect some of that modern, flawed thinking in education. I’m a bit sad, chemistry seems to be the loser here.
I knew in 2014 that the new tougher norms were permanent; we saw the same thing happen in biology. Like Adrian, I don’t understand why they would so drastically decrease the number of successful students. If one is trotting out a controversial new course, one would expect that every effort would be made to ensure a successful and popular roll-out. Instead, they did the opposite, uniting those who disliked the revision from its inception, (like me) with those who are furious with the grading standards. I am convinced that the responsible party or parties are statisticians, not educators. In addition, the College Board has lied, in writing, about the impact of the 2014 exam on students. Very sad.
Screw this. I got a 5 on Bio last year with minimal studying. Now I have been taken down by a fkn buffer question. wth. I even did the MC for the 2015 and 2016 practice exams in class and there were about 2 buffer questions on each. Screw the college board and their twisted ideology.
By the way, I got a 46/50 on the 2015 MC and 44/50 on 2016 MC
However, I thought this years MC was so much easier. The calculation problems on the MC were easy af (and I even practiced with pre-2014 MC and those were harder) and the concepts were super easy.
The reason I’m pissed is cause the #3 question I got 2/10 (at most) on will screw my score up. And I’m going to college for a bio-chem major. So if I have a reservoir of 5s on insignificant subjects like European history and then a 4 on chemistry (the one that matters), either the colleges will think I’m inept in the subject or I didn’t study. Both are equally bad. And on top of that sh*t, I need to increase my ACT score from a 32 to a 34 to even be competitive for Princeton.
If this be the case with the grading standards/norms, my fear is the number of students wanting to take AP classes will drop off, especially in chemistry. Students are driven to score well on AP exams to achieve the college credit. Grading practices coupled with the manner in which questions are being asked might have a reverse effect on the popularity of AP testing in the long run. Test the chemistry, make it rigorous, and score it appropriately. Very frustrating, as an educator, to work so hard preparing the students for the test only to have each test normed on previous years. Each year should stand on it’s own merit. Thanks Adrian for sharing your insights on this.
Scott – it will remain to be seen if this backfires on the CB in terms of numbers, but I think the potential is there. Many teachers will be going back to classes this Fall saying, ‘The CB has now made it really, really, really difficult for you to get a 5’.
I agree with what you are saying. Chemistry at this level has not changed but the test is becoming less fact based/problem solving. The “forced context” questions are convoluted and could have been written by patent attorneys. I feel the is squeezing more students into a middle of the road performance on the exam.
The US has produced the top chemists in the world using the tried and true methods. The issue with that is, tried and true methods will propagate and expand the gap between those who can and those who cannot (or will and will not). The new tests are designed to make chemistry (and science in general) more inclusive. This sounds noble, but will not be effective in the long term. Those focused on real world applications should also know that doing real chemistry costs time and money. We (in the US) can only afford the best and most dedicated people practicing this science (independent of their backgrounds and pedigrees). The focus should be on how best to cultivate and motivate these students, not neutralize them.
“2. There is pressure from colleges, that are finding that kids with high scores are still unprepared for college chemistry.”
I think this explanation is probably one of the main driving forces behind the change. I worked in the chemistry department of a top-ranked research university for 12 years before returning to teaching high school. And, I can tell you that the majority of my colleagues at this university felt that scoring a 5 on the AP Chemistry exam was not a good indicator of how the student would perform in college-level chemistry classes. In fact, students who were planning to major in a science and scored a 5 on the AP exam were for all intents and purposes required to sign-up for a one-semester Honors General Chemistry class. This requirement was enforced within the chemistry department with very few exceptions even though the University would give the students credit for both semesters of general chemistry. Students in this situation were told to rescind their AP credit for the second semester course and take the Honors General Chemistry class instead.
Interesting, however, there also seems to be some contradictory anecdotal evidence, where teachers report that many of their best (5) students, tell them that college chem 101 was a piece of cake!