Here’s my take on the 2016 AP Chemistry Scores. Actually, it’s more a repetition of what I have said about the re-distribution of scores that we have seen on new exam, and a re-examination of the 2015 scores. Bear with me!
Something just doesn’t make sense about the AP chemistry scores on the new exam. A few facts.
- Honestly, I’m not a conspiracy theorist.
- I’m pretty sure that I understand the process that leads to the production of AP chemistry scores, at least to the extent that it is possible given what the College Board allows us to know – it’s hardly a transparent process despite the de-briefs that happen at the summer gatherings of the AP national conference, ChemEd or BCCE.
In order to comment on 2016 AP chemistry scores, we really need to go back to 2013, and the first examination of the re-designed AP biology course.
AP biology was re-designed one year prior to AP chemistry, and as a result we had a new exam in 2013. When those scores were published in July of 2013, a shockwave went through the AP biology community. In a nutshell, the %’s of kids receiving 5’s went from what I might characterize as a ‘traditional number’ of just under 20%, to 5%. In addition, the ‘traditional’ % of 1’s awarded shrank from around 35% in 2011 and 2012, to 7% in 2013. Wow! Essentially 5’s and 1’s were DRAMATICALLY squeezed. In AP biology a very similar pattern was then repeated in 2014, 2015 and 2016.
Obviously this dramatic change raises some serious questions. The one answer that the CB has given is actually completely valid! They say that given that this is an entirely new exam, there can be no valid comparison made between the legacy exam and the new exam. Fair enough, the CB has clearly (in the case of chemistry) decided that they wanted to move significantly away from what I call ‘hardcore chemistry content’, and toward making the AP chemistry exam an examination of language skills and reading comprehension. It would appear that something similar may have happened in AP biology.
Of course, this lurch makes perfect sense when you consider the general wind of edubabble change, driven by schools of education and ‘gurus’, that is a self-serving industry and unrelated to the subject content of biology, chemistry or indeed, any other subject. Such a shift in emphasis is misguided and dangerous in my opinion, BUT having said that it is absolutely the prerogative of the CB to run their exam, in whatever way they see fit. If ‘we’ don’t like it, we have to campaign to abandon the AP test, and replace it with something that is better and by the way, that is already available.
Now let’s analyze the re-designed AP chemistry exam in a similar way to the new AP biology exam. Prior to the first examination of the re-designed course, one could expect ‘traditional’ %’s to be in the mid to high teen range for 5’s, and in the high 20’s for 1’s. In 2014 (the first examination of the new course), we saw a very similar – if not quite so dramatic – change in the the numbers as AP biology, with 10.1% now awarded 5’s, and 21.8% awarded 1’s. This was no surprise given what we had learned the year before in biology, and is largely inline with the new philosophy of the CB. 2015 produced a further fall of 5’s to 8.4%, and the number of 1’s changing to 22.8%. 2016 has the numbers at 9.7% and 22.4% respectively. Again, if the CB create a new exam and want to go in a different direction in terms of what they assess, that’s their business.
So, aside from what I consider to be the dismantling of emphasis on solid, chemical content (calculations, Lewis acids/bases, colligative properties, phases diagrams etc.), and given the acknowledgement that the CB can do whatever they chose with their own exam, what’s the problem?
For me, there are three issues.
- The damage occurring to chemical education. That’s not a rant for this post, but its outlined in part above in terms of the de-emphasis of certain aspects within the course, and the increasing emphasis on others. Such a huge new emphasis on reading, writing and comprehension skiills is, I believe, to the detriment of chemistry.
- One question remains wholly unanswered for me. If the chemistry – the actual nuts and bolts of stuff reacting with stuff – has not changed, and the kids have not changed, then why do we now declare that the %’s of kids receiving 5’s needs to be axed in half? Suddenly these kids don’t understand chemistry as well as they did previously? I don’t think so, rather I believe it’s that the emphasis on the exam has dramatically changed, and that teachers are perhaps not caught-up. Alternatively, it could be that teachers ARE caught-up, but are refusing to swallow the CB’s new emphasis, are continuing to teach what they believe to be a solid, advanced high school chemistry course, and as a result, kids receive what teachers believe to be a great grounding in college 101 chemistry, but that does not dovetail with the new exam very well. (Now, there could be a bunch of other answers to the question, one of which is that colleges want less people testing out of college 101 courses for financial reasons, but as I said above, I’m not really a conspiracy theorist. BTW – not being a conspiracy theorist does not mean that I don’t suggest that you follow the money when you observe things that look odd!)
- What happened in 2015? Something really odd occurred in 2015 IMO. The number of 5’s dropped to its lowest level ever, 8.4%, but take a look at the data from the kids that I teach.
There are a lot of data there, but here’s the piece that I want to focus on. From 2002 to 2013, the old (legacy) exam, I see an average of 76.22% of my kids getting 5’s. Then take a look at years 2014 and 2016 of the new exam (ignoring 2015 for a moment), and you’ll see an average % of 5’s equal to 75.48%. I believe that illustrates the fact that I (despite my better chemical judgment which would have me ignoring all of the Big Ideas, Enduring Understanding, Essential knowledge and Learning Objective nonsense), have been able to adjust my AP chemistry course to pretty much make a perfectly seamless transition from the old to the new course in terms of exam preparation.
Now consider 2015. With the same ability kids, in the same school, with the same teacher, delivering the same course, with the same attitude (and you should note, with very similar 5%’s nationally from 2014-16), we see the %’s of 5’s drop from an average of 76.22% on the legacy exam over 12 years, and 75.48 over the 2 other years, to 38.33%! This makes NO sense at all, and as a result I believe that something went awry with the setting of grade boundaries in 2015.
Honestly, I was expecting my 2016 scores to be similar to the the 2015 numbers, as what I thought might be a new norm was established, i.e., significantly lower %’s of 5’s amongst my own population. However, the violent swing back toward legacy %’s of 5’s for my kids in 2016, has only served me to re-open my questioning of the 2015 grade boundary setting, that I wrote about last year here, and here.
BTW, ICYMI.
One of the problems in discerning the College Board’s motives is that their spokesman, Mr. Packer, LIES ALL THE TIME! It is not only the documented lies about gender differences. If you read the letter sent out in summer 2014…first they try to suggest that the decrease in numbers of 5’s was NOT pre-planned. No, it it the result of questionnaires that asked AP teachers to pick a “5” score for each free response question. We know that this is utter bunk, since nearly every redesign led to fewer top scores. Of COURSE it was pre-planned, so why lie about it? Then, in the same letter, amazingly, Packer goes on to explain why the % of fives in chemistry SHOULD be smaller, based on some study of how well AP students did in organic chemistry. This, of course, an after the fact rationalization. Are there similar studies in biology, or any of the other AP subjects that had their top scores severely reduced? Lies upon lies.
I don’t know what the real reasons for the change were, but they are certainly not what Mr. Packer is claiming. Among the possibilities are – the number of top-tier colleges granting credit for 5’s has been declining steadily. Perhaps they hope to change this by decreasing the number of 5’s, and thus claiming greater value. From the college’s point of view, however, since the new course leaves out about 25 % of what they teach in gen chem, I would think they would be less likely to grant credit.
Another possibility – the College Board wishes to infuse high schools with its peculiar, edubabble based methodologies. By lowering the numbers of top scores, perhaps they hope to force change. “Do it our way, and your scores will go up.” That hasn’t happened yet, and won’t happen, but not to worry. They are perfectly capable of secretly “tweaking” the standard to make it come out any way they choose.
I did not see the big change in grades from 2014 to 2015 that Adrian did. And the % of 5’s this year is closer to the 2015 figure than to the 2014 figure, isn’t it? It will NEVER go over 10 % again, using the current method of setting parameters, unless the College Board deliberately reconfigures the norms.
The problem is not benchmarking off of basic traditional frosh college chemistry. Instead they have disappeared up their own assholes with the psychometric BS.
If you think that Chemistry has problems, AP Physics 1 is a slow motion train wreck. Everything the college board did to Chemistry that deviates it from a true college course, you can square that for Physics. They essentially split the old AP Physics B into AP Physics 1 and AP Physics 2 Algebra Based (it should be named Language and Composition). They’ve limited AP1 almost exclusively to mechanics…most likely as a bias towards the modeling community.
The scores in 2015 were 4.1% 5s, 12.8% 4s, 20% 3s, 30.2% 2s, and 32.9% 1s. The scores in 2016 were statistically indistinguishable, 4.3% 5s, 13.6% 4s, 21.3% 3s, 30.7% 2s, and 30.1% 1s. In Biology during it’s roll out, the 1’s and 5’s disappeared. What was more insulting was that Trevor Packer tweeted it as an improvement. Meanwhile, it was the 2nd worst result in the history of all AP testing, with last year being the worst.
There are some considerations when looking at this data. The number of test takers literally doubled, which occurred because the college board recommended most schools convert their 1st year honors courses to AP1. So, the percentages may take into account that 50% of the students in it are either not ready for the subject or never belonged in the first place. That being said…if you eliminate 50% of the test takers within this subject, you then start to approach parity with other AP score distributions. I guess one 1 is a tragedy, 100,000 1s is a statistic.
What’s alarming is that this is the 2nd year of the rollout, yet we see no reduction in test takers and a nearly identical score distribution. My guess is that the college board did actually nothing to address the scoring cutoffs. I think this whole redesign is the biggest blow to science education in several decades. They’ve eliminated important material in Chemistry, and the Chemistry community obviously takes issue with it. In Physics, they’ve created a situation in which kids no longer see 50% of the set of topics typically found in a 1st year Physics course. And…the data indicates, not many opt for the 2nd year after going through that 1st year of torture. As it stands, we now have a net loss of 75,000 of our best students in the country who no longer get exposure to Thermodynamics, Fluids, Optics, E&M, or Modern Physics.
This is pathetic. I’ve heard the splitting of physics as being laughable. Also the word on the street is that in physics 1, a score of 5 could be achieved by not having to write a single number. True? Schools of Education are killing……education. Sad.
The feedback from the students is that they pick up their calculator a grand total of two times max on the exam. That’s not saying much when most students reach for the calculator to calculate 10/5. Yes it’s true, you can achieve a 5 without ever writing out a single number. The schools of education has indoctrinated people into pushing the idea that mathematical problem solving and demonstrating true understanding are mutually exclusive.
As someone who holds degrees in Chemistry, but happens to currently teach Physics, it’s refreshing to see that the sentiment that I have is not unique. Unfortunately, the AP Physics community for the most part seems to be content to sit back and enjoy the abuse and in a lot of cases blindly support it. They’ve written off the bad results and are content to wait and see and not “rush to judgement”. One of my goals this week is to read through the past 3 years of your blog just to see how much correlation there truly is in the redesigns of both of these subjects.
1. The math has been removed
2. Explanation and writing is now preferred
3. Verbosity is ubiquitous throughout the exam
4. Big ideas are just a distraction
One of the things that has changed though is that the Physics has gotten a lot harder. I think a large portion of new teachers are setup for failure when they are being advised by the curriculum framework to teach a concept conceptually, but not mathematically.
Something similar has happened in chemistry where the CB talks a LOT about the importance of lab work in the new curriculum. In terms of AP scores, NOTHING could be further from the truth. Often, inexperienced teachers suffer (with bad scores) as a result. In chemistry as in physics, I think it’s central to understanding that the math comes with the concept. It cements everything.
Ahhh…modeling BS. Tremendous!
Hold on a second. I took “PSSC” physics in HS in 1983. It was the harder of the two algebra-based options (Harvard Project was the artier one). I was at a top 10 suburban DC high school (this was pre TJ). There was no option to take a calculus based physics (even as a second course, how bio and chem are done).
Even at that time, it was universally understood that AP Physics B was irrelevant for anyone going into STEM major in college. And I’m not talking about MIT. I mean any state engineering school. They all expected a calculus based physics course. That was and is the NORM. So, it was common knowledge that taking AP Phys B exam was pointless. Nothing has changed since then and this should not be “new news”! So how they screw up the Phys B exam is completely pointless, since the exam is pointless.
Now, Phys C? That’s another kettle of fish. It did (and I assume/hope still does) approximate a calc-based physics course.
I took the Phys C exam for the heck of it and mechanics was no problem. Any kid who has calc or even pre-calc is screaming to use derivatives and integrals for the whole position, velocity, acceleration kinematics topic. Rotary motion was not (really) covered in PSSC (just a little bit of centripetal forces). But the equations for angular mechanics are almost completely analogous to translational motion. Use moment of inertia instead of mass, etc. Was able to cram my way to a 4 on AP C mech.
E&M was too tough to cram. I really didn’t know how to do surface integrals (for charge) and just thinking about those spheres with charge spread out was hard/different. Very little of the test covered extremely simple circuit analysis of PSSC. And there was no optics coverage either. Got a 2.
Ended up being surprised and getting validation of first semester physics in school. Always felt that I missed out on learning super hard gyroscope problems, but never had any issues with rotary motion in p-chem, etc.
Bottom line: Physics B is irrelevant and always has been. Just kill it with fire already. It is/was an example of AP brand being used to cover advanced HS courses that are not the common intro college class.
What you say is what my AP physics teaching colleague says (36 years into his career), i.e., that an ‘AP’ course that is trig based is a waste of time for any subsequent, real physics work. Period.
I have some real questions about APUSH and APUSG, too. The first was well established in 1983; the latter came out in 1984.
Now, I went to a non-traditional college (in that it was traditional, haha). But still. We had required Western Civ 1 and 2 in frosh year. Sort of approximating the AP courses (I assume). At least of the time. They are probably all multicultural now. But still. Basic world history. But overview US history and CIVICS were definitely considered to be HS courses. We didn’t even offer them as electives in college. Maybe diving deeper like a Civil War elective. But no required or even optional US history in college. Isn’t that high school stuff?
So am I wrong? Do/did normal schools have basic US history as a course in college? Does anyone actually validate anything from taking the exam?
I would argue that AP Physics B was just as important from an academic standpoint, if not more important for someone going into Chem. AP Physics C is limited to Mechanics and E&M. As someone who is finishing up a PhD in Physical Chemistry and went to undergraduate in Chemistry (yet ironically teaches high school Physics), the topics like thermodynamic process, diffraction, fluids mechanics, and optics do wonders for your understanding of Chemistry and all that neat instrumentation you are likely to use as a chemist. Exposure to those topics is also very important to possibly getting someone to consider Physics as a major. These things are not taught in Physics C. There’s no doubt that the Physics found in C and math behind it is something all Physical Science or Engineering majors should be exposed to. That being said, not everyone is pursuing those majors, and there are plenty of majors within college that don’t require a calc based Physics course. In that respect, AP Physics B filled that niche. That’s why it was always up to schools on whether or not they would award credit for APB. Basically, Science and Engineering majors could only get credit for C while other majors had a shot at getting credit for B.
Several schools such as the ones I’ve taught at would offer AP Physics B as a prereq to entering AP Physics C. The split into AP1 & AP2 has forced most schools that did that to go from APB to AP1 and AP2 enrollment has been very low. As a result, nationwide, you now have 75,000 less kids getting exposure to Thermodynamics, Optics, Waves, Fluids, Electricity, Magnetism, and Modern Physics. This is bad for science in general.
Beyond that, several teachers have chosen to have their students take both the AP Physics 1 and Mechanics C exams simultaneously and in each case I’ve seen, their students score higher on Mechanics C? What kinda of stupid system has an algebra based course harder than their respective calculus based course?
optics is covered in high school physics. If you really believe your argument, you should be making the point to change the COLLEGE physics requirement for Chemistry to add a third semester after mech, E&M. After all, AP I supposed to allow validating COLLEGE COURSE content.
And your response also failed to address that algebra based mechanics and E&M are NOT college requirements for chem degree. Halliday and Resnick is. So what course do you VALIDATE (??) by passing AP Phys B?
There are other majors besides Chemistry and course equivalents to AP Physics B exist in plenty of schools. I would like to point out however, that since AP1 has been so royally screwed with by the schools of education, there exists no college course equivalent to that in the entire country.
Well your earlier comment was about chem majors. And chem majors need to take calc based physics. So an AP B exam is useless to them. Good, bad or indifferent. It won’t allow them to pass out of their college physics. If you want to run different sorts of honors high school physics classes fine. But AP B is not a real college physics course and that is what everyone was saying 30 years ago. And this isn’t even some professor/teacher attitude–this is from the student perspective.
AP should match college courses. Traditional college courses. General college courses. Not some weird Caltech calculus with real analysis mixed in. But the traditional Thomas Finny style of course that has been around since Granville. Same applies in physics, chem, bio.
College Board needs to stop playing edubabble politics and trying to drive reform content. Just match the college content. and conservatively and as a lagging indicator, not a leader. If AP wants to expand they should stop moving into being de factor high school honors classes, but start covering higher up college courses. No reason they couldn’t do calc 3 and ODEs.
Here in Arizona, we have factors conspiring against us…the reduction in the number of 4’s and 5’s on the exam and no credit given for a 3 from the state’s largest community college system or any of the 3 state universities!
No credit for 3 is not that new a story. As a student in the 1980s, a 3 was considered very chancy to get credit. 4 or 5 needed.
It’s relatively new here and has only changed in the last 5-7 years. Neighboring states (CA, CO, NM, UT) still offer credit in many institutions for a 3. As a teacher throughout the 1990’s and, as I said, up until very recently, many of my students obtained 4+ credit hours within the state for a score of 3.
And if this has become common, why is a 3 considered passing? 🙁