Freddie Starr Ate My Hamster (AKA AP discussions #ChemEd15)

August 11, 2015

At least one member of the AP Chemistry Test Development Committee (TDC) has described this blog as no more than, in his words, a ‘tabloid’. I’m not quite sure what he means by that, but presumably it’s an attack on me because he perceives that what I write here to be somehow invalid, or perhaps sensationalist.

Nothing could be further from the truth, and it exposes a deep misunderstanding about me and what I am trying to achieve in my exposure of the mess that is currently AP chemistry. If I wanted sensationalism I’d go for something more akin to The Sun, and perhaps start reporting on the eating habits of 1970’s/80’s celebrities.


So, back to the matter in hand.

This post is about the two AP sessions (that basically turned into a single session) held at ChemEd 2015 on July 31st, and the questions that I feel were not answered.

During opening remarks, Harvey Gendreau described some of the discussions regarding the 2015 AP chemistry scores that had taken place on the AP chemistry forum as, ‘downright nasty’, and he highlighted the large changes in grade distributions by using Total Registration stats from recent years. He suggested that we (AP chemistry teachers) are being ‘short changed’ (his words), and that the CB has some, in his words, ‘splaining’ to do. I agreed, and so did many in the room judging by the nodding!

Harvey’s remarks were followed by a two and half hour presentation by Roger Kugel (Chief Reader, AP chemistry) and question and answer session with Paul Price and Dave Yaron from the TDC. (I’m beginning to think that Jamie Benigna does not exist, since he was originally advertised as being at ChemEd and he has failed to respond to umpteen emails that I have sent him over the years).

Kugel’s presentation of the 2015 questions and answers from the released exam provided very little that I didn’t already know, but there were a few nuggets that were of interest to me. Below I have highlighted some things that I think are worth repeating.

  • 153,557 total exams in 2015 – 349 Readers & Leaders

My original calculation of the national average AP chemistry score of 2.61, was based upon the previously released number by the College Board of 150,000 test takes. With the new numbers, the math determines that the actual national average is 2.609, which is actually slightly worse than the original numbers posted by the College Board would suggest! Here is my Tweet of July 6th that gives perspective over the last 14 years (with the original number that I calculated for 2015 still shown).

AP Averages Tweet

  • Bizarrely, apparently the College Board have now gone BACK to allowing a tolerance of +/- 1 sig. fig on the sig. fig question. WHY OH WHY OH WHY (tabloid moment) can’t they come up with a definitive, once and for all statement on this? It’s annoying, uncommunicative and completely unnecessary to leave us all guessing about the policy on this year after year. In addition to that, I *think* that I heard Kugel say that the sig. fig point could be applied across the whole of one of the FRQ questions. As far as I am aware, most people that I know (including me), thought that the sig. fig point would only be applied to one part of one FRQ (not potentially all parts of one FRQ). Perhaps I have been wrong about this all along, but the fact that someone like me, with his ear so close to the AP chemistry ground, is not clear about this, just serves to show the disaster than College Board communication has become.
  • The bond angle accepted for the C-O-H bond in ethanol was between 100 and 115 degrees! 115 degrees!? The range is understandable (and consistent with previous precedent), but 115? I have no idea how that can possibly be justified.

But for me perhaps the most important thing that was confirmed from the exam discussion part of the presentation, was something that ‘we’ have suspected right from the beginning of the new exam. Because Kugel said time and time again that the way to improve student performance was to get kids to practice writing out their answers, it is absolutely crystal clear to me that there has been a shift in emphasis AWAY from chemistry, and toward English skills. There can be no doubt about this IMO. On multiple occasions he stressed the need for careful reading of (the increasingly esoteric) questions, and time and time again he said that kids should be practicing writing out their answers longhand during their AP courses.

In addition, some of the anecdotal evidence that has starting coming in on the AP discussion group about the relatively poor performance on the exam from kids who have English as a second language, categorically backs this up. In short, communication is now a bigger part of the AP chemistry exam than it ever has been in the past, and chemistry has been de-emphasized in relative terms. In the past, if you knew your chemistry then that was enough, that is categorically no longer the case IMO.

Moving on to comments about score setting and the catastrophic decimation of the number of 5’s awarded in the new exam.

Essentially, there was the usual smoke and mirrors conversation about statistics and the questions that appear from year to year that act as ‘equaters’ that help to set the score boundaries, but the central question remained unanswered;

What agenda does the College Board have, that would justify the scores plummeting as they have, when the chemistry on the new exam is getting no more difficult?

That question remains unanswered as far as I am concerned. This despite the fact that I fully understand that subsequent exams are compared to the first new one (2014), and not earlier exams, and that ‘equater’ questions (those that appears on multiple exams) are used for comparison. Those conversations do NOT explain why essentially the same group of kids, with an exam that has no, more difficult chemistry on it, has resulted in the number of 5’s being cut in half. I’m not a conspiracy theorist in general, but I am a believer that agendas have been set, and that action is now being taken to artificially reduce the number of 5’s on the AP chemistry exam. One person in the room described the new situation as, ‘The May Lottery’ – I feel much the same way. For me there IS an answer, and that it is linked to external bodies believing that English communication skills are perhaps as important as chemistry – that makes ZERO sense to me on a CHEMISTRY exam. I think it stinks.

Here are a few highlights from Roger Kugel on the scoring setting process, and on the decisions about which schools get alternative exams.

  • On score setting (i.e., the process that determines the cut-offs for each grade boundary); ‘I don’t have answers’. ‘Some things I understand, some things I don’t’!
  • He then described his role at the score setting as that of being present simply to, ‘say OK’, and that, “The statisticians are the stars of the show’.
  • When asked how decisions are made regarding which schools get the alternative exam he said; ‘I don’t know how the algorithm works’.
  • On the College Board explaining (or not), the way that score boundaries were determined, Kugel said; “Communication has been poor”

All in all I felt a bit sorry for him – really I did. He looked a little tired of questions that he clearly has no answers for. He is clearly almost completely out of the loop when it comes to score setting (something that I already knew), but how can this be right? How can the single most important person in the country in terms of the CHEMISTRY of AP chemistry exam, have such a non-existent role in score setting? It makes no sense.

There then was the (what turned out to be bizarre), ‘Letter of Explanation’ saga. It’s just another example of the right hand not knowing what the left hand is doing at The College Board.

Paul Price announced that Trevor Packer had written a letter that went some way to outline why the massive grade re-distribution and he suggested that we contact Serena Magrogan to get a copy. It then turns out that Trevor had not written such a letter, rather that Serena had, but it is not supposed to be posted in the public domain. Why not?? We ALL need to see this letter regardless of whether we are getting heat from stakeholders about the new scores. I’ve seen it, wanted to post it here and also had my links to it removed from the AP forum. Why? It largely says nothing of substance but the question is, why the bizzaro secrecy? We ALL need to have this information in the public domain. The College Board just gets odder and odder.

The bottom line is this. If I could drop AP chemistry from my school’s curriculum, I would. I believe that I can achieve two things, independently of the College Board.

One, I can deliver a higher quality advanced chemistry course than the AP chemistry curriculum now requires and in the process move away from an emphasis on NON-chemistry skills – this I feel is REALLY important to chemistry education going forward.

Two, I would not have to put up with the frankly increasingly odd behavior of The College Board. From the audit, to the curriculum changes re: oxidizing and reducing agents/Lewis acids and bases/phase diagrams etc., to the move away from chemistry skills, to the weird and secretive behavior regarding score setting, it’s all getting a little too peculiar for this AP chemistry teacher.


  1. Paul Cohen

    Jamie Benigna exists! I have met him, and have exchanged emails with him. Other than on that point, I agree with everything Adrian writes here!

    I would also like to know what idiot made the “Tabloid” crack. Please email me privately…

  2. Julia Winter

    Yes, Jamie does exist. He teaches right down the street.

    AND, I have been on a mission (some at my school call a soapbox) to cut ties with the College Board with regards to chemistry. The ACS has a great program with the Chemistry Olympiad exams, it could be modified to serve as an alternative to the College Board’s AP exam. Just an idea…

    Another option is to remove AP from the title of the course, call it ‘College Chemistry’ and let the students decide if it’s worth the $80+ to take the exam. For many, especially those going to UMichigan, it matters not. They’ll take a placement test anyway and move into organic chemistry. (Yes, I know they don’t know that in their junior year.) It’s time to start breaking this monopoly.

    So, in complete agreement with Adrian and Paul.

  3. AP Chem Teacher

    I have to post anonymously for fear of repercussions at work.

    For years, you have been on point, and you have been one of the few courageous voices to take on the generally nameless, faceless, unaccountable monopoly that is the Collegeboard. The energy that you have spent in this battle against incompetence and mis-education is a marvel.

    I have been waiting for the ultimate punchline, which is the final recourse of those who see that their voices will be ignored or vilified; that punchline is, “If I could drop AP chemistry from my school’s curriculum, I would”.

    It really is time to walk away from this disaster, especially since there is an army of apologists and sycophants who will ensure that more of the same counterproductive changes will occur. Amazingly, as bad as the changes have been in AP Chem, they pale in comparison to the debacle that is now AP Physics 1 and AP Physics 2 (formerly AP Physics B, algebra-trig college level physics) – check out what has happened to the exam (now for advanced English majors) and its scoring (plummeted from about 16% fives to 4% fives!!, 66% of the test-takers got a two or one!).

    There are alternatives. Schools can develop a course with a nearby college to teach the actual college chem curriculum. For example, New York has SUPA, Syracuse University Project Advance, which carefully trains and vets its adjunct HS teachers, and the students take the actual Syracuse University chem exams. That this is not the standard in NY is terrible for the “advanced” students. There is too much money in the CB monopoly and the linked HS rankings published in TABLOIDS like US News and World Report (somehow, that’s the sole arbiter of HS quality!).

    I’m walking away from the CB. I will not utter the “AP” moniker, other than to follow it with pure vitriol.

    The sycophants of CB will have some cheap soundbite: “only a coward walks away from a battle”, but that is just more irrational drivel. Quitting can be as great if not a greater virtue than fighting a Sisyphys-like battle with no chance of success.

    I’m sure I speak for legions when I say how grateful we are for your tireless efforts, reasoned opinions, and philosophy; we appreciate your tremendous efforts, and wish you continued success in publishing and teaching.

    • Adrian

      Thanks for your kind words – they mean a LOT to me.

  4. Ron Brandt

    I participated in today’s webinar with Roger Kugel. My recollection of your math comments are that a maximum of 1 point is at risk each for sig digits and computational mistakes within a question. Roger confirmed the +/- 1 allowance on the significant digits

    The setting of score bands really bothered me. Based on 10 MC questions embedded in last year’s test, it was determined that this year’s population was less qualified. This then pre-determined fewer 4 and 5’s, regardless of how the population did on the rest of the test. I don’t understand this logic at all. This also means that the cut offs for each score could be significantly different from one year to the next. They were not willing to release the cut off scores for 2015. Merely to state that 5’s were reduced to 8.4% of the population. This was set by the CB statisticians.

    It is no longer about mastery of science. What if all students did well on the test? Will they skewer the scores so only 10% or less get a 5?

  5. AP reader

    I graded the free response question this year on the international exam that checked for significant figures. There were at least two different parts of the question that involved math. One was addition/subtraction, the other was multiplication/division. On the addition part, the Sig figs had to be perfect. You couldn’t be off by even one digit. On the multiplication part, the answer could be off by 1 Sig fig. Interesting.

    • Adrian

      Of good grief – now it’s basically a free-for-all with absolutely no consistency!

  6. Phil Palko

    Fewer fives and fours is an easy way to appease colleges and universities that want the tuition money instead of granting credit for an exam. The college in high school movement still provides revenue to colleges because students pay tuition but is a much easier route to credit for students. I totally agree that the college board is very very poor at communicating just about anything to even those that “work” for them. Wish I could have been at the meetings. Must have been “entertaining” to say the least.

  7. Frank Gasparro

    Adrian – I am in 100% agreement with you!

  8. Frank Gasparro

    Adrian, I am in 100% agreement!



Submit a Comment

Your email address will not be published. Required fields are marked *