For those of you (students and teachers) with (relatively) disappointing AP chemistry scores this year, read on, but after you’ve read this post.
The College Board have decided, that up until 2013, about half of the kids that were scoring 5’s on the exam were not really deserving of such accolades. Sure they were reaching the standard that the CB themselves set, BUT suddenly it became apparent that this was not good enough. Insufficient ‘mastery’ and inadequate ‘depth of explanation’ were being exhibited by those children, and it had to stop! In order to correct this horrible lack of chemistry education, the CB decided that a whole new level of literacy, both in terms of the questions and crucially the answers, was required, and that these new standards (on the 2014 and 2015 exams), would guarantee a whole new crop of brilliant people getting 5’s, and the ‘good-but-not-quite-so-brilliant-ex-5’s‘, would become the new 4’s.
Contrary to what you might think, I have no particular issue with this per se (even though I disagree with the thinking behind it – another, different rant). The CB can do what they like, re-align standards and exams how they choose, and buy into whatever 21C edubabble they pay for from the educational ‘researchers’ that don’t teach AP chemistry to children, all day, everyday in schools across the country. After all, it’s their exam and their business. HOWEVER, what I do object to profoundly, is the fact that we, in many cases, have no idea what these new answer requirements look like. Take one example, the Le Châtelier/non-standard cell debacle. In a nutshell, this is where a Le Châtelier argument for explaining non-standard cell voltages used to be acceptable, then without warning it became unacceptable. The CB has failed miserably in communicating such things properly, and that is why we have seen a catastrophic drop in 5’s at the top end of the grading scale. BTW, please be clear that I am not talking about things that were never acceptable, for example stating trends as reasons, rather I am talking about far more subtle, philosophical changes in answer requirements.
Table leaders and graders agree with me and have been (publicly) extremely critical of the process. My own experience as an (albeit brief) employee of the CB confirms the philosophy, too, and on grader’s observations are given below;
“As a reader, I saw a continuation of the practice of being VERY PARTICULAR about what answer would get credit. In both 2014 and 2015, one discussion section on the overall question that I graded (granted I only graded 3 out of the 14 questions these past 2 years) was interpreted very strictly while the rest were much more broad in acceptable approaches. On my own tests, I have a “challenge” question or two that helps me differentiate between the top students. I suspect that the strict interpretation of answers on one section per question is to help better differentiate between the 4s and 5s…
..I think it is clear that CB wants to reduce the number of 5s (and 4s?) across the board in AP science and so the standards have been raised much higher than before.”
As I have said above, perhaps surprisingly, I have no issue with that per se, but the problem is that we have largely no idea where these new ‘very particular’ answers might be required, or what they might be. I cannot hit a moving (or indeed totally hidden) target.
In one strange way, if you’ve never taught the legacy course (prior to 2014), you have an enormous ADvantage over those of us that used to know what would score a 5.
*This year, I include myself in that ‘disappointed’ category.
I mildly disagree with some of the above. I have been reading AP papers for nine years. There were ALWAYS times where I disagreed how a particular item was graded. I don’t see that we are grading the new exams differently from the way we graded the old ones. (remember the flap over joules per mol rxn? I graded that one my first year!) The grading standards are determined largely by people like me.Had I been assigned to question 1, it is possible that I could have persuaded the leadership group to allow answers that referred to driving the reaction less forward. On my question,(2) some of my suggestions were followed, others not, but there was no agenda imposed upon us.
Notice that the reader you quoted writes “I saw a CONTINUATION of the practice” (emphasis mine) of being very particular…” I really don’t see an important difference in how we are grading now, compared with the legacy exams. However, it is probably statistically significant that a 1 point decision on the new exam has 50% more impact than it did on the old exams. There are now 100 total points, where there used to be 150, or sometimes 160.
The catastrophic drop in fives AND fours, in my view, has little to do with grading standards. It is the result of deliberate statistical manipulation, and it has occurred in ALL of the sciences; it is even worse in physics and biology.The number of 5’s dropped because the non-scientists at the College Board wanted them to drop. Anything that they try to sell you about “deeper understanding” is just hokum. The 2012 AP exam required a score of about 73% of the points for a 5. Had it been 75 or 76 %, how much lower would the % of fives, ( 15 % that year) have been?
There is some evidence that the % for a 5 WAS set somewhere above 75% this year. I don’t know what they will reveal to us about that.
Paul
Interesting. So, what you are saying is that generally speaking, we are simply talking about a increased % being required for a 5? That’s not the message that I’m getting from Paul P.! Maybe I misunderstand him.
So, if this is all about simply reducing the number of kids with 5’s, what’s the real, underlying reason/agenda for THAT??
I don’t know whether the % for a 5 this year will ever be revealed, but last year we were told that it would have been 75, but 3 points were deducted in response to complaints about time pressure. With no time pressure this time, and a somewhat easier (IMHO) free response section, I would expect about 76. That would be the highest % needed for a 5, since they have (occasionally) listed them.
Paul P. is on the testing committee, but he was not involved in the grading of the main exam (he was in charge of one of the “secret” versions). While, as always, I disagreed with some of the grading schemes, there did not seem to be any change. It’s the same people ultimately making the decisions, and they are very competent, and most appear to have no agenda at all. I don’t think that “Fussier” grading contributed significantly to the decline in scores, if at all. As to the underlying reason? I don’t know, since some of the spokespeople cannot be trusted, but Bernie may be on to something. Fewer and fewer colleges have been granting credit for 4’s and 5’s, so with the course clearly LESS rigorous, perhaps the College Board felt that it needed to demonstrate that the standards were now MORE rigorous.
At ChemEd 2015 last Friday, Roger Kugel suggested that the grade boundaries would NOT be released this year.
Of course. Where we ask for MORE transparency, they give us less. The scores are deliberately manufactured to fit a preconceived notion of what they ought to be. This seems to be true across the AP board. Having no competition at all, the College Board can do whatever it wishes to. And state and local departments of education force schools into compliance with the opinions of this private entity, which is answerable to no one. Were there some other, completing national program with a valid final examination, the College Board would be racing to accommodate us. But they own the monopoly, don’t they?
Well there is IB and the option of dropping AP altogether. I know that both of those have very limited chance of application in most schools and districts, BUT I do think that dual enrollment has a real chance of taking off as a result of these changes.
Interesting finding this blog. Also interesting when I found the free response to the new questions. I took the AP chem and scored 5 a long time ago (2008) and attended Emory University and decided to take ochem. However, I did often observe many friends and acquaintances going through general chemistry. The interesting thing was, since Emory has many pre-medical students, many of such students opted to forfeit their 4s and 5s to retake general chemistry for an “easy A”. Unfortunately, if such students signed up for a reasonably rigorous instructor (normally the ones that teach ever year), most ended up in the B range when their score should meant that they easily earn an A because the material is a repeat, however, what actually could have happened is that the competition is of course more stiff, especially at selective institutions. The fact that a 75% on the AP can result in a 5 is telling because I know to get an A/A- in general chemistry, one would usually need to be scoring in the high 80’s at Emory (and that is only because the homework counts for a little). A 75% being a 5/A suggests that the mean on the exam may be in the 50’s (or lower…I don’t know you guys know more than me). While the exam (especially the free response portion) was not particularly low in rigor.
In fact it was solid: Adrian, I know you critiqued it because of the random apps thrown in and the framing of certain prompts, but I see some value in doing that since many will go on to major in the physical sciences or become pre-med and thus tackle problems where they have to sift through information and exclude the useless. In the case of a pre-med, the MCAT requires this, but is multiple choice. Also, the most challenging college instructor’s exams are NOT straight forward and are known for the same tricks of throwing people off who can’t see the big picture and are easily distracted. Some of the biology and organic chemistry instructors at Emory are notorious for this, and so are the most rigorous gen. chem instructors. They’ll present prompts in context of some application and sometimes it asks the student to do something as small as a calculation and sometimes it asks them to go a lot deeper/do a difficult task. This often results in questions having fairly long passages and/or irrelevant information associated with them and sometimes several figures. Some of it is clearly just to test the students’ test taking skills or endurance/focus and again, see if they can sift through BS and simplify the questions to something they know.
I digress however: In comparison to the most difficult exams the top instructors give in general chemistry, that exam is on par or less difficult in problem types and expectations and the exams I refer to still typically average in the mid-high 60s, much higher than what I would expect to be the AP exam average. Given this, I would have to wonder if a 75% does deserve a 5. The AP is certainly challenging, but it can easily be coached and you are told exactly what will go on it and the problem types do not deviate much from what you’ve done (you for sure know what is coming and how it is coming: Similar tactics are used by college instructors who decides to water down their course/spoonfeed students when they want good evaluations). A difficult college level exam and the instructor that gives it is usually more onerous and has a lower level of predictability, often throwing curveballs (some think its bad they do this, but it has its utility if you are attempting to also challenge the brightest among the students) that may require problem solving techniques not covered or assigned for practice (as in a completely new problem type or angle not explicitly covered in class), but ones that must be essentially derived on the spot in the span of a 75-90 minute exam. Either that or they have familiar prompts formatted in a completely different way. For example, last term I observed an exam of a student (who I tutored) in what is supposedly the most rigorous instructor’s section. One of the rate law questions was very long and was restructured as a word problem with a student running a series of experiments instead of the standard data tables and a series of parts that ask them to use that table to find the order of the reaction and make some calculation involving the progress of the reaction. Instead, the student had to put together their own table. On top of this, the question required students to figure out if it was 1st or 2nd order even though they were never explicitly ask to (the instructor was tricky in phrasing it as a fact of the experimental set-up saying in the passage: “an experiment was run to determine if it was first or second order” meaning the student, after noticing that the result wasn’t given, was supposed to ask the question themselves and use the data to figure it out). All of this to simply figure out the amount of time it takes for the concentration of the start reagent to drop by 97% (how much time to convert your 97% of your original oil into biodiesel? One may argue that that could confuse..I disagree)..I can imagine the shock and confusion after getting used to the data tables and having the whole process broken into several parts for the student when prepping for the exam. That wasn’t the only instance on that exam where a problem type was reformatted. In addition, the multiple choice was turned into time killers by making most of them traditionally formatted equilibria and rate law problems involving calculations. The mean on this caliber exam ended up being 71. So by AP scaling, the average person would have gotten maybe a 4, but in this class, consistent performances in that range will result in a C+ or B-, not the B/B+ anticipated by a 4.
However, things are complicated by the fact that I think the AP is supposed to measure up well to or better than the traditional college general chemistry class taught at a normal level (I think) in which case, I would say that most 5’s would get A’s (or should) even on the old distributions (I have seen gen. chem midterms at many other schools, and bluntly, many are below AP rigor and at some schools, sadly Georgia Tech being one of the them and GSU being another, the midterms and final are only multiple choice). At the same time, the meaning of a 4 or 5 at a school with a more rigorous than standard general chemistry sequence (especially at pre-med heavy schools where such courses function as weeders and are often taught and tested in a manner that takes into account that many taking it have scored 4/5 are taking it and much less or have at least taken AP) is much less obvious. It becomes even greyer when a high caliber (or any) school decides to change its general chemistry curriculum (I actually think Emory is doing this in 2016 and it may not allow anyone to opt out of the 1 semester course that will supposedly replace the 2 semester sequence it has now). Given this, CB is artificially deflating the distribution for the sake of “appearing” more rigorous (unlike their new exams in my opinion) or it may be taking into account the potential results of students who go on to take more rigorous chemistry courses either as a result of their credit or in lieu of it (in the case of forfeiting credit). Despite all that I have said, I suspect it is the former, but the devil’s advocate in me wants to believe it is the latter (the idea that a 5 will general equal an A-/A in any level gen. chem sequence-many people who get strong AP scores are considering these more competitive or some very selective institutions for science). It is clear to me that AP provides an advantage and those who took or scored well will certainly do at least as well as those without it and in most cases better, but the question is how much better? And furthermore, how well does it predict that a student will do extremely well in a more competitive or rigorous environment than when they took the test. I have seen many 4s and 5’s in the past retake gen. chem with an air of overconfidence (perhaps this alone can effect performance) and that can be very dangerous in a competitive classroom.
My experiences are a bit different, but Emory has a very fine reputation, and I don’t doubt that Bernie is reporting accurately. I lectured at Brooklyn College for 15 years, a highly rated college in the City University of New York, and I think closer to the mainstream than Emory. (I believe Emory to be far more competitive – based on limited knowledge). In my lecture sections, students who had gotten 5 on the AP exam ALWAYS got A’s in general chemistry, and 4’s usually did. A student of mine who got a 4 ran straight A in chem at Barnard. (Students at Princeton, Penn and MIT were not even offered college credit, and found that a very high proportion of the students in their chemistry sections had taken AP chem, and gotten 4 or 5.)
Also, remember when comparing a 75 on the AP exam with a similar grade on a college final, that the AP exam is on the whole year’s material. A typical chem 2 final, for example omits most of the material in chem 1.