Since the announced changes to the SAT in March 2014, College Board officials have been on the world tour of high schools and education conferences trying to wow educators with their shiny new toy, the 15th iteration of the SAT. They’ve published exhaustive treatises on the research and specifications behind the changes, hosted dozens of gatherings and yet have provided no real information for the students who will actually take the test. These kids have been left to decode marketing-speak extolling the virtues of a test “more aligned with school work” and “based on a foundation of research.” Newspapers have picked up on the College Board’s talking points and parroted them without providing clarification, further confusing families and adding to the anxiety surrounding an already fraught time. So this leaves little ole me with the herculean task of laying plain that which has been obfuscated. I’ve been trying to work through each of the “8 Key Changes” and translate them into laymen’s terms so that they are more easily digested. Previously, I analyzed “Founding Documents and the Great Global Conversation“. As with that analysis, here I’ll also seek to answer these three key questions:
- What does this really mean?
- What level of impact will this change have for test takers?
- Is this really a change or is it simply a redistribution of the same ole same?
So, let’s do this thing!
Problems Grounded In Real World Context
College Board (CB) has shouted from the rooftops that the SAT will no longer test students on “tricky logic puzzles” but will instead focus on Problems Grounded in Real World Context. But they haven’t done a good job of defining simply and clearly what the hell “Problems Grounded in Real World Context” are and, in contrast, what are those terrible fake (virtual? cyber? Tron? Cybertron?) world problems that were on the old terrible outdated misaligned SAT that CB is working so hard to distance themselves from.
In order to understand what this jargon means, let’s first take a look at what the College Board published (buried?) in their 200+ page test specifications document (TSD).
On page 16 they give us:
“Real-world literacy requires a deep reading, careful analysis, and thorough understanding of a wide variety of sources, including both text and data; real-world mathematics requires sustained chains of reasoning and application. The redesigned SAT showcases problems in which literacy and numeracy unlock insights within rich, authentic contexts.“
They go on to add on page 147:
In response to evidence about essential prerequisites for college and career readiness and success, the redesigned SAT’s Math Test requires students to apply their mathematics knowledge, skills, and understandings in challenging, authentic contexts. Students taking the Math Test will encounter a range of disciplines and will be asked to address real-world problems drawn from science, social studies, and careers and demonstrate a capacity for sustained reasoning over the multiple steps required to answer many of the questions on the exam. In these ways, the Math Test also rewards and incentivizes valuable work in the classroom.
I’m not sure above actually explains anything so much as repeats that College Board thinks that real-world problems (whatever those are) are important to college, career, and success (in this blog I won’t try to tackle what they mean by career and success but that is well worthy of questioning). Let’s try again to understand this with some sample problems:
While providing a little more clarification this problematic juxtaposition seems to show a “logic” question (number 4 in section 6) as compared to an algebra problem that is presented in a context students should be familiar with from school and could potentially be a situation they would have to solve in their adult lives. This all seems reasonable until you analyze it more closely.
First, the juxtaposition is false. One is a formal logic problem and the other is an algebra problem. They are testing different skills. They have different focuses for different careers. They are different. Full stop.
In fact, when you look at that logic question it looks an awful lot like a simple form of an Law School Admission Test game question (download a sample LSAT), if then math logic (often found in 8th grade math class), or a basic logic question (I still have bad dreams about Nietzsche and Kierkegaard from my first year of college). So what’s College Board suggesting by bashing these logic questions? What is the message they are trying to deliver? These logic problems seem likely to have relevance to certain careers, especially law, philosophy, politics, and computer programming. Is the College Board trying to tell us that these careers are no longer relevant? Hopefully, David Coleman will answer that for me one day. Further, when you look at the occurrence of these logic questions on the old SAT you find that they appeared fairly infrequently (yes these conclusions are drawn from some hand-crafted artisanal data drawn from three old SATs: the Free 2014-15, Oct 2013, and Free 2013-14).
So, much like with Historical Documents, this Key Change is addressing a minor “problem” and pretending that its major. Additionally the “solution” was already a significant portion of the old test. So CB has removed the logic problems that represented at best 9% of the test and more commonly less than 5% and replaced it with a problem type that was already roughly a quarter of the exam. Seemingly much ado about nothing.
But let’s move on from their strange bashing of logic skills and try to identify real world math. These 3 problems from the TSD seem to align with the real world math that CB is holding out as the epitome of math testing:
College Board explains the first problem above in the TSD as follows:
[The problem] is based on real-world methods (aerial observations of wintering spots, or synoptic counts) used by the U.S. Fish and Wildlife Service to count manatees, a type of sea mammal. This type of item is an excellent way of connecting linear functions to statistics. In this item, students are not required to model the line of best fit completely, but they are required to decontextualize the item to understand that they must compute the slope of the line of best fit to get the correct answer, 150.
This explanation seems oddly focused on pointing out that this problem has value simply because there is a career that uses the math in this problem. Can we not find some career that uses all math? How is CB distinguishing the ones that are “real” and the ones that aren’t? Also if these 3 problems are samples of the “real” world how many of us live in a world where we are regularly converting rupees to dollars?
Another odd tidbit is that if we look at a few more questions from that same old SAT test from which that solitary logic question is drawn you find the following questions:
So even if we believe that the juxtaposition of the logic problem and the “real world” problem is a worthwhile one, it seems that framing this as a “change” is questionable. All three of these questions, and quite a few more from the old SAT, are just as real world-ish as those offered in the TSD and in the sample redesigned SATs. All this weird positioning and odd comparisons makes one think that the changes to the test are more designed to appear to address the criticisms of the SAT rather than to make substantive changes to the content. These rifts between what real changes were made and how CB has talked about the changes have made the conspiracy theorist among us suspicious of College Board’s motives. The more I look at the changes the more I wonder why CB is going through all these games and disguises and not just admitting what they are doing.
Back to the issue of real world problems, there is a substantive shift in the math on SAT version 15.0 vs SAT version 14.0, though it seems that CB is loathe to (or incapable of?) discuss it publicly. That change is a movement away from not only Geometry questions (which used to be about 30% of the test but now is less than 7%) but also away from “pure math” problems like these (from the Free 2014-15 test):
College Board has seemed to make a concerted effort to include a larger percentage of questions that are presented in terms of a theoretical situation. This change seems consistent with the changes that underpin Common Core math standards (which should be no surprise given that David Coleman was one of the main architects of the Common Core standards). The issue that is most strange in all of this is that when you look at the percent of questions that were “pure math” versus the percent that is now pure math, the difference is so minor that I’m not sure it rises to the level of a key change.
So at the end of the day its seems that the College Board could have done a much better job informing students, parents, and educators about the changes to the SAT math by simply saying “More Word Problems and Less Geometry” and staying away from the jargony marketing speak that obscures the actual change that was made to the test.