New SAT – How Hard is It?

blogging in hawaii
Yup, I wrote this blog while vacationing in Hawai’i

Earlier this August, the College Board released raw score to scaled score conversion charts for the redesigned SAT, giving the world the final piece of the puzzle to the structure, format, and scoring of this, the newest version of the SAT. With this last nugget of information, we in the test prep world can begin to comprehensively understand how (whether) the redesign will impact student performance. If you, like me, have been following the trail of breadcrumbs released by the College Board since the March 5th of 2014 announcement, finally having a conversion chart is like opening your last gift on Christmas morning (and yeah I mean both that feeling of disappointment when you get a sweater or socks and that feeling of excitement when you get your brand new Atari).

So with the release of the conversion chart (ostensibly after exhaustive beta testing, research, calibrating, and equating by the College Board), we finally have official word on how raw scores will be converted to scaled scores and how many questions you’ll need to answer correctly in order to get the score you want. Let’s look at what we know so far and how the new SAT and old SAT compare. Specifically, let’s investigate how “difficult” it is to get a 500 math score on either test. Why 500? Because 500 is the approximate median score on the current SAT and very likely the median score on the redesigned SAT. Additionally 500 is the much ballyhooed college readiness benchmark which College Board seems hell bent on trying to convince us there is causation with success in college. Anyway let’s dive right in.

 

New SAT

To get a 500 on the redesigned SAT a test-taker will have to achieve a raw score of 24 out of a maximum possible 58 raw score points. Since the new SAT uses a simple scoring system awarding 1 raw point for each question correctly answer and with no wrong answer penalty, getting a raw score of 24 simply means answering 24 questions correctly (or answering fewer than 24 and guessing the remaining questions correctly, but in this comparison we won’t factor in random guessing and luck). So on the redesigned test to get a 500, the likely median score, a test-taker will have to answer correctly approximately 41% of the questions.

Current SAT

Conversely on the currently, to get a 500 a test-taker has to achieve a raw score of 25 (out of the maximum possible raw score of 54). This means that a test-taker will have to a get between 46% and 57% of the questions correct. Because the current SAT takes off a quarter point for each incorrect response the scoring is more complex and there are 23 different ways a test-taker could get a 25.

All 23 ways to get a 500 on the current SAT

The verdict is…

When you take into account the greater accuracy level required and the complexity of the scoring system, getting an average score on the current form of the SAT seems to be harder than it will be on the redesigned SAT. So when we look at the scoring system one has to say that the new SAT is going to make it easier for a student to get a score that indicates “college readiness.” Now, if only the content is easier on the redesigned test as well… but that’s another vacation’s writing entirely.

Share

T3$T P@#P part 2: the illegitimate stepchild of education

When most educators and parents hear the words “test prep” you immediately see the corners of their mouths curl up in a snarl of disgust, disdain, derision, and other d-words. What is “test prep” and why does it inspire such misplaced ire and disdain from students, parents, and educators

The hostility that many teachers and educators show toward the time they spend “preparing students for tests” and the rhetoric surrounding that characterization surprises and befuddles me.

The disdain for that many educators show for test prep is confusing to me because while they will rant for hours about the time wasted on “test prep,” the people who can clearly articulate what test

This reaction always befuddles me as they think of hours of wasted time as for children drill in the nuances of filling in bubbles on a Scantron form. As a 20 year vet of the test prep industry, I’m torn about the validity of this assessment and would like to weigh in on this topic and perhaps add some clarity and color to a discussion that is far from black and white.

The key issue with this reaction is that it overlooks the fundamental truths of testing and it disassociates “regular” classroom learning from whats tested All instruction is test preparation. When a child learns something new he is

Share

Did Khan Pull Out His Disruptor?

Over the last few days, I’ve been texted, tweeted, gchatted, emailed and called about the release of College Board’s/Khan Academy’s SAT prep resources. I’ve been forwarded article, after article, after article, about the playing field leveling that College Board is touting its partnership with Khan Academy will bring.  I’ve been asked for my opinion and thoughts on Khan’s resources and the implications for my job and industry. So here it is, my unfiltered (mostly) thoughts on Khan Academy “Official SAT Practice.”

No self-respecting child of the 80’s and sci-fi fan should ever speak of Khan without a passing reference to Star Trek.

 

The partnering of Khan Academy and College Board is certainly a significant step for low income students. The key benefits of Khan’s SAT prep are:

  1. Greater opportunity and access for free – This relationship creates the opportunity for low income students to have reliable free practice, which until now has not been easily found. Students, parents, teachers, and counselors now know immediately where they can send a student to find practice tools for the SAT.
  1. High quality practice questions – Khan’s practice questions will be high quality because their relationship with the College Board will give KA a resource to verify that accuracy and appropriateness of their questions. Prior to this relationship the validity of free online SAT practice materials was questionable at best. (I’ve seen some terrible practice SAT materials both in stores and online, in fact most of the free SAT resources online are terrible.)
  1. Ease of use and access – Setting up and using KA’s site is relatively painless and in this day and age most students have probably already accessed it at some point. This means by adding SAT practice tools, they are simply improving an already useful tool. This is great for students.
  1. Integration with College Board results - In the fall there are plans to integrate further with College Board test results. PSAT and SAT test-takers will be able to add their test results to their KA accounts and get analysis and feedback. This again is great since it allows a one-stop shopping for information and analysis.
  1. Boys and Girls Clubs of America - The most interesting and least clear part of the College Board’s venture into test prep is the partnership not with Khan Academy but with the Boys and Girls Club. Reports are spotty but have indicated everything from College Board providing support setting up computer labs to College Board putting tutors in the Boys and Girls Clubs to provide actual teaching. If there is large scale free instruction supported by the College Board that will be truly interesting and helpful for low income students.
  1. Ongoing improvement - It seems that KA has an active team of professionals working to improve the product. This is amazing and bodes well because I’m anticipating that the College Board hasn’t yet finished tinkering with the SAT (and won’t finish until May 2016).

So while clearly there are great potential benefits to the advent of KA’s SAT tools, it’s also important to be aware of the limitations. Most of the articles I’ve seen about KA have ignored completely or paid scant attention to the potential problems with KA. These articles are touting Khan as the grand equalizer of economically and racially aligned score discrepancies on the SAT.  It’s not. Khan is a tool. It’s a nice, well-designed free tool. And like any tool it will only be as good as those using it. I don’t object to the existence of KA SAT tools (in fact I’m excited by them), my concern is about the impact of touting it as a solution to inequalities. Let’s explore some of the key limitations with KA:

 

  1. Access is not effectiveness – Khan provides OPPORTUNITY to practice. It’s ACCESS to materials. But just because you are provided access and opportunity that does not mean it will be used and if it’s not used then no matter how good it is there will be no effect or leveling of the playing field. One of my concerns about Khan’s effectiveness is about access (since there are lots of studies saying low income students do not have the same internet access or solely access the web via mobile). Living Without Broadband In 2015: How 55 Million Americans Find Jobs, Study, Watch YouTube
  1. Access is not engagement- Another of my concerns is engagement. Logging on to KA periodically when you have a sticky math problem in homework is very different from the consistent practice generally necessary to improve SAT scores. Will students be engaged enough to use the site? Historically, College Board’s prep tools have only been mildly used (I’ve been told by districts that usage of College Board’s SAT Online Course which comes free with most SAT School Day contracts is less than 10% as is use of the My College Quickstart site that is included for every PSAT test taker). Are these videos enough to keep student engaged? Pew Research: U.S. Smartphone Use in 2015
  1. Free access is free for everyone – No matter how great Khan is at providing resources for low income students, high income students will also be able to access those resources to supplement their high priced tutoring programs. Any claims that this tool will minimize the score differentials overlooks that KA tools are available to all regardless of how much the family makes.
  1. Academic preparation is not test preparation - There is a big difference between academic learning and preparing specifically for a test. College Board and Khan provide more of the academic learning (they’ve actually said so). They are focused on more academic approaches. Here is an interesting comparison by Stacey Howe-Lott of Stellar Scores on how a test prep person might do a question vs how an academic might do the same question.
  1. Testing is not the same – All the great practice in the world is generally not sufficient to replicate the experience of take a proctored exam in a crowded room with other kids sniffling and tapping and stressing. Khan will never be able to truly simulate the experience of taking the test.

 

So what’s the upshot of all of this? 

While Khan is shaping up to be a great resource it’s important to not get too enamored with the potential of the shiny new toy. Khan will help those who have had no access to quality free resources, but it will probably not level a playing field that is slanted at every level of education starting in utero and culminating in the workplace.  Additionally Khan has been around for years delivering lessons for everything from algebra to physics and yet somehow the teaching industry has not been disrupted, it’s unlikely this will upend the test prep industry.

 

The keys for taking advantage of Khan will be to start using it early and over a sustained period of time to build academic skills and gain comfort with the material on the SAT. If you don’t make the gains you want or have very little time, then it might make sense to look into actual test preparation options.

 

What’s your thoughts ? What did I miss? What articles or research do I need to read or link to? Please put it in the comments!


A couple of articles that get it right

Share

The no-newer-than-any-other-test-you’ve-never-taken SAT

As we approach the launch of the latest iteration of the SAT, more and more students, counselors, and parents are becoming worried about the “new SAT” and how these changes are going to impact scores and college admissions. These concerns have ranged from reasonable (is there some different that is unlike other tests that we know of) to illogical (will the new SAT test the “new math” and thus unknown and unknowable) to irrational (will colleges not accept the new SAT and taking it keep my child from the college of their dreams).

In hopes alleviating some of the fear and panic around this test I’ve put together this handy list of who should and who shouldn’t be concerned about the upcoming changes to the SAT. And to be clear, I’m not saying that the SAT doesn’t matter and won’t continue to matter, because it will. I’m advocating thinking carefully about how the fact that there are changes to the content of the SAT will have limited impact on most of the people who are concerned about those changes.

Let’s start with those who should ignore the “changes” hype (and just pay attention to preparing well for the test you choose to take… since you still have a choice between SAT and ACT).

  1. Classes of 2018 and onward
    Changes only matter if you have prior context or experience. Having very likely never taken any version of the SAT or PSAT, the classes of 2018 and after should completely ignore all the talk of the changes to the SAT. The version of the SAT they take will be the first and likely only version of the SAT they have ever seen. They should prepare for it much like students have been preparing for the SAT for years. These students should currently be planning their college application timelines and determining when they are going to take exams and focusing on the academic skills they are learning in school that will have lasting value. Any discussion of the old SAT versus the new SAT are completely irrelevant and distracting noise. Students in the class of 2018 and later should no more be worried about the new SAT and its changes than they are about the changes college professors are making to their grading criteria.
  2. Students who are planning to take the redesigned SAT
    Any student who has already determined that they will be taking the redesigned SAT (whether it’s because they are seeking the National Merit Scholarship, schedule doesn’t permit them to take the SAT before the new test launches in March, their school is providing the new SAT for free, etc) should also ignore all the chatter about the changes. Once you’ve reached a decision about which test you are taking it no longer matters that that test is different than it used to be, it only matters what it will be when you take it. The only reason to be aware that it has changed is to ensure that you aren’t practicing with outdated material or information.
  3. Parents
    As a parent your concern shouldn’t be with the newness or changedness of the test but rather with how to make sure your child is best prepared for the test. That preparation might mean choosing to take a different test (since there is the option of SAT or ACT) so as to best display her academic abilities. For most parents your role will remain what it has always been and will continue to be for a long time: the planner, the payer, and the helper. This was true for the 1926 SAT, the 1974 version, as well as the 1994 and 2005 versions (Want to see the history of SAT changes? click here) and will be true when they revise the SAT again in 6 – 10 years. Until then you just need to worry about figuring out how to guide and support your child in taking the test they choose to take, not worrying about the details of the changes.

And now who should be concerned about the fact that the SAT has changed.

  1. College Board
    College Board should and is really concerned about the new SAT. They are concerned that the new test might drive more test-takers to their competition (the ACT) in the short-term and exacerbate the market shift that’s been happening over the last few years. The College Board also has the additional concern that the new test won’t perform as advertised, hoped, or projected. What if, instead of scores being distributed on a nice happy normally distributed bell curve, the results for the redesigned SAT are skewed toward an average score of 650 per section (anyone else remember recentering aka tweeking the curve)? What will happen if the results are completely out of wack with all beta testing? Will colleges still require or accept these scores? Will the changes to the SAT to “align it with the work of schools” (read: align with Common Core) exacerbate the Test Optional movement? The College Board has lots of reasons to be concerned.
  2. ACT
    The team at the Iowa-based newly crowned king of college testing has lots of reasons to worry about the changes to the SAT. Since many of the changes to the SAT address criticisms of the old SAT, ACT is rightly concerned that test-takers may return to ETS’s warm bosom and the comfort of the “known” college admission exam. In redesigning its test, College Board not only address many criticisms of its exam but also took direct shots at perceived areas of superiority that the ACT possessed (and often touted). ACT has lots to be concerned about as it must wonder if this newest SAT (or doppelganger ACT) will retake its position at the top of the psychometric food chain.
  3. Test prep companies and tutors
    Test prep professionals are the ones who should be most concerned about the changes to the SAT. These changes mean that if you’re a test prep person doing your job right you’ve got to learn new stuff and you’ve got to create or find new materials. Changes to the test, while usually causing a spike in business, cause a great deal of work and research for test prep folk. It will take test prep community a while to gather and analyze sample questions and tests in order to learn the ins and out of the new test and the quarks and nuances that can most easily be turned into points. Test prep folks are and should be concerned about the changes, burning the midnight oil, and clicking refresh thousands of times on release days for new samples from College Board.
  4. College counselors of students in the class of 2017
    College counselors should be mildly concerned about the changes to the SAT, especially for the next 12 – 18 months where there are still 3 viable options for college admissions tests. While choice is often good, too many choices just create a big confusing mess, this will probably be true for counselors and students in the class of 2017 until March 2016. Counselors are going to struggle to figure out which of the three college admission tests to recommend, how to interpret the results of the new SAT and PSAT, and how colleges are going to handle comparing students taking different tests. Yes, counselors working with current sophomores will have a bit to be concerned about relative to the changes.
  5. Class of 2017
    Students in the class of 2017 are right to be a little concerned about changes. This class took the old format PSAT, and if they liked it, they may be rightly concerned that the new test might not be as easy for them to get a good score (anytime there is change, how an individual student will handle that change is nigh-impossible to predict). The introduction of the new test will create confusion and thus concern for students. These students who’ve taken the PSAT formatted after the current SAT had some understanding of what they were soon to face, however the introduction of the new SAT could make that 2.5 hour preview entirely moot.

To make a long blog short, not many students or their families should be worried about the newness or changedness of the test until they determine that it actually impacts them. If you’ve determined that you’re going to take the new SAT than its not the newness or changedness of the test that matters so much as making sure you properly prepare (which means ensuring that you have up-to-date practice materials and resources. Unsurprisingly these same preparation concerns have been true since the invention of testing and thus the real concern is not that changes but proper preparation.

Share

The Dire Consequences of Teaching Ambiguity

"test" of dubious veracityRecently I came across this image on the Book of Faces (the book that makes you learn surprising and frightening things about people you thought you knew and now might not want to know. But I digress.) and number of people who seem to miss core issues that this image brings to a head surprises me. While for many this image shows a funny creative student who probably didn’t study and tried to get away with it; for me this test is an example of one of the reasons that many students struggle on admissions tests (and other standardized tests). It is also indicative of how enforced conformation to societal “norms” stymies our most creative thinkers. Yup, for me, this test is an indictment of not the student but the educational system that doesn’t properly train teachers or students. It’s an indictment of the refusal of many of those in power to look inward and acknowledge their own errors and flaws.

And for many students, especially our most vulnerable, this is indicative of the type of interaction that hurts their educational progress, stymies their growth, and ill-prepares them for future tests and future work. But don’t take my word for it, let’s breakdown what’s going on in this image.

Continue reading The Dire Consequences of Teaching Ambiguity

Share

The Questions I Get Asked

Recently, I was invited by a small group of mothers to a home to discuss the college admissions process and standardized testing. These moms are well to do and college educated. They are also mostly African-American. Among the many questions and topics of discussion were the questions listed below. I’m just going to post them without commentary and allow you to draw from it what you will.

 

-How much do test scores really matter to colleges and universities?
-How should I prepare my child for the college admissions tests?
-In general, what are some strategies and support that position students of color for test taking success?
– Do students of color spend as much time preparing for standardized tests?

– Do you perceive students of color to feel as confident about taking the tests? Will the new SAT be helpful to test results in general?

Share

A Tale of Two College Processes

With close of the “college application season,” I’m once again reminded of the stark differences in the college application process for students from low income families and that for the 1%. With every blog or article I read, I’m continually reminded how divided America is in every way. Divided informationally. Divided experiencially. Divided economically. Divided racially. Divided educationally. Just divided. This division is no small matter, it’s a chasm that starts with access to pre-k programming, blossoms in primary school, matures throughout higher education, and culminates in the workplace. The college admissions process is not immune to these conditions and concerns (despite its theoretical claims to a meritocracy). In fact, the college application and admission process may be the very nexus of these myriad issues and bring to a head the years-long growth of inequity between the haves and have-nots. But since I can’t address, cure, or understand all the ills of society, I’ll just explore differences between the college preparation and application process for the haves and have-nots (and yes I’ll ignore America’s shrinking middle class).

Continue reading A Tale of Two College Processes

Share

The Perils of Calculator Permission

As the College Board gears up to launch the revised SAT in March of 2016, one of the changes coming is a seemingly minor revision of the rules for the Math sections. This revision will change the 20 year old policy that has allowed the indiscriminate the use of calculators on math sections and may have a huge impact on test-takers. The current SAT has 3 scored math sections and in each section test-takers are allowed to use a calculator, or not, as they see fit. The current test makes no distinction (either implicit or explicit) among the math sections about the necessity or appropriateness of calculator usage. Well Interestingly, when the revised SAT launches it will have 2 scored math sections and in one of the two sections calculator use will be forbidden. While the College Board seems to be soft-selling this as if it will be no major change, I’m not certain at all that the impact of this change in procedure won’t have a deleterious impact. Having worked with teens in test preparation for more than 20 years, this distinction has me worried about unintended consequences. What immediately pops to mind are the following questions:

  • Will the inclusion of a “calculator permitted” section translate to this calculator dependent generation as “calculator necessary”?
  • Will the mention of calculator permitted cause additional stress for students who do not have a calculator or cannot afford a “good” calculator?
  • Is the college board assuming that students will all be made aware of the fine distinction between permitted and necessary prior to taking the exam?
  • Will this seemingly small change to the SAT hurt scores of the most vulnerable populations?

Continue reading The Perils of Calculator Permission

Share