Standardized Testing: The Temporal Scan of Education

As the pandemic rages and the issue of testing is discussed in a medical context, I’ve found myself increasingly noticing the parallels between the the ways in which America has discussed and politicized Covid testing and the way in which America has politicized and discussed educational testing. These parallels between medical and educational testing have me wondering if it’s a human or American failing that leads to the being so easily seduced by as Alfred Binet, father of IQ testing, put it “a simple, brutal number, which can have only a deceptive precision.

Continue reading “Standardized Testing: The Temporal Scan of Education”

Don’t Believe the Hype

Brigham tried to warn you in 1936. PE tried to warn you in 1988. Bigham tried to warn you repeatedly since 1990.

But, unfortunately, since the marketing machine of the testing agencies got their grips into the Stanvard Universities the narrative of the SAT providing access has taken root in the American psyche and far too many have bought into the hype about what standardized testing does and doesn’t do. Specifically about what it does and doesn’t do for Black and Hispanic people as a group.

Continue reading “Don’t Believe the Hype”

It’s Time to Fix Standardized Testing

The global pandemic has wreaked havoc and the educational status quo and disrupted not only how students are taught and tested. Schools are radically adjusting how they deliver lessons and testing agencies are revamping how they deliver their assessment. The inability to gather in a room has forced the cancellation of statewide assessments, the shortening of AP exams, and major admissions tests (like the ACTGMATGRE, ISEELSATSAT, and SSAT) to develop online at-home testing options. This is the moment to fix many of the things that are broken about assessments. 

Since their beginnings in the late 1800s, standardized tests have become an oppressive force in U.S. education and have influenced many other areas of society, yet in that same time there have been only marginal changes in the tests themselves. Despite reams of papers and scores of conference presentations, a high score on a 4 or 5 answer-choice aggressively-timed test continues to be treated as the epitome of intellectual demonstration. This has gone so far that, in the midst of a global pandemic, one (intentionally unnamed or linked to) author questioned whether epidemiologists should be heeded above economists, asking, “How smart are they? What are their average GRE scores?”

Continue reading “It’s Time to Fix Standardized Testing”

College, Career, and Cremation Benchmarks

As the new school year begins, I am anxiously awaiting (read: dreading) the forthcoming SAT and ACT annual reports and with them the inevitable exaggerations, hand-wringings, misinterpretations, and statistical paralogisms that will follow. The College Board’s Total Group Reports and ACT’s Condition of College and Career Readiness Reports (or Profile Reports) will not only spark the annual “sky-is-falling because district scores have dropped .005 points” responses but will also likely lead to an uptick in the “SAT/ACT scores show students not ready to succeed in college, career, life, liberty or the pursuit of happiness.”

Continue reading “College, Career, and Cremation Benchmarks”

Are Changes Coming to College Admissions?

We know it doesn’t feel like it, but it’s been less than 6 months since the Justice Department announced the indictments resulting from Operation Varsity Blues, so it’s no surprise that few universities have announced any substantial policy changes in their admissions procedures. If most big institutions move slowly, universities look at them and wonder, “What’s the big rush there?”  There are still committees to be convened in order to create sub-committees that can issue memos that can be circulated in order to be approved as official reports by committees who can then move items forward for approval by the faculty and/or Board of Trustees. In other words, don’t expect big changes in how colleges admit students anytime soon.

Continue reading “Are Changes Coming to College Admissions?”

Stanford shows they’re not so smart after all

Making the rounds in the college world this week is the story of Stanford University’s “demand”* that college applicant Malala Yousafzai take the SAT. Correction, that’s Noble Laureate, educational activist, assassination attempt survivor, and still “kicking ass and taking names” while advocating for education for women, 18-year-old Malala Yousafzai. Stanford University, in their apparent quest for additional bad press, has let the story of Ms Yousafzai’s desire to apply to the college become one of a “demand” for a test that many see as worthless and not indicative of any of the true characteristics of college bound students. It’s stunning to me that this university, with what I assume is a million dollar team of PR professionals, would let this potentially huge PR win become another example of the evil that colleges do in their quest for rankings and their love of test scores.

Continue reading “Stanford shows they’re not so smart after all”

Did Khan Pull Out His Disruptor?

Pew Research: U.S. Smartphone Use in 2015

Over the last few days, I’ve been texted, tweeted, gchatted, emailed and called about the release of College Board’s/Khan Academy’s SAT prep resources. I’ve been forwarded article, after article, after article, about the playing field leveling that College Board is touting its partnership with Khan Academy will bring.  I’ve been asked for my opinion and thoughts on Khan’s resources and the implications for my job and industry. So here it is, my unfiltered (mostly) thoughts on Khan Academy “Official SAT Practice.”

No self-respecting child of the 80’s and sci-fi fan should ever speak of Khan without a passing reference to Star Trek.

The partnering of Khan Academy and College Board is certainly a significant step for low income students. The key benefits of Khan’s SAT prep are:

  1. Greater opportunity and access for free – This relationship creates the opportunity for low income students to have reliable free practice, which until now has not been easily found. Students, parents, teachers, and counselors now know immediately where they can send a student to find practice tools for the SAT.
  2. High quality practice questions – Khan’s practice questions will be high quality because their relationship with the College Board will give KA a resource to verify that accuracy and appropriateness of their questions. Prior to this relationship the validity of free online SAT practice materials was questionable at best. (I’ve seen some terrible practice SAT materials both in stores and online, in fact most of the free SAT resources online are terrible.)
  3. Ease of use and access – Setting up and using KA’s site is relatively painless and in this day and age most students have probably already accessed it at some point. This means by adding SAT practice tools, they are simply improving an already useful tool. This is great for students.
  4. Integration with College Board results – In the fall there are plans to integrate further with College Board test results. PSAT and SAT test-takers will be able to add their test results to their KA accounts and get analysis and feedback. This again is great since it allows a one-stop shopping for information and analysis.
  5. Boys and Girls Clubs of America – The most interesting and least clear part of the College Board’s venture into test prep is the partnership not with Khan Academy but with the Boys and Girls Club. Reports are spotty but have indicated everything from College Board providing support setting up computer labs to College Board putting tutors in the Boys and Girls Clubs to provide actual teaching. If there is large scale free instruction supported by the College Board that will be truly interesting and helpful for low income students.
  6. Ongoing improvement – It seems that KA has an active team of professionals working to improve the product. This is amazing and bodes well because I’m anticipating that the College Board hasn’t yet finished tinkering with the SAT (and won’t finish until May 2016).

So while clearly there are great potential benefits to the advent of KA’s SAT tools, it’s also important to be aware of the limitations. Most of the articles I’ve seen about KA have ignored completely or paid scant attention to the potential problems with KA. These articles are touting Khan as the grand equalizer of economically and racially aligned score discrepancies on the SAT.  It’s not. Khan is a tool. It’s a nice, well-designed free tool. And like any tool it will only be as good as those using it. I don’t object to the existence of KA SAT tools (in fact I’m excited by them), my concern is about the impact of touting it as a solution to inequalities. Let’s explore some of the key limitations with KA:

  1. Access is not effectiveness – Khan provides OPPORTUNITY to practice. It’s ACCESS to materials. But just because you are provided access and opportunity that does not mean it will be used and if it’s not used then no matter how good it is there will be no effect or leveling of the playing field. One of my concerns about Khan’s effectiveness is about access (since there are lots of studies saying low income students do not have the same internet access or solely access the web via mobile). 
  2. Access is not engagement– Another of my concerns is engagement. Logging on to KA periodically when you have a sticky math problem in homework is very different from the consistent practice generally necessary to improve SAT scores. Will students be engaged enough to use the site? Historically, College Board’s prep tools have only been mildly used (I’ve been told by districts that usage of College Board’s SAT Online Course which comes free with most SAT School Day contracts is less than 10% as is use of the My College Quickstart site that is included for every PSAT test taker). Are these videos enough to keep student engaged? 
  3. Free access is free for everyone – No matter how great Khan is at providing resources for low income students, high income students will also be able to access those resources to supplement their high priced tutoring programs. Any claims that this tool will minimize the score differentials overlooks that KA tools are available to all regardless of how much the family makes.
  4. Academic preparation is not test preparation – There is a big difference between academic learning and preparing specifically for a test. College Board and Khan provide more of the academic learning (they’ve actually said so). They are focused on more academic approaches. Here is an interesting comparison by Stacey Howe-Lott of Stellar Scores on how a test prep person might do a question vs how an academic might do the same question.
  5. Testing is not the same – All the great practice in the world is generally not sufficient to replicate the experience of take a proctored exam in a crowded room with other kids sniffling and tapping and stressing. Khan will never be able to truly simulate the experience of taking the test.

So what’s the upshot of all of this? 

While Khan is shaping up to be a great resource it’s important to not get too enamored with the potential of the shiny new toy. Khan will help those who have had no access to quality free resources, but it will probably not level a playing field that is slanted at every level of education starting in utero and culminating in the workplace.  Additionally Khan has been around for years delivering lessons for everything from algebra to physics and yet somehow the teaching industry has not been disrupted, it’s unlikely this will upend the test prep industry.

The keys for taking advantage of Khan will be to start using it early and over a sustained period of time to build academic skills and gain comfort with the material on the SAT. If you don’t make the gains you want or have very little time, then it might make sense to look into actual test preparation options.

What’s your thoughts ? What did I miss? What articles or research do I need to read or link to? Please put it in the comments!


A couple of articles that get it right