The global pandemic has wreaked havoc and the educational status quo and disrupted not only how students are taught and tested. Schools are radically adjusting how they deliver lessons and testing agencies are revamping how they deliver their assessment. The inability to gather in a room has forced the cancellation of statewide assessments, the shortening of AP exams, and major admissions tests (like the ACT, GMAT, GRE, ISEE, LSAT, SAT, and SSAT) to develop online at-home testing options. This is the moment to fix many of the things that are broken about assessments.
Since their beginnings in the late 1800s, standardized tests have become an oppressive force in U.S. education and have influenced manyother areas of society, yet in that same time there have been only marginal changes in the tests themselves. Despite reams of papers and scores of conference presentations, a high score on a 4 or 5 answer-choice aggressively-timed test continues to be treated as the epitome of intellectual demonstration. This has gone so far that, in the midst of a global pandemic, one (intentionally unnamed or linked to) author questioned whether epidemiologists should be heeded above economists, asking, “How smart are they? What are their average GRE scores?”
As the new school year begins, I am anxiously awaiting (read: dreading) the forthcoming SAT and ACT annual reports and with them the inevitable exaggerations, hand-wringings, misinterpretations, and statistical paralogisms that will follow. The College Board’s Total Group Reports and ACT’s Condition of College and Career Readiness Reports (or Profile Reports) will not only spark the annual “sky-is-falling because district scores have dropped .005 points” responses but will also likely lead to an uptick in the “SAT/ACT scores show students not ready to succeed in college, career, life, liberty or the pursuit of happiness.”
We know it doesn’t feel like it, but it’s been less than 6 months since the Justice Department announced the indictments resulting from Operation Varsity Blues, so it’s no surprise that few universities have announced any substantial policy changes in their admissions procedures. If most big institutions move slowly, universities look at them and wonder, “What’s the big rush there?” There are still committees to be convened in order to create sub-committees that can issue memos that can be circulated in order to be approved as official reports by committees who can then move items forward for approval by the faculty and/or Board of Trustees. In other words, don’t expect big changes in how colleges admit students anytime soon.
Recently Marten Roorda, CEO of ACT reminded us that throwing away the thermometer won’t get rid of a fever and he’s 100% right. Of course, no doctor in the world thinks that tossing the thermometer will cure an identified ailment. Mr. Roorda’s analogy in defense of the ACT (and attacking the test optional movement) was really subtle and I think many will miss all the ways in which the analogy works. Since I’m a fan of a good analogy (except when they are put on a test) I’m going to help make sure everyone understands why this is an amazing analogy.
Since the 1980s, Educational Testing Service (ETS), which dominated educational admission testing from 1940 – 1980, has been hemorrhaging product lines. In its heyday (SAT word) ETS was the Sauron to US education’s Middle Earth, providing admissions tests for the vast majority of professional certification programs and higher ed admissions. Their services ranged from teacher certification exams to the SAT, GRE, GMAT, MCAT, and LSAT. In the last decade or so, ETS business strategy has changed and the organization has begun to aggressively market their most popular remaining assessment product, the Graduate Record Exam (commonly known by its initialism – GRE), as “the One Test to Assess Them All.” This strategic market grab, while an interesting business strategy, raises significant questions about all admission tests. Specifically, the expansion of the GRE into fields beyond its design should force responsible test users to reevaluate long-held assumptions about what information is being gained by requiring the GRE (and all its brethren) and at what cost.
Recently, I had the honor of delivering the Keynote address at the 2019 IACAC Conference. I thought it would be cool to share some of the highlights of that talk and some of the reference resources here.
The core message of my talk was that the American educational system is not a meritocracy as most people think of the word, but instead is a meritocracy in the original sense of the word as it was satirically coined by Micheal Young in 1958.
Since the inception of the SAT in 1926, the admission world has debated (1976, 2001, 2008, 2015, 2018, 2019) the impact of and validity of the SAT (and later the ACT, CLT, CCTST, etc) on the pool of applicants and enrolled students at a university. Recently, more and more colleges have been asking themselves should they diminish the role of testing in their admission process and declare a test optional admissions policy.
As anyone who works with low income, first generation or underrepresented students could tell you, the vast majority of these students lack not only the funds to compete with the 1% but also the “social capital” that greases the wheels of higher education access. Networks of chatty parents sharing new discoveries about demonstrated interest, hooks, gap years, PPY, ED/EDII/EA/EAII/REA, supplemental essays, recommended (not really) tests, super-scoring, super-duper-scoring, test optional/flexible, and a host of other insider secrets help the most informed more easily navigate an increasingly complex system.
Unlike their rich counterparts, low-income parents do not have the opportunities to learn the intricacies of admissions to feeder middle schools, selective high schools, and highly selective colleges and graduate schools from a social circle that includes deans of admissions, board members, CEOs, and college presidents. Instead, first generation parents rely on objective sources (catalogues and webpages), their next door neighbors (who are likely also first generation, low income, and underrepresented), their school counselors, and their friends to pool the little information they have in hopes of hitting the access lotto and gaining a spot at a selective institution.
Making the rounds in the college world this week is the story of Stanford University’s “demand”* that college applicant Malala Yousafzai take the SAT. Correction, that’s Noble Laureate, educational activist, assassination attempt survivor, and still “kicking ass and taking names” while advocating for education for women, 18-year-old Malala Yousafzai. Stanford University, in their apparent quest for additional bad press, has let the story of Ms Yousafzai’s desire to apply to the college become one of a “demand” for a test that many see as worthless and not indicative of any of the true characteristics of college bound students. It’s stunning to me that this university, with what I assume is a million dollar team of PR professionals, would let this potentially huge PR win become another example of the evil that colleges do in their quest for rankings and their love of test scores.
Over the last few days, I’ve been texted, tweeted, gchatted, emailed and called about the release of College Board’s/Khan Academy’s SAT prep resources. I’ve been forwarded article, after article, after article, about the playing field leveling that College Board is touting its partnership with Khan Academy will bring. I’ve been asked for my opinion and thoughts on Khan’s resources and the implications for my job and industry. So here it is, my unfiltered (mostly) thoughts on Khan Academy “Official SAT Practice.”
No self-respecting child of the 80’s and sci-fi fan should ever speak of Khan without a passing reference to Star Trek.
The partnering of Khan Academy and College Board is certainly a significant step for low income students. The key benefits of Khan’s SAT prep are:
Greater opportunityand access for free – This relationship creates the opportunity for low income students to have reliable free practice, which until now has not been easily found. Students, parents, teachers, and counselors now know immediately where they can send a student to find practice tools for the SAT.
High quality practice questions – Khan’s practice questions will be high quality because their relationship with the College Board will give KA a resource to verify that accuracy and appropriateness of their questions. Prior to this relationship the validity of free online SAT practice materials was questionable at best. (I’ve seen some terrible practice SAT materials both in stores and online, in fact most of the free SAT resources online are terrible.)
Ease of use and access – Setting up and using KA’s site is relatively painless and in this day and age most students have probably already accessed it at some point. This means by adding SAT practice tools, they are simply improving an already useful tool. This is great for students.
Integration with College Board results – In the fall there are plans to integrate further with College Board test results. PSAT and SAT test-takers will be able to add their test results to their KA accounts and get analysis and feedback. This again is great since it allows a one-stop shopping for information and analysis.
Boys and Girls Clubs of America – The most interesting and least clear part of the College Board’s venture into test prep is the partnership not with Khan Academy but with the Boys and Girls Club. Reports are spotty but have indicated everything from College Board providing support setting up computer labs to College Board putting tutors in the Boys and Girls Clubs to provide actual teaching. If there is large scale free instruction supported by the College Board that will be truly interesting and helpful for low income students.
Ongoing improvement – It seems that KA has an active team of professionals working to improve the product. This is amazing and bodes well because I’m anticipating that the College Board hasn’t yet finished tinkering with the SAT (and won’t finish until May 2016).
So while clearly there are great potential benefits to the advent of KA’s SAT tools, it’s also important to be aware of the limitations. Most of the articles I’ve seen about KA have ignored completely or paid scant attention to the potential problems with KA. These articles are touting Khan as the grand equalizer of economically and racially aligned score discrepancies on the SAT. It’s not. Khan is a tool. It’s a nice, well-designed free tool. And like any tool it will only be as good as those using it. I don’t object to the existence of KA SAT tools (in fact I’m excited by them), my concern is about the impact of touting it as a solution to inequalities. Let’s explore some of the key limitations with KA:
Access is not effectiveness – Khan provides OPPORTUNITY to practice. It’s ACCESS to materials. But just because you are provided access and opportunity that does not mean it will be used and if it’s not used then no matter how good it is there will be no effect or leveling of the playing field. One of my concerns about Khan’s effectiveness is about access (since there are lots of studies saying low income students do not have the same internet access or solely access the web via mobile).
Access is not engagement– Another of my concerns is engagement. Logging on to KA periodically when you have a sticky math problem in homework is very different from the consistent practice generally necessary to improve SAT scores. Will students be engaged enough to use the site? Historically, College Board’s prep tools have only been mildly used (I’ve been told by districts that usage of College Board’s SAT Online Course which comes free with most SAT School Day contracts is less than 10% as is use of the My College Quickstart site that is included for every PSAT test taker). Are these videos enough to keep student engaged?
Free access is free for everyone – No matter how great Khan is at providing resources for low income students, high income students will also be able to access those resources to supplement their high priced tutoring programs. Any claims that this tool will minimize the score differentials overlooks that KA tools are available to all regardless of how much the family makes.
Academic preparation is not test preparation – There is a big difference between academic learning and preparing specifically for a test. College Board and Khan provide more of the academic learning (they’ve actually said so). They are focused on more academic approaches. Here is an interesting comparison by Stacey Howe-Lott of Stellar Scores on how a test prep person might do a question vs how an academic might do the same question.
Testing is not the same – All the great practice in the world is generally not sufficient to replicate the experience of take a proctored exam in a crowded room with other kids sniffling and tapping and stressing. Khan will never be able to truly simulate the experience of taking the test.
So what’s the upshot of all of this?
While Khan is shaping up to be a great resource it’s important to not get too enamored with the potential of the shiny new toy. Khan will help those who have had no access to quality free resources, but it will probably not level a playing field that is slanted at every level of education starting in utero and culminating in the workplace. Additionally Khan has been around for years delivering lessons for everything from algebra to physics and yet somehow the teaching industry has not been disrupted, it’s unlikely this will upend the test prep industry.
The keys for taking advantage of Khan will be to start using it early and over a sustained period of time to build academic skills and gain comfort with the material on the SAT. If you don’t make the gains you want or have very little time, then it might make sense to look into actual test preparation options.
What’s your thoughts ? What did I miss? What articles or research do I need to read or link to? Please put it in the comments!
Bellowings 