Are New York City’s teachers as smart as their students? John Sexton, the ex-president of New York University, thinks not. During a talk he gave on the future of American universities at the Library of Congress last week, he claimed that in the past five years, New York City public schools have been hiring “teachers that have lower SAT scores than the students you are graduating. That’s a ticket for failure, because you’re hiring from the bottom half of the existing class.”
In a year with a mis-timed SAT (leading to 2 sections not counting), cheating scandals, delayed PSAT results, incessant marketing of a practice tool as the solution to societal inequity and educational injustice, and fights between SAT and ACT, it’s easy to understand why the one of the few positive changes that came with the revised SAT would have been largely overlooked and unmentioned. This past January, College Board completely revamped the way it delivered scores to schools. The organization that owns and is responsible for designing the SAT not only revamped its test (for the first time since 2005) but also revamped its reporting portal for school counselors for the first time since the dark ages. The new school reporting portal changes should provide significant benefit to schools and teachers around the country. To put these changes in context let me point out that until very recently, the College Board sent dot-matrix printed labels with PSAT scores on them to high school counseling offices and additional data schools had to be purchased and was mailed on CD to schools. Yes you read that right. In the 21st century the multi-billion dollar educational institution was sending dot-matrix labels and mailing CDs.
Welcome to the internet era College Board
In the fall of 2015, College Board went entirely online with their (singular their) reporting portal and has been steadily improving it ever since! The portal currently allows users to review and export almost all the data a school counselor or teacher could want. The College Board SAT educator’s portal is what I want from every standardized test administered in a public school. Timely, up-to-date access to my students’ test scores and performance data. While there is certainly room for improvement, this portal is miles ahead of anything I’ve seen from major national standardized tests. The College Board has created a tool that allows counselors to look either at the macro level (how did students do overall this year vs last year) of student performance or the micro (how did Malala answer on question 7 in the reading section). This is all available online from the comfort of your cubical at work. No waiting for a disc to be mailed, no asking a student to print and bring in their score report. Timely, detailed, and accessible. College Board has, with this one portal, potentially made the SAT a useful tool for teachers and schools. Approximately a month after a test is administered a school would have back complete data on all students, ready to be shared with the academic team and ready to be used to improve instruction. This is pretty close to exactly what testing (at least test result reporting) should be.
But I ramble, let me do more showing and less telling.
Question Analysis Report
- Performance comparison – the report compares the schools performance on the test to the district, state, and nation. Allowing a school to easily identify possible indicators of holes in the curriculum.
- Links to questions – the portal provide direct links to the questions from the PSAT (always) and the SAT (for released forms of the SAT) so that educators can easily access and see the questions.
- Answer selection percentiles – knowing the percentiles rates at which each answer choice (when combined with the actual question) is selected lets educators gain insight into the types of mistakes made.
Also of note are links to state standards (I’m told that its actually linked to the specific state standards), categorization of questions (yeah it’s still CB quasi-meaningful categories), sortable tables, filterable data sets, and ability to make sub-groups of students. Check out the screen shot to the right of the report options and below of the Question Analysis Report. The biggest surprise from the portal for me was that with the release of May SAT scores, the actual questions from two of the three forms given in May were available in the portal. Historically, CB made the Question and Answer Service available for purchase ($18) only by those who had taken that test administrations (thus creating a healthy grey market for QAS booklets that advantages students with tutors or a lot of info about dodgy internet corners). Schools were never given easy access to the questions from released tests and even those students who ordered the QAS service waited 2 – 3 months to receive a copy of the questions. Now the questions and answers are relatively quickly available in the school to the educators who can do something with it! Even if this is only available 3 of the 7 ( or 10 if you count the three SAT School Day dates or 17 if count Sunday test dates also) test dates. This is a boon to schools and students.
Most of the other reports in the dropdown (2nd screenshot in post) are just a variation on the screenshot below. They provide different ways to look at student scores, allowing you to look at the scores (versus district, state, nation) or the benchmarks (which kids reached CB’s benchmarks), or by fee waivers used. Most of these reports are mildly interesting to schools and teachers but not particularly actionable in terms of improving student performance. The benchmark report is probably the most interesting of the other reports. While, I’m not convinced that College Board’s benchmarks tell enough of a story to be independently useful for a student, I think they are useful as a comparison point for a school, district, and maybe even teacher. Being able to see where your school underperforms or out-performs schools in your district, state, or the nation can be used to inform revision of curriculum (when paired with analysis of the questions that feed info those benchmarks) or refocus instruction.
(Update: I found a few more gems after originally posting this that I wanted to point out) A few other reports that are great for counselors:
- View/Print Student Score Report
- View/Print Student Essay
- View/Print Student score sends (there is a whole post on the process and costs of sends that might be cooking).
How to make the SAT portal a game changer
If College Board really wants to make this product a game-changer here are a few “simple” things they could do:
1. Find a way to make this available to community based organizations
Given the sad state of the caseload of most school counselors a decent portion of the work of supporting students for the SAT (and college access) falls to community based organizations. Currently CBOs have no way to access group reports on the students they work with.
2. Add content tags to questions
When I ran my own test prep company, our database allowed me to tag question with up to 30 different content areas (not just Harts who love Algebra but also exponents, variable, and manipulating equations) and thus make it sortable and searchable by a wide range of characteristics. Adding this feature to the portal would allow teachers to search/sort questions by content areas by terms they know better.
3. Create an “content performance report”
The portal needs an easy way for the test coordinator/school counselor/principal to get the information out of the portal and put it in front of the teachers and academic department heads.
Despite my objection to categorizations of the SAT as an unbiased predictive assessment of college success, this new portal goes a long way toward making the SAT a useful assessment of certain academic performance. For me any assessment is valuable only when an educator or student can relatively quickly following the test review performance on that test and take steps to improve that performance. With the changes to its reporting portal College Board has done a lot to make the SAT such an assessment, so I say to David, Aaron, Steven, Shameek, Farhad, Teirney, and the team at CB: