Screen Shot 2016-08-28 at 4.06.28 PM

Assessing the Assessment: SHSAT


Screen Shot 2016-08-28 at 4.06.28 PMAs we approach September in NY, I enter that special time of year affectionately know as “fall hell”, when parents of students in every transition grade realize that admission testing is looming and they need to figure it out. Parents of 11th and 12th graders have to think about SAT and FAFSA, parents of 7th and 8th graders are thinking about the ISEE, SSAT, SHSAT, TACHS, COOP, and/or HSPT, parents of 4th and 5th graders are thinking about the SSAT, ISEE and/or Hunter High School exam. This time of year can clearly bring lots of stress and confusion.

In this post, I’ll take on the SHSAT and give a little insight into what makes it what it is. Let’s go…

What is the SHSAT?

The SHSAT is the Specialized High School Admission Test. It is the sole criteria used to determine which of 27,000 test-takers are given one of about 5,100 offers (18% of test-takers) for a seat at the 8 specialized schools.  The SHSAT is the primary (and nearly only) gatekeeper for admission to these schools (how a student ranks the schools is the other factor but that’s too much to get into today).

 

Here is how the test is structured:

Questions Question types/Topics Time
Verbal 45 Scrambled Paragraphs
Logical Reasoning
Reading Comprehension
75 minutes recommended
Math 50  Percents, Proportions,  Ratios, Word Problems, Number Properties, Algebra, Geometry, 75 minutes recommended

 

How is it different? 

Unlike tests taken in school or statewide assessments the SHSAT is designed to predict performance at a small set of highly rigorous high schools. As such, the SHSAT has a much looser relationship to state or national standards and only a passing similarity to the content of a 6th or 7th grade classroom. The narrowness of the use of the exam creates a test what is by design and intent different from the statewide assessment test, which is designed for all students studying under state standards.

The state wants all students to achieve proficient or better on the statewide tests, the DOE doesn’t want nor can it have all students achieve a score good enough for admission to the specialized schools.

This difference in philosophy leads to differences in test design, timing, and even scoring. Since many of the technical elements of the test are probably too much to discuss here, let’s explore some differences in the reading section.

SHSAT Reading vs NYS ELA Assessments

The NYS ELA Assessment (NYSELA) is designed to assess whether a student has achieved proficiency on the learning standards defined by the state, which means for your typical 7th grader being able to read and answer questions from passages at grade level by the end of the school year. The NYSELA thus uses reading passages that are written in a style and using vocabulary that your typical 7th grader should be used to. I examined the released items from the 2015 ELA exam and below you can see some statistics from the Flesch-Kincaid analysis (you can learn more about reading scales here).

 

Number of words Sentences per paragraph Words in sentence Characters per word % passive sentences FK grade
ELA 1 799 3.2 17.7 4.5 4% 8.4
ELA 2 1003 5.2 17.2 4 5% 5.7
ELA 3 789 1.9 17.7 4.7 7% 9.9
ELA 4 775 3.8 22.9 3.7 16% 5.6
ELA 5 907 3.5 18.5 4.5 14% 8.4
Averages 854.6 3.52 18.8 4.28 9% 7.6

Note that the average grade level score for this test seems appropriate to a 7th grader. By comparison, this NY Times article about Kaepernick stand scores a 5.8 and this blog post (only from “unlike tests taken through “reading scales here” scores a 7.5). On the other hand the SHSAT in order to weed out significant portions of students has to use reading passages that are much more complex. Here are the numbers from the same evaluation of the set of 5 passages (every test has 5 passages and 6 questions per passage) from the two sample tests in the 2013 DOE Handbook:

 

Number of words Sentences per paragraph Words in sentence Characters per word % passive sentences FK grade
SHSAT ’13 – 1 428 4 17.8 5.1 33% 10.7
SHSAT ’13 – 2 470 4.4 21.3 5 36% 12.9
SHSAT ’13 – 3 474 6.2 15.2 5 19% 9.8
SHSAT ’13 – 4 370 6 20.5 4.9 33% 11.8
SHSAT ’13 – 5 447 4.6 19.4 4.9 13% 11.3
Averages 437.8 5.04 18.84 4.98 27% 11.3

 

Looking at the results from this sample (admittedly small) we can see that while the passages are shorter on the SHSAT, they are generally more complex. The typical passage is at least 3 grades above that of the passages on NYSELA. Not only that the rage of the passages is wider on the NYSELA which to me means that on the SHSAT its much more likely that a student who has trouble with reading will encounter passage after passage that is challenging if not incomprehensible. On the NYSELA, having some easier and some harder passages means that a student will have moments of relief as he is able to understand some of the passages given.

 

But complexity of writing is not the only factor that impacts why a child doing well in school and on state tests might not excel on the SHSAT. Let’s next look at the timing of the two exams.

 

How is the timing different? 

The timing of an exam is very import because not only does it impact how many questions you get done but it also impacts the feeling of rushing that a student must deal with during testing. The more rushed a student feels the more likely they are to make mistakes.

First in terms of timing, the SHSAT gives 150 minutes to complete all 95 questions (both math and verbal) and the statewide tests decided this year to make the tests untimed. This makes this evaluation a little bit more complex. For the purpose of comparison, I’m going to use the timing data for the NYSELA from 2015 when the NYSELA had a (generous) time limit (and had a few more questions).

 

Reading Questions Passages Words Time Minutes per question Words per min
SHSAT 30 5 2189 60 2 36.48
NYS Assess. 51 11 8546 150+ 2.9+ 56.97

As you can see the time allotted for the state test is much more generous even before you account for the fact that this year there were no limits. Not only does this give students more time to work on the test but it also removes the perceived pressure to work quickly, which in my experience leads to a huge percentage of errors on tests.

So given these differences in the tests purposes, design and timing it’s not surprising that a student might perform very differently on these two tests.

 

What do you do about it?

Don’t assume that because your student does well in school that they will do well on any other test or in any other setting. Kids who do the best on the test are those who go into confident and prepared. Don’t make assumptions your kid will do well. If you’re thinking of a Specialized High School start looking into the test and preparation in 6th grade. Explore the DREAM – SHSI program run by the DOE or at very least have your child take a practice test to see how they would do on the SHSAT so that you have plenty of time to prepare if you need to.

Parents should constantly help their children improve their reading abilities by getting the to read often. Reading things like the newspaper, comic books, and even exciting blog posts like this will help them develop their reading.

Also test preparation can help to mitigate these differences and help students understand what the test is and how it’s different from what they may know.

 


If you’re interested in SHSAT and NYC Specialized schools, here are a few stories links you should read.

  • City Announces Programs to Boost Diversity at Specialized High Schools http://dnain.fo/24DFmUP
  • Diversity in New York’s specialized schools: A deeper data dive http://disq.us/t/29jwwog
  • NYC Education Department urges renewal of Pearson entrance-test contract  http://on.wsj.com/2bvMS0k

 

Here are some more organizations that help students prepare for admissions to various school types:

Middle and Elementary School into High School

  • Exam School Partnership Initiative – http://espi.nyc/
  • PASSNYC – http://www.passnyc.org/
  • Breakthrough NY (also exists in other cities) – http://www.btny.org/
  • Oliver Scholars – http://www.oliverscholars.org/
  • A Better Chance – http://www.abetterchance.org/
  • Say Yes to Education – http://sayyestoeducation.org/

High School into College

  • Breakthrough NY (also exists in other cities) – http://www.btny.org/
  • Sadie Nash Leadership Project – http://www.sadienash.org/
  • The Wight Foundation – http://www.wightfoundation.org/
  • NJ SEEDS – https://njseeds.org/
  • The Opportunity Network – https://opportunitynetwork.org/
  • Give Something Back Foundation – https://www.givesomethingbackfoundation.org/

Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>