01rfd-image-custom1

Choosing a Test Prep Program


01rfd-image-custom1Often I’m asked for help selecting a test prep program, course, or tutor (usually in an email that provides almost no details of the situation of the asker…but that’s another story) and of course, if you ask me a question this leads to a 500+ word screed as a response. Since ranting in private spaces only helps a few, I’ve decided to share my thoughts with a wider audience (all 6 of you who read my blog posts).

 

The criteria and considerations below are some of the things you should look for when you are deciding how to prepare for an admission test. You may find other things have to be included in your decision-making but I highly recommend that you at a minimum consider these factors. I’ll outline my rationale for each factor as I go. 

 

First, your brother’s/sister’s/friend’s/aunt Mabel’s experience will probably not be yours and shouldn’t be the sole/main driving factor. When talking to friends and family about their experiences with something, you should seek concrete quantifiable information (“the materials had at least one typo per page” or “the teacher constantly assumed we knew rules and was hesitant to answer my questions, which he more than once implied were simplistic”) rather than subjective feelings (“the books were bad” or “the teacher didn’t teach well”). You should seek consensus and trends rather than individual stories. Individual stories (also known as small batch artisanal data) of either success or failure are not particularly reliable unless you know the entire story from both sides, so while these stories might color and inform the data you gather, they shouldn’t be your primary source. Second, all methods and plans can improve scores, but none are guaranteed to improve your (or your child’s) score. You are trying to select the program or method that is most likely to combine the features, approaches, and resources that you believe will get you (your child) the result you want. Third, teaching admission tests prep is teaching, and like any teaching the learning that happens is dependent on teacher, student, and curriculum. Test prep is not magic. Test prep teachers do not have a secret pill to take a student from knowing nothing to a stellar score. Preparing for tests take work and guidance. Choosing a test prep program is choosing the best guidance for your needs and budget, but then requires putting in the effort to make that guidance work.

 

These first decisions will help you quickly cut through the vast array of offerings. Though these factors will also be heavily influenced by budget but I recommend you delay considering budget until you’ve decided on the other factors. After you’ve ranked your prep methods, companies, and options then you should look at what you can afford among the ranked options you’ve identified.

 

1. Type of Instruction
To prepare for any test you have 3 basic options:
  • self-study (books, online tools, DVDs, apps)
  • classes
  • one-on-one tutoring 
Your first decision should be which of these methods you plan to use (or start with). Tutoring is generally the most effective because it is (should be) personalized and flexible. Classes are the next best option providing structure and organization while still providing a live instructor to provide feedback and answers. Self-study is the most time-consuming option since it requires the student to play the roles of course administrator, teacher (online resources can mitigate this), and student simultaneously.

 

2. Modality of Preparation
The two main modalities of prep are online and in-person. Taking a class online is vastly different from taking a class in a traditional classroom. In order to effectively take a class online a student must be able to focus and avoid distractions (Facebook, email, texting) since there isn’t a teacher looking at them to hold them accountable. The student must also be comfortable speaking up since teachers do not have the luxury of looking at faces for signs of confusion. There is no standard marketing lingo to distinguish between online recorded lessons and live lessons delivered online. If you are looking into online programs make sure you are clear whether it will be live class conducted online or a recorded class that is watched. Some programs will offer “50 hours of prep” and have on 10 of those hours with an live instructor and the rest is videos.

 

Once you had decided on type of program and modality of preparation you’re ready to start evaluating which company or person you’ll hire to deliver that program.

 

 

Selecting a provider
Here are some elements that you should look for when selecting a program provider:

 

Teaching quality
The quality of the teacher you have is probably the single biggest factor in how well you’ll improve (after your own effort), so it’s imperative that you look for the best possible teacher you can find. Whether you’re considering private tutors or a group course run by a large multi-national company the following questions will help you judge the quality of the teachers they employ. Also check out these 6 questions to ask a potential tutor by Mike aka Pwn The SAT (except for question 3). 
  1. What are the criteria for hiring teachers?
    Ideally a preparation company will have minimum standards for hiring teachers that go beyond “they got a good score.” A good score tells you the person can take the test not that they can teach the test.
  2. Are new employees provided training?
    Whether someone is working for themselves or for a large test prep company unless the teacher is Stanley Kaplan, Adam Robinson, or John Katzman they didn’t invent test preparation and likely received training somewhere from someone. The benefit of a formal training in teaching tests is the ability to learn from people who have made the mistakes and learned from them. A first time SAT teacher usually doesn’t have the experience to understand where most students will get stuck or how to get them unstuck. Training is a good proxy for verifiable high quality experience.  In all companies in which I’ve worked our new teachers were given 25 – 50 hours of training before they were allowed to teach paying customers.
  3. What are the key skills you look for in your new teachers?
    Good teaching is more than simple knowing material, it’s a combination of soft and hard skills that allow the teacher to break down material so that students can understand it while concurrently enlivening that material so that students remain engaged. I look for teachers (or companies) that can convince me that they are at least aware that being a good teacher is more than being able to recite the structure of a test.

Program Duration

Since test prep is unregulated, you’ll find a huge range in the number of “required” or “recommended” hours to “fully” prepare for an exam (LSAT courses can have as few as 20 hours or as many as 100) typically based solely on the business model of the company/tutor. When evaluating programs, pay attention to the total number of instructional  hours (live teaching not tests, workshops, or study sessions) and ensure that you know what you are comparing. One way to compare companies is to look at the dollar per teaching hour costs. Generally speaking, the more instructional hours the better, though this doesn’t convey quality, it does convey opportunity to ask a knowledgeable person to help you figure stuff out.

 

There are two types of materials that are used in most programs: proprietary and official. Proprietary materials are created by a test prep company for their use in their programs. Sample questions in proprietary materials, when correctly written, should be indistinguishable by most people (apparently some people like Debbie who spent a year taking SATs can tell) from the official materials. Official materials are those put out by the companies that write the actual exams (e.g. College Board for the SAT and GMAC for the GMAT). Official materials are almost exclusively practice materials with limited instructional value (think of a big book of practice questions rather than strategy and advice). Also there is a distinction between proprietary classroom materials and proprietary bookstore materials (bookstore books are not designed to be used in a classroom and if a company uses bookstore books in a classroom that isn’t great).

 

The best program design will combine proprietary and official materials for most test types (the exception would be for LSAT prep, in which almost no reputable company writes their own LSAT questions). I’m a fan of a company having materials they have written themselves (mostly/especially for classes) because it allows for better organization of teaching. The official test companies do not organize their released materials by topic area or difficulty so using that in a classroom is cumbersome at best. Additionally, the released official practice materials don’t always fully cover the way a topic is tested and thus proprietary materials allow teachers to fill in the gaps (this is especially true of tests like SSAT, which only release 1 or 2 official tests).  Further, a company having proprietary materials suggest that the company has a research and development department and thus will likely have a better knowledge of the test.
Keep in mind that practicing in the modality that the test is given is key. You can’t effectively practice for a computer based test if you only ever practice on paper tests. Its important to have practice and especially practice tests that are delivered in the same way (computer or paper). 

 

Instructional Focus
One important factor in preparing for an exam is finding a program that teaches what you need to know and in the ways that will best help your score. Some programs focus almost exclusively on providing test preparation strategies (plugging in, memorizing common means of solving questions, identifying test patterns) and others focus on trying to reteach you all of the content you need for the test (algebra, grammar rules, etc). Few programs effectively strike a balance between teaching the appropriate depth of content (so that you can effectively leverage test preparation strategies) and teaching you strategies (which help you effectively apply your knowledge of content). This is especially difficult to do because not everyone has the same needs. A test-taker with a strong math foundation will primarily need test preparation strategies to improve their score while someone with a weaker grasp of foundational math (fractions, manipulating equations,etc) would greatly benefit from improving the underlying skills before attempting to layer on strategies and time-saving techniques.

 

If you’re unsure which instructional focus is best, take two full-length practice tests: one untimed and the other timed. If the score on both tests are fairly close (with in the SEM for that test) that implies that the issue is content related rather than strategy related. If the score is vastly better on the untimed test that suggests that the issue is timing and strategy. (If the score is better on the timed test, that’s weirder and I’ve got no generalization for that situation.)

 

Style of Instruction
Different individuals and companies have different teaching styles. Some primarily lecture, while others use a more Socratic method. Different students respond differently to each method and if you can identify which method best helps you it will help you make a selection. This criteria is a bit more challenging to identify because if you can’t watch all teachers that work for a company or all tutors you’re considering teach.

 

The best way to figure out the style of a program is to visit a teaching session. Many test prep companies offer free informational sessions or sample classes where you can learn about the test and their program. These sessions are great to attend. If you do attend one of these sessions, keep in mind that the person conducting the lesson is likely one of that companies better teachers (since they trusted them to do marketing). If this teacher comes across as uninformed or uninspiring it likely reflects on how that company measures good teachers.

 

Extras
Other factors I’d look for are online practice resources, extra-help policy (what happens if i need help when doing my homework?), extra resources (if i want to take an extra practice test what happens). Ignore refund and guarantee policies (those are predicated on you not being happy and are marketing devices not measures of quality).

 

Taking all of the above and combining that with your impression from meetings or info sessions should help you effectively make a decision.

Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>