Scaled score inflated?

Post Reply
rylie_hess
Posts: 1
Joined: Tue Jan 07, 2020 10:13 pm

Scaled score inflated?

Post by rylie_hess » Wed Jan 08, 2020 1:06 pm

I have been getting around my goal score on the practice tests, but I have noticed that the Next Step full length scaled scores are more inflated compared to AAMC full lengths. For example, I got a 126 on Next Step’s chem/phys with 31/59 correct. However, on AAMC’s chem/phys, I got a 124 with 32/59 correct. Does this mean my score on test day will be lower than what I have been getting on Next Step?
NS_Tutor_Mathias
Posts: 480
Joined: Sat Mar 30, 2019 8:39 pm

Re: Scaled score inflated?

Post by NS_Tutor_Mathias » Wed Jan 08, 2020 1:20 pm

This question comes up a lot, so here is a detailed response we keep at the ready:
Thanks very much for reaching out! Your question is a good one, since it can be difficult to figure out how each company devises its scoring scales. Forgive the long email, but I want you to have as much information as possible.

Basically, a scale has two components: "raw-to-scaled" and "scaled-to-percentile." Scaled-to-percentile (like the idea that a 127 correlates to the 79th percentile) is pretty easy, as one can just monitor data from students who have taken official MCATs and mimic it as well as possible. There also isn't much variation, since a 127 will always be around the same position on the curve of test-takers.

What you're talking about, though, is the "raw-to-scaled" score, which is the tricky one. This is where we convert how many questions you answered correctly to a scaled score, like 126. Any test prep company could easily just mimic the exact same scale the AAMC uses for one of their scored exams. This would actually be far simpler than devising our own scale.

So why do we not do this? Because one of our top priorities is making sure our students' scores accurately predict their scores on the official exam (not necessarily their scores on the AAMC practice tests). When our exams were first released, we used a scale that was generally 1-2 points more "harsh" than our current scale (meaning it gave students lower scores, like what you saw on your AAMC test). However, once we had collected data from hundreds of our students who took the first 2015 tests, we saw that this scale was consistently predicting scores too low - in other words, it was giving scores that were lower than the great majority actually got on the real test. At this point, we adjusted our scale to match the student data we collected, and we constantly revisit that data to make sure further changes are not necessary.

Even more importantly, though, note that even the AAMC varies significantly in its scoring from exam to exam. This is harder to notice now, since the AAMC has only published three scored exams. But in many cases, if you took both scored AAMC tests and got the exact same number of questions correct, you would get a different score in one or more sections.

This is true across the board for AAMC practice exams and for real MCATs. If we used one of the exact scales that the AAMC uses for all 10 of our exams, it wouldn't account for the difficulty differences that are present to some degree in our tests (and certainly in the official MCAT - difficulty varies significantly from section to section and between exams).

In conclusion, we've developed the scale that best fits our large amount of student data, but this will inevitably deviate slightly from some AAMC scales. Let us know anytime if you have additional questions!
In short, scoring is adjusted to reflect the same distributions that are present on AAMC exams, and as such will be highly variable. Not every exam is equally difficult, so every single exam, AAMC or NextStep, has its own scoring scale. If you're scoring higher on our exams, odds are you're just doing a little better, that is all.
Post Reply