|
Talent:
Is there a bias in recruiting rankings?
Do
teams and athletes from certain regions/teams/conferences get
ranked higher than athletes from other
regions/teams/conferences? Does southern and western talent get
overrated? Do Big East teams get underrated?
One
way to determine bias is to look at which teams overachieved and
which teams underachieved during a certain period of time based
on the talent they allegedly recruited. Did many teams that
allegedly had a lot of talent but didn't play well come from a
particular conference or region of the country? If so, there's a
good chance their talent was overrated, which is why they didn't
play as well as expected. Or are they spread out across regions?
Maybe there's a different explanation, like obvious coaching
problems; a coach in his first year or two is probably going to
struggle no matter how strong the talent.
This chart of
achievement cross-matches the alleged talent teams got from 2003
to 2008, according to Rivals, with their final Sagarin rankings
from 2007 to 2009, when the talent recruited earlier would have
made up the majority of players on the field. The numbers for
all BCS Bowl Conference teams and Notre Dame were tallied and
the teams were ranked relative to each other.
Raw
Data
So,
for example, with a total of 15,957 points USC was considered to
have recruited the best talent from 2003 to 2008, and got a
ranking of 1 for talent. But USC's average Sagarin rank for the
years 2007 to 2009 was 9, which works out to only 4th
best. Thus, USC
underachieved (a little). Given the team's troubles in 2009,
this makes perfect sense, assuming they got all the talent the
recruiting services said they did. Currently there is no reason
to believe they didn't.
Two
things jump out from this graph:
1. By and large, the
recruiting rankings get it right. The center line represents
100% accuracy between the level of talent and the play on the
field that you would expect to result. The closer a team is to
the line the more accurate the prediction; the further away the
less accurate. In the case of Indiana, the team that allegedly
got the 64th
best talent (out
of 66 teams) finished with the 64th
best Sagarin
rank, an average of 98.
Allowing
for some human error, the shaded gray area above and below the
line also represents situations where the talent recruited led
to approximately the expected result. USC got the 1st
best talent and
finished with the 4th
best results, a
little bit of underachievement—but still pretty good. USC,
as well as all the other teams in the gray area, don't give much
reason to believe the recruiters were far from the mark.
Thirty-six out of 66 teams lie within the shaded area, and
roughly 10 more are close—a sign of high overall accuracy.
2.
SOME teams from certain conferences and regions greatly
overachieved and underachieved AND there is a clear pattern.
Anti-Big East bias
Why
do 6 out of 8 Big East teams appear as having overachieved?
Pittsburgh and even Rutgers are close to having been accurate,
but that still leaves half the conference—West Virginia,
South Florida, Connecticut and Cincinnati, most of all—with
Sagarin ranks far above where they should have been. South
Florida allegedly had the 56th
best talent yet
finished with an average Sagarin rank of 34, the 25th
best. West
Virginia allegedly had only the 44th
best talent yet
finished with an average Sagarin rank of 20, 13th
best.
You
might be able to argue that the elevation of Cincinnati,
Connecticut and South Florida from midmajor to BCS Bowl
conference level during this period threw off recruiters, but
that doesn't explain why West Virginia and Rutgers both
“overachieved,” too. Rutgers average ranking over
this period was 37, on par with North Carolina, South Carolina
and Miami—yet if recruiters are to be believed Rutgers did
it with the 52nd
best talent while
South Carolina did it with the 17th
best. Is it a
coincidence that all of those teams with “more talent”
but similar achievement come from the southeast or is there a
pro-southeast bias that rates their talent as better than it
is?
And since Sagarin takes into account Strength of
Schedule you cannot argue that Big East teams got to where they
did by beating up on each other. If they were all so weak their
Sagarin ranks would reflect that by being lower. There is clear
bias against teams in the Big East present, if not bias against
talent from the northeast in addition. They had better talent
all along than they were given credit for.
Other
Conferences
This
is all the more obvious when you look at the teams from other
conferences. EVERY SINGLE TEAM from the Big Ten, for example,
falls within the gray area except Illinois, which still lies
close to the center, and Michigan, which has an obvious reason
for having done so much worse than expected. How did the
recruiters get it so right for the Big Ten and so wrong for the
Big East?
Furthermore, many of the other outliers have a
good explanation for how they got there. Michigan, the greatest
underachiever of all, played so poorly because of a transition
to a new coach with a radically different style of play. Notre
Dame, another great underachiever, also had coaching or team
problems, as did Miami. Perhaps their talent was still
overrated, but there is a pretty clear explanation for why they
underachieved. To boot, few other teams in their conferences are
far from where they should have been.
Other
bias
There
is also evident a bias against “small name” schools
and for “big name” schools in this graph, which
explains why so many of the alleged overachievers are where they
are and why so many of the alleged underachievers are where THEY
are. There are no blue-chip names among the teams that greatly
overachieved. Granted, when you're expected to finish say 5th
it's hard to
overachieve too much, but even solid programs like Wisconsin,
Clemson, Oklahoma State, California, Mississippi or Arkansas
didn't greatly overachieve. (The only team that even comes close
to fitting this profile is Virginia Tech, finishing with the 7th
best Sagarin rank
despite allegedly having only the 23rd
best talent--a
team which, incidentally, was in the Big East for the first half
of the recruiting period.)
Only
teams with recruiting classes alleged to have been 38th
(Missouri) or
worse greatly overachieved. Why did only teams among the very
“lowest on talent” surprise and nobody else did? And
why are most of those particular teams from schools that don't
have legendary names?
|
