The NCAA is conducting its mock NCAA tournament selection exercise this week (the results should come out today), and so talk has turned to the method by which the committee selects teams. That, of course, means criticism of the RPI, a ranking system that most people find to be inadequate compared to other options.

Seth Davis, though, tweeted out this morning, “I don’t get the argument that NCAA should value Pomeroy ratings ahead of RPI. Would you rather be efficient and lose or inefficient and win?”

(Last year, I talked about how RPI is measured, and criticized it, so check that out if you don’t know what this is about.)

Basically, RPI does not use any margin of victory, and then bases “strength of schedule” on RPI. Pomeroy takes into account underlying efficiency stats and per possession numbers, to calculate team ratings and strength of schedule.

But how much of a straw man is this statement about losing efficiently versus winning inefficiently? I immediately responded, “I feel like this falsely states the issue. New Mexico and Southern Miss are both 20-4 against DI. Systems rank differently.”

I thought I would go into more detail and actually do some research here. Davis’ statement would imply that the Pomeroy rankings feature a bunch of teams that lose a lot more games but look really good when they do, while the RPI rewards winners.

I went through all teams currently rated in the Top 30 in either the RPI or Pomeroy (or both), where there was a difference of at least 6 spots in the relative rankings. That gave me a nice round eleven teams in each group – those rated higher by RPI and those rated higher by Pomeroy. We can view these as teams where there are differences, so let’s see if the “RPI teams” do win a lot more games.

The RPI teams won 77.9% of their games against Division I opponents (19.4 wins to 5.5 losses on average). The Pomeroy teams won 75.3% (19.2 wins to 6.3 losses on average).

That’s pretty similar, especially considering that the RPI teams played more non-D-I opponents, featured more non-BCS conference teams, and were higher rated by RPI (11.7) than the Pomeroy teams were rated by his system (14.2). This belief that the RPI has winners and Pomeroy has lovable losers is false. They just measure teams differently. I know this is shocking, Pomeroy’s rankings tend to like teams that win.

Here are teams where RPI and Pomeroy are near mirror images of each other in the two ratings, one liking one team over the other. The RPI-favored team is listed first.

  • Duke (22-4, 3rd RPI, 14th Pom) versus Ohio State (22-4, 8th RPI, 2nd Pom)
  • Baylor (22-4, 7th RPI, 15th Pom) versus Wichita State (23-4, 16th RPI, 9th Pom)
  • Marquette (21-5, 11th RPI, 20th Pom) versus St. Louis (21-5, 22nd RPI, 10th Pom)
  • UNLV (22-5, 12th RPI, 28th Pom) versus New Mexico (21-4, 29th RPI, 11th Pom)
  • Gonzaga (21-4, 18th RPI, 30th Pom) versus California (21-6, 35th RPI, 18th Pom)
  • Temple  (20-5, 17th RPI, 34th Pom) versus Texas (17-9, 38th RPI, 19th Pom)
  • Connecticut (16-9, 21st RPI, 37th Pom) versus Virginia (19-6, 40th RPI, 23rd Pom)
  • Seton Hall (18-8, 30th RPI, 45th Pom) versus Alabama (16-9, 39th RPI, 29th Pom)
  • San Diego State (20-5, 27th RPI, 57th Pom) versus Kansas State (17-8, 61st RPI, 27th Pom)

Look at that list. Do the RPI teams on the left really look like a bunch of “inefficient winners” compared to those “efficient losers” on the right side? The interesting thing here: The Pomeroy teams in the pairs are 2-1 head to head: Ohio State beat Duke by 22. Texas beat Temple by 12, while UNLV beat New Mexico by 17 earlier (and they rematch tomorrow at New Mexico).

Three weeks ago, I did a post on the teams where there was the largest disagreement between RPI and the Pomeroy and Sagarin ratings, among possible tournament at-large teams.

In the meantime, what has happened to those RPI All-Stars? They have seen their RPI rating decline by 5 spots on average, while their Pomeroy ranking stayed the same. The largest decline was also the team that was #1 on the list, Colorado State, who have dropped 14 more spots from 22 to 36 in the RPI.

Meanwhile, the Pomeroy and Sagarin All-Stars have seen the RPI rankings actually move toward them, improving in RPI by 11.2 spots on average, with Texas, St. Louis, and Indiana making large jumps. Meanwhile, their Pomeroy rankings have stayed stable (actually dropping 1.4 slots on average).

Guess what that means when one ranking system changes by moving toward the other, in both cases? Which one do you think is more accurate?

So, RPI is not the only tool, that’s what this later tweet suggests:

This creates a false contrast. No one is arguing that the committee use a rating system like Pomeroy, Sagarin, the new ESPN-created BPI, or whatever, exclusively to select and seed teams either. I agree that the committee will not seed Southern Miss as a #3 seed, but in a post-Pomeroy world where those ratings would be used as a tool, no one is seeding Wisconsin as a #2 either.

Think of it as building a house. Would you use just one tool to build a house? No. You wouldn’t. But you would be far better at your job if you used a nail gun and compressor, rather than a rusty old hammer and hand nailing everything.

Or think of it as the underlying language. The committee may say that they use it as a tool only. They probably don’t seed Southern Miss as the tenth best team. However, I can tell you that going through these exercises and teams is time-consuming. Language aids us in taking shortcuts in communicating information. So while they may, after looking at a team, recognize that Southern Miss and Colorado State are overrated, do they then go through each resume after that? No, because that language still pervades the meeting room. Why do you think “bracketologists” are pretty good at getting the teams right? They know the garbled language the committee relies on.

And it is garbled. With the various weights, with no account for how teams won, it becomes like the whisper game, where Colorado State looks better, so Southern Miss and Duke look better, and it keeps going. So despite what they claim, “Colorado State” becomes converted to “Top 40 win” and it shows up on a nice little summary that is printed out.

You would think they would want to improve the language. Right now, though, we can’t get past misrepresenting what the languages are actually saying. The teams ranked higher by Pomeroy aren’t typically losers.

[photo via U.S. Presswire]