Wednesday, October 18, 2006

College Football Ranking Systems and the Laws of Retrodictive Accuracy and Cyclic Triads

So, you know all about sports?

Then the concepts of "retrodictive accuracy" and "cyclic triads" should be familiar terrain to you.

As B. Jay Coleman informs us in his article, Minimizing Game Score Violations in College Football Rankings, Interfaces 35(6), pp. 483–496, ©2005 INFORMS:

"[R]etrodictive accuracy is equivalent to minimizing game score violations: the number of times a past game's winner is ranked behind the loser".

Unfortunately, this is not always possible, since, as Coleman informs us, we face the problem of "cyclic triads", where Team A beats Team B, Team B beats Team C, and Team C beats Team A.

Nevertheless, the amount of retrodictive inaccuracy in college football ranking systems is far greater than cyclic triads alone would engender, Coleman has found that even the current best ranking system from the standpoint of retrodictive accuracy still contains violations which are at least 38% higher than the minimum which can actually be achieved.

Coleman thus developed a minimum-violations ranking solution - MinV, which ranks football teams so that the minimum number of retrodictive violations occur. As Coleman writes:

"The minimum number of game score violations MinV identified for the NCAA Division 1-A college football seasons from 1994–2003 was far superior to the numbers that 58, 68, 75, and 93 different ranking systems (Massey 2004b, c) produced in the years 2000, 2001, 2002, and 2003, respectively. Massey did not report ranking violation statistics prior to 2000."

He then examined the college football games from 1994 to 2003 and applied his results:

"to determine whether the consensus top five teams in any of these 10 seasons would have been changed if one had enforced a minimum violations criterion."

Using his system, the national champion would not have changed, but the rankings of teams below Nr. 1 would have changed, in part, substantially:

MinV ranking (Actual Massey Consensus Ranking in Parentheses)

Year...Number 1...........Number 2............Number 3...........Number 4..........Number 5

1994...Nebraska (1)......Penn State (2)......Colorado (3).......Alabama (5).......Texas A&M (8)
1995...Nebraska (1)......Florida (2)...........Tennessee (3)......Florida St. (4)....Colorado (5)
1996...Florida (1).........Ohio St. (2)...........Florida St. (3).....Arizona St. (5)...Nebraska (4)
1997...Nebraska (1)......Michigan (3)........Florida (4)..........Florida St. (2)....Tennessee (5)
1998...Tennessee (1).....Ohio St. (2)..........Florida St. (3).....Wisconsin (4).....Florida (5)
1999...Florida St. (1).....Nebraska (2).......Va. Tech (3)´......Michigan (4)......Kansas St. (5)
2000..Oklahoma (1).....Nebraska (8)........Washington (6)...Miami (FL) (2)...Florida St. (3)
2001...Miami (FL) (1)...Tennessee (3)......Florida (2)..........Oregon (4).........LSU (9)
2002...Ohio St. (1)........Miami (FL) (3).....Georgia (4).........Oklahoma (5).....Texas (6)
2003...LSU (1)..............Southern Cal (2)..Oklahoma (3)......Georgia (4)........Miami, OH (6)


Table 3: Although the national champions would not have been affected, the remainder of the consensus top
five NCAA Division 1-A college football teams for the 1994–2003 seasons often would have been adjusted
had a minimum-violations restriction been enforced (consensus rankings are in parentheses). I used Massey’s (2004b, c) consensus rankings for 1996–2003 and the final Associated Press poll in 1994 and 1995. Massey did not report a consensus ranking for 1994or 1995.

Coleman also examined:

"whether the participants in any of the BCS national championship games from 1998 through 2003 would have been altered by a minimum-violations restriction."

The answer is yes, as this table by Coleman shows:

............Minimum
............number of........Actual BCS..................................Participants with
Year.....violations........participants................................MinV adjustments


1998.....41...................Tennessee, Florida St...................Tennessee (1), Florida St. (2)
1999.....58...................Florida St., Va. Tech.....................Florida St. (1), Va. Tech. (2)
2000....51...................Oklahoma, Florida St....................Oklahoma (1), Washington (4)
2001....51...................Miami (FL), Nebraska...................Miami (FL) (1), Oregon (4)
2002....50...................Ohio St., Miami (FL).....................Ohio St. (1), Miami (FL) (2)
2003....59...................Oklahoma, LSU............................Oklahoma (1), LSU (2)


Table 4: The Bowl Championship Series (BCS), begun in 1998, would have
selected different participants in its national championship games in 2000
and 2001 if it had adjusted its final ranking to adhere to the minimum
number of violations (actual BCS rankings are in parentheses).


But note that the retrodictive approach is not a cure-all for ranking systems.

As Coleman himself admits:

[T]he MinV ranking may not make the best predictions of future games; as Martinich (2002) demonstrated, retrodictive accuracy does not necessarily result in predictive accuracy. However, no prior researcher has assessed the relationship between these two perspectives, because none have developed systems that optimize the retrodictive component (as measured by violation percentage) for college football. MinV allows that as a future research objective.

The entire article is definitely worth a read here.

LawPundit Post Archive

The ISandIS Network

Our Websites and Blogs: 3D Printing and More 99 is not 100 Aabecis AK Photo Blog Ancient Egypt Weblog Ancient Signs (the book) Ancient World Blog AndisKaulins.com Anthropomorphic Design Archaeology Travel Photos (blog) Archaeology Travel Photos (Flickr) Archaeo Pundit Arts Pundit Astrology and Birth Baltic Coachman Bible Pundit Biotechnology Pundit Book Pundit Chronology of the Ancient World Computer Pundit DVD Pundit Easter Island Script Echolat edu.edu Einstein’s Voice Energy Environment and Climate Blog Etruscan Bronze Liver of Piacenza EU Laws EU Legal EU Pundit FaceBook Pundit Gadget Pundit Garden Pundit Golf Pundit Google Pundit Gourmet Pundit Hand Proof HousePundit Human Migrations Idea Pundit Illyrian Language Indus Valley Script Infinity One : The Secret of the First Disk (the game) Jostandis Journal Pundit Kaulins Genealogy Blog Kaulinsium Kiel & Kieler Latvian Blog LawPundit.com Law Pundit Blog LexiLine.com LexiLine Group Lexiline Journal Library Pundit Lingwhizt LinkedIn Literary Pundit Magnifichess Make it Music Maps and Cartography Megalithic World Megaliths Blog) Megaliths.net Minoan Culture Mutatis Mutandis Nanotech Pundit Nostratic Languages Official Pundit Phaistos Disc Pharaonic Hieroglyphs Photo Blog of the World Pinterest Prehistoric Art Pundit Private Wealth Blog PunditMania Quanticalian Quick to Travel Quill Pundit Road Pundit Shelfari SlideShare (akaulins) Sport Pundit Star Pundit Stars Stones and Scholars (blog) Stars Stones and Scholars (book) Stonehenge Pundit The Enchanted Glass Twitter Pundit UbiquitousPundit Vision of Change VoicePundit WatchPundit Wine Pundit Word Pundit xistmz YahooPundit zistmz