Post a reply

ELO Rankings for Snooker

Postby elonmerkki

I have been lurking here for a bit, might as well my introductions now :hatoff:

As a computer nerd someone's (snookerfan maybe?) mention of cuetracker caused me to calculate ELO-ratings for all of snooker history. ELO-ratings calculated by snookeranalyst.org are done based on winning frames, not matches, which makes no sense to me. That means that just winning a match is not enough, a high-rated players must always win by a large frame margin to improve.

First of all, current ELO-ratings for top 64 and their ELO-history for four years. The rectangle at top-left shows an approximation how their rating has changed recently (red=bad, green=good).

https://www.dropbox.com/s/kngtkxq7u51bijf/snooker1.png

And then then the history. The graph shows snooker ELO ratings for every year since 1970 and the name for who is leading after the world championship.

https://www.dropbox.com/s/19d37qofw7hi8ki/snooker_history.png

That should be fun for any Ronnie fan :sad:

It also shows that the snooker quality has stayed at about the same level for 20 years, the best years being 2000-2005.

Re: ELO Rankings for Snooker

Postby Skullman

Interesting topic.

How do you calculate these ELOs (I vaguely know that beating players around the same as you increases scores, beating lesser players increases it by less and losing to lesser players causes a sharper decrease)?

And if it's based solely on match wins, I'm not sure how you can justify how the standard is constant, as you don't take into account the level the player has played through just the wins.

Re: ELO Rankings for Snooker

Postby elonmerkki

ELO-ratings have been calculate very much the same way as described in wikipedia, http://en.wikipedia.org/wiki/Elo_rating_system.

Beating any player increases the rating, but beating a higher rated rated player increases it more than beating a lower rated player.

Of course the ELO-rating is comparable only to all players currently (and in the past) playing, so saying anything really definitive about the past is not possible.

Re: ELO Rankings for Snooker

Postby shankly

Didn't want to hijack the German masters qualies thread.. Do you have a list of the players and their current ELO score by any chance?

Re: ELO Rankings for Snooker

Postby elonmerkki

An updated list with top 64 players is in url

Players with most increased/decreased ratings since the beginning of current season are here (including results from german masters qualifiers so far).

I think the Selt fanboys will really like to see this :bowdown:

1: Matthew Selt, 1561.70/+121.12
2: James Cahill, 1315.95/+117.62,
3: Fraser Patrick, 1365.86/+115.01
4: Oliver Lines, 1415.87/+101.80,
5: Yan Bingtao, 1313.03/+93.94

1: Tom Ford, 1390.32/-85.36,
2: Kurt Maflin, 1453.96/-79.16
3: John Higgins, 1587.72/-74.31
4: Kyren Wilson, 1445.32/-73.52
5: Dave Harold, 1352.77/-73.05

Re: ELO Rankings for Snooker

Postby elonmerkki

Yeah, it is all Matlab, including the parser to get new scores from http://livescores.worldsnookerdata.com/. And Matlab is not really meant for that <cool>

A couple of points about the ELO-ratings, in case anyone cares:

The ratings don't change if someone doesn't play, so players who have not played as much as others may have unrealistical ratings. That might explain for example why Robin Hull is so highly ranked. He had a good rating when he had to quit and after his comeback hasn't played all that many tournaments, so his current rating is a bit suspect.

Difference in ratings gives the odds how likely the player wins another player. If someone has 100 higher rating than another player he should win 64% of the time, for 200 difference 76% of the time, 300=85% and 400=91%. The lowest rated player has now about 960 points, so if he were to play the highest rated player (ROS with almost 1900) he should have 0.4% chance of winning.

And because you didn't ask the ratings for tennis look quite similar, although after the top the ratings drop quite sharply unlike in snooker,

1. Djokovic, Novak, 1880
2. Federer, Roger, 1812
3. Nadal, Rafael, 1791

If someone knows a source to get historical data for football (all results from european leagues, champions league, etc), I'd be interested ;-)

Re: ELO Rankings for Snooker

Postby elonmerkki

And continuing while nobody cares, the highest ever rated players are (numbers are their highest rating and year/month when it happened):

1 Ronnie O'Sullivan 1951 2005/05
2 Mark Williams 1925 2003/11
3 Stephen Hendry 1906 1995/12
4 John Higgins 1884 2001/10
5 Paul Hunter 1840 2004/03
6 Ding Junhui 1831 2014/03
7 Stephen Lee 1822 2002/09
8 Ken Doherty 1814 2002/01
9 Steve Davis 1813 1994/05
10 Neil Robertson 1811 2014/01

So Hendry is not even the second best snooker player of all time. Joking aside, comparing ratings between different years is not really possible (http://en.wikipedia.org/wiki/Elo_rating_system and check the inflation/deflation bit) and a small mistake in the data, for example changing the order of two tournaments, could change the above list.

Anyway, I was quite surprised to see Paul Hunter, Stephen Lee and Ken Doherty so high on that list. Too bad two of them had to end their careers early.

Re: ELO Rankings for Snooker

Postby McChazzz

A few more comments about Elo ratings:

1) A key factor in the accuracy of the numbers is what multiplier you use when adjusting the ratings. If Matt Selt beats Ronnie, does that make Selt better than Ronnie? No, but after he'd won (say) ten in a row, you might think differently. Using too low a multiplier will mean that improving players like Cahill/O.Lines are never allowed to catch the top dogs, using too high a multiplier means that the ratings are too variable on recent form and don't reflect true ability.

2) How do you deal with unrated players? Picking the "correct" number to introduce them on has knock-on effects on other ratings and is a cause of inflation/deflation.

3) Using matches won instead of frames won makes the ratings highly arguable. For example, you say a player with a rating 100pts higher should win 64% of the time, but as the match length increases, so should that probability. Ronnie is about 80% to beat Selt in a best of seven PTC, but increases to more like 90% in a Crucible best of 19.

Further on this point; under the above methodology any win will increase your rating. However, I think it's fair to say that a player's rating can actually DROP after a win (and indeed, you can gain points from a loss). Take Mitchell Mann's performance against Neil Robertson last night; bookmakers estimated his probability of winning the match at around 6½%, and technically he underperformed that expectation by 6½%. However, they also estimated his probability of winning any given frame at 27%, and he outperformed that admirably, winning 44.44% of frames played. If Ronnie were to beat Ahmed Saif 10-9, it would be counter-intuitive to lower Saif's rating.

Re: ELO Rankings for Snooker

Postby elonmerkki

Nice that someone cares :-D

1. The ELO-ratings work so that beating a low-rated player increases the rating only a little, while beating a high-rated player increases it a lot. The corresponding increase/decrease is always the same, so when Neil Robertson won Mitchell Mann he got only 1.9 points and Mitchell lost 1.9, while Graeme Dott losing to Michael Georgiou meant that he lost 21.8 points and Michael won 21.8. Losing to low-rated players can decrease the rating quite quickly, look at John Higgins for example.

2. All players start with rating 1200. That's arbitrary, but so is anything :-)

3. I agree, the match length should have an effect. I actually use that as factor in the sense that ELO-rating changes less in PTCs (and even less in shorter championship league etc matches). But I don't want to over-emphasize that, otherwise winning the only tournament with long matches, the world championship, would have too much weight.

About the last point that rating going up even when losing I think the way I do it is fair. The rating loss is very small when a player loses to a high rated player, and on the other hand a win is always a win. If counting just frames a high-rated player loses always points unless he wins by a big margin, and that's bulls*t.

Re: ELO Rankings for Snooker

Postby McChazzz

I understand that Elo (it's lowercase; invented by Arpad Elo) is zero-sum, I was just making the point that the rate of increase/decrease is a key factor in what the numbers mean. By extrapolating out it appears you're using a multiplier of 25, which means Georgiou would have to beat Graeme Dott in eight consecutive best of nine matches before you considered it a 50-50 matchup. I'm not saying that's right or wrong, just that using 25 as opposed to 2/10/50/other will yield either convergent or divergent results.

As you say, everyone has to start somewhere, and 1200 may seem as good as anywhere. But is an unknown amateur 4/1 to beat Matthew Stevens? I'd argue not; it may be an arbitrary number, but you can still make it as accurate as possible. Also as I said, where you pitch that number will cause either inflation or deflation, so finding the "correct" starting point is important if you want to compare present and past ratings.

I personally think using the formula on frames won is the only way forward; high-rated players *should* be beating lower ones by large margins, not scraping through in deciders. Anyone surprised by the relatively poor form of Robertson and Ding Junhui this season should look at how many close matches they won by the skin of their teeth last year. They still come out on top of the rating tree, just not by as far as your method would suggest.

...and in the end, it's all just playing with numbers. I'd be fascinated, really fascinated, to find out what players on tour think their chances are of winning a match (or of others winning theirs) prior to the breakoff, and comparing it to what the betting markets think.

Re: ELO Rankings for Snooker

Postby elonmerkki

The factor is indeed 25, decided by running through historical data and checking which factor gives the best predictions. So any number works, but at least that is not completely random.

I don't think that rating new players is too much of a problem. If a new players enters the circuit and is way better than expected, then he will win a lot of matches and his rating will improve quickly. Some older players will lose more points than they "should" because the player was underrated, but *shrug*. Matthew Stevens btw is currently at 1442 points, so he is about 80% favorite against a new player at rating 1200.

Inflation or deflation happens only when players enter and then quit, and give up or take away some of their 1200 starting points. So Stephen Hendry quitting took some points away from the pool of total available points, but on the average it seems like most players quit with the same points they started with.

Both Robertson and Ding have gone down in the ratings (as well as Selby), so it seems to be working fine. I still think that any win is better than not winning.

Re: ELO Rankings for Snooker

Postby McChazzz

The inflation/deflation can happen from players like Hendry leaving, but it's more easily seen with poor players in the APTC, which ties in to the problem of rating new players. Zhang Hanxuan, Xiong Jun, Wang Shi, Gao Le, Chen Bo and others are all far worse than a 1200 rating would suggest, and will have inflated their opponents ratings, unlikely to ever return to the pool again.

I'm only suggesting things to help the accuracy of the numbers, but your current ones are certainly a reasonable reflection if you're not too fussed about 100% precision. A good read!

Re: ELO Rankings for Snooker

Postby elonmerkki

The image linked somewhere upthread, https://www.dropbox.com/s/19d37qofw7hi8ki/snooker_history.png, shows the average rating of current players which is directly linked to inflation/deflation. The average changes from 1200 only when players leave with more or less points than 1200, so it looks like it has been quite stable.

But anyway I am pleased about your insightful comments :spot on:

Re: ELO Rankings for Snooker

Postby Dan-cat

I'm late to the party.

Wow, how did I miss this, absolutely brilliant. Elonmerrki that is some killer work you did there. As a fellow techy I'm pretty blown away by what you've done. I'm still trying to get my head around it!

Dan

Re: ELO Rankings for Snooker

Postby shankly

The frames vs match wins debate is a tough one. I suppose in a perfectly controlled environment where a player's performance is fairly constant, doing it by frames obviously makes the most sense as Mcchazz has rightly reasoned.

However, the truth is players often have wild swings in performance. If I'm Robbo or whatever, ultimately I don't care whether I win 5-0 or 5-4. You often see the top boys playing down to the level of their opposition, whether they're conserving energy, or lack of focus, or whatever. Say Judd plays Mcleod and doesn't feel threatened at all, plays some loose shots and wins 5-2. Then he plays Bingham, knows he's dangerous and tightens up a bit and raises his game. Bingham also plays very well and takes his chances, and Judd wins 5-2. Surely you can see how they are not the same? To reflect their ability accurately, Bingham deserves more credit for taking two frames off a peak Trump, McLeod less for two frames off an unfocused Trump.

Obviously the example is very specific and it's impossible to account for all the different scenarios.. and as such maybe the most prudent approach is Elonmerkki's -- in which ultimately you judge the players based on what they are valuing, i.e. match wins.

Haha sorry, I haven't expressed myself very well but I think you can get the gist of what I'm trying to say. In any case, kudos to anyone putting in the effort to generate Elo/alternative rankings; always interesting to see!

EDITED TO ADD: Somewhat relevant is the all time stats for % of matches and frames won, which you can see on cuetracker... Obviously over such a large sample size you'd expect the ranking to be the same - the person with the highest match win % also has the highest frame win % (no surprises there, it's Ronnie). However the margins separating the players are significantly smaller in the frame win %, which I guess both of you have noticed in your Elo rankings as well. My gut reaction in this case is to lean towards the system which gives you a greater degree of separation (i.e. doing it by match wins)... but I'm sure if I thought about it enough I can convince myself of the opposite!

Re: ELO Rankings for Snooker

Postby McChazzz

The way I would respond to the McLeod/Bingham example is twofold. Firstly, it's tough or even impossible to know a player's mental situation at any given time. It's easy to speculate, but people write the narrative that fits with what they see ("he's started to loosen up") and change it repeatedly throughout a match based on events that occur. Even players in post-match pressers can be led to answer questions as if they thought one thing, whereas during the match they might have been thinking quite another. To have an impartial rating system, you need to treat each frame individually from a standing start, with no assumptions.

How many people thought it was impossible for Bond to beat Hawkins from 5-0 down in the UK just gone? "The momentum is with Barry, Nigel just wants to get out of there", etc, etc. It wasn't impossible, it was just a very big price to occur. Similarly, at 5-5 the commentators announced there was no way Hawkins could win (and the betting markets showed similar bias), forgetting the insane Hawkins - Dale World Championship Quarter Final where Dale had an even bigger comeback, only for Hawkins to get over the line anyway. As soon as you try to stamp your narrative on a match, you've lost impartiality.

The second point is that if Trump has in his game the tendency to go for loose shots some of the time, and lose frames because of it, then his rating *should* be lower than if he consistently plays the right shot. That will be how he sometimes loses matches to a McLeod or a Joyce, and it should be reflected in his rating. Each frame is a mini-match, and ultimately your ability to win matches depends on them.

Re: ELO Rankings for Snooker

Postby elonmerkki

I just checked what odds my script gives for Masters based on the Elo-ratings and the tournament structure and it does not look too different to what for example oddschecker says. The numbers are decimal betting odds.

1, Ronnie O'Sullivan, 2.88
2, Judd Trump, 10.48
3, Ding Junhui, 10.74
4, Neil Robertson, 11.98
5, Mark Allen,13.67
6, Mark Selby, 17.70
7, Stuart Bingham, 19.76
8, Shaun Murphy, 19.93
9, Stephen Maguire, 27.28
10, Barry Hawkins, 36.76
11, Ricky Walden, 54.02
12, Marco Fu, 60.05
13, Allister Carter, 60.48
14, Joe Perry, 61.04
15, John Higgins, 98.70
16, Robert Milkins, 116.97

Maybe I should make some money with this :-)

Re: ELO Rankings for Snooker

Postby McChazzz

1. Ronnie O'Sullivan, 3.202
2. Judd Trump, 7.625
3. Neil Robertson, 11.926
4. Ding Junhui, 12.613
5. Shaun Murphy, 13.528
6. Mark Selby, 14.175
7. Stuart Bingham, 17.004
8. Mark Allen, 23.898
9.Barry Hawkins, 39.415
10. Stephen Maguire, 41.539
11. Ali Carter, 44.013
12. John Higgins, 53.665
13. Joe Perry, 55.841
14. Ricky Walden, 66.867
15.Marco Fu, 67.824
16.Robert Milkins, 101.537

There's mine. One thing we can probably both agree on is that the market underrates Stuart Bingham!

Re: ELO Rankings for Snooker

Postby Dan-cat

Just to clarify, as I'm a little confused. Are the ELO ratings in the graphic based on frames won or matches won?

Re: ELO Rankings for Snooker

Postby shankly

The graphics in this thread are made by Elonmerkki and are based on matches won.

I'm guessing Mcchazzz owns the snookeranalyst website or supports that approach, which generates Elo rankings based on frames won.

EDIT: Sorry, don't think Mcchazzz owns the snookeranalyst site! Don't think his Elo rankings are available...? SnookerAnalyst does generate frame-based Elo rankings though.

Re: ELO Rankings for Snooker

Postby Dan-cat

Thanks. Surely matches won is the only thing worth counting? True champions raise their game when in trouble.

Re: ELO Rankings for Snooker

Postby shankly

Yes, it's a tough one to call but that's what I'm leaning towards as well. I do think however that Mcchazz's odds are probably a better representation of the players form, but that's only part (albeit a big one) of the equation when deciding how likely someone is to win a match. We all know if it comes down to the wire form goes out the window and it's who deals with the pressure best that counts.

Re: ELO Rankings for Snooker

Postby McChazzz

"True champions raise their game when in trouble" is a contentious, narrative-stamped phrase (i.e you tend to forget the times when even the best have just capitulated in matches), but I'm willing to agree with it here. Why? Because that is actually an argument in favour of using frames won as opposed to matches. If a true champion wins the matches he should win, and then raises his game in matches others would lose, that will be reflected in winning more frames than his peers.

Both routes (matches and frames) will lead to very similar numbers, as shown by the passes made at outright prices for the Masters by elonmerkki and myself. The difference is that frames won is using more of the information you have to hand than simply 'Player A won, Player B lost'. Think of the number of fine margins there can be in a frame of snooker. A good example might be the recent Allen - Williams semi-final at the International Championship; using elonmerkki's numbers, Allen will have gained ten or so points for his win, WIllo will have lost the same, and the next time they face Allen will be expected to win 65% of the time (up from 62%).

But what if Willo had made the do-or-die red and won the decider? Why, he'd have gained 15 points, Allen lost the same, and the next time they played Allen would have been about a 58% favourite. Do you really believe that one shot was the difference between 65% and 58%? By using frames won instead, as memory serves the two ratings remained unchanged (Allen may have actually lost a fraction), which is a much better reflection of the match.

I say again, consider how many fine margins there are in a single frame of snooker. The kick you did or didn't get, that did or didn't send the object ball off-line. The last red that wiped it's feet and dropped/stayed up. The incredible, mind-blowing, could-not-be-repeated-if-you-tried in-off after you went into the bunch. The fluke that kept your clearance alive. If you strip out all of these things and boil it down to just who won in the end, you have a) something that reads like a World Snooker report and b) a very simplistic approach to the data.

From a technical standpoint, snooker is very difficult to model as there is only one set 'state', the one at the start of each frame. Whereas in baseball you can talk with some certainty about the probabilities with (say) a man on first and two outs, in snooker everything goes out the window as soon as the white strikes the reds. You can come up with convincing estimates for (say) how often a player clears the colours, their long-potting percentage or their ability to win from snookers behind, but they will be only estimates - it can vary wildly based on a single ball being a few centimetres off! Effectively then, we can only talk about how proficient a player is at converting the starting state into a win or a loss; everything else is conjecture, albeit quite enjoyable conjecture!

I'm not Snookeranalyst (I've had a look and agree with a lot of his stuff, though), but I do firmly believe that how good a player is at winning a frame is a better indication of his abilities than his matches won. You can find some players who have surprising differences between the two, and this frequently indicates a player that cannot deal well with pressure situations, which is normally the first counter-argument. However, dealing with pressure is often just a matter of repetition. Recent players to fit the description? Barry Hawkins and Stuart Bingham. Hawkins consistently buckled under pressure, before his Shootout win gave him some confidence; he turned it into the Australian Open title, and two Crucible losses to Ronnie at the final/semi-final stage. Bingham has been even more successful (though not yet in Sheffield), yet still inexplicably struggles to shake the 'journeyman' tag.

In short, if you are really good at winning frames of snooker, you will eventually be really good at winning matches, simple as that sounds. If judging how good a player was involved looking only at their matches won, snooker bookmakers like myself would be out of a job!

Re: ELO Rankings for Snooker

Postby Dan-cat

It really is a great thread and deserves to be bumped.

Re: ELO Rankings for Snooker

Postby TheSaviour

Lots of players seems to be claiming already having Stephen Hendry´s number. It is not just about Ronnie, or a nutter like Peter Ebdon claiming that. Basically everyone with an education is claiming that. It is nothing unusual, really.

Re: ELO Rankings for Snooker

Postby SnookerAnalyst

I am responsible for the ratings on snookeranalyst.com. Sorry for missing the original discussion but I've not dedicated much time to this for the last couple years.

On the frames v matches debate, I think that the one of the main reasons I used frames was that I started developing my system in the pre-Hearn era, when there were only 6 or 7 ranking events a year. For players only completing 10-12 matches in a season this didn't give much to go on, but if these comprised of at least 70-80 frames then it felt like there was some validity to their ratings. Although there are far more tournaments these days, there is still a potential issue around the amount of time it takes players to have completed a 'sufficient' number of matches.

I was also never that interested in developing a rating system in order to predict the results of future matches (i.e. for betting purposes). My interest was more in forming a reasonably reliable picture of the relative ability of different players so that I could then carry out further analysis. For some of this analysis it would be useful to have an estimate of the probability that one player would defeat another in a single frame (e.g. in assessing whether some players are relatively good at winning deciding frames). I haven't actually done anywhere near as much of this analysis I would have liked, but maybe in the future...

Ultimately, I doubt that either system would give radically different results - particularly for the top players. It's just a matter of what you are interested in using the outputs for and which method suits that purpose better.

Thanks,
James