First shot at a banzuke algorithm

[Ichimawashi 573]

I like to predict sekitori honbasho results bout by bout.  Over the past year I have developed a spreadsheet for recording the predictions and results, setting it up each time with updated ages, ranks, weights and heights for each wrestler down to Ms5w, the top 80.  It's a quick job after the end of the last bout each day to enter the ranks (from which the shikona are looked up from the main list) in order for the next day's torikumi, and have a quick shot and picking winners before I fall asleep.  At the end of Haru I thought why not extend this further and see if I could come up with a formula to predict the next banzuke.  Bearing in mind that this only goes to the top 80, so meteoric rises into Juryo might be missed but drops out of Juryo shouldn't be, this is what I came up with:

Each rikishi has the following data from the most recent tournament (examples given are for Hakuho, Endo, Enho, and Seiro):

Rank (Y1e, M1w, J2w, Ms3w)

Rank Number (1, 11, 46, 76)

Rank Value (1000, 968.96, 929,98, 906.95): the formula here is yz - 1.005 ^ (80 - rank) * rank - bumps.  yz is a constant set so that Y1e always works out to 1000.  rank is the rank number, bumps is 10 if below the lowest yokozuna rank and 17 if below the lowest ozeki rank.  The idea here is to space out the top ranks more than the lower ranks, with major hurdles at the sekiwake-ozeki-yokozuna borders.  There is about 1.5 points between ranks near the top of the banzuke and about 0.7 at the low end of Juryo.

Wins minus non-wins (15, -1, 1, 5)

New Points: (1033.25, 966.95, 931.40, 912.21): the formula here is rankvalue + (wins minus non-wins) * 1.01 ^ (81 - rank number).  A small but adjustable bonus (penalty) for wins (losses) near the top.

So essentially we give each rank a point value and add (wins minus non-wins) times a small bonus factor, then resort and see what we get.  The 1.005 in rank value, and the 1.01 in new points are controlling knobs we can adjust to our liking, and I did a little experimenting to get things to work our the way I thought it should.  Here is the resulting list:

Y1e: Hakuho (from Y1e)
Y1w: Kakuryu (from Y1w)
O1e: Goeido (from O1w)
O1w: Takayasu (from O1e)
O2e: Takakeisho (from S1e)
S1e: Ichinojo (from M4w)
S1w: Tochinoshin (from O2e)
K1e: Aoiyama (from M7e)
K1w: Mitakeumi (from K1e)

M1e: Hokutofuji (from K1w)
M1w: Endo (from M1w)
M2e: Daieisho (from M2e)
M2w: Tamawashi (from S1w)
M3e: Kotoshogiku (from M8w)
M3w: Chiyotairyu (from M5e)
M4e: Myogiryu (from M2w)

M4w: Okinoumi (from M6e)
M5e: Abi (from M6w)
M5w: Takarafuji (from M7w)
M6e: Ryuden (from M11e)
M6w: Shodai (from M3w)
M7e: Kaisei (from M1e)
M7w: Nishikigi (from M3e)
M8e: Asanoyama (from M8e)
M8w: Yoshikaze (from M12w)
M9e: Meisei (from M11w)
M9w: Onosho (from M5w)
M10e: Shimanoumi (from J1e)
M10w: Tomokaze (from M13e)
M11e: Shohozan (from M10e)
M11w: Kagayaki (from M13w)
M12e: Tochiozan (from M4e)
M12w: Yago (from M10w)
M13e: Sadanoumi (from M9e)
M13w: Chiyomaru (from J1w)
M14e: Kotoeko (from M15w)
M14w: Terutsuyoshi (from M14e)
M15e: Daishoho (from M16e)
M15w: Ishiura (from M15e)
M16e: Chiyoshoma (from M17e)
M16w: Enho (from J2w)
M17e: Toyonoshima (from M14w)

J1e: Tokushoryu (from J4w)
J1w: Ikioi (from M9w)
J2e: Takagenji (from J4e)
J2w: Wakatakakage (from J5e)
J3e: Daiamami (from J3w)
J3w: Kyokushuho (from J6e)
J4e: Kotoyuki (from J2e)
J4w: Azumaryu (from J7e)
J5e: Sokokurai (from J7w)
J5w: Gagamaru (from J8e)
J6e: Yutakayama (from M16w)
J6w: Hidenoumi (from J10e)
J7e: Takanosho (from J13e)
J7w: Daishomaru (from J5w)
J8e: Kyokutaisei (from J9e)
J8w: Chiyonokuni (from M12e)
J9e: Tsurugisho (from J6w)
J9w: Chiyonoumi (from J8w)
J10e: Mitoryu (from J12e)
J10w: Hakuyozan (from J3e)
J11e: Aminishiki (from J11w)
J11w: Arawashi (from J12w)
J12e: Tobizaru (from J11e)
J12w: Kiribayama (from J14w)
J13e: Irodori (from Ms1e)
J13w: Churanoumi (from Ms5e)
J14e: Seiro (from Ms3w)
J14w: Akiseyama (from J9w)

Ms1e: Wakamotoharu (from J10w)
Ms1w: Fujiazuma (from Ms2e)
Ms2e: Takanofuji (from J13w)
Ms2w: Ryuko (from Ms1w)
Ms3e: Kotokamatani (from Ms5w)
Ms3w: Jokoryu (from Ms3e)
Ms4e: Kaisho (from Ms4e)
Ms4w: Kotodaigo (from Ms4w)
Ms5e: Kotokuzan (from Ms2w)
Ms5w: Daiseido (from J14e)

 

Not bad for about an hour's work with the existing data.  We have the three promotions into Juryo, the sanyaku as expected, and three promotions from Juryo that could easily be five with some tweaking: ranks 38-45 are within a few points of one another.  The thing that the spreadsheet doesn't do well is resolve banzuke luck outliers: Chiyoshoma going up two spots from the M17e with a 7-8 because so many above him are dropping faster seems like something the banzuke committee will try to smooth out, to name just one example.  I don't know a lot about how big the moves are as many banzuke experts do.  And I suspect that the banzuke makers consider more than just the last basho in close cases, to ensure that no rikishi gets a succession of banzuke luck over several basho.  But it seems to be a decent start and maybe I can work at this and make it better.
 

 

[Jabbamaru 166]

Its possible to get promote after a makekoshi?

[Randomitsuki 2,826]

Nice idea! You should validate your system by „predicting“ banzuke from the past. You could also look how your algorithm fares against the best players in the Guess the Banzuke sumo game!

Edited March 28, 2019 by Randomitsuki

[Kintamayama 45,140]

26 minutes ago, Randomitsuki said:

Nice idea! You should validate your system by „predicting“ banzuke from the past. You could also look how your algorithm fares against the best players in the Guess the Banzuke sumo game!

This never works.　Promoting Kotoekou and Daishouhou with a makekoshi ? Seriously? Terutsuyoshi and  Ishiura with a 6-9 move half a rank down? Banzuke luck and all but this is not happening.

Edited March 28, 2019 by Kintamayama

[Gurowake 4,053]

11 minutes ago, Jabbamaru said:

Its possible to get promote after a makekoshi?

That's the main problem with a system like this.  No matter how much it makes sense that someone should get promoted with a 7-8 (or 3-4), they have only done that during the yaocho scandal fallout.

[Kintamayama 45,140]

41 minutes ago, Ichimawashi said:

 

Chiyoshoma going up two spots from the M17e with a 7-8 because so many above him are dropping faster seems like something the banzuke committee will try to smooth out,

 

Never happened, never will, without a mass retirement . Best that can happen to a makekoshi rikishi is he saves his rank.

[Ichimawashi 573]

Good points from all.  It was clear to me from the start that at some point in the process there would need to be intervention.  I don't see a way around this drawback through programming: a stopgap function might work to prevent MKs from going up and KKs from going down, until there is a weird set of results where such limits result in bigger problems.  I was too focused on promotions and demotions to/from Juryo and within sanyaku and didn't look closely enough at the MKs and KKs.  Back to the drawing board!

[Pandaazuma 1,310]

4 hours ago, Ichimawashi said:

Good points from all.  It was clear to me from the start that at some point in the process there would need to be intervention.  I don't see a way around this drawback through programming: a stopgap function might work to prevent MKs from going up and KKs from going down, until there is a weird set of results where such limits result in bigger problems.  I was too focused on promotions and demotions to/from Juryo and within sanyaku and didn't look closely enough at the MKs and KKs.  Back to the drawing board!

Nice attempt! As pointed out by others, certain demotions/promotions will never happen outside of emergencies like the yaocho scandal, so it would be necessary for you to write limits into your algorithm. For example, the minimum a 6-9 lad can go down is 1 full rank, I think. Half a rank would not happen on an ordinary banzuke. If there is a space to be filled half a rank down, it will have to come from someone higher up with a worse record, or someone lower down who will be the recipient of banzuke luck. On the database you can search to find the historic minimums for each record.

And as Rando says, you can tweak the effectiveness on past banzuke. And test it real-time in the GTB game. I just entered my picks and briefly compared them with yours above and it looks like we have quite different predictions.

[Eikokurai 3,437]

5 hours ago, Jabbamaru said:

Its possible to get promote after a makekoshi?

In jonokuchi it’s pretty normal!

[Jakusotsu 5,969]

Reminds me of my old self 16 years ago.

Before making my first GTB entry in 2003, I developed a semi-automated Excel sheet based on historical data (which I can't even remember anymore where I got it from - was it banzuke.com?). That immediately earned me 3rd place, and I really thought I crafted the philosopher's stone.

Of course it turned out to be a massive fluke, and I wisened up soon enough to abandon the whole idea.

[Akinomaki 40,522]

3 hours ago, Pandaazuma said:

it would be necessary for you to write limits into your algorithm. For example, the minimum a 6-9 lad can go down is 1 full rank, I think. Half a rank would not happen on an ordinary banzuke. If there is a space to be filled half a rank down, it will have to come from someone higher up with a worse record, or someone lower down who will be the recipient of banzuke luck.

You need 2 limitations:

a makekoshi can't be promoted except from west to east

6-9 may stay at the same rank in that way: http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m3&form1_wins=6&form1_losses=9&form2_rank=m3

a kachikoshi can't be demoted except from east to west

I also wrote me a banzuke prediction in hypertalk on my mac back in 1992, but you soon end up to try to add compensation rules for the real banzuke movements that happen and the results still won't get better.

Edited March 29, 2019 by Akinomaki

[Jakusotsu 5,969]

I just finished my first draft for the next GTB entry, and every algorithm from M14 down is bound to fail, no matter how sophisticated.

[Kintamayama 45,140]

2 hours ago, Akinomaki said:

You need 2 limitations:

a makekoshi can't be promoted except from west to east

6-9 may stay at the same rank in that way: http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m3&form1_wins=6&form1_losses=9&form2_rank=m3

 

Can you give us an example of a makekoshi that was promoted from west to east in the last 50 years in Makuuchi, not counting the scandal? The 6-9 demotion last happened  48 years ago and it's pretty safe to say it won't happen again soon in normal circumstances.

[Akinomaki 40,522]

23 minutes ago, Kintamayama said:

Can you give us an example of a makekoshi that was promoted from west to east in the last 50 years in Makuuchi, not counting the scandal? The 6-9 demotion last happened  48 years ago and it's pretty safe to say it won't happen again soon in normal circumstances.

I can't, these are just the rules, examples are all from the distant past. But we often have circumstances which are not normal - and the banzuke makers change their ways all the time - who knows if they won't go back to ancient customs.

[Kintamayama 45,140]

5 minutes ago, Akinomaki said:

I can't, these are just the rules, examples are all from the distant past. But we often have circumstances which are not normal - and the banzuke makers change their ways all the time - who knows if they won't go back to ancient customs.

That can be said about almost anything..　Isn't it best to say no way a makekoshi gets promoted?　It's like saying A Sekiwake with an excellent zensho should be able to become Yokozuna the next basho because who knows what can happen..

[Tsuchinoninjin 1,269]

It's a really good try for a first whack at this kind of thing. It really helps to have an iterative algorithm with some kind of penalty function for really ugly banzuke situations where's there no 'perfect' answer.

 

It's also really difficult to do all of the divisions at once because the rules for promotion over boundaries are a bit different compared to inner division ones

[Akinomaki 40,522]

2 hours ago, Kintamayama said:

That can be said about almost anything..　Isn't it best to say no way a makekoshi gets promoted?　It's like saying A Sekiwake with an excellent zensho should be able to become Yokozuna the next basho because who knows what can happen..

is it ? Has that happened?

The rules I mentioned have all several examples in the (distant) past

Edited March 29, 2019 by Akinomaki

[Kintamayama 45,140]

32 minutes ago, Akinomaki said:

is it ? Has that happened?

The rules I mentioned have all several examples in the (distant) past

Can you point me to them?

[Shinrei 47]

4 hours ago, Akinomaki said:

the banzuke makers change their ways all the time

Is there a summary online, in English, of what's known about the process?

[Jabbamaru 166]

12 hours ago, Eikokurai said:

In jonokuchi it’s pretty normal!

For Hattori's luck!

[Akinomaki 40,522]

2 hours ago, Kintamayama said:

Can you point me to them?

• a makekoshi can't be promoted except from west to east - 7 cases

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m1w&form1_wins=7&form1_losses=8&form2_rank=m1e

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m4w&form1_wins=7&form1_losses=8&form2_rank=m4e

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m6w&form1_wins=7&form1_losses=8&form2_rank=m6e

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m9w&form1_wins=7&form1_losses=8&form2_rank=m9e

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m10w&form1_wins=7&form1_losses=8&form2_rank=m10e

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m13w&form1_wins=7&form1_losses=8&form2_rank=m13e

• 6-9 may stay at the same rank in that way:

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=m3&form1_wins=6&form1_losses=9&form2_rank=m3

you have to change the rank in the search to find them all, I'd say more than 10 examples

• a kachikoshi can't be demoted except from east to west

this only is done with sanyaku, but is quite common

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=k1e&form1_wins=8&form1_losses=7&form2_rank=k1w

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=s1e&form1_wins=8&form1_losses=7&form2_rank=s1w

http://sumodb.sumogames.de/Query.aspx?show_form=0&form1_rank=s1&form1_wins=8&form1_losses=7&form2_rank=s2

[Sue 508]

The fact that GTB exists makes this an interesting case for trying reinforcement learning. Is there an easy way to get a full dump of sumodb, or is one limited to just doing queries against it?

Edited March 29, 2019 by Sue

[Tsuchinoninjin 1,269]

I think the issue with that is the banzuke gets made differently during different time periods, so training against that will just result in the average action which isn't going to win many GTB especially since there are some people who have figured out the current committees more nebulous preferences.

 

It's really easy to find queries that have contradictory results, like figuring out from how many ranks back a 9-6 will pass a 8-7 or a 6-9 dropping down below a way lower ranked 10-5 for example. These depend on what's going on around other ranks sometimes so a penalty function may be more appropriate.

[Benihana 1,929]

And then you have to count in the "we don't like that guy, so we don't promote him, albeit the numbers deserve him promotion" stuff and vice versa. But nonetheless nice effort!

Edited March 29, 2019 by Benihana

[Ryoshishokunin 288]

2 hours ago, Akinomaki said:

this only is done with sanyaku, but is quite common

This is theoretically a misunderstanding of what's going on--the san'yaku ranks are "supposed" to be substantively equal on both sides, so it's "not" a dmeotion to move someone from E to W if the other guy is moving from W to E.  This is what happens with the Ozeki and Yokozuna every basho, and happens in the lower sanyaku iff there are a bunch of KKs in those ranks.