The Global StarCraft League (GSL) represents the pinnacle of StarCraft II competition for the last ten years. Despite its prestige, is it good at determining the better players? This article explores the hidden side of GSL tournament format.
GSL tournament format
Since 2011 July, GSL has been using a mixed tournament format for its main event.
My focus is on the group stage. There are four players in each group. The top two players advance to the next stage, while the bottom two get eliminated. Is the existing format the best way to determine which players should advance to the next stage?
Before I attempt to answer this question, I’m going to first state three tournament format factors that are relevant to this question. I will then highlight an issue with the existing format. Lastly, I will shed light on this issue with available data.
Tournament format trade off
There are virtually countless types of tournament formats. No format is perfect, as each has its pros and cons. There are three considerations that are relevant to this discussion:
- Total number of games
- Minimum number of games played by one participant
- Confidence in determining the better player
Total number of games
The total number of games must fit within the schedule. Some formats have more games than others. For example, once you have at least a certain number of participants, single-elimination and double-elimination have fewer games than round-robin and Swiss-system.
The lower the total number of games, the more consequential each game is. Thus, generally, a format with fewer games punishes a defeat harder than another with more games. For instance, losing a series in single-elimination means you are eliminated, but losing one in round-robin simply has a negative impact on your accumulated score. This factor directly affects how marketable the games are. The current English Premier League table is a good example of how a format (i.e., round-robin) can kill the excitement, because there is almost no way Liverpool is not crowned the champion (see image below). Liverpool basically have to lose all the eight remaining games, while Manchester City win all of their games. Of course, the 25 points gap reflects the difference in performance of the two teams this season. However, in contrast, if the top two teams were to play a one-off elimination game, it would be a lot more exciting. We know that Liverpool are the better team based on the table, but we know there is a decent chance that Manchester City can win it.
Minimum number of games played by one participant
While having fewer and more consequential games makes each game more marketable, a cut-throat format can have a negative impact on the product. Let’s say you are a big Maru fan, and Maru got knocked out in Ro32 in a single-elimination tournament. Your desire to keep up with the tournament drops drastically.
It is also relatively difficult to come up with a story line to market the tournament when half of the participants got eliminated in each round. With Maru eliminated, it is normal to shift the Maru story line to the person who defeated him. “Wow! he defeated one of the best players, is he going to take it all?!” Then, he loses in the next round, now what?
Thus, we often see tournament organisers combine different formats as a compromise. The main event of FIFA World Cup is a good example of finding a compromise, whereby both group stage round-robin and single-elimination are used. Each team has to play a total of three games against the other three teams in the group. Even if a team loses every single game, it plays three games before exiting the tournament. Imagine fans flying all the way to the host country to see their teams lose one game and go home. That makes a poor product.
Confidence in determining the better player
Some formats are better than others at determining standing of the players. For instance, a Swiss-system is better than single-elimination in identifying who is second, third, fourth etc. Imagine you have Serral, Maru, and six amateurs in a single-elimination format, and Serral and Maru are matched in ro8. One of them is the champion, and the other is ranked 5th-8th. Clearly, the loser of that series is the second best player in the tournament, but the results do not reflect it. Put it in a different way, one has relatively low confidence in drawing the conclusion that the silver medalist is the second best player in the tournament. Moreover, you cannot identify the positions of all eight players.
Confidence in determining the relative positions of players is very important in the GSL group stage, and this brings us to the next part.
Ranking the four players
The four players in a GSL group are ranked at the end of the five series. The first and second players advance, while the third and fourth players get eliminated. Using a double-elimination format, the players essentially need to win two series to advance, and the players play a minimum of two series. This seems to fit the bill of not having too many games, having a decent minimum number of games played by one player, and identifying the ranking of the four players.
However, I have always wondered whether the format is good at discriminating between the second and third place players. The positioning of second versus third is determined in the fifth series. The winner is the second place who moves to the next round, while the loser is the third place who gets eliminated. This single do-or-die series ignores the earlier games played in the group. The loser of the fifth series might have defeated the winner of the fifth series earlier, so their head-to-head series score is 1-1. Can we confidently conclude that, based solely on the results of the five series in the group, the second place is objectively better than the third place?
The image below should help to clarify my point. The results show three different ro32 groups in the current GSL season:
- Group A: The second and third place players are 1-1 in head-to-head series score.
- Group B: The second and third place players did not play each other before the fifth series.
- Group C: The second place player defeated the third place player previously, so their head-to-head series score is 2-0.
We can confidently conclude that the rankings of Group B and C reflect the performance of the players, whereby the second place performed better than the third place. In Group A, however, can we claim that the second place performed better than the third place? They both performed better than the fourth place but worse than the first place, but it is difficult to judge who should be second place.
A > B
C > D
C > A
B > D
B > A
Who is the better player from this result, A or B?
Also, importantly, how do we define better? I understand that fans generally have a ranking internally, and hence, there are “upsets” in tournaments. However, from a meritocratic tournament perspective, the winner is the better player. Then, is the player who advances in second place the second best player in the group or simply the better player of the fifth series?
This leads to a question of whether we should take the earlier series into account. Should the results of the fifth series carry more weight than the earlier series when determining the second best player? The answer of the current format is yes, because all it matters is the fifth series between the two players ultimately. Is the earlier series truly independent of the later series though? Put it in another way, is the result of the earlier series predictive of the fifth series? We can examine this using the data from GSL.
Data and analyses
Since the double elimination group format was only implemented in 2011 July, we only consider Code S seasons from 2012 to 2019. Non-full season tournaments (e.g., Super Tournament) are not included. There are a total of 23 seasons. In these 23 seasons, there were 134 occurrences where the players in the fifth series had played against each other earlier in that group.
Quantitatively, we are examining whether the winner of the fifth series is the same (or different) to the winner of the earlier series. This is essentially a classic hypothesis testing of the coin flipping scenario, whereby you want to test if a coin is fair (i.e., it doesn’t favor either head or tail quantitatively). The outcome is binary, as the winner of the fifth series is either the same or different to the winner of the earlier series. A simple binomial test is sufficient in testing this.
The results show that the outcome of the fifth series and the earlier series is independent (p = .666). The same player won both series in 70 of the 134, and a different player won in later series in 64 of the 134.
Next, we examined if expansion plays a role in the results. A chi-squared test shows that expansion does not matter (p = .572). Given that the number of occurrences in the three expansions is not proportionate (WoL = 30, HotS = 25, LotV = 79), we did the same analysis by splitting the data into three separate expansions. The results show that the outcomes of the fifth series and the earlier series are independent in Wings of Liberty (p = 1.000), Heart of the Swarm (p = .690), and Legacy of the Void (p = .368).
Is there racial asymmetry in this scenario? There are 28 TvZ, 37 PvT, 36 PvZ, and 33 mirror matches. Only non-mirror match ups are included in the analyses. It appears that match up does not affect whether the winner of the repeated series is the same (p = .994). The race of the winner in the earlier series (p = .664) and the race of the winner in the fifth series (p = .648) also do not matter.
Discussion and implication
The data suggests the outcome of the fifth series is independent to the outcome of the earlier series. The expansion, match up, and race are not boundary factors either.
This means, simply judging from the results of the fifth series, we can only claim that the winner of the fifth series is the better player of that series. Independently, it is limited in claiming that the winner of the fifth series is also the second best player in the group. This is not an issue from a meritocracy stand point when the winner of the fifth series also won the earlier series against the same opponent. However, 47.8% of the time, we have different winners in the two series. Therefore, for about half of the time, the current GSL format concludes who the second best player is in the group before the results allow us to confidently make that claim.
This begs the question of whether we should only use the fifth series to determine who advances when the two players have played each other earlier in that group. The obvious alternative is to use a round-robin format instead of double-elimination in the group stage. A round-robin format ensures each series carries equal weight in ranking the players. Round-robin increases the number of series played by one to six, but that is not the main weakness of the format. When players have to play against everyone in the group, there will be possible redundant games. The match between a player who won two series and another who lost two series is not going to affect who advances to the next round. The other weakness is issues with tie breakers. It is possible to have three players with 2-1 in series score (fourth player lost all three) or three players with 1-2 in series score (fourth player won all three).
In fact, GSL was employing a modified round-robin format in the group stage for three seasons in 2011 before the current format. The below image shows how the format played out (click image to enlarge). The first two series in each group are self-explanatory. The third series is always between the winner of the first and the loser of the second, and the fourth series is always between the loser of the first and the winner of the second.
The fifth and occasional sixth series did not follow the typical round-robin rules. Using group G as example, the fifth and sixth series should be TricKsteR vs. FruitDealer and anypro vs. Genius in a normal round-robin format. The match ups in the fifth and occasional sixth series were paired in such a way that we could quickly rank the players. While this modified round-robin format eliminates the potential issue with tie breaker, it is flawed in identifying the best two players in the group. In a normal round-robin format, if TricKsteR defeated FruitDealer and Genius defeated anypro in the fifth and sixth series, we would then have:
- TricKsteR > anypro
- FruitDealer > Genius
- TricKsteR > Genius
- anypro > FruitDealer
- TricKsteR > Fruitdealer*
- Genius > anypro*
- TricKsteR: 3-0
- FruitDealer: 1-2
- Genius: 1-2
- anypro: 1-2
This modified round-robin format selectively paired the players in the fifth round and paired anypro and FruitDealer again. After playing just two series, TricKsteR was already ranked first of the group for winning two out of two, while Genius was already ranked last for losing two out of two. However, the results could be completely different if the order of the first two series was changed. Let’s say, the order was Genius vs. anypro and TricKsteR vs. FruitDealer. We would have this instead with the modified format:
- Genius > anypro
- TricKsteR > FruitDealer
- FruitDealer > Genius
- TricKsteR > anypro
- FruitDealer vs. Genius*
I did not change the results of the first four series (using the full round-robin results above), as I merely shuffled the order of the matches. The GSL system would then select FruitDealer vs. Genius for the fifth series, with anypro eliminated. However, as you can see from the actual results back in 2011, anypro advanced from the group. Therefore, while this modified round-robin format prevents tie breaker situations, it is highly susceptible to the variance of match order.
The reason I brought this old GSL format up is to show that neither a standard round-robin nor a GSL modified round-robin format is clearly more suitable than the current format. Then, how should we fix the issue with the current format?
One solution is to play one additional game if the winner of the fifth series is different to the winner of the earlier series. Using the earlier example (group A of ro32 in the current season), after Bunny had defeated sOs in the fifth series, the two of them should play one more game as the tie breaker. Having just one additional game with the same player is not going to affect the schedule. This should complement the current double-elimination format in determining the second best player in the group.
Do you have a better idea in improving the current format? Leave a comment below.
If you enjoyed this article, I’d love you to share it with one friend. You can follow me on Twitter and Facebook. If you really like my work, you can help to sustain the site by contributing via PayPal and Patreon. You can also support me and enjoy quality tea with a 15% discount at AFKTea by using the “TERRAN” code. See you in the next article!