The Global StarCraft League (GSL) represents the pinnacle of StarCraft II competition for the last ten years. Despite its prestige, is it good at determining the better players? This article explores the hidden side of GSL tournament format.
GSL tournament format
Since 2011 July, GSL has been using a mixed tournament format for its main event.
- Ro32 + Ro16: Group based double-elimination.
- Ro8 and later: Single elimination.
My focus is on the group stage. There are four players in each group. The top two players advance to the next stage, while the bottom two get eliminated. Is the existing format the best way to determine which players should advance to the next stage?
Before I attempt to answer this question, I’m going to first state three tournament format factors that are relevant to this question. I will then highlight an issue with the existing format. Lastly, I will shed light on this issue with available data.
Tournament format trade off
There are virtually countless types of tournament formats. No format is perfect, as each has its pros and cons. There are three considerations that are relevant to this discussion:
- Total number of games
- Minimum number of games played by one participant
- Confidence in determining the better player
Total number of games
The total number of games must fit within the schedule. Some formats have more games than others. For example, once you have at least a certain number of participants, single-elimination and double-elimination have fewer games than round-robin and Swiss-system.
The lower the total number of games, the more consequential each game is. Thus, generally, a format with fewer games punishes a defeat harder than another with more games. For instance, losing a series in single-elimination means you are eliminated, but losing one in round-robin simply has a negative impact on your accumulated score. This factor directly affects how marketable the games are. The current English Premier League table is a good example of how a format (i.e., round-robin) can kill the excitement, because there is almost no way Liverpool is not crowned the champion (see image below). Liverpool basically have to lose all the eight remaining games, while Manchester City win all of their games. Of course, the 25 points gap reflects the difference in performance of the two teams this season. However, in contrast, if the top two teams were to play a one-off elimination game, it would be a lot more exciting. We know that Liverpool are the better team based on the table, but we know there is a decent chance that Manchester City can win it.
Minimum number of games played by one participant
While having fewer and more consequential games makes each game more marketable, a cut-throat format can have a negative impact on the product. Let’s say you are a big Maru fan, and Maru got knocked out in Ro32 in a single-elimination tournament. Your desire to keep up with the tournament drops drastically.
It is also relatively difficult to come up with a story line to market the tournament when half of the participants got eliminated in each round. With Maru eliminated, it is normal to shift the Maru story line to the person who defeated him. “Wow! he defeated one of the best players, is he going to take it all?!” Then, he loses in the next round, now what?
Thus, we often see tournament organisers combine different formats as a compromise. The main event of FIFA World Cup is a good example of finding a compromise, whereby both group stage round-robin and single-elimination are used. Each team has to play a total of three games against the other three teams in the group. Even if a team loses every single game, it plays three games before exiting the tournament. Imagine fans flying all the way to the host country to see their teams lose one game and go home. That makes a poor product.
Confidence in determining the better player
Some formats are better than others at determining standing of the players. For instance, a Swiss-system is better than single-elimination in identifying who is second, third, fourth etc. Imagine you have Serral, Maru, and six amateurs in a single-elimination format, and Serral and Maru are matched in ro8. One of them is the champion, and the other is ranked 5th-8th. Clearly, the loser of that series is the second best player in the tournament, but the results do not reflect it. Put it in a different way, one has relatively low confidence in drawing the conclusion that the silver medalist is the second best player in the tournament. Moreover, you cannot identify the positions of all eight players.
Confidence in determining the relative positions of players is very important in the GSL group stage, and this brings us to the next part.
Ranking the four players
The four players in a GSL group are ranked at the end of the five series. The first and second players advance, while the third and fourth players get eliminated. Using a double-elimination format, the players essentially need to win two series to advance, and the players play a minimum of two series. This seems to fit the bill of not having too many games, having a decent minimum number of games played by one player, and identifying the ranking of the four players.
However, I have always wondered whether the format is good at discriminating between the second and third place players. The positioning of second versus third is determined in the fifth series. The winner is the second place who moves to the next round, while the loser is the third place who gets eliminated. This single do-or-die series ignores the earlier games played in the group. The loser of the fifth series might have defeated the winner of the fifth series earlier, so their head-to-head series score is 1-1. Can we confidently conclude that, based solely on the results of the five series in the group, the second place is objectively better than the third place?
The image below should help to clarify my point. The results show three different ro32 groups in the current GSL season:
- Group A: The second and third place players are 1-1 in head-to-head series score.
- Group B: The second and third place players did not play each other before the fifth series.
- Group C: The second place player defeated the third place player previously, so their head-to-head series score is 2-0.
We can confidently conclude that the rankings of Group B and C reflect the performance of the players, whereby the second place performed better than the third place. In Group A, however, can we claim that the second place performed better than the third place? They both performed better than the fourth place but worse than the first place, but it is difficult to judge who should be second place.
A > B
C > D
C > A
B > D
B > A
Who is the better player from this result, A or B?
Also, importantly, how do we define better? I understand that fans generally have a ranking internally, and hence, there are “upsets” in tournaments. However, from a meritocratic tournament perspective, the winner is the better player. Then, is the player who advances in second place the second best player in the group or simply the better player of the fifth series?
This leads to a question of whether we should take the earlier series into account. Should the results of the fifth series carry more weight than the earlier series when determining the second best player? The answer of the current format is yes, because all it matters is the fifth series between the two players ultimately. Is the earlier series truly independent of the later series though? Put it in another way, is the result of the earlier series predictive of the fifth series? We can examine this using the data from GSL.
Data and analyses
Ben did all the heavy lifting in getting the data from Liquipedia, and he is a co-contributor of this article. His programming skill saves me from manually getting the data from Liquipedia.
Since the double elimination group format was only implemented in 2011 July, we only consider Code S seasons from 2012 to 2019. Non-full season tournaments (e.g., Super Tournament) are not included. There are a total of 23 seasons. In these 23 seasons, there were 134 occurrences where the players in the fifth series had played against each other earlier in that group.
Quantitatively, we are examining whether the winner of the fifth series is the same (or different) to the winner of the earlier series. This is essentially a classic hypothesis testing of the coin flipping scenario, whereby you want to test if a coin is fair (i.e., it doesn’t favor either head or tail quantitatively). The outcome is binary, as the winner of the fifth series is either the same or different to the winner of the earlier series. A simple binomial test is sufficient in testing this.
The results show that the outcome of the fifth series and the earlier series is independent (p = .666). The same player won both series in 70 of the 134, and a different player won in later series in 64 of the 134.
Next, we examined if expansion plays a role in the results. A chi-squared test shows that expansion does not matter (p = .572). Given that the number of occurrences in the three expansions is not proportionate (WoL = 30, HotS = 25, LotV = 79), we did the same analysis by splitting the data into three separate expansions. The results show that the outcomes of the fifth series and the earlier series are independent in Wings of Liberty (p = 1.000), Heart of the Swarm (p = .690), and Legacy of the Void (p = .368).
Is there racial asymmetry in this scenario? There are 28 TvZ, 37 PvT, 36 PvZ, and 33 mirror matches. Only non-mirror match ups are included in the analyses. It appears that match up does not affect whether the winner of the repeated series is the same (p = .994). The race of the winner in the earlier series (p = .664) and the race of the winner in the fifth series (p = .648) also do not matter.
Discussion and implication
The data suggests the outcome of the fifth series is independent to the outcome of the earlier series. The expansion, match up, and race are not boundary factors either.
This means, simply judging from the results of the fifth series, we can only claim that the winner of the fifth series is the better player of that series. Independently, it is limited in claiming that the winner of the fifth series is also the second best player in the group. This is not an issue from a meritocracy stand point when the winner of the fifth series also won the earlier series against the same opponent. However, 47.8% of the time, we have different winners in the two series. Therefore, for about half of the time, the current GSL format concludes who the second best player is in the group before the results allow us to confidently make that claim.
This begs the question of whether we should only use the fifth series to determine who advances when the two players have played each other earlier in that group. The obvious alternative is to use a round-robin format instead of double-elimination in the group stage. A round-robin format ensures each series carries equal weight in ranking the players. Round-robin increases the number of series played by one to six, but that is not the main weakness of the format. When players have to play against everyone in the group, there will be possible redundant games. The match between a player who won two series and another who lost two series is not going to affect who advances to the next round. The other weakness is issues with tie breakers. It is possible to have three players with 2-1 in series score (fourth player lost all three) or three players with 1-2 in series score (fourth player won all three).
In fact, GSL was employing a modified round-robin format in the group stage for three seasons in 2011 before the current format. The below image shows how the format played out (click image to enlarge). The first two series in each group are self-explanatory. The third series is always between the winner of the first and the loser of the second, and the fourth series is always between the loser of the first and the winner of the second.
The fifth and occasional sixth series did not follow the typical round-robin rules. Using group G as example, the fifth and sixth series should be TricKsteR vs. FruitDealer and anypro vs. Genius in a normal round-robin format. The match ups in the fifth and occasional sixth series were paired in such a way that we could quickly rank the players. While this modified round-robin format eliminates the potential issue with tie breaker, it is flawed in identifying the best two players in the group. In a normal round-robin format, if TricKsteR defeated FruitDealer and Genius defeated anypro in the fifth and sixth series, we would then have:
- TricKsteR > anypro
- FruitDealer > Genius
- TricKsteR > Genius
- anypro > FruitDealer
- TricKsteR > Fruitdealer*
- Genius > anypro*
- TricKsteR: 3-0
- FruitDealer: 1-2
- Genius: 1-2
- anypro: 1-2
This modified round-robin format selectively paired the players in the fifth round and paired anypro and FruitDealer again. After playing just two series, TricKsteR was already ranked first of the group for winning two out of two, while Genius was already ranked last for losing two out of two. However, the results could be completely different if the order of the first two series was changed. Let’s say, the order was Genius vs. anypro and TricKsteR vs. FruitDealer. We would have this instead with the modified format:
- Genius > anypro
- TricKsteR > FruitDealer
- FruitDealer > Genius
- TricKsteR > anypro
- FruitDealer vs. Genius*
I did not change the results of the first four series (using the full round-robin results above), as I merely shuffled the order of the matches. The GSL system would then select FruitDealer vs. Genius for the fifth series, with anypro eliminated. However, as you can see from the actual results back in 2011, anypro advanced from the group. Therefore, while this modified round-robin format prevents tie breaker situations, it is highly susceptible to the variance of match order.
The reason I brought this old GSL format up is to show that neither a standard round-robin nor a GSL modified round-robin format is clearly more suitable than the current format. Then, how should we fix the issue with the current format?
One solution is to play one additional game if the winner of the fifth series is different to the winner of the earlier series. Using the earlier example (group A of ro32 in the current season), after Bunny had defeated sOs in the fifth series, the two of them should play one more game as the tie breaker. Having just one additional game with the same player is not going to affect the schedule. This should complement the current double-elimination format in determining the second best player in the group.
Do you have a better idea in improving the current format? Leave a comment below.
If you enjoyed this article, I’d love you to share it with one friend. You can follow me on Twitter and Facebook. If you really like my work, you can help to sustain the site by contributing via PayPal and Patreon. You can also support me and enjoy quality tea with a 15% discount at AFKTea by using the “TERRAN” code. See you in the next article!
TerranCraft 
The image of the 2011 groups doesn’t link to a larger version.
Having one extra game as a tie breaker sounds really exciting! Not only is it more fair, but it would also add some drama. They could make it a special thing, like Proleague did with the ace match.
Thanks, I updated the image link!
I could never quite put a finger on this phenomenon. A tie breaking game, yes.
Hi, nice piece.
I do not like a tie breaker as it is like the extended series idea which I never liked. I think the winner of this last chance match deserves to progress.
I semi recall the RR format being used and not really working when 3 players kept beating and losing to each other, paralyze was one of them, but I can’t exactly remember when it was (I think it was to replace a player who had qualified but dropped out)
I see your point, however I do not think the added complexity in the format is worth it.
As far as I understand the format the results indicate that it is a very hard decision to make in most cases. The players are very close in skill and just because one player wins their first encounter is a terrible predictor (as you show) of who wins in the rematch. I am not convinced that an additional game will make us much more sure of who the better player really is.
I can see why an additional game is not appealing. As the comments on reddit have shown, people cannot comprehend the arguments in this article, so the added complexity may be perceived as negative.
I always wondered what if in the GSL format, deciding games aka series 3-5 was a best of 5 rather then best of 3. Or would that take too long? Hmmmmmmm. Anyway great article, I do have more to say but will say this for now.
Is it an alternative way to interpret the independence that players understand the fifth match is more important and approach it differently, so that it’s independent from the previous game? Consider that when there’s a clear favorite and a clear worst player in a group and you’re not facing them in your first match, it makes sense to play not as hard and hide strategies since you know there’s a high chance that you’ll face the same opponent in the more important decider match. And this could be the cause of the independence.
BTW, the proposed hypothetical scenarios also more likely leads to a repeated fifth match.
Good point. This is an assumption that the data cannot tell you. I won’t be surprised if players hide builds.
Often as a viewer it feels like the first match has an upset because the worse player prepared better strategies, and then in the rematch the better player wins in more normal games.
For instance the case with sos-bunny in the example: sos did a ridiculous proxy nexus allin in the first series but didn’t have anything as successful in the second series, thus bunny won with solid standard play.
..and following this line of thinking, basing the result on the latter series seems fine.
You are making an argument based on presumed player superiority. If the so-called worse player prepare better strategies, does it not suggest this player is in fact better?
I think a better way to put nept’s argument is that you don’t know your fifth match’s opponent for sure before the match day, and therefore that will test your on-the-fly playmaking more. But I disagree with nept that this could be a reason that the fifth match should be more important. GSL, especially towards the playoffs, puts a lot of emphasis on match preparation. This is actually one of GSL’s distinctive property vs. foreign tournaments. So along this line, if anything, it make sense to emphasize match preparation more, rather than less, in the more important match.