Recently (for more than two months), Terran players have argued that TvP is Protoss favored. The recent results may seem to support this assertion. I want to share my thoughts in two posts. One is from a quantitative perspective, and the other from a qualitative perspective. This post is my thoughts from a quantitative perspective.

Just a head start, don’t read too much into technical part. I simply want to make sense out of the data at the end of the day.

Let’s start with the basic statistics.

I do not want to use progamers’ games because I want to have a more holistic look of the match up. The most objective way to evaluate a player’s skill is by ladder ranking. It is not the perfect way, but I don’t see a better and realistic way.

Taken from SC2 Ranks (10/10/2013)

There have been many posts on reddit/r/starcraft regarding the relatively low number of Terran in Grandmaster. Indeed, the number of Protoss in Grandmaster is more than two times the number of Terran. However, simply by looking at the Grandmaster league, it does not capture the whole picture (the same logic for progamers’ games). The ladder ranking is arguably the best sampling frame available (perhaps it can be considered the population itself).

Taken from SC2 Ranks (10/10/2013)

Globally, the number of Protoss is slightly higher, even though the difference is marginal. The relative high number of Protoss in Grandmaster may be due to the fact that there are more Protoss in the population to begin with. With the assumption of a normal distribution, there should be more Protoss in Grandmaster than Terran.

“But those in the Gold and below should not be considered because their skill level is irrelevant…”

It is hard to have a cut off objectively of which leagues are irrelevant. From quantitative point of view, it is better to include every piece of data in initial analysis without any influence of “experts’ opinion”. I ran a simple one-way ANOVA based on the above data. However, I must stress that I did not enter every case (almost half a million) in data entry for practical reason. Given the large sample size and how well distributed the total number of players for each race, I took a short-cut.  First, I removed the random “race” from the data set. Then, I divided the number of players for a race in a certain league by the total number of players (exclude random), and times 1000. I wanted to times 100 for percentage, but the number of Grandmaster players would result as less than 1. As a result, the number of players in each race is about 300+. This is just to make the data more manageable. It should not affect the actual result to the extent that it tells a different story. (Let me know if you get a significant different result when the raw data is used)

I coded Grandmaster as 1, Master as 2 … Bronze as 7. That is why the minimum is 1.00 and maximum is 7.00. Race is the independent variable, while league placement is the dependent variable.

Here is the result.

The purpose of this is to test if playing a specific race affects league placement. Given the nature of the variables, it is not meaningful to look at the mean and standard deviation. More importantly, there was a significant effect of race on league placement at the p<.05 level for the three conditions [F(2, 997) = 3.451, p = .032]. Another words, the relationship between race and league placement is unlikely to happen just by chance (3.2%). In layman term, race indeed affects your league placement.

I ran a post-hoc least square difference test to test if the races are significantly different from each other for the sake of completion.

Basically, it compares each race with one another. Zerg-Protoss = .065 (not significantly different at p < .05), Zerg-Terran = .011 (significantly different at p< .05), Protoss-Terran = .468 (not significantly different at p < .05). This is not meaningful because the number of players in each league is structured to be proportionate to the total number of players to begin with. The mean of league placement is irrelevant.

In my opinion, I will take the results with a pinch of salt because of various limitations. But my point is that it is better than just using frequency statistics of Grandmaster to tell a story. (So please don’t challenge me that the statistics is irrelevant)

Here is a simple graphical presentation.

At face value, the race distribution for Diamond and above does not really suggest Protoss is stronger than Terran. The pattern looks consistent although Protoss have more players than Terran. If you trace it from Grandmaster to Silver, the pattern of Protoss and Terran cannot look much better to call it “balance”. Just to be clear, although the difference between Protoss and Terran is relatively bigger at Gold and Silver, this does not mean that Protoss is stronger than Terran at Gold and Silver. You want to compare it across the entire “league placement scale”, since the scale is the measurement of “strength”.

Interestingly, the graph fluctuates for Gold and below, while it is consistent for Platinum and above. Remember that the ANOVA analysis earlier suggests that race does affect your league placement. The analysis was conducted with all seven leagues. The results could be due to the fluctuation for Gold and below. If Protoss is truly stronger than Terran based on league placement, the effect should be statistically significant if I trim the data to just Platinum and above. By the look of the graph, I can tell that the results will not be statistically significant. Anyway, why not just run it.

There wasn’t  a significant effect of race on league placement at the p<.05 level for the three conditions [F(2, 191) = 0.42, p = .959]. Another words, the relationship between race and league placement is likely to happen just by chance (95.9%). In layman term, race does not affect your league placement.

The post-hoc test also suggests that the difference between the races themselves is not statistically significant.

Now compare the first ANOVA test (Grandmaster to Bronze) and second ANOVA test (Grandmaster to Platinum). The first ANOVA test suggests that race affects league placement, but the second ANOVA test suggests otherwise. Thus, the “racial imbalance” can be argued to happen in Gold to Bronze. More accurately, I would argue that there is much inconsistency in Gold to Bronze.

I stated at the beginning that it is not wise to make a cut off for the data to be included in the analysis since it defeats the purpose of quantitative research. Then why do I remove Gold to Bronze in the second ANOVA? I only did that after I have a grasp of the holistic view of the interested topic. I then tried to make sense by combining the ANOVA results and the graph. This helps to get a clearer picture of the problem.

If you look closely, the racial distribution balance starts to converge from Gold. It stablizes in Platinum, and the overall distribution is consistent all the way to Grandmaster.

All in all, the results of the two ANOVA tests do not provide strong evidence that race affects league placement. I operationalize the strength of the race using the frequency distribution of league placement. Thus, indirectly, the results suggest that the three races are rather equally matched. 

—

Why do Terran often win tournament?

This is a very interesting question. I will use logic to put my thoughts forward.

First, I must again point out: the analyzes above indicate that the races are equally matched. Thus, this observation could be due to the players. It can be argued that the best few players happen to play Terran. If it is the race that is strong, then there should be a significant high number of that race in later stages of the tournaments. This is based on the logic that the race advantage, if any, should not only be “activated” on the top few Terran players. The top 16 and top 8 of recent tournaments that Terran won do not have significantly more Terran. Therefore, I will conclude that it is the “player” that won the tournament and not the “race”.