Rumours about scripting, handicapping and momentum (SHM going forward) have haunted the FIFA community for years. Despite EA's repeated denials, polls suggest that a majority in the FIFA community believe that the game is manipulated.

When I first heard about SHM, I definitely could relate to the frustration which nurtured these claims. But at the same time, there was a nag of scepticism in the back of my head.

Features like rubber banding are common in video games. The reason we know is that game designers typically are completely transparent about it. A good example is EA's own Battlefield series, which uses behemoths to level the playing field. Another good example is in fact FIFA's kick-off mode, which shows a dialogue, where it is announced that it determines the difficulty dynamically when you open a new account!

And what strikes me about SHM in this context is that I really can't think of a good reason to implement it secretly. The controversial part of SHM wouldn't be the things it does to the game – it would be the fact that it is done secretly!

Another thought, which has nurtured my growing scepticism, is that SHM also might work as a convenient excuse. From a perspective of human psychology, it is easier to explain to yourself that you lost because of foul play and not because you aren't as good as you hope. And convenient excuses are common everywhere in competitive environments. An example that leaps to my mind is the complaints about referees in real football.

As an undecided sceptic, I decided to take the debate a step further back in 2014 where I published a couple of articles on the topic. Soon thereafter, I started the FIFA fact checking blog called FUTfacts.com together with a friend.

The idea behind FUTfacts.com is that we want to collect and test evidence with relevancy to the SHM debate to see if we could bring the debate to a conclusion.

Since then, we have analysed and written about every piece of alleged evidence that we possibly could get our hands on as well as conducted research on our own.

You may of course ask: Is it really possible to reach a certain conclusion over these matters? My answer is a clear yes. Below, I will go into detail with the research we have conducted and explain what our conclusions are and why we are so certain about them.

Flimsy claims about FUT scripting

When we started our work, we needed to get our hands around the claims we were to test. But defining scripting, handicapping and momentum is not as easy as it sounds.

The terms scripting, handicapping and momentum cover a broad range of more or less consistent and more or less compatible claims. The bullets below should give you an impression of some of the main streams, although there are other claims and variants as well:

• Scripting typically refers to the belief that natural or unnatural events deliberately are put into the game for some reason.
• Momentum can either refer to the belief that EA imposes momentum shifts in matches or to the belief that EA invokes winning or losing streaks on players.
• Handicapping can either refer to the idea that the better player is handicapped or that the better team is handicapped, meaning that stat differences either are neutralized or even reversed.

I often hear people claim that there is a consensus in the community around that SHM exists, and this alleged consensus is claimed to be evidence in itself. But given that SHM isn't a belief but rather an umbrella term covering a multitude of beliefs, I had to question whether the alleged consensus was real.

To test this question, I made a survey where I asked the respondents whether they considered ten common claims about SHM likely or not. I wanted to know to what extent people shared a common set of beliefs across the ten claims or not. It turned out they didn't. Far from actually.

The survey confirmed that there is a consensus around the overarching idea that FIFA matches are manipulated, but people disagree over how and why the alleged manipulation happens.

Some people find it likely that the game favours the lesser player. Others believe that it favours the lesser team. And then there are some who believe both things are true, despite that this would mean that EA was helping both players in some matches, which honestly doesn't make a lot of sense

Neither the unclear nature nor the lack of a consensus proves these claims wrong per se. But when two people hold contradictory beliefs, minimum one of them is wrong. And when 1000 people hold 800 different beliefs, most of them are wrong.

What that brings me to is first and foremost that the alleged consensus doesn't exist. And if it did exist, it really wouldn't tell us anything about whether SHM exists or not. People, even a majority, can be wrong.

 

What claims did we check?

Fact checking isn't exactly easy when claims go in all sorts of opposite directions. But some claims are more common than others.

Two particular claims that appear very often in posts related to SHM are:

• Matches are manipulated in order to make them more even - either dynamically or through a preset handicap.
• Players who use better squads are handicapped.

The bulk of our work has been centered around the two ideas above. Below, I will give you a brief overview of the relevant evidence for each of them and explain our conclusion.

 

Claim 1: Does EA make matches even?

The community has seen quite a lot of different evidence allegedly suggesting that FUT matches are made even. I would like to mention a couple of examples, either because they are typical examples of what people consider evidence, or because they attracted a lot of attention.

An example of the latter dates back to 2016, when someone dug out some code from a FIFA-file on the PC. It contained references to a concept called "adaptive difficulty" whereby the difficulty would be adjusted depending on the score line. There are multiple reasons to refute this as evidence. First and foremost, the code in question clearly wasn't related to FUT but to career mode. And while it is absolutely likely that adaptive difficulty is implemented in career mode, that just doesn't lead us to conclude that it also is implemented in multiplayer game modes like FUT.

Another example, which attracted a lot of evidence, was the discovery of two new EA patents – namely DDA (Dynamic difficulty adjustments) end EOMM (Engagement optimized matchmaking), which were accompanied by research papers written by EA's own research team. Particularly DDA attracted attention, because it bore some resemblance with momentum. While it is certain that EA has build this concept, nothing suggests that it is implemented in FUT. And the mere fact that something perhaps could be done, just isn't evidence that it in fact was done.

I could go on for hours explaining why different pieces of evidence just isn't evidence. For something to become evidence, it as a minimum has to rule out other explanations than SHM. And a recurring problem with the alleged evidence is that it mostly can be explained by other – natural – causes as well. But if you aren't even considering that option, you of course won't see it.

So, could we actually create an experiment, which allows to test whether matches are being leveled? Indeed we could, and we did. In fact, we created three different experiments, which have in common that if matches are being leveled to any significant extent, we would catch it.

 

FIFA Manipulation Claims

In the first experiment, we wanted to test whether better players in fact reported more manipulation than lesser skilled players. If matches are being made even, we would expect more successful players to report losing more often due to manipulation than less successful players.

To test our hypothesis, we asked 1000 players to provide sufficient information for us to establish their skill level relative to other players. Further, we asked them how often they felt that their matches were subject to manipulation.

As it happened, we found absolutely no connection between a person's skill level and his inclination to feel that his matches were manipulated. In fact, less successful FIFA players were just as likely to attribute their losses to manipulation as highly successful players.

What this leads us to conclude more than anything is that the experiences that people refer to as scripting, handicapping and momentum, apparently don't follow a pattern where they would be likely to create more even matches.

This study alone is however not even remotely the most convincing evidence we have at our hands.

Even Results Claim in FUT

In the second experiment, we wanted to test whether FUT matches in fact are more even than they ought to be. How can you possibly test this, I hear you say? The idea for this experiment came from the observation that sports with more goals than football also generate fewer draws. In the same way, football leagues with higher goal ratios also generate fewer draws.

Based on data from more than 100 real football leagues, we were able to calculate a formula for the relationship between goal ratio and draw ratio (see the trend line in the chart below). If FUT seasons matches are made even, we would expect to see them contain a high draw ratio when compared to real football leagues with a similar goal frequency. But as the red dot illustrates, FUT falls nicely into the pattern of all the real leagues.

Where this brings us is that if matches indeed are made even, it clearly doesn't generate an unusual percentage of draws. In fact, FUT – with its very high goal ratio – has a much lower draw ratio than most real football leagues, which is exactly what you would expect if things were as they should be.

 

Claim About Better Players Losing the Match in FIFA

Our third experiment was aimed at testing whether it is a common phenomenon that the loser of a match evidently is the better among the two players involved. If matches really are made even, we would expect to find a lot of cases where the better player ended up losing.

Our test was conducted back when you could download data about players and their ten latest matches. We did this for 1100 matches, and calculated the percentage of matches where the losing party (1) dominated on all the available performance metrics (passing, possession and shooting) and (2) had a better historic track record. This turned out to be the case in a mere 4 % of the 1100 matches in our sample. And when we took a closer look at the 40 matches in question, we found no traces of the gaming helping out new players or the likes. In fact, all the 40 matches involved players with similar track records.

So, what we have above is three separate pieces of evidence, which would have yielded a certain result if matches are made even. But none of them does. At the same time, I have to repeat that we have found no evidence which suggests the opposite.

Based on the overall state of evidence, the only reasonable conclusion has to be that matches aren't made even. And that sadly also means that you lose either because you just were unlucky or because the opponent was a better player.

 

Claim 2: Are better squads handicapped?

The fact that matches aren't made even does not rule out that EA could be handicapping the player with the better squad. So, what exactly is the state of evidence in relation to handicapping?

A redditor called Arlington69 analysed the performance of icons and found that the prime version sometimes didn't create as many goals as the base version. He, then, concluded that this suggests that the game "dumbs down" better teams.

A major problem with that conclusion is however is that Arlington69 completely ignores that most of his data in fact suggests that prime icons perform better than the cheaper versions. Additionally, the decision to focus on extremely rare items like icons means that Arlington69's samples are far too small to support any conclusions in regard to the relationship between stats and performance.

We carried out similar experiments multiple times for cards released in much larger volumes, and our data repeatedly confirmed that higher rated versions in general score more goals, cf. the table below.

And in all fairness, I think anything else would have been a surprise of epic proportions. EA sells packs, and when you are in that line of business, it doesn't make sense to make it a disadvantage to use an expensive squad.

 

Why do people believe in SHM?

I wrote earlier that SHM may be a convenient excuse, both to yourself and to your friends, when you lose more often than you would like to. This is up until this point mainly an unsubstantiated claim, and I would like to add a few additional words to my argument before I wrap this up.

A noteworthy fact that I would like to bring up here is that, unlike what FUT players in general like to think, a lot of them have an unrealistic picture of their own skill level and a poor understanding of football as a game. This statement is controversial, but I do have the data to back it up.

Among the observations I have made is that people in general tend to believe that they are better at FIFA than they really are. A tricky, little survey published by FUTfacts co-editor Coldspurs in /r/FIFA asked a very simple question: How good are you compared to other people responding to this poll?

A staggering 70 % thought they were above average!

Another observation is that people in general fail to acknowledge certain inevitable truths about football and hence also FIFA. I'm aware that a lot of people will object when I compare those two things, but the fact is that FIFA is a football simulation. Therefore, it absolutely makes sense to assume that many of the properties of real football are present in FIFA as well. And one of those properties is coincidence.

A German professor called Martin Lames found that 44 % of all goals in real football matches are random. It is reasonable to assume that the same goes for a lot of FIFA goals.

But in a survey from 2017, only 3 in 10 SHM believers acknowledges that FIFA is a highly random game like real football. When you don't acknowledge the presence of randomness, you also don't acknowledge the importance of luck, despite the fact that it is a decisive factor in a game like FIFA.

The same survey also showed that SHM believes in general fail to realize that the same things that work to their advantage, works to the opponent's advantage. 8 in 10 SHM believers completely rejected the idea that the opponent is able to influence your ability to pass, shoot, head and sprint, thereby effectively saying that those matches where you perform poorly on those parameters could have anything to do with the opponent. And yet, the same proportion - 8 in 10 of the SHM believers - said that they themselves were able to influence the opponent's ability to pass, shoot, head and sprint! SHM believers are likely to attribute their own success to skill while attributing their opponent’s success to scripting, handicapping or momentum.

The point I want to make here is that the by far most likely explanation to why people believe in SHM is that the reality is difficult to handle: Namely that you lose more than you would like to, because the opponents are better or just luckier than you, and not because some big, evil corporation is preventing you from taking your game to the skies.