FreeSO, an unofficial reincarnation of The Sims Online received a huge interest from simmers all around the world. Just a day before the scheduled open beta release (January 6th 2017), the FreeSO servers were under a DDOS attack which caused the site and servers not to load at all. After trying to set up a registration page so that people can create their FreeSO account and use them to log in to the game, over 500 people tried to register at once which caused the whole thing to crash.
In a new blog post from FreeSO, the devs have explained the issue and what they’re trying to solve them:
- One day before launch, DOS attacks began hitting the wordpress and forum servers. These were mitigated as they were pretty small scale, and against normal web servers which can deal with a reduced load much better.
- When setting up the API server on our large scale servers, DOS attacks began coming through to the API server recently added to the DNS. These were rate limited, but somehow were still enough to take down the server. It became clear something was wrong with the Nancy driven API server from this.
- By rate limiting to 10 requests a minute (yikes!) we were able to reduce the load on the registration enough to keep it alive, before it was even announced to anyone. During this period, I was able to log in with what seemed like no issues. In this period, 100 users registered.
- I released the registration link on the discord channel. The whole thing blew up, immediately 500 concurrent requests were made to the api server, taking it down immediatly. Hours later, these have still not simmered down – there are a few hundred requests being made and simultaneously active. The large delay each of these requests is incurring is causing the majority of them to fail to get through.
So what is the plan?
We will perform further load and api testing, look into the root causes of load issues and the server stability. The current plan is to rewrite the API server using ASP.NET instead of Nancy, as it is a lot more “battle-tested”. This could take some amount of time.
We cannot announce a retry date. We might even need a new launch plan altogther, to avoid the load of a few thousand users wanting to play the game on day one. Whatever the plan is, it will be posted on here eventually.