Facebook was down for about 2.5 hours on Thursday, an outage that the company called its worst in four years.
While Facebook was down, users complained on Twitter about the outage. Screenshot: Sam Diaz/ZDNet.com
Reports of Facebook problems from users mostly said that the site has been up and down all of Thursday evening. Facebook acknowledged that it was experiencing "latency issues" with its API, but it was not immediately clear what was causing those issues.
Facebook released a statement on Thursday evening explaining what sparked the outage.
"This is the worst outage we've had in over four years, and we wanted to first of all apologise for it," Robert Johnson, the company's director of software engineering, wrote in the statement.
"Today we made a change to the persistent copy of a configuration value that was interpreted as invalid. This meant that every single client saw the invalid value and attempted to fix it. Because the fix involves making a query to a cluster of databases, that cluster was quickly overwhelmed by hundreds of thousands of queries a second," Johnson wrote.
Read this
Facebook, Twitter: Is social media out of control at work?
A few key measures can help organisations benefit from social media and also shut out the criminals, says Alan Calder
"We had entered a feedback loop that didn't allow the databases to recover. The way to stop the feedback cycle was quite painful — we had to stop all traffic to this database cluster, which meant turning off the site. Once the databases had recovered and the root cause had been fixed, we slowly allowed more people back onto the site," he added.
For more on this ZDNet UK-selected story, see Facebook outages reported; Qwest sees packet loss on CNET News.
