{"UUID":"4a4e2060-38cb-4ced-9788-3c4e305a3c3c","URL":"http://users.csc.calpoly.edu/~jdalbey/SWE/Papers/att_collapse.html","StartTime":"0001-01-01T00:00:00Z","EndTime":"0001-01-01T00:00:00Z","Categories":["postmortem"],"Company":"AT\u0026T","Product":"","Description":"A bad line of C code introduced a race hazard which in due course collapsed the phone network. After a planned outage, the quickfire resumption messages triggered the race,  causing more reboots which retriggered the problem. \"The problem repeated iteratively throughout the 114 switches in the network, blocking over 50 million calls in the nine hours it took to stabilize the system.\" From 1990."}