On October 17, 2020, at approximately 10:45 PM CET, the production database for Leaderboarded.com was accidentally deleted. This incident resulted in the immediate loss of over 300,000 scoreboards and their associated data.

The root cause was a misconfiguration in the application’s environment. A function intended to wipe a local development database was inadvertently executed against the live production database. Although the function’s code hardcoded ‘localhost’ for the database host, an incorrect FLASK_ENV setting caused the application to connect to the production environment.

The customer impact was significant data loss. Any scoreboards created or scores added between 3:47 PM CET and 11:21 PM CET on October 17, 2020, were permanently lost. The service was put into maintenance mode shortly after the incident began.

Within 30 minutes, by 11:15 PM CET, the service was brought back online using a daily backup provided by DigitalOcean. While this allowed for quick recovery, the data created during the 7-hour window between the last backup and the incident was irrecoverable. The incident highlighted the danger of having database-deleting functions and the critical importance of reliable, quick-recovery backups. The developer plans to implement measures to prevent this specific error from recurring.