On Sunday, June 12, 2016 at 10:13 PDT, Sentry received an external security report that one of its S3 buckets — used for partial database backups including user accounts and API tokens — had permissions that allowed any authenticated AWS user who knew the bucket name to read and modify it. The team confirmed the issue at 10:20 and immediately revoked permissions and credentials on the affected bucket, then audited every other bucket and confirmed the misconfiguration was isolated.

The cause was two conflicting S3 ACL policies on the same bucket: a permissive one that allowed any authenticated AWS user, and a restrictive one that limited access to specific accounts. AWS resolved the conflict by allowing the access (the “more-correct” restrictive policy was not the one that won).

While running the threat-mitigation workstream in parallel with the investigation, Sentry rotated internal credentials for its SSO providers (Google and GitHub) by 11:14 — every active session was invalidated and users had to re-authenticate. By 12:40 the affected bucket had been moved to a less-guessable name with corrected ACLs. They contacted integration partners (notably Atlassian, since the JIRA plugin stored passwords in plaintext as part of its integration config). On the afternoon of June 12 they built a password-expiration flow and held off enforcing it pending evidence of actual access. On June 13 at 15:12 they force-expired all pre-existing passwords and emailed every potentially-affected customer. At 16:49 they shipped pre-encrypting backups before sending them to S3.

On June 14 at 12:12 PDT the investigation concluded no third party had actually accessed the backups. Sentry consequently dropped the forced API-token rotation and removed the forced-password-expiration flag on accounts still pending it.

Committed follow-ups: ship multi-factor auth (already in production for the Sentry team for several weeks at the time), encrypt API tokens and integration settings at rest in the database so a backup leak alone is insufficient, and migrate integrations like JIRA to OAuth flows where possible to avoid storing third-party passwords.