On December 16, 2022, a CircleCI engineer’s laptop was compromised by malware, leading to the theft of a 2FA-backed SSO session. This allowed an unauthorized third party to gain access to CircleCI’s production systems, with reconnaissance activity observed on December 19, 2022, and data exfiltration occurring on December 22, 2022. CircleCI was alerted to suspicious GitHub OAuth activity by a customer on December 29, 2022, which initiated a deeper security review.

The root cause was malware deployed to an engineer’s laptop, which was not detected by antivirus software. This malware executed session cookie theft, enabling the unauthorized third party to impersonate the targeted employee. Leveraging the employee’s privileges to generate production access tokens, the attacker escalated access to a subset of CircleCI’s production systems.

The unauthorized third party accessed and exfiltrated data from a subset of databases and stores, including customer environment variables, tokens, and keys. Although the exfiltrated data was encrypted at rest, encryption keys were also extracted, potentially allowing access to the encrypted information. Customers were advised to assume their secrets were accessed and to rotate them, with fewer than five customers reporting unauthorized access to their third-party systems as a direct result.

In response, CircleCI took immediate actions starting December 31, 2022, by proactively rotating all GitHub OAuth tokens. On January 4, 2023, they shut down access for the compromised employee, restricted production access for most employees, and rotated all potentially exposed production hosts. Subsequent actions included revoking all Project and Personal API Tokens, rotating Bitbucket tokens, and notifying customers about potentially affected AWS tokens, with all major rotations completed by January 7, 2023.

CircleCI implemented additional security measures, including enhanced malware detection, stricter production access controls, step-up authentication, and improved monitoring. They also developed new tools for customers, such as a secret finding script, API changes for SHA-256 signatures and updated_at fields, and made audit logs accessible to all. Future plans include automatic OAuth token rotation, a shift to GitHub apps, and more ephemeral system permissions.