{"UUID":"a130b649-66de-4e16-bf41-8b84bb0a8e48","URL":"https://github.blog/news-insights/company-news/github-availability-report-july-2024/","ArchiveURL":"","Title":"GitHub Copilot degradation on July 13, 2024","StartTime":"2024-07-13T00:01:00Z","EndTime":"2024-07-13T19:27:00Z","Categories":["automation","config-change","security"],"Keywords":["github","copilot","copilot chat","code scanning","autofix","partner service","degradation","resource cleanup"],"Company":"GitHub","Product":"github copilot","SourcePublishedAt":"2024-08-14T22:00:06Z","SourceFetchedAt":"2026-05-04T19:52:20.971392Z","Summary":"A scheduled resource-cleanup job at a partner service mistakenly targeted a resource group containing essential Copilot infrastructure, removing it. Copilot Chat error rate peaked at 63%; rerouting Chat traffic mitigated most impact while resources were progressively restored over 19h26m.","Description":"On July 13, 2024, between 00:01 UTC and 19:27 UTC, the GitHub Copilot service experienced significant degradation. This incident lasted for 19 hours and 26 minutes, affecting various Copilot functionalities.\n\nDuring the incident, the error rate for Copilot code completions reached 1.16%, while GitHub Copilot Chat experienced a peak error rate of 63%. Customers using Copilot may have encountered delays, errors, or timeouts. Additionally, GitHub code scanning autofix dropped suggested fixes between 00:01 UTC and 12:38 UTC, and subsequently delayed suggested fixes until 21:38 UTC.\n\nThe root cause was identified as a scheduled resource cleanup job executed by a partner service. This job mistakenly targeted a resource group containing essential Copilot infrastructure, leading to the removal of critical resources.\n\nTo mitigate the impact, GitHub rerouted Copilot Chat traffic between 01:00 and 02:00 UTC, which successfully reduced Copilot Chat error rates to below 6%. The erroneous cleanup job was stopped in time to preserve some resources, allowing GitHub to progressively restore services.\n\nMoving forward, GitHub is collaborating with partner services to implement safeguards against similar incidents. They are also enhancing their traffic rerouting processes to expedite future mitigation efforts and minimize customer impact."}