Google Workspace Status Dashboard
Incident affecting ChromeOS
Incident began at 2024-08-06 15:00 and ended at 2024-08-07 10:29 (times are in Coordinated Universal Time (UTC)).
Date | Time | Description | |
---|---|---|---|
| Aug 15, 2024 | 7:46 PM UTC | Incident ReportSummaryOn 06 August at 08:00 US/Pacific, Chrome OS devices experienced issues with logins globally for a duration of 19 hours 29 minutes. During this time, users on ChromeOS devices were able to complete the full sign-in process, including any Multi-factor Authentication when required. However, some users were immediately redirected back to the login page after logging in. Root CauseGoogle’s internal authentication system uses an HTTP endpoint microservice that provides clients with a list of accounts currently in the authentication cookies. This endpoint is mainly used for various account switcher user interfaces, providing a list of accounts the current session has access to. The root cause of the issue is an incorrect configuration made to the Google authentication service, which resulted in a malformed response to the queries from this microservice in a particular condition. As a result, Chrome OS was unable to get the account details from the internal Google authentication service, which caused the sign-in failures. Remediation and PreventionGoogle engineers were alerted to the outage via a support case on Tuesday, 06 August at 15:51 US/Pacific, and immediately started an investigation. Once the nature and scope of the issue became clear, the configuration change was reverted and Google engineers ensured that sign-in worked again. Google is committed to preventing a repeat of this issue in the future and is completing the following actions:
Google is committed to quickly and continually improving our technology and operations to prevent service disruptions. We appreciate your patience and apologize again for the impact to your organization. We thank you for your business. Detailed Description of Impact
|
| Aug 7, 2024 | 5:46 PM UTC | Mini Incident ReportWe apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced an impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213. (All Times US/Pacific) Incident Start: 2024-08-06 08:00 Incident End: 2024-08-07 03:29 Duration: 19 hours 29 minutes Affected Services and Features: Chrome OS Regions/Zones: Global Description: Chrome OS devices experienced issues with login due to a recent change to a backend identity service, causing incorrect parsing of account details. Google will complete a full IR in the following days that will provide a full root cause. Customer Impact:
|
| Aug 7, 2024 | 10:44 AM UTC | The problem with ChromeOS has been resolved. We apologize for the inconvenience and thank you for your patience and continued support. |
| Aug 7, 2024 | 8:00 AM UTC | Summary:ChromeOS sign in issues affecting multiple customers throughout the globe Description:We are experiencing an issue with ChromeOS. Our engineering team continues to investigate the issue. We will provide an update by Wednesday, 2024-08-07 06:00 US/Pacific with current details. Diagnosis:During the full sign in process ChromeOS devices complete the sign in process (even Multi Factor Authentication (MFA) if applicable) but are eventually brought back to the login page. The issue is affecting ChromeOS versions 120 to 128 (potentially more versions). Workaround:None at this time. |
| Aug 7, 2024 | 6:01 AM UTC | Summary:ChromeOS sign in issues affecting multiple customers throughout the globe Description:We are experiencing an issue with ChromeOS. Our engineering team continues to investigate the issue. We will provide an update by Wednesday, 2024-08-07 01:30 US/Pacific with current details. Diagnosis:During the full sign in process ChromeOS devices complete the sign in process (even Multi Factor Authentication (MFA) if applicable) but are eventually brought back to the login page. The issue is affecting multiple ChromeOS versions Workaround:None at this time. |
| Aug 7, 2024 | 5:06 AM UTC | Summary:ChromeOS sign in issues affecting multiple customers throughout the globe Description:We are experiencing an issue with ChromeOS. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2024-08-06 23:30 US/Pacific with current details. Diagnosis:During the full sign in process ChromeOS devices complete the sign in process (even Multi Factor Authentication (MFA) if applicable) but are eventually brought back to the login page. The issue is affecting multiple ChromeOS versions Workaround:None at this time. |
- Times are listed in Coordinated Universal Time (UTC)