Office 365 Down | September 28, 2020

Office 365 Down | September 28, 2020

PRINCETON, N.J. September 28, 2020 – Microsoft Office 365 is having major connectivity and authentication issues today.

As of 5:33P EST Microsoft O365 issues have been reported across several countries.  Downdetector had over 18,500 users report issues just a few hours ago.  Users of O365 services are experiencing various issues such as Outlook not connecting and O365 account prompting for passwords, but not authenticating even with the correct password.  Microsoft acknowledged the issue was which caused by a change they deployed earlier today.  The update has been rolled back and some users are reporting connectivity once again.

We expect the rollback to resolve this issue in the next few hours, although no official time frame has been given by Microsoft.  At this point we suggest waiting until 10:00P EST and if the issue is not resolved for you, to restart your device (PC, Mac, tablet, phone, etc.) after 10:00P.  You can visit Microsoft's public site directly for an "official" Microsoft 365 Service Health status.

For those that have access to the O365 admin portal, we suggest logging into your portal and checking the O365 Service health page "Can't access Microsoft 365 services".  The last description under that incident is below.

 


via Microsoft

Title: Can't access Microsoft 365 services

User Impact: Users may be unable to access multiple Microsoft 365 services.

More info: Users would be unable to access Outlook.com, Microsoft Teams including Teams Live Events, and Office.com. Additionally, Power Platform and Dynamics365 properties are affected by this incident. Existing customer sessions are not impacted and any user who is logged in to an existing session would be able to continue their sessions.

Current status: We’ve determined that a specific portion of our infrastructure is not processing authentication requests in a timely manner. We’re pursuing mitigation steps for this issue. In parallel, we’re rerouting traffic to alternate systems to provide further relief to the affected users.

Scope of impact: Any user may experience access problems for Microsoft 365 services.

Start time: Monday, September 28, 2020, 5:25 PM (9:25 PM UTC)

 

UPDATE: 9:09P - Fix Deployed
Current status: Our mitigation strategy was successful in allowing users to sign into the previously impacted services. Our internal monitoring has validated this recovery and we have received positive confirmation from customer reports. We’ll continue to monitor the service and provide updates on full recovery to remaining impacted users.

 

UPDATE: 10:33P - Resolved
Final status: After extended monitoring of our service, we have confirmed that the issue has been fully mitigated and is running within optimal service levels. If you remain impacted please engage with your support representatives to further assist with any additional issues.

Scope of impact: Any user may have experienced problems accessing services that leveraged Azure Active Directory.
Start time: Monday, September 28, 2020, 5:25 PM (9:25 PM UTC)
End time: Monday, September 28, 2020, 10:25 PM (9/29/2020, 2:25 AM UTC)

Preliminary root cause: A code issue caused a portion of our infrastructure to experience delays processing authentication requests, which prevented users from being able to access multiple M365 services.

Next steps: - We’re reviewing our code to understand what caused the code to stop processing authentication requests in a timely fashion.

We'll publish a post-incident report within five business days.


 

This post first appeared on the Blueclone Networks site blueclone.com.  Blueclone Networks is a Princeton based IT Company with focus on Computer Systems Management, Cybersecurity, and Cloud Services for small & medium sized business in Central NJ.