Users may be unable to access or use some Microsoft 365 services and features. | ||||||
---|---|---|---|---|---|---|
ID | Workload Feature | Title | Classification | Severity | Start Time | Last Updated |
MO941162 | Microsoft 365 suite Portal Access | Users may be unable to access or use some Microsoft 365 services and features | incident | 24-Nov-2024 20:20 | 27-Nov-2024 20:11 | |
Update: 27-Nov-2024 20:11 A post-incident report has been published. Update: 26-Nov-2024 14:51 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact scenarios were as follows: Exchange Online - Users may have been unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may have experienced mail transport delays. Microsoft Teams - Users may have been unable to create or update Virtual Events, including webinars and Town Halls. - Users may have been unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users may have been unable to create chat, add users and create or edited meetings. - Users may have been unable to create or modify new teams and channels. - Users may have been unable to update presence. - Users may have been unable to use the search function. - Users may not have seen an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may have been unable to access the Purview Portal, or Purview Solutions. - Users may have experienced delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may have been unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may have been unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may have been unable to use the search feature. Microsoft Defender for Office365 and the XDR portal - Users may have been unable to create simulations, simulation payloads or end user notifications. - Users may have experienced issues with delivery for end user notifications and simulation messages. - Some users may have experienced failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may have experienced issues with viewing simulation reports, and content. - Users may have received a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may have been unable to Print via Universal Print. - Users may have been unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may have been unable to Register Printers via Universal Print. Power Automate for Desktop - Users may have experienced errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may have been unable to access their bookings. Microsoft Viva Engage - Users may have been unable to load topics in feeds and threads. - Users may have seen searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not have shown user names. - Users may not have been able to load their home feeds. Microsoft Copilot - Users may have been unable to use the personal Copilot panel in meetings and post meetings. - Users may have been unable to see historic Copilot conversation history in meetings and post meetings. - Users may have been unable to load Copilot experiences within the Viva Engage service. - Users may have been unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may have received an apology or "at capacity" message. Final Status: After a period of extended monitoring, we've confirmed through customer reports and service health telemetry that the issue is resolved. Scope of Impact: This issue could have impacted any user hosted globally; however, users experienced varying degrees of impact depending on how they were routed through the affected infrastructure and what action they were trying to perform. Impacted users who attempted to use the functionalities outlined in the More info section of this communication may have been affected by this event Start Time: Monday, November 25, 2024, at 1:20 AM UTC End Time: Tuesday, November 26, 2024, at 11:00 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and affected servers' processing capabilities, which led to impact. Next Steps: - We're examining the parameters required to decommission backend services so we can better anticipate, test for, and avoid or prevent similar scenarios. - We're assessing monitoring optimizations we can better detect and more quickly remediate router service issues. We'll provide a preliminary Post-Incident Report within two business days and a final Post-Incident Report within five business days. Update: 26-Nov-2024 12:57 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We're continuing to conduct additional optimizations to further improve service availability, which remains in a healthy state. We're monitoring telemetry closely while this work progresses to ensure service availability is fully restored. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 8:00 PM UTC Update: 26-Nov-2024 10:58 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We're conducting additional optimizations to further improve service availability, which remains healthy. We're closing in on a full mitigation to address any remaining impact and we're closely monitoring telemetry to ensure this trend continues. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 6:00 PM UTC Update: 26-Nov-2024 07:53 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We're continuing our period of monitoring service telemetry, which shows the service availability has remained healthy. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 4:00 PM UTC Update: 26-Nov-2024 05:59 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: From monitoring service telemetry, most users should now experience relief. We've completed our optimizations and we're continuing our period of extended monitoring to ensure the availability remains stable. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 1:00 PM UTC Update: 26-Nov-2024 05:02 While we continue our period of extended monitoring, the availability of the Outlook on the Web service has reached expected availability levels. We're continuing to optimize the environment to address the remaining impact. This quick update is designed to give the latest information on this issue. Update: 26-Nov-2024 04:03 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We're still addressing the remaining impact to the Outlook on the web service that is affecting some users. We've applied mitigation actions to reduce the mail queues and we're continuing the extended period of monitoring to ensure stability continues. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 11:00 AM UTC Update: 26-Nov-2024 03:04 We've isolated the cause of mail queue delays and have restarted the affected infrastructure to drain stalled queues. We'll remain in an extended monitoring phase until this draining is completed and we can consider the incident fully recovered. This quick update is designed to give the latest information on this issue. Update: 26-Nov-2024 00:04 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We're continuing to address lingering impact to the Outlook on the web service that is still affecting some users. In parallel, we're investigating some mail queuing delays that is resulting in mail taking longer than expected to be delivered. Due to the impact of this incident, we will enter a period of extended monitoring prior to declaring this issue resolved. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 9:00 AM UTC Update: 25-Nov-2024 23:32 Impact to core services have been restored with the exception of Outlook on the web, which we'll continue to monitor and actively troubleshoot until full recovery. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 22:31 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We've successfully restored functionality for all previously impacted services and users with the exception of Outlook on the web, which is showing prolonged impact for a small number of users. We'll continue carefully monitoring the service health and focus on troubleshooting this persisting impact to fully recover for the remaining affected users. We'll provide a new timeline within the next update. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 5:00 AM UTC Update: 25-Nov-2024 21:10 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading the calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see an updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experience issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors when running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: Our monitoring indicates that a large portion of affected users and services are seeing recovery following our mitigation efforts. We're working on addressing the lingering regions that are still seeing small impact to fully restore service availability, which we still expect to complete by Tuesday, November 26, 2024, at 3:00 AM UTC. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Estimated time to resolve: We estimate full recovery by Tuesday, November 26, 2024, at 3:00 AM UTC. Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 3:30 AM UTC Update: 25-Nov-2024 19:01 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels. - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature. Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: The service is continuing to incrementally recover. We're forecasting that impact will be fully remediated by Tuesday, November 26, 2024, at 3:00 AM UTC, though we're closely monitoring our progress and will provide an updated timeline should the estimate change. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Estimated time to resolve: We estimate full recovery by Tuesday, November 26, 2024, at 3:00 AM UTC. Preliminary Root Cause:Due to a recent change that decommissioned a backend service, requests were directed to an incorrect endpoint. This resulted in request handling issues and ultimately affected machines' processing capabilities, which led to impact. Next Update By: Tuesday, November 26, 2024, at 2:00 AM UTC Update: 25-Nov-2024 17:09 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels within - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature within Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: We identified a change in the environment that resulted in an influx in request retries routed through affected servers. Our optimizations, which enhanced the infrastructure's processing capabilities, continue to provide incremental relief. We're monitoring the service and continuing our work to perform any follow-up actions or opening additional workstreams needed to fully resolve the problem. We understand the significant impact of this event to your organization, we're treating this issue with the highest priority, and we're working to provide relief as soon as possible. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change resulted in a portion of infrastructure not operating as expected. Next Update By: Tuesday, November 26, 2024, at 12:00 AM UTC Update: 25-Nov-2024 15:25 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels within - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature within Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Viva Engage - Users may be unable to load topics in feeds and threads. - Users may see searches or autocomplete fail for topic experiences. - Users' Microsoft Teams Q&A experiences may not show user names. - Users may not be able to load their home feeds. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. - Users may not be able to load Copilot experiences within the Viva Engage service. - Users may be unable to submit queries to Copilot within OneNote through the chat pane and context menu. Users may receive an apology or "at capacity" message." Current Status: Services are showing significant signs of restoration following our infrastructure optimizations to redistribute traffic and restart affected servers, upwards of 90 percent recovery as we continue to increase processing capability. We're continuing to perform appropriate optimizations on the subset of remaining affected machines to resolve the issue for all users. Scope of Impact: This issue could impact any user hosted globally; however, users are experiencing varying degrees of impact depending on how they are routed through the affected infrastructure and what action they are trying to perform. Impacted users attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change resulted in a portion of infrastructure not operating as expected. Next Update By: Monday, November 25, 2024, at 10:00 PM UTC Update: 25-Nov-2024 14:52 We're continuing to redirect traffic and apply targeted remediation to impacted servers. We've observed initially positive results in some areas, and we're monitoring service health closely as our mitigation workstreams continue in parallel. We understand the significant impact of this event to your organization, we're treating this issue with the highest priority, and we're working to provide relief as soon as possible. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 14:15 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels within - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature within Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. Current Status: Our mitigative actions haven't provided relief as expected, and a portion of infrastructure remains in an unhealthy state. We determined that some of the targeted server restarts did not succeed due to processing issues, which are under investigation. We're currently focused on spreading traffic to healthy infrastructure, and we're seeing some recovery. Scope of Impact: Any user routed through affected infrastructure and attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change has resulted in a portion of infrastructure not operating as expected. Next Update By: Monday, November 25, 2024, at 9:00 PM UTC Update: 25-Nov-2024 12:37 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The impacted services and their impact are as follows: Exchange Online - Users may be unable to access using the following impacted connection methods: Outlook on the web, Outlook desktop client, Representational State Transfer (REST), Exchange ActiveSync (EAS) - Users may experience mail transport delays. Microsoft Teams - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Microsoft Purview - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Microsoft Fabric - Users may be unable to export content or set and view labels within - Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. SharePoint Online - Users may be unable to use the search feature within Microsoft Defender for Office365 - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Universal Print - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Power Automate for Desktop - Users may experience errors running flows that utilize cloud connectors. Microsoft Bookings - Users may be unable to access their bookings. Microsoft Copilot - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. Current Status: We're continuing to reroute traffic to alternate infrastructure and have reinitiated targeted server restarts to ensure the fix takes effect as expected. We're monitoring to confirm the restarts proceed successfully. We don't yet have an estimated time to resolution; however, we'll provide one as soon as it becomes available. Scope of Impact: Any user routed through affected infrastructure and attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change has resulted in a portion of infrastructure not operating as expected. Next Update By: Monday, November 25, 2024, at 7:00 PM UTC Update: 25-Nov-2024 11:44 We're rerouting traffic to alternate infrastructure to expedite mitigation. In parallel, we're resolving complications leading to the delays in targeted server restarts. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 11:15 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may be unable to use the following features within Microsoft Teams: - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Users may experience the following issues with Microsoft Purview: - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Users may be unable to export content or set and view labels within Microsoft Fabric. Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. Users may be unable to use the search feature within SharePoint Online. Users may be unable to perform the following actions within Microsoft Defender for Office365: - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. - Users may get a "You can't access this section" error when accessing sections of the Defender XDR portal, such as the Incidents and Alerts pages, that include affected Defender for Office 365 shared components. Users leveraging Universal Print may experience the following issues: - Users may be unable to Print via Universal Print. - Users may be unable to list Printers/Printer Shares on the Azure Portal Universal Print blade. - Users may be unable to Register Printers via Universal Print. Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop. Users may be unable to access their bookings within Microsoft Bookings. Users may be unable to use some Microsoft Copilot features including: - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. Current Status: The fix which is addressing impact is continuing to deploy and has now reached 98 percent of the affected environment. Regarding recovery, we've encountered delays in our targeted restarts; however, we're identifying alternative options to expedite recovery of the affected infrastructure. Scope of Impact: Any user routed through affected infrastructure and attempting to use the functionalities outlined in the More info section of this communication may be affected by this event. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change has resulted in a portion of infrastructure not operating as expected. Next Update By: Monday, November 25, 2024, at 5:30 PM UTC Update: 25-Nov-2024 11:02 The patch has deployed 90 percent, and according to telemetry, service availability is recovering. An estimated time to completion of the fix is not yet available. Targeted server restarts are underway to address routing service issues, with priority on customers who are currently in business hours or beginning their business day. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 09:27 We're monitoring the progress of the fix, which has been deployed to approximately 60% of the affected environments. We're continuing our manual restarts on the remaining impacted machines. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 08:58 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may be unable to use the following features within Microsoft Teams: - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Users may experience the following issues with Microsoft Purview: - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Users may be unable to export content or set and view labels within Microsoft Fabric. Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. Users may be unable to use the search feature within SharePoint Online. Users may be unable to perform the following actions within Microsoft Defender for Office365. - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. - Users may experiences issues with viewing simulation reports, and content. Users may also be unable to print via Universal Print. Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop. Users may be unable to access their bookings within Microsoft Bookings. Users may be unable to use some Microsoft Copilot features including: - Users are unable to use the personal Copilot panel in meetings and post meetings. - Users are unable to see historic Copilot conversation history in meetings and post meetings. Current Status: We've started to deploy the fix which is currently progressing through the affected environment. While this progresses, we're beginning manual restarts on a subset of machines that are in an unhealthy state. Scope of Impact: Impact is specific to users who are served through the affected infrastructure. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A recent change has resulted in a portion of infrastructure not operating as expected. Next Update By: Monday, November 25, 2024, at 4:00 PM UTC Update: 25-Nov-2024 08:30 We've isolated the root cause and have developed a patch, which we're testing on a subset of components before full deployment. We're making sure that once we deploy the script, we're able to ensure a full recovery. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 06:50 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may be unable to use the following features within Microsoft Teams: - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Users may experience the following issues with Microsoft Purview: - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Users may be unable to export content or set and view labels within Microsoft Fabric. Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. Users may be unable to use the search feature within SharePoint Online. Users may be unable to perform the following actions within Microsoft Defender for Office365. - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Users may experiences issues with viewing simulation reports, and content. - Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center. Users may also be unable to print via Universal Print. Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop. Users may be unable to access their bookings within Microsoft Bookings. Current Status: We've identified a recent change which we believe has resulted in impact. We've begun to revert the change and are investigating what additional actions are required to mitigate the issue. Scope of Impact: Impact is specific to users who are served through the affected infrastructure. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 2:00 PM UTC Update: 25-Nov-2024 05:35 Title: Users may be unable to access or use some Microsoft 365 services and features User Impact: Users may be unable to access or use some Microsoft 365 services and features. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may be unable to use the following features within Microsoft Teams: - Users are unable to create or update Virtual Events, including webinars and Town Halls. - Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - Users are unable to create chat, add users and create or edited meetings. - Users are unable to create or modify new teams and channels. - Users may be unable to update presence. - Users may be unable to use the search function. - Users may not see updated list of files and links failing to load within the Chat shared tab. Users may experience the following issues with Microsoft Purview: - Users may be unable to access the Purview Portal, or Purview Solutions. - Users may experience delays in policy stamping and with Adaptive Scope Evaluations. Users may be unable to export content or set and view labels within Microsoft Fabric. Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied. Users may be unable to use the search feature within SharePoint Online. Users may be unable to perform the following actions within Microsoft Defender for Office365. - Users may be unable to create simulations, simulation payloads or end user notifications. - Users may experience issues with delivery for end user notifications and simulation messages - Users may experiences issues with viewing simulation reports, and content. Current Status: We've updated the More Info section to include a list of impacted services and scenarios. We're continuing to investigate recent changes, service telemetry and the failure paths to determine the root cause of impact. Scope of Impact: Impact is specific to users who are served through the affected infrastructure. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 12:00 PM UTC Update: 25-Nov-2024 03:54 Title: Users unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may also be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. Current Status: While we're continuing the restarts to improve service availability, we're thoroughly analyzing recent changes and the specific failure path to determine the next steps required to resolve the issue. Scope of Impact: Some users may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempting to use their calendar in Microsoft Teams. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 11:00 AM UTC Update: 25-Nov-2024 03:18 We are systematically restarting the affected components while continuing our investigation into the cause of the errors. This quick update is designed to give the latest information on this issue. Update: 25-Nov-2024 02:21 Title: Users unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Team User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may also be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. Current Status: Our ongoing service monitoring has determined that the issue is not limited to users in Asia Pacific, and instead that all users worldwide could be intermittently impacted. As such, we're expanding our restart efforts to ensure the issue is resolved for all users. Scope of Impact: Some users may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempting to use their calendar in Microsoft Teams. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 9:00 AM UTC Update: 25-Nov-2024 00:46 Title: Users in Asia Pacific unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Team User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may also be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. Current Status: We're continuing to work to restart the small subset of machines which have previously failed to restart. We're also continuing to add additional capacity to the affected environment to mitigate impact as quickly as possible. At this time, we expect the majority of users will experience relief from impact as we work to address the remaining issues. Scope of Impact: Some users in the Asia Pacific region may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempting to use their calendar in Microsoft Teams. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 8:00 AM UTC Update: 24-Nov-2024 23:56 Title: Users in Asia Pacific unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Team User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may also be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. Current Status: We've identified a small subset of machines which are failing to restart, resulting in persisting impact. We're further investigating what is causing the restarts to fail to determine our next steps. Scope of Impact: Some users in the Asia Pacific region may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempting to use their calendar in Microsoft Teams. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Preliminary Root Cause:A portion of infrastructure which supports mailbox and calendar functionality isn't operating as expected, resulting in impact. Next Update By: Monday, November 25, 2024, at 6:00 AM UTC Update: 24-Nov-2024 23:18 In addition to the infrastructure restarts, we're also working to add additional capacity to the affected environment to further assist in our impact mitigation efforts. This quick update is designed to give the latest information on this issue. Update: 24-Nov-2024 22:50 Title: Users in Asia Pacific unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Team User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: The affected scenarios are as follows: Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Users may also be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. Current Status: We've determined via our internal telemetry that the impact to Microsoft Teams calendars has self-recovered. To fully resolve the issue, we initiated restarts on affected infrastructure and are monitoring further to determine whether additional steps are required. We're also continuing our investigation into the root cause of this issue. Scope of Impact: Some users in the Asia Pacific region may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempting to use their calendar in Microsoft Teams. Start Time: Monday, November 25, 2024, at 1:20 AM UTC Next Update By: Monday, November 25, 2024, at 5:00 AM UTC Update: 24-Nov-2024 22:29 While we continue to investigate the source of the issue, we're restarting affected machines in an attempt to mitigate the user impact. This quick update is designed to give the latest information on this issue. Update: 24-Nov-2024 22:00 Title: Users in Asia Pacific unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Team User Impact: Users may be unable to access their Exchange Online mailboxes or their calendar functionality in Microsoft Teams. More Info: Here are the service impact scenarios affected: Exchange Online may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client - Representational State Transfer (REST) - Exchange ActiveSync (EAS) Current Status: Our automated systems alerted us to an issue in which some users may be intermittently unable to access their mailboxes in Exchange Online using any connection method or their calendar functionality in Microsoft Teams. We're analyzing a portion of Exchange Online infrastructure responsible for accessing mailboxes to understand the root cause of the issue and determine our next troubleshooting steps. Scope of Impact: Some users in the Asia Pacific region may intermittently be affected by this issue when attempting to access their mailboxes using any connection method in Exchange Online or attempt to use their calendar in Microsoft Teams. Next Update By: Monday, November 25, 2024, at 4:00 AM UTC |