Degraded performance impacting Orchestrator and Document Understanding in Europe

Incident Report for UiPath

Postmortem

Customer Impact

Between June 10, 2026, 14:15 and 16:35 UTC, customers hosted in the Europe region experienced delays or failures in services that rely on internal event processing.

Affected functionality included:

  • Jobs remaining in a pending state or failing to start
  • UiPath Document Understanding™ operations timing out or failing
  • Automation Solutions installation and uninstallation operations timing out
  • Delays in synchronization of resources such as folders, processes, queues, and tags

No data was lost during the incident. Events that could not be processed during the disruption were automatically processed after service was restored.

Total duration: approximately 2 hours and 20 minutes.

Root Cause

The incident was caused by a configuration issue during a planned infrastructure migration in the North Europe region.

A required configuration update was not fully applied across all service components before traffic was moved to the new infrastructure. As a result, some services were unable to communicate with the messaging system they depend on, causing delays and failures in affected operations.

Recovery took longer than expected because a separate issue temporarily delayed the rollout of the corrective configuration. Once the configuration was successfully applied, service was restored.

Detection

At 14:52 UTC, our messaging team observed a drop in traffic to the original backend. This was initially attributed to the normal, seasonal decline in traffic that occurs after business hours. On closer inspection it became clear this was a genuine anomaly: traffic to the original backend had dropped as expected when the failover was triggered, but the corresponding increase in traffic to the new backend did not fully materialize. This discrepancy prompted an immediate investigation.

Internal service alerts for pending jobs, Document Understanding test failures, and Automation Solutions timeouts also fired during this period, corroborating the customer impact. We confirmed at 15:25 UTC that Orchestrator was receiving "not found" errors when attempting to reach the messaging component.

Response

  • 14:07 UTC — A configuration change to trigger the on-demand failover was merged.
  • 14:15 UTC — Orchestrator began receiving "not found" errors when connecting to the messaging component.
  • 14:52 UTC — Our messaging team detected a traffic anomaly and began investigating.
  • 15:03 UTC — A change to revert the failover was prepared.
  • 15:10 UTC — The revert was merged; however, the configuration-propagation automation did not run due to an ongoing GitHub incident.
  • 15:20 UTC — GitHub publicly reported its incident, confirming why our configuration changes were not propagating.
  • 15:25 UTC — We confirmed the impact to Orchestrator and engaged the team responsible for configuration propagation.
  • 16:35 UTC — The corrective configuration propagated to the affected clusters once GitHub recovered; errors stopped and service was restored. Events that were not delivered during the impact window were replayed.

Follow-Up

We are implementing the following improvements:

  1. Improved alerting in Orchestrator to detect and surface errors from internal messaging endpoints more quickly, reducing the time to detection.
  2. Pre-change verification to confirm that new backend configuration has fully reached all clusters before a traffic switch is initiated.
  3. Reducing our dependence on a single external provider in the configuration-propagation path, so that corrective changes can still be applied when an upstream provider is degraded.
Posted Jun 12, 2026 - 08:21 UTC

Resolved

A fix has been applied, and the issue impacting Orchestrator workflows, Document Understanding validation experiences, and Agents jobs in Europe has been resolved.

Impact: Users may have experienced degraded performance for validation tasks started through Document Understanding APIs, delays or missing exceptions for review in the Build page, and Agents jobs stuck in a pending state.

We are continuing to monitor service health to ensure stability.
Posted Jun 10, 2026 - 17:05 UTC

Identified

We have identified degraded performance impacting Orchestrator workflows, Document Understanding validation experiences, and Agents jobs in Europe.

Impact: Users may experience degraded performance for validation tasks started through Document Understanding APIs. Exceptions for review may also be delayed or may not appear in the Build page. Users may also see Agents jobs stuck in a pending state.

Our teams are working on mitigating the issue and assessing the broader service impact, as additional services may be affected. We will share more details as mitigation progresses.
Posted Jun 10, 2026 - 16:40 UTC

Update

We are continuing to investigate degraded performance impacting Orchestrator workflows, Document Understanding validation experiences, and Agents jobs in Europe.

Impact: Users may experience degraded performance for validation tasks started through Document Understanding APIs. Exceptions for review may also be delayed or may not appear in the Build page. Users may also see Agents jobs stuck in a pending state.
Posted Jun 10, 2026 - 16:20 UTC

Update

We are continuing to investigate this issue.
Posted Jun 10, 2026 - 16:18 UTC

Update

We are continuing to investigate degraded performance impacting Orchestrator workflows and Document Understanding validation experiences in Europe.

Our teams are also assessing the broader service impact, as additional services may be affected. We will share more details as the investigation and rollback progress.
Posted Jun 10, 2026 - 15:59 UTC

Investigating

We are investigating reports of degraded performance impacting Orchestrator workflows and Document Understanding validation experiences in Europe.

Impact: Users may experience degraded performance for validation tasks started through Document Understanding APIs. Exceptions for review may also be delayed or may not appear in the Build page.
Posted Jun 10, 2026 - 15:51 UTC
This incident affected: European Union (Orchestrator, Document Understanding, Agents) and Delayed EU (Orchestrator, Document Understanding).