UI issue - reporting engines states

Incident Report for Firebolt

Resolved

We have corrected the final set of engines and all services are available. Please contact support if you have any issues.
Posted Jan 19, 2023 - 18:43 UTC

Update

We have resolved the issue with a majority of the engines and are working through the last small fraction of them. We will post when completely resolved. In the interim if you see some engines running and others failed please notify support.
Posted Jan 19, 2023 - 18:20 UTC

Update

We have confirmed the fix and are actively working on applying the fix across all engines. This fix will take approx 1 hour. As mentioned prior this issue should be purely UI related. If you notice connectivity issues during this hour please reach out to support as that is most likely an unrelated issue.
Posted Jan 19, 2023 - 17:15 UTC

Update

We are currently working through the solution which will propagate across accounts over the next 15 minutes. We will continue to update as we resolve the issue and will mention when it's completely fixed.
Posted Jan 19, 2023 - 16:49 UTC

Update

We are still working on a resolution and can confirm in the majority case it is a UI issue. There are a few cases where engines have become unresponsive which we need to fix manually.
If an engine is not responsive via the API/SDK, please contact us, and do not restart/stop the engine.
Posted Jan 19, 2023 - 16:28 UTC

Identified

We identified the problem and working on a fix.
We will provide an ETA in 15 minutes.
Posted Jan 19, 2023 - 16:10 UTC

Update

We are still investigating and will update in 10 minutes or when we have a clear ETA
Posted Jan 19, 2023 - 15:56 UTC

Update

We are actively working on a fix for the UI issue. Please do not restart or stop engines until we have the issue resolved. We will have an ETA on a fix in 15 minutes.
Posted Jan 19, 2023 - 15:44 UTC

Investigating

We have a current multi account and multi engine incident. We're still identifying the impact and severity, but as of now it looks like it impacts only the UI and not the engines themselves.
We are still checking it.
Posted Jan 19, 2023 - 15:27 UTC
This incident affected: Firebolt Components (Web Application).