Certain GPU VMs in us-northcentral1-a are unavailable

Incident Report for Crusoe Cloud

Resolved

We have mitigated the issue, internal services are back online and customers can start turning their VMs on now.

We will provide a postmortem later this week.
Posted Jul 25, 2023 - 03:47 UTC

Update

We are continuing to work towards resolution, and internal services are back online. We are running additional testing and will notify customers when VMs are able to come back online.

We will provide another update no later than 0330 UTC on July 25th.
Posted Jul 25, 2023 - 03:01 UTC

Update

We are continuing to work towards resolution, and internal services are coming back online. We will update customers when VMs are ready to come online

We will provide another update no later than 0300 UTC on July 25th.
Posted Jul 25, 2023 - 02:18 UTC

Update

We are continuing to work towards resolution, no specific changes since the last update.

We will provide another update at 0215 UTC on July 25th.
Posted Jul 25, 2023 - 01:46 UTC

Update

We are continuing to work towards resolution, no specific changes since the last update.

We will provide another update at 0130 UTC on July 25th.
Posted Jul 25, 2023 - 01:15 UTC

Update

We are continuing to work towards resolution, no specific changes since the last update.

We will provide another update at 0100 UTC on July 25th.
Posted Jul 25, 2023 - 00:35 UTC

Update

We are continuing to work towards resolution, no specific changes since the last update.

We will provide another update at 0030 UTC on July 25th.
Posted Jul 25, 2023 - 00:07 UTC

Update

We are continuing to work towards resolution, no specific changes since the last update.

We will provide another update at 2400 UTC.
Posted Jul 24, 2023 - 23:35 UTC

Update

Certain GPU enabled VMs in `us-northcentral1-a` remain unavailable, as do VM related API operations.

We will provide another update at 2330 UTC.
Posted Jul 24, 2023 - 23:07 UTC

Identified

Certain GPU enabled VMs in `us-northcentral1-a` are currently unavailable. All API operations to start/stop VMs are also unavailable.

We will update at 2300 UTC (~15 minutes) with further updates.
Posted Jul 24, 2023 - 22:43 UTC
This incident affected: API (API) and GPU Virtual Machines (us-northcentral1).