Cobalt Technology - Datacenter Instability – Incident details

This incident has been resolved. Earlier today we implemented a change in our core Dallas datacenter network that was intended to increase redundancy in the network, but it inadvertently injected duplicate routes in the network and caused traffic to get disrupted. We worked throughout the afternoon and evening to resolve the issues and we believe we have restored stability to the network. This issue came as a result of our overarching plan to bring a higher level of redundancy and resiliency to our network to protect ourselves from future outages. We will continue to implement carrier-grade practices in our network to increase stability as we grow.

We have implemented a fix that appears to have stabilized the network. We are going to keep an eye on it to see if the problem creeps back in. We are leaving the incident open for now and categorizing it as Degraded Performance until we are confident the issue has been resolved.

Connectivity in our Dallas datacenter network appears to be cutting in and out. We are in talks with our network engineers to implement a fix.