Cobalt Technology - Degraded Network Performance – Incident details

Wichita Falls (Stadium Tower) experiencing major outage

Degraded Network Performance

Resolved
Degraded performance
Started about 1 year agoLasted about 9 hours

Affected

Wichita Falls (Stadium Tower)

Degraded performance from 1:00 PM to 9:41 PM

Holliday Tower

Degraded performance from 1:00 PM to 9:41 PM

Iowa Park ( City Limits)

Degraded performance from 1:00 PM to 9:41 PM

Iowa Park (Rural-South)

Degraded performance from 1:00 PM to 9:41 PM

Lakeside City

Degraded performance from 1:00 PM to 9:41 PM

Datacenter

Degraded performance from 1:00 PM to 9:41 PM

Updates
  • Monitoring
    UTC
    Monitoring

    We have implemented a fix for the Amazon AWS connectivity issues, and it appears to have resolved the problems. We will continue to monitor the situation to see if any additional fixes are needed.

  • Resolved
    UTC
    Resolved

    This incident has been resolved. We got confirmation from Amazon's network engineers that they were having issues with websites hosted in their datacenters, which caused a cascading affect across internet service providers. The problem was resolved at 4:41 PM today.

  • Update
    UTC
    Update

    We are in contact with Amazon AWS engineers, but they have not provided an estimated time to restoration. All other services are operational.

  • Identified
    UTC
    Identified

    We have identified the source of the connectivity issues. Many of Amazon AWS's servers are unreachable from our network, pointing towards a routing issue with an upstream provider or Amazon. We are working with Amazon engineers to resolve the issue.
    This will affect customer connections due to the fact that a large amount of websites and services are hosted on Amazon's web servers.

  • Investigating
    UTC
    Investigating

    We are investigating an issue causing connectivity issues to specific websites. Internet service is online across our network, but some websites are having issues loading. We are investigating the root cause of the issue and will bring a resolution ASAP.