Cobalt Technology - Fiber Outage in Ft Worth Causing Congestion – Incident details

Fiber Outage in Ft Worth Causing Congestion

Resolved
Degraded performance
Started 10 months agoLasted 2 days

Affected

Wichita Falls (Stadium Tower)

Degraded performance from 8:41 PM to 12:07 AM

Iowa Park ( City Limits)

Degraded performance from 8:41 PM to 12:07 AM

Iowa Park (Rural-South)

Degraded performance from 8:41 PM to 12:07 AM

Lakeside City

Degraded performance from 8:41 PM to 12:07 AM

Wichita Falls (Western Rural - Arena Rd. Tower)

Degraded performance from 8:41 PM to 12:07 AM

Updates
  • Resolved
    Resolved

    The upstream carrier has reported that the outage has finally been resolved. We are extremely displeased with the length of time it took for the issue to be resolved, so we have already begun the process of adding an additional upstream connectivity provider to strengthen the network and prevent future problems such as this. We apologize for the extended issues and are committed to improving the redundancy systems as soon as possible.

  • Identified
    Identified

    The national fiber carrier updated us on the situation. They are having ongoing issues with equipment in Ft Worth that is affecting a lot of their circuits across North Texas. We have escalated our ticket once more, but they already have personnel on-site working on their issue in Ft. Worth.

  • Monitoring
    Monitoring

    The national fiber carrier has been taking an unusually long amount of time to find the issue. They dispatched ground crews in the middle of the night to find the damaged fiber but have not updated us on an estimated resolution time as of yet. Our system continues to run on the backup circuit, but like we posted in a previous update, we have decided to add an additional fiber upstream provider in our network in the near future to mitigate these sorts of issues.

  • Identified
    Identified

    The same national fiber carrier that was having issues yesterday, and whom had fixed the issue last night, is having another outage in Ft. Worth. We have escalated the issue and are working with a high level repair team to get the issue resolved ASAP.

    As a result of this instability we will be signing a new contract with an additional fiber carrier to increase refundancy in our network.