Cobalt Technology - Upstream Fiber Cut Causing Degraded Speeds for Wichita Falls, Iowa Park, Lakeside City, and Memorial Stadium areas. – Incident details

Upstream Fiber Cut Causing Degraded Speeds for Wichita Falls, Iowa Park, Lakeside City, and Memorial Stadium areas.

Resolved
Degraded performance
Started over 1 year agoLasted 1 day

Affected

Wichita Falls (Stadium Tower)

Degraded performance from 11:35 PM to 2:05 AM

Iowa Park ( City Limits)

Degraded performance from 11:35 PM to 2:05 AM

Iowa Park (Rural-South)

Degraded performance from 11:35 PM to 2:05 AM

Lakeside City

Degraded performance from 11:35 PM to 2:05 AM

Wichita Falls (Western Rural - Arena Rd. Tower)

Degraded performance from 11:35 PM to 2:05 AM

Updates
  • Resolved
    Resolved

    The fiber cut has been repaired and speeds are back to normal.

  • Update
    Update

    The upstream fiber provider has given an update. The repair crew has bored the new hole and are about to pull conduit and fiber through it. The location that was damaged affected multiple major Tier 1 carriers at the same time, so crews are working diligently to repair the lines.

    Our network is outside of peak usage times right now, so customers should not se degraded speeds until overall usage begins to pick up around 6 PM. We are expecting connection to be back to full capacity by then, but we cannot know for sure since we are relying on a national ISP to repair the damage.

    These fiber cuts have become exceedingly frequent in the last 6 months so we are working on an internal plan to add capacity to alleviate load during times like these.

  • Identified
    Identified

    One of our upstream fiber providers had a fiber cut between Wichita Falls and DFW. The Wichita Falls, Iowa Park, Lakeside City, and Memorial Stadium networks are currently running on the backup service until the fiber is repaired.

    Holliday is at full speeds and is not affected by the fiber cut.

    Some people have expressed concern about the frequency of fiber cuts, so here is a brief explanation of why it keeps happening.
    All internet connectivity to the greater-Wichita Falls area comes in through fiber optics lines from Lubbock, DFW, or Oklahoma City. Running fiber 200+ miles is extremely expensive, so it is done by the large Tier 1 companies such as AT&T and others. Smaller providers then rent space on the fiber optic lines to get connection to the big city networks.
    There is a great deal of construction going on between Wichita Falls and DFW, and a backhoe or boring crew hitting a line anywhere between the cities will cause a complete outage to anyone down the line, just like hitting a water main will. Many of the crews doing work in DFW are not following proper procedures to call in underground utility locates and are hitting many different utilities underground, including fiber optic lines.

    We protect against this by paying for more than 1 fiber optic line that takes diverse geographical routes to DFW so that the likelihood of a total outage is greatly reduced. That is why the network is able to continue running during a fiber cut.

    We will provide updates as the upstream provider provides updates. It will most likely be repaired during the night.