In my day job I have the arduous task of working with Telcos from across the globe. I’ve been working in and around networking for 14 years so it’s nothing new, carriers are a fact of life, part of a network guys existence, however from experience it never gets any easier.
In the last 4 weeks, my colleagues and I have been dealing with 2-3 outages per week, none of them planned or changes we’ve made. Each time the issue was solely the fault of the provider… I completely understand, providers have faults, failure happens and you need to plan for it! That’s why we have backup services, alternate connectivity and strategies for dealing with outages.
However what really bugs me is the response, or more like lack of response during failures. Thinking back, I can’t remember a good experience with a Telco NOC in recent times. Support Engineers working in these NOCs are hidden away behind ticketing systems or email. There is almost no direct access and there is little regard for fixing your problem in a timely manner.
A provider I deal with recently started creating pre-emptive tickets in their system for site down emergencies, but do they do anything with them? Arh no! During site outages, you’re the one responsible for contacting them. It’s you that needs to chase ETRs and the NOC shows no interest or urgency in fixing your problem.
9 out of 10 times, I’m forced to escalate the fault to my local Account Manager, who then contacts the Duty Manager, who then ups the priority on the case. Why is service so poor? We pay good money for these services and expect to receive the appropriate level of service.
In 2015 I’ll be looking into my options here, there is a lot of talk about SDN WAN, after the last few weeks it’s now at the top of my priority list.
