This is an interesting topic and I expect to get some heat from this one.  There are a few things you are almost guaranteed in telecom.  Most are avoidable and all are able to be engineered around as to mitigate impact.  I love the statement “ This is costing me Millions of Dollars”  this may be true, but then why is it not worth avoiding?

Unrealistic Expectations and Resolves.

• If this circuit is not installed by XYZ date “ I am losing millions of dollars”.
  • Valid Argument, however if you are ordering any service expect delays.  Most all services above a T1 enter on Fiber.  Yes even a DS3 because they are delivered at least on an OC3.  If there is no fiber present a build out must occur.  If fiber is present there must be capacity and the equipment carded out properly.  This takes time.  If anyone else is on the current infrastructure it must be scheduled.  You should get continuous updates.  You should know in advance if you are going to miss a target date.  If you need it in by a certain date,  give it twice the expected time. A T1 is 4 to 6 weeks  Give it 8 to 12 Weeks.  It is from the time you place the order.  Don’t sit on it until it is only a week out then say you need it in 5 days. Did I mention a facility check will not happen until an order is placed?
  • T1’s can have an issue as well.  It is more common in rural areas. Lack of copper Pairs.
  • Ethernet is another fun one.  Ethernet can deliver over fiber or copper (EoC) and can take some time to turn-up.
• The circuit is down and it needs to be back up now because I am losing millions of dollars.
  • If you need a circuit at its most critical point it will fail.  Let’s face it Murphy’s law is alive and true.  If this is true it screams redundancy.  The type of redundancy is dependent on your need.  If you don’t know how important it is then I encourage you to go pull the wire on the T1 or remove a cable on the DS3. If your business screeches to a stop you need redundancy.  If your consultant or technician turns red in the face and steam comes out of his ears it is one of two things.
    • (S)HE has not thought of the issue. You need to find a new consultant or engineer.
    • (S)HE has been told no to proposals for whatever reason. And you are about to get hit with something hard so run.  This is your fault, because remember “I am losing millions of dollars”.
• Ethernet for WAN connectivity.  Expect delays in diagnosis.  A tech may need to be physically dispatched to each connection point to test.  This takes time.  Much more time than traditional telecom services.  It is difficult to coordinate multiple people and takes time.  Did I mention this takes time?  Did I mention Redundancy because this takes time?  Did you pull the plug yet?
• The SLA says 99.99999999999999999999999999999999999 Up time.
  • Who in their right mind believes more 9’s = more up time?
    • This is a token.  You are paying $ 14 a day for a T1. Your circuit is down for 1 to 4 hours.   An SLA will not get you up any faster. 
    • NO Carrier can Guarantee you will NEVER GO DOWN.  No Auto Manufacture can guarantee you will never have a Car Problem (Toyota are you listening? Probably not.)
    • SLA = Money back for an outage not to excede the cost of the circuit.

Real current issue.
Spending thousands of dollars for a radio add.  Internet Connection goes down and all of the voice calls were directed to a VOIP phone number.  Bad Choice, but Ok  How could It be made better? 

This is an opinion and there are numerous ways to prevent form loosing these potential inbound calls.  It all comes down to planning.

If you pull the plug and your tech still sits there with his feet on the desk eating his sandwich and drinking his soda.  Sit and wait for 5 to 10 Min.  If you don’t have half the company ringing his phone off the hook or running in.  You have an amazing tech give him a raise and a well deserved hi five.

If the Tech logs into his computer and does something for a minute or two and goes back to his sandwich.  Again High five and give him a slightly bigger budget.  This one is thinking out of the box and doing the best with what you are giving him.

Happy testing.