4:05 pm

The Virtual PBX we use, RingCentral.com completely went down nation wide and all Phone service was completely interrupted.

================= 

4:15 pm

The delicious annectdote in all this is we had just five minutes before the outage fixed a number of configuration and programming issues on the local phones for one of our clients.

Virtual PBX used to not work at all and per our experiments over the summer it seems to have come of age.  Getting the correct firmware and configurations for the phone with the RingCentral service, however was a 3 day troubleshooting project, which we were proud to have finished and to report to the client with our success, at 4pm - and then the whole system went kaput!  Sweet irony, right?

Calls to Ring Central engineering don't work correctly - we can hear them but they can't hear us.  Some of the same problems we experienced in setting up configurations on our local network.

Eventually we get through and ---- they tell us service is down.  Estimate is One Hour Minimum for repair, so I advise our clients it's out for the rest of the day.

We expend so much effort finding the right solutions and working to integrate them into Solutions for clients, the worst thing to then have to tell a client the system just doesn't work and there's nothing we can do.

The real question in the back of my mind is --- will it happen again?

============

4:55 pm

I received a call from a client.  Phones working!  ---- NOT.  Apparently ring central fixed the calling within the network, but upon checking from my cell phone, dialing IN to the lines doesn't work. 

=============

6:45 pm

A call to our switchboard and to our clients indicates the phones are working again.

Technology is sometimes like a patient lover, a "sure thing," when it's all working well.  Imagine the complexity of the Power Grid that brings electricity to the wall - never do we wake up thankful that we're getting power.  For the most part that's been debugged, with the exception of the occasional cataclysm.  But I can imagine in 1915 it was a whole different matter to be thankful power was working on a given day.  Amazing to think the internet is only a teenager and these digital services, especially at our level deploying Enterprise Infrastructure to small and medium businesses are at such an early stage, and still working so amazingly well most of the time.  But when they don't it feels like hell hath no fury like a system down.