IT4 Case Study
At about 9:45pm on Saturday, May 3rd, IT4 received a call from a long-term customer that their business was offline and in jeopardy.
The company is based internationally and has established themselves as an esteemed email marketing
service. Without any local IT staff of their own, they rely on one service provider for support. After
experiencing sporadic, but significant network downtime due to repeated DDoS attacks, the company
found themselves as the direct target of the attacks. Before they would restore services, the service
provider insisted that the company contract with a third party for DDoS mitigation services.
The company immediately signed for always-on protection with 2 leading providers of these services – Akamai and CloudFlare. The company experienced more attacks, but the DDoS mitigation provided protection. Not long after, the attacks shifted to adjacent IP blocks that belonged to other service provider’s hosted customers. Although the customer’s IP ranges were not the target of this attack the service provider believed that the attackers were trying to target them and took them offline again – this time indefinitely. It was late on Saturday night, the international customer had no employees in the United States, and they needed to contract with a new provider to get 500Mbps bandwidth installed as soon as possible.
IT4 reached out to on-net fiber and bandwidth providers immediately via voicemail and email to see who could help. Realistically, even if a service agreement was made on Sunday, it would take several weeks to get installed even with paid expedites because industry standard is 60-90 days for installation of these services. We needed another solution that would get them up faster.
On Sunday morning, IT4 reached out to another former customer, an India-based company called Web Werks, which collocates in the same service provider’s Data Center as the company in jeopardy. After setting up a conference cal” IT4 and Web Werks came up with a solution that could be installed in about 48 hours. A fiber cross connect from Web Werks would be run to the customer company to create a GRE tunnel to bring their IPs back online via the Web Werks connection. While the customer was extremely happy that we had come up with a short term solution, they wanted to continue swiftly down the path of reducing their reliance on their service provider. Their top three technical resources had immediate flights to the United States. IT4 decided to meet with them at the service provider’s Data Center on Monday morning to begin architecting a plan.
On Monday, IT 4 and the company in jeopardy devise a plan to:
1) Establish a collocation presence in a secondary data center site
2) Bring in 500Mbps bandwidth from an alternate ISP that serves the second data center site ensuring that it is routed from a different major Internet hub than the connections in Conshohocken
3) Establish a 10Gb connection between the existing and new data center for data replication
By Tuesday morning, the company is up and running on the connection from Web Werks and back online. IT4 sets up a meeting to tour the Sungard Availability Services colo facility near Philadelphia on Wednesday. By close of business, IT4 has proposals in hand from both Sungard and Xand for a private caged space to house six cabinets of equipment.
On Thursday, the company chooses Xand as their data center provider of choice and selects Sunesys for the fiber and bandwidth. IT4 worked with both providers to improve pricing and obtain an installation commitment of 15 calendar days for the customer. Later that afternoon a meeting was set up with the Xand facility with all involved parties to review the solution face-to-face and the customer executed agreements for the following services:
– Private colo cage with Xand for 6 cabinets and associated power
– 500Mbps InterNAP Route Optimized Bandwidth delivered to Xand data center via dedicated fiber from Sunesys to 3701 Market St. POP providing redundancy to 401 N. Broad St. POP
– 10Gb dedicated fiber from Sunesys between their service provider and Xand data centers for active/active data replication
For the next 2 weeks, IT4 remained in contact with all parties to ensure that things were moving smoothly and that we encountered no delays with the orders. As promised and on-schedule, all contracted services were installed and available for the customer as of 5/23. This is Phase 1 of a 3-Phased plan to set them up for future success and to reduce their exposure to outages of any type that can occur by using a single hosting and network provider.