On June 8, 2021, we experienced an outage beginning at approximately 2:30 p.m. ET, lasting until 4:36 p.m. ET. Your service may have been intermittently impacted for some or all of this time and we apologize for the inconvenience.
This outage was caused initially by a network loop inadvertently introduced into our West Coast Data Center (WCDC). This network loop resulted in packet loss and connectivity issues that ultimately led to our primary ns1 nameserver failing and becoming unresponsive. The ns1 failure then routed all nameserver traffic to our ns2 nameserver, which was undergoing maintenance at the time, resulting in DNS resolution to our nameservers to fail, impacting both internal assets and customer websites and email.
The initial maintenance on our ns2 nameserver was intended to be non-intrusive and improve our DNS infrastructure. This improvement was to help us prevent DNS related service outages in the future and faster DNS resolution in the event that nameserver-related services need to be restarted.
Completing this maintenance is still a priority for us, but we will take preventative measures that were learned in yesterday’s events to avoid such outages in the future, to include load testing up to multiple times (4x-6x, or more) the expected load.
We understand how disruptive these events can be, and we appreciate your patience and understanding as we continue to bring enhancements to our network to improve your web hosting experience. We apologize for any inconvenience caused due to this situation.
If you have any questions or concerns, please contact our Customer Service and Technical Support departments, available 24/7/365.
