Investigating Intermittent DNS Problems for Some Users

We are currently investigating an influx of tickets related to connectivity problems. At this time, the issue is hard to nail down as we are getting reports from multiple operating systems and applications, suggesting that it was not a software update that is causing the issue.

Further, the issue is causing outages on all clusters for some users, and only some clusters for others.

Even worse, we are unable to replicate the problem in the lab on any of the reported operating systems.

We will continue investigating this issue and we are working with multiple customers who have this issue to get it resolved.

If you are currently having connectivity problems, please contact us at customer care and give us any information that you can about your operating system, ISP, and the clusters that are affected for you. It could contribute to us finding a solution faster.

The vast majority of our users appear to be unaffected, as all of our clusters have active network activity.

Thank you for your patience with this issue. We will nail it down quickly.

Edit: We have found a temporary solution. The problem seems to be related to OpenNIC DNS and tied to how our servers forward DNS traffic. We are implementing a temporary measure server-side to get everyone online while we further investigate the issue. All servers will temporarily be using OpenDNS in the interim.

Server clusters will be issued rolling restarts as we implement this change, downtime of the clusters will be less than 1 second. If you are disconnected from the service, you can simply reconnect immediately.

< last
next >