In a new revelation from the Snowden files, it has been revealed that the NSA has been tapping the unencrypted links between datacenters of major email providers and possibly more.
The program, dubbed "Muscular" places a tap on the links between datacenters for content delivery network style services. Apparently these links are unencrypted, which amazes me unless these companies were compelled to do so. It does create plausible deniability for the companies transferring the data. These networks are private, but the size and scope of Google facilities and the technical expertise of state-sponsored eavesdropping still make it horribly irresponsible to move customer data around "in the clear".
What is important here is these transfers are not metadata. This is the full content of the email. To understand why it is impossible that this is metadata you need to fundamentally understand how a Content Delivery Network (called a CDN for the rest of this article) works.
A CDN is a network layout for optimizing performance. Conceptually, a CDN clones data accross many servers in order to serve websites, email, streaming content, and other files in order to serve them to the end-user faster. Doing this has multiple advantages. In bringing the content physically closer to the user, the data requires less travel to get to the user. This means faster delivery of data, and more responsiveness for interactive web programs, games, and other systems.
So, for a system like Gmail, you need to clone your data across all facilities in a timely manner. This is so if i send my colleague Micah Greene an email on Gmail, and it goes to my local CDN datacenter, and Micah checks his email five minutes later, and it is checking a different datacenter on the CDN because he is physically closer to it, it needs to already be there.
So these datacenters "sync" with one another on a regular, timely basis. This data syncing is not encrypted (WTF) and this is the data the NSA is tapping in to. This is not metadata. It is the full email / file / video with no protections to keep your data safe. They are sifting through this mountain of information and pulling out the bits they like, to the tune of 60GB per day according to the report. (
60GB of email is a LOT of email. The iconic novel Les Misérables, which runs 1488 pages in English, and is one of the longest reads of all time, is about 871KB. This is 68000 of those PER DAY being collected, and those are only the ones they are keeping. If we made a stack of those novels, it would be over two and a half miles tall with one day of collection. A Mount Everest every three days.
It is also important to note that Google and Yahoo, who were specifically named in this latest release, were not aware of this intrusion into their infrastructure. Google has said it is now "in an arms race" to encrypt all of the internal communications between the interlinked datacenters. As of this writing, Yahoo has not made an announcement to reassure users that this vulnerability is being closed.