Thread: U.S. intelligence mining data from 9 U.S. Internet companies in broad secret program

U.S. intelligence mining data from nine U.S. Internet companies in broad secret program

By Barton Gellman and Laura Poitras, Barton Gellman and Laura Poitras Jun 06, 2013 09:43 PM EDT
The Washington Post
Thursday, June 6, 2:43 PM E-mail the writer

The National Security Agency and the FBI are tapping directly into the central servers of nine leading U.S. Internet companies, extracting audio, video, photographs, e-mails, documents and connection logs that enable analysts to track a person’s movements and contacts over time.
The highly classified program, code-named PRISM, has not been disclosed publicly before. Its establishment in 2007 and six years of exponential growth took place beneath the surface of a roiling debate over the boundaries of surveillance and privacy. Even late last year, when critics of the foreign intelligence statute argued for changes, the only members of Congress who know about PRISM were bound by oaths of office to hold their tongues.

What has the government been doing? Is it legal? Does it mean some bureaucrat somewhere has heard all your phone calls? Read on to find out.

An internal presentation on the Silicon Valley operation, intended for senior analysts in the NSA’s Signals Intelligence Directorate, described the new tool as the most prolific contributor to the President’s Daily Brief, which cited PRISM data in 1,477 articles last year. According to the briefing slides, obtained by The Washington Post, “NSA reporting increasingly relies on PRISM” as its leading source of raw material, accounting for nearly 1 in 7 intelligence reports.
That is a remarkable figure in an agency that measures annual intake in the trillions of communications. It is all the more striking because the NSA, whose lawful mission is foreign intelligence, is reaching deep inside the machinery of American companies that host hundreds of millions of American-held accounts on American soil.
The technology companies, which participate knowingly in PRISM operations, include most of the dominant global players of Silicon Valley. They are listed on a roster that bears their logos in order of entry into the program: “Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” PalTalk, although much smaller, has hosted significant traffic during the Arab Spring and in the ongoing Syrian civil war.
Dropbox , the cloud storage and synchronization service, is described as “coming soon.”
Government officials declined to comment for this story.

Roots in the ’70s

PRISM is an heir, in one sense, to a history of intelligence alliances with as many as 100 trusted U.S. companies since the 1970s. The NSA calls these Special Source Operations, and PRISM falls under that rubric.
The Silicon Valley operation works alongside a parallel program, code-named BLARNEY, that gathers up “metadata” — address packets, device signatures and the like — as it streams past choke points along the backbone of the Internet. BLARNEY’s top-secret program summary, set down alongside a cartoon insignia of a shamrock and a leprechaun hat, describes it as “an ongoing collection program that leverages IC [intelligence community] and commercial partnerships to gain access and exploit foreign intelligence obtained from global networks.”
But the PRISM program appears more nearly to resemble the most controversial of the warrantless surveillance orders issued by President George W. Bush after the al-Qaeda attacks of Sept. 11, 2001. Its history, in which President Obama presided over “exponential growth” in a program that candidate Obama criticized, shows how fundamentally surveillance law and practice have shifted away from individual suspicion in favor of systematic, mass collection techniques.

The PRISM program is not a dragnet, exactly. From inside a company’s data stream the NSA is capable of pulling out anything it likes, but under current rules the agency does not try to collect it all.
Analysts who use the system from a Web portal at Fort Meade key in “selectors,” or search terms, that are designed to produce at least 51 percent confidence in a target’s “foreignness.” That is not a very stringent test. Training materials obtained by the Post instruct new analysts to submit accidentally collected U.S. content for a quarterly report, “but it’s nothing to worry about.”

Related stories
‘No Such Agency’ spies on the communications of the world

Anne Gearan 2:35 PM ET
The National Security Agency, nicknamed such for years, is the U.S. government’s eavesdropper-in-chief.


Report: NSA asked Verizon for all U.S. call data

Ellen Nakashima 4:51 AM ET
If document requiring company to submit phone records for millions of Americans is authentic, it would be the broadest surveillance order known to date.


All about the NSA surveillance program.

Timothy B. Lee 12:46 PM ET
What has the government been doing? Is it legal? Does it mean some bureaucrat somewhere has heard all your phone calls? Read on to find out.

Even when the system works just as advertised, with no American singled out for targeting, the NSA routinely collects a great deal of American content. That is described as “incidental,” and it is inherent in contact chaining, one of the basic tools of the trade. To collect on a suspected spy or foreign terrorist means, at minimum, that everyone in the suspect’s inbox or outbox is swept in. Intelligence analysts are typically taught to chain through contacts two “hops” out from their target, which increases “incidental collection” exponentially. The same math explains the aphorism, from the John Guare play, that no one is more than “six degrees of separation” from Kevin Bacon.
A ‘directive’
Formally, in exchange for immunity from lawsuits, companies like Yahoo and AOL are obliged accept a “directive” from the attorney general and the director of national intelligence to open their servers to the FBI’s Data Intercept Technology Unit, which handles liaison to U.S. companies from the NSA. In 2008, Congress gave the Justice Department authority to for a secret order from the Foreign Surveillance Intelligence Court to compel a reluctant company “to comply.”
In practice, there is room for a company to maneuver, delay or resist. When a clandestine intelligence program meets a highly regulated industry, said a lawyer with experience in bridging the gaps, neither side wants to risk a public fight. The engineering problems so immense, in systems of such complexity and frequent change, that the FBI and NSA would be hard pressed to build in back doors without active help from each company.
Apple demonstrated that resistance is possible, for reasons unknown, when it held out for more than five years after Microsoft became PRISM’s first corporate partner in May 2007. Twitter, which has cultivated a reputation for aggressive defense of its users’ privacy, is still conspicuous by its absence from the list of “private sector partners.”
“Google cares deeply about the security of our users’ data,” a company spokesman said. “We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government ‘back door’ into our systems, but Google does not have a ‘back door’ for the government to access private user data.”
Like market researchers, but with far more privileged access, collection managers in the NSA’s Special Source Operations group, which oversees the PRISM program, are drawn to the wealth of information about their subjects in online accounts. For much the same reason, civil libertarians and some ordinary users may be troubled by the menu available to analysts who hold the required clearances to “task” the PRISM system.
There has been “continued exponential growth in tasking to Facebook and Skype,” according to the 41 PRISM slides. With a few clicks and an affirmation that the subject is believed to be engaged in terrorism, espionage or nuclear proliferation, an
analyst obtains full access to Facebook’s “extensive search and surveillance capabilities against the variety of online social networking services.”
According to a separate “User’s Guide for PRISM Skype Collection,” that service can be monitored for audio when one end of the call is a conventional telephone and for any combination of “audio, video, chat, and file transfers” when Skype users connect by computer alone. Google’s offerings include Gmail, voice and video chat, Google Drive files, photo libraries, and live surveillance of search terms.
Firsthand experience with these systems, and horror at their capabilities, is what drove a career intelligence officer to provide PowerPoint slides about PRISM and supporting materials to The Washington Post in order to expose what he believes to be a gross intrusion on privacy. “They quite literally can watch your ideas form as you type,” the officer said.

Julie Tate and Robert O’Harrow Jr. contributed to this report.

Graphic: NSA slides explain the PRISM data-collection program

http://www.washingtonpost.com/investigations/us-intelligence-mining-data-from-nine-us-internet-companies-in-broad-secret-program/2013/06/06/3a0c0da8-cebf-11e2-8845-d970ccb04497_story.html

RELATED http://www.alipac.us/f9/senate-intel...lawful-280240/

CNN BREAKING NEWS U.S. intel agencies extracted private communications from big Internet companies, Guardian and Washington Post report.

And the media told us about this before the internet bloggers knew anything about it and made up more cool stuff to go along with it. But never fear. The bloggers will, and soon.

RELATED

How does the NSA's phone-record program work?

RELATED http://www.alipac.us/f9/top-u-s-inte...t-data-280298/

With databases, size has mattered--think of you own PC and when it gets full, which has restricted data mining (business intelligence). Now through Hadoop, the invention of Google, does away with all that.

http://strata.oreilly.com/2011/01/what-is-hadoop.html

"Where did Hadoop come from?


Mike OlsonMike Olson: The underlying technology was invented by Google back in their earlier days so they could usefully index all the rich textural and structural information they were collecting, and then present meaningful and actionable results to users. There was nothing on the market that would let them do that, so they built their own platform. Google’s innovations were incorporated into Nutch, an open source project, and Hadoop was later spun-off from that. Yahoo has played a key role developing Hadoop for enterprise applications.

What problems can Hadoop solve?


Mike Olson: The Hadoop platform was designed to solve problems where you have a lot of data — perhaps a mixture of complex and structured data — and it doesn’t fit nicely into tables. It’s for situations where you want to run analytics that are deep and computationally extensive, like clustering and targeting. That’s exactly what Google was doing when it was indexing the web and examining user behavior to improve performance algorithms.
Hadoop applies to a bunch of markets. In finance, if you want to do accurate portfolio evaluation and risk analysis, you can build sophisticated models that are hard to jam into a database engine. But Hadoop can handle it. In online retail, if you want to deliver better search answers to your customers so they’re more likely to buy the thing you show them, that sort of problem is well addressed by the platform Google built. Those are just a few examples....

[the rest of the discussion is mainly technical and not included here]

http://www.alipac.us/f9/inside-ameri...valley-280398/

http://www.alipac.us/f9/how-companie...online-280428/