Sam Biddle, The Intercept:
Thiel represents a perfect nexus of government clout with the kind of corporate swagger Trump loves. The Intercept can now reveal that Palantir has worked for years to boost the global dragnet of the NSA and its international partners, and was in fact co-created with American spies.
Palantir’s chief appeal is that it’s not designed to do any single thing in particular, but is flexible and powerful enough to accommodate the requirements of any organization that needs to process large amounts of both personal and abstract data.
It’s hard to square this purported commitment to privacy with proof, garnered from documents provided by Edward Snowden, that Palantir has helped expand and accelerate the NSA’s global spy network, which is jointly administered with allied foreign agencies around the world. Notably, the partnership has included building software specifically to facilitate, augment, and accelerate the use of XKEYSCORE, one of the most expansive and potentially intrusive tools in the NSA’s arsenal. According to Snowden documents published by The Guardian in 2013, XKEYSCORE is by the NSA’s own admission its “widest reaching” program, capturing “nearly everything a typical user does on the internet.” A subsequent report by The Intercept showed that XKEYSCORE’s “collected communications not only include emails, chats, and web-browsing traffic, but also pictures, documents, voice calls, webcam photos, web searches, advertising analytics traffic, social media traffic, botnet traffic, logged keystrokes, computer network exploitation targeting, intercepted username and password pairs, file uploads to online services, Skype sessions, and more.” For the NSA and its global partners, XKEYSCORE makes all of this as searchable as a hotel reservation site.
But how do you make so much data comprehensible for human spies? As the additional documents published with this article demonstrate, Palantir sold its services to make one of the most powerful surveillance systems ever devised even more powerful, bringing clarity and slick visuals to an ocean of surveillance data.
The fake scenario’s target, a cartoonishly sinister religious sect called “the Paraiso Movement,” was suspected of a terrorist bombing, but the unmentioned and obvious subtext of the experiment was the fact that such techniques could be applied to de-anonymize and track members of any political or ideological group.
However anxious British intelligence was about Palantir’s self-promotion, the worry must not have lasted very long. Within two years, documents show that at least three members of the “Five Eyes” spy alliance between the United States, the U.K., Australia, New Zealand, and Canada were employing Palantir to help gather and process data from around the world. Palantir excels at making connections between enormous, separate databases, pulling big buckets of information (call records, IP addresses, financial transactions, names, conversations, travel records) into one centralized heap and visualizing them coherently, thus solving one of the persistent problems of modern intelligence gathering: data overload.
Above all, these documents depict Palantir’s software as a sort of consolidating agent, allowing Five Eyes analysts to make sense of tremendous amounts of data that might have been otherwise unintelligible or highly time-consuming to digest. In a 2011 presentation to the NSA, classified top secret, an NDIST operative noted the “good collection” of personal data among the Five Eyes alliance but lamented the “poor analytics,” and described the attempt to find new tools for SIGINT analysis, in which it “conducted a review of 14 different systems that might work.” The review considered services from Lockheed Martin and Detica (a subsidiary of BAE Systems) but decided on the up-and-comer from Palo Alto.
Palantir is described as having been funded not only by In-Q-Tel, the CIA’s venture capital branch, but furthermore created “through [an] iterative collaboration between Palantir computer scientists and analysts from various intelligence agencies over the course of nearly three years.” While it’s long been known that Palantir got on its feet with the intelligence community’s money, it has not been previously reported that the intelligence community actually helped build the software.
This collaborative environment also produced a piece of software called “XKEYSCORE Helper,” a tool programmed with Palantir (and thoroughly stamped with its logo) that allowed analysts to essentially import data from the NSA’s pipeline, investigate and visualize it through Palantir, and then presumably pass it to fellow analysts or Five Eyes intelligence partners. One of XKEYSCORE’s only apparent failings is that it’s so incredibly powerful, so effective at vacuuming personal metadata from the entire internet, that the volume of information it extracts can be overwhelming. Imagine trying to search your Gmail account, only the results are pulled from every Gmail inbox in the world.
An analyst using XKEYSCORE could pull every IP address in Moscow and Tehran that visited a given website or made a Skype call at 14:15 Eastern Time, for example, and then import the resulting data set into Palantir in order to identify additional connections between the addresses or plot their positions using Google Earth.
Palantir was also used as part of a GCHQ project code-named LOVELY HORSE, which sought to improve the agency’s ability to collect so-called open source intelligence — data available on the public internet, like tweets, blog posts, and news articles. Given the “unstructured” nature of this kind of data, Palantir was cited as “an enrichment to existing [LOVELY HORSE] investigations … the content should then be viewable in a human readable format within Palantir.”
What would a privacy-minded conversation about privacy-breaching software look like? How had a privacy and civil liberties council navigated the fact that Palantir’s clientele had directly engaged in one of the greatest privacy and civil liberties breaches of all time? It’s hard to find an answer.