December 8, 2016

Wikileaks publishes classified documents from inside German NSA inquiry commission

On December 1, Wikileaks published 90 gigabytes of classified documents from the German parliamentary commission that investigates NSA spying and the cooperation between NSA and the German foreign intelligence service BND. The documents include 125 files from BND, 33 from the security service BfV and 72 from the information security agency BSI.

It should be noted though that all documents are from the lowest classification level and lots of them are just formal letters, copies of press reports and duplications within e-mail threads. Nonetheless, the files also provide interesting new details, for example about the German classification system, BND's internal structure, the way they handled the Snowden-revelations and the use of XKEYSCORE.

The German parliamentary investigation commission just before a hearing
(photo: DPA)


Some background information was provided in an article from the newspaper Die Zeit, which says that only documents with the lowest classification level (VS NfD or RESTRICTED) are scanned and made available to the investigation commission on a government server. They are also available at the federal Chancellery.

Documents with a higher classification level are not digitalized and have to be read in a secure room (German: Geheimschutzstelle) in the parliament building. Most of the documents classified Top Secret can only be viewed at the Chancellery or the new Berlin headquarters of BND.

Classified documents provided to the investigation commission
(still from the ARD documentary Schattenwelt BND)

Regarding the source of this leak, IT experts of the German parliament said that they found no indications of a hack. Der Spiegel suggests that the source might be a member of the parliamentary commission for foreign affairs or for the affairs of the European Union, because one document published by Wikileaks (meanwhile removed) was only available to members of those two commissions.

Wikileaks hasn't redacted anything. Almost everything that is redacted is in blue, which is apparently the way BND is redacting its documents. Therefore, the files still contain all the internal organizational designators as well as the e-mail aliasses or addresses of many German government units and employees.

Internal BND e-mail from the EAD branch for the relationships with western countries &
cooperation partners, and the EADD unit for relationships with North America & Oceania
(click to enlarge)

BND classifications

Documents from BND are classified according to the official German classification system, which has four levels, corresponding to those used in many other countries:

color code: blue or black; equivalent: RESTRICTED

color code: blue or black; equivalent: CONFIDENTIAL

- GEHEIM (Geh. / Stufe I)
color code: red; equivalent: SECRET

- STRENG GEHEIM (Str. Geh. / Stufe II)
color code: red; equivalent: TOP SECRET

Besides these common classification levels, it was suspected that there would be at least one higher or more restrictive category to protect highly sensitive information. This has now been confirmed by various letters from the Wikileaks trove, which mention the following two classification markings:


color code: ?; equivalent: TOP SECRET/SCI

The use of these markings is apparently a secret itself, because also members of the parliamentary commission puzzled about their exact meaning and usage. It seems though that these categories are rather similar to the US Classification System, which was explained here earlier.

The German marking ANRECHT apparently means that certain information is classified Secret or Top Secret, but that within that particular level, it's only meant for those people who have a need-to-know (German: Anrecht), apparently especially when it comes to signals intelligence. In the United States this is realized through a range of different dissemination markings.

The marking SCHUTZWORT is also meant to restrict access, but in this case, the originator of a particular document determines a codeword (German: Schutzwort) which he provides only to those people who are allowed access to that document. This is similar to the system of Sensitive Compartmented Information (SCI) used in the US, where meanwhile several formerly secret codewords have been declassified.

A security manual from the German armed forces from 1988 also mentions special classification categories, like for example SCHUTZWORT and KRYPTO, the latter apparently for classified cryptographic information.

Letter from the Chancellery which was classified STRENG GEHEIM-ANRECHT,
which was marked as cancelled (UNGÜLTIG) after the attached
documents at that classification level were removed
(click to enlarge)

BND organization

The files published by Wikileaks also contain a set of charts showing the organizational structure of BND between the year 2000 and 2014. There are some changes in the agency's divisions, with a reorganization in 2009, as can be seen in the following charts:

BND organization chart, situation until 2009
(click to enlarge)

BND organization chart, situation since 2009
(click to enlarge)

A more detailed BND organization chart was among the Snowden documents and was published earlier by Der Spiegel.

Internal designators

The BND's divisions, branches and units are designated by codes that consist of letters, written in capitals. In the current situation the main divisions have a two-letter designator which is more or less an abbreviation of their full name. The SIGINT division is for example TA, which stands for Technische Aufklärung.

From the e-mails published by Wikileaks we learn that lower units are designated by adding additional letters or words to the division designator. It seems that these addtional letters can be the first letter of a full name, a more or less random letter, or A for the first unit, B for the second unit, etc.

For example, "PLSA-HH-Recht-SI" is the first branch (A) of PLS, which is the BND president's staff. The term "Recht" indicates that this is apparently a unit for legal issues. A simpler designator is "GLAAY", which is a unit of the division GL (Gesamtlage)

By combining several documents related to XKEYSCORE, the following list of designators for BND's field stations could be reconstructed:
- 3D10: Schöningen or Rheinhausen (satellite interception)
- 3D20: Schöningen or Rheinhausen (satellite interception)
- 3D30: Bad Aibling (satellite interception)
- 3D40: Gablingen (HF radio interception)*

Some divisions

The organization charts for BND's structure since 2009 shows that there are four divisions for analysis and production, which is where analysts prepare intelligence reports:
- Two divisions are for topical missions: TE for international terrorism and organized crime, and TW for proliferation of weapon systems and ABC weapons.
- The other two divisions, LA and LB, are responsible for a geographical area. From their logos in the signature block in internal e-mails we learn that LB is responsible for Africa, the Middle East and Afghanistan, while LA has the rest of the world:


According to Wikileaks, one of the more interesting documents from their release is one that allegedly proofs that "a BND employee will be tasked to use and write software for XKeyscore." However, the German tech website Golem says that this seems to be based on a text section that only refers to BND employee A.S. who helped install XKEYSCORE at the Berlin headquarters of the domestic security service BfV, which uses this system only for analysing terrorism-related data sets.

More interesting are several other documents about XKEYSCORE. For example In a list of answers prepared for the meeting of the parliamentary oversight commission on November 6, 2013 it is said that XKEYSCORE is used since 2007 in Bad Aibling and that this system is being tested since February 2013 at the satellite intercept stations Schöningen and Rheinhausen. It was planned to use XKEYSCORE on a regular basis at the latter two locations too.

An internal BND e-mail from November 5, 2013, explains that at Schöningen and Rheinhausen, XKEYSCORE is used for intercepting foreign satellite communications. The specific purpose for the system is determining which satellite links are most useful and subsequently checking whether the traffic contains the communications of people the BND is looking for (so-called survey):

Internal BND e-mail about the use of XKEYSCORE at BND's satellite stations
(source: Wikileaks, pdf-page 248 - click to enlarge)

This is a rather unexpected use of XKEYSCORE, because for NSA and GCHQ the strength of the system lies in its capability to reassemble internet packets, filter them and allow analysts to search buffered content. It is still not fully clear whether BND uses XKEYSCORE also in this way.

In November 2014, W.K. from BND's SIGINT division testified that XKEYSCORE was used for decoding and demodulating IP traffic. Decoding for making things readable happens both online and on stored data, while (demodulating for) selecting the proper satellite links only happens on online data streams.

At Schöningen and Rheinhausen XKEYSCORE was only used for the latter purposes, in the pre-analysis stage. This also came forward from some testimonies before the investigation commission. For example E.B., head of the Schöningen station, said that XKEYSCORE was only used for looking at a few days of satellite traffic to determine which communication links where in it.

An earlier presentation about satellite interception at Menwith Hill Station in the UK shows that NSA and GCHQ have other systems, like DARKQUEST, for surveying satellite links, after which XKEYSCORE is used for processing and analysing the data.

IBM servers

The Wikileaks files also contain an internal BND order form from February 25, 2014, used for ordering six servers for field station 3D20: two IBM X3650 M4 and four IBM X3550 M4 servers, with a total cost of 58.000,- euros. A separate text explains that these servers were needed for both PDBD and XKEYSCORE:

- PDBD was the new centralized BND tasking database, which would replace the proprietary tasking databases used at the various field stations.

- XKEYSCORE is described as a system that decodes packet-switched telecommunicatiosn traffic like e-mail, messenger, chat, geolocation information, etc. and is used for analysing telecommuncations traffic. At BND the system was needed because it became increasingly difficult to extract relevant information from the ever growing amount of data. The servers were needed to move XKEYSCORE from test to operational status.

Internal BND order form for several IBM servers to be used for XKEYSCORE and PBDB
(source: Wikileaks, pdf-page 72 - click to enlarge)


A large file from the commission documents is about the reaction on the revelation of PRISM. In August 2013, members of the Bundestag asked so many questions about this NSA program, that one BND employee complained that it was unreasonable to expect that his agency could provide all the answers.

At that time, many details about PRISM weren't clear yet and statements from the US government and from internet companies seemed to contradict eachother. Among the documents that BND forwarded to the parliamentary commission was also one report from July 2013, which summarizes what was known about PRISM at that time.

This report was made by people from unit ÖS I 3 of the Public Safety division of the German Interior Ministry (BMI). After summarizing what was known from the press reports, the report also describes a second tool that is named PRISM - based upon an earlier article on this weblog:

Summary of a second PRISM program as described on this weblog
(source: Wikileaks, pdf-page 104 - click to enlarge)

Shortly after the existance of PRISM was revealed early June 2013, much was unclear, so I did some open source research and found that the US military uses a program named PRISM, which in this case is an acronym for "Planning tool for Resource Integration, Synchronization and Management".

Shortly afterwards, in July 2013, German press published an NSA letter saying that there are actually three different programs with the name PRISM: one that collects data from the big internet companies, one that is used as a military tasking and planning tool, and finally one that is used for internal data sharing in NSA's Information Assurance Directorate (IAD).


On July 29, 2013, the German magazine Der Spiegel published a chart from the NSA tool BOUNDLESSINFORMANT. The chart was related to Germany and it was thought that it showed that NSA had intercepted over 550 million pieces of communications traffic.

But within just a few days, BND contacted Der Spiegel, saying that they collected those data, and shared them with NSA. The SIGADs US-987LA and US-987LB designated collection at the BND satellite station in Bad Aibling and (wireless) interception of phone calls in Afghanistan, respectively. This was confirmed by NSA and published by Der Spiegel on August 5, 2013.

BOUNDLESSINFORMANT screenshot showing metadata related to Germany
as being published by Der Spiegel on July 29, 2013
(click to enlarge)

An e-mail published by Wikileaks shows that meanwhile, M.J. from unit 3D3D of the Bad Aibling station was comparing the numbers from the BOUNDLESSINFORMANT chart with those from his logfiles and Nagios Checks. In the e-mail, from August 12, 2013 to his boss R.U., he concluded that at the beginning of the month there was a relatively clear similarity with the chart from Der Spiegel:

The chart that seems to be prepared by BND employee M.J. to compare
with the one from BOUNDLESSINFORMANT (note the different scale)
(click to enlarge)

It should be noted that BND didn't count the numbers of metadata they provided to NSA, they did so only for content, so the numbers from M.J.'s chart may not be fully accurate. Even more puzzling is a table that was also with the e-mail from M.J. and contains the daily numbers for the metadata during this period:

The chart that seems to be prepared by BND employee M.J. to compare
with the one from BOUNDLESSINFORMANT (note the different scale)
(click to enlarge)

The strange thing here is that on the right side, the table has daily numbers broken down for several processing systems - strange because the chart from Der Spiegel only provided aggregated numbers, and because three codenames weren't seen in the published BOUNDLESSINFORMANT charts: POPTOP, CRON and SNOWHAZE. Did NSA provide these more detailed numbers so BND could compare them?


Finally, a list of some of the most interesting files found so far (would have been useful when Wikileaks provided this kind of index though):

- MAT_A_BND-8a (contacts with GCHQ, cooperation between BND and NSA, reports about the refugee interview unit, internal G10 manual)

- MAT_A_BND-1-3a_2 (employees of US military and intelligence contractors in Germany)

- MAT_A_BND-3a (very extensive index of topics used by BND)

- MAT_A_BND-1-5 (NSA's bulk metadata collection, PRISM and XKEYSCORE)

- MAT_A_BND-3-1a (BND organization charts from 2000-2014)


- MAT_A_BND-1-11c (page 315: options how NSA could have intercepted Merkel's cell phone)

More to follow...

November 21, 2016

Data sharing systems used within the Five Eyes partnership

(Updated: November 28, 2016)

From the Snowden revelations, the general public learned about the Five Eyes partnership between the signals intelligence agencies of the United States, the United Kingdom, Canada, Australia and New Zealand, but details about this cooperation remained shrouded in secrecy.

Now, a batch of internal newsletters of the NSA's Signals Intelligence Directorate (SID), published last August by the website The Intercept, provides new information about various systems for sharing information, metadata, content and reports among the Five Eyes partners.

- From BRUSA to Five Eyes
- Joint Executive for SIGINT Interoperability (JESI)
- Information sharing: IWS
- Interoperable access control: PKI
- Sharing metadata: MAINWAY
- Federated metadata queries: GLOBALREACH
- Sharing content: TICKETWINDOW
- Sharing end reports: CATAPULT
- SIDtoday newsletters

From BRUSA to Five Eyes

The Five Eyes community grew out of the cooperation between Britain and the United States during World War II. On March 5, 1946 both countries signed the BRUSA (now known as UKUSA) Agreement on communications intelligence cooperation. This is not only about collecting signals intelligence, but also about security measures, like the use of codewords to restrict access to highly sensitive sources and reports.*

In June 1948 the UKUSA Agreement was established, which Canada, Australia and New Zealand signed on along with the UK as "Second Parties". A separate agreement between Canada and the USA (CANUSA) was signed in November 1949, followed by one with Australia in September 1953.*

Finally, in May 1954, the BRUSA Agreement was renamed UKUSA, which became also the name for the complex network created by these often overlapping agreements, appendices and memoranda of understanding.* Australia acted on behalf of New Zealand until the latter became a full member in 1955 or 1977.

The (signals) intelligence agencies that have less close bilateral relationships with NSA are called Third Party partners. Currently, there are over 30 Third Party partners, see: NSA's Foreign Partnerships

When the term Five Eyes (for classification purposes abbreviated as FVEY) came in use is not clear, but the SIDtoday newsletter from August 5, 2003 confirms that "Five Eyes" is derived "from the "US/UK/CAN/AUS/NZ EYES ONLY" caveat that limits the distribution of SIGINT reports to the listed Second Party countries."

The initial network of bilateral relationships between the five partner countries was eventually transformed into a "group partnership" in 1993 - as was revealed in a newsletter from August 25, 2003. It's not explained what this means, but it's sounds like a shift to a more multilateral framework for cooperation among eachother.

The British-U.S. Communication Intelligence Agreement from 1946
(the full text as pdf - click to enlarge)

Joint Executive for SIGINT Interoperability (JESI)

In 1998, the agencies of the Five Eyes group established the Joint Executive for SIGINT Interoperability (JESI, pronouncesd as "jessy"). In the newsletter from August 25, 2003, JESI is described as a "multi-national executive body responsible for ensuring continued interaction and interoperability among the five SIGINT partners". JESI doesn't have its own staff, it's just a collaboration platform.

Officials from the Five Eyes agencies also meet at an annual JESI conference. In July 2003 this meeting was held in the Australian capital Canberra and was focused on the mission objectives of the partner agencies and how they relate to the 5-EYES SIGINT Partnership Business Vision, which was published earlier that year. They addressed the following topics:
- Mission collaboration and knowledge sharing
- Enabling SIGINT operations through information assurance
- Exchange of finished intelligence
- Maintaining business continuity

For a more efficient cooperation among the Five Eyes partners, the following systems were created, most of them initiated by JESI in 2002-2003, as described in the SIDtoday newsletter from August 25, 2003:

Information sharing: IWS

A collaboration tool called InfoWorkSpace (IWS) was created to exchange information between NSA, the US military and partner countries during Operation Enduring Freedom in Afghanistan.

IWS is a software tool that provides chat communications as well as audio and video conferencing, file sharing, virtual whiteboards, and shared desktop views through desktop computers connected to a secure network.* As within the Five Eyes it's about signals intelligence, IWS most likely ran, and maybe still runs on NSANet.

According to a SIDtoday newsletter from September 10, 2003 IWS was already used by over 4000 NSA and their Second Party counterparts at the working levels. They collaborated on topics like Operation Enduring Freedom, international terrorism, real-time collection coordination, SIGINT development and multi-intelligence tasking.

This succesful use of IWS led JESI decide that the system should also be used at leadership-level. As of 2003, the SIGINT directors of the Five Eyes partners would use IWS to enhance their collaboration on subjects ranging from current intelligence objectives to future collection planning. They would get access to one of the IWS servers managed by NSA, codenamed VOTEDOOR.

InfoWorkSpace, here being used during the Joint
Expeditionary Force Experiment (JEFX) 2006
(photo: CHIPS Magazine)

In another newsletter from December 19, 2003, it is said that not long before, the SIGINT directors of NSA, the Canadian CSE, the Australian DSD and New Zealand's GCSB held their first virtual meeting using the InfoWorkSpace tool. However, their counterpart at "GCHQ was unable to attend due to a computer failure."

According to the newsletter, this first meeting lasted over an hour and was mainly about "efforts against terrorism, especially ways to extend cooperation across the SIGINT community, and to include the HUMINT [Human Intelligence] community". A next virtual meeting using IWS was scheduled for the middle of January 2004.

The tech website Motherboard found the following video presentation of the InfoWorkSpace (IWS) tool, which was developed by ezenia!, a small company from Salem, New Hampshire:

Interoperable access control: PKI

In order to give Second Party employees access to joint collaboration systems, JESI pushed the partner agencies to deploy interoparable Public Key Infrastructure (PKI). The NSA's PKI is a comprehensive encryption system to protect classified information against:

- Unauthorized disclosure and modification through digital signing
- Unauthorized access through access controls and authorization services
- False user idenfications

An SIDtoday newsletter from July 8, 2003 explains that the new PKI system would replace the ICARUS e-mail encryption system by October 2003. A valid PKI certificate was also needed to use applications like Peoplesoft and CONCERTO. The latter is NSA's internal personnel system, which has separate parts for human resource and security clearance information.

The new PKI certificates were first issued to NSA employees who were US citizens and held a blue, green, or gold badge. Later, PKI certificates would also be issued to employees of Second Party agencies and to non-US citizens. This PKI system seems to be a software solution without two-factor authentication with a token like the CAC-smartcard of the US military.

Sharing metadata: MAINWAY

Since 2006 it was thought that MAINWAY was a repository just for telephone metadata, but based upon recently leaked and declassified documents, it was explained on this weblog that MAINWAY also contains internet metadata as well as the domestic phone records NSA previously collected under the authority of Section 215 of the USA PATRIOT Act.

Rather unexpected, the SIDtoday newsletter from August 25, 2003 now also reveals that "MAINWAY, a system that uses phone call contact chaining to identify targets of interest, was provided to each of our partners. The partners now supply additional contact information to the database to enhance the joint ability to identify targets".

So MAINWAY is not only fed with the domestic US telephone records and the foreign telephone and internet metadata collected by NSA, but also with foreign metadata provided by GCHQ, CSE, DSD and GCSB. According to the quid pro quo rule for intelligence cooperation, all Five Eyes partners can apparently also query the MAINWAY database for their national security interests.

However, Second Party analysts have no access to the domestic US phone records, but so far there are no documents that mention this explicitly (recently published dataflow diagrams show that MAINWAY has separate BRF [BR FISA or Section 215 records] partitions though).

A GCHQ presentation from 2010, which was published earlier, shows the user interface of the IMMINGLE tool with check boxes for direct access to various metadata repositories, including MAINWAY II:

User interface of GCHQ's IMMINGLE tool for access to various metadata repositories
(the full presentation as pdf - click to enlarge)

Federated metadata queries: GLOBALREACH

Besides direct access to the metadata contained in MAINWAY, analysts from the Five Eyes partners can also use the GLOBALREACH system. In documents that were published earlier, this system is described as a "federated query service via accounts and access verified by PKI certificates" which probably runs on NSANet.

As a federated service, GLOBALREACH can be used to query multiple metadata databases with one single login. A 2005 document says that for example CIA would provide metadata "from non-SIGINT sources for inclusion in the dataset searched by GLOBALREACH" and it's likely that it can also search the foreign metadata from MAINWAY.

A pilot for a similar federated query tool codenamed ICREACH for the US Intelligence Community (IC) was started in 2007. After NSA "persuaded other US IC agencies to make almost 100 bn previously NOFORN records shareable with the 5-eyes via GLOBAL REACH", agreements were reached with the Second Party agencies, whereafter they started to provide ICREACH with telephony metadata, making them accessible to over 1000 analysts across 23 US intelligence agencies.

After establishing ICREACH, these analysts got access to more communication modes (including landline, mobile, satellite and VoiP call records), the types of metadata increased from 5 fields to 33 fields and the total volume rose from 50 billion to over 850 billion records - ca. 126 billion of which from Second Party partners. 1-2 billion records were said to be added daily, so by now, ICREACH may provide access to over 5 trillion metadata records.

Architecture of the ICREACH federated query system
(the full presentation as pdf - click to enlarge)

Sharing content: TICKETWINDOW

An older collaboration system for the Five Eyes partners is described in a SIDtoday newsletter from November 7, 2003: TICKETWINDOW. This system was established in 1999 by the NSA's Data Acquisition division to enable reciprocal data sharing with Second Party parters - without revealing sensitive sources and collection methods, which often restricted data sharing. Within TICKETWINDOW, NSA shares most data, but the other partners also contribute from their own collection.

In 2003, TICKETWINDOW was regarded a success story: new sources from the partner countries helped NSA to be more productive, while for the Australian DSD, more than 40% of their product reporting was from TICKETWINDOW collection, particularly from NSA collection. Both the British GCHQ and the Canadian CSE had doubled their output of TICKETWINDOW reports in 2002. Maybe this system is somehow related to the mysterious SIGADs starting with DS, which seem to denote collection by Second Party countries.

A similar data sharing system for the SIGINT Seniors Europe (SSEUR) group of Third Party partners is the Signals Intelligence Data System (SIGDASYS).

Sharing end reports: CATAPULT

Finally, there's also a system for sharing intelligence reports among the Five Eyes partners. According to a newsletter from May 8, 2003, NSA and the Canadian CSE set up a prototype portal to exchange SIGINT products between NSA and its Second Party partners under the codename CATAPULT.

The CATAPULT portal "contains all 2nd party viewable product shared with CSE to include multimedia reporting, CRITICOMM released product, and SIGINT on Demand (SOD) items", all of which is accessible from NSANet through a browser interface. CATAPULT is based on CSE's SLINGSHOT system, which delivers SIGINT reports to Canadian "customers" like policy and decision makers.

CATAPULT was brought under the JOURNEYMAN umbrella program for modernizing the way SIGINT analysts can write and disseminate their reports. As CATAPULT started as a prototype, it may have been replaced by a system that includes all Five Eyes partners.

Besides the systems described above, JESI also initiated the creation of several protected websites to allow employees of the Second Party agencies to securely share data within specific communities of interest.

As close as the cooperation between these agencies may have become, the sharing mechanisms are still meant to support each member's foreign intelligence tasks. The Five Eyes are not a body of its own with its own goals or targets, like for example a rather ridiculous target list on Wikipedia suggests.

Also, the data sharing system TICKETWINDOW isn't the successor of ECHELON, as Wayne Madsen wrote on the website Intrepid Report. ECHELON was (and under the name FORNSAT still is) a worldwide network of satellite intercept stations to provide in the information needs of each of the Second Party countries.

SIDtoday newsletters

In May 2016, The Intercept started publishing large batches of documents from the Snowden archive, to begin with the SIDtoday newsletters from 2003, all the way to the most recent available ones from 2012. A second batch came in August 2016 and so far, a total number of 429 SIDtoday newsletters have been published, from March 2003 to July 2005.

These newsletters are an interesting source for historical research as they add or confirm many details about NSA. Although some of them are about operations that could be controversial, taking away full nine years of SIDtoday newsletters isn't proportionate and forms an example of where Snowden wasn't very selective.

Links and sources
- The Intercept: All published editions of SIDtoday
- About Canada and the Five Eyes Intelligence Community (pdf)
- Martin Ruder: Hunters and Gatherers: The Intelligence Coalition Against Islamic Terrorism
- NSA: UKUSA Agreement Release 1940-1956

October 19, 2016

With NSA contractor Martin arrested, other leakers may still be at large

(Latest UPDATE: October 31, 2016)

Earlier this month we learned the name of a second person who stole top secret documents from the US National Security Agency (NSA). After Edward Snowden admitted doing so publicly in June 2013, the FBI has now arrested the 51-year old Harold T. Martin III at his home in Maryland.

Martin hoarded lots of classified documents, not only from NSA but also from a number of other military and intelligence agencies. The FBI is still comparing them with those from the recent Shadow Brokers leak and a range of other NSA leaks from the past few years, but given what's known now, it seems likely that at least one other leaker is still at large.

The house of Harold T. Martin III in Glen Burnie, Maryland
(photo: Jose Luis Magana/The Associated Press)

The New York Times reported that when the FBI raided Martin's house on August 27, they found paper documents and many terabytes of highly classified information, even going back the 1990s. At least six documents were from 2014. It was reported that Martin first took the classified documents on paper, later on CDs and more recently on thumb drives.

The reason why Harold Martin brought home and stored such large numbers of top secret documents isn't yet clarified. One suggestion is that he may have used them for research for his dissertation about "new methods for remote analysis of heterogeneous & cloud computing architectures", which he was working on at the University of Maryland.

Documents from multiple agencies

It should be noted that not everything Martin stole comes from NSA. In the official charges there are no names of the agencies where the documents come from, they are only described as highly classified, including ones that are marked as Top Secret and Sensitive Compartmented Information (SCI).

With the documents going back to the 1990s, he may well have started hoarding them from the places where he worked in those days. From 1987 to 2000, Martin served at the US Navy, achieving the rank of lieutenant, but he left active duty in 1992.

As the Washington Post found out, he then took a variety of tech jobs with government contractors, like at Computer Sciences Corp. (CSC) somewhere in the 1990s and later, until 2009, at Tenacity Solutions, for which he worked at the Office of the Director of National Intelligence (ODNI). Over the course of 18 years, Martin worked for a total of 8 different defense contractors.

In 2009, Harold Martin started to work for Booz Allen Hamilton, for which he was a contractor at NSA from 2012 to 2015, when Booz transferred him to the Pentagon’s Office of Acquisition, Technology and Logistics (AT&L), which is responsible for often highly sensitive and classified procurement programs. There he stayed until the moment of his arrest last August, after which he was also fired by Booz.

Officials have meanwhile said that Martin took classified documents not only from NSA, but also from his other workplaces, including ODNI and AT&L.

It's interesting as well that in the charges against Martin, a whole paragraph is dedicated to the at least six documents from 2014, which are described as being produced "through sensitive government sources, methods, and capabilities". As signals intelligence is traditionally seen as the most sensitive capability, maybe just these six documents are from NSA.

The building of the Office of the Director of National Intelligence (ODNI)
where Harold Martin worked as a contractor before 2009
(photo: Microsoft, via - click to enlarge)

Shadow Brokers investigation

After the "Shadow Brokers" disclosed a large set of secret NSA hacking tools last August, the FBI began investigating this leak. At the same time there was a lot of speculation: was NSA hacked from the outside? Had an NSA hacker been sloppy? Were the tools leaked by an insider? Maybe the same insider responsible for earlier leaks that hadn't been attributed to Snowden?

On September 22, it was reported that during the FBI investigation, NSA officials had said that a former agency operative had carelessly left the hacking tool files available on a remote computer, where Russian hackers found them. If that's correct, then it seems likely that the FBI traced Harold Martin when they were looking for that careless NSA hacker. It has not yet been confirmed that Martin was that person though.

Harold Martin was working at NSA's hacking division TAO around the time when the tools were considered to be left exposed, somewhere after October 18, 2013, but a former TAO hacker told the Washington Post that Martin "worked in the unit’s front office carrying out support roles such as setting up accounts, not conducting actual operations."

Even if Martin was the man who left the hacking tools exposed, then we still don't know who found them and published them under the name Shadow Brokers. It's not very likely that this was done by Martin himself, as Shadow Brokers published additional messages on August 28, October 1, October 15, and October 31, when he was already in custody. The actual publication can therefore be the work of for example Russian, Iranian or North Korean hackers or even independent hacktivists.

Other sources?

Could Harold Martin also be the source of earlier leaks, that were not attributed to Edward Snowden? In theory he could have been that "second source" next to Snowden: none of these other leaked documents (like the TAO catalog, XKEYSCORE code, tasking lists and end reports) are newer than 2015, when Martin left NSA. Contrary to this Martin is described as very patriotic, which doesn't fit the fact that these particular leaks were clearly meant to harm and embarrass the US and NSA.

Also, Martin hasn't (yet) been charged with espionage or the attempt to provide classified information to a third party or a foreign government - which doesn't seem something the US government would leave out or keep secret after the recent and unprecedented statement in which the Office of the Director of National Intelligence accused Russia of hacking the Democratic National Committee (DNC) and other political organizations.

Should the FBI investigation confirm that Harold Martin was only responsible for leaking the NSA hacking tools (after which unknown others published them) and that none of his documents were provided to foreign intelligence agencies or showed up in the earlier revelations, then there's most likely yet another leaker from inside NSA.

The Shadow Brokers leak standing alone and not related to the earlier non-Snowden leaks is of some importance, because only among the stuff published by the Shadow Brokers there are files with a date (October 18, 2013) after the day that Snowden left NSA (May 20, 2013).

This means that when Harold Martin is the initial source of the Shadow Brokers files, we can no longer exclude the possibility that the earlier leaks do come from the Snowden trove. If that would be the case, then someone with access to them went rogue and had them published on his own account. But it should also be noted that both Glenn Greenwald and Bruce Schneier explicitly said that some of these leaked documents did not come from Snowden.

The more likely option is therefore that there's still another leaker at large, someone with a more evil intent than Harold Martin and Edward Snowden - a conclusion which is not very comforting and which also raises questions about NSA's internal security...

Some NSA buildings at the Friendship Annex (FANX) complex near Baltimore
(photo:, via - click to enlarge)

NSA's internal security measures

The NSA's hacking division TAO, where Harold Martin worked for some time, is apparently not located in the well-known NSA headquarters building at Fort Meade, but in one or more leased office buildings outside, one of them at an office complex called Friendship Annex (FANX) near Baltimore. TAO also has units at NSA's four Cryptologic Centers across the US.

Entrance to the highly secured TAO headquarters building is strictly controlled: one has to go through an imposing steel door, protected by armed guards, and entrance is only possible after entering a six-digit code and passing a retinal scanner to ensure that only specially cleared individuals are allowed in.

Such security measures are more aimed at keeping outsiders out, than at insiders in. And when it comes to finding inside moles of hostile foreign intelligence agencies, the NSA is also said to have a rather bad track record. The Manning and Snowden leaks made NSA painfully aware of this and so preventive insider-threat detection programs were put in place.

It's not clear whether these new systems failed in the case of Harold Martin, or that they simply weren't yet implemented at the TAO location where he worked - anti-leak software that was designed by Raytheon to "spot attempts by unauthorized people to access or download data" was also not yet installed at the NSA facility in Hawaii when Snowden was working there.

Tracking what employees are doing inside is one thing, checking what they take out is another. But according to The Washington Post, the NSA (like other agencies) does not impose universal checks of personnel and their belongings as they enter and leave agency buildings. Security guards only conduct random checks and use their discretion in order to keep en build the trust of the employees.

"If you have a bag full of stuff, you’re probably going to get stopped" said a former TAO operator to the Post, but, in general, "Disneyland has more physical security checks than we had". This was confirmed by two other former NSA employees, saying that "nobody does pocket checks" and that "Anything that could fit in a pocket could go out undetected".

It would also take hours to screen every person leaving NSA buildings, and because the vast majority of employees go through extensive vetting, so there's an inherent amount of faith in staff at the agency. Besides checks, NSA facilities will also have detection gates, but it seems that it was easier for Snowden to walk out with his thousands of documents than many would have thought.

As former NSA general counsel Rajesh De explained, it is unlikely "you’re going to be able to stop every incident of somebody taking documents if they’re determined to do so. But the real question is how quickly can you detect it, how quickly can you mitigate the harm of any such incident."

An old sign inside the NSA headquarters building
showing what kind of items are not allowed in.
(screenshot from a documentary about NSA)


Harold Martin stole a lot of classified documents from multiple military and intelligence agencies where he worked over the past 20 years, with maybe just a small number from NSA. The still ongoing FBI investigation has to make clear whether Martin was responsible for exposing the TAO hacking tools.

If not, then there has to be yet another careless NSA employee, but then it's also still possible that the hacking tools came from a source responsible for a range of earlier leaks. So far it seems that Martin isn't the source of those earlier leaks, which means that the so-called "second source" is still at large.

The case of Harold Martin also made clear that security measures at NSA, and other US agencies, were not as strict and tight as outsiders would have expected: even for someone without a strong ideological or financial drive like Martin it was apparently not that difficult to regularly walk out with top secret documents.

Many things have not yet been confirmed or clarified, but at least the Shadow Brokers leak and the subsquent arrest of Harold Martin created more awareness among the American public of the fact that there have been more leaks than just those from Snowden.

In August 2014, Bruce Schneier was probably one of the first who identified a second and a third leaker besides Snowden. Many more similar leaks followed and a full list of them was compiled on this weblog in December 2015 (still being updated). As an excerpt of this listing, a short overview of the most important non-Snowden leaks was published in The New York Times last week.

Update #1:

Shortly after this blog posting was published, The New York Times came with a new report saying that the volume of classified documents Harold Martin had in his possesion seems larger than those stolen by Edward Snowden and even than those of the Panama Papers from 2015.
FBI investigators apparently also found that the TAO hacking tools were among Martin's documents, but because he is not very cooperative, it is still not clear how they came in the hands of the mysterious Shadow Brokers, who subsequently published them. So far there's no evidence that Martin was hacked or that he sold information.
He seems to have hoarded all these documents in order to get better at his job, as he is described as someone who imagined himself a top spy and an important player in the world of digital espionage.

Update #2:

On October 20, it was reported that the FBI had found the huge amount of 50 terabytes of data at Martin's home, but it is not yet clear how much of that is actually classified. Also found were "hard-copy documents that were seized from various locations during the search that comprise six full bankers’ boxes worth of documents" with many of the documents marked Secret and Top Secret.
One document was marked Top Secret/SCI and had this additional caveat at the top of the document: "THIS CONOP [Concept of Operation] CONTAINS INFORMATION CONCERNING EXTREMELY SENSITIVE U.S. PLANNING AND OPERATIONS THAT WILL BE DISCUSSED AND DISSEMINATED ONLY ON AN ABSOLUTE NEED TO KNOW BASIS. EXTREME OPSEC [Operational Security] PRECAUTIONS MUST BE TAKEN" - Martin had no need to know for this operation.

Update #3:

Harold Martin appeared in court for the first time on Friday, October 21. There, his lawyer said that things like an "unlocked garden shed, stuffed with more classified documents than the contractor [...] could ever read, might be a symptom of a mental disorder" - and also that keeping top secret material in plain view in his home and car was not the conduct of a spy or a political activist.
Although he was charged with the relatively minor criminal offenses of theft of government property and unauthorized retention of classified material, Martin had to stay in jail because he could be a threat to national security as investigators couldn't rule out that he might have hidden classified information in other, yet undisclosed locations.
Even after seven weeks of investigation, the FBI was still not able to show whether Martin gave any of his documents to anyone else, nor could they link him to the Shadow Brokers.

Update #4:

A legal document filed by federal prosecutors on October 27 says that the information stolen by Harold Martin included numerous names of intelligence officials working under cover outside the United States. It is not clear whether these officials were from NSA or from other US intelligence agencies where Martin had worked as a contractor.

We also don't know how many documents Harold Martin actually stole from NSA: everything that matters within that agency is classified under the SCI compartment SI (Special Intelligence), but so far, the FBI investigation only mentioned very few documents that were classified as Top Secret/SCI.

To be continued...

Links and Sources
- New York Times: N.S.A. Appears to Have Missed ‘Big Red Flags’ in Suspect’s Behavior
- John Schindler: It’s Time to Rename NSA the National INsecurity Agency
- The Washington Post: NSA contractor thought to have taken classified material the old-fashioned way
- Daily Beast: Democrats Say WikiLeaks Is a Russian Front, U.S. Intelligence Isn’t So Sure
- Defense One: Data-Theft Arrest Shows that Insider Threat Remains Despite Post-Snowden Security Improvements
- John Schindler: Has the Russian Mole inside NSA finally been arrested?
- New York Times: N.S.A. Suspect Is a Hoarder. But a Leaker? Investigators Aren’t Sure.
- The Cipher Brief: First on The Cipher Brief: Snowden's Boss Shares Lessons Learned

September 22, 2016

Secret report reveals: German BND also uses XKEYSCORE for data collection

(Updated: December 3, 2016)

Over the past few years we learned a lot about Germany's foreign intelligence service BND, although not from leaks, but from the public hearings of the parliamentary commission that investigates NSA spying operations and its cooperation with German agencies.

Recently however a secret government report was leaked to German media, which not only identifies violations of the data protection act but also reveals the codenames for several BND systems and the fact that BND uses the American XKEYSCORE system not only for analysis, but also for collection purposes.

Here, the new information from the secret report is combined with things we know from earlier sources and reportings.

- A secret report
- The SUSLAG liaison office
- Selectors provided by NSA: TND and SCRABBLE
      - BND's selector database: PBDB
- Operations SMARAGD and ZABBO
- Metadata analysis: VERAS
- Analysis and collection: XKEYSCORE
- Integrated analysis: MIRA 4
- Legal defects

The BND satellite intercept station at Bad Aibling, Germany
(Photo: AFP/Getty Images)

A secret report

The report that now has been published goes back to September 2013, when the then federal data protection commissioner Peter Schaar ordered a thorough inspection of the BND satellite intercept station in Bad Aibling, which took place on December 3 and 4 of that year.

In October 2014, Schaar's successor Andrea Voßhoff conducted a second visit to Bad Aibling, which in July 2015 resulted in an extensive and detailed report (German: Sachstandsbericht) about all the systems used at this BND station. This report was (and still is) classified as Top Secret.

Additionally, Voßhoff made a legal assessment based upon the Sachstandsbericht. This was finished in March 2016 and sent to then BND president Schindler and the federal chancellery. It was classified as Secret, but was leaked to regional broadcasters NDR and WDR and a transcription of the full document was published by the digital rights platform on September 1.

Both reports are about the cooperation between BND and NSA, which goes back to 2004, when the Americans turned their satellite intercept station Bad Aibling (codenamed GARLICK) over to German intelligence. In return, BND had to share the results from its satellite collection with the NSA, for which the latter provided selectors, like e-mail addresses, phone numbers, etc. of the targets they were interested in.

Google Maps view of the Mangfall Barracks in Bad Aibling, Germany.
The building at the very top seems to be the BND facility,
the one nearby with the white roof NSA's "Tin Can".

The SUSLAG liaison office

After taking over the Bad Aibling satellite station, BND seems to have moved the control facility to the nearby Mangfall Barracks, which were taken over from the German armed forces (Bundeswehr) in 2002. For the Special US Liaison Activity Germany (SUSLAG), which is the liaison office of NSA for Germany, a new highly secure container building was built on the Mangfall Barracks premises in 2003 (nicknamed "tin can" or Blechdose).

According to the commissioner's report, the SUSLAG building and the building with BND servers and equipment are connected through a 100 MBit/s fiber optic cable. SUSLAG also has a technical data link to the NSA's primary communications hub in Europe, the European Technical Center (ETC) in the Mainz-Kastel district of the city of Wiesbaden.

Cooperation between the US and Germany in the Joint SIGINT Activity (JSA, 2004-2012) took place inside the BND building, for which NSA personnel had access permissions. After the JSA was terminated, SUSLAG personnel kept their entrance rights for the BND building, but it has separate rooms for highly sensitive information to which none of the Americans have access.

A letter from BND from October 15, 2015 says that at that moment, 10 people from NSA worked at SUSLAG, with following access rights:
- 2 have access to building 7 (SUSLAG) only
- 4 have access to building 7 and building 4 (Administration)
- 4 have access to building 7 and building 8 (BND)

The SUSLAG building is only used by NSA personnel and BND claims that the data protection commissioner has no jurisdiction over the SUSLAG, but she disputes that and says the SUSLAG building is simply part of the BND complex. She also regrets that SUSLAG doesn't recognize her oversight authority.

Selectors provided by NSA: TND and SCRABBLE

For the satellite interception in Bad Aibling, some 4 out of 5 selectors come from NSA, the rest from BND. According to Süddeutsche Zeitung, NSA provided BND with roughly 690.000 phone numbers and 7,8 million internet identifiers between 2002 and 2013. That is an average of something like 60.000 phone numbers and 700.000 internet identifiers a year, or 164 phone numbers and over 1900 internet identifiers each day.

From the parliamentary hearings we already knew that BND personnel pulls the American selectors from an NSA server, and the commissioner's report now reveals that this server is in NSA's ETC in Wiesbaden. On this server BND puts back any results for these selectors. These data transfers from and to ETC go through the SUSLAG facility, but BND is able to get direct access to the NSA server in Wiesbaden through an FTP-gateway (a "BACOM system").

Selector databases

From an earlier parliamentary hearing we know that BND stores the selectors from NSA in two databases: one for IP selectors (from NSA only), and one for telephone selectors (from both NSA and BND). Each agency had access to its own IP database; the phone database was managed jointly, but BND could only approve or disapprove NSA selectors, and NSA could only do so with those from BND.

The names of these databases were not known until now, but the commissioner's report mentions them, along with some additional details:
- Target Number Database (TND), which exists since 2008 and holds the telephone selectors from both NSA and BND. The latter either come from BND's own tasking database PBDB or are provided by domestic security services.

- SCRABBLE, which only holds selectors for packet-switched (internet) communications provided by NSA, after their format has been converted. These selectors initially had no description (Deutung, like a justification for the target). Because of this, BND temporarily stopped using them as of May 2015, and for the commissioner any results from them are unlawful because BND was not able to determine whether they are necessary for its mission.

Their names indicate that these database systems were provided by NSA, and together with the fact that they also contain NSA-provided selectors, this is likely the reason why these names were never mentioned during the parliamentary hearings - unlike those of BND's own systems.

It was noticed that TND and SCRABBLE were actually mentioned once during the parliamentary hearings, when former BND president Schindler said that "the US has [its own] databases TND and SCRABBLE".

- PBDB - During a parliamentary hearing on November 9, 2016 it came out that BND's own tasking database PBDB (PersonenBezogene DatenBestände) became operational in the Summer of 2014, after a test period that started late 2012. Both in this system and in the previous system, it is/was logged when for example a selector was deactivated. An even older system had no such logging capability. Before 2014, BND field stations had their own proprietary tasking databases, at least some of them maintaining their selectors using Excell lists.
The PBDB is maintained by the T2-branch from BND headquarters. Analysts can enter any selectors (often multiple ones for a particular target) into PBDB that they assume useful for foreign intelligence purposes. Newly entered selectors are checked (through the DAFIS system) at BND headquarters to make sure they don't pull in German communications.
Results generated by approved and activated selectors are enriched with PBDB data in order to attribute them to their target. Maybe results are also stored in the PBDB database, where they can be accessed by groups of 4 to 5 analysts working on the particular topic. After it came out that BND itself also used selectors related to partner countries, those selectors were moved to a separate partition (called Gruppenliste) of the PBDB database in October 2013, so they couldn't be tasked anymore.


Before being stored in the SCRABBLE and TND databases, both the telephone and internet selectors have to pass the DAFIS filtering system, which checks whether they belong to German citizens or companies or may otherwise contradict German interests. Accordingly, the selectors are marked as "allowed" or "protected".

Those marked "allowed" are subsequently being activated ("tasked") on the actual data collection systems. The report says that for this, hard selectors like phone numbers and e-mail addresses can be freely combined with content search terms (Inhaltssuchbegriffe) like key words, which could refer to the GENESIS language used for more complex XKEYSCORE searches.

According to the report, selectors marked as "protected" are send back to NSA and are also deactivated in the TND and SCRABBLE databases - to make sure that they won't get activated when NSA provides them a second time (this confirms that there's no separate database (Ablehnungsdatei) with rejected selectors as was suggested during the earlier parliamentary commission hearings).

BND refused the data protection commissioner access to TND and SCRABBLE, so she wasn't able to check the individual selectors. She regarded that as a massive restriction of her supervision authority.

Operations SMARAGD and ZABBO

Selectors that have been approved are send to the systems that filter out communications that match those selectors. Some of these systems are in Germany, others are abroad. The report of commissioner Voßhoff for the first time discloses two specific data collection operations and their codewords:

- SMARAGD, a cable tapping operation somewhere outside Europe and in cooperation with another foreign intelligence agency.

- ZABBO, collection in Bad Aibling of satellite communications from Afghanistan.

There's no explanation for why the commissioner only mentions these two operations. The satellite antennas in Bad Aibling undoubtedly collect from many more countries, but maybe these are the only operations from which, during the investigation period, data were shared with NSA.


The way SMARAGD is described perfectly fits a certain type of operations in which a 3rd Party partner of NSA like in this case BND, cooperates with yet another country that secretly provides access to data traffic, which is then also shared with NSA. According to the book Der NSA Komplex, BND and NSA conducted about half a dozen of such operations in recent years.

In its english version of the news report about this issue, the website points to an NSA document that was published earlier by Der Spiegel. In it, we see EMERALD mentioned as an alternate codename for the NSA operation WHARPDRIVE, which is exactly such a trilateral program in which a third secret service participates.

WHARPDRIVE was still active in 2013, but in the Spring of that year, employees of the private company that operated the communication cables, accidently discovered the clandestine BND/NSA equipment, but the operation was rescued by providing a plausible cover story.*

The NSA report from April 2013 however said that "WHARPDRIVE has been identified for possible termination due to fiscal constraints", but this may have been coincided with the exposure of the program in the book Der NSA Komplex in March 2014.

It should also be noted that came up with this identification by translating the German codename SMARAGD into its English equivalent EMERALD. It is possible that the Americans also translated the German codeword SMARAGD into EMERALD, but just as likely is that it's a different program (maybe as a successor with the same set-up).

During a parliamentary hearing on November 9, 2016, member of parliament Renner said that SMARAGD is identical with EMERALD and that the operation was deactivated after Snowden, because it was mentioned in documents. BND-employee R.U. said that a cable access which terminates in Bad Aibling (likely the one from the SMARAGD operation), provided just a minimal data stream, by fault of the foreign intelligence service (probably the 3rd partner involved).

Operation Eikonal

But there's another codeword connection: from 2004 till 2008, NSA cooperated with BND in operation EIKONAL in order to get access to fiber optic cables from Deutsche Telekom in Frankfurt.

From the parliamentary hearings we know that operation EIKONAL had GRANAT as its internal BND codename. And with GRANAT being German for garnet, and SMARAGD for emerald, we see that both operations are actually named after a gemstone, which often indicates some kind of similarity.

In October 2014, the Danish paper Information reported that the WHARPDRIVE access was opened in February 2013 and had the same size as EIKANOL. This operation EIKANOL or EIKONAL was a typical example of the way NSA cooperates with 3rd Party partner agencies under its RAMPART-A program, but unlike the SMARAGD/WHARPDRIVE operations with the cable access point being inside Germany:

Left: bilateral cable access operation (RAMPART-A) - Right: trilateral cable access operation
In the cases discussed here, Germany would be "Country X"
(click to enlarge)

It is tempting to identify SMARAGD and ZABBO as the two collection programs (SIGADs US-987LA and US-987LB) from the BOUNDLESSINFORMANT chart for Germany that was published in July 2013. For both facilities together, more than 552 million metadata records were counted between December 10, 2012 and January 8, 2013.

Provided that this chart shows the only data shared by BND, it's very well possible that the satellite collection program ZABBO is one of them. For the cable access SMARAGD this is less certain and depends on when this program started and whether it is identical with WHARPDRIVE (which started in February 2013).

BOUNDLESSINFORMANT screenshot showing metadata provided by BND
(click to enlarge)

Data transfer

The report of the data protection commissioner also provides an impression of the BND networks through which collected data are brought back to headquarters.

Data collected abroad are send back to Germany over the operational network ISNoVPN (apparently something that goes "over VPN" for secure tunneling) and then arrives at a dedicated demilitarized zone (DMZ) network for data collection (Datenabholungs-DMZ).

In this DMZ network there's a virtual machine (VM) that acts as a host for data that come in from each collection facility (Erfassungsansatz). The report mentions the virtual machines "Import VM SMARAGD" and "Import VM ZABBO" for the operations SMARAGD and ZABBO respectively.

In these virtual machines, the metadata go through an Application Level Gateway (ALG), which is a security components combined with a firewall. Such an ALG is able to detect, filter and when necessary, delete data from an incoming data stream. Again, there's an ALG for each collection facility: for example SMARAGD-ALG for data from the SMARAGD collection effort.

Finally, the collected data arrive at a network called NG-Netz, which is the back-end in Bad Aibling of the transfer system that pulls in data collected at a front-end access point (Erfassungskopf) somewhere abroad.

(click to enlarge)

Metadata analysis: VERAS

The system that BND uses for analysing bulk metadata from circuit-switched communications is called VERAS, which stands for Verkehrs-Analyse-System or Traffic Analysis System. VERAS stores metadata only for up to 90 days and according to the commissioner's report they are derived from two sources:

- Metadata that come with communications collected after matching with specific selectors (the related content goes to the INBE database)

- All the metadata from selected communication links (satellite frequencies and fiber optic channels) that are regarded useful for intelligence purposes, but only after passing the DAFIS filter.

According to the manual for VERAS version 4.3.x from 2010, the system has a topology mode, in which connections can be created level after level, similar to the "hops" we know from the NSA's contact chaining method. There's no limitation to the number of levels that can be added and analysts can also focus on specific targets to create patterns-of-life (Bewegungsprofile) for them.

This kind of contact-chaining and metadata analysis inevitably involves metadata from innocent people. BND distinguished between directly and indirectly relevant. Directly relevant are metadata related to people who are already known or suspected for being relevant for intelligence purposes.

Indirectly relevant are metadata related to people who have some kind of connection to directly relevant people, or when such metadata are being stored from a "geographical point of view", which apparently refers to metadata of people being somewhere near a target without having been in direct contact.

The report says that metadata connected on such a geographical basis results in much more people being involved than when using call or connection chaining. Data related to indirectly relevant people are also used by BND, for example as new selectors.

VERAS was introduced in 2002 and recently, VERAS 4 has been replaced by VERAS version 6, which was developed by the German armed forces (Bundeswehr) as part of the VERBA (VERkehrs-Beziehungs-Analyse) project.

For VERAS 6 there's not yet a database establishing order (see below), but in February 2015 BND sent the commissioner a draft version, which she already considers illegal because BND admits that it is technically impossible to prevent that data of innocent people are being used in the VERAS system.

Analysis and collection: XKEYSCORE

Already in July 2013, Der Spiegel reported that BND president Schindler had informed the parliamentary intelligence oversight commission (PKGr) that his agency was using NSA's XKEYSCORE system since 2007, but only for analysis, not for data collection. This was confirmed by W. K., a sub-division manager in the BND's Signals Intelligence division, during a parliamentary hearing.

But now, the report of the data protection commissioner says that BND uses XKEYSCORE not just for analysis, but also for the collection of both metadata and content.

The report explains that in its data collection, or front-end function, XKEYSCORE uses selectors, single ones or freely combined ones in the form of fingerprints, to search for matches in IP traffic of both public and privat networks, and stores anything that matches these selectors.

Remarkably enough, the commissioner writes that XKEYSCORE searches all internet traffic worldwide ("weltweit den gesamten Internetverkehr"), which seems to be a copy/paste from sensationalistic press reports, as XKEYSCORE can only search data which are collected at some physical access points and not even NSA has access to all the world's communications traffic, let alone BND.

Slide from an NSA presentation about the XKEYSCORE system

Besides picking out and storing communications that match specific selectors, XKEYSCORE is also able to store a so-called "full take", a temporary rolling buffer of all data from a particular link. This in order to find files which aren't directly associated with specific selectors - which was heralded as its unique capability.

The commissioner's report only mentions this buffer function when it cites a BND response calling XKEYSCORE "a local and temporary buffering of data" which in their opinion doesn't make it a database. The commissioner disagrees and says it's a database, because even when it's just for a short time, the data are available for usage. This means a there should have been a database establishing order for XKEYSCORE (see below).

Front-end and back-end

The report doesn't explain what XKEYSCORE actually does in its function as a back-end analysis tool. But maybe instead of distinguishing between collection and analysis, we should look at the difference between the front-end and the back-end functions of the system, which is explained in a manual for its so-called Deepdive version.

This learns us that the back-end performs high-speed filtering and selection using both strong selectors (like e-mail addresses) and soft selectors (like key words), and also uses various plug-ins to extract and index the metadata, which are also used for the rolling buffer-functionality of XKEYSCORE:

Diagram showing the dataflow for the DeepDive version of XKEYSCORE

The front-end is where the intercepted data streams come in, which are first reassembled by the METTLESOME and xFip components. Then, only the most useful streams are forwarded based upon rules using country codes, keywords and such. Finally, the Defrag component conducts full sessionizing, which means that the separate IP packets that travel over the internet are reassembled into their original readable form again.

The commissioner's report says that initially the sessionizing of data from a particular communications link was conducted by another NSA system codenamed WEALTHYCLUSTER (WC, which is for lower data rates), but that this kind of processing was more and more taken over by XKEYSCORE (XKS).

So, if the distinction between collection and analysing corresponds to that between front-end and back-end, that means that the new thing we learned from the commissioner's report is that BND apparently also uses XKEYSCORE for sessionizing the data they collect, and not only for filtering and analysing them.

This sessionizing might seem rather obvious, but real-time filtering and sessionizing at data rates as high as 10 Mbit/s requires very fast, specialized and expensive equipment. Well-known manufacturers are Narus and Verint, and it seems likely that their equipment is used for XKEYSCORE too.

As XKEYSCORE is only used for internet communications, the NSA selectors are derived from the SCRABBLE database. The results of the collection are transferred to NSA, after having been filtered by DAFIS to get rid of data related to Germans.

Integrated analysis: MIRA 4

Besides all the systems mentioned before, BND also uses MIRA 4, which stands for Modulare Integrierte Ressourcen Architektur or Modular Integrated Ressource Architecture, version 4. According to a letter from BND from February 2015, this system is used to store all the content, whether from e-mail, voice, fax or teletype messages, within a certain BND station and apparently also enables software to process and select raw data in order to create intelligence reports (Meldungen).

This was however contradicted by a letter from BND from December 2015 which said that MIRA 4 is only used to store just those Meldungen. The commissioner replied that she would be thankful when BND could clarify this discrepancy.

Apparently not noticed by the commissioner is an NSA report from 2006, which was published by earlier Der Spiegel, and which says that German analytic tool suites like MIRA 4:
"integrate multiple database analytic functions (such as viewing voice and listening to fax [sic]), much like NSA Headquarters has UIS (User Integrated Services). In some ways, these tools have features that surpass US SIGINT capabilities. Among a series of interesting items, NSA analysts noted that BND analysts could seamlessly move from VERAS (call-chaining software) to the associated voice cuts."

Later on, the 2006 NSA report says: "The BND responded positively to NSA's request for a copy of MIRA4 and VERAS software, and made several requests from NSA concerning target and tool development and data".

During a parliamentary hearing in October 2014, BND's own data protection officer Ms. H. F. said that in 2010, MIRA 4 was replaced by INBE as a system that apparently not only stores the content of communications, but also makes it available for analysis.

The 2016 commissioner's report says that data stored in MIRA 4 were not migrated to INBE, when the latter system became operational in 2011. Data in MIRA 4 seem to have been automatically "aged off" after 90 days and the last backup of the system was destroyed in the Summer of 2014.

Legal defects

The purpose of the secret report by federal data protection commissioner Andrea Voßhoff was to determine the legality of the data collection, processing, storing and analysing systems at the BND field station in Bad Aibling. The two main problems she identified are about necessity and the lack of database establishing orders.


According to the German data protection law, BND is only allowed to receive, store, process and analyse personal data after checking that they are necessary and relevant for its foreign intelligence mission as authorized by German law. In various cases, especially when it comes to bulk collection of metadata and receiving the selectors from NSA, the agency doesn't or cannot check the necessity for each piece of data. This makes it unlawful for BND to posess and use those data.

The problem behind this is that when such laws were made, there was no awareness of secret services using large sets of metadata, which also includes those of innocent people. Also in this particular case, almost all data collected in Bad Aibling and shared with NSA will be collected from crisis zones like Afghanistan, which makes them more relevant for BND's mission and less likely of containing German communications.

Database establishing orders

Another major legal defect the commissioner found was that for the BND databases VERAS 4, VERAS 6, XKEYSCORE, TND, SCRABBLE, INBE, and DAFIS there was no database establishing order (Dateianordnung) and that they were also set up without prior approval by the commissioner. This makes the existance of these databases unlawful, which means the data they contain should be deleted immediatly until an establishing order is provided.

BND argued that the absence of a database establishing order is just a formal defect and doesn't affect the legal status of a database and its content. The commissioner doesn't agree with that and says that one of the functions of an establishing order is to determine the purpose of a database, which limits and restricts the use of the personal data in it. The lack of such an order also means that there are no rules for when approvals by oversight bodies are required, thus making the use of these databases both unlawful and uncontrolled.

In response

Meanwhile, on September 7, the German interior ministry released a draft for a new data protection act, in which it is proposed that in the future, the data protection commissioner will not have the authority anymore to impose sanctions or fines on the secret services - so restricting the commissioner's authority rather than strenghten it.

Finally, on September 15, Edward Snowden also mentioned the commissioner's report on Twitter, saying that it "confirms mass surveillance". Apparently he didn't read the report, as it is actually about the lack of specific legal restrictions, not about the scope of BND's collection efforts.

Links and Sources
- Rolf Weber: Der geleakte BND-Bericht der BfDI Voßhoff -- wie gewohnt bei näherem Hinsehen wenig skandalträchtig
- Netzpolitik: Secret Report: German Federal Intelligence Service BND Violates Laws And Constitution By The Dozen
- Der Spiegel: NSA-Standorte in Deutschland: Wiesbaden
- Wikipedia: Operation Eikonal