Restricted Data

I wrote this piece up several months ago, and was thinking about what to do with it, where to try and publish it, and so forth. Eventually I came to the conclusion that it would require a whole lot of cutting for anyone to take it up, especially as the “news” aspect of it slipped away. So I’ve decided to publish it on the blog, in a series of two parts. Click here for Part II.

Last October, the US Department of Energy released the full, un-redacted, uncensored transcripts from J. Robert Oppenheimer’s 1954 security board hearing. Oppenheimer, the “father of the atomic bomb,” had his security clearance revoked in late 1953 after accusations were made that he had been a Communist spy. He appealed the revocation, and set into motion the trial of his life. Over the course of four weeks, the details of Oppenheimer’s actions, allegiances, opinions, and personal failures were rehashed and scrutinized under the pretense of evaluating his “character, associations, and loyalty.” At issue was whether Oppenheimer could have continued access to atomic secrets. The government’s judgment was negative, effectively excluding Oppenheimer from any further government service. The transcripts were published shortly thereafter, but with considerable deletions made in the name of security. Does the unmasking of 60-year-old secrets change our understanding of Oppenheimer and his hearing? And why did it take until now for them to be released?

The Oppenheimer security hearing took place behind closed doors, in a temporary building on the National Mall. But the world soon learned of their contents when they were published by the US Government Printing Office (GPO). This was rather remarkable: normally the contents of a security board review were considered confidential information, for fairly obvious reasons relating to both privacy and national security. Each of the forty witnesses called to testify (including Oppenheimer himself) was told that what he or she said was going to be treated as “strictly confidential,” and that the Atomic Energy Commission (AEC) would “take no initiative in the public release of any information relating to these proceedings.” And yet, within days of the conclusion of the hearings, they had become part of public record.

The circumstances behind the publication of the hearings were unusual. Shortly after the Oppenheimer hearing concluded, the Atomic Energy Commission thought they had lost a copy of the transcripts. Assuming they would be leaked to the press anyway, they decided to preemptively publish them. But just before publication, the lost copy was located, yet they decided to publish them anyway. The real reason for the publication was that the primary antagonist of the Oppenheimer affair — AEC Chairman Lewis Strauss — thought that Oppenheimer was getting too much public sympathy. In his mind, if the public could actually see what the decision to deny Oppenheimer’s clearance had been based on, they would see Oppenheimer as the unreliable villain that Strauss felt he was. Strauss’ view was, in retrospect, shortsighted. Almost nobody has read the entire hearing (it is nearly 1,000 pages of dense Government Printing Office typeface, often with no indication of who is answering questions at any given time, and is very repetitive), but the overall tone of the thing is that of an inquisitional character assassination.

First page of the Government Printing Office edition of the Oppenheimer security hearing transcript, which was published soon after the final decision had been made.

Some of the antagonism was inherent to the nature of this sort of hearing. Oppenheimer wasn’t on trial for anything he had specifically done; rather, it was his “character” that was being explicitly evaluated. But some of it was because of dirty dealing on behalf of the Atomic Energy Commission: they were treating it as a prosecutorial trial, except that Oppenheimer was not given any of the legal protections that normally exist in actual criminal prosecutions, such as the assumption of innocence or even prior knowledge of which witnesses would be called. Even worse, as the historian Priscilla McMillan documented in her 2005 book, The Ruin of J. Robert Oppenheimer, the FBI was wiretapping conversations between Oppenheimer and his counsel and giving them, illegally, to the prosecutor. It is not a coincidence that the overall impression one gets from the hearing transcript is that Oppenheimer was set up.

Oppenheimer himself was not even entitled to a full, unexpurgated copy of the hearing transcript for his own personal use during the hearing. This is perhaps one of the most curious aspects of the hearing and its subsequent release and recent declassification. The hearing was a consequence of Oppenheimer’s appealing the suspending of his security clearance. Because his security clearance was suspended, including during the hearing itself, he was not allowed to have access to classified information — even classified information that he was involved in producing. So while a stenographer recorded every utterance said during the hearing itself, a censored copy had to be made daily for use by Oppenheimer. That Oppenheimer was in the room when most of these “classified” utterances occurred made no difference. There were a few “classified sessions” where Oppenheimer was not present, but otherwise he was present throughout the hearing sessions — including the one that took up the entirety of his 50th birthday.

So in principle the hearings were meant to be unclassified, as the defendant, Oppenheimer, was no longer cleared to hear classified information. The information was meant to be “confidential” but not legally secret (it did not require a security clearance to hear). Given the nature of the material under discussion, which involved at times quite subtle technical disagreements over the history of the American thermonuclear program and the military’s plans for using nuclear weapons against the Soviet Union, quite a lot of classified information did end up being discussed, and thus had to be deleted from the transcripts before Oppenheimer could see them. These deletions were initially made by means of a razor blade or pen-knife, literally cut out of the transcript pages themselves. At the National Archives facility in College Park, Maryland, there is a folder that contains these little cut out secrets — the detritus of Cold War secrecy.

I.I. Rabi denouncing the suspension of Oppenheimer’s clearance, as seen in the GPO version of the hearing transcripts. “We have an A-bomb and a whole series of it, * * * and what more do you want, mermaids?” The asterisks indicate a removal by the unnamed and unseen censor.

When the decision was made to publish the hearings in 1954, the classification officers of the Atomic Energy Commission went over the transcripts one more time. The version released to the public contains many conspicuous deletions, indicated with a series of asterisks. For example, in a supporter of Oppenheimer’s, I.I. Rabi, is famously recorded in the published transcript as saying: “We have an A-bomb and a whole series of it, * * * and what more do you want, mermaids?” Rabi was expressing exasperation of the persecution of Oppenheimer. The asterisks, here, denote something that did not survive the censor’s blade — whether the removal was minor or major could not be known.

That the hearings contained omissions was of course noticed by commentators and later historians. What was missing from the Oppenheimer hearing transcript? Did the censors remove only technical information, or much more? Were the censors themselves biased in their operation? Were the technical omissions crucial or minor? Without access to the originals, one could never know. The problem is, nobody seemed to know where the original, unexpurgated transcript was, or whether it had even been kept.

* * *

I first started looking for the uncensored transcript in 2004. I was at a conference on the 100th anniversary of Oppenheimer’s birth, held at the University of California, Berkeley, the spring before I started graduate school. One of the speakers was Richard Polenberg, a historian who had edited an abridged version of the Oppenheimer hearings. One of Polenberg’s remarks before the conference audience was that the original, uncensored version of the transcript appeared to be lost, and he issued something of an open challenge for people to find it. As a budding historian, I was interested in such challenges. Five years later, in 2009, I was doing research at the National Archives facility (“Archives II”) in College Park, Maryland, where most of the records of US federal agencies are kept. By this point I was in the final stages of writing a dissertation on the history of nuclear secrecy in the United States, and had been on many trips to the National Archives and was used to its idiosyncrasies.

The inner storage carousels at the National Archives II facility, where most of the US federal records are kept. These stacks are off-limit to researchers. Image source.

People who have not done research in the National Archives either imagine that it is organized and user-friendly or that it is similar to the sprawling warehouse shown at the end of Raiders of the Lost Ark. The reality is somewhere in between these extremes. The archives are indeed vast and sprawling, though they are kept in neat, clean, moveable, high-volume storage shelving. Researchers are not allowed to browse the stacks (I was allowed to see them once, briefly). Instead, the researcher consults paper “Finding Aids” that are bound (sometimes haphazardly) in three-ring binders. The Finding Aids give some hints at what is kept in the stacks, but they can be a crude metric. They are often photocopies-of-photocopies of documents prepared decades previously using typewriters, with handwritten annotations.

Once one has identified something useful from the Finding Aid, one then has to cross-reference the entry with something called a “Master Location Register,” a different set of materials in a different three-ring binder. The Master Location Register tells researchers and archivists which shelves the boxes in question are supposed to be on. Having acquired that information, the researcher can then fill out a records request (“pull”) form, which has its own idiosyncratic rules for how it should be filled out. Having written out the request, the researcher then presents it to a reference librarian who scrutinizes it for formal adherence to a set of unwritten basic requirements. If it is judged to be formally sound, it is time-stamped and put into a bin.

The researcher, mind you, cannot simply request as many boxes as they want. There are limits to how many boxes of records will be retrieved, how many collections you can request records from at once, and how many “pull” requests you can make over the course of a day. There are five designated pull times on weekdays. The earliest is at 10:00am, the last is at 3:30pm. Any requests submitted between the hours of 11:05am and 1:30pm will not start to be processed until the 1:30pm pull time. Miss the 3:30pm pull time, and your records will not be pulled until 10:00am the next day. It can take between 20 minutes and an hour to actually see the fruits of any given records request. Sometimes the results are the records you asked for. Sometimes they are yellow slips of paper indicating that the records were not found in the place you said they should be, or that you violated a rule in filling out the request form, or that another researcher is already using the records (sometimes said researcher is a government employee working in a separate, inaccessible part of the facility).

A familiar sight to Archives II researchers: “You done messed up.”

If this sounds like a complicated system where a lot can go wrong, it is, and it is unusual among American archives in its complexity. When novice researchers ask me about using the records at the College Park facility, I tell them to factor in about twice as much research time as they might take at a “normal” archive, and to expect to spend the entirety of the first day learning how the system works to the point where they can actually file research requests that get useful records as a result. Of course, for the researcher, the real work only begins once the records have arrived. The downsides of such a system are obvious. The upside for a historian, though, is that in such a maze of paper there are sometimes still treasures to be found.

I had been at the archives for a week, and had exhausted all of my pull requests, except one. Because a set of records I had wanted to see proved useless, I found myself unexpectedly with some free time. Rather than going back to my hotel, I decided to do a little “fishing.” Sometimes Finding Aids are wrong, and sometimes they are out of sync with the MLR records. Sometimes records in the MLR lack Finding Aids, making them rarely used by researchers. A trick I had found over the years was to go over the MLR very carefully and look for anomalies: things that were in one database but not another, or were mislabeled. Doing this for the records of the Atomic Energy Commission, I found an unusual entry of files relating to the Oppenheimer hearing that was labeled as classified (and thus off-limits to me), but it was housed in a part of the facility that was for unclassified or declassified records. The Finding Aid provided no useful information about it.

I thought it was worth a chance to try and request it, because it seemed like the MLR might just have incorrect information in it, and it wasn’t at all clear what these files actually were. In the worst-case scenario, the pull request would come back as invalid, or it might just be one of the many copious files relating to Oppenheimer that scholars had looked at a dozens of times before over the years.

The cover and first page of the original Oppenheimer hearing transcript. In the left photo, I am holding back a “Top Secret/Restricted Data” cover sheet. I have cropped out my declassification slug. The color photos of the transcripts are from a 2011 follow-up trip to NARA I made; the original photos I took in 2009 were grayscale (as is my usual archival practice), which is why I am illustrating this post with the 2011 photographs. Note that the transcriber, on the first page at right, got Oppenheimer’s name wrong at the top — “Oppenheim” plus a handwritten “er.”

When the “pull” came back, I was genuinely surprised to find that this mysterious, erroneous entry contained many of the original, un-redacted volumes of the Oppenheimer hearing. These were small blue stenography books produced for the use of the security hearing board itself by the Alderson Reporting Company, not for publication. On their covers were the stamps that characterize government document: “ORIGINAL,” “SECRET,” “RESTRICTED DATA.” Hold your breath, open the cover: instead of asterisks denoting classified omissions, they contained the missing text, circled in the red-orange pencil of the censor. I took photographs of all of the pages with removals on them, glancing over m shoulder the whole time, not wanting to let on my excitement.

There was only one problem: not all of the volumes had been declassified. Of 25 books worth of material, I had 17. Enough to see that I had found something wonderful, but not enough to do anything with it — nobody cares about finding most of the original Oppenheimer transcripts. Those that were missing had in their place “Withdrawal Notices,” pieces of card stock which say, in essence, “Sorry, you can’t see this.”  Fortunately they contain notations on them that can tell the archive where the still-officially-secret originals are being kept in in some other, more tightly-guarded part of the archive, and can be used to aid in Freedom of Information Act (FOIA) requests that compel the government to review the materials for release.

In theory, all classified documents contain the record of their classification status, and how it changes over the years, stamped on them. (This procedure, which is now common throughout the American classification system, was begun with Manhattan Project records in early 1946 at the recommendation of a committee of scientists that included Oppenheimer.) A close look at the documents and their containers revealed that the Department of Energy had transferred them to the National Archives in 1991, and that in May 1992, someone had started declassifying them. But after 17 volumes, they stopped. Why? It isn’t clear. The early 1990s were a period of classification reform and “Openness” under President Clinton’s Secretary of Energy, Hazel O’Leary, and it would poetic if the Oppenheimer transcript fell to the wayside because they were too busy declassifying other things (and, eventually, fighting back against Congressional Republicans who eventually stopped the “Openness Initiative” in its tracks). In any case, it looks like things got stopped mid-declassification and that this was responsible for the sort of “limbo” these records ended up in — with an incorrect MLR entry and nobody quite knowing what had happened to them, until I stumbled across them 17 years later.

The stamps on a declassified document can tell you its classification history, if you know how to decode them. The cover of the Oppenheimer transcript indicates evidence of its original review in 1954 for publication, the record of it being catalogued by the AEC office of history (where it stayed until 1991, when it was transferred to NARA), and evidence of its declassification in 1992 by a DOE contractor. The “X”s through the “Restricted Data” stamp and the original 1954 note are meant to indicate that they are not longer valid (they should have drawn a line through the “SECRET” stamp, too, but this is often neglected).

When I got back from my archive trip I immediately filed a Freedom of Information Act request for the remaining volumes. I knew this entailed a little bit of risk of being scooped. FOIA requests are not confidential; there are people who file FOIA requests to find out what other people filing FOIA requests on. In principle there is nothing wrong with this. Scholars have no proprietary claims on information that the government created, and once the government declassifies something it is available to everyone. But as with scientists, priority matters for historians: we like to take credit for what we find. But until I had the missing volumes, I felt I to be fairly quiet about the entire thing, telling only a few trusted colleagues.

The speed of response to a FOIA request can vary by the material and by the agency. The FBI, in my experience, is quite fast, despite (or maybe because) of their reputation for secrecy. They manage to process even quite large files usually within six months to a year. The Department of Energy is also relatively efficient. Waiting a year or two when you are trying to finish a dissertation or a book is a long time, but one cultivates a sense of patience about these things. Unfortunately, to get records that are already in the National Archives declassified, you have to file a FOIA request to the National Archives and Records Administration (NARA), who in turn has to turn them over to the actual declassifying agency (whichever agency, or its heirs, made the information classified in the first place). As the caretaker of all government archives, NARA receives huge volumes of FOIA requests on all topics, and so has a massive backlog. So my FOIA request for the Oppenheimer hearings would have to worm its way through the NARA system in order to be forwarded to the Department of Energy, the institutional heir of the Atomic Energy Commission, so the actual work of declassification review could begin.

Like Oppenheimer, one must cultivate a sense of Zen while waiting for classified documents to be reviewed. Photo source: Emilio Segrè Visual Archives at the AIP Niels Bohr Library. (The first Oppenheimer photo at the top of this post is also from the ESVA.)

So I expected this to be a slow process. But it was much slower than I’d have guessed. For three years, NARA did nothing with my request. At regular intervals I checked in, via e-mail, on its status, and every time was told that it was simply in a very long queue. The NARA employee I corresponded with was sympathetic and friendly, but insisted that he could do nothing to improve the speed of the system. So I waited — not for them to actually declassify the records, but for them to even start processing them, so that they could be sent to the Department of Energy for the actual declassification effort.

Finally, in 2012, I was told they were “out for review,” having finally been sent to the Department of Energy. It seemed like things might finally pick up. Still, I heard almost nothing for another two years. That is, until October, when I saw that the Department of Energy had declassified the entire transcript and posted it onto their OpenNet website… without informing me. A contact of mine at the Department of Energy has assured me that they did not realize there was a FOIA request associated with these records, and my contact at the National Archives has apologized over e-mail for the way this got handled.¹

As you can imagine, I was more than a little surprised that a process that made no obvious steps forward over the course of six years suddenly burst into the public eye in the most rapid way possible. In their defense, NARA seemed just as surprised as I was that the files had been posted online, which complicated their own screening process — as often happens in the Federal Government, the left hand didn’t quite know what the right hand was doing. As someone who studies the history of nuclear secrecy, I have allowed myself to be amused by the way this has all transpired. To have my priority claims on finding a secret document dashed by excessive openness on behalf of the government is perhaps an appropriately ironic fate, is it not? One of the key points of my (someday) forthcoming book is that revelation can be as much as a problematic activity as concealment, though this in this case it was a bit more personal than usual!

Part II, which contextualizes the newly revealed content and its impact on Oppenheimer’s legacy, is available here.

1. In full disclosure, I worked briefly for the Department of Energy while in graduate school — I was the Edward Teller Graduate Fellow in Science and Security Studies for 2007-2008. This gets me no advantages other than knowing who to contact in their history division when I have questions (and I have good relations with the history division), and knowing a bit about how their system works. [↩]