Members of the IRENE project have recently used the AI (artificial intelligence) application ChatGPT to analyse an 1800s sound recording, unlocking a new AI data analysis technique that could transform their research to uncover the sounds and voices of the past.

Project IRENE started at Berkeley Lab over 20 years ago as a collaboration with the Library of Congress to develop optical techniques to restore the earliest sound recordings ever made. One of the team’s earlier major efforts resulted in the digitization of over 3000 wax recordings of Native Americans from the recorded sound collection of the UC Berkeley Phoebe Hearst Museum of Anthropology.

The team’s current focus is recovering and digitizing sound from about 200 experimental recordings that originated in the 1880s in the Volta Laboratory of Alexander Graham Bell. These recordings are part of the collections at the Smithsonian’s National Museum of American History and document the very beginning of sound recording technology and the process of invention and discovery in the United States at that time. This project is the most recent phase of a collaboration that began in 2009 with Berkeley Lab, the Library of Congress, and the Museum.

Among the items in the Volta Laboratory collection were a set of delicate discs made out of a thin coating of wax on “binder board,” a material similar to the hardcover of a book. One such item contained the voice of Alexander Graham Bell himself.

“These techniques are non-contact, so even a very delicate or damaged item can, in principle, be restored,” according to the project’s principal investigator, Carl Haber of Berkeley Lab’s Physics Division.

The team recently came upon another such disc, excessively riddled with cracks. Using IRENE technology, they made an optical scan of this early disc, and the data was analyzed by UC Berkeley undergraduate Evan Odell using a software package referred to as Weaver – largely developed by Earl Cornell of the Lab’s Engineering Division – which reads high-resolution image files gathered by IRENE and converts the image data to audio, optionally removing defects that can degrade the playback experience.

In spite of this filtering, the recordings from these early experiments are often noisy and difficult to understand. In this case, the speaker was clearly reciting something, vaguely poetic, but no one could quite understand the content:

Odell then had the idea to feed this audio into an AI software application called Whisper – developed by OpenAI and available for download from GitHub – and it returned this transcription:
Not at one was heard, not at the email he made, not his thoughts so unpassed to be heard, not his favor described so far when it was done, or the praise that I hear over and over again.

After making the initial transcription using Whisper, Odell then copied the transcription over to ChatGPT and asked it to search for 19th-century literature with a similar meter and form to the transcription that Whisper made. ChatGPT returned this new transcription:
Not a tone was heard, not a murmur he made, not his thoughts so unpassed to be heard, nor his favor described so far when it was done, or the praise that I hear over and over again.

ChatGPT also returned this additional note: That resembles the somber, formal tone found in elegiac poetry of the 18th and 19th centuries. Strong Possibility: “The Burial of Sir John Moore after Corunna” by Charles Wolfe (1817). Here’s the opening stanza:
Not a drum was heard, not a funeral note,
As his corse to the rampart we hurried;
Not a soldier discharged his farewell shot
O’er the grave where our hero was buried.

ChatGPT concluded that, Your transcription most likely refers to “The Burial of Sir John Moore after Corunna” by Charles Wolfe, written in 1817. The garbled phrases appear to be a poor recognition of the original poem’s solemn, repetitive form.

And if we listen again to the same audio recording and follow this text of Wolfe’s poem, we can recognize the verses much more clearly:

“It seems highly plausible that ChatGPT has correctly identified this recording, and this seems like a perfect application of these large statistical models, which may contain such a broad collection of data as to make these identifications possible,” explains Haber.

Going forward, the IRENE team hopes to use these tools to further illuminate this early period of invention and innovation, and to develop new methods of noise reduction, as well.

The project to recover recorded sound from the collection at the National Museum of American History is supported by the National Park Service through a Save America’s Treasures grant, with matching funds from the Linda and Mike Curb Foundation, Seal Storage, SEDDi Inc., the Filecoin Foundation for the Decentralized Web, and the Alexander and Mabel Bell Legacy Foundation. The development of IRENE, and earlier projects, were supported by the Library of Congress, the National Science Foundation, the University of California, the National Archives, the Institute of Museum and Library Services, and the National Endowment for the Humanities.