The Adams Papers Editorial Project and John Quincy Adams Diary Digital Project
The Adams Papers is a documentary editing project located at the Massachusetts Historical Society (MHS) that publishes in print and digital form the letters and other writing of founders John and Abigail Adams and three generations of their family (1735–1889). Established in 1954, the project was conceived as a 100-plus volume letterpress edition in three main series: diaries, family correspondence, and papers of the Adams statesmen. To date, the project has published 51 print volumes across the three series, 45 of which are now available online in the Adams Papers Digital Edition (APDE). Other digital initiatives, which are all made freely available on the MHS website (www.masshist.org/adams), include a family tree, timeline, and biographical sketches. The project has also digitized its control file; the Online Adams Catalog (OAC) represents a fully searchable electronic database of all known Adams documents, providing access to over 110,000 records, with some 30,000 cross-reference links to online, printed, and microfilm editions of the items or to websites of the holding repositories. Each record contains information on a document’s author, recipient, date, and location of the original, if known.
|
The Adams Papers has a successful formula for producing quality content in a timely fashion, yet the majority of work beyond the first generation of Adamses remains to be done. As the project looks toward fulfilling its mission to publish the multigenerational writings of this prominent American family, it becomes necessary to develop innovative ways to increase accessibility to the material while still adhering to the core standards of the field. The key tenets of documentary editing are to provide an accurate, verified transcription and to contextualize the documents within their historical time, both tasks that are best completed by experienced editors. Keeping these core standards in mind, we conceived the John Quincy Adams Diary Digital Project.
The John Quincy Adams Diary Digital Project (DJQA)
John Quincy Adams (JQA, 1767–1848) is one of America’s great statesmen. The oldest son of founders John and Abigail Adams, his distinguished career in public service spanned six decades and included roles as diplomat, secretary of state, president, and congressman. For 68 years, JQA kept a diary of his public and private experiences. The 51-volume diary comprises the longest continuous record of any American of the time and provides an unparalleled resource not only for scholars—the traditional audience for this type of publication—but for educators, students, and a general public interested in history. Building on a project completed more than a decade ago that digitized the entire JQA manuscript diary and created basic metadata for every date entry, the DJQA project will make JQA’s diary truly accessible for the first time by presenting a verified and searchable transcription of each entry on the MHS website. Transcriptions alone will improve access, but with more than 15,000 pages, the usefulness of the diary can only be leveraged through improved search options.
In a traditional documentary editing project, annotation is what provides the context that enables users to better understand a document or group of documents. Annotation is a labor-intensive undertaking, however, and does not necessarily address the needs of all audiences. The DJQA project offers a new approach to providing deeper engagement with the material without undertaking traditional annotation. The solution we envision is the development of robust digital tools that will allow name or subject searches of the text.
Throughout his life, JQA interacted with thousands of individuals. A working names list has been generated with each individual’s full name, birth and death dates, date first mentioned in the diary, brief biography, and a unique identification tag (XML ID). This list, produced in consultation with a subject expert who is analyzing each diary entry’s contents, will allow the MHS web developer to build a search tool to enable users to search for individuals, even when they have not been expressly named by JQA in the text. For example, if JQA mentions “father and mother,” the names will be encoded with the XML IDs as adams-john and adams-abigail, or when JQA mentions “the Consul at the Cape de Verd Islands,” the XML ID will be hodges-samuel. Once the relevant search tool has been created, users will be able to search for any individual with an XML ID, regardless of how they were referenced by JQA.
Another means of providing increased discoverability is topical searchability. To facilitate topical searches of the diary, DJQA project staff consulted with members of our target audience—the K-12 community—to identify approximately 100 themes related to American history from 1817 to 1848. These form the basis of the subject analysis where a subject expert reviews the text and assigns relevant topical headings for each entry. For example, in a particular diary entry, topical access could be added for Industrialization, Native Americans, Science and Technology, or Slave Trade—subjects of particular interest to students and teachers and utilized in history courses across the country—even though JQA may not have written those specific words in his diary. To streamline search results across a diary with many multipage entries, this subject access will be employed at the entry level instead of for each page of a date entry. Both the name and subject search tools will enable more complex and informed discoverability on nationally significant historical topics than a keyword search alone could provide, and they specifically address the needs identified in consultation with our target audience.
The DJQA project then offers a good example of how placing the public’s needs at the forefront of project planning can necessitate limited public participation in some aspects of public digital archives projects. The DJQA project places audience at the center of its planning discussions, attempting to balance the needs of the traditional users of a documentary edition—scholars—with the goal of broadening that audience, specifically making it useful to the K-12 community. K-12 educators and staff at the Center for the Teaching of History (CTH) at the MHS were engaged in the initial phase to determine usable outcomes for their community. From these discussions we learned that while online transcriptions address the accessibility issue for students who have difficulty reading handwritten documents, keyword searching alone cannot adequately reveal the full, rich content of historical materials like the JQA diaries. These conversations established the framework for undertaking the subject analysis, and DJQA staff collaborated with educators and CTH staff to generate the topical headings list. Once the search tools have been built, K-12 educators will be included in the beta-testing and refinement phase before the tools are made available on the DJQA website.
While involving the target audience in multiple steps of the process addresses the needs of this core community of users, it will also benefit both the traditional audience and general-interest user. Scholars, whose most immediate need is a verified transcription, will benefit from the regularization of names and the tagging of individuals not identified in the text, enabling more thorough targeted searches of the diary’s text. The broader public audience benefits from the availability of an accurate transcription, more refined search functions, as well as from headnotes—in short and long form—that explicate the chronology of JQA’s life. In these ways, our audiences shaped the development of the project and will continue to shape its final version.
Participatory engagement with transcription of a historical archive can be a powerful marketing tool, but adequate quality controls must be built in to a project in order to yield a transcription that meets the standard of the documentary editing community. In the summer of 2017, the MHS attempted to bridge this divide when it held its first transcribe-a-thon to commemorate JQA’s birthday. Over the course of several hours, 30 participants transcribed nearly 80 pages of the diary. To provide some control, we asked each participant to start by transcribing the same short paragraph that contained many of JQA’s quirks of handwriting. This gave us a qualitative baseline by which to review the transcriptions and make quick determinations about usability. Overall, approximately 90 percent of the transcriptions were incorporated into the project’s transcription files. The transcribe-a-thon also yielded a handful of volunteer transcribers, another way the DJQA project is trying to engage the public in its work. All transcriptions, however, will be verified before being published online, a vital step of the process and one that is best done by project staff in order to maintain the standards of the Adams Papers edition as a whole.
Along with increased participation in transcription, we envision a few steps of the process where the DJQA project could further incorporate public engagement. One possibility might be providing the ability for audience tagging of the topical tags with a final review completed by a subject expert. Another might be to develop a more dynamic presentation of the material. Algorithms that prioritize topical search results would allow the public’s needs to drive searches of the material. For example, if a user wanted to view JQA diary entries related to the Missouri Compromise, they might engage the search feature and find the results that provide all mentions of the topic but highlight the five most viewed by other users.
Our work on the DJQA project demonstrates the importance of keeping audience at the center of decision-making when creating digital archives projects. However, public engagement also requires practical limits in terms of verifying transcriptions and providing historical context through tagging. By being realistic about when and how the public can benefit a digital project, we can establish parameters to maximize public involvement while maintaining the core standards of documentary editing.