Artificial Intelligence (AI), crowdsourcing 2.0 and Handwritten Text Recognition (HTR), enable a smart search through 200,000 scans of historical texts word by word. The goal our project is creating a prototype for a smart archival search for our archival partners: the Amsterdam City Archives (Stadsarchief Amsterdam), the Dutch National Archives (Nationaal Archief) and the North Holland Archives (Noord-Hollands Archief).
However, searching for locations in historical texts is problematic for several reasons, while the need for this is significant. Together with the archives and a crowd of volunteers, Aincient and partners created AI datasets for finding historical locations in transcriptions. We are developing a prototype search environment where scans, transcriptions, historical maps and (historical) images can be explored. Aincient has formed a consortium of heritage and AI specialists, consisting of Aincient, Picturae, Sioux Technologies and Islands of Meaning.
The AI application offers a solution to the searchability problem of the archives and the users of the archives, especially in the area of locations. Every day, archives are searched online by thousands of users, partly as an important source for scientific research. In addition, this solution can be applied in many heritage institutions in the public sector, in the Netherlands and abroad.
In recent years there has been an enormous development within AI in the field of Natural Language Processing (NLP). We focus within NLP on Named Entity Recognition (NER) for the automatic recognition of locations in historical texts. We are using the open source tool BERT by Google, a deep learning AI solution. In the previous SBIR phase, the initial results achieved with BERT were good.
What challenge does it solve?
This project solves several challenges. The first challenge is the fact that locations are hard to find in the extensive online archives, while there is a great need for them. Even just for researching the history of your own house or city. This applies to searching through text as well as through maps. The other challenge is the scarcity of datasets to train AI in the heritage sector, also when it comes to locations. Together with the archives and a crowd of volunteers, we created AI datasets for locations, names and dates. For this purpose we launched the project ‘Tag the text’ on crowdsourcing platform VeleHanden. Over 10,000 historical texts have been tagged and checked within two months.
Small Business Innovation Research (SBIR)
Aincient and partners participate in the SBIR ‘Artificial intelligence for public services’. Phase 1 has been successfully completed and we are currently working on phase 2, the development of a prototype. We also participate in the SBIR for the National Archives, of which phase 2 has been completed and the implementation phase will be finished soon. In addition, Aincient is a member of the working group Culture of the Netherlands AI Coalition.
SBIR makes use of an exception in the procurement legislation for applied research and development. A societal question posed by the government is central. SBIR works in a tiered innovation competition towards the innovative solution(s). The best proposals execute a feasibility study and the best feasibility studies are commissioned to develop an innovation. This innovation is ideally tested in practice by the potential government customer.
In collaboration with:
Aincient is carrying out the SBIR AI project in collaboration with the Amsterdam City Archives (Stadsarchief Amsterdam), the Dutch National Archives (Nationaal Archief) and the North Holland Archives (Noord-Hollands Archief), Picturae, Sioux Technologies, and Islands of Meaning.
This article is a moderated version of the arcticle published online by the NLAIC, the Netherlands AI Coalition.