HIMANIS a, depuis le début, des liens avec l’infrastructure READ (Recognition and Enrichment of Archival Documents, https://read.transkribus.eu/), financée par l’Europe et porteuse du logiciel Transkribus. En effet, l’Université polytechnique de Valencia est partenaire de READ.
L’IRHT et le CNRS sont en train de signer une convention de partenariat avec READ pour pouvoir travailler ensemble de façon plus intégrée et préparer l’avenir, maintenant qu’HIMANIS est achevé et pour après la fin du financement européen de READ. On devait simplement signer un “Memorandum of Understanding”, mais les complexités légales du CNRS font que le “Memorandum” préparé le 12 juin 2017 et que quatre-vingts institutions ont déjà signé sans modification a dû être revu et n’est toujours pas signé par le CNRS…
Mais cela ne nous empêche pas de travailler ! Et voici les résultats…
Nous avons envoyé à READ les images du corpus ainsi qu’une extraction du texte “brut” des actes édités pour lesquels Sébastien Hamel avait repéré les “zones” sur les images (1726 zones sur 1263 images: 1334 zones pour l’édition de Guérin, 75 pour l’édition de Viard), ainsi que les liens nécessaires pour utiliser l’édition du formulaire de Morchesne. READ a aligné les éditions ligne à ligne, à la main (contrairement à ce que nous avons fait dans Himanis, où l’alignement a été fait automatiquement, mais, en conséquence, avec des résultats de moindre qualité).
A partir de cela, READ vient d’entraîner des “modèles optiques” pour la reconnaissance de l’écriture manuscrite et a testé sur quelques dizaines de pages. Les résultats sont très impressionnants, comme le montrent les captures d’écran suivantes.
La comparaison des deux versions montre l’excellente qualité des modèles entraînés, d’autant que de nombreuses “erreurs” de transcription sont des artefacts de l’édition (présence de ponctuation, segmentation de mots et apostrophes…).
Il reste à faire pour étendre la reconnaissance de texte en séparant le corpus d’entraînement et le corpus de test, mais ces résultats absolument extraordinaires sont plus qu’encourageants !
Artificial Intelligence has unlocked the access to the text of medieval manuscripts! The partners of the European research project HIMANIS implemented, for the first time, the indexing and plain text querying of thousands of pages of medieval manuscripts. The large scale of the corpus and the possibility to search in plain text for handwritten sources are unheard of in medieval studies, so that the results present a major shift for historians. The challenge of multilingualism, script variation and abbreviations, which are crucial for HTR on medieval sources, has been successfully met.
Some statistics show that we are having an emerging, large community of users. A growing number of people are reading this blog: ca. 12’500 unique visitors in 2017, with more than 34’000 pages visited; 7’400 and 30’000 pages visited for the first six months of 2018.
There are also 301 followers on Twitter, which is a good start.
On MONK, there is very intense use, with amazing statistics (more than 1 million actions, partly from bots).
Last but not least, on the public search engine, there have been:
more than 813’000 search actions: around 40’000 queries per month from Oct. 2017 to May 2018, jumping at 495’455 queries in June 2018 (hurray!)
more than 15’000 user feedback actions, from all over the world (see map below)
A second conference from the project HIMANIS (Historical Manuscripts for user-controlled Search), about handwriting recognition by computers of the ancient writings, is held on Tuesday, May 29, 2018 at the Archives Nationales (Paris site – Hôtel de Soubise, chambre du Prince).
Since 2015, the Archives Nationales became an associated partner with the IRHT-led HIMANIS (Historical Manuscripts for User-controlled Search) research project, which goals were to study the records of the French Royal Chancery of the 14th and 15th centuries. The major technological challenge was to carry out computer recognition of medieval writing that was sufficiently reliable to provide the community with a full-text search engine of the registers, many of which were not yet covered by inventories or index. A web interface, combining access to images, transcripts and archival analyzes, has also been developed.
Vingt registres supplémentaires ont été intégrés à l’interface de recherche. Parmi les nouvelles recherches possibles, nous trouvons une nouvelle preuve de la présence quotidienne des livres d’heures.
A la Saint-Martin, le 11 novembre, 1447, Raoul est trouvé noyé dans un estang. Suicide ou accident ? Ses parents demandent grâce au roi pour que le doute ne jette pas le déshonneur sur la famille et n’empêche pas l’application des règles normales d’héritage. Que reste-t-il dudit Raoul au bord de l’étang ? Son livre d’heures, son manteau, ses chaussures et son chapeau (« et ses heures, mantel, patins et chappel estans sur ladicte chausee pres de la bonde dudit estang ») !
Revenant du Mexique, je souhaite à tous une bonne fête des Morts…
To celebrate the achievements and the end of the HIMANIS project, the Institut de Recherche et d’Histoire des Textes (IRHT) is organizng a conference on Friday, 27 October 2017, to present, explain and discuss the impact of handwriting recognition technologies for medieval studies and heritage institutions.
The European project HIMANIS (HIstorical MANuscript Indexing for user-controlled search), led by the IRHT, brought together several research teams in engineering sciences, both private (A2iA, France), and public (PRHLT-UPVLC, Spain and Alice-Groningen Univ., The Netherlands), with the support of the National Archives and the National Library of France.
The main result of this project is that it is now possible to carry out keyword searches in the corpus of registers of the French royal chancery of the 14th and 15th centuries. For the moment there are 167 volumes of the National Archives series, JJ 7 – JJ 209 and 20 volumes of the Bibliothèque nationale de France. The interface (still preliminary) can be accessed at the following address: http://prhlt-kws.prhlt.upv.es/himanis/. It is a test interface, which will evolve in the coming months, with additional manuscripts and new functionalities, notably to
navigate between the results, to display the regests and editions. HIMANIS also aims at computer-assisted analysis of the different hands involved in each volume, to study the functioning of the royal chancellery and the progressive construction of the centralized state of the French monarchy.
Research Fellow / Ingénieur de recherche en analyse de sources
Institution: Institut de Recherche et d’Histoire des Textes
Location: Paris, France
Application Due: 31 Oct. 2017
Type: Full Time
Duration: 1 year, renewable until 3 years
Starting Date: between January 1, 2018 and April 1, 2018
The Institut de Recherche et d’Histoire des Textes (CNRS) invites applications for a Research Fellow (“Ingénieur(e) de recherché en analyse de sources”) for the project “HORAE – Hours, Analysis, Editions” (ANR-17-CE38-0008), under the supervision of Mr. Dominique Stutzmann, Principal Investigator.
We are delighted to announce that the French research agency is funding the HORAE project (ANR-17-CE38-0008). HIMANIS also highlights why it is important with one example:
un certain lieu ou il estoit alé disant ses heures
(translation: a place where he went saying his Hours; image Paris, Archives Nationales, JJ44, fol. 98r)
In another example, the book of hours of a Jehannette, daughter of Jehan Petit is central to the plot, since it is the cause of the crime! Used to force a date with the young Jehannette, it created a fight between the father and the daughter’s friend…
Thomas, par esbatement print les heures de ladite Jehannette qui estoient sur une fenestre dudit hostel en la presence de ladite Jehannette et de son consentement les emporta et dist a ycelle Jehannette qu’elle les alast querir a vespres en l’eglise, laquelle lui respondi que si feroit elle. Et tantost aprés, ledit Jehan Petit ala aprés ledit Thomas ou cymetiere de ladite eglise ou il estoit et de felon courage lui dist qu’il lui rendist ses heures, ou si non qu’il l’en poursuiroit et que faulsement et mauvaisement il les lui avoit emblees.
At the core of the late medieval society…
HORAE (Hours – Recognition, Analysis, Editions) is a cross-disciplinary research project studying religious practices and experiences in the late Middle Ages through the books of hours, the absolute medieval best seller. In a public-private partnership, it gathers three partners in the Humanities and Computer Science: the Institut de Recherche et d’Histoire des Textes (IRHT, UPR 841), the private company TEKLIA and the Laboratoire des Sciences du Numérique de Nantes (LS2N, UMR6004). Books of hours forms, with more than 5000 preserved manuscripts, a vast and crucial ensemble to understand the medieval mindset. Yet, their textual content is very scarcely studied, although the massive production of such a large number of manuscripts is a pivotal cultural and industrial phenomenon to the profound changes in the medieval religious life. Not only a proto-industrial book production arises and the economics of demand shifts to the economics of offer, but also the awakening of the individual and the internalization of faith take places in an increasingly church controlled world. Books of hours are at the same time deluxe objects, implying social staging, and intimate objects, with devotional, memorial and eschatological challenges.
HORAE combines the research and expertise of all three partners in artificial intelligence applied to Computer Vision and image analysis, in Natural Language Processing (NLP), and in book history and religious practices. The research project aims at creating an integrated chain from image treatment to producing new knowledge by placing the end user at the center of the developments, by managing formats, interoperability and long term preservation, but also ergonomics and data visualization.The aims encompass: (1) re-using the many digitized manuscripts, which are online but underused resources; (2) new open source software for Handwritten Text Recognition (HTR); (3) tools for segmentation and plagiarism detection, adapted to the transcription of medieval manuscripts produced by the machine in order to identify the texts in the books of hours; (4) identifying and editing unpublished texts; (5) visualization of manuscript clusters, which share the same textual characteristics in the order of the different parts (Officium Beatae Mariae Virginis, votive offices, suffrages, prayers), but also in the order of textual units to identify the liturgical use; (6) studying the diffusion and circulation of devotional and liturgical texts at the end of the Middle Ages in order to better understand the cultures and faith in the 13th c.-16th c.
With its aims and methods, HORAE profoundly changes the ways of doing research in auxiliary sciences and tackles the challenges of big data. Books of hours have been too scarcely studied until new because they are too numerous, too complex and their text is very repetitive: now, their very number, repetitions and complexities make possible for this project to develop new and efficient technologies and methodologies and gain new knowledge on the Middle Ages.
We improve the way to correct the results by adding two new functionalities:
1/ After a query, on the spotted words, when you click on the word you are asked if the word is correctly spotted. Not entirely sure of the result you got, we add a third option (“not sure”) to “correct” or “incorrect”.
2/ When clicking on a word to see if it has been recognized, you have now a dropdown menu suggestions of the recognized word in order of confidence. You can now just select the correct answer from the list rather then typing it. Of course, if none of the suggested result is good you can still type the correct word manually.
The bounding box surrounding the word now appears and they are in color (green, yellow, red orgrey) so you can have an idea of the confidence level of the word you are going to select. And don’t worry if the suggest word in the bounding box have nothing to do with the one you clicked: the system will only take into account the pixel on which you clicked on.
Lemmatized search is on it’s way. We are still testing it on a small sample of the corpus but it will be implement in the coming month.
New documents have just been added to the HIMANIS search engine. The new documents come from BNF digitized microfilm available on Gallica and that are related to the French royal chancery. Those are either originals or copies of chancery registers that are for various reasons located at the BNF rather then at the AN. Actual shelf marks of the chancery registers of the Trésor des Chartes come from the system put in place around 1380 by Gerard of Montaigu in the last version of his inventory of the libri utiles (actual JJ 115) who numbered the registers from I (JJ 11) to CXV (JJ 115). When the JJ serie was organized, between 1852 and 1854, numerous registers where added to the original serie that were not originaly in it:
Some came from what was considered by Montaigu as libi inutiles (JJ 42B, 79B, 267 à 270, 288, JJNN et JJOO).
Others came from the J serie (JJ 20, 24B -24D, 274 and 304-320) or even P (JJ 30B), Q1 (JJ 24A2) and X (JJ 54B) series.
Numerous are inventories and copies from the 13th to the 18th century (JJ 12A-JJ 128, JJ 9A, 9B, 27A-27C, 271-273, 275-303, 534-5884 et JJ A-JJ K).
In 1862 an exchange of documents took place between the AN and the BN and all of the above (except JJ 27C, 272 and 273) have been transferred to the BN in exchange for various inventories (nowadays JJ 12B-127).
The HIMANIS project is funded by the Joint Programming Initiative on Cultural Heritage and Global Change” (JPI-CH) of the European Union. The partners are developing cost-effective solutions for querying large sets of handwritten document images. It gathers Computer Science, Humanities and Cultural Heritage institutions in order to produce technology to generate new, research-based knowledge from historical manuscripts. As a challenging and particularly interesting case study, the large collection of the Trésor des Chartes’ registers produced by the French royal chancery (Paris, Archives Nationales, JJ35 – JJ211). The expected outcomes of this project are:
a new indexing/searching technology for historical manuscripts
a new paradigm to study our historical heritage, as conveyed by manuscripts, by using full text search technology.
a new vision of the raise of nation states in Europe via a new study of the corpus under this paradigm.
You can search the plain text in the Trésor des Chartes’ registers and provide feedback!
How to search?
A query box is provided to type the queries
The search engine is case insensitive (“Paris” and “paris” retrieve the same results). The search engine allows for “AND”, “OR” and “EXPRESSION” queries (see below).
Abbreviations are not queried as they appear in the images; instead the corresponding expanded forms are used. For example, use “paris” to find “Paris”, “par.”, “par^s”, “Pars.”, etc.
Query text must be diacritcs-folded (e.g., “sebastien”, rather than “Sébastien”).
A confidence box and slider are provided to express a confidence threshold which determines the desired precision-recall tradeoff. The default is 50%. A higher value results in higher precision (little or none wrong spots) and lower recall (some, or many existing instances of the query may not be retrieved); a lower value results in lower precision (some, or many wrong spots) and higher recall (all or most the existing instances can be retrieved).
A ‘Max. results’ box is provided to express the maximum number of spots to be retrived and shown. For example, if this number is set to 20 and the confidence is set to 40%, the system shows at most 20 spots with confidence higher than 40%; and if no (or 0) confidence is set, the system retrieves at most 20 spots with the highest confidence.
Search: AND, OR, SEQUENCE
The search engine allows compound queries (boolean AND, OR, NOT) and “sequences”. For compound queries, the results are computed at Page level, and the number of matches is the total number of word occurrences matching the query.
Relations are interpreted at the full page-image level.
The operators are: AND: “&&” (or just blank space), OR: “||”, NOT: “-” (before the words which have to be negated). Parenthesis “(“, “)”, can be used for grouping
For “Sequence”, the number of match is computed at Page level and one match corresponds to the complete expression.
A query can be a sequence of words, expressed using the symbols “[” and “]”. Sequences are not exact strings; they allow (a few) extra words to appear among the stated words. Examples: “[ludovico francorum]” (where “francorum” is an expanded abbreviastion) would retrieve pages with strings such as “ludovico franc.”, “Ludovico rege franc.”, “ludovico dei gra. francorum”, etc. In the current system version, sequences can not be mixed with boolean operators.
You can search for single words (click on each word to see the results).
The current presentation of results is bound by the hierarchy of the collection and we know that it is not always easy to flip through the results. We ask for your patience and your help!
We ask for your patience and your help!
It is possible to:
browse all registers
browse all pages in one register
see the list of registers with the number of matching pages (not the number of matches in the volume)
see the list of pages with the number of matches
Queries are interpreted, and results are shown, hierarchically. For example, assume a collection is composed of many Volumes and each Volume of many Pages. Then, if the query is issued at the top collection level, the system first shows the Volumes where the query may appear with confidence above the threshold specified. Then if the same query is issued for a specific Volume, the system shows all the Pages where the query may appear with a confidence above the threshold. Finally if it is issued for a specific Page, the system shows locations where the words involved in the query may appear. In the PRHLT interface, default levels of the hierarchy are called: “Home”, “Collection”, “Chapter” and “Page”.
The navigation from one page goes either to the previous or the next one in the same volume, or back to a higher level (volume or corpus). There is no possibility to jump directly to the next match.
To go back to the superior level, please navigate with the breadcrumb.
Miniatures. At each level of the hierarchy (except at the lowest one, the page level), the system shows the elements of this level by means of miniature images, with some information associated; namely,
an identifier of the element,
a bar representing the confidence of the system that the query appears somewhere in this element
the number of elements in the lower level where the query may appear with a confidence above the threshold specified.
By hovering the mouse over the confidence bar, the actual confidence value is shown.
Pan and Zoom: When a (part of a) page image is shown, it can be explored by moving the mouse while holding its right button. In addition, the mouse wheel can be used to zoom in and out.
If a word appears more than once in a text line, only the instance with greatest confidence is shown.
The approximate locations of spotted queries in each page are marked with rectangles (called “bounding boxes”) surrounding the corresponding words. The color of a rectangle expresses the degree of confidence the system has in the corresponding spot. Exact confidence values can be seen by hovering the mouse over the rectangles.
Your feedback is needed!
YOU CAN GIVE YOUR FEEDBACK
On the spotted words
When you see the results of a query a “word level”, you can click on the word and you are asked if the word is correctly spotted. Please help us and say “correct” or “incorrect”.
If you see a missed word
In the example below, the word “pedagiis” has been correctly spotted on the lower right part of the image.
If you double-click on the image, you can add the word to our index!
Hints for the Chancery collection
Click on the Chancery miniature with no query to see the 147 Volumes indexed, each showing the number of images it contains (ranging from less then 100 to more than 1000).
Queries can be formulated at any level: full Collection (Chancery), specific Chapter (Volume JJ…), or individual Page.
Remember to use the Confidence and/or Max.results levers depending on the precision-recall tradeoff you wish to achieve.
The number of the “page” indicated is the number of the file.
There are 147 volumes; 32 additionnal volumes are being digitized by the French National Archive
In the near future…
This is a presentation of the Beta Version. In a near future, you will be able to scroll through the results and provide feedback directly on a hit list of the results.
You will be able to get the correct foliation of each image and retrieve metadata from the inventories and text editions.
As mentioned in previous posts, we are dealing with plenty of inventories in the HIMANIS project: handwritten systematic inventories, printed ones, cardboard files, geographic inventories, indexes, etc.
Now, the issue we have, is of the coherence of the formats we are using. The text edition we use as a training data set and ground truth, namely the Actes royaux du Poitou edited by Paul Guérin and converted by the Ecole nationales des Chartes, is in TEI-P5 format. There are twelve independent TEI files. This is normal because it is a text edition.
But what would you do with inventories? Some are plain text, some are in EAD format. We could convert them to TEI and consider that these inventories are, each of them, one “text”, so that we could convert each of them to a single TEI file, using the <body> to render the core of the inventory.
For the systematic inventories, we also could consider that each of the inventories contains “manuscript descriptions”: each register can be rendered within a <msDesc> and each text is a <msItem> within the <msContents> part.
The volumes which are copied the one from the other (such as JJ 42B and JJ41) makes it difficult to keep the same structure, except if you want to multiply the descriptions, with one <msDesc> for each of the registers and each <msItem> having the same content but the respective foliation.
In HIMANIS, we are making a new choice: considering each volume as a <TEI> entity (combining them thanks to <teiCorpus>), whose text gathers the <group> of all contained royal charters, each of which being a <text>. The abstract and analysis remains in the <front> part, with an empty <body>.
This solution is, obviously, borderline, since we end up having no text in the whole TEI document, but it is an opening towards a future in which the text could be added as if it where a scholarly digital edition. On the top of that, of course, it allows us to homogenize all source formats, including the text editions, at the (almost) the same level towards an integrated environment. (Not exactly the same level, since the <TEI> file for Guérin is one volume of the edition, not one volume of the register).
For all comments of the community, we are grateful!
On December 5, during our second plenar meeting, held in Valencia (Spain), we discussed (among other topics) various issues that we have encountered with the inventories of the French royal chancery registers. Until the HIMANIS project, those were the only tools available to find particular information in those registers. They exist in different formats: printed or handwritten; some are already converted in EAD (in different flavours), but, despite the efforts of the Archives Nationales, not all available for the general public yet.
As we intend to use these inventories in the project in order to collect correct identifications for place and person names, as well as to evaluate the precision/recall of indexing, we need to have them in a interoperable electronic format (TEI / EAD).
The oldest inventory which we will use, and the only one covering the entire corpus, is the three volume manuscript inventory made in the nineteenth century by A. Longnon and A. Coulon, providing very short Latin summaries for each document. This inventory was retyped by the Archives Nationales and converted in the EAD format.
Modern systematic inventories elaborated by the Archives Nationales come in different formats: printed, typed or are still at manuscript stage (Registres du Trésor des Chartes, Inventaires analytique, 6 vol. in 3 t.).
T. I: Philippe IV (1300-1314) [JJ 37 – 50]
OCR by AN; EAD with subjects index coded
Index (place names + persons) : TEI by IRHT
T. II: Fils de Philippe le Bel (1314-1328)
T.II.1 : Louis X, Philippe V (1314-1321) [JJ 50 – 60]: EAD by AN
T.II.2 : Charles IV (1322-1328) [JJ 61 – 64]: TEI by IRHT
No index existing
T. III: Philippe de Valois (1328-1349)
T. III.1 1328-1349 [JJ 65A – 69]: OCR by IRHT; TEI
T. III.2 1336-1346 [JJ 70 – 75]: OCR by AN; EAD
T. III.3 1339-1350 [JJ 76 – 79B]: OCR by AN; EAD
Index (subjects, person, place names, chancery notes): OCR by IRHT; TEI
Typed unpublished inventories have been retyped by the Archives Nationales (AN) :
Jean II and Charles Dauphin
JJ 80 – JJ 81: OCR by AN; plain text converted to TEI
JJ 83: OCR by AN; .xls to be converted to TEI
JJ 82 – 95: Ms. Index on cardboard files (persons and subjects) about 30.000 files; re-typed by AN
JJ 96 – 97: Ms. Index on cardboard files (persons, place names, subject): 4.500 files; retyped by AN
Charles V (1364-1366)rc
JJ 96 to JJ 98: abstracts and index in .xls to be converted to TEI
Last but not least, our main concern were with the 12 inventories that were still in handwritten stage (JJ84-95 or about 1400 pages). Those where remained in the manuscript (JJ84-95 for the reign of Jean II and a part of Charles V). We therefore asked our partner A2ia if they can perform HTR on those. But how did we proceed to achieve this task?
First, the AN provided us with high resolution reproduction of those 12 inventories, or 1429 pages, at 300 dpi in tiff format. We then produce ground truth, which consisted of the 4 first pages of each inventories (1079 lines of text). Then A2ia trained and performed HTR. We were lucky as the inventory have only one hand and as the layout is very stable from one inventories to the other. Using a standard French model and lexicon from the printed inventories, this operation was able to be carried out quickly in less then 4 weeks and produced results that went beyond our expectations with a very high recognition rate and transcript almost usable as is.
During the colloquium L’acte original, entre conceptions médiévales et concept diplomatique (Nancy, 6-7 oct. 2016), we presented a paper about the practice of “insertion”, which consists in transcribing an earlier document into the body of another act. This practice is used, according to the Vocabulaire international de diplomatique (International vocabulary of diplomatics), either to give legal support to this act (for example, by transcribing the powers of prosecutors or commissions), or to confer authenticity to the transcribed act, known as “vidimus”. This double practice is massive in the registers of the French Royal Chancery, going as far as to produce vidimus of vidimus.
Using the materials accumulated so far in the HIMANIS project, encompassing several text editions and all available inventories (despite the differences in structure), we quantified this practice and documented its evolution from the reign of Philip IV (1286-1314) until Louis XI (1461-1483) and were able to show diachronic variation (with a maximum under the reigns of Philip IV’s sons and a sharp decrease under Charles V and his successors) as well as geographic differences.
We also studied the diplomatic features (beneficiaries etc.) and indications of collation (becoming quasi systematic under Philip VI) and mise en texte within the registers.
During a workshop at the University of Saint-Quentin-en-Yvelines on digital text editions, we had the pleasure to make a provocative pun. Scholarly Digital Editions, a concept largely defined by our friends James Cummings and Franz Fischer (cf. their “Criteria for Reviewing Scholarly Digital Editions” online : http://www.i-d-e.de/publikationen/weitereschriften/criteria-version-1-1/) and Elena Pierazzo (cf. her book DIgital Scholarly Editing: Theories, Models, Methods, available on HAL-SHS), are not only thought for the Humanists, but also for the machines. Indeed the better the edition is, the better it is to teach computer to read reading skills on medieval documents to beginners, i.e. Computers. We demonstrated that and how electronic editions can have a second life as a teacher for Artificial Intelligences, far from their original intent. In fact, our “ground-truth” is mostly based on the electronic edition by Ecole de Chartes of chancery documents published by Paul Guerin more than a hundred years ago: but to be good teachers, they need to be enhanced and upgraded to a more elaborate scholarly level.
Of course, this is not all. They also serve, along with indexes and inventories, to measure the accuracy and recall, and we all have to think about what it means to index (and not to edit) for the remnant parts and what conclusions we can draw from uncertain questions. That was the rest of our talk 😉
HIstorical MANuscript Indexing for user-controlled Search