Plain text access to two centuries of medieval registers (French royal chancery)

Artificial Intelligence has unlocked the access to the text of medieval manuscripts! The partners of the European research project HIMANIS implemented, for the first time, the indexing and plain text querying of thousands of pages of medieval manuscripts. The large scale of the corpus and the possibility to search in plain text for handwritten sources are unheard of in medieval studies, so that the results present a major shift for historians. The challenge of multilingualism, script variation and abbreviations, which are crucial for HTR on medieval sources, has been successfully met.

Overview presentation: Dominique Stutzmann, Christopher Kermorvant, Enrique Vidal, Sukalpa Chanda, Sébastien Hamel, Joan Puigcerver Pérez , Lambert Schomaker, Alejandro H. Toselli, “Handwritten Text Recognition, Keyword Indexing, and Plain Text Search in Medieval Manuscripts”, Digital Humanities 2018 Puentes-Bridges: Book of Abstracts Libro de resúmenes (Mexico City, 26-29 June 2018), Mexico, 2018, p. 298-302, PDF en ligne [http://dh2018.adho.org/wp-content/uploads/2018/06/dh2018_abstracts.pdf], HTML [https://dh2018.adho.org/en/handwritten-text-recognition-keyword-indexing-and-plain-text-search-in-medieval-manuscripts/], XML [https://dh2018.adho.org/wp-content/uploads/2018/05/STUTZMANN_Dominique_Handwritten_Text_Recognition__Keyword_In.xml]

Follow us on Twitter: @himanis6
Search engine: http://bit.ly/himanis-search
User manual: https://himanis.hypotheses.org/105
Official website: https://himanis.org

Himanis users: an emerging community!

The HIMANIS web presence is complex: the present weblog https://himanis.hypotheses.org/ , the beta version of the search engine provided by Univ. Politecnica Valencia http://prhlt-kws.prhlt.upv.es/himanis/, the twitter account (@Himanis6), the official website https://himanis.org/

Some statistics show that we are having an emerging, large community of users.  A growing number of people are reading this blog: ca. 12’500 unique visitors in 2017, with more than 34’000 pages visited; 7’400 and 30’000 pages visited for the first six months of 2018.

There are also 301 followers on Twitter, which is a good start.

On MONK, there is very intense use, with amazing statistics (more than 1 million actions, partly from bots).

Last but not least, on the public search engine, there have been:

  • more than 813’000 search actions: around 40’000 queries per month from Oct. 2017 to May 2018, jumping at 495’455 queries in June 2018 (hurray!)
  • more than 15’000 user feedback actions, from all over the world (see map below)
Georgraphic origin of the users who participate in crowdsourcing the corpus
Statistiques de consultation du blog Himanis en 2017
Statistiques de consultation du blog Himanis de janvier à juin 2018

 

HIMANIS Project New Conference on 29 May 2018 at the Archives Nationales

A second conference from the project HIMANIS (Historical Manuscripts for user-controlled Search), about handwriting recognition by computers of the ancient writings, is held on Tuesday, May 29, 2018 at the Archives Nationales (Paris site – Hôtel de Soubise, chambre du Prince).

Since 2015, the Archives Nationales became an associated partner with the IRHT-led HIMANIS (Historical Manuscripts for User-controlled Search) research project, which goals were to study the records of the French Royal Chancery of the 14th and 15th centuries. The major technological challenge was to carry out computer recognition of medieval writing that was sufficiently reliable to provide the community with a full-text search engine of the registers, many of which were not yet covered by inventories or index. A web interface, combining access to images, transcripts and archival analyzes, has also been developed.

Continue reading HIMANIS Project New Conference on 29 May 2018 at the Archives Nationales

Medieval Music, episode 3

Suite de mes recherches… Dans de futurs billets, nous parlerons de jongleurs. Pour l’instant, nous continuons avec les ménestrels.  Pour l’instant, aucune vérification n’est faite pour savoir combien sont connus aux historiens de la musique et combien manquent. Ce travail viendra plus tard encore…


Source : Arras, BM, 0863 (1043), f. 149v
(http://initiale.irht.cnrs.fr/decor/44017)

Voici nos petites découvertes à partir de JJ 100 :

  • “Denisot Le Feve (?), menestrier de …”, en octobre 1369 (JJ 100, f. 165r, n° 542);
  • “Jehannin Dorgeret, menestrel de haux instrumens” et ses camarades, en mai 1374 (JJ 105, f. 182r, n° 337)
    • Le jour de la Chandeleur derreniere passee, aprés ce que ledit Jehannin et Oudinet du Gué, menestrel desdiz instrumens, ores [sic] souspé en l’ostel du Veau devant Saint Merry avec Jehannin, filz du maire de jonoie, Jehannin Proudomme, Perrin Chevalon, Rogerin Le Ligueur, et Oudinet le Couroier, et que yceulx menestrelz orent joué desdiz instrumens, ilz se feussent partis dudit hostel avec une lanterne et une chandelle…

  • à Senarpont, “une feste es halles d’icelle ville, en laquelle feste avoit un menestrel jouuant d’une cornemuse” en juillet 1374 (AN JJ 105, f. 231r, n° 452) et (AN JJ 106, f. , 154v, n° 191) ;

Continue reading Medieval Music, episode 3

More Medieval Music (AN JJ 87 – JJ 99) / Encore plus de musique médiévale : les ménestrels des registres de la chancellerie AN JJ 87 – JJ 99

Dijon BM ms 493, fol. 15v (bvmm.irht.cnrs.fr/consult/consult.php?VUE_ID=1252786)

Voici la suite de nos ménestrels, en passant à des registres non dépouillés (retrouver l’épisode 1 ici). Au passage, rappelons que Menestrel est un site de ressource capital pour les médiévistes) :

Charles etc. Savoir faisons a touz presens et avenir que oye hymble supplicacion de Colin Carle, escuier, contenant que comme environ la Toussains l’an LVII il feust venuz en la ville de Pons Sainte Maixance pour soy jouer et esbatre ; et illeuc eust trouvé un homme estrange du païs de Haynau qui estoit devant le moustier de ladicte ville aprés souper, lequel homme avoit un menestre avé soy qui jouoit de la cornemuse et illeuc se feust embatu ledit suppliant pour dancer et jouer sicomme jeunes le font volentiers ; et incontinent que ledit homme estrange aperçut ledit suppliant, il dist audit menestre que il laissast le jouer ; et quant ledit suppliant vit cesser ledit menestre de jouer, il ala par devers lui et li demanda pourquoy il avoit laissié a jouer ; lequel menestre respondi que le dessusdit homme li avoit deffendu ; pour quoy ledit suppliant, esmeu de chaleur, cuidant que il le faisoit en despit de lui, prist la cornemuse dudit menestre et la geta a terre ; et ce fait ledit homme estrange survint en la place et donna une buffe

Continue reading More Medieval Music (AN JJ 87 – JJ 99) / Encore plus de musique médiévale : les ménestrels des registres de la chancellerie AN JJ 87 – JJ 99

Some Medieval Music for a Happy New Year \(^o^)/ Un peu de musique pour la nouvelle année

Nous vous souhaitons une très belle et heureuse année 2018, pleine de découvertes et de recherches fructueuses ! Nous espérons que vous trouverez grâce à Himanis des informations utiles et surprenantes 😀

A l’occasion d’une discussion avec Isabelle Ragnard, nous avons cherché quelques informations nouvelles sur les musiciens au Moyen Âge. Grâce à HIMANIS, de nombreuses sources nouvelles sont apparues en un rien de temps .

Quelques histoires de cornemuse (d’où l’illustration) : en septembre 1360 (JJ 88, fol. 63r), le 11 juillet 1374 (JJ 105, fol. 230v, n° 452), le 12 décembre 1375 (JJ 107, fol. 188r), en  juillet 1385 (JJ 127, fol. 30r).

Continue reading Some Medieval Music for a Happy New Year \(^o^)/ Un peu de musique pour la nouvelle année

Nouveau corpus, nouvelles heures

Vingt registres supplémentaires ont été intégrés à l’interface de recherche. Parmi les nouvelles recherches possibles, nous trouvons une nouvelle preuve de la présence quotidienne des livres d’heures.

Paris, Arch. nat., JJ 179, fol. 81v, n° 150

A la Saint-Martin, le 11 novembre, 1447, Raoul est trouvé noyé dans un estang. Suicide ou accident ? Ses parents demandent grâce au roi pour que le doute ne jette pas le déshonneur sur la famille et n’empêche pas l’application des règles normales d’héritage. Que reste-t-il dudit Raoul au bord de l’étang ? Son livre d’heures, son manteau, ses chaussures et son chapeau (« et ses heures, mantel, patins et chappel estans sur ladicte chausee pres de la bonde dudit estang ») !
Revenant du Mexique, je souhaite à tous une bonne fête des Morts…

Concluding conference of the HIMANIS project on 27 October 2017

To celebrate the achievements and the end of the HIMANIS project, the Institut de Recherche et d’Histoire des Textes (IRHT) is organizng a conference on Friday, 27 October 2017, to present, explain and discuss the impact of handwriting recognition technologies for medieval studies and heritage institutions.

The European project HIMANIS (HIstorical MANuscript Indexing for user-controlled search), led by the IRHT, brought together several research teams in engineering sciences, both private (A2iA, France), and public (PRHLT-UPVLC, Spain and Alice-Groningen Univ., The Netherlands), with the support of the National Archives and the National Library of France.

The main result of this project is that it is now possible to carry out keyword searches in the corpus of registers of the French royal chancery of the 14th and 15th centuries. For the moment there are 167 volumes of the National Archives series, JJ 7 – JJ 209 and 20 volumes of the Bibliothèque nationale de France. The interface (still preliminary) can be accessed at the following address: http://prhlt-kws.prhlt.upv.es/himanis/. It is a test interface, which will evolve in the coming months, with additional manuscripts and new functionalities, notably to
navigate between the results, to display the regests and editions. HIMANIS also aims at computer-assisted analysis of the different hands involved in each volume, to study the functioning of the royal chancellery and the progressive construction of the centralized state of the French monarchy.

Entry is free but registration is required: Registration form.

Continue reading Concluding conference of the HIMANIS project on 27 October 2017

The Royal Highness / L’altesse royale / regalis celsitudo : une thématique de préambule pour mesurer l’utilité d’HIMANIS

Il y a quelques années, deux de mes amis ont publié une somme magistrale sur les préambules des rois de France, étudiant la production des actes royaux et ses acteurs au regard des actions juridiques et des thématiques et du style rhétorique des préambules : Barret, Sébastien, et Benoît Grévin. Regalis excellentia :  les préambules des actes des rois de France au XIVe siècle, 1300-1380. Paris: École des chartes, 2014 (Mémoires et documents de l’École des chartes, 98), ci-après abrégé « Barret-Grévin ».

La finesse de ce travail, sa complétude et son existence même, preuve que les méthodes les plus austères de l’érudition ouvrent des vues riches et chatoyantes sur les pratiques intellectuelles et l’histoire culturelle.

Pour ce billet, c’est principalement la partie documentaire de l’ouvrage qui sera utilisée, afin de mesurer l’utilité des recherches et développements faits dans le projet HIMANIS, même si je tenterai, in fine, quelques remarques sur les préambules eux-mêmes.

Méthodologie : performance, précision, rappel et résultats

Après l’énorme travail d’indexation automatique, il faut évaluer la performance de l’indexation.

Continue reading The Royal Highness / L’altesse royale / regalis celsitudo : une thématique de préambule pour mesurer l’utilité d’HIMANIS

Job: Research Fellow position for the HORAE project / Ingénieur de recherche pour le projet HORAE

Job description / Fiche de poste (pdf) :

IRHT-HORAE-ResearchFellow-Ingénieur

I got a job for you… (Paris, Bibliothèque nationale de France, Dép. MSS, Rothschild 2535 (24 b)

English version: Research Fellow / Ingénieur de recherche
    Position focus
    Required Education, Skills and Experience
    About the project HORAE
    Host institution
    Salary
    How to Apply

Version française: Ingénieur de recherche en analyse de sources
    Missions
    Compétences
    Le projet HORAE
    Institution d’accueil
    Rémunération
    Candidature

 

Research Fellow / Ingénieur de recherche en analyse de sources

Institution: Institut de Recherche et d’Histoire des Textes

Location: Paris, France

Application Due: 31 Oct. 2017

Type: Full Time

Duration: 1 year, renewable until 3 years

Starting Date: between January 1, 2018 and April 1, 2018

 

The Institut de Recherche et d’Histoire des Textes (CNRS) invites applications for a Research Fellow (“Ingénieur(e) de recherché en analyse de sources”) for the project “HORAE – Hours, Analysis, Editions” (ANR-17-CE38-0008), under the supervision of Mr. Dominique Stutzmann, Principal Investigator.

Continue reading Job: Research Fellow position for the HORAE project / Ingénieur de recherche pour le projet HORAE

Saint-Omer in the registers of the French royal chancery (99.6% precision!)

As some of the readers will know, I am particularly interested in the city of Saint-Omer in Northern France. This is a good reason to test and search for the first time unknown occurrences of this city, its city officials and chapter officials in the registers of the French royal chancery.

As a matter of fact, the results are quite impressive…

Registrum factum in viagio sancti Audomari (Paris, Archives Nationales, JJ 88)

 

In total, 173 occurrences are found for “audomarum” OR “audomari” or “audomaro” and 73 occurrences for “saint omer”. I noticed bypassing that “osmer” is also present in the registers, but not in our index (generally silently regularized to “omer”).
Continue reading Saint-Omer in the registers of the French royal chancery (99.6% precision!)

“disant ses heures” : from HIMANIS to HORAE

We are delighted to announce that the French research agency is funding the HORAE project (ANR-17-CE38-0008). HIMANIS also highlights why it is important with one example:

Disant ses heures… (Paris, Archives Nationales, JJ44, fol. 98r)

un certain lieu ou il estoit alé disant ses heures

(translation: a place where he went saying his Hours; image Paris, Archives Nationales, JJ44, fol. 98r)

In another example, the book of hours of a Jehannette, daughter of Jehan Petit is central to the plot, since it is the cause of the crime! Used to force a date with the young Jehannette, it created a fight between the father and the daughter’s friend…

Thomas, par esbatement print les heures de ladite Jehannette qui estoient sur une fenestre dudit hostel en la presence de ladite Jehannette et de son consentement les emporta et dist a ycelle Jehannette qu’elle les alast querir a vespres en l’eglise, laquelle lui respondi que si feroit elle. Et tantost aprés, ledit Jehan Petit ala aprés ledit Thomas ou cymetiere de ladite eglise ou il estoit et de felon courage lui dist qu’il lui rendist ses heures, ou si non qu’il l’en poursuiroit et que faulsement et mauvaisement il les lui avoit emblees.

qu’il lui rendist ses heures (Paris, Archives Nationales, JJ 162)

At the core of the late medieval society…

HORAE (Hours – Recognition, Analysis, Editions) is a cross-disciplinary research project studying religious practices and experiences in the late Middle Ages through the books of hours, the absolute medieval best seller. In a public-private partnership, it gathers three partners in the Humanities and Computer Science: the Institut de Recherche et d’Histoire des Textes (IRHT, UPR 841), the private company TEKLIA and the Laboratoire des Sciences du Numérique de Nantes (LS2N, UMR6004). Books of hours forms, with more than 5000 preserved manuscripts, a vast and crucial ensemble to understand the medieval mindset. Yet, their textual content is very scarcely studied, although the massive production of such a large number of manuscripts is a pivotal cultural and industrial phenomenon to the profound changes in the medieval religious life. Not only a proto-industrial book production arises and the economics of demand shifts to the economics of offer, but also the awakening of the individual and the internalization of faith take places in an increasingly church controlled world. Books of hours are at the same time deluxe objects, implying social staging, and intimate objects, with devotional, memorial and eschatological challenges.

HORAE combines the research and expertise of all three partners in artificial intelligence applied to Computer Vision and image analysis, in Natural Language Processing (NLP), and in book history and religious practices. The research project aims at creating an integrated chain from image treatment to producing new knowledge by placing the end user at the center of the developments, by managing formats, interoperability and long term preservation, but also ergonomics and data visualization.The aims encompass: (1) re-using the many digitized manuscripts, which are online but underused resources; (2) new open source software for Handwritten Text Recognition (HTR); (3) tools for segmentation and plagiarism detection, adapted to the transcription of medieval manuscripts produced by the machine in order to identify the texts in the books of hours; (4) identifying and editing unpublished texts; (5) visualization of manuscript clusters, which share the same textual characteristics in the order of the different parts (Officium Beatae Mariae Virginis, votive offices, suffrages, prayers), but also in the order of textual units to identify the liturgical use; (6) studying the diffusion and circulation of devotional and liturgical texts at the end of the Middle Ages in order to better understand the cultures and faith in the 13th c.-16th c.

With its aims and methods, HORAE profoundly changes the ways of doing research in auxiliary sciences and tackles the challenges of big data. Books of hours have been too scarcely studied until new because they are too numerous, too complex and their text is very repetitive: now, their very number, repetitions and complexities make possible for this project to develop new and efficient technologies and methodologies and gain new knowledge on the Middle Ages.

 

 

Language for acts registration at the French Royal Chancery: a fresh look at the data

 

The simple query of part of the protocol formula [ dei gratia francorum ] in the HIMANIS search engine return results that confirm what we already know about language used by the French Royal Chancery for recording acts.

http://prhlt-kws.prhlt.upv.es/himanis/index.php/ui/chapters/chancery?q=%5B+dei+gratia+francorum+%5D&t=50&feedback=1

Some warning.

  1. The complete formula ([name of the king] dei gratia Francorum rex…) is often skip and replace by “[name of the king] etc.”
  2. As the search engine is not yet lemmatized, distinction is made between “gratia” and “gracia”. To have more precise results, one can add the results of a second query [ dei gracia francorum ]
  3. Some noise might also be introduced by the vidimus copy of an act within an act.

Nevertheless, as from the first register (JJ35) the French Royal Chancery used both Latin and French and that one prevailed over the other at different periods, all major transitions are outlined by this query.

Serge Lusignan, in his 2004 book La langue des rois au Moyen Âge, point out how langage used evolved. From his observations, we can explain the major breaks-up in the following graph that we made from our query (number of query results are in red in the graph; in blue are the total number of acts for each register):

  1. JJ35-49: French is seldom used as language of registration by Philippe IV (5%)
  2. French is a little more used by Philippe IV two elder sons: JJ50-52 Louis X (19%) ; JJ53-60: Philippe V (10%).
  3. JJ53-65A: The French conquest begins slowly under his younger son, Charles IV (11% at the beginning; 26% at the end).
  4. JJ65B-79: Major break-up appears during the reign of Philippe VI. No reason is known, but in October 1330 French overthrow Latin for around 80% of act are registered in French from this date.
  5. JJ80-95: When Jean II become king (JJ80), Latin regain his place (2% of French only) until his capture. During the Regency of the Dolphin Charles (JJ84-95), French progressively take back his ground: from 32 % in 1358 it reaches 75% by the end of 1360. Back from his captivity, Latin is back again, but not with the past glory (21% French).
  6. JJ96-117: When Charles V becomes king in 1364 what today could be named a “Language policy” is put in place. From now, language is determined by the nature of the act recorded: French for remission; Latin for ennoblement, legitimation, Amortization and almost everything Church related. As Remission count for 62% of the registered act during his reign, French became predominant from now on.

99 pilgrimages / 99 “pelerinage(s)”: accuracy and historical research

Good news! We travelled across the registers, looking for pilgrimages… A search for pilgrimages in the registers of the French royal chancery is a fantastic example of what HIMANIS is bringing to the historians and interested citizens.

Search for “pelerinage” or “pelerinages” with a confidence level as low as 0 gives 54 matches with an average confidence of 46.6. This means 54 registers and 96 pages corresponding to the search, and 99 spotted occurrences. We searched in French only and, here, the results are amazingly accurate: All spotted occurrences are correct! And they even include an occurrence in a black-and-white register, scanned from the microfilm… A very good information for further research.

The table below shows the list of occurrences with direct links. Of course, there are some artefacts and in one occurrence, we see a bizarre result: http://prhlt-kws.prhlt.upv.es/himanis/index.php/ui/show/chancery/259/149?q=pelerinage+%7C%7C+pelerinages&feedback=1 in which there are two (red) boxes around the word, which is “pelerignage” but has also been indexed as “pelerinage”. In this case, I have to confess that I have clicked on “correct” to confirm the reading, before realizing that it was, technically speaking, not a correct match 😉
Continue reading 99 pilgrimages / 99 “pelerinage(s)”: accuracy and historical research

NEW : improved fonctionalities for correcting query results

We improve the way to correct the results by adding two new functionalities:

1/ After a query, on the spotted words, when you click on the word you are asked if the word is correctly spotted. Not entirely sure of the result you got, we add a third option (“not sure”)  to “correct” or “incorrect”.

2/ When clicking on a word to see if it has been recognized, you have now a dropdown menu suggestions of the recognized word in order of confidence. You can now just select the correct answer from the list rather then typing it. Of course, if none of the suggested result is good you can still type the correct word manually.

The bounding box surrounding the word now appears and they are in color (green, yellow, red or grey) so you can have an idea of the confidence level of the word you are going to select. And don’t worry if the suggest word in the bounding box have nothing to do with the one you clicked: the system will only take into account the pixel on which you clicked on.

Lemmatized search is on it’s way. We are still testing it on a small sample of the corpus but it will be implement in the coming month.

HIstorical MANuscript Indexing for user-controlled Search