LLMs and Medieval Charters

DiDip invests time into creating machine learning models for medieval charters. Since the start of the project, machine learning has come out of currency being replaced by “artificial intelligence” (AI). This AI term covers the emerging capabilities of language generation models trained on huge data sets. This “generative AI”, in the general public often called “chatGPT” (and recently increasingly “Claude”, much less “mistral”), is capable of things for which it wasn’t primarily designed: Text prediction can, for instance, be prompted to analyse text, and by this perform NLP tasks. Text prediction can be used to create analogue texts in a different language, and by this be used for translation tasks. (Brown et al. 2020)

Recently, generative AI includes also image information and combines this with textual information enable through the visual-language models like Gemini 3 or Llama 4. Prompting a generative AI service to create images from a text description is a fascinating playground, but its use for the study of medieval charters and their content is limited. Using the visual capacities of current LLMs (or better visual LLM or multimodal LLMs) for transcription is much more attractive. Mark Humphries and his colleagues promote this application in their “transcriptionPearl” (Humphries et al. 2025). They have good reasons, because their examples (18./19. century English letters) provide impressive results and they even outperform the ‘classic’, i.e. Transkribus.

How do these technologies perform on medieval documents? I did the ordinary experiment, with no systematic ground truth to evaluate and only reduced prompt engineering. I used abritrarily selected documents: one from the Bavarian Main State Archives (Hochstift Freising Urkunden 270, dated 1351), and one from the archive of St. Gall Abbey (St. Gallen FF.4.E.2, dated 1282).

Autosummary: BayHStA, Hochstift Freising Urkunden 270

I imagined an history scholar with not much palaeographical and diplomatics training who wants to know what is recorded in a document. I asked gemini (3 Flash) with a slightly contextualised prompt: “You are an expert in late medieval German documents from the Bavarian Hauptstaatsarchiv. I need you to read this document and create an abstract of the main legal action recorded.”

I get highly structured information – on persons not mentioned, a century later than the charter text itself claims to be (1451 instead of 1351), on non-existant archival marks on the parchment. But I also get the content structured by parts of the text which diplomatists might call “clauses”: the renounciation, the feudal obligations, the legal guarantee. Gemini returned also a description as “letter patent”. This is not wrong, but of no particular interest in this kind of legal action.

I turned to a more detailed approach, starting with a visual analysis. This was not of much help, but at least gave good descriptions of the external features a requested to identify. Next step would be transcription: “Please provide me full transcription of the document, marking line breaks with `||` and expansions of abbreviations with `{}`. Keep as close as possible to the original orthography” with a rather impressive result

Ich Andre pflegl vnd Amptman ze Is{maning} vergich offentlich mit disem brief fur mich vnd mein eliche hausvrow Margret || mit sampt In fur mich vnd fur vnser baid erben. Das wir vns{er}m genädigen heren pischof albrecht ze freisinge{n}. vn{nd} sine{n} || Nachomen mit dem Gemaurte{n} Turn In d{er} Ringmaur ze Is{maning} den er vns ze rechtem lehen lihen hat. als vns{er} brief || sagent di er vns dar vmb gebn hat. warten sullen hinfür mit geud vn{nd} zwen knecht mit armbroste{n} dar vf sten. wen es im || Notdurftig sind. vn{nd} sulln den selbe{n} turn v{er}kauffen noch in d{er} hain wie d{er} nummen. an vnser vorgen{ant} h{er}rschaft wissn vn{nd} gunst. || ouch daz wir mit vnrechten vn{nd} mit vnbillichen sachen wid d{er} h{err}schaft vn{nd} daz gothus ze freysingen vn{nd} di stat ze Is{maning} || gemainsch{a}ft täten. damit man billich lassen wern sol. vñ wen auir der mit d{er} wärheit an mit erbn lüten v{er}uunden wurdn || so sol d{er} egenant turn d{er} vorgen{an}tn h{err}schaft vñ gothus ledig vñ vultkñ sei. Auch sülln wir In di nider hast ze Is{maning} || mit all{er} zugehörd ob wir si von vns{er}m wegen Innhet{en}. dar vf er vns sibe{n} Jar besucht hat als vns{er} brief sagent. di er vns dar || vmb gebn hat. nach den selbn all{e} pauen vn{nd} antwurten bi alles weichen / wen er vns des amant od{er} vadert mit sine{n} offen || briefen. od{er} sine{n} Nachomen di in d{e}r schuld ze kome genade{n} wen. od{er} dem moren tail daz e capitel ze freisingen. ob d{er} h{er}r || pischof vñ sein erz ze bestalte{n} am d{er} vorgen{an}t h{er}rschaft. ouch soll vns vorgen{ant} h{er}rschaft. In de{n} || tagen Alst österen mit enthuse{n} von d{er} vorgen{an}tn stat best ze Is{maning}. Erer ouch dã. daz wir vns redlich vn{nd} vnbillich sach vn{nd} si || täte{n}. damit mit billich haufung thern vñ danken blibe. so sol di egen{ant} haufung absein. Vn{nd} daz alles stat vñ vnzbrochen || belibe. gebn wir disen offen brief v{er}sigelt mit meind vorgen{ant} Andre Insigel d{er} dar An hanget. d{er} vns vñ bint ich mich || d{er} vorgen{ant} Margret. rüstig ze haltn vñ ze leisten. aller daz da vorgeschribn stat. Gebn ze hneim do man galt von d{er} Cristi || Geburt dreizehenhundert Jar vñ in dem ain vñ funfzigiste{n} Jar. An sant Lucien tag.

The transcription was accompanied by explainations about earlier errors (date) and an explanation of the term “pressel” (there is none in the document, only the cut in the plica where the seal was once attached probably by stripes of parchment, i.e. a Pressel) and the lost seal. The transcription is quite impressive, if you don’t care about the jurisdiction of the issuer (Niklas, bailliff of “Lok” in the text) and the castle assigned. Spelling is less precise, but at least the original task should now improve. I prompeted Gemini to compare the summary with the one available on Monasterium.net itself (“Which errors did you make when including the information from https://www.monasterium.net/mom/DE-BayHStA/HochstiftFreising/Hochstift_Freising_Urkunden_270/charter into your consideration?”). Lots of Wikipedia information kicked in: Bishop Albrecht was identified as Albert II. of Hohenberg. Generic palaeographical knowledge guided the model to avoid decision: “The script is a 14th-century Textualis/Cursiva transition, not a mid-15th-century Bastarda.” The Lehnrevers from the abstract was explained as “formal acknowledgment of feudal tenure and service”.

However, geographically it still remained undecided. Freising had large property in Krain – and “Lok” has taken its modern name from this as “Bischoflack” (now Škofja Loka) in Slovenia. This external information was missing, as obviously the archival context of the document given in the prompt distracted the LLM.

Visual analysis and transcription: St. Gall FF.4.E.2

For the example from St. Gall, I went for the highly uninformed short prompt: “This is a Latin charter from St. Gall from the 1280s. Can you identify on the image (e.g. with an overlay) the areas of the very charter, the seal and the text on the charter?” and used it with GPT 5.1 in the chat interfacce.

Of course, we can be insistent with the bots, so asking more precisely “Redo the task and make sure that you return the full image as submitted.” returned a much better result:

Following up on this with “Transcribe the text of the charter!” and the suggestion by the bot “If you want, I can produce a fully expanded diplomatic transcription with all abbreviations expanded and normalized to modern Latin spelling, which would make it easier to translate and analyze for content. This is often done in charter editions.”, I ended up with the following transcription:

Universis presentes litteras inspecturis frater Adam presbyter et Egidius de sancto Asaphach miles omnipotentis Dei amicos salutem. Noveritis nos per hoc scriptum praesentium audivisse et monachos sancti Galli tam annorum subscriptorum, memorandum, quod ego Ammundtters mihi meis per dioceses prebet fecit beneficium et manutenendo potestatem cella.
Tenens et alios idem assensu et observantia et fidelis promotores exhibitores presentium reddimus ad servitium. Habet terram in pendula quiete illas per illa monachos quos suscipimus et communi in sancti Asaphach terrulo sedis.
Tamen nec concessisse prebet et quiete nec possessiones in supradictis succedentibus et fructus medii redditus. Titulam propriam et firmam et honorem quondam fuerint non perpetua obligata regno et renunciato. Per monachos sancti percipiendum ad manum et alios et omnipotentis prebet concessum et sancti Galli conservare prebet et remittere.
Et cui necessitatibus necessario remissis evidentiis amplificamus prebet monum assensum. Datum apud hanc apud … idem sub anno domini millesimo ducentesimo octuagesimo secundo [1282] die festivitatis …

Sounds nice, doesn’t it? And, yes, it is completely wrong. This did not change even when trying to push the bot into a more meticulous behaviour with a new prompt “Redo your original transcription and look more detailed at the image. Transcribe line by line and make sure you capture all words correctly as they are written on the document. Be meticulous and recheck your results.” The bot returned 22 lines instead of the 11 of the charter …

Doing it with a prompt similar to the one suggested by Mark Humphries the result looks better, and there are several things to be learned when comparing it to the transcription provided in Monasterium.net:

We see that the automatic transcript can be

• more precise, for instance in the invocatio where it uses the abbreviation based X for XPI, distinguishes between V and U by position, e.g. in Vniuersis or reueverendi.
• It can invent and omit text, for instance the final dating clause with the indiction (indictione Xa) is missing, a generic address is added (omnibus fidelibus), or inserting a long arenga replacing the justification related to the monastery.
• It replaces crucial information, for instance 4 lb (quaturo libras usualis monete) with free men (liberos); the resignatio becomes a recognitio.
• It has problems with proper names, for instance converting “Rorschach” to “Aschach”, “Tvfenbach” to “Isenbach”, “Romo” with “Hermann”, “Veltkilch” to “Rorschach” in the dating clause

Agentic AI: DHCraft co-OCR/HTR application

OK, I know that my (former) PhD students are much better than myself in creating systems that make really good use of generative AI solutions. For instance, Christopher Pollin, who is successfully selling this skill in his company DHCraft. He and his colleague Christian Steiner founded DHCraft with a bit of support of our university start-up program UniCorn two years ago. Christopher built a prototype for AI assisted editing: https://dhcraft.org/co-ocr-htr/. I’m pretty proud of his skills, and he is obviously much better in contextualising the transcription task:

Vniuerfis xpı̄ fidelibus preſens ſcriptum īſpecturıs Egelolf⁹ de Erſchbach mıles omīū ıllıus monastıı
sānctı Gallı tam nouıs quā vetııb⁹ [?] Ad notıcıā ōmı̄m dıco fıngloꝝ contextu p̄ſencıū cupıo deue-
nıre quod ego Anımaduētens ı̄mām meā qꝫ p̄deceſſoꝝ meoꝝ [?] facere benefıcıā ā monāstı̄o sāctı Gallı pluꝝes
decıās & alıas edēm ı̄mı̄ ı̄ obſequııſ fıdelıū pmotorııs exhıbıtor p̄ter [?] dıctū feudū quā quod ā dıcto mon
ı̄ hı̄dıtā tenuı̄ ı̄ feudū p̄ı̄dıdı̄ quı̄tūque lıberas popūlııs monete quı̄s anglııs de ōmı̄ m [?] erſchbach tıtulo feudı
[illegible] mıe cōt̄ dıcē pacıfıce & quıete necnō poſſeſſıōem ı̄ [?] ı̄ſenbach

Still, the transcription does not really convince, but at least we have Egelolfus as the issuer, even if his origin is not really fitting. I’m sure, Christopher will be able to explain why the system stopped transcription in line 6 or has already modified the code in a way that the behaviour above cannot be reproduced anymore.

Lessons learned

What can we learn from these examples? Blaming the bot is fun, but not really useful. I recently learned that moral philosophers consider it impossible to blame without suffering – and not even Christopher will suffer, if I discuss the errors with him, as he would correctly argue that his system is not replacing the experts but supporting them, amplifying their capacities – and, of course, he would use the lessons learned to improve his system.

Comparing the results to the even worse results from a completely uncontrolled transcription in the “Wolpi AI assistant” from Transkribus shows one problem to be considered:

Cumusio ꝙī fidelibus prespoa scarpum preuentibꝰ Cꝓololi de di aschech miles onnustialis onovastio
sancti celli alium cum notitia substi ipsi Ad noticiam omnium ac singulorum contenetu presentium atpio deve
mire quod ego animaducens mielii meis quod per decesseribus varia fere beneficia si ministris grati colli pluries
Vensa et alias edem animi in obsequiere et fidelis pro amororis exhibitatem fore assisaum feodum quod a iamdico non
hacke tenui in suad videlicet quatiue libras posualia monete quas singulis de quia in E orschach titulo seod
quam sine contra dictae plafice et quiete necnon possessionem in et igenbach sive redditus et frenatus michi ibidem
tialo pignatis per dominum L’amoniam quondam Altein non seperfati obligatos ipsigno et itemitto episcopi non istario fac
rigusse ad anamus reverendi domini villi eti gratia electi et confirmati sepefati monasterium sancti Galli
conferere presentibus et remisisse. In ejus in esignacionis necnon remissions eiudentiam amplium presentibus meum angustinus
apadi acta sunt hec sepud Velti ilch sonno domini Anno c c LXXXII In die festuitatis Beate ead
me magdalene Indicte

Already the first word of the transcription tells the slightly informed user the problem: The word “Cumusio” does not exist in Latin. Going through the transcription demostrates that Transkribus does the better job than the bots: the errors are much easier to interprete on a visual level. The bots replace the insufficient training in the language and visuals of the charters by generalised text taken from medieval charters. The usual answer of the system is didactically interesting: the bot adds general contextual knowledge on palaeography, prosopography, and terminology, which is helpful for the uninformed. However, precisely the uninformed is rather easy seduced by this and is maybe not really capable of identifying the risks.

At the moment, my main lesson is: it is not worth to extend my carbon footprint by setting up a systematic test, calculating evaluation metrics that I could use to compare them to more traditional methods. However, it calls for DiDip’s work to be prepared for the future: It is probable that we can refactor the task, get models that include more examples from our domain, and models that are more advanced in general, with the end that one day there will be a better experience than the one above. DiDip will hopefully have the benchmarks ready to measure this.

---

PS: The Freising document has been transcribe by Joseph Zahn in the 1870s: https://www.monasterium.net/mom/FreisBm/1352_XII_13/charter

References

• Brown, Tom B., Benjamin Mann, Nick Ryder, u. a. 2020. „Language models are few-shot learners“. Proceedings of the 34th International Conference on Neural Information Processing Systems (Red Hook, NY, USA), NIPS ’20, Dezember 6, 1877–901. https://dl.acm.org/doi/10.5555/3495724.3495883.
• Humphries, Mark, Lianne C. Leddy, Quinn Downton, u. a. 2025. „Unlocking the Archives: Using Large Language Models to Transcribe Handwritten Historical Documents“. Historical Methods: A Journal of Quantitative and Interdisciplinary History 58 (3): 175–93. https://doi.org/10.1080/01615440.2025.2500309.

---

The text only may be used under licence Creative Commons Attribution 4.0 International. All other elements (illustrations, imported files) are “All rights reserved”, unless otherwise stated.

---

---