Invited talk – Struggles of evaluating LLMs beyond English and Anglo-Saxon culture by Assistant Professor Jeremy Barnes

In this talk, Assistant Professor Jeremy Barnes will describe the ongoing efforts in the HiTZ group (University of the Basque Country) and their collaborators, to create a comprehensive evaluation framework for models in Iberian languages: Basque, Galician, Catalan, Portuguese, and Spanish, among others. He will first outline the common best practices in the evaluation of LLMs, how these need to be adapted to new languages and cultures, and discuss common pitfalls.