ChatPULI: Enhancement to the first Hungarian conversational model

Yang, Zijian Győző and Bánfi, Ágnes and Dodé, Réka and Ferenczi, Gergő and Földesi, Flóra and Hatvani, Péter and Héja, Enikő and Lengyel, Mariann and Madarász, Gábor and Osváth, Mátyás and Sárossy, Bence and Varga, Kristóf and Váradi, Tamás and Prószéky, Gábor and Ligeti-Nagy, Noémi (2025) ChatPULI: Enhancement to the first Hungarian conversational model. ANNALES MATHEMATICAE ET INFORMATICAE, 61. pp. 261-274. ISSN 1787-6117

Preview

Text 261_274_yang.pdf - Published Version Download (482kB) | Preview

Abstract

This paper presents the development and evaluation of PULILlumiX- Llama-3.1 Chat and PULI Trio Q Chat, the first Hungarian-focused conversational large language models based on the Llama 3.1 and Qwen 2.5 architectures. Extending previous work on Hungarian instruction-following models, we applied continual pre-training on multilingual and Hungarian corpora, followed by supervised fine-tuning on an expanded instruction dataset including Hungarian, English, and Chinese prompts. Our models demonstrate significant performance improvements on Hungarian language understanding benchmarks, as well as on few-shot and zero-shot tasks, compared to earlier PULI models. Additionally, they show enhanced capabilities in machine translation and multi-turn dialogue handling. These results highlight the effectiveness of continual pre-training and fine-tuning strategies for adapting large language models to low-resource languages like Hungarian, and provide a foundation for future research in conversational AI for underrepresented languages.

Actions (login required)

Edit Item