Author: Halyna Hyryavets

Speaker: Domen Vreš
Affiliation: Faculty of Computer and Information Science, University of Ljubljana
Title of the lecture: Advances in Artificial Intelligence for the Slovenian Language: Training and Scaling the GaMS Large Language Model
Format: Online
Date and Time: 17 September 2025 at 10:00 CEST

About webinar
This webinar will provide an in-depth look at the development of GaMS, a large language model (LLM) tailored for the Slovenian language. We will begin by exploring the general principles of LLM training and explain why this process requires substantial computational resources and data. Participants will learn about the entire workflow, from data preparation through training to model performance evaluation.

We will then focus on scaling LLM training on EuroHPC infrastructure, specifically on the Leonardo supercomputer. This step made it possible to achieve higher performance and quality of the model. Domen Vreš will also share insights into the technical and organisational aspects of collaborating on such a project.

Finally, we will delve into the specific challenges of developing an LLM for a low-resource language such as Slovenian. What makes this process more demanding? How can the limitations of available data be overcome? And what creative solutions did the team apply in the GaMS model?

A dedicated part of the webinar will address the challenges of evaluating LLMs in low-resource environments, where established benchmarks and comprehensive evaluation datasets are often lacking. The speaker will explain how this gap can be bridged through the collection and integration of human feedback, and how such data is used to further improve and fine-tune the model.

Target audience:
The webinar is suitable for artificial intelligence professionals, researchers in natural language processing, as well as students, technology enthusiasts, and anyone interested in the development and applications of large language models in minority languages.

Whether you are an AI expert or simply seeking inspiration, this webinar offers a clear and engaging insight into the groundbreaking work behind the GaMS model and its significance for the future of Slovenian language processing.
Don’t miss the opportunity to participate and gain valuable insights directly from an expert at the University of Ljubljana.

These experiments should demonstrate the benefits of HPC for the broader SME ecosystem in Europe and showcase concrete success stories that can inspire other businesses to adopt advanced computational technologies. The call is particularly suitable for companies that have not yet had experience with HPC but are facing a technical or business challenge that could be addressed through the use of HPC.

Key Information about the Call:

• Proposal submission deadline: August 26, 2025 at 17:00 (Brussels time)
• Expected duration of experiments: maximum 15 months, planned start date: January 1, 2026
• Total call budget: €4 million
• Proposal language: English
• Submission method: exclusively electronic, in accordance with the requirements set out in the document. Announcement and Proposer’s Guide

The call is open to SMEs from countries associated with the Digital Europe Programme. Please note that the eligibility condition was updated on July 18, 2025 – the decisive factor for determining eligibility is the status of a country’s association as of the call deadline (August 26, 2025).

Key points to address in the proposal:

• Identify the business challenge and describe how HPC can address it.
• Present the expected benefits and positive impact on the business.
• Define the specific objectives, partners, and resources required for the implementation of the experiment.
• Address data protection aspects, budget, and implementation plan.
• Commit to creating a case study on the impacts of the experiment.

Each proposal must be prepared in accordance with the structure outlined in the Proposer’s Guide. The use of the official proposal templates (Part A and Part B) is mandatory.

Are you interested in participating but not sure where to start?
Do not hesitate to contact us. We will be happy to provide you with a consultation on the call, help you understand the conditions, verify the eligibility of your project, and connect you with suitable partners or research institutions. We can also assist you in preparing your proposal and provide feedback on its quality before submission.

Our ambition is to support Slovak SMEs in entering the digital space and help them fully leverage the potential of HPC. Working with us can simplify the entire process and increase your chances of success in the call.

Contact us and take part in an initiative that can significantly advance your business.

The meeting was marked by themes of inclusion, openness, and a shared vision of a digital future that leaves no one behind.In the beginning was the Word. And if the future is to belong to everyone, it too must begin with the Word,”echoed during the conversations that went beyond the realm of technology and touched upon the very essence of what a 21st-century society should look like.

Lucia Maličková has long worked at the intersection of technological innovation and societal benefit. In her role at the National Competence Centre for HPC, she focuses on connecting Slovak research institutions, companies, and the public sector with the opportunities offered by supercomputing and artificial intelligence. Her goal is to ensure that high-performance computing is not the privilege of a narrow group of experts, but a tool that serves people – in science, industry, and social innovation.

Ján Bihari, founder of Tibi Digital, is a long-standing activist and expert in bridging the digital divide. With passion and dedication, he develops educational and technological tools aimed at the Roma community – particularly through language, gaming, and artificial intelligence. In his projects, he collaborates with Roma user experience testers, connecting cutting-edge development with authentic life experiences and real community needs.

Together, they explored ways to connect high-performance computing, artificial intelligence, and digital inclusion. Their discussions focused on the possibilities of creating inclusive digital tools that could serve marginalized communities – not merely as a form of assistance, but as a genuine means of empowerment and participation in the digital economy.

Their shared vision is rooted in the belief that technological progress must go hand in hand with social responsibility. Both Maličková and Bihari see meaning in connecting technology with real human stories – in places where artificial intelligence and supercomputing can help bridge divides, level the playing field, and turn prejudice into opportunity.

Bihari’s approach is grounded in a deep belief that technology holds the potential to bring about equality – if harnessed correctly. He develops applications that respond to demographic challenges, such as population aging, while simultaneously creating opportunities for Roma women in the field of caregiving.

Lucia Maličková praised his work as clear evidence that innovation can—and must—be inclusive if it is to have a real and lasting impact.

“We look forward to future collaboration. Because technological progress only truly matters when it is accessible to everyone – and when it transforms lives for the better. Inclusion is not just a goal; it is a fundamental prerequisite for a just society.“

The webinar featured leading experts in the field: Prof. Dr. F. LeRon Shults from NORCE and the University of Agder (Norway), who specializes in cultural cybernetics and modeling extremism, and Dr. Justin Lane, co-founder of CulturePulse AI and Scientific Director of the DEKK Institute, a pioneer in multi-agent AI and the creator of societal digital twins.

The webinar showcased real-world applications of these technologies, including simulations developed for the United Nations, the Norwegian government, and conflict modeling such as the Israel–Palestine crisis. It also introduced the Slovak Digital Twin initiative, which leverages the Devana supercomputer to run large-scale national simulations.

The program also included a discussion on the ethical aspects of using AI and the opportunities for international collaboration.

The webinar was organized by the National Supercomputing Centre Slovakia in collaboration with NCC Norway and NCC Portugal as part of the EuroCC project activities.

Lucia Malíčková encourages anyone interested in WHPC, mentoring, or getting involved in the community to reach out to her. As an active member of the community, she shares her experience, inspires other professionals, and helps foster an inclusive environment in research, technology, and computational science.

This engagement also aligns with the broader celebration of Women’s History Month, which aims to highlight the contributions of women across all sectors — including the technology space, where there is still work to be done to achieve equal representation.

“Being part of this inspiring network of professionals who support diversity and growth in the field of HPC is a meaningful step for me,” Malíčková wrote in her LinkedIn post.

ITER: Research, Innovation, and Sustainability

ITER is one of the most prominent technological institutes in Spain, with a strong focus on renewable energy sources, green technologies, and innovative solutions for a sustainable future. The center plays a key role in bridging science and industry, creating conditions for the development of solutions with significant societal and environmental impact.

Part of this research ecosystem is Teide HPC, one of the most powerful supercomputers in Spain.

Teide HPC: A Symbol of the Union Between Nature and Computational Power

The Teide HPC supercomputer is housed in a seemingly unremarkable building in the industrial area of Tenerife. However, inside lies a technological gem capable of tackling the most demanding scientific and technical computations.

It is named after Mount Teide, the highest peak in Spain, whose symbolism is reflected in the graphic design of the facility itself. This design expresses the connection between the forces of nature and the power of computation – perfectly capturing the mission of the center.

Teide HPC is used not only for research in the field of renewable energy but also supports a wide range of multidisciplinary projects — from bioinformatics and artificial intelligence to simulations in energy, transportation, and climate science.

First Steps Toward Cooperation Between Slovakia and Spain

Lucia Malíčková’s visit represents an important step in strengthening international ties between Slovak and Spanish partners. During discussions with ITER’s leadership, several potential areas of cooperation were identified — including knowledge exchange, participation in joint research projects, expert mobility, and opportunities in education and training for HPC users.

The representative of NSCC and NCC also presented the Slovak high-performance computing ecosystem, as well as the services it offers to Slovak users from the academic, public, and industrial sectors.

An Inspiring Visit with a Vision for the Future

“This visit was incredibly inspiring,” said Lucia Malíčková upon her return. “I’m glad we had the opportunity to establish contact with such an important partner as ITER and Teide HPC. I believe this is the first step toward long-term and fruitful collaboration that can deliver tangible results not only for our institutions but also for the entire European research and innovation community.”

In the coming months, the National Competence Centre for HPC will continue its dialogue and explore concrete forms of collaboration.

Michal Valko currently serves as Chief Models Officer at a stealth startup, is a researcher at the prestigious French institute Inria, and lectures at ENS Paris-Saclay. In 2024, he also became the principal engineer for Llama at Meta, where he is building the online reinforcement learning infrastructure and conducting research for the Llama 3 models.

His research focuses on designing algorithms that can operate with as little human intervention as possible. This includes deep reinforcement learning, bandit algorithms, unsupervised learning, as well as so-called self play — learning by comparing one’s own outcomes. Recently, he has concentrated on data representations, language models, and developing deep learning algorithms with solid theoretical foundations. He is currently working on algorithmic solutions that enable more efficient fine-tuning and better alignment of large language models (LMMs).

Michal Valko comes from Slovakia, where he attended the Alejová Gymnasium in Košice. Looking back, he especially values the close collaboration between this school and the Faculty of Science at UPJŠ, which sparked his interest in science. He went on to study artificial intelligence and mathematical methods in computer science at the Faculty of Mathematics, Physics and Informatics of Comenius University. In 2011, he earned his PhD at the University of Pittsburgh under the supervision of Miloš Hauskrecht. He then completed his postdoctoral studies with Rémi Munos, with whom he later co-founded Google DeepMind Paris.

Throughout his career, he has also worked on projects with companies such as Intel, Adobe, Technicolor, and Microsoft Research. As early as 2009–2010, during an internship at Intel Labs in Silicon Valley, he contributed to the development of autonomous systems designed to help visually impaired people recognize faces.

The National Supercomputing Centre and the National Competence Centre for HPC will continue to actively seek opportunities to connect Slovakia’s HPC infrastructures with global leaders in the field of artificial intelligence. They believe that this meeting will also lay the groundwork for new ambitious initiatives.

The current collaboration focuses on leveraging high-performance computing to analyze texts that shape social and cultural discourse. The aim is to uncover patterns of thought within various religious and societal groups and to explore pathways toward mutual understanding.

Thanks to the capabilities of large language models, we are able to simulate scenarios of social reconciliation. Starting this autumn, students of the Religious and Cultural Mediation program will also actively participate in the research, working directly with the data and learning to interpret its societal impact.

Lucia Malíčková, Milan Urbančok, and Jozef Žuffa recently met at the faculty to discuss opportunities for connecting big data analysis with socio-theological research. This continues our collaboration aimed at gaining deeper insights into phenomena that create tension in today’s society — from questions of identity, the relationship between spirituality and secularism, through migration and ecology, to the issue of truth and disinformation.

The outcome will be interdisciplinary results that aim to enrich not only academic discourse but also broader societal dialogue. In this context, artificial intelligence becomes a tool for understanding — analyzing the language, discourses, and meaning frameworks of different groups, and helping to uncover new ways to foster meaningful social conversations.

In this way, we are cultivating a space where theological reflection, social engagement, and technological innovation meet — all in the service of easing tensions and seeking shared solutions.

Participants were introduced to several innovative approaches aimed at reducing linguistic inequality in the era of artificial intelligence:

• Generation of bilingual datasets: We used a database of professionally edited Slovak books and the LLaMA 3.3 70B Instruct model to translate between Slovak and English, resulting in datasets for training translation models and improving machine-translated Slovak.
• Summarization of scientific texts: Using the Gemini Flash Experimental model and the PLOS database, we generated Slovak summaries of scientific articles, contributing to the development of domain-specific language in large language models (LLMs).
• Enhancing cultural context: We are preparing a dataset based on Slovak sources aimed at improving the models’ ability to understand culturally specific topics and local context.

The webinar was hosted by Marek Dobeš, with Radovan Garabík and Peter Bednár as co-authors of the project. The research is conducted using high-performance computing infrastructure – the Slovak supercomputer Devana and the Italian supercomputer Leonardo, operated by the Cineca supercomputing center in Italy.

The case study highlights the potential for applying these methodologies to other low-resource languages. We believe that the insights gained from this project can inspire experts around the world.

The Technical University of Košice has long been among the leaders in the fields of materials research, metallurgy, and technological innovation. High-performance computing (HPC), provided by NSCC Slovakia, is a key tool for supporting research projects with demanding requirements for simulation, modeling, and data analysis. Connecting these two worlds—research and infrastructure—can open up new opportunities not only for scientists, but also for Slovak industry and education.

„We discussed specific ways to strengthen collaboration between universities and national institutions in the field of high-performance computing (HPC),“ said Lucia Malíčková. „The potential for joint projects, access to computing resources, and support for students in this field are areas where we see significant room for progress.“

The meeting represents a first step toward a long-term connection of infrastructure, expertise, and shared goals. Both parties aim not only to make use of available tools, but also to actively contribute to their development for the benefit of the scientific community and Slovakia’s innovative future.