\n<\/p>\n
The Institute for Language and Speech Processing (ILSP) at the Athena Research Center \u2013 the only institute in Greece that has developed machines capable of answering questions and solving problems by mimicking human intelligence \u2013 has just launched its second large language model (LLM), Llama-Krikri.\u00a0<\/p>\n
Llama-Krikri follows in the footsteps of the institute\u2019s first language model, Meltemi. While it may not rival the scale of DeepSeek or ChatGPT, it performs the same fundamental tasks. A conversation with the team behind it feels like a deep dive into the fascinating world of artificial neural networks, which expertly process and generate human language.<\/p>\n
\n\u2018It\u2019s like setting the language model to sift through your stored data and use it as \u201ccheat sheet\u201d to generate responses\u2019<\/p>\n<\/blockquote>\n
Why the name Llama-Krikri? \u201cIt\u2019s a word association game,\u201d explains Vassilis Katsouros, director of the ILSP at the Athena Research Center, with a smile. \u201cThe name comes from the base model we used \u2013 the open-source Llama 3.1 model from Meta \u2013 combined with the Greek equivalent of the llama, the kri-kri [a type of wild goat native to Crete].\u201d<\/p>\n
Llama-Krikri is significantly smaller in computational scale than DeepSeek, but by how much? \u201cIf DeepSeek\u2019s neural network has 671 billion parameters, of which 37 billion are activated based on the user\u2019s query, our model has only 8 billion parameters,\u201d explains Katsouros. To put it simply, by translating the Greek language into numbers \u2013 since computers process numbers, not words \u2013 Llama-Krikri represents it using 8 billion numerical values. \u201cIt\u2019s a different scale of investment,\u201d he adds. \u201cHigh-end language models require vast amounts of data, immense computational power, and significant financial resources.\u201d<\/p>\n
Digital aides<\/h3>\n
Llama-Krikri accomplishes everything that Meltemi did \u2013 and does a much better job. What exactly did Meltemi do? It empowered the Greek academic, research and business communities to build their own digital assistants. \u201cWhile it doesn\u2019t provide a service for answering direct queries from the general public, it does allow anyone with some programming knowledge to customize it to their needs, submit queries and receive answers based on their own data,\u201d explains Sokratis Sofianopoulos, who, together with Prokopis Prokopidis, Georgios Paraskevopoulos, Leon Voukoutis and Dimitris Roussis \u2013 all scientific associates at ILSP \u2013 developed Llama-Krikri.<\/p>\n
\u201cIt\u2019s like setting the language model to sift through your stored data and use it as \u2018cheat sheet\u2019 to generate responses,\u201d he adds. And these responses come in fluent modern Greek as well as in English. \u201cMeltemi made a huge impact \u2013 tens of thousands of downloads on Hugging Face, a platform where AI developers share code. It was downloaded by students and academics, used in countless applications and research projects, boosted research activity in Greece and increased productivity in small and medium-sized enterprises. It offered an affordable solution for rapid project execution,\u201d says Sofianopoulos. In addition, the institute has supported public organizations and businesses in building their own digital assistants. \u00a0\u201cWithin the framework of the European Digital Innovation Hubs (EDIHs), we work with companies that request customized adaptations of our language model to suit their specific needs,\u201d adds Katsouros.<\/p>\n
So, what improvements does Llama-Krikri bring? \u201cIt has a much stronger memory \u2013 it doesn\u2019t lose track during lengthy conversations. While the previous model could remember up to 8,000 words and sustain a dialogue for only a few minutes, Llama-Krikri can remember up to 128,000 words \u2013 an entire 250-page book \u2013 and can handle a sequence of multiple questions without forgetting what was said at the beginning. It can process larger volumes of text to generate its responses. It produces higher-quality text; it writes in the polytonic system, can compose in Ancient Greek, and translates into Modern Greek \u2013 something the previous model wasn\u2019t trained to do. It also writes better code, as models aren\u2019t limited to natural language \u2013 they also write computer languages. Additionally, it solves mathematical problems more accurately and exhibits stronger reasoning. In other words, it can perform complex tasks that require several intermediate steps of logical thinking,\u201d explains Katsouros.<\/p>\n
At the same time, the research team is creating and publishing a wealth of linguistic resources, including datasets that translate English language data into Greek. This effort allows for a thorough evaluation of Greek language model performance, and all of this is made openly available. As Maria Giagkou, a scientific associate at the institute, says, \u201cMeltemi and Llama-Krikri are open-source models, meaning that any researcher or industry professional can freely examine and customize their workings. This is the power of open source. In contrast, OpenAI\u2019s model is closed \u2013 you get what it gives you, with no possibility for tailoring to your specific needs.\u201d<\/p>\n
Katsouros further explains that the Greek ecosystem of small and medium-sized enterprises \u201cneeds technology in Greek that caters to its unique requirements and financial capabilities.\u201d Moreover, as Sofianopoulos adds, a Greek company doesn\u2019t need ChatGPT to support its services. It needs a smaller model \u2013 or perhaps even a second or third one \u2013 on which to assign specific tasks, all managed internally to avoid the risk of data leaks and large fees charged by OpenAI. This is especially important when services are provided via subscription, which is essential to ensure security. In such models, the cost is based on the number of word segments generated by each query and response. The more segments a word is broken into, the higher the cost. OpenAI and Anthropic often decompose each Greek word into multiple subwords \u2013 sometimes even character by character \u2013 thus generating Greek text by piecing these fragments together. \u201cIn our approach, we break down Greek words into the largest possible segments, significantly reducing the computational cost. On average, we split a Greek word into 1.5 segments, whereas other models break them into 2.5, 6, or even 7 segments,\u201d he explains.<\/p>\n
Cultural diet<\/h3>\n
But these are not the only reasons we need Greek language models. \u201cOur language carries our culture and shapes the way we think and communicate in Greek. You need a service that, when you ask a culturally specific question, say, \u2018Recommend a healthy breakfast,\u2019 responds appropriately rather than suggesting something like \u2018beans with bacon,\u2019 which is a British concept of breakfast. It\u2019s crucial to train large language models \u2013 and all language technologies \u2013 on texts originally written in Greek rather than on translations from other languages,\u201d explains Giagkou. \u201cIt is also a matter of digital survival for our language. We know that unwritten languages can disappear; today, languages without a digital presence will also vanish. For a language to survive, it must have a digital footprint, which means being supported by language technologies, including large language models. And because we are a very small market, large companies show little commercial interest in our language. If we don\u2019t invest in Greek language technologies, no one else will.\u201d<\/p>\n
A European DeepSeek?<\/h3>\n
Could Europe develop a DeepSeek? Katsouros responds: \u201cIt certainly could. DeepSeek utilized computational techniques that are well known in the research community. Similar methods have been adopted by the French company Mistral AI, one of Europe\u2019s leading firms in language model development. What\u2019s needed is more experimentation, continuous training, access to resources, hard work and constant adaptation to the latest technological advancements.<\/p>\n
With open-source models, the effort is shared \u2013 the French build on the work of the Chinese, the Greeks build on the French, and so on \u2013 so a major European language model could indeed emerge and make a significant impact. However, Europe must remain actively involved and develop a more balanced regulatory framework for data copyrights \u2013 perhaps allowing 10% or 20% free use, similar to allowances made for educational purposes. [French President Emmanuel] Macron once observed that in Europe, we regulate too much and innovate too little.\u201d<\/p>\n
\u201cRight now, a European alliance for language technologies has been formed, with 17 countries participating so far. The goal is to collect linguistic data and develop new large-scale language models. Additionally, two major European projects have been funded to create a multilingual model that will support all official European languages equally, in keeping with Europe\u2019s multilingual policy,\u201d says Giagkou.<\/p>\n
Sofianopoulos is less optimistic, reminding us that Europe has not yet developed entirely new architectures for large language models \u2013 at least not on a scale that can compete with initiatives from the US and China. One initiative addressing part of this challenge is Pharos, \u201cthe Greek AI factory,\u201d a project involving multiple domestic research and academic organizations. Key partners include the National Infrastructures for Research and Technology (GRNET), the Demokritos Center for Scientific Research, the National Technical University of Athens (NTUA), Athena RC and the Hellenic Corporation of Assets and Participations. Pharos will operate using computational resources from the high-performance computer (HPC) Daedalus.\u00a0<\/p>\n
\u201cThe more people that get involved, the faster we will see innovative results,\u201d Katsouros concludes.<\/p>\n<\/p><\/div>\n