Teaching Exotic Programming Languages to Large Language Models

Speaker: Alessandro Giagnorio, Università della Svizzera italiana

Abstract: Large Language Models (LLMs) are transforming the landscape of software development, with some commercial models now capable of generating entire projects from scratch based on simple natural language descriptions of the desired functionalities. While LLMs are becoming increasingly proficient at generating code in popular programming languages, such as Python or Java, they still face challenges when it comes to uncommon languages, typically developed in-house by companies to address specific domain needs. Such languages are often not represented in the training data of LLMs, leading to suboptimal performance in code generation tasks and thus being ‘exotic’ for these models. In this seminar, I will present my recent research advancements in this area, showcasing the current state of LLMs’ performance on uncommon programming languages and the benefits they could achieve using popular techniques such as few-shot learning, Retrieval-Augmented Generation (RAG), and various training approaches. Additionally, I will discuss potential alternatives and solutions for achieving better results using publicly available open-weight models.

Biography: Alessandro Giagnorio is a Ph.D. student in the Faculty of Informatics at the Università della Svizzera italiana (USI), Switzerland, where he is part of the SoftwarE Analytics Research Team (SEART). He received his double Master’s degree in Software System Security (Università degli Studi del Molise, Italy), and Software and Data Engineering (Università della Svizzera italiana, Switzerland) in July 2023. He is currently researching new methodologies for customizing deep-learning models to support developers on code-related tasks.

Chair: Joey Bevilacqua

*************************

In February 2019, the Software Institute started its SI Seminar Series. Every Thursday afternoon, a researcher of the Institute will publicly give a short talk on a software engineering argument of their choice. Examples include, but are not limited to novel interesting papers, seminal papers, personal research overview, discussion of preliminary research ideas, tutorials, and small experiments.

On our YouTube playlist you can watch some of the past seminars. On the SI website you can find more details on the next seminar, the upcoming seminars, and an archive of the past speakers.