LLM is a type of AI designed to process, understand, and generate human-like text. Built on vast datasets, it serves as the foundation for creating virtual assistants useful in everyday life.

Developing Vietnamese LLMs to build virtual assistants tailored for Vietnamese users is deemed highly significant by the Ministry of Information and Communications (now MOST).

Many activities have been undertaken recently to promote the development of Vietnamese LLMs and virtual assistants designed specifically for Vietnamese. Among these efforts is the evaluation and ranking of Vietnamese LLMs developed by domestic digital technology firms such as Viettel, VNPT, FPT, MISA, VNG, and others.

The National Digital Transformation Agency (NDTA), an arm of MOST, on March 17 stated that preparations for the evaluation and ranking of Vietnamese LLMs—such as compiling a question bank and test materials, developing tools to integrate enterprise services, and conducting trial connections before the official assessment—have been largely completed.

NDTA said the evaluation process is slated for March, with official results to be announced by MOST in April. The evaluation panel will consist of experts from Vietnamese training institutions and research institutes.

NDTA highlighted the key goal of enhancing the experience of local users.

For Vietnamese LLMs to effectively serve Vietnamese users, evaluating and comparing them with other models is essential. The evaluation results will help developers refine their models, and produce more natural responses, contextually accurate answers, culturally appropriate content, and outputs that are easily understood by most users. This is particularly critical for applications like chatbots, virtual assistants, and customer support services in Vietnamese.

Additionally, the evaluation process will foster domestic AI research and development. Once a dedicated evaluation framework for Vietnamese is launched, local research groups will be motivated to create standardized datasets, evaluation benchmarks, and techniques to improve their models. This initiative will cultivate an AI development ecosystem in Vietnam, opening up numerous collaboration opportunities among organizations, enterprises, and universities.

Beyond boosting international competitiveness, evaluating Vietnamese LLMs will pave the way for diverse practical applications. Specifically, thoroughly assessed LLMs will gain trust and be readily deployable across sectors such as education, healthcare, banking and finance, and e-government.

Earlier, in July 2024, the Ministry of Information and Communications (now MOST) issued basic requirements and evaluation methods for Vietnamese LLMs and virtual assistants.

This guideline supports the evaluation and selection of platforms under the Program to Promote the Development and Use of National Digital Platforms for Digital Transformation, as well as the development of digital government, economy, and society.

Van Anh