India is making significant strides in establishing its own independent artificial intelligence capabilities. Sarvam AI, a rapidly growing startup based in Bengaluru, has been entrusted with the crucial role of leading the development of India’s first sovereign large language model (LLM) under the IndiaAI Mission. This ambitious project underscores India’s dedication to fostering technological independence and leveraging the power of AI for the betterment of its citizens.
A Vision for Indigenous AI
The core of this initiative is a profound vision: to create an AI model that is not only indigenous but also possesses advanced reasoning skills, sophisticated speech processing capabilities, and seamless fluency in a wide range of Indian languages. This model will be deeply rooted in the Indian linguistic and cultural landscape, reflecting the nation’s unique identity and heritage. The model will be designed to understand and respond to the nuances of Indian languages, cultures, and societal contexts. This includes the ability to process regional dialects, understand cultural references, and generate content that is relevant and appropriate for Indian audiences.
To facilitate the realization of this vision, Sarvam AI will be granted access to a formidable arsenal of computational resources, including 4,086 NVIDIA H100 GPUs, over a six-month period. This access will enable the startup to build the LLM from the ground up, tailoring it to the specific needs and aspirations of the Indian context. This massive computational power will be essential for training the large language model on vast datasets of text and code. The GPUs will enable Sarvam AI to perform the complex calculations required for training the model efficiently and effectively.
Three Distinct Variants
The development of this sovereign LLM will encompass three distinct variants, each designed to cater to a specific set of applications and requirements:
Sarvam-Large: This variant will be engineered to excel in complex reasoning and generation tasks, enabling it to tackle intricate problems and generate sophisticated content. It will be designed for applications that require a high level of accuracy and understanding, such as research, analysis, and content creation.
Sarvam-Small: This variant will be optimized for real-time interactive applications, ensuring swift and responsive interactions with users in various scenarios. This variant will be suitable for applications such as chatbots, virtual assistants, and real-time translation services.
Sarvam-Edge: This variant will be tailored for on-device operations, allowing it to function seamlessly on resource-constrained devices without requiring constant connectivity to the cloud. This variant will be ideal for applications on mobile phones, tablets, and other devices where connectivity may be limited or unreliable.
In a collaborative endeavor, Sarvam AI will partner with AI4Bharat, an initiative of IIT Madras, to ensure that the models are deeply embedded in Indian linguistic and cultural contexts. This collaboration will leverage AI4Bharat’s expertise in natural language processing and its rich repository of Indian language resources. AI4Bharat will provide valuable data, tools, and expertise to help Sarvam AI build models that are truly representative of the Indian linguistic landscape.
Sarvam AI’s Proven Track Record
Sarvam AI has already distinguished itself as a frontrunner in the Indian AI landscape, particularly in the realm of multilingual AI. The company’s track record of innovation and its commitment to addressing the unique challenges of the Indian context have positioned it as a natural choice to lead this ambitious project. The company has demonstrated its ability to build high-quality AI models that are tailored to the specific needs of Indian users.
In October 2024, Sarvam AI unveiled Sarvam-1, a 2-billion-parameter LLM specifically designed and optimized for Indian languages. This model boasts support for ten major Indian languages, including Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu, in addition to English. This makes Sarvam-1 one of the most comprehensive multilingual AI models available for Indian languages. The model is able to understand and generate text in all of these languages with a high degree of accuracy and fluency.
Unlike many existing models that struggle with token inefficiency when processing Indic scripts, Sarvam-1 achieves fertility rates of 1.4 to 2.1 tokens per word. This remarkable feat significantly enhances processing efficiency, enabling the model to handle Indian languages with greater speed and accuracy. Token inefficiency is a common problem with many AI models when processing Indic scripts, which can lead to slower processing times and reduced accuracy. Sarvam-1’s high fertility rates demonstrate its ability to overcome this challenge.
Domestic Training and Infrastructure
Sarvam-1 was trained entirely within India, leveraging domestic AI infrastructure powered by NVIDIA H100 Tensor Core GPUs, Yotta’s data centers, and AI4Bharat’s language resources. This end-to-end domestic approach underscores India’s growing capabilities in AI development and its commitment to building a self-reliant AI ecosystem. By training the model entirely within India, Sarvam AI is able to ensure that the model is tailored to the specific needs of Indian users and that the data used to train the model is representative of the Indian linguistic landscape.
Performance benchmarks have revealed that Sarvam-1 not only matches but, in some cases, surpasses larger models like Meta’s Llama 3.1 8B and Google’s Gemma-2-9B, particularly in tasks involving Indic languages. This impressive performance highlights the effectiveness of Sarvam AI’s approach and its ability to compete with global AI leaders. Sarvam-1’s performance demonstrates that it is possible to build high-quality AI models that are specifically tailored to the needs of Indian users and that can compete with the best models in the world.
On the TriviaQA benchmark across Indic languages, Sarvam-1 achieved an accuracy of 86.11, outperforming Llama-3.1 8B’s score of 61.47. This significant margin demonstrates Sarvam-1’s superior capabilities in understanding and processing information in Indian languages. The TriviaQA benchmark is a widely used measure of a model’s ability to answer questions about factual information. Sarvam-1’s high score on this benchmark demonstrates its ability to understand and process information in Indian languages with a high degree of accuracy.
Challenges Ahead
While Sarvam AI has demonstrated its capabilities with Sarvam-1, the task of building the first indigenous foundation model is not without its challenges. Overcoming these challenges will require ingenuity, perseverance, and a collaborative spirit. The development of a foundation model is a complex and demanding undertaking that requires significant resources and expertise.
Infrastructure Scaling
One of the most significant hurdles is scaling up infrastructure to meet the demands of training large models. Training these models requires access to massive computational power over extended periods. While the government’s provision of thousands of NVIDIA H100 GPUs is a significant step forward, managing, optimizing, and maintaining such high-end resources is a complex undertaking. The sheer scale of the computational resources required to train large language models presents a significant logistical and technical challenge.
Effective resource management will be crucial to ensure that the training process is efficient and cost-effective. This will involve optimizing the use of GPUs, managing memory allocation, and implementing strategies to mitigate potential bottlenecks. Sarvam AI will need to develop sophisticated resource management techniques to ensure that it is able to make the most of the available computational resources. This will involve careful planning, monitoring, and optimization of the training process.
Data Curation
Another critical challenge lies in curating high-quality, diverse datasets. India’s linguistic landscape is incredibly complex, with variations not only between languages but also within dialects, cultures, and writing styles. Creating a balanced dataset that truly captures this diversity without introducing biases is essential but extremely challenging. The quality and diversity of the data used to train a large language model is crucial to its performance. A biased or incomplete dataset can lead to a model that is inaccurate, unfair, or discriminatory.
The dataset must be representative of the various regions, communities, and social groups within India. It must also be free from biases that could lead to unfair or discriminatory outcomes. Careful attention must be paid to the selection and annotation of data to ensure that it meets these criteria. This will require a significant investment in data curation and annotation. Sarvam AI will need to develop robust processes for collecting, cleaning, and labeling data to ensure that it is of the highest quality.
Linguistic Nuances
Furthermore, the models must be able to capture the subtle nuances of Indian languages, including idioms, metaphors, and cultural references. This requires a deep understanding of the cultural context in which these languages are used. Language is deeply intertwined with culture, and a model that is not able to understand the cultural context in which a language is used will not be able to generate text that is truly meaningful or relevant.
Sarvam AI’s collaboration with AI4Bharat will be instrumental in addressing these challenges. AI4Bharat’s expertise in Indian languages and its access to a vast repository of linguistic resources will provide valuable support in the development of the sovereign LLM. AI4Bharat’s deep understanding of Indian languages and cultures will be invaluable in helping Sarvam AI to build models that are truly representative of the Indian linguistic landscape.
Implications for India
The development of a sovereign LLM holds profound implications for India’s technological landscape and its role in the global AI arena. This initiative has the potential to transform various sectors, including education, healthcare, finance, and governance. The availability of a high-quality, multilingual AI model that is tailored to the specific needs of Indian users will unlock a wide range of new opportunities for innovation and economic growth.
Economic Growth
By fostering innovation and driving economic growth, the sovereign LLM can create new opportunities for Indian businesses and entrepreneurs. It can also help to bridge the digital divide by providing access to information and services in local languages. The LLM can be used to develop new products and services that are specifically tailored to the needs of Indian users. It can also be used to improve the efficiency and effectiveness of existing businesses.
Empowerment
Moreover, the LLM can empower citizens by providing them with access to personalized education, healthcare, and other essential services. It can also help to promote social inclusion by breaking down language barriers and fostering communication between different communities. The LLM can be used to provide personalized learning experiences for students, to improve access to healthcare services, and to facilitate communication between people who speak different languages.
Strategic Independence
Ultimately, the development of a sovereign LLM is a strategic imperative for India. It will enable the nation to develop its own AI capabilities, reducing its reliance on foreign technology and ensuring its digital sovereignty. By developing its own AI capabilities, India will be able to control its own data and technology and to ensure that its AI systems are aligned with its values and priorities.
A Collaborative Ecosystem
The success of this ambitious endeavor hinges on the creation of a collaborative ecosystem that brings together government, industry, academia, and the startup community. By working together, these stakeholders can leverage their collective expertise and resources to drive innovation and accelerate the development of AI in India. Collaboration is essential to the success of any major technological undertaking. By bringing together the best and brightest minds from different sectors, India can accelerate the development of AI and ensure that it is used for the benefit of all its citizens.
The government’s support for Sarvam AI and its commitment to providing access to computational resources are crucial enablers of this ecosystem. Industry partnerships can provide access to real-world data and expertise, while academic institutions can contribute cutting-edge research and talent. The government can play a key role in fostering collaboration by providing funding, infrastructure, and regulatory support. Industry can provide valuable data, expertise, and market access. Academic institutions can contribute cutting-edge research and talent.
A Future Powered by AI
As India embarks on this transformative journey, the nation stands poised to unlock the immense potential of AI and create a future powered by innovation, inclusivity, and self-reliance. The development of a sovereign LLM is a testament to India’s ambition and its unwavering commitment to shaping its own destiny in the age of artificial intelligence. AI has the potential to transform every aspect of Indian society, from education and healthcare to agriculture and manufacturing. By investing in AI, India can create a more prosperous, equitable, and sustainable future for all its citizens. The LLM serves as a critical component in creating a truly digital India.