NVIDIA and the Indian research group AI4Bharat introduced Nemotron-pretraining-Indic-v1 on 16 October 2026, a synthetic pretraining corpus of about 2.8 trillion tokens covering 22 Indian languages, according to NVIDIA’s blog on Hugging Face and reports by Unite.AI, Hyper.ai and The Chenab Times. The release includes an open, GPU-accelerated pipeline for generating such data.
A pretraining corpus is the large body of text on which a language model first learns. The reports say that data scarcity is the main obstacle for Indian-language models, because the languages are spoken by hundreds of millions of people but have far less written text online than English. The 22 languages are those listed in India’s Constitution, which are the first languages of 97 per cent of Indians, nearly 1.2 billion citizens at the 2011 Census, according to the release.
The numbers are given in detail. The corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, counted with the Gemma-4 tokenizer, according to the release. Added together they come to 2,825.6 billion, or about 2.83 trillion tokens, and romanized text is about 8 per cent of the total, calculations by IndiaX24. A token is a unit of text, usually a word or part of a word, and counts depend on the tokenizer used, which is why the release names the tokenizer.
Twelve of the languages have both native-script and romanized records: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu and Urdu, according to the release. Ten have native-script data only: Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali and Sindhi. Romanized text, meaning Indian languages written in the Latin alphabet, reflects how many people type on phones.
The data is synthetic. The pipeline translates curated English material into the Indian languages while preserving technical content such as code and equations, according to the release. That approach lets the builders use high-quality English sources, but a translated corpus can carry translation errors and may not reflect how native speakers write. The reports reviewed do not give an evaluation of the corpus, so IndiaX24 cannot say how models trained on it perform.
Openness is the main claim. The pipeline is described as open and reproducible, so that other groups can generate their own data in the same way, according to the reports. The reports reviewed do not state the licence of the corpus or the cost of generating it, and IndiaX24 has not verified either.
The release adds to a set of Indian-language efforts that IndiaX24 has covered, including Sarvam AI’s models, which a survey of investors discussed in an earlier article. NVIDIA said in February that Sarvam had released models of 3, 30 and 100 billion parameters across 22 Indic languages, according to NVIDIA’s blog from 17 February, so the new corpus is a resource for developers beyond any single company.
The partnership between a US chip maker and an Indian academic initiative reflects the interest of hardware vendors in Indian markets. The reports reviewed do not say how many GPU hours were used or where the processing was done, and they do not name any model that will be trained on the corpus.
Researchers will want to know how much of the 2.8 trillion tokens is available for the smaller languages. The reports give the language list but not the token counts for each language, so IndiaX24 cannot say how evenly the data is spread. The ten languages with native-script data only are likely to have much less text than Hindi or Bengali, which is IndiaX24’s expectation and not a figure from the release.
IndiaX24 will report on models trained on the corpus and on any independent evaluation.
Image: NVIDIA’s headquarters in Santa Clara, California (file photo). Photo by Coolcaesar, licensed under CC BY-SA 4.0, via Wikimedia Commons.
Leave a Reply