OpenAI on Tuesday unveiled IndQA, a benchmark designed to improve AI understanding of India’s languages, traditions and ways of life. The company said the tool evaluates how well systems answer questions rooted in Indian cultural and linguistic contexts.
Developed with 261 domain experts across the country, IndQA contains 2,278 questions in 12 Indian languages and covers 10 cultural areas such as literature, cuisine, spirituality, history, everyday life and the arts. Unlike many multilingual datasets that rely on translation, IndQA uses questions written natively in regional languages to reflect how people actually think and speak.
“India is one of the most linguistically diverse nations in the world. If AI is to serve everyone, it must understand how people actually speak and reason within their own cultures,” OpenAI said in a blog post.
IndQA focuses on reasoning and cultural context rather than simple translation or multiple choice. Each item includes a culturally grounded prompt, an English translation for verification and rubric based grading criteria set by experts in each field.
The benchmark tests models across 10 categories including Arts and Culture, Law and Ethics and Religion and Spirituality. Languages in the dataset include Hindi, Tamil, Telugu, Kannada, Malayalam, Marathi, Gujarati, Bengali, Odia, Punjabi, Hinglish and English. Hinglish was included to reflect common code switching in everyday speech.
Responses are checked by a model based evaluator against expert criteria. The final score measures how closely a model matches human like cultural reasoning.
Questions were authored and reviewed by journalists, linguists, artists, academics and industry practitioners. Contributors include a Nandi Award winning Telugu actor and writer, a Marathi editor from a regional daily, an international chess grandmaster and an award winning Tamil poet and activist.
OpenAI said it filtered questions by testing them against its strongest models such as GPT 4o, OpenAI o3, GPT 4.5 and partially GPT 5, keeping only items models found challenging. IndQA is intended to track progress within model families over time rather than to rank languages against each other. The company plans to extend the framework to other regions and languages.
Also read: Viksit Workforce for a Viksit Bharat
Do Follow: The Mainstream formerly known as CIO News LinkedIn Account | The Mainstream formerly known as CIO News Facebook | The Mainstream formerly known as CIO News Youtube | The Mainstream formerly known as CIO News Twitter
About us:
The Mainstream formerly known as CIO News is a premier platform dedicated to delivering latest news, updates, and insights from the tech industry. With its strong foundation of intellectual property and thought leadership, the platform is well-positioned to stay ahead of the curve and lead conversations about how technology shapes our world. From its early days as CIO News to its rebranding as The Mainstream on November 28, 2024, it has been expanding its global reach, targeting key markets in the Middle East & Africa, ASEAN, the USA, and the UK. The Mainstream is a vision to put technology at the center of every conversation, inspiring professionals and organizations to embrace the future of tech.
