What makes Tamil hard for AI

What’s different

Tamil has 247 letters: 12 vowels, 18 consonants, 216 consonant–vowel combinations, and ஃ. Computers don’t store most of them as one character. கி is stored as க plus the vowel sign ி, and க் is stored as க plus the dot (புள்ளி). தமிழ் is 3 letters to a reader but 5 characters to a computer, and 15 bytes.

Why AI gets it wrong

Many AI tokenizers cut a consonant away from its vowel sign before they do anything else. In our tests, Qwen3 cut 98.8% of Tamil words this way, and it needs about 4 times as many tokens per word as Gemma 3. That makes Tamil slower and costlier, and harder for the model to learn.

What we’re doing

• A public tokenizer leaderboard (Uraikal D4).
• A one-line fix for the splitting.
• A Tamil-aware tokenizer.

What’s different

Tamil builds meaning by adding endings to a root. படித்துக்கொண்டேயிருப்பவர்களுக்காகவும் is a single word (21 letters). It means “also for the sake of those who keep on reading”. It’s built from படி (read), then markers for “having”, “keep on”, “those who”, plural, “for”, “for the sake of” and “also”.

Why AI gets it wrong

Models built for English expect short words. A Tamil root shows up in thousands of forms, and each looks like a new word unless the model learns the pattern.

What we’re doing

Tokenizers and word analysers that understand Tamil word structure.

What’s different

When two words or word parts join, letters can appear, change or disappear:

• வாழை + பழம் → வாழைப்பழம் (a ப் appears)
• மரம் + வேர் → மரவேர் (ம் disappears)
• கல் + தூண் → கற்றூண் (letters change)
• நாடு + இல் → நாட்டில் (உ drops and ட doubles)

Getting it wrong can change the meaning, so writers are careful with it.

Why AI gets it wrong

நாட்டில் doesn’t contain the string நாடு. So simple search and spell-check tools can’t tell they’re the same word.

What we’re doing

A sandhi checker and word analyser, tested against examples written by Tamil writers.

What’s different

In Tamil, where a letter sits in the word decides how it sounds:

• க is “k” in கடல், “kk” in பக்கம், “g” in தங்கம் and close to “h” in மகன்.
• த is “th” in தலை and “dh” in அது.

Why AI gets it wrong

Pronunciation tools that go letter by letter read மகன் as “makan”. People say “magan” or “mahan”, and that’s also how they type it in English letters.

What we’re doing

A Tamil pronunciation library that follows these rules, for our voice and our speech recognition.

What’s different

Tamil tells apart sounds that English treats as one:

• ல / ள / ழ: வலி (pain), வளி (wind), வழி (path).
• ன / ண: மனம் (mind), மணம் (fragrance).
• ர / ற: மரம் (tree), மறம் (valour).
• Short and long sounds: பல் (tooth), பால் (milk); கடை (shop), கட்டை (log).

Why AI gets it wrong

Speech systems trained mostly on English may blur these, and one blurred sound changes the word. We don’t yet know how much this happens. That’s exactly what we’re measuring.

What we’re doing

A test with these word pairs, read by speakers from every region.

What’s different

Written Tamil and everyday spoken Tamil differ in regular, rule-governed ways:

Why AI gets it wrong

Models learn mostly from written text, so they answer like a newsreader. Speech recognition trained on read speech struggles with natural talk.

What we’re doing

Models that understand both and answer in warm spoken Tamil, with tests for each.

What’s different

Kongu Tamil has its own respect forms (ஏனுங்க). Tirunelveli has அண்ணாச்சி. Jaffna Tamil says ஓம் for “yes” (ஆமாம்) and கதைக்கிறது for “speaking”. Chennai Tamil borrows freely from English, Telugu and Urdu.

Why AI gets it wrong

Training data over-represents a few cities and written Tamil.

What we’re doing

Speech test sets from nine regions, from Chennai to Jaffna to Singapore. Our headline score averages across them, so no region gets left behind.

What’s different

When English words enter Tamil, they take Tamil endings: “Application-அ online-ல submit பண்ணினேன்”. English verbs need a Tamil helper verb: “login பண்ணுங்க”, “confuse ஆயிட்டேன்”. In English letters, one word is spelled many ways: போறேன் becomes poren, pooren or porean, and ழ becomes zh, l or z.

Why AI gets it wrong

Tools treat Tanglish as noise, or as English.

What we’re doing

Tanglish test sets, and conversion between Tanglish and Tamil script.

What’s different

Tamil puts the verb at the end and descriptions before the noun, with no “who” or “which”:

மதுரையில் வாழ்ந்த ஆசிரியரால் எழுதப்பட்ட புத்தகத்தை வாங்கிய பையன் வந்தான்.

Word for word, that’s “[in Madurai lived] [by the teacher written] [the book bought] boy came”. In English: “The boy who bought the book written by the teacher who lived in Madurai came.”

Why AI gets it wrong

Text translated from English comes out in unnatural English word order.

What we’re doing

Training on natural Tamil, and checking answers for natural style.

What’s different

• The verb shows who acted, so Tamil often leaves out “I”, “he” or “they”. வந்தான், வந்தாள், வந்தார், வந்தது.
• There are two words for “we”: நாம் includes the listener, and நாங்கள் doesn’t. A service that says “நம் கணக்கு” (our account, yours and mine) means something different from “எங்கள் கணக்கு” (our institution’s account).
• There are three levels of “you”: நீ, நீங்கள், தாங்கள்.
• Family words are precise: சித்தப்பா, பெரியப்பா and மாமா are all “uncle” in English.

Why AI gets it wrong

English has one “we”, one “you” and one “uncle”. So translated text loses the distinction, or gets the politeness wrong.

What we’re doing

Our assistant matches its politeness to the person, and we have tests for kinship and pronouns.

What’s different

• 90 is தொண்ணூறு, and 900 is தொள்ளாயிரம். These are easy to mix up, and neither is “nine tens” or “nine hundreds”.
• Fractions are everyday words: அரை (½), கால் (¼), முக்கால் (¾), ஒன்றரை (1½).
• Large numbers go in lakhs and crores, as in ஒன்றரை லட்சம் (1,50,000).
• Old Tamil numerals multiply: ௨௱௫ is 2 × 100 + 5 = 205.

Why AI gets it wrong

Models are good with digits and weak with number words. That matters most for people who speak rather than type.

What we’re doing

Number handling in both directions, from words to digits and back, already released in our open library, plus tests for models.

What’s different

• Gold is weighed in சவரன் (8 grams).
• Land is measured in செண்ட் (435.6 sq ft) and ஏக்கர் (100 cents).
• Interest is quoted by the month. “2 ரூபாய் வட்டி” means 2% a month, which is 24% a year.

Why AI gets it wrong

A model that reads “2 rupees interest” as 2% a year gets a farmer’s loan wrong by 12 times.

What we’re doing

Exact unit and interest conversion, done by tested code, not by the model’s guesswork.

What’s different

Tamil prosody (யாப்பு) has precise rules:

• Letters are short (குறில்), long (நெடில்) or a bare consonant (ஒற்று).
• Letters group into அசை, அசை group into சீர், and neighbouring சீர் must link by rules (தளை).
• Every Thirukkural couplet has seven சீர், four in the first line and three in the second.

Why AI gets it wrong

Language models invent verses that look right but break the metre, or present made-up couplets as genuine.

What we’re doing

• A prosody checker tested on all 1,330 Kurals.
• Exact quotes from trusted texts, never from model memory.

What’s different

Many government orders, court judgments and older textbooks were typed in pre-Unicode Tamil fonts, such as Bamini, TAB, TAM and TSCII. On screen they look like Tamil. But the characters underneath are other symbols, often plain English letters, sometimes with the vowel sign stored before the consonant.

Why AI gets it wrong

Copied text, and text from search engines, comes out as gibberish, so this material is invisible to AI.

What we’re doing

A converter that detects each old font and turns it into proper Unicode Tamil.