Reference
Two vocabularies for the same thing
Computational linguistics and Tamil grammar describe much of the same territory in different words, and neither field learned the other's. This table is the map between them.
It is written for a reader who knows வேற்றுமை and விகுதி and has no reason to know "lemma" or "finite-state transducer". It works in the other direction too, for an engineer who knows the technical terms and is meeting பகுபத உறுப்பு for the first time.
The mapping is close, not exact. Some technical terms have no clean Tamil equivalent, and some Tamil categories have nothing to map onto. திணை, the rational and non-rational distinction, is the standing example: it is central to Tamil agreement and the international standard for annotating grammar cannot express it. The full document lists those mismatches, and they are the interesting part.
| Technical term | Nearest Tamil concept | In one line |
|---|---|---|
| Morphology | சொல் இலக்கணம் (word-internal structure) | How words are built out of parts |
| Morpheme | உறுப்பு (as in பகுபத உறுப்பு) | The smallest piece that carries meaning or grammar |
| Agglutinative | ஒட்டுநிலை மொழி | Parts chain onto a word, each keeping its own identity |
| Surface form | the word as actually written | மரத்தில் |
| Lemma | அகராதி வடிவம் — the headword | மரம் |
| Stem | the form suffixes attach to | மரத்- |
| Root | வேர்ச்சொல் | the historical seed of a word family |
| Affix / suffix | விகுதி, இடைநிலை, சாரியை | a part added to a stem |
| Inflection | same word, different grammatical form | மரம் → மரத்தில் |
| Derivation | a new word made from an old one | செய் → செயல் |
| Feature | இலக்கணக் கூறு | one grammatical property (tense, case, பால்…) |
| Feature bundle | the full grammatical description | "noun, அஃறிணை, singular, 7th வேற்றுமை" |
| POS tag | சொல் வகை | பெயர் / வினை / இடை / உரி |
| Case | வேற்றுமை | the eight |
| Sandhi | புணர்ச்சி | sound change where two forms join |
| Analysis | பகுத்தல் | word → parts |
| Generation | ஆக்கல் | parts → word |
| Segmentation | பிரித்தல் | marking where the parts divide |
| Disambiguation | choosing among possible readings | which analysis of a form is right *here* |
| Paradigm | வாய்பாடு / inflection table | the full set of forms for a word class |
| Corpus | தொகுப்பு | a large collection of text |
| Treebank | grammatically annotated sentence collection | Aalamaram |
| Token | whatever unit a model counts | usually *not* a word |
| OOV | out of vocabulary | a word the system has never seen |
24 terms, copied 2026-08-19 from the project's
full glossary by scripts/sync-glossary.py. The full
version explains each one properly and works two examples end to end.
Three that cause the most trouble
- Lemma, stem and root
- Three different things that get used interchangeably, and Tamil grammar slices them differently again. For மரத்தில் the lemma is மரம், the dictionary headword. The stem is மரத்-, which is what the suffix attaches to. The root is a historical claim about a word family. A tool that conflates them will confidently return the wrong one.
- Token
- In this field a token is whatever unit a model happens to count, which is usually not a word and often not even a syllable. Almost every claim about how much text a model can hold depends on this, and for Tamil the answer is unflattering. That is the subject of why this project exists.
- Gold, silver and disputed
- Data that has been verified, data that has been generated and looks right, and data where the sources genuinely disagree. Keeping them apart is what makes a corpus worth training on. Collapsing them is how a plausible error becomes permanent.