ஆலமரம்Aalamaram
The largest public Tamil treebank we know of: around ten thousand sentences annotated for part of speech, named entities, morphology and dependencies. We have adopted it in the design and we have not been able to find where the data lives.
Published
WILDRE workshop at LREC 2024
Size
≈10,000 sentences, POS, NER, morphology, dependencies
Notable
Universal Dependencies adjusted for Tamil clitics and segmentation
Blocked on
Locating the distribution, and reading its licence
Aalamaram: A large-scale linguistically annotated treebank for the Tamil language
A. M. Abirami, W. Q. Leong, H. Rengarajan, D. Anitha, R. Suganya, H. Singh, Kengatharaiyer Sarveswaran, W. C. Tjhi, R. R. Shah
Proceedings of the 7th Workshop on Indian Language Data Resource and Evaluation (WILDRE), Turin, 2024.
What we would do with it
Today we check our morphology one word at a time, against rules. A treebank lets us check it against ten thousand sentences that somebody has already annotated by hand, which is a different order of confidence.
- Cross-check the analyser at scale. Where our output and the treebank disagree, one of us is wrong and it is worth knowing which.
- Evaluation with real context. Our test words currently sit in isolation. A sentence is where ambiguity actually gets resolved.
- Sentence support. The step after single words needs contextual disambiguation, and that needs annotated sentences to learn from and to test against.
- Corpus for anything later. Every long-term plan in this project needs annotated Tamil, and there is very little of it.
A correction we made about it, publicly
An earlier version of our own plan proposed replacing our equivalents word list with Aalamaram. That was wrong, and it is recorded as wrong in our decision log rather than edited away. Aalamaram is a treebank. It says how sentences are structured. It is not a list of borrowed words and their Tamil equivalents, and the two jobs have nothing to do with each other.
The mistake is worth leaving visible because it is the exact failure this project is built to prevent: a confident sentence about a source, from someone who had read about it rather than read it.
What we are asking
Only where to look. We could not find the data on Hugging Face or in a public repository, and the paper is the trail we have. Two questions:
- Where is the corpus distributed?
- Under what licence, and are there conditions on using it to evaluate a tool that is itself public?
Until both are answered it stays adopted-in-principle and unused, which is the honest state and not a comfortable one. thamizh@ief-global.org
One of its authors also wrote ThamizhiMorph, which we depend on, and is senior author on ILAKKANAM, which is how we would measure ourselves. A great deal of what makes Tamil computable comes from a small number of people, and that is worth saying rather than quietly benefiting from.