arXiv NLP
SuTRA : Structurally-Unified Tokenization with Root Awareness
arXiv:2608.18087v1 Announce Type: new Abstract: Existing subword tokenizers optimize statistical compression but ignore morphological structure, particularly the relationship between roots and affixes...
arXiv NLP