A multilingual tokenizer to split a string into tokens
updated at Feb. 22, 2024, 4:04 p.m.
6 +0
90 +0
11 +0
Pragmatic Segmenter is a rule-based sentence boundary detection gem that works out-of-the-box across many languages.
updated at May 29, 2024, 12:43 p.m.
16 +0
539 +1
51 +0