A multilingual tokenizer to split a string into tokens
updated at Feb. 22, 2024, 4:04 p.m.
6 +0
90 +0
11 +0
Pragmatic Segmenter is a rule-based sentence boundary detection gem that works out-of-the-box across many languages.
updated at May 10, 2024, 7:25 p.m.
16 +0
537 +2
51 +1