Hi! I need something like DetokenizationOperation.MERGE_BOTH to train a Tokenizer from NameFinder data. A sample of the data is:
... devolva - me o livro .... (give the book back to me) I need detokenize it to "devolva-me o livro" So I would need to add the hyphen to the detokenizer dictionary and configure it to something like MERGE_BOTH, but we don't have such option. Do you see another way of doing it or should I extend the the DetokenizationOperation ? Thanks William
