Package: wnpp Severity: wishlist Owner: Simon Josefsson <[email protected]>
* Package name : golang-github-eliben-go-sentencepiece Version : 0.7.0-1 Upstream Author : Eli Bendersky * URL : https://github.com/eliben/go-sentencepiece * License : Apache-2.0 Programming Lang: Go Description : Go implementation of the SentencePiece tokenizer This is a pure Go implementation of encoding and decoding text with the SentencePiece tokenizer (https://github.com/google/sentencepiece). . "Encoding" is the operation used to split text into tokens, using a trained tokenizer model. "Decoding" is the reverse process - converting a list of tokens into the original text. Needed by google.golang.org/genai. https://salsa.debian.org/go-team/packages/golang-github-eliben-go-sentencepiece /Simon
signature.asc
Description: PGP signature

