Source: tokenizers Version: 0.23.1+dfsg-2 Severity: important Tags: patch User: [email protected] Usertags: compact-str-0.10 X-Debbugs-Cc: [email protected]
Dear maintainer, I am preparing a rust-compact-str transition, which will be uploaded soon. tokienizers needs a small patch for its Cargo.toml. I have attached a debdiff fixing this, which can be uploaded at any time. Best regards, Simon Quigley [email protected]
diff -Nru tokenizers-0.23.1+dfsg/debian/changelog tokenizers-0.23.1+dfsg/debian/changelog --- tokenizers-0.23.1+dfsg/debian/changelog 2026-08-10 21:46:56.000000000 -0500 +++ tokenizers-0.23.1+dfsg/debian/changelog 2026-08-24 18:44:58.000000000 -0500 @@ -1,3 +1,10 @@ +tokenizers (0.23.1+dfsg-2.1) unstable; urgency=medium + + * Non-maintainer upload. + * Allow compact_str 0.10. + + -- Simon Quigley <[email protected]> Mon, 24 Aug 2026 18:44:58 -0500 + tokenizers (0.23.1+dfsg-2) unstable; urgency=medium * Mask a test case for armhf and i386 to unblock migration. diff -Nru tokenizers-0.23.1+dfsg/debian/patches/set_version.patch tokenizers-0.23.1+dfsg/debian/patches/set_version.patch --- tokenizers-0.23.1+dfsg/debian/patches/set_version.patch 2026-07-09 20:49:26.000000000 -0500 +++ tokenizers-0.23.1+dfsg/debian/patches/set_version.patch 2026-08-21 00:06:16.000000000 -0500 @@ -38,11 +38,13 @@ +monostate = "1.0" ahash = { version = "0.8.11", features = ["serde"] } dary_heap = { version = "0.3.6", features = ["serde"] } - compact_str = { version = "0.9", features = ["serde"] } -Index: tokenizers/tokenizers/src/models/bpe/word.rs -=================================================================== ---- tokenizers.orig/tokenizers/src/models/bpe/word.rs -+++ tokenizers/tokenizers/src/models/bpe/word.rs +-compact_str = { version = "0.9", features = ["serde"] } ++compact_str = { version = ">= 0.9, < 0.11", features = ["serde"] } + + [features] + default = ["progressbar", "onig", "esaxx_fast"] +--- a/tokenizers/src/models/bpe/word.rs ++++ b/tokenizers/src/models/bpe/word.rs @@ -1,7 +1,8 @@ use super::Pair; use ahash::AHashMap;
OpenPGP_signature.asc
Description: OpenPGP digital signature
