Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-tokenizers for openSUSE:Factory checked in at 2026-07-21 23:11:07 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-tokenizers (Old) and /work/SRC/openSUSE:Factory/.python-tokenizers.new.24530 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-tokenizers" Tue Jul 21 23:11:07 2026 rev:8 rq:1366916 version:0.23.1 Changes: -------- --- /work/SRC/openSUSE:Factory/python-tokenizers/python-tokenizers.changes 2026-03-01 22:15:46.108290285 +0100 +++ /work/SRC/openSUSE:Factory/.python-tokenizers.new.24530/python-tokenizers.changes 2026-07-21 23:11:25.860411632 +0200 @@ -1,0 +2,20 @@ +Wed Jul 15 05:49:45 UTC 2026 - Martin Pluskal <[email protected]> + +- Update to 0.23.1: + * First stable release in the 0.23 series (0.23.0 was never + published as a stable version) + * Add support for Python 3.14, including the free-threaded + (3.14t) interpreter + * Ship complete type hints (.pyi stubs) for all public classes + * Performance improvements on the BPE and added-vocabulary + tokenization paths + * add_tokens() now normalizes token content when tokens are + added; a re-saved tokenizer.json may differ in its + added_tokens block, but existing files load unchanged + * Drop support for Python 3.9 (now requires Python 3.10 or newer) +- Recompile the installed modules as hash-based bytecode and drop a + stray %fdupes call in %build, clearing the + python-bytecode-inconsistent-mtime and rpm-buildroot-usage rpmlint + errors the package previously carried + +------------------------------------------------------------------- Old: ---- tokenizers-0.22.0.tar.gz New: ---- tokenizers-0.23.1.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-tokenizers.spec ++++++ --- /var/tmp/diff_new_pack.i8Rs2Z/_old 2026-07-21 23:11:27.996484674 +0200 +++ /var/tmp/diff_new_pack.i8Rs2Z/_new 2026-07-21 23:11:27.996484674 +0200 @@ -19,10 +19,9 @@ %if 0%{?suse_version} && 0%{?suse_version} < 1550 %global force_gcc_version 13 %endif - %{?sle15_python_module_pythons} Name: python-tokenizers -Version: 0.22.0 +Version: 0.23.1 Release: 0 Summary: Provides an implementation of today's most used tokenizers License: Apache-2.0 @@ -38,8 +37,8 @@ BuildRequires: gcc%{?force_gcc_version}-c++ BuildRequires: python-rpm-macros BuildRequires: zstd -ExclusiveArch: %{rust_tier1_arches} riscv64 Requires: python-huggingface-hub +ExclusiveArch: %{rust_tier1_arches} riscv64 %python_subpackages %description @@ -59,7 +58,7 @@ %prep %autosetup -p1 -n tokenizers-%{version} rm -rfv .cargo -tar xf %{S:1} -C $PWD +tar xf %{SOURCE1} -C $PWD %build export CARGO_HOME=$PWD/.cargo @@ -73,7 +72,6 @@ %endif pushd bindings/python %pyproject_wheel -%python_expand %fdupes %{buildroot}%{$python_sitearch} %install export CARGO_NET_OFFLINE=true @@ -82,6 +80,10 @@ export CARGO_PROFILE_RELEASE_STRIP=false pushd bindings/python %pyproject_install +# Recompile the installed modules as hash-based bytecode so .pyc invalidation +# does not depend on the reproducibility-clamped .py mtimes (otherwise trips +# python-bytecode-inconsistent-mtime). +%python_expand $python -m compileall -q -f --invalidation-mode=unchecked-hash -o 0 -o 1 -s %{buildroot} %{buildroot}%{$python_sitearch}/tokenizers %python_expand %fdupes %{buildroot}/%{$python_sitearch}/* %check ++++++ registry.tar.zst ++++++ /work/SRC/openSUSE:Factory/python-tokenizers/registry.tar.zst /work/SRC/openSUSE:Factory/.python-tokenizers.new.24530/registry.tar.zst differ: char 258637, line 953 ++++++ tokenizers-0.22.0.tar.gz -> tokenizers-0.23.1.tar.gz ++++++ ++++ 25640 lines of diff (skipped)
