Hi Michael,
On 27/01/2026 11:36, Michael Tremer wrote:
Hello Adolf,
On 26 Jan 2026, at 19:07, Adolf Belka <[email protected]> wrote:
Hi Michael & Stefan,
On 26/01/2026 18:23, Michael Tremer wrote:
Hello Stefan,
Thanks for looking into this.
I would suggest that for the cleanup project, it would be best to keep two
versions of rust-syn. Obviously this will inflate the number of packages for
now, but actually starting to update and therefore potentially add more
dependencies does not sounds wise to me. We would mix up too many changes into
one which is never good.
I am pretty certain from my work on the python update, which required rust
updates, that you will find that you will need two different versions of
rust-syn as one rust module will require an older version and another rust
module will require a newer version and the two won't overlap. In my work there
were even some rust modules where I ended up with the latest version plus two
older versions being required.
I hope that at some point someone will be able to explain to me what the
benefit is to ship an older version of a crate that will have some bugs that
have been fixed in the newer version. And if this is regarding LTS or breaking
changes, other libraries totally manage this...
I believe this is due to some packages that use another rust module not having
been updated for some time (1 or 2 years) and the module build is linked to the
older version of the additional required module.
It might be as simple as a rust module just being updated to use the new other
modules or there might be some work to make them work but the original module
developer needs to do the update.
I just did a search for option-ext and crates.io came up with 5 results. Th
oldest last updated is almost 5 years (works with a 2018 edition of Rust) and
the youngest updated is over 1 year ago (works with a 2021 edition of Rust,
which is not particularly recent).
Being tied to an older edition of Rust doesn't mean it won't build but it
increases the chances of the module needing to be re-written to be updated and
work with the latest Rust edition.
Searching for rust modules with the search text of option found 46425 results
and on the first page there is one module updated 13 days ago and another that
was updated almost 8 years ago.
From my work on adding required rust modules for the python build it is not
clear to me that a module developer ensures they are using other up to date
rust modules. I think they find they need a rust module and that might be up to
date but it might be using other modules that are quite old and that large
chain of dependencies can easily end up with requiring different versions of
some other modules than another module that also requires the same dependency.
Regards,
Adolf.
Please start a list with all those packages that we would have to have a look
at once the cleanup has been completed and we will start with updating them.
That way we should be able keep this cleaner and hopefully won’t introduce too
many new dependencies.
In the end, I guess we will have to run this cleanup more than just this time,
because we never know which package has dropped any dependencies. This is
however rather unlikely.
Best,
-Michael
On 26 Jan 2026, at 15:31, Stefan Schantl <[email protected]> wrote:
Hello list it's me again,
the build process now reached python-cryptography which requires rust-
asn1, which requires rust-ans1_derive, which did not build because of a
to new version of rust-syn.
rust-asn1_derive (0.12.2)
[ 1 ][ FAIL ]
make: Nothing to be done for 'download'.
make: Leaving directory '/home/ipfire-2.x/lfs'
make: Entering directory '/usr/src/lfs'
toml-0.8.19.tar.gz checksum OK
make: Nothing to be done for 'install'.
make: Leaving directory '/usr/src/lfs'
Jän 26 15:18:59: Building rust-asn1_derive make: Entering directory
'/home/ipfire-2.x/lfs'
make: Nothing to be done for 'download'.
make: Leaving directory '/home/ipfire-2.x/lfs'
make: Entering directory '/usr/src/lfs'
asn1_derive-0.12.2.tar.gz checksum OK
====================================== Installing asn1_derive-
0.12.2 ...
Install started; saving file list to /usr/src/lsalr ...
cd /usr/src/asn1_derive-0.12.2 && if [ -f Cargo.toml.orig ]; then \
rm -f Cargo.toml.orig; \
fi; \
cd /usr/src/asn1_derive-0.12.2 && mkdir -p /usr/src/asn1_derive-
0.12.2/.cargo && echo "${CARGO_CONFIG}" > /usr/src/asn1_derive-
0.12.2/.cargo/config && rm -f Cargo.lock
cd /usr/src/asn1_derive-0.12.2 && CARGOPATH=/usr/src/asn1_derive-
0.12.2/.cargo RUSTC_BOOTSTRAP=1 cargo --offline build --release -Z
avoid-dev-deps -j12
warning: `/usr/src/asn1_derive-0.12.2/.cargo/config` is deprecated
in favor of `config.toml`
|
= help: if you need to support cargo 1.38 or earlier, you can
symlink `config` to `config.toml`
error: failed to select a version for the requirement `syn =
"^1.0.58"`
candidate versions found which didn't match: 2.0.114
location searched: directory source `/usr/share/cargo/registry`
(which is replacing registry `crates-io`)
required by package `asn1_derive v0.12.2 (/usr/src/asn1_derive-
0.12.2)`
perhaps a crate was updated and forgotten to be re-vendored?
As a reminder, you're using offline mode (--offline) which can
sometimes cause surprising resolution failures, if this error is too
confusing you may wish to retry without `--offline`.
make: *** [rust-asn1_derive:78: /usr/src/log/asn1_derive-0.12.2]
Error 101
make: Leaving directory '/usr/src/lfs'
ERROR: Building rust-asn1_derive
[ FAIL ]
Check /home/ipfire-2.x/log_x86_64/_build.ipfire.log for errors if
applicable
[ FAIL ]
root@localhost:/home/ipfire-2.x#
Currently there is an older version of the rust-syn packaged, which
would allow me to bypass this issue, but would violence the goal of
getting rid of unneccessary rust modules.
Theoretically I also could update the asn1_derive crate to the latest
version but this may break building the next modules.
May this could act as starting point for the python update, where all
the rust stuff also needs to be touched.....
@Adolf, @Michael what do you think about that?
Thanks in advance,
-Stefan
Hello list,
currently I'm working on cleaning up the rust packages.
For these I disabled all rust modules in the make.sh file and perform
a
clean build as Michael suggested.
At the moment I'm past the stage where "cbindgen" successfully has
been
build and have 103 rust modules (inlcluding there sub-dependencies)
only for this one tool.
An additional rust module is required to build suricata. This is
because of patching the source code the required rust module is not
part of their source tarball.
This currently summs to 104 rust modules for the moment.
I'm looking forward when python-cryptography kicks in its module
whishes....
Best regards,
-Stefan
Hello Adolf,
Hello Michael,
I would give the rust cleanup a try in the next few days.
Adolf may I can ask you to put your current state of the python
update
into a git repositry?
As my last work from November last year was not completed and is based on an
older status now and also on python3.13 vs the current 3.14, I think it makes
more sense that once Stefan has completed the clean up of the rust modules, I
then take that as the starting point and go through all my changes as done
before until I get to the problem I experience with python-cryptography not
being able to find any of the rust modules required by python-maturin. That has
always been the point where I got stuck.
Did you try to update python-cryptography first without touching Python and
after that try the Python update?
At that time I will then put that git branch I am working on into my personal
IPFire git repo so that the two of you can look at it to see what I am doing
wrong at that stage.
That way, I can still contribute with all the update steps that I can do but
hand it over when it gets to the step that has consistently beat me.
Regards,
Adolf.
Thanks in advance,
-Stefan
Hello Adolf,
On 23 Jan 2026, at 11:06, Adolf Belka <[email protected]>
wrote:
Hi Michael,
On 23/01/2026 11:31, Michael Tremer wrote:
Hello Stefan,
Hello list,
Thank you for looking at this. Of course it is very important
that we are able to stay on the latest version of Suricata.
I have merged your monster of a patch so that we can move on
for
now, but I have a couple of bigger questions that we all
should
have a look at:
Adolf has in the past spent a lot of time on updating Rust.
This
is all tapping into Python - or rather python-cryptography -
having some Rust code that has further dependencies. In
essence,
it has been a huge headache to update this. Maybe Adolf even
has
some other words for this all.
My words on this are that I have now tried multiple times to
get
a
new python update built. Each time I have done it a bit
different
but the end result has been the same and that is that python-
cryptography (which requires rust modules to be built) ends up
requiring python-maturin that requires more rust modules but at
the
end of this the python-cryptography fails to find the built
rust
modules.
I have been stuck at this last point so many times that I have
realised that I am finding lots of reasons not to go and work
on
the python update.
That is not a good position and also python has now moved from
3.13
to 3.14 so things are moving away from me.
I have come to the conclusion that someone else, more capable
than
me needs to have a go at the python update, so I am giving up
on
it
but will continue working on other things.
Hmm okay, you sound like you are giving up on this :) I know how
many
hours (we probably need to measure those in days or even weeks)
you
have spent on this though.
Let’s pool resources together and finally get this done.
Hopefully
this will be a smoother ride as a combined effort.
Just building cbindgen has required a further ~98 Rust crates
to
be packaged. Often we have the same crate in different
versions
because other crates have pinned a specific version. In
total,
we
currently have ~790 packages in IPFire. Out of those, there
are
202 packages in the rust-* namespace. That is pretty much a
quarter of the distribution. Although not a lot in size, this
is
a considerable maintenance burden.
ClamAV and Suricata have (recently?) started to bundle all
their
Rust dependencies with their release tarballs. Although this
is
not a good thing for many other reasons, it will move the
onus
onto the upstream projects to provide whatever they need. If
their dependencies (and the dependencies of their
dependencies)
explode, this is not really our problem any more as well as
any
supply chain problems. Great - within reason.
That leaves us with only very few packages that would
actually
require any external Rust crates (Suricata is even configured
to
*exclusively* use their bundled crates): cbindgen as a new
thing,
python-cryptography, anything else? We might actually only
need
a
fraction of the Rust crates that we currently have as the
only
packages that may actually tap into our locally built
repository
are only those two.
Unfortunately there is the addon oci-python-sdk that uses
python-
cryptography.
python-cryptography was on my list. oci-python-sdk only uses Rust
indirectly through python-cryptography, right?
Is anyone happy to give this all a try and cleanup any old
Rust
deps? That way, I hope we will have a much smoother ride
moving
forward with a Python update.
I can take the current status, before Stefan's patches, and see
how
many existing rust modules can be removed. Anything that can be
removed is a step forward.
Yes, I think we should try to shrink what we have now if that is
possible at all. As most packages are bundling all Rust deps,
there
should be some we won’t need any more in the system.
Then, we hopefully have much less to update/worry about in any
other
way when we start touching python-cryptography.
So who is volunteering to do this? Commenting out all Rust
packages,
then build python-cryptography which will fail as it requires
some
Rust crates. Those will be there so they will only have to be
commented in again. Once the package builds, we should then have
a
couple of packages still commented that we can drop.
I think a problem moving forward is that more python modules
are
ending up being a combination of python and rust as the
cryptography and maturin modules have already done. I have also
seen a lot of rust modules covering the same stuff as covered
by
python modules. So the future I think looks like it will
continue
to be very frustrating.
Yes it does, but we will have to find a way whether we want it or
not.
-Michael
Regards,
Adolf.
All the best,
-Michael
On 22 Jan 2026, at 17:38, Stefan Schantl
<[email protected]> wrote:
Hello list followers,
I'm currently updating rust and affected modules.
This happends mainly because I'm trying to fix the
"suricata
cache
grows infinite" problem, which a lot of people are
affected.
To archive this, I ported the patches from suricata main
development
branch to our used suricata version (8.0.3).
To perform a full build, a new tool called cbindgen - which
is
a rust
to c bindings generator, is required.
Sadly this tool is also written in rust and requires some
new
dependencies and a more up to date rust compiler.
I hope to send a patchset for all this very soon to the
mailing
list.
Best regards,
-Stefan
