Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
ср, 10 февр. 2021 г. в 01:16, Tim Harder : > > On 2021-02-09 Tue 17:51, Benda Xu wrote: > > I am wondering how useable pkgcore is on alpha, hppa, etc. Maybe it's > > time for us to plan for a Gentoo without essential Python dependency. > > Just to keep misinformation down, pkgcore currently has nothing to do > with rust as it's implemented in python due to basically being deemed > portage-ng when it forked ~15 years ago. It previously did have some > extensions written in C which have mostly been removed in the current > day from CPython catching up in a number of areas. > > That being said, alternative languages and related support has been > looked at for a number of reasons/features and development could move in > that direction, but hasn't yet in a public fashion. > > Tim > Another far-fetched option is to compile CPython to WebAssembly+WASI [1] (even if CPython itself will require Rust in future), and run it on the non-Rust-supported architecture via an interpreter [2]. If this works at all, bootstrap will probably be not trivial. And such version of interpreted python may need to be added to PYTHON_TARGETS to enable testing of this on more usual architectures like amd64. [1] https://wasi.dev/ [2] https://github.com/wasm3/wasm3
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
Mike Gilbert wrote: > > > The first big blocker we're going to hit is trustme [3] package that > > > relies on cryptography API pretty heavily to generate TLS certs for > > > testing. > > > > For which testing? Could it be changed to generate certs differently? > > He is probably referring to the test suite in dev-python/urllib3, > which uses the python "trustme" package in several places. > > https://github.com/urllib3/urllib3/search?q=trustme > > Someone would need to convince the urllib3 developers to use something > else to manage certificates in their unit tests. Ah - non-Gentoo packages - thanks, I understand. The trustme API isn't very large, so it seems doable to create a different implementation of it without rust dependencies, add virtual/trustme and be done? //Peter
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Wed, Feb 10, 2021 at 4:57 PM Peter Stuge wrote: > > The first big blocker we're going to hit is trustme [3] package that > > relies on cryptography API pretty heavily to generate TLS certs for > > testing. > > For which testing? Could it be changed to generate certs differently? He is probably referring to the test suite in dev-python/urllib3, which uses the python "trustme" package in several places. https://github.com/urllib3/urllib3/search?q=trustme Someone would need to convince the urllib3 developers to use something else to manage certificates in their unit tests.
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
Michał Górny wrote: > I'm seriously wondering why I'm wasting so much effort on open source. Open source only ever works when taking responsibility for one's problems. > I don't see a good way out of it. I see a couple. Of course all require some effort. One was already mentioned; move Gentoo package management from Python to some compiled language. High effort but maximum independence/reward. Another option, less effort and less reward, is to investigate how much CPython portage in fact requires, and make that a special package in Gentoo. This essentially means a special-purpose fork of CPython, only for running portage. Obviously portage development must then be comfortable without using the latest shiny Python language stuff that only future RustPython will offer. I guess that's not a problem. Yet another is a variant on the previous, but even less effort and much less reward; freeze what language stuff is allowed in portage code and always run portage with some chosen existing/later CPython version. Like libressl and gtk2 this thread also converges on the common point in my argumentation: it's not per se bad, and sometimes supremely wise, to quit chasing the latest version, and rest on a known platform. Coupled with independent efforts to place security-relevant parts in isolated environments (see sandbox) - ongoing effort regardless of Python - I don't see why portage couldn't depend on a CPython interpreter of its own, some last version that works well and is then copied and renamed. It seems like that would be rather straightforward. It might also be a good thing to take portage out of the overall Gentoo Python picture? I don't know here - this bit is just a guess. > arrogant zealots who want to destroy everything in their path LOL, priceless. > The first big blocker we're going to hit is trustme [3] package that > relies on cryptography API pretty heavily to generate TLS certs for > testing. For which testing? Could it be changed to generate certs differently? CAs aren't magic. Attached is a basic script of mine. //Peter mkca.sh Description: Bourne shell script
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Wed, 2021-02-10 at 08:51 +0800, Benda Xu wrote: > > > > The first big blocker we're going to hit is trustme [3] package that > > relies on cryptography API pretty heavily to generate TLS certs for > > testing. If we managed to convince upstream to support an alternate > > crypto backend, we'd be able to retain minor keywords a lot of packages > > without too much pain. > > I could feel the pain. > > Bootstraping Rust on Prefix is somewhere between alpha, hppa, ia64, > m68k, s390 and amd64[1]. The problem was exposed by > gnome-base/librsvg[2]. > > I am wondering how useable pkgcore is on alpha, hppa, etc. Maybe it's > time for us to plan for a Gentoo without essential Python dependency. It's not usable anywhere. We keep updating the PMS, and the council keeps voting to approve the new versions, and we teach all new developers that they need to respect both the PMS and council decisions... and then from that day on, everyone completely, publicly, ignores it, adding thousands of packages across entire ecosystems that don't work properly without portage. I'd like to say it's one of the craziest things I've ever seen, but, there was 2020. We "started" with three package managers, and now we're down to one. The council needs to grow some balls and enforce the PMS before that can change. Pkgcore could be salvaged if you could actually update your system with it. For my "me too," I've been told by upstream that the next major version of clamav will require rust. There are a lot of "real" UNIX machines relying on clamav for e.g. PCI compliance that will be screwed by that, not to mention all of the small business routers and mail gateways on obscure or limited hardware. Personally, I just haven't spent the last 20 years contributing to free software to be casually migrated to a system of bundled binary blobs; nor am I able to set aside 8GB of RAM for hours to rebuild the latest version of rust and its myriad unofficial dependencies every day on a production mail server. Eventually clamav will disappear, and we'll likely move a few big customers to Microsoft O365 as a result.
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On 2021-02-09 Tue 17:51, Benda Xu wrote: > I am wondering how useable pkgcore is on alpha, hppa, etc. Maybe it's > time for us to plan for a Gentoo without essential Python dependency. Just to keep misinformation down, pkgcore currently has nothing to do with rust as it's implemented in python due to basically being deemed portage-ng when it forked ~15 years ago. It previously did have some extensions written in C which have mostly been removed in the current day from CPython catching up in a number of areas. That being said, alternative languages and related support has been looked at for a number of reasons/features and development could move in that direction, but hasn't yet in a public fashion. Tim
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
Hi Michał, Michał Górny writes: > So it seems that upstream has practically closed the discussion, > and the short summary is that they only care for the 'majority' of > users, they don't care for minor platforms (but we're free to port > LLVM/Rust to them) and -- unsurprisingly -- this is a part of crusade > towards promoting Rust. > > Given the aggressive opinions of a number of Python core devs > participating in the discussion, I'm afraid that it is quite probable > that a future version of CPython may require Rust. In fact, they've > already started having knee-jerk reactions to the problem at hand [1]. > To be honest, I've never thought I'd be this disappointed in Python > upstream. > > Good news is that they've promised to keep a LTS branch with security > fixes to the non-Rust version. Until end-of-year. And they've pretty > aggressively stated that they won't fix anything except security bugs > with a CVE assigned. So if it stops building for whatever reason, we're > on our own. > > I've reached out to Debian and they're planning to remove support for > minor architectures for this package in the next release. However, > Python is not as central to them as it is to us. Alpine is also > affected but seems intent on pushing Rust forward, so they'll probably > drop these architectures as well. > > Mike's submitted a PR to remove (unnecessary) cryptography dep from our > urllib3/requests packages [2]. This should make it possible to avoid > cryptography at least on some systems. However, it is still an indirect > test dependency of these packages, so we're going to have a hard time > keeping them properly tested. > > At this point, I'm really depressed about this and I'm seriously > wondering why I'm wasting so much effort on open source. I don't see > a good way out of it. Rust could be a nice language -- but it won't if > it continues to be surround by arrogant zealots who want to destroy > everything in their path towards promoting it. > > The first big blocker we're going to hit is trustme [3] package that > relies on cryptography API pretty heavily to generate TLS certs for > testing. If we managed to convince upstream to support an alternate > crypto backend, we'd be able to retain minor keywords a lot of packages > without too much pain. I could feel the pain. Bootstraping Rust on Prefix is somewhere between alpha, hppa, ia64, m68k, s390 and amd64[1]. The problem was exposed by gnome-base/librsvg[2]. I am wondering how useable pkgcore is on alpha, hppa, etc. Maybe it's time for us to plan for a Gentoo without essential Python dependency. Looking forward to gcc-rust for saving the world in the end. Benda 1. https://bugs.gentoo.org/689160 2. https://bugs.gentoo.org/739574
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, 2021-02-08 at 12:19 +0100, Michał Górny wrote: > FYI the developers of dev-python/cryptography decided that Rust is going > to be mandatory for 1.5+ versions. It's unlikely that they're going to > provide LTS support or security fixes for the old versions. > > Since cryptography is a very important package in the Python ecosystem, > and it is an indirect dependency of Portage, this means that we will > probably have to entirely drop support for architectures that are not > supported by Rust. > > I...] > > I've raised a protest on the cryptography bug tracker [2] but apparently > upstream considers Rust's 'memory safety' more important than ability to > actually use the package. > > Honestly, I don't think it likely that Rust will gain support for these > platforms. This involves a lot of work, starting with writing a new > LLVM backend and getting it accepted (getting new code into LLVM is very > hard unless you're doing that on behalf one of the big companies). You > can imagine how much effort that involves compared to rewriting the new > code from Cryptography into C. > > If we can't convince upstream, I'm afraid we'll either have to drop > these architectures entirely or fork Cryptography. > > > [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html > [2] https://github.com/pyca/cryptography/issues/5771 So it seems that upstream has practically closed the discussion, and the short summary is that they only care for the 'majority' of users, they don't care for minor platforms (but we're free to port LLVM/Rust to them) and -- unsurprisingly -- this is a part of crusade towards promoting Rust. Given the aggressive opinions of a number of Python core devs participating in the discussion, I'm afraid that it is quite probable that a future version of CPython may require Rust. In fact, they've already started having knee-jerk reactions to the problem at hand [1]. To be honest, I've never thought I'd be this disappointed in Python upstream. Good news is that they've promised to keep a LTS branch with security fixes to the non-Rust version. Until end-of-year. And they've pretty aggressively stated that they won't fix anything except security bugs with a CVE assigned. So if it stops building for whatever reason, we're on our own. I've reached out to Debian and they're planning to remove support for minor architectures for this package in the next release. However, Python is not as central to them as it is to us. Alpine is also affected but seems intent on pushing Rust forward, so they'll probably drop these architectures as well. Mike's submitted a PR to remove (unnecessary) cryptography dep from our urllib3/requests packages [2]. This should make it possible to avoid cryptography at least on some systems. However, it is still an indirect test dependency of these packages, so we're going to have a hard time keeping them properly tested. At this point, I'm really depressed about this and I'm seriously wondering why I'm wasting so much effort on open source. I don't see a good way out of it. Rust could be a nice language -- but it won't if it continues to be surround by arrogant zealots who want to destroy everything in their path towards promoting it. The first big blocker we're going to hit is trustme [3] package that relies on cryptography API pretty heavily to generate TLS certs for testing. If we managed to convince upstream to support an alternate crypto backend, we'd be able to retain minor keywords a lot of packages without too much pain. [1] https://bugs.python.org/issue43179 [2] https://github.com/gentoo/gentoo/pull/19383 [3] https://github.com/python-trio/trustme -- Best regards, Michał Górny
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, Feb 8, 2021 at 1:35 PM Brian Evans wrote: > > On 2/8/2021 12:53 PM, Brian Evans wrote: > > On 2/8/2021 6:19 AM, Michał Górny wrote: > >> Hi, > >> > >> FYI the developers of dev-python/cryptography decided that Rust is going > >> to be mandatory for 1.5+ versions. It's unlikely that they're going to > >> provide LTS support or security fixes for the old versions. > >> > >> Since cryptography is a very important package in the Python ecosystem, > >> and it is an indirect dependency of Portage, this means that we will > >> probably have to entirely drop support for architectures that are not > >> supported by Rust. > >> > > > > For the portage indirect dependency, can it be swapped for pycurl? > > > > AFAICT, it is just used to pull GPG sigs in gemato via dev-python/requests. > > > > This at least would at least keep the arches in question with some > > support but not necessarily all of python world until a clearer plan can > > be made. > > > > Brian > > > > After discussion in #gentoo-dev and simple chroot testing, it seems like > dev-python/requests nor dev-python/urlllib3 needs > dev-python/cryptography at all (when run with stable Python, tested with > 3.8). So unless there's a really good reason outside of gemato sync, > this looks to be a non-issue for portage and more of a dependency fix. I created a PR to drop this and related deps from dev-python/urllib3 and dev-python/requests. Please review. https://github.com/gentoo/gentoo/pull/19383
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, Feb 08, 2021 at 10:02:43AM -0800, Alec Warner wrote: > On Mon, Feb 8, 2021 at 9:56 AM Alessandro Barbieri > wrote: > > > > Il Lun 8 Feb 2021, 12:19 Michał Górny ha scritto: > >> > > > > Should we shed tears for those legacy architectures or move forward? Does > > anyone really use them in production? > > Many users don't use gentoo in 'production' so I'm not sure how that > matters in our decision making. I'm not sure the trade off here > (taking rust as a core dep) is reasonable and it looks like we can > avoid it. > Do we have anything to support this claim? -Aaron signature.asc Description: PGP signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, 2021-02-08 at 18:56 +0100, Alessandro Barbieri wrote: > Il Lun 8 Feb 2021, 12:19 Michał Górny ha scritto: > > > Hi, > > > > FYI the developers of dev-python/cryptography decided that Rust is going > > to be mandatory for 1.5+ versions. It's unlikely that they're going to > > provide LTS support or security fixes for the old versions. > > > > Since cryptography is a very important package in the Python ecosystem, > > and it is an indirect dependency of Portage, this means that we will > > probably have to entirely drop support for architectures that are not > > supported by Rust. > > > > According to upstream platform support information [1], this probably > > means (eventually) entirely removing the following architectures: > > - alpha (stable) > > - hppa (stable) > > - ia64 (stable) > > - m68k (exp) > > - s390 (except for s390x, exp) > > > > Furthermore, the Gentoo Rust packages are missing support > > for the following platforms, apparently supported upstream: > > - mips (exp) > > - ppc (32) (stable) > > - sparc (stable) > > - s390x (exp) > > - riscv (stable) > > > > Apparently it's non-trivial to bootstrap Rust on these platforms, > > so it's unclear when Gentoo is going to start providing Rust on them. > > > > I've raised a protest on the cryptography bug tracker [2] but apparently > > upstream considers Rust's 'memory safety' more important than ability to > > actually use the package. > > > > Honestly, I don't think it likely that Rust will gain support for these > > platforms. This involves a lot of work, starting with writing a new > > LLVM backend and getting it accepted (getting new code into LLVM is very > > hard unless you're doing that on behalf one of the big companies). You > > can imagine how much effort that involves compared to rewriting the new > > code from Cryptography into C. > > > > If we can't convince upstream, I'm afraid we'll either have to drop > > these architectures entirely or fork Cryptography. > > > > > > [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html > > [2] https://github.com/pyca/cryptography/issues/5771 > > > > -- > > Best regards, > > Michał Górny > > > > Should we shed tears for those legacy architectures or move forward? Does > anyone really use them in production? I'm pretty sure people do use them in production. Whether we want to really continue supporting them is a separate topic. But the last thing I'd like to do is drop support for arches because of a single upstream maintainer being a Rust lover. -- Best regards, Michał Górny
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On 2/8/2021 12:53 PM, Brian Evans wrote: On 2/8/2021 6:19 AM, Michał Górny wrote: Hi, FYI the developers of dev-python/cryptography decided that Rust is going to be mandatory for 1.5+ versions. It's unlikely that they're going to provide LTS support or security fixes for the old versions. Since cryptography is a very important package in the Python ecosystem, and it is an indirect dependency of Portage, this means that we will probably have to entirely drop support for architectures that are not supported by Rust. For the portage indirect dependency, can it be swapped for pycurl? AFAICT, it is just used to pull GPG sigs in gemato via dev-python/requests. This at least would at least keep the arches in question with some support but not necessarily all of python world until a clearer plan can be made. Brian After discussion in #gentoo-dev and simple chroot testing, it seems like dev-python/requests nor dev-python/urlllib3 needs dev-python/cryptography at all (when run with stable Python, tested with 3.8). So unless there's a really good reason outside of gemato sync, this looks to be a non-issue for portage and more of a dependency fix. Brian OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, 2021-02-08 at 12:53 -0500, Brian Evans wrote: > On 2/8/2021 6:19 AM, Michał Górny wrote: > > Hi, > > > > FYI the developers of dev-python/cryptography decided that Rust is going > > to be mandatory for 1.5+ versions. It's unlikely that they're going to > > provide LTS support or security fixes for the old versions. > > > > Since cryptography is a very important package in the Python ecosystem, > > and it is an indirect dependency of Portage, this means that we will > > probably have to entirely drop support for architectures that are not > > supported by Rust. > > > > For the portage indirect dependency, can it be swapped for pycurl? > > AFAICT, it is just used to pull GPG sigs in gemato via dev-python/requests. I'll probably switch it to built-in urllib.request. The py3 versions should be good enough (previously urllib3 had better TLS support than old urllib AFAIR). -- Best regards, Michał Górny
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, Feb 8, 2021 at 9:56 AM Alessandro Barbieri wrote: > > Il Lun 8 Feb 2021, 12:19 Michał Górny ha scritto: >> >> Hi, >> >> FYI the developers of dev-python/cryptography decided that Rust is going >> to be mandatory for 1.5+ versions. It's unlikely that they're going to >> provide LTS support or security fixes for the old versions. >> >> Since cryptography is a very important package in the Python ecosystem, >> and it is an indirect dependency of Portage, this means that we will >> probably have to entirely drop support for architectures that are not >> supported by Rust. >> >> According to upstream platform support information [1], this probably >> means (eventually) entirely removing the following architectures: >> - alpha (stable) >> - hppa (stable) >> - ia64 (stable) >> - m68k (exp) >> - s390 (except for s390x, exp) >> >> Furthermore, the Gentoo Rust packages are missing support >> for the following platforms, apparently supported upstream: >> - mips (exp) >> - ppc (32) (stable) >> - sparc (stable) >> - s390x (exp) >> - riscv (stable) >> >> Apparently it's non-trivial to bootstrap Rust on these platforms, >> so it's unclear when Gentoo is going to start providing Rust on them. >> >> I've raised a protest on the cryptography bug tracker [2] but apparently >> upstream considers Rust's 'memory safety' more important than ability to >> actually use the package. >> >> Honestly, I don't think it likely that Rust will gain support for these >> platforms. This involves a lot of work, starting with writing a new >> LLVM backend and getting it accepted (getting new code into LLVM is very >> hard unless you're doing that on behalf one of the big companies). You >> can imagine how much effort that involves compared to rewriting the new >> code from Cryptography into C. >> >> If we can't convince upstream, I'm afraid we'll either have to drop >> these architectures entirely or fork Cryptography. >> >> >> [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html >> [2] https://github.com/pyca/cryptography/issues/5771 >> >> -- >> Best regards, >> Michał Górny > > > Should we shed tears for those legacy architectures or move forward? Does > anyone really use them in production? Many users don't use gentoo in 'production' so I'm not sure how that matters in our decision making. I'm not sure the trade off here (taking rust as a core dep) is reasonable and it looks like we can avoid it. -A
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On 2/8/21 7:53 PM, Brian Evans wrote: > > AFAICT, it is just used to pull GPG sigs in gemato via > dev-python/requests. > And it that's true, isn't this only relevant when syncing the tree using rsync? So using git would still be viable? -- juippis OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
Il Lun 8 Feb 2021, 12:19 Michał Górny ha scritto: > Hi, > > FYI the developers of dev-python/cryptography decided that Rust is going > to be mandatory for 1.5+ versions. It's unlikely that they're going to > provide LTS support or security fixes for the old versions. > > Since cryptography is a very important package in the Python ecosystem, > and it is an indirect dependency of Portage, this means that we will > probably have to entirely drop support for architectures that are not > supported by Rust. > > According to upstream platform support information [1], this probably > means (eventually) entirely removing the following architectures: > - alpha (stable) > - hppa (stable) > - ia64 (stable) > - m68k (exp) > - s390 (except for s390x, exp) > > Furthermore, the Gentoo Rust packages are missing support > for the following platforms, apparently supported upstream: > - mips (exp) > - ppc (32) (stable) > - sparc (stable) > - s390x (exp) > - riscv (stable) > > Apparently it's non-trivial to bootstrap Rust on these platforms, > so it's unclear when Gentoo is going to start providing Rust on them. > > I've raised a protest on the cryptography bug tracker [2] but apparently > upstream considers Rust's 'memory safety' more important than ability to > actually use the package. > > Honestly, I don't think it likely that Rust will gain support for these > platforms. This involves a lot of work, starting with writing a new > LLVM backend and getting it accepted (getting new code into LLVM is very > hard unless you're doing that on behalf one of the big companies). You > can imagine how much effort that involves compared to rewriting the new > code from Cryptography into C. > > If we can't convince upstream, I'm afraid we'll either have to drop > these architectures entirely or fork Cryptography. > > > [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html > [2] https://github.com/pyca/cryptography/issues/5771 > > -- > Best regards, > Michał Górny > Should we shed tears for those legacy architectures or move forward? Does anyone really use them in production? >
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On 2/8/2021 6:19 AM, Michał Górny wrote: Hi, FYI the developers of dev-python/cryptography decided that Rust is going to be mandatory for 1.5+ versions. It's unlikely that they're going to provide LTS support or security fixes for the old versions. Since cryptography is a very important package in the Python ecosystem, and it is an indirect dependency of Portage, this means that we will probably have to entirely drop support for architectures that are not supported by Rust. For the portage indirect dependency, can it be swapped for pycurl? AFAICT, it is just used to pull GPG sigs in gemato via dev-python/requests. This at least would at least keep the arches in question with some support but not necessarily all of python world until a clearer plan can be made. Brian OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On 2/8/21 2:27 PM, Michael Orlitzky wrote: > Not only that, but you will be dropping support for at least two of my > machines that are literally incapable of building the 10+ GiB bundled > rust package due to the amount of disk space and RAM required. > Pardon my intervention, but there is rust-bin available at least for amd64, x86, arm, arm64 and ppc64. Just a PSA to anyone not aware. -- juippis OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, 2021-02-08 at 12:19 +0100, Michał Górny wrote: > > Since cryptography is a very important package in the Python ecosystem, > and it is an indirect dependency of Portage, this means that we will > probably have to entirely drop support for architectures that are not > supported by Rust. > Not only that, but you will be dropping support for at least two of my machines that are literally incapable of building the 10+ GiB bundled rust package due to the amount of disk space and RAM required.
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
> On 8 Feb 2021, at 11:19, Michał Górny wrote: > > Hi, > > FYI the developers of dev-python/cryptography decided that Rust is going > to be mandatory for 1.5+ versions. It's unlikely that they're going to > provide LTS support or security fixes for the old versions. > > Since cryptography is a very important package in the Python ecosystem, > and it is an indirect dependency of Portage, this means that we will > probably have to entirely drop support for architectures that are not > supported by Rust. > It seems that there was no mention of this in release notes or the changelog - neither to warn packagers or consult with them for their views on this, which would’ve been the best place to do so. > [snip] > > Furthermore, the Gentoo Rust packages are missing support > for the following platforms, apparently supported upstream: > - mips (exp) > - ppc (32) (stable) > - sparc (stable) > - s390x (exp) > - riscv (stable) We also need to add support for armv4/armv5. > > Apparently it's non-trivial to bootstrap Rust on these platforms, > so it's unclear when Gentoo is going to start providing Rust on them. > > [snip] Filed a Gentoo bug too: https://bugs.gentoo.org/769482. > [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html > [2] https://github.com/pyca/cryptography/issues/5771 > > -- > Best regards, > Michał Górny > > > signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
On Mon, 08 Feb 2021 12:19:13 +0100 Michał Górny wrote: > Hi, > > FYI the developers of dev-python/cryptography decided that Rust is going > to be mandatory for 1.5+ versions. It's unlikely that they're going to > provide LTS support or security fixes for the old versions. > > > > Honestly, I don't think it likely that Rust will gain support for these > platforms. This involves a lot of work, starting with writing a new > LLVM backend and getting it accepted (getting new code into LLVM is very > hard unless you're doing that on behalf one of the big companies). You > can imagine how much effort that involves compared to rewriting the new > code from Cryptography into C. I do know that there is ongoing work to get Rust going on m68k but it's very slow and I'm not entirely confident they'll get there. At least I don't have this package on my m68k system! -- James Le Cuirot (chewi) Gentoo Linux Developer pgpEXub3MDL7c.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
El lun, 08-02-2021 a las 12:19 +0100, Michał Górny escribió: > Hi, > > FYI the developers of dev-python/cryptography decided that Rust is going > to be mandatory for 1.5+ versions. It's unlikely that they're going to > provide LTS support or security fixes for the old versions. > > Since cryptography is a very important package in the Python ecosystem, > and it is an indirect dependency of Portage, this means that we will > probably have to entirely drop support for architectures that are not > supported by Rust. > > According to upstream platform support information [1], this probably > means (eventually) entirely removing the following architectures: > - alpha (stable) > - hppa (stable) > - ia64 (stable) > - m68k (exp) > - s390 (except for s390x, exp) > > Furthermore, the Gentoo Rust packages are missing support > for the following platforms, apparently supported upstream: > - mips (exp) > - ppc (32) (stable) > - sparc (stable) > - s390x (exp) > - riscv (stable) > > Apparently it's non-trivial to bootstrap Rust on these platforms, > so it's unclear when Gentoo is going to start providing Rust on them. > > I've raised a protest on the cryptography bug tracker [2] but apparently > upstream considers Rust's 'memory safety' more important than ability to > actually use the package. > > Honestly, I don't think it likely that Rust will gain support for these > platforms. This involves a lot of work, starting with writing a new > LLVM backend and getting it accepted (getting new code into LLVM is very > hard unless you're doing that on behalf one of the big companies). You > can imagine how much effort that involves compared to rewriting the new > code from Cryptography into C. > > If we can't convince upstream, I'm afraid we'll either have to drop > these architectures entirely or fork Cryptography. > > > [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html > [2] https://github.com/pyca/cryptography/issues/5771 > It will also affect non-SSE2 x86 setups as upstream is neither providing prebuilt package for them :/ https://bugs.gentoo.org/741708 signature.asc Description: This is a digitally signed message part
[gentoo-dev] dev-python/cryptography to use rust, effectively killing alpha, hppa, ia64, m68k, s390
Hi, FYI the developers of dev-python/cryptography decided that Rust is going to be mandatory for 1.5+ versions. It's unlikely that they're going to provide LTS support or security fixes for the old versions. Since cryptography is a very important package in the Python ecosystem, and it is an indirect dependency of Portage, this means that we will probably have to entirely drop support for architectures that are not supported by Rust. According to upstream platform support information [1], this probably means (eventually) entirely removing the following architectures: - alpha (stable) - hppa (stable) - ia64 (stable) - m68k (exp) - s390 (except for s390x, exp) Furthermore, the Gentoo Rust packages are missing support for the following platforms, apparently supported upstream: - mips (exp) - ppc (32) (stable) - sparc (stable) - s390x (exp) - riscv (stable) Apparently it's non-trivial to bootstrap Rust on these platforms, so it's unclear when Gentoo is going to start providing Rust on them. I've raised a protest on the cryptography bug tracker [2] but apparently upstream considers Rust's 'memory safety' more important than ability to actually use the package. Honestly, I don't think it likely that Rust will gain support for these platforms. This involves a lot of work, starting with writing a new LLVM backend and getting it accepted (getting new code into LLVM is very hard unless you're doing that on behalf one of the big companies). You can imagine how much effort that involves compared to rewriting the new code from Cryptography into C. If we can't convince upstream, I'm afraid we'll either have to drop these architectures entirely or fork Cryptography. [1] https://doc.rust-lang.org/nightly/rustc/platform-support.html [2] https://github.com/pyca/cryptography/issues/5771 -- Best regards, Michał Górny