Re: [gentoo-dev] moving kernel config checks forward: potential config checking tool
On Mon, Sep 27, 2021 at 11:47:38PM +0200, Michał Górny wrote: > > Can we consider moving the checks for set A somewhere else, such that we > > don't check the kernel config during package compile & install time, but > > only check it later? This also meaningfully resolves that cases where > > the system that has package building isn't where the packages are being > > used. > I'm not sure if I understand you correctly but if you mean not doing > checks before compiling/installing, then I have to disagree. There is > value in knowing about this kind of problems early (hey, that's why we > have pkg_pretend in the first place!) Ebuilds should be able to call the tool (but it could be made optional easily), which does the checks MORE efficiently than the present eclass code. The ebuilds would be responsible for suitable warnings or failures based on the tool's output. E.g. maybe you're in a rescue environment and you know the tooling will work fine on your final environment. > There's certainly value in knowing 'I need to rebuild my kernel' early > vs learning only after you've spent significant time waiting for some > package to build. One thing to this is if you're doing pkg_pretend for multiple packages in a single emerge call, the tool could greatly amortize the cost of the checks, as well as having them available after merge. Great thought I had would be this tool could ALSO run on boot and warn if some packages are unlikely to work -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature
[gentoo-dev] RFC: dev-libs/openssl USE=bindist removal
Deadline for responses: 2021/10/14! The Foundation would like to propose that RedHat/Fedora "hobble" patch presently applied when USE=bindist is true shall be removed from dev-libs/openssl. RedHat's stated reasons for the patch were originally to avoid any patent concerns, but they have also morphed over time to present some "insecure" things from being used entirely: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-the-system-wide-cryptographic-policies_security-hardening "All ECC curves < 224 bits (since RHEL 6)" "All binary field ECC curves (since RHEL 6)" However, the Foundation would also like to be sure that no users feel that patchset provides something critical to their usage of Gentoo. If nobody speaks up as saying that the "hobble" patch is REQUIRED for their use cases, the Foundation proposes that usage of the patchset be dropped from the main tree. Any users who might be concerned about patent compliance are encouraged to do their own due diligence, as OpenSSL was the only Gentoo package that shipped this type of patch, and even Fedora's upstream did not completely patch out EC in other packages. Below shows which EC curves are present in major distributions. - RHEL/Fedora is the most restrictive list, with only 5 curves kept - OpenSUSE is next, with 41 curves - Gentoo, Debian, Ubuntu all have the same 88 curves available. Fedora # openssl ecparam -list_curves secp224r1 : NIST/SECG curve over a 224 bit prime field secp256k1 : SECG curve over a 256 bit prime field secp384r1 : NIST/SECG curve over a 384 bit prime field secp521r1 : NIST/SECG curve over a 521 bit prime field prime256v1: X9.62/SECG curve over a 256 bit prime field OpenSUSE Leap # openssl ecparam -list_curves secp112r1 : SECG/WTLS curve over a 112 bit prime field secp112r2 : SECG curve over a 112 bit prime field secp128r1 : SECG curve over a 128 bit prime field secp128r2 : SECG curve over a 128 bit prime field secp160k1 : SECG curve over a 160 bit prime field secp160r1 : SECG curve over a 160 bit prime field secp160r2 : SECG/WTLS curve over a 160 bit prime field secp192k1 : SECG curve over a 192 bit prime field secp224k1 : SECG curve over a 224 bit prime field secp224r1 : NIST/SECG curve over a 224 bit prime field secp256k1 : SECG curve over a 256 bit prime field secp384r1 : NIST/SECG curve over a 384 bit prime field secp521r1 : NIST/SECG curve over a 521 bit prime field prime192v1: NIST/X9.62/SECG curve over a 192 bit prime field prime192v2: X9.62 curve over a 192 bit prime field prime192v3: X9.62 curve over a 192 bit prime field prime239v1: X9.62 curve over a 239 bit prime field prime239v2: X9.62 curve over a 239 bit prime field prime239v3: X9.62 curve over a 239 bit prime field prime256v1: X9.62/SECG curve over a 256 bit prime field wap-wsg-idm-ecid-wtls6: SECG/WTLS curve over a 112 bit prime field wap-wsg-idm-ecid-wtls7: SECG/WTLS curve over a 160 bit prime field wap-wsg-idm-ecid-wtls8: WTLS curve over a 112 bit prime field wap-wsg-idm-ecid-wtls9: WTLS curve over a 160 bit prime field wap-wsg-idm-ecid-wtls12: WTLS curve over a 224 bit prime field brainpoolP160r1: RFC 5639 curve over a 160 bit prime field brainpoolP160t1: RFC 5639 curve over a 160 bit prime field brainpoolP192r1: RFC 5639 curve over a 192 bit prime field brainpoolP192t1: RFC 5639 curve over a 192 bit prime field brainpoolP224r1: RFC 5639 curve over a 224 bit prime field brainpoolP224t1: RFC 5639 curve over a 224 bit prime field brainpoolP256r1: RFC 5639 curve over a 256 bit prime field brainpoolP256t1: RFC 5639 curve over a 256 bit prime field brainpoolP320r1: RFC 5639 curve over a 320 bit prime field brainpoolP320t1: RFC 5639 curve over a 320 bit prime field brainpoolP384r1: RFC 5639 curve over a 384 bit prime field brainpoolP384t1: RFC 5639 curve over a 384 bit prime field brainpoolP512r1: RFC 5639 curve over a 512 bit prime field brainpoolP512t1: RFC 5639 curve over a 512 bit prime field SM2 : SM2 curve over a 256 bit prime field Gentoo, Ubuntu, Debian # openssl ecparam -list_curves secp112r1 : SECG/WTLS curve over a 112 bit prime field secp112r2 : SECG curve over a 112 bit prime field secp128r1 : SECG curve over a 128 bit prime field secp128r2 : SECG curve over a 128 bit prime field secp160k1 : SECG curve over a 160 bit prime field secp160r1 : SECG curve over a 160 bit prime field secp160r2 : SECG/WTLS curve over a 160 bit prime field secp192k1 : SECG curve over a 192 bit prime field secp224k1 : SECG curve over a 224 bit prime field secp224r1 : NIST/SECG curve over a 224 bit prime field secp256k1 : SECG curve over a 256 bit prime field secp384r1 : NIST/SECG curve over a 384 bit prime field secp521r1 : NIST/SECG curve over a 521 bit prime field prime192v1: NIST/X9.62/SECG curve over a 192 bit prime field prime192v2: X9.62 curve over a 192 bit prime field prime192v3: X9.62 c
Re: [gentoo-dev] moving kernel config checks forward: potential config checking tool
On Mon, 2021-09-27 at 19:14 +, Robin H. Johnson wrote: > I wanted to break the prior thread to discuss the root issue. > > We have some set of packages (A) which collectively depend on one or > more kernel options being set in specific ways, and the options need to > REMAIN set if you want the packages to continue work. > > There are also a subset of packages (B), usually kernel modules themselves > that will outright fail to compile if specific options are/are not set. > > Can we consider moving the checks for set A somewhere else, such that we > don't check the kernel config during package compile & install time, but > only check it later? This also meaningfully resolves that cases where > the system that has package building isn't where the packages are being > used. > I'm not sure if I understand you correctly but if you mean not doing checks before compiling/installing, then I have to disagree. There is value in knowing about this kind of problems early (hey, that's why we have pkg_pretend in the first place!) There's certainly value in knowing 'I need to rebuild my kernel' early vs learning only after you've spent significant time waiting for some package to build. -- Best regards, Michał Górny
Re: [gentoo-dev] Guidance on distributed patented software
(This is all tangential to the main issue of this thread and just discussing internet history - skip as you wish...) On Mon, Sep 27, 2021 at 2:14 PM Marek Szuba wrote: > > I am no expert on US law but from what I have read (in many different > sources, with me having begun using PGP in either late 1996 or early > 1997 i.e. when it was still very much subject to US export restrictions) > about this case, both the publishing of the source-code book and it > having subsequently been taken out of the country has been legal - the > former due to publishing the first amendment Well, based on the little I know of US export controls, I doubt that being in book form vs some other form really has any bearing in principle. HOWEVER, I think it was probably done specifically as a challenge to the constitutionality of the law. Ie, the argument would be that it ought to be legal to take the source code out in any form. By doing it via a formally published book though they take away all the "exotic internetness" out of the equation and this way all the 60 year old judges (in the 90s) who might get involved are forced to confront it as suppression of book distribution. In principle though I think most of us would agree that there is no difference in sharing information no matter the way in which it is conveyed. It was probably their hope that if it did go to court any ruling that secured the right to distribute via book could then be leveraged against other modes. I'm guessing that it was never challenged in court precisely for this reason. US export controls cover the communication of information via any mode: https://www.bis.doc.gov/index.php/policy-guidance/deemed-exports If they had fought the export of this book it is quite possible that there would have been a ruling that just finds all export controls to be illegal. Really when you think about it any sort of restriction on communication of classified information or whatever is going to run into the 1st amendment. Courts are going to tend to make their rulings on what can be restricted narrow as a result. The government probably prefers to maintain some FUD around where those boundaries lie, to get companies in particular to follow policies that in the end might not be enforceable. (Note I'm not arguing that dissemination of classified info ought to be legal, but as you move away from things like locations of troops or blueprints of specific aircraft or whatever, into more generic topics like entire classes of technology, I think you're going down a slippery slope...) The main reason that all of this went away though was that I think it was Clinton that specifically granted an exception to cryptography software from ITAR. The concerns were that the cat was basically out of the bag anyway, and consumers were potentially going to be harmed by things like web browsers getting distributed with 40-bit key limitations. Sure, secure browsers were also probably available to people who knew the right links to click, but do we really want somebody reputable like Google to end up having to have a link on their website for a non-secure version of their software, and where the non-secure link just gives you a download, while the secure link makes you jump through hoops to verify your location/etc? The popular use of SSL for entering credit card info on e-commerce sites really was what drove it IMO. -- Rich
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
Mike Gilbert wrote: > > > It's a waste of time and effort to pepper random ebuilds with checks > > > for options that everyone should have enabled in the first place. > > > > It's not for you to say what everyone should have enabled in their kernel. > > Do you not agree that there are some options that should always be > enabled, or at least that we can assume are enabled? "Should be enabled" no, but I agree with the latter - some assumed set should be fine. I think it would be good if it's (somehow) documented though. > To use my earlier example, should every package that uses AF_INET > check for CONFIG_INET in the kernel? CONFIG_INET is a perhaps surprisingly tricky example! A package could e.g. use getaddrinfo() with no address family hint but because of USE=-ipv6 exclude all AF_INET6 address results and so end up using AF_INET based on whether it's available in the running kernel or even based on third party DNS entries. I'm not sure about the best approach for very basic options, CONFIG_NET could be another such candidate. Thinking towards robbat2's proposal (which I like) it might make sense to try to map requirements of packages, but there will probably be cases where it can't really be done successfully. Ultimately that work should not be the responsibility of distribution package maintainers but something upstreams deliver, similar to systemd units, but maybe we'll invent it (if noone else has).. //Peter
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 4:28 PM Jason A. Donenfeld wrote: > > On Mon, Sep 27, 2021 at 11:44 AM Robin H. Johnson wrote: > > I think we need to strip out a lot of the crap about trying to detect > > things in the stuff being built, and reduce the check to the simplest > > possible form: > > $ time zgrep -w CONFIG_PACKET /proc/config.gz > > CONFIG_PACKET=y > > The results from our current method could of course be cached, and > even cached between runs, by just relying on `/proc/version` changing > for different kernels. This seems easy enough to do. > > Alternatively, sure, we could require that everyone has /proc/config > in their kernel config. Many kernels don't have this (mine doesn't), > but we could add it to the base requirements. I think it would be reasonable to check a couple fallback locations: /proc/config.gz /lib/modules/$(uname -r)/build/.config /boot/config-$(uname-r) The slowness really comes from guessing the kernel build location and invoking the kernel build system via make. Avoiding that is a big improvement.
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 11:44 AM Robin H. Johnson wrote: > I think we need to strip out a lot of the crap about trying to detect > things in the stuff being built, and reduce the check to the simplest > possible form: > $ time zgrep -w CONFIG_PACKET /proc/config.gz > CONFIG_PACKET=y The results from our current method could of course be cached, and even cached between runs, by just relying on `/proc/version` changing for different kernels. This seems easy enough to do. Alternatively, sure, we could require that everyone has /proc/config in their kernel config. Many kernels don't have this (mine doesn't), but we could add it to the base requirements.
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 3:11 PM Peter Stuge wrote: > > Mike Gilbert wrote: > > It's a waste of time and effort to pepper random ebuilds with checks > > for options that everyone should have enabled in the first place. > > It's not for you to say what everyone should have enabled in their kernel. Do you not agree that there are some options that should always be enabled, or at least that we can assume are enabled? To use my earlier example, should every package that uses AF_INET check for CONFIG_INET in the kernel?
Re: [gentoo-dev] moving kernel config checks forward: potential config checking tool
Robin H. Johnson wrote: > We have some set of packages (A) which collectively depend on one or > more kernel options being set in specific ways, and the options need to > REMAIN set if you want the packages to continue work. .. > Can we consider moving the checks for set A somewhere else, such that we > don't check the kernel config during package compile & install time, but > only check it later? I only see one problem with this; that USE flags could influence which kernel options are required, which would be useful to know at/before compile time. > It would need to keep long-term state about which packages want > specific options set/unset/modular, as well as short-term state > about the config from each boot. It's a cool idea. A standardized solution would be quite nice for the whole Linux ecosystem. //Peter
[gentoo-dev] moving kernel config checks forward: potential config checking tool
I wanted to break the prior thread to discuss the root issue. We have some set of packages (A) which collectively depend on one or more kernel options being set in specific ways, and the options need to REMAIN set if you want the packages to continue work. There are also a subset of packages (B), usually kernel modules themselves that will outright fail to compile if specific options are/are not set. Can we consider moving the checks for set A somewhere else, such that we don't check the kernel config during package compile & install time, but only check it later? This also meaningfully resolves that cases where the system that has package building isn't where the packages are being used. This secondary tooling COULD be called from pkg_setup much less, and could do a much more efficient job of checking the state of multiple flags. At boot, it needs to load the present config into some easy to check for, and then it can be verified against in a lightweight manner. Also a lot easier for users to say "i accept the responsbility of my stuff breaking", AND for users to say "hey, why did package X broken when I rebooted into new kernel" (because some config option changed). It would need to keep long-term state about which packages want specific options set/unset/modular, as well as short-term state about the config from each boot. -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
Mike Gilbert wrote: > It's a waste of time and effort to pepper random ebuilds with checks > for options that everyone should have enabled in the first place. It's not for you to say what everyone should have enabled in their kernel. There's significant value in ebuilds documenting required kernel options in a structured manner. So I welcome simplifications in the checking to achieve millisecond test times. But the benefit of structured requirements is given even without any runtime checking at all, as long as required options continue to be codified /somehow/. Thank you for your work! //Peter
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 1:56 PM Rich Freeman wrote: > > On Mon, Sep 27, 2021 at 1:44 PM Robin H. Johnson wrote: > > > > I think we need to strip out a lot of the crap about trying to detect > > things in the stuff being built, and reduce the check to the simplest > > possible form: > > $ time zgrep -w CONFIG_PACKET /proc/config.gz > > CONFIG_PACKET=y > > > > Sure, just please don't strip out the part that actually does what you > just did - check the running kernel. I don't think we should assume > that the user has the configured+prepared sources for a kernel just > lying around all the time. I don't build in /usr/src (which is > apparently discouraged by upstream anyway), so /proc/config.gz is > really the only reliable indication of how things are setup. Well, > that or a config file in /boot which I'm sure others don't use (but > make install puts it there). I like the idea of a stripped-down implementation that just checks the running kernel config when packages get installed. I think that probably means creating a new eclass though, or moving some of the current complexity into linux-mod.eclass.
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 12:23 PM Mike Pagano wrote: > > On 9/27/21 12:10 PM, Mike Gilbert wrote: > > I'm looking to solicit opinions on when it is appropriate for an > > ebuild to check for kernel config options using linux-info.eclass. I > > don't think we have any guidelines documented, instead leaving it up > > to the "common sense" of package maintainers. > > > > Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns > > when running emerge, so we should do so only when there is a > > compensating benefit. It doesn't make sense to check for kernel > > options that are very commonly enabled. But what is "very common"? > > > > An obvious example would be CONFIG_INET, which controls IPv4 support > > in the kernel. It does not make sense to check for that in every > > package that uses AF_INET sockets. > > > > A less obvious example: a user filed a bug against net-misc/dhcpcd > > today asking that we check for CONFIG_PACKET [1]. My first thought was > > "why would you ever disable that?". The option description even says > > "if unsure, say Y". However, I suppose it is technically possible to > > run a Linux system with it disabled. > > > > I think a reasonable rule of thumb would be to assume we can rely on > > options that are enabled by "make defconfig". If the user chooses to > > disable them, they are responsible for anything that breaks. > > > > Thoughts? > > > > [1] https://bugs.gentoo.org/815064 > > > > The challenge I see is that these config checks head off bugs and issues > without our intervention. > > We as kernel maintainers depend on ebuild maintainers to check these things > so they don't become "kernel bugs" to figure out. I guess I'm proposing that we set some minimal standard for kernel configs. It's a waste of time and effort to pepper random ebuilds with checks for options that everyone should have enabled in the first place.
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On 9/27/21 2:15 PM, Mike Gilbert wrote: On Mon, Sep 27, 2021 at 1:44 PM Robin H. Johnson wrote: On Mon, Sep 27, 2021 at 01:15:10PM -0400, Mike Gilbert wrote: On Mon, Sep 27, 2021 at 12:23 PM Mike Pagano wrote: Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns when running emerge, so we should do so only when there is a compensating benefit. Is this a significant slowdown? Do you have any numbers? Adding a check for CONFIG_PACKET to the dhcpcd ebuild adds around 7 seconds of delay time to the pkg_setup and/or pkg_pretend phase on my system. That's ok if a small number of packages are doing it, but it would become quite annoying if a significant number of them get queued up. 7 seconds is ridiculous. I think the horribly slow performance is mainly due to this change in how we detect the kernel version: https://gitweb.gentoo.org/repo/gentoo.git/commit/eclass/linux-info.eclass?id=d1ea596f034285cd044006b107a60902ea059793 That causes "make" to be called 4 times, and each make invocation adds a couple of seconds to the get_version call. Changes to these eclasses are like a prison of circular breakage for which there is no escape.
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 1:44 PM Robin H. Johnson wrote: > > On Mon, Sep 27, 2021 at 01:15:10PM -0400, Mike Gilbert wrote: > > On Mon, Sep 27, 2021 at 12:23 PM Mike Pagano wrote: > > > > Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns > > > > when running emerge, so we should do so only when there is a > > > > compensating benefit. > > > > > > Is this a significant slowdown? Do you have any numbers? > > > > Adding a check for CONFIG_PACKET to the dhcpcd ebuild adds around 7 > > seconds of delay time to the pkg_setup and/or pkg_pretend phase on my > > system. > > > > That's ok if a small number of packages are doing it, but it would > > become quite annoying if a significant number of them get queued up. > 7 seconds is ridiculous. I think the horribly slow performance is mainly due to this change in how we detect the kernel version: https://gitweb.gentoo.org/repo/gentoo.git/commit/eclass/linux-info.eclass?id=d1ea596f034285cd044006b107a60902ea059793 That causes "make" to be called 4 times, and each make invocation adds a couple of seconds to the get_version call.
Re: [gentoo-dev] Guidance on distributed patented software
On 2021-09-26 21:20, Rich Freeman wrote: Back in the PGP ITAR days I believe somebody went through some loopholes to publish the software outside the US, Yes, PGP 2.6 source code got published as an OCR-friendly book (https://dl.acm.org/doi/book/10./207390) which was then legally taken from the US abroad. and it is probably debatable whether that was legal under US law, I am no expert on US law but from what I have read (in many different sources, with me having begun using PGP in either late 1996 or early 1997 i.e. when it was still very much subject to US export restrictions) about this case, both the publishing of the source-code book and it having subsequently been taken out of the country has been legal - the former due to publishing the first amendment and the second due to the scope of ITAR as far as crypto software was concerned. but presumably the people who did it didn't care, and enforcement was unlikely at all, and especially unlikely if you didn't have plans to visit the US after bragging about distributing it. I don't know if Ståle Schumacher (the person who scanned the book and subsequently published "international" versions of PGP 2 in Norway) ever visited the US afterwards. On the other hand the source-code book itself, the purpose of which was rather clear given it even contained notes on how to OCR it, was written by a US person (Phil Zimmermann himself) and published by a US company (MIT Press) - so I am not quite convinced they either thought they would be our of reach of US law (indeed, wasn't PRZ still being persecuted by US Customs at the time?), or didn't care. Not that any of this changes the point you have tried to make regarding due diligence, mind you. -- Marecki
Re: [gentoo-dev] [PATCH 1/2] bzr.eclass: Reinstate eclass
On Mon, 2021-09-27 at 20:01 +0200, Ulrich Mueller wrote: > > > > > > On Mon, 27 Sep 2021, Michał Górny wrote: > > > You seem to be missing the fact that all of them hardcode "bzr". > > Yes, and it is updated to "brz" in a followup commit. Would you prefer > leaving them as "bzr"? I guess that backwards compatibility link will > stay there for a long time. What I'm saying is that if anyone overrides these commands, they will be broken when the symlink disappears. Then ofc there's the basic problem if anyone will ever actually need to do that. This doesn't sound likely for eclass that we expect to have approximately one consumer. -- Best regards, Michał Górny
Re: [gentoo-dev] [PATCH 1/2] bzr.eclass: Reinstate eclass
> On Mon, 27 Sep 2021, Michał Górny wrote: > You seem to be missing the fact that all of them hardcode "bzr". Yes, and it is updated to "brz" in a followup commit. Would you prefer leaving them as "bzr"? I guess that backwards compatibility link will stay there for a long time. signature.asc Description: PGP signature
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 1:44 PM Robin H. Johnson wrote: > > I think we need to strip out a lot of the crap about trying to detect > things in the stuff being built, and reduce the check to the simplest > possible form: > $ time zgrep -w CONFIG_PACKET /proc/config.gz > CONFIG_PACKET=y > Sure, just please don't strip out the part that actually does what you just did - check the running kernel. I don't think we should assume that the user has the configured+prepared sources for a kernel just lying around all the time. I don't build in /usr/src (which is apparently discouraged by upstream anyway), so /proc/config.gz is really the only reliable indication of how things are setup. Well, that or a config file in /boot which I'm sure others don't use (but make install puts it there). -- Rich
Re: [gentoo-dev] [PATCH 1/2] bzr.eclass: Reinstate eclass
On Mon, 2021-09-27 at 19:25 +0200, Ulrich Mueller wrote:
> > > > > > On Sat, 25 Sep 2021, Michał Górny wrote:
>
> > > +EBZR="bzr.eclass"
>
> > Why do we need this? It seems as if someone is reinventing
> > ${ECLASS}.
>
> Replaced by ${ECLASS}.
>
> > > +# @ECLASS-VARIABLE: EBZR_STORE_DIR
>
> > @USER_VARIABLE?
>
> Added.
>
> > > +# @DESCRIPTION:
> > > +# The directory to store all fetched Bazaar live sources.
> > > +: ${EBZR_STORE_DIR:=${PORTAGE_ACTUAL_DISTDIR:-${DISTDIR}}/bzr-
> > > src}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_UNPACK_DIR
> > > +# @DESCRIPTION:
> > > +# The working directory where the sources are copied to.
> > > +: ${EBZR_UNPACK_DIR:=${WORKDIR}/${P}}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_INIT_REPO_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to initialise a shared repository.
> > > +: ${EBZR_INIT_REPO_CMD:="bzr init-repository --no-trees"}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_FETCH_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to fetch the sources.
> > > +: ${EBZR_FETCH_CMD:="bzr branch --no-tree"}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_UPDATE_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to update the sources.
> > > +: ${EBZR_UPDATE_CMD:="bzr pull --overwrite-tags"}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_EXPORT_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to export a branch.
> > > +: ${EBZR_EXPORT_CMD:="bzr export"}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_CHECKOUT_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to checkout a branch.
> > > +: ${EBZR_CHECKOUT_CMD:="bzr checkout --lightweight -q"}
> > > +
> > > +# @ECLASS-VARIABLE: EBZR_REVNO_CMD
> > > +# @DESCRIPTION:
> > > +# The Bazaar command to list a revision number of the branch.
> > > +: ${EBZR_REVNO_CMD:="bzr revno"}
>
> > Are you sure that having these overrides is a good idea?
>
> Yes. Ebuilds had used them in the past.
>
> > Your followup patch pretty much proves that every ebuild redefining
> > them would get broken by switch to breezy.
>
> The only one where syntax has changed is EBZR_INIT_REPO_CMD (which has
> init-shared-repository instead of init-repository).
You seem to be missing the fact that all of them hardcode "bzr".
--
Best regards,
Michał Górny
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 01:15:10PM -0400, Mike Gilbert wrote: > On Mon, Sep 27, 2021 at 12:23 PM Mike Pagano wrote: > > > Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns > > > when running emerge, so we should do so only when there is a > > > compensating benefit. > > > > Is this a significant slowdown? Do you have any numbers? > > Adding a check for CONFIG_PACKET to the dhcpcd ebuild adds around 7 > seconds of delay time to the pkg_setup and/or pkg_pretend phase on my > system. > > That's ok if a small number of packages are doing it, but it would > become quite annoying if a significant number of them get queued up. 7 seconds is ridiculous. I think we need to strip out a lot of the crap about trying to detect things in the stuff being built, and reduce the check to the simplest possible form: $ time zgrep -w CONFIG_PACKET /proc/config.gz CONFIG_PACKET=y real0m0.032s user0m0.021s sys 0m0.018s -- Robin Hugh Johnson Gentoo Linux: Dev, Infra Lead, Foundation Treasurer E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 1/2] bzr.eclass: Reinstate eclass
> On Sat, 25 Sep 2021, Michał Górny wrote:
>> +EBZR="bzr.eclass"
> Why do we need this? It seems as if someone is reinventing ${ECLASS}.
Replaced by ${ECLASS}.
>> +# @ECLASS-VARIABLE: EBZR_STORE_DIR
> @USER_VARIABLE?
Added.
>> +# @DESCRIPTION:
>> +# The directory to store all fetched Bazaar live sources.
>> +: ${EBZR_STORE_DIR:=${PORTAGE_ACTUAL_DISTDIR:-${DISTDIR}}/bzr-src}
>> +
>> +# @ECLASS-VARIABLE: EBZR_UNPACK_DIR
>> +# @DESCRIPTION:
>> +# The working directory where the sources are copied to.
>> +: ${EBZR_UNPACK_DIR:=${WORKDIR}/${P}}
>> +
>> +# @ECLASS-VARIABLE: EBZR_INIT_REPO_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to initialise a shared repository.
>> +: ${EBZR_INIT_REPO_CMD:="bzr init-repository --no-trees"}
>> +
>> +# @ECLASS-VARIABLE: EBZR_FETCH_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to fetch the sources.
>> +: ${EBZR_FETCH_CMD:="bzr branch --no-tree"}
>> +
>> +# @ECLASS-VARIABLE: EBZR_UPDATE_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to update the sources.
>> +: ${EBZR_UPDATE_CMD:="bzr pull --overwrite-tags"}
>> +
>> +# @ECLASS-VARIABLE: EBZR_EXPORT_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to export a branch.
>> +: ${EBZR_EXPORT_CMD:="bzr export"}
>> +
>> +# @ECLASS-VARIABLE: EBZR_CHECKOUT_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to checkout a branch.
>> +: ${EBZR_CHECKOUT_CMD:="bzr checkout --lightweight -q"}
>> +
>> +# @ECLASS-VARIABLE: EBZR_REVNO_CMD
>> +# @DESCRIPTION:
>> +# The Bazaar command to list a revision number of the branch.
>> +: ${EBZR_REVNO_CMD:="bzr revno"}
> Are you sure that having these overrides is a good idea?
Yes. Ebuilds had used them in the past.
> Your followup patch pretty much proves that every ebuild redefining
> them would get broken by switch to breezy.
The only one where syntax has changed is EBZR_INIT_REPO_CMD (which has
init-shared-repository instead of init-repository).
>> +# @ECLASS-VARIABLE: EBZR_OPTIONS
>> +# @DEFAULT_UNSET
>> +# @DESCRIPTION:
>> +# The options passed to the fetch and update commands.
> Is this intended to be set by ebuild or by user?
Ebuild.
>> +# @ECLASS-VARIABLE: EBZR_OFFLINE
> @USER_VARIABLE?
>> +# @ECLASS-VARIABLE: EVCS_UMASK
> @USER_VARIABLE?
Both added.
>> +# @FUNCTION: bzr_initial_fetch
> @INTERNAL?
Added, and renamed to _bzr_initial_fetch.
>> +if [[ -n "${EBZR_OFFLINE}" ]]; then
>> +ewarn "EBZR_OFFLINE cannot be used when there is no local
>> branch yet."
> I dare say this is incorrect. If user says "no online operations", then
> the eclass should just fail, not ignore the user.
Fixed.
>> +${EBZR_FETCH_CMD} ${EBZR_OPTIONS} "${repo_uri}" "${branch_dir}" \
>> +|| die "${EBZR}: can't branch from ${repo_uri}"
> You can replace the backslash with '||'.
I think that splitting the line before the operator is clearer. (It is
also what GNU coding standards recommend for C.)
>> +# @FUNCTION: bzr_update
> @INTERNAL?
Added, and renamed to _bzr_update.
>> +bzr_fetch() {
>> +local repo_dir branch_dir
>> +local save_sandbox_write=${SANDBOX_WRITE} save_umask
>> +
>> +[[ -n ${EBZR_REPO_URI} ]] || die "${EBZR}: EBZR_REPO_URI is empty"
>> +
>> +if [[ ! -d ${EBZR_STORE_DIR} ]] ; then
>> +addwrite /
>> +mkdir -p "${EBZR_STORE_DIR}" \
>> +|| die "${EBZR}: can't mkdir ${EBZR_STORE_DIR}"
>> +SANDBOX_WRITE=${save_sandbox_write}
> Looks like abusing implementation details.
Replaced by subshell.
>> +if [[ -z ${EBZR_INITIAL_URI} ]]; then
>> +bzr_initial_fetch "${EBZR_REPO_URI}" "${branch_dir}"
>> +else
>> +# Workaround for faster initial download. This clones
>> the
>> +# branch from a fast server (which may be out of date),
>> and
>> +# subsequently pulls from the slow original repository.
>> +bzr_initial_fetch "${EBZR_INITIAL_URI}" "${branch_dir}"
>> +if [[ ${EBZR_REPO_URI} != "${EBZR_INITIAL_URI}" ]]; then
>> +EBZR_UPDATE_CMD="${EBZR_UPDATE_CMD} --remember
>> --overwrite" \
>> +EBZR_OFFLINE="" \
> Why do you override EBZR_OFFLINE here?
The whole else clause is dropped in a followup commit.
Ulrich
signature.asc
Description: PGP signature
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On Mon, Sep 27, 2021 at 12:23 PM Mike Pagano wrote: > > Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns > > when running emerge, so we should do so only when there is a > > compensating benefit. > > Is this a significant slowdown? Do you have any numbers? Adding a check for CONFIG_PACKET to the dhcpcd ebuild adds around 7 seconds of delay time to the pkg_setup and/or pkg_pretend phase on my system. That's ok if a small number of packages are doing it, but it would become quite annoying if a significant number of them get queued up.
[gentoo-dev] Re: Guidance on adding kernel config checks to ebuilds
On 27/09/21 18:10, Mike Gilbert wrote: > I'm looking to solicit opinions on when it is appropriate for an > ebuild to check for kernel config options using linux-info.eclass. I > don't think we have any guidelines documented, instead leaving it up > to the "common sense" of package maintainers. > > Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns > when running emerge, so we should do so only when there is a > compensating benefit. It doesn't make sense to check for kernel > options that are very commonly enabled. But what is "very common"? > > An obvious example would be CONFIG_INET, which controls IPv4 support > in the kernel. It does not make sense to check for that in every > package that uses AF_INET sockets. > > A less obvious example: a user filed a bug against net-misc/dhcpcd > today asking that we check for CONFIG_PACKET [1]. My first thought was > "why would you ever disable that?". The option description even says > "if unsure, say Y". However, I suppose it is technically possible to > run a Linux system with it disabled. > > I think a reasonable rule of thumb would be to assume we can rely on > options that are enabled by "make defconfig". If the user chooses to > disable them, they are responsible for anything that breaks. > > Thoughts? We can document in the wiki that going against defconfig means you keep the pieces when something explodes colorfully and/or come up with a even smaller list of config items expected. lu
Re: [gentoo-dev] [PATCH] distutils-r1.eclass: fix formatting
On 9/25/21 9:28 PM, Arthur Zamarin wrote:
> - mark _distutils-r1_check_all_phase_mismatch as @INTERNAL
> - fix list appearance for distutils_enable_tests options
>
> Signed-off-by: Arthur Zamarin
> ---
> eclass/distutils-r1.eclass | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
> index 1326809a8..3513a74c4 100644
> --- a/eclass/distutils-r1.eclass
> +++ b/eclass/distutils-r1.eclass
> @@ -369,8 +369,11 @@ distutils_enable_sphinx() {
> # of RDEPEND to test?-BDEPEND. The test-runner argument must be one of:
> #
> # - nose: nosetests (dev-python/nose)
> +#
> # - pytest: dev-python/pytest
> +#
> # - setup.py: setup.py test (no deps included)
> +#
> # - unittest: for built-in Python unittest module
> #
> # Additionally, if --install is passed as the first parameter,
> @@ -618,6 +621,7 @@ _distutils-r1_handle_pyproject_toml() {
> }
>
> # @FUNCTION: _distutils-r1_check_all_phase_mismatch
> +# @INTERNAL
> # @DESCRIPTION:
> # Verify whether *_all phase impls is not called from from non-*_all
> # subphase.
>
Merged
--
Arthur Zamarin
arthur...@gentoo.org
Gentoo Linux developer (Python, GURU)
OpenPGP_signature
Description: OpenPGP digital signature
Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds
On 9/27/21 12:10 PM, Mike Gilbert wrote: I'm looking to solicit opinions on when it is appropriate for an ebuild to check for kernel config options using linux-info.eclass. I don't think we have any guidelines documented, instead leaving it up to the "common sense" of package maintainers. Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns when running emerge, so we should do so only when there is a compensating benefit. It doesn't make sense to check for kernel options that are very commonly enabled. But what is "very common"? An obvious example would be CONFIG_INET, which controls IPv4 support in the kernel. It does not make sense to check for that in every package that uses AF_INET sockets. A less obvious example: a user filed a bug against net-misc/dhcpcd today asking that we check for CONFIG_PACKET [1]. My first thought was "why would you ever disable that?". The option description even says "if unsure, say Y". However, I suppose it is technically possible to run a Linux system with it disabled. I think a reasonable rule of thumb would be to assume we can rely on options that are enabled by "make defconfig". If the user chooses to disable them, they are responsible for anything that breaks. Thoughts? [1] https://bugs.gentoo.org/815064 The challenge I see is that these config checks head off bugs and issues without our intervention. We as kernel maintainers depend on ebuild maintainers to check these things so they don't become "kernel bugs" to figure out. Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns when running emerge, so we should do so only when there is a compensating benefit. Is this a significant slowdown? Do you have any numbers?
[gentoo-dev] Guidance on adding kernel config checks to ebuilds
I'm looking to solicit opinions on when it is appropriate for an ebuild to check for kernel config options using linux-info.eclass. I don't think we have any guidelines documented, instead leaving it up to the "common sense" of package maintainers. Adding linux-info calls to pkg_pretend or pkg_setup causes slowdowns when running emerge, so we should do so only when there is a compensating benefit. It doesn't make sense to check for kernel options that are very commonly enabled. But what is "very common"? An obvious example would be CONFIG_INET, which controls IPv4 support in the kernel. It does not make sense to check for that in every package that uses AF_INET sockets. A less obvious example: a user filed a bug against net-misc/dhcpcd today asking that we check for CONFIG_PACKET [1]. My first thought was "why would you ever disable that?". The option description even says "if unsure, say Y". However, I suppose it is technically possible to run a Linux system with it disabled. I think a reasonable rule of thumb would be to assume we can rely on options that are enabled by "make defconfig". If the user chooses to disable them, they are responsible for anything that breaks. Thoughts? [1] https://bugs.gentoo.org/815064
