Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Sat, Jun 04, 2022 at 06:19:30PM +0200, Alessandro Barbieri wrote:
> When I use sed is for dynamic content and mostly like to do this:
> s|lib|$(get_libdir)|g
> In this case esed would be deleterious because it would fail on 32 bit
> arches
This case is noted in the docs fwiw. How to handle will depend on
preference but best route to ensure the change is always right
would be either to patch to replace lib by e.g. @GENTOO_LIBDIR@,
/then/ sed, or insert a variable/option that could be passed and
perhaps even upstreamed.
Lazier approach that use esed (or upcoming erepl) could be:
[[ $(get_libdir) != lib ]] && erepl /lib /$(get_libdir) file
This also avoids unnecessarily editing files when there's nothing,
albeit more invasive and runs get_libdir twice. EPREFIX is a bit
nicer given we can use `use prefix`:
use prefix && erepl /usr "${EPREFIX}"/usr file
This is a bit like replacing
rm -f exists-with-USE-gui-only
by
use !gui || rm exists-with-USE-gui-only || die
(bash exit code logic always fun wrt that double negation...)
Both are going to do the job at first, but one may end up missing
that the file was moved in another directory on bump and is now
getting wrongly installed.
--
ionen
signature.asc
Description: PGP signature
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
Il giorno mar 31 mag 2022 alle ore 13:23 Ionen Wolkens ha scritto: > Often preferable to use patches so this happens, but sed have its > uses/convenience and this intend to help reduce the amount of old > broken seds causing issues that go unnoticed on bumps. > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > this is for more deterministic use in ebuilds. > > Also slightly shortens sed use, -i is default, and no need to || die. > (see @EXAMPLE in eclass for a quick usage overview). > > Implementation / available wrappers / usefulness still up for debate, > but without further comments I consider this ready (albeit first time > touching / making an eclass, so I could be overlooking simple things). > Also partly uses >=bash-4.4, so EAPI-7 is not considered. > > See github PR[1] for old changelog background. > > Up to maintainers but personally would encourage to slowly replace > (almost) all use of sed with either this or patches. Some cases > where it can be inconvenient like eclasses "guessing" that a package > may or may not have something to replace, and that nothing happened > is not an issue. > > [1] https://github.com/gentoo/gentoo/pull/25662 > > Ionen Wolkens (2): > esed.eclass: new eclass > eclass/tests/esed.sh: basic tests for esed.eclass > > eclass/esed.eclass | 199 +++ > eclass/tests/esed.sh | 173 + > 2 files changed, 372 insertions(+) > create mode 100644 eclass/esed.eclass > create mode 100755 eclass/tests/esed.sh > > -- > 2.35.1 > > > I use patches for static content, not a big fan of wall of sed in ebuilds When I use sed is for dynamic content and mostly like to do this: s|lib|$(get_libdir)|g In this case esed would be deleterious because it would fail on 32 bit arches
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Fri, Jun 03, 2022 at 06:09:38AM +0200, Michał Górny wrote: > On Tue, 2022-05-31 at 07:23 -0400, Ionen Wolkens wrote: > > Often preferable to use patches so this happens, but sed have its > > uses/convenience and this intend to help reduce the amount of old > > broken seds causing issues that go unnoticed on bumps. > > > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > > this is for more deterministic use in ebuilds. > > > > Also slightly shortens sed use, -i is default, and no need to || die. > > (see @EXAMPLE in eclass for a quick usage overview). > > > > To be honest, I strongly dislike this. It really feels like trying to > make an adapter for a square wheel, while the right solution would be to > replace the wheel. On top of that, ton of evals which are pretty much > a huge "no-no". As for using this or not, I don't overly mind ether way. I felt like giving it a try given it's been requested but if there's not much interest that's fine too (there's still qa-sed fwiw, with some limitations). > > Perhaps it would be better to forget about trying to work miracles with > sed and instead write a trivial shell replacement for the most common > use cases. One thing I'd love to see is a simple substitution command > that would work for paths/CFLAGS on RHS without having to worry about > them conflicting with the pattern delimiter. > > -- > Best regards, > Michał Górny > > -- ionen signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
> On 3 Jun 2022, at 05:59, Sam James wrote: > > > >> On 31 May 2022, at 12:23, Ionen Wolkens wrote: >> >> Often preferable to use patches so this happens, but sed have its >> uses/convenience and this intend to help reduce the amount of old >> broken seds causing issues that go unnoticed on bumps. >> >> Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but >> this is for more deterministic use in ebuilds. >> >> Also slightly shortens sed use, -i is default, and no need to || die. >> (see @EXAMPLE in eclass for a quick usage overview). >> >> Implementation / available wrappers / usefulness still up for debate, >> but without further comments I consider this ready (albeit first time >> touching / making an eclass, so I could be overlooking simple things). >> Also partly uses >=bash-4.4, so EAPI-7 is not considered. >> >> See github PR[1] for old changelog background. >> >> Up to maintainers but personally would encourage to slowly replace >> (almost) all use of sed with either this or patches. Some cases >> where it can be inconvenient like eclasses "guessing" that a package >> may or may not have something to replace, and that nothing happened >> is not an issue. >> > > I like it. I'd prefer it if GNU sed supported it by itself (exit code > indicating > if any changes were made) but it doesn't right now. > > Ebuilds use sed aplenty and making it easier to be "less bad" is a good > thing. It's a practical solution to a real problem we have (zombie seds, > or more rarely, overzealous-and-not-realising-it seds). > > I don't want us to have to keep it forever and I wouldn't want > people to actively use this instead of patches, but they certainly should > instead of sed. > Also, I like that we let ourselves experiment a bit with edo.eclass, and I don't see this as too different from that. Unless something is absolutely the wrong approach and we know it won't go anywhere, I think exploring options is generally a good thing. Plus, we've seen the actual need for this from iwdevtools usage (warns on such zombie seds). Let's give it a go, I think? >> [1] https://github.com/gentoo/gentoo/pull/25662 >> >> Ionen Wolkens (2): >> esed.eclass: new eclass >> eclass/tests/esed.sh: basic tests for esed.eclass >> >> eclass/esed.eclass | 199 +++ >> eclass/tests/esed.sh | 173 + >> 2 files changed, 372 insertions(+) >> create mode 100644 eclass/esed.eclass >> create mode 100755 eclass/tests/esed.sh >> >> -- >> 2.35.1 >> > > best, > sam signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Fri, Jun 03, 2022 at 09:47:51AM +0500, Anna V wrote: > On 2022-06-03 00:45, Ionen Wolkens wrote: > > On Fri, Jun 03, 2022 at 06:09:38AM +0200, Michał Górny wrote: > > > On Tue, 2022-05-31 at 07:23 -0400, Ionen Wolkens wrote: > > > > Often preferable to use patches so this happens, but sed have its > > > > uses/convenience and this intend to help reduce the amount of old > > > > broken seds causing issues that go unnoticed on bumps. > > > > > > > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > > > > this is for more deterministic use in ebuilds. > > > > > > > > Also slightly shortens sed use, -i is default, and no need to || die. > > > > (see @EXAMPLE in eclass for a quick usage overview). > > > > > > > > > > To be honest, I strongly dislike this. It really feels like trying to > > > make an adapter for a square wheel, while the right solution would be to > > > replace the wheel. On top of that, ton of evals which are pretty much > > > a huge "no-no". > > > > About evals, the two eval is just to silence this: > > > > var=$(printf "\0") > > printf -v var "\0" That was just for the sake of showing what happens, the eclass is reading a file and isn't using printf. aka: var=$( > > The 2>/dev/null doesn't work without wrapping it, aka > > > > eval 'var=$(printf "\0")' 2>/dev/null > > > > No variables are expanded pre-eval so it's just evaluating a > > static statement. eval can be removed, but it'll be noisy > > if someone happens to sed binary files. > > > > > > > > Perhaps it would be better to forget about trying to work miracles with > > > sed and instead write a trivial shell replacement for the most common > > > use cases. One thing I'd love to see is a simple substitution command > > > that would work for paths/CFLAGS on RHS without having to worry about > > > them conflicting with the pattern delimiter. > > > > > > -- > > > Best regards, > > > Michał Górny > > > > > > > > > > -- > > ionen > > > -- ionen signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
> On 31 May 2022, at 12:23, Ionen Wolkens wrote: > > Often preferable to use patches so this happens, but sed have its > uses/convenience and this intend to help reduce the amount of old > broken seds causing issues that go unnoticed on bumps. > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > this is for more deterministic use in ebuilds. > > Also slightly shortens sed use, -i is default, and no need to || die. > (see @EXAMPLE in eclass for a quick usage overview). > > Implementation / available wrappers / usefulness still up for debate, > but without further comments I consider this ready (albeit first time > touching / making an eclass, so I could be overlooking simple things). > Also partly uses >=bash-4.4, so EAPI-7 is not considered. > > See github PR[1] for old changelog background. > > Up to maintainers but personally would encourage to slowly replace > (almost) all use of sed with either this or patches. Some cases > where it can be inconvenient like eclasses "guessing" that a package > may or may not have something to replace, and that nothing happened > is not an issue. > I like it. I'd prefer it if GNU sed supported it by itself (exit code indicating if any changes were made) but it doesn't right now. Ebuilds use sed aplenty and making it easier to be "less bad" is a good thing. It's a practical solution to a real problem we have (zombie seds, or more rarely, overzealous-and-not-realising-it seds). I don't want us to have to keep it forever and I wouldn't want people to actively use this instead of patches, but they certainly should instead of sed. > [1] https://github.com/gentoo/gentoo/pull/25662 > > Ionen Wolkens (2): > esed.eclass: new eclass > eclass/tests/esed.sh: basic tests for esed.eclass > > eclass/esed.eclass | 199 +++ > eclass/tests/esed.sh | 173 + > 2 files changed, 372 insertions(+) > create mode 100644 eclass/esed.eclass > create mode 100755 eclass/tests/esed.sh > > -- > 2.35.1 > best, sam signature.asc Description: Message signed with OpenPGP
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Fri, Jun 03, 2022 at 06:09:38AM +0200, Michał Górny wrote: > On Tue, 2022-05-31 at 07:23 -0400, Ionen Wolkens wrote: > > Often preferable to use patches so this happens, but sed have its > > uses/convenience and this intend to help reduce the amount of old > > broken seds causing issues that go unnoticed on bumps. > > > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > > this is for more deterministic use in ebuilds. > > > > Also slightly shortens sed use, -i is default, and no need to || die. > > (see @EXAMPLE in eclass for a quick usage overview). > > > > To be honest, I strongly dislike this. It really feels like trying to > make an adapter for a square wheel, while the right solution would be to > replace the wheel. On top of that, ton of evals which are pretty much > a huge "no-no". About evals, the two eval is just to silence this: var=$(printf "\0") The 2>/dev/null doesn't work without wrapping it, aka eval 'var=$(printf "\0")' 2>/dev/null No variables are expanded pre-eval so it's just evaluating a static statement. eval can be removed, but it'll be noisy if someone happens to sed binary files. > > Perhaps it would be better to forget about trying to work miracles with > sed and instead write a trivial shell replacement for the most common > use cases. One thing I'd love to see is a simple substitution command > that would work for paths/CFLAGS on RHS without having to worry about > them conflicting with the pattern delimiter. > > -- > Best regards, > Michał Górny > > -- ionen signature.asc Description: PGP signature
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Tue, 2022-05-31 at 07:23 -0400, Ionen Wolkens wrote: > Often preferable to use patches so this happens, but sed have its > uses/convenience and this intend to help reduce the amount of old > broken seds causing issues that go unnoticed on bumps. > > Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but > this is for more deterministic use in ebuilds. > > Also slightly shortens sed use, -i is default, and no need to || die. > (see @EXAMPLE in eclass for a quick usage overview). > To be honest, I strongly dislike this. It really feels like trying to make an adapter for a square wheel, while the right solution would be to replace the wheel. On top of that, ton of evals which are pretty much a huge "no-no". Perhaps it would be better to forget about trying to work miracles with sed and instead write a trivial shell replacement for the most common use cases. One thing I'd love to see is a simple substitution command that would work for paths/CFLAGS on RHS without having to worry about them conflicting with the pattern delimiter. -- Best regards, Michał Górny
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
Hi, On 2022/05/31 16:29, Ionen Wolkens wrote: > esed does bring back the -i/die skipping but that's not its inherent > purpose and GNU sed currently does not support a mean to report if > changes occurred (if this happens, esed may well become obsolete too). > Haven't checked the code to validate. But I'm in support of esed eclass for the sole reason it ensures the sed actually still changes something. Not that it verifies the change is actually intended mind you, so it doesn't completely take away the due diligence checks, but the verbose options should help devs to ensure the changes are the intended at testing time. One *could* also add a change-counter check, eg, count the number of lines starting with + and - (but not +++ and ---) from diff, and ensure they match the expected count. Just to catch further possible cases. Kind Regards, Jaco
Re: [gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
On Tue, May 31, 2022 at 06:54:21PM +0500, Anna wrote: > On 2022-05-31 07:23, Ionen Wolkens wrote: > > Implementation / available wrappers / usefulness still up for debate, > > but without further comments I consider this ready (albeit first time > > touching / making an eclass, so I could be overlooking simple things). > > Also partly uses >=bash-4.4, so EAPI-7 is not considered. > > From ebuild(5): > > Beginning with EAPI 4, the dosed helper no longer exists > > Ebuilds should call sed(1) directly (and assume that it is GNU > > sed). > > What were the reasons to remove this helper? Do they still apply to esed > as an eclass? Haven't looked up exact history, but afaik dosed existed because of sed formerly not having -i. And editing files in-place was tedious in ebuilds with cp/mv. With -i becoming commonplace (PMS also requires GNU sed) it lost its purpose beside just allowing to skip -i/||die. esed does bring back the -i/die skipping but that's not its inherent purpose and GNU sed currently does not support a mean to report if changes occurred (if this happens, esed may well become obsolete too). -- ionen signature.asc Description: PGP signature
[gentoo-dev] [PATCH 0/2] Add esed.eclass for sed that dies if caused no changes
Often preferable to use patches so this happens, but sed have its uses/convenience and this intend to help reduce the amount of old broken seds causing issues that go unnoticed on bumps. Inspired by app-portage/iwdevtools' qa-sed (warns on any seds), but this is for more deterministic use in ebuilds. Also slightly shortens sed use, -i is default, and no need to || die. (see @EXAMPLE in eclass for a quick usage overview). Implementation / available wrappers / usefulness still up for debate, but without further comments I consider this ready (albeit first time touching / making an eclass, so I could be overlooking simple things). Also partly uses >=bash-4.4, so EAPI-7 is not considered. See github PR[1] for old changelog background. Up to maintainers but personally would encourage to slowly replace (almost) all use of sed with either this or patches. Some cases where it can be inconvenient like eclasses "guessing" that a package may or may not have something to replace, and that nothing happened is not an issue. [1] https://github.com/gentoo/gentoo/pull/25662 Ionen Wolkens (2): esed.eclass: new eclass eclass/tests/esed.sh: basic tests for esed.eclass eclass/esed.eclass | 199 +++ eclass/tests/esed.sh | 173 + 2 files changed, 372 insertions(+) create mode 100644 eclass/esed.eclass create mode 100755 eclass/tests/esed.sh -- 2.35.1
