Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Jonathan Wakely]

On 28/03/19 08:17 +, Tomasz Kłoczko wrote:

On Wed, 27 Mar 2019 at 21:27, Jonathan Wakely 
wrote:


On 26/03/19 11:40 +, Tomasz Kłoczko wrote:
>On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely 
>wrote:
>[..]
>
>> >What does this 42 means in this case? It means that during whole gcc
build
>> >are repeated 42 times some subset of *autoconf tests*. How it was
possible
>> >to loose that?!? 樂
>> >gcc is quite monolithic and it should have only one configure.ac. Yes,
>>
>> Why?
>>
>
>Really?
>Really do you want me to answer on the question "why there is no any sense
>repeat 42 times some tests on the source code configuration stage?" ??

Yes, because you repeatedly make the mistake of assuming that one
dimension of a problem is the only one that matters, and that all
other considerations are irrelevant.



Just to be clear ..
So you want to say that I'm making mistake because I'm assuming that
speed/performance matters?


No, that's clearly not what I wrote. Try again.


Could you please confirm that is is what you are thinking reading what I
wrote?
Could you please as well crush my assumption using logical arguments and
facts?


What I wrote was clear. Focus on the words "the only one that matters"
and "other considerations are irrelevant".
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Przemek Klosowski]

On 3/28/19 4:17 AM, Tomasz Kłoczko wrote:


>Really?
>Really do you want me to answer on the question "why there is no
any sense
>repeat 42 times some tests on the source code configuration
stage?" ??

Yes, because you repeatedly make the mistake of assuming that one
dimension of a problem is the only one that matters, and that all
other considerations are irrelevant.


Just to be clear ..
So you want to say that I'm making mistake because I'm assuming that 
speed/performance matters?


I think people pointed out that GCC bundles several subsystems that are 
independently developed and therefore have their own autoconf processes. 
Trying to merge them together, like you propose, would make sense from 
the performance point of view but does not reflect the reality of how 
they are developed.


Jonathan is saying that efficiency has multiple axis: there's the simple 
'CPU cycles' but there are others, e.g. 'how many keystrokes does a 
developer have to type, and how often' and 'how many people have to 
coordinate changes to this file'.


It's good that people like you re-examine various assumptions, because 
some things can be improved on all axes simultaneously, and changing 
them makes life immediately better for everyone.


Other times, however, improvements on one axis make other axes worse, at 
least initially. If that's the case, you have to persuade people to 
adopt this change,  by addressing their concerns and showing a clear 
path forward, in a way that's acceptable to everyone. This is usually 
harder than solving the specific technical issue.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Wed, 27 Mar 2019 at 21:27, Jonathan Wakely 
wrote:

> On 26/03/19 11:40 +, Tomasz Kłoczko wrote:
> >On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely 
> >wrote:
> >[..]
> >
> >> >What does this 42 means in this case? It means that during whole gcc
> build
> >> >are repeated 42 times some subset of *autoconf tests*. How it was
> possible
> >> >to loose that?!? 樂
> >> >gcc is quite monolithic and it should have only one configure.ac. Yes,
> >>
> >> Why?
> >>
> >
> >Really?
> >Really do you want me to answer on the question "why there is no any sense
> >repeat 42 times some tests on the source code configuration stage?" ??
>
> Yes, because you repeatedly make the mistake of assuming that one
> dimension of a problem is the only one that matters, and that all
> other considerations are irrelevant.
>

Just to be clear ..
So you want to say that I'm making mistake because I'm assuming that
speed/performance matters?
Could you please confirm that is is what you are thinking reading what I
wrote?
Could you please as well crush my assumption using logical arguments and
facts?

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Jonathan Wakely]

On 26/03/19 11:40 +, Tomasz Kłoczko wrote:

On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely 
wrote:
[..]


>What does this 42 means in this case? It means that during whole gcc build
>are repeated 42 times some subset of *autoconf tests*. How it was possible
>to loose that?!? 樂
>gcc is quite monolithic and it should have only one configure.ac. Yes,

Why?



Really?
Really do you want me to answer on the question "why there is no any sense
repeat 42 times some tests on the source code configuration stage?" ??


Yes, because you repeatedly make the mistake of assuming that one
dimension of a problem is the only one that matters, and that all
other considerations are irrelevant.

I haven't quoted the rest of your email because it's the usual
rambling diatribe and I gave up trying to follow it.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-27 16:23, Nico Kadel-Garcia a écrit :


That wasn't me. I was just asking about whether using pushd and popd
in a .spec file made sense. I think it does not, it's safer to specify
the target directory explicitly rather than rely on these tools.


The way rpm packaging works, you compose many different shell snippets 
(either via packaging guidelines or via rpm macros). Very rigid systems 
that assume things are exactly one way, without variation, tend to 
compose badly.


--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nico Kadel-Garcia]

On Wed, Mar 27, 2019 at 3:32 AM Nicolas Mailhot
 wrote:
>
> Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit :
> > On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot
> >  wrote:
>
> >> POSIX is dead as a shell compatibility target. You want to replace
> >> bash
> >> with something faster, by all means do it. With something that
> >> includes
> >> the GNU extensions like pushd/popd that most packagers expect today.
> >
> > Is there any reason to *ever* use pushd or popd in %build  or %install
> > today?
>
> That is totally the wrong attitude when you want to replace an
> implementation used for years by thousands of volunteer people in tens
> of thousands of interdependent files.

*I* wasn't suggesting altering the files. I'm merely startled by the
use of "pushd" and "popd" in build tools, especially within %build or
%install.

> It is used now, ergo packagers (the people who did the work) find it
> useful and convenient. You want them to do something else, you need to
> make it worth their effort to something else. Winning 10 minutes of CPU
> time in a single pathological spec like gcc isn't it.

Looking at https://src.fedoraproject.org/cgit/rpms/gcc.git/tree/gcc.spec
. *ouch*. Yeah, that is a good example. I disagree with that part
of the .spec file's logic, but wouldn't want to take on testing and
debugging something that takes this long to compile.

> Yet another is to propose a syntax with is clearly simpler, more
> expressive, more productive and better documented for humans (Not CPUs.
> CPUs do not get to vote). But, that solves "new spec and macro code"
> problem, not the "existing code" problem.
>
> Hazing people with negative terms like bashism never convinced anyone.
> Especially when others are doing the work, not you. In my language, that
> is called “arriving after the battle”: complaining loudly at the people
> who sweated and blooded doing some work, that they didn't do it well
> enough, when you were safely somewhere else, at the time help was
> needed.

That wasn't me. I was just asking about whether using pushd and popd
in a .spec file made sense. I think it does not, it's safer to specify
the target directory explicitly rather than rely on these tools. But
that's a matter of policy and best practices.

> Sincerely,
>
> --
> Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Chris Adams]

Once upon a time, Vít Ondruch  said:
> Is there a way to do something similar to:

A case of:

  pushd someplace
  [do some things]
  popd

can be replaced with:

  cd someplace
  [do some things]
  cd -

If there's some more complicated stuff that has more directory changes
in between the pushd/popd, you can do:

  returnto=$(pwd)
  cd someplace
  [do some things]
  cd $returnto

I went through a lot of this many years ago when I used RPM for managing
third-party software on DEC Unix (where /bin/sh was actual Bourne
shell), but... I don't see the value for Fedora in such churn.  I don't
see a compelling reason to switch /bin/sh to something other than bash,
and it would just be confusing to run shell scriptlets with something
other than /bin/sh.

-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Vít Ondruch]

Dne 27. 03. 19 v 10:30 Dridi Boukelmoune napsal(a):
> On Wed, Mar 27, 2019 at 8:33 AM Nicolas Mailhot
>  wrote:
>> Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit :
>>> On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot
>>>  wrote:
 POSIX is dead as a shell compatibility target. You want to replace
 bash
 with something faster, by all means do it. With something that
 includes
 the GNU extensions like pushd/popd that most packagers expect today.
>>> Is there any reason to *ever* use pushd or popd in %build  or %install
>>> today?



Is there a way to do something similar to:


~~~


%check   
pushd .%{gem_instdir}

# Load nio4r_ext.so.   
rspec -I$(dirs +1)%{gem_extdir_mri} spec
   
popd

~~~


But honestly I don't care if /bin/sh is Bash or something different as
long as my script runs in our build environment.



Vít



>> That is totally the wrong attitude when you want to replace an
>> implementation used for years by thousands of volunteer people in tens
>> of thousands of interdependent files.
> Again, not interested in the side of the discussion about replacing
> bash as the default interpreter. This is a reminder that I would
> personally prefer that bash weren't behind /bin/sh.
>
>> It is used now, ergo packagers (the people who did the work) find it
>> useful and convenient. You want them to do something else, you need to
> I strongly disagree with the shortcut you make here.
>
> The only use of pushd/popd I made in scriptlets was because it was
> part of the Python packaging guidelines and I happen to maintain a
> bunch of Python packages. So what I was doing then wasn't to choose
> something I would deem useful and convenient but rather follow
> guidelines to the best of my abilities.
>
> So sure some people may find it useful and convenient, but others may
> not even be aware or care that this construct is bash-specific so
> please refrain from authoritatively introduce causality where there
> isn't.
>
> And on the convenience factor:
>
> pushd somewhere
> ...
> popd
>
> is not harder than:
>
> cd somewhere
> ...
> cd -
>
> The bash construct becomes interesting only when you need to stack
> directories and not keep explicit track of where you are returning to.
>
> This convenience I do not wish to remove from package maintainers,
> even in scriptlets.
>
>> make it worth their effort to something else. Winning 10 minutes of CPU
>> time in a single pathological spec like gcc isn't it.
>>
>> The easiest way to make it worth their effort is to reduce the effort to
>> zero, ie implement the capabilities commonly people use in your target
>> replacement shell. That will be way way easier than trying to invent
>> something compelling enough for them to change their habits. 10% of
>> specs doing something is definitely common use.
>>
>> Another way is to take the conversion work unto yourself. But that does
>> not solve the ongoing effort of helping packagers that try to use bash
>> syntax in their spec because they need to do something, and find out it
>> does not work, and give up because the additional work of looking for
>> alternatives makes the cost/benefit analysis of packaging something
>> negative. We have many packages where the cost/benefit hovers next to
>> the limit, because we have nice volunteers that will go to their limits
>> for Fedora.
> Conversely whenever as a Fedora end user I run into a failing script
> on a non Fedora system because it uses GNUisms it makes my life harder
> to work on the multiple systems I have to deal with. By having bash as
> /bin/sh Fedora defers the moment when I realize this is happening and
> pain ensues. That is because Fedora is my main system and others are
> second-class citizens in my workflow. I'd hate to elect a different
> one as main my system for obvious reasons.
>
>> Yet another is to propose a syntax with is clearly simpler, more
>> expressive, more productive and better documented for humans (Not CPUs.
>> CPUs do not get to vote). But, that solves "new spec and macro code"
>> problem, not the "existing code" problem.
>>
>> Hazing people with negative terms like bashism never convinced anyone.
> I'm sure some comments of this nature actually got a "fair enough"
> response from a couple people.
>
>> Especially when others are doing the work, not you. In my language, that
>> is called “arriving after the battle”: complaining loudly at the people
>> who sweated and blooded doing some work, that they didn't do it well
>> enough, when you were safely somewhere else, at the time help was
>> needed.
> I am personally taking an opportunity to raise my concern about the
> default shell seeing this after-the-battle thread.
>
> Maybe I should start a different one on /bin/sh?
>
> I was not aware of this until the link was shared in this thread but
> this is an interesting read on the topic:
>
> https://wiki.ubuntu.com/DashAsBinSh
>
> If RPM goes somewhere on this topic [1] maybe that could be a change for f31?

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Wed, Mar 27, 2019 at 8:33 AM Nicolas Mailhot
 wrote:
>
> Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit :
> > On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot
> >  wrote:
>
> >> POSIX is dead as a shell compatibility target. You want to replace
> >> bash
> >> with something faster, by all means do it. With something that
> >> includes
> >> the GNU extensions like pushd/popd that most packagers expect today.
> >
> > Is there any reason to *ever* use pushd or popd in %build  or %install
> > today?
>
> That is totally the wrong attitude when you want to replace an
> implementation used for years by thousands of volunteer people in tens
> of thousands of interdependent files.

Again, not interested in the side of the discussion about replacing
bash as the default interpreter. This is a reminder that I would
personally prefer that bash weren't behind /bin/sh.

> It is used now, ergo packagers (the people who did the work) find it
> useful and convenient. You want them to do something else, you need to

I strongly disagree with the shortcut you make here.

The only use of pushd/popd I made in scriptlets was because it was
part of the Python packaging guidelines and I happen to maintain a
bunch of Python packages. So what I was doing then wasn't to choose
something I would deem useful and convenient but rather follow
guidelines to the best of my abilities.

So sure some people may find it useful and convenient, but others may
not even be aware or care that this construct is bash-specific so
please refrain from authoritatively introduce causality where there
isn't.

And on the convenience factor:

pushd somewhere
...
popd

is not harder than:

cd somewhere
...
cd -

The bash construct becomes interesting only when you need to stack
directories and not keep explicit track of where you are returning to.

This convenience I do not wish to remove from package maintainers,
even in scriptlets.

> make it worth their effort to something else. Winning 10 minutes of CPU
> time in a single pathological spec like gcc isn't it.
>
> The easiest way to make it worth their effort is to reduce the effort to
> zero, ie implement the capabilities commonly people use in your target
> replacement shell. That will be way way easier than trying to invent
> something compelling enough for them to change their habits. 10% of
> specs doing something is definitely common use.
>
> Another way is to take the conversion work unto yourself. But that does
> not solve the ongoing effort of helping packagers that try to use bash
> syntax in their spec because they need to do something, and find out it
> does not work, and give up because the additional work of looking for
> alternatives makes the cost/benefit analysis of packaging something
> negative. We have many packages where the cost/benefit hovers next to
> the limit, because we have nice volunteers that will go to their limits
> for Fedora.

Conversely whenever as a Fedora end user I run into a failing script
on a non Fedora system because it uses GNUisms it makes my life harder
to work on the multiple systems I have to deal with. By having bash as
/bin/sh Fedora defers the moment when I realize this is happening and
pain ensues. That is because Fedora is my main system and others are
second-class citizens in my workflow. I'd hate to elect a different
one as main my system for obvious reasons.

> Yet another is to propose a syntax with is clearly simpler, more
> expressive, more productive and better documented for humans (Not CPUs.
> CPUs do not get to vote). But, that solves "new spec and macro code"
> problem, not the "existing code" problem.
>
> Hazing people with negative terms like bashism never convinced anyone.

I'm sure some comments of this nature actually got a "fair enough"
response from a couple people.

> Especially when others are doing the work, not you. In my language, that
> is called “arriving after the battle”: complaining loudly at the people
> who sweated and blooded doing some work, that they didn't do it well
> enough, when you were safely somewhere else, at the time help was
> needed.

I am personally taking an opportunity to raise my concern about the
default shell seeing this after-the-battle thread.

Maybe I should start a different one on /bin/sh?

I was not aware of this until the link was shared in this thread but
this is an interesting read on the topic:

https://wiki.ubuntu.com/DashAsBinSh

If RPM goes somewhere on this topic [1] maybe that could be a change for f31?

Dridi

[1] https://github.com/rpm-software-management/rpm/issues/646
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-26 16:51, Japheth Cleaver a écrit :


The tooling around the original ecosystem seemed to have no rhyme or
reason with it, JPackage efforts never *fully* went anywhere,


JPackage certainly went somewhere, and this somewhere is Fedora/Red Hat.

To recap, JPackage existed because the JDK was not open source, 
preventing the packaging of Java software in Linux definitions. JPackage 
was successful first in making some of the key Java software of the time 
available on Linux systems on a controlled QAed way, and second in 
helping to convince SUN it needed to open the JDK code or it would lose 
control of the Java ecosystem (I can tell you that SUN and other big 
Java companies were definitely aware and looking at JPackage at the time 
SUN made its openjdk decision).


With the opening of the JDK the core reason for JPackage to exist 
outside distributions vanished and Red Hat promptly hired most of the 
core JPackage team and told it to work for RHEL/Fedora.


Therefore, since a decade at least the stewardship of defining of how to 
integrate Java software in Linux packages, and convincing upstream Java 
projects to do package-friendly things, has rested on Fedora and Red 
Hat. That is a much heavier burden than working with language 
communities that already agreed to be  Linux distribution friendly.


We see now it has not turned out well. I wasn't involved in this phase, 
so I have no idea why. Certainly, Red Hat, which is now the official 
maintainer of several OpenJDK versions upstream, which bought JBoss, 
which had several initiatives (like 389 server) that relied on Java 
parts, which is now part of IBM (another huge Java stakeholder), had all 
the required weight to make things happen. Certainly more than the small 
JPackage team (which managed a definite impact at the time).


I can only explain it with lots of complacency Red Hat and Fedora side, 
Java was not going anywhere, no one was ever going to write Enterprisey 
free software in another language, no need to expend a lot of effort to 
get past what JPackage had already achieved, the Java & JPackage 
investment was safe. Well Kubernetes was rewritten from Java to Go and 
is massively used in Enterprises today, and the whole generation of 
cloud-oriented enterprise software is looking like it will abandon Java 
altogether. You rip what you sow.


If it was just me I'd tell all the various free software Java teams 
within Red Hat/Fedora to agree on a single Java component tech (probably 
Java modules), define a standard FHS-compatible materialization of those 
components, define basic sanity versioning rules ("if component N 
version X needs itself to build, it MUST work with version X-1") and 
then set a deadline, after which Red Hat/Fedora releases that do not 
conform to this model are not accepted anymore. That would get the ball 
rolling Java dev side.


But it's not me and I have decided quite a long time ago to work on 
other things than Java software, so whatever actually happens is up to 
Fedora and Red Hat/IBM leadership. I certainly hope that someone within 
Fedora instances alerted the main sponsor that things were starting to 
burn.


--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-27 03:46, Nico Kadel-Garcia a écrit :

On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot
 wrote:


POSIX is dead as a shell compatibility target. You want to replace 
bash
with something faster, by all means do it. With something that 
includes

the GNU extensions like pushd/popd that most packagers expect today.


Is there any reason to *ever* use pushd or popd in %build  or %install 
today?


That is totally the wrong attitude when you want to replace an 
implementation used for years by thousands of volunteer people in tens 
of thousands of interdependent files.


It is used now, ergo packagers (the people who did the work) find it 
useful and convenient. You want them to do something else, you need to 
make it worth their effort to something else. Winning 10 minutes of CPU 
time in a single pathological spec like gcc isn't it.


The easiest way to make it worth their effort is to reduce the effort to 
zero, ie implement the capabilities commonly people use in your target 
replacement shell. That will be way way easier than trying to invent 
something compelling enough for them to change their habits. 10% of 
specs doing something is definitely common use.


Another way is to take the conversion work unto yourself. But that does 
not solve the ongoing effort of helping packagers that try to use bash 
syntax in their spec because they need to do something, and find out it 
does not work, and give up because the additional work of looking for 
alternatives makes the cost/benefit analysis of packaging something 
negative. We have many packages where the cost/benefit hovers next to 
the limit, because we have nice volunteers that will go to their limits 
for Fedora.


Yet another is to propose a syntax with is clearly simpler, more 
expressive, more productive and better documented for humans (Not CPUs. 
CPUs do not get to vote). But, that solves "new spec and macro code" 
problem, not the "existing code" problem.


Hazing people with negative terms like bashism never convinced anyone. 
Especially when others are doing the work, not you. In my language, that 
is called “arriving after the battle”: complaining loudly at the people 
who sweated and blooded doing some work, that they didn't do it well 
enough, when you were safely somewhere else, at the time help was 
needed.


Sincerely,

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Miroslav Suchý]

Dne 26. 03. 19 v 18:38 Tomasz Kłoczko napsal(a):
> If yes I can propose simpler solution like create .post_build_preserve.lst 
> file in source build root and whatever will
> be added to that file should be archived in some 
> --.tar.xz file to be accessible over koji
> web interface.
> If such file will be created during manually executed "rpmbuild -ba" it will 
> not leave any garbage behind or will be no
> flooding stdout during build.

In fact, this can be quite easy, because we already have:

https://github.com/rpm-software-management/mock/wiki/Plugin-ChrootScan

I will be happy to review and merge enhancement to this plugin, which read or 
add to the regexes from a file.

Miroslav
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nico Kadel-Garcia]

On Tue, Mar 26, 2019 at 3:44 AM Nicolas Mailhot
 wrote:
>
> Le 2019-03-25 22:47, Japheth Cleaver a écrit :
> > If you can take a one-time hit to
> > remove bashisms and get a 25-40% improvement,
>
> CPU time is cheap, packager time is not. Exchanging CPU time for "you
> all should learn to write POSIX-only shell scripts" would be an awful
> deal. The Java part of Fedora is slowly imploding right now because a
> lot of people pushed their complexity on packagers, and the packagers
> could not cope. The Fedora target should be to help packagers achieve
> more with less work, not achieve less with more work.
>
> POSIX is dead as a shell compatibility target. You want to replace bash
> with something faster, by all means do it. With something that includes
> the GNU extensions like pushd/popd that most packagers expect today.

Is there any reason to *ever* use pushd or popd in %build  or %install today?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 8:14 AM, Adam Williamson wrote:

On Tue, 2019-03-26 at 09:08 -0500, mcatanz...@gnome.org wrote:

On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?=
 wrote:

[tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
/bin/sh

How can this discussion still be ongoing? Why not just change it to
/bin/bash and move on?

Japheth Cleaver explained why in response to me a couple of days ago:
apparently changing it would also change the shell used for some
scriptlets...


FYI: https://github.com/rpm-software-management/rpm/issues/646

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 10:57 AM, Nicolas Mailhot wrote:

You want something faster than bash – write something faster than bash
with as expressive a syntax (and ideally the same syntax). Winning CPU
time by consuming packager time is not going to work.


This seems like it's begging the question. "The same syntax as bash" is 
just going to be bash. And if it's going to be bash, we should call that 
out as "bash," which is Step 0 for anything else.


Removing bashisms is /a/ change, but it's not a huge change for shell 
snippets here and there. (Unlike other parts of the distribution that 
would probably have a lot of gettext substitution to deal with.) Yes, 
there are syntax changes occurring, but most of it's going to be more 
along the lines of "Please tweak these C++-style comments so we can use 
a C compiler" than anything else.


-jc

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Ralf Corsepius]

On 3/26/19 5:44 PM, Tomasz Kłoczko wrote:


$ rpm --showrc | egrep -e "popd|pushd"


This scriptlet of yours isn't posix compliant, either.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Feel free to "fix" all the spec and macro code you want. As it is now
and as it changes as people continuously rewrite and add to it.

You'll find little sympathy to adopt a spec syntax less featured and
convenient than the current one, in code that is largely not
performance-sensitive. So as the interested party the "fixing" is up to
you.

And if you intend to make it everyone's problem by forcing something
other than bash in the Fedora default rpm configuration, in a take it
or leave mode, packagers will just leave.

As is happening right now in the Fedora+Java+module subset.

You want something faster than bash – write something faster than bash
with as expressive a syntax (and ideally the same syntax). Winning CPU
time by consuming packager time is not going to work.

-- 
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Tue, 26 Mar 2019 at 16:44, Tomasz Kłoczko 
wrote:
[..]

> If above is correct IMO collecting such files  (it those files are really
> needed and used) should be done outside of the scope of regular "rpmbuild
> -ba".
> Collecting and preserving some build logs always should be part of the
> build automation.
>

One more thing (and sorry again related do gcc.spec)
(again *Warning*: wear welder mask before start reading that file!)

I just realised that in gcc.spec at end of the %check is possible to find
few funny lines which I'll quote here:


mkdir testlogs-%{_target_platform}-%{version}-%{release}
for i in `find . -name \*.log | grep -F testsuite/ | grep -v
'config.log\|acats.*/tests/'`; do
  ln $i testlogs-%{_target_platform}-%{version}-%{release}/ ||:
done
tar cf - testlogs-%{_target_platform}-%{version}-%{release} | xz -9e \
  | uuencode testlogs-%{_target_platform}.tar.xz ||:
rm -rf testlogs-%{_target_platform}-%{version}-%{release}


if it is hard to see about what this part is this is about archiving some
files in build logs :o)
Just tar | uunecode and output to stdout. Pure JFDIN :)
That uuencoded part lands in gcc build logs. Funny old school .. but is it
really necessary/useful?

Quick check:

[tkloczko@barrel SPECS.fedora]$ grep uuencode * -l
binutils.spec
gcc.spec
gdb.spec
ghc-dataenc.spec
nacl-arm-binutils.spec
nacl-binutils.spec
perl-Convert-UU.spec
sharutils.spec
uudeview.spec

>From that list it is possible to remove last three specs.
I think that all those uuencode use cases have been done by the same person
-> kind of exception.
What above says?
That probably there are some needs to preserve some files beyond regular
build process and some people already started taking care of that without
thinking to much.

Question only is: are those needs are really real (real-real) or only
imaginary?

If yes I can propose simpler solution like create .post_build_preserve.lst
file in source build root and whatever will be added to that file should be
archived in some --.tar.xz file to be accessible
over koji web interface.
If such file will be created during manually executed "rpmbuild -ba" it
will not leave any garbage behind or will be no flooding stdout during
build.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Tue, 26 Mar 2019 at 15:42, Dridi Boukelmoune 
wrote:
[..]

> > Japheth Cleaver explained why in response to me a couple of days ago:
> > apparently changing it would also change the shell used for some
> > scriptlets...
>
> He also posted this link:
> http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877


Just FTR: Fedora macros are already polluted by bashisms;

$ rpm --showrc | egrep -e "popd|pushd"
pushd %{buildsubdir}
popd
$ egrep -e "popd|pushd" /usr/lib/rpm/macros.d/*
/usr/lib/rpm/macros.d/macros.perl:pushd %{buildsubdir} \
/usr/lib/rpm/macros.d/macros.perl:popd \

So here it is whole macro which is using pushd/popd:

%__perl_check_pre %{expand: \
%{?__spec_check_pre} \
pushd %{buildsubdir} \
%define perl_br_testdir %{buildroot}%{perl_testdir}/%{cpan_dist_name} \
%{__mkdir_p} %{perl_br_testdir} \
%{__tar} -cf - %{__perl_test_dirs} | ( cd %{perl_br_testdir} && %{__tar}
-xf - ) \
find . -maxdepth 1 -type f -name '*META*' -exec %{__cp} -vp {}
%{perl_br_testdir} ';' \
find %{perl_br_testdir} -type f -exec %{__chmod} -c -x {} ';' \
T_FILES=`find %{perl_br_testdir} -type f -name '*.t'` \
%fix_shbang_line $T_FILES \
%{__chmod} +x $T_FILES \
%{_fixperms} %{perl_br_testdir} \
popd \
}

and:
%perl_testdir   %{_libexecdir}/perl5-tests

That %check_pre extension looks like it is about archiving some post %build
files (what???).
Just checked:

[tkloczko@domek SPECS.fedora]$ grep perl5-tests *
perl.spec:%global perl5_testdir   %{_libexecdir}/perl5-tests

I could be wrong so please correct me if I'm wrong.
Looks like someone to add something in *single perl package* build process
one additional macro has been added to global rpm macros.
If above is correct IMO collecting such files  (it those files are really
needed and used) should be done outside of the scope of regular "rpmbuild
-ba".
Collecting and preserving some build logs always should be part of the
build automation.

Conclusion: whole macro should be removed and there is nothing to fix about
fixing some bashisms here.
If that macro is needed in perl build process it should be moved to the
perl.spec (I would even verify that part carefully as well).
It would be good if someone else as well will carefully review whole
content of the macros.perl. I would be not surprised it that file contains
more such .

That is result of abandoning strict controlling macros used during rpm
build process.
I've been telling here something like +2y ago that spreading macros across
many packages will blow up.

Nevertheless .. cuts_counter--

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 8:47 AM, Japheth Cleaver wrote:

On 3/26/2019 8:14 AM, Adam Williamson wrote:

On Tue, 2019-03-26 at 09:08 -0500, mcatanz...@gnome.org wrote:

On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?=
 wrote:

[tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
/bin/sh

How can this discussion still be ongoing? Why not just change it to
/bin/bash and move on?

Japheth Cleaver explained why in response to me a couple of days ago:
apparently changing it would also change the shell used for some
scriptlets...


Well, each of the build-time scriptlets (%prep vs %check, for example) 
have distinct shell macros that can be overriden -- they just default 
to whatever buildshell is if not.


The problem is that the *install-time* default shell can't be 
overriden at all.


Sorry, this was a bit too cavalier. I'm speaking as a Fedora end-user, 
not in Fedora Infrastructure.


From an infrastructure/builder perspective, there's are some avenues 
for effecting this shell change, which may or may not be desired. For 
any consumers of built RPMs, there isn't. Distinct issues.


-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 9:04 AM, Nicolas Mailhot wrote:

Le 2019-03-26 16:51, Japheth Cleaver a écrit :


As far as actually making changes in specs, let's try addressing that
separately and perhaps asynchronously. Most of the changes for things
likely to run in scriptlets are probably just straight mechanical
replacements. (We won't know until someone does a full scan, but I'd
bet if you're writing *really* complex stuff there, you've probably
already moved that to a packaged script that's executed as part of the
scriptlet instead.)


Or you used macros. That will inject shell snippets right and left.

Macros should be the most trivial of things to clean up, though. And 
would represent the most obvious low-hanging fruit.


In fact, it would be nice to do that regardless of where this goes.

-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-26 16:51, Japheth Cleaver a écrit :


As far as actually making changes in specs, let's try addressing that
separately and perhaps asynchronously. Most of the changes for things
likely to run in scriptlets are probably just straight mechanical
replacements. (We won't know until someone does a full scan, but I'd
bet if you're writing *really* complex stuff there, you've probably
already moved that to a packaged script that's executed as part of the
scriptlet instead.)


Or you used macros. That will inject shell snippets right and left.

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 5:24 AM, Nicolas Mailhot wrote:

Le 2019-03-26 12:29, Dridi Boukelmoune a écrit :

On Tue, Mar 26, 2019 at 8:43 AM Nicolas Mailhot
 wrote:


Le 2019-03-25 22:47, Japheth Cleaver a écrit :
> If you can take a one-time hit to
> remove bashisms and get a 25-40% improvement,

CPU time is cheap, packager time is not. Exchanging CPU time for "you
all should learn to write POSIX-only shell scripts" would be an awful
deal. The Java part of Fedora is slowly imploding right now because a
lot of people pushed their complexity on packagers, and the packagers
could not cope. The Fedora target should be to help packagers achieve
more with less work, not achieve less with more work.


I think the Java ecosystem is before all imploding because of build 
tools

promoting a quadratic complexity of dependencies in a "community" not
bothered to maintain compatibility in libraries and as a result the 
other

side of the community coin not updating dependencies they consume
unless they feel like they need it.


It's an ecosystem where tools do not orient devs towards easy to 
integrate choices. It's imploding because someone decided a long time 
ago @SUN not to bother optimising the integrator/packager time,and all 
the efforts to change the Java community values since have failed (and 
someone should look hard @RH why it has not leveraged its stake in 
JBoss/OpenJDK to improve the situation).


Another victim of those choices and Java ecosystem values was Oracle 
that found itself unable to release timely JDK security fixes when it 
had to in the first years after it bought SUN, the integrator chain it 
bought from SUN was that much broken.


Packager time is not cheap, it's not inexhaustible, it runs out. 
Wasting it on bashisms is not smart.


Packager time -- either within the Fedora project, or among the system 
users, system administrators, and system engineers of Linux 
distributions derived directly or indirectly from Fedora -- is not 
cheap, I agree.  But getting things Right is important, because it 
provides a foundation for those below you to make competent decisions 
based the calls made earlier.


Having had to deal with packaging java apps up myself for quite a while 
now, I'd agree that it's a hot mess and has been for some time. The 
tooling around the original ecosystem seemed to have no rhyme or reason 
with it, JPackage efforts never *fully* went anywhere, modern Java devs 
still (in my experience) have difficulty getting onboard with attempts 
by local OpsEng and admins to wrangle some site-level sense at things. 
It doesn't work for anyone. (I'm sure I'm not the only admin who rues 
this every time they have a 15-line process string in their 'ps' 
output.) Arguably the "Linux is too hard to have to grok" mentality 
among java app devs has driven some part of the tech industry's current 
trends.



However, I think the parallel with bashisms fails. Getting shell right 
is only as difficult as you make it and we're talking about syntax 
cleanup, not imposing a new paradigm on a lawless ecosystem. True, the 
other major Linux ecosystem that did this had always held a 
/bin/sh-stays-POSIX distinction in mind, and had made occasional audit 
efforts before, so it was easier. But they faced their own issues at the 
same time when they did their conversion... (Their init script 
environment (IMHO) was far less disciplined than RH's was at the time 
(and still is in rhel6), so there's that.) Nevertheless, we have a clear 
example here of how it's possible to get it done.


The fixes to RPM to make this even possible shouldn't be difficult. 
Bourne shell remains the shell language (unless otherwise specified for 
that scriptlet), but the "default Bourne shell" should be configurable 
-- everything else is. In theory, RPM is still the specified package 
format for LSB, so the more flexibility for weird environments the 
better. And Fedora overriding that to /bin/bash after that (if desired) 
seems trivial as well.



As far as actually making changes in specs, let's try addressing that 
separately and perhaps asynchronously. Most of the changes for things 
likely to run in scriptlets are probably just straight mechanical 
replacements. (We won't know until someone does a full scan, but I'd bet 
if you're writing *really* complex stuff there, you've probably already 
moved that to a packaged script that's executed as part of the scriptlet 
instead.) That advancement because a nearly insurmountable effort if we 
continue to, by policy, refuse at a core level to distinguish between sh 
and bash.


-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/26/2019 8:14 AM, Adam Williamson wrote:

On Tue, 2019-03-26 at 09:08 -0500, mcatanz...@gnome.org wrote:

On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?=
 wrote:

[tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
/bin/sh

How can this discussion still be ongoing? Why not just change it to
/bin/bash and move on?

Japheth Cleaver explained why in response to me a couple of days ago:
apparently changing it would also change the shell used for some
scriptlets...


Well, each of the build-time scriptlets (%prep vs %check, for example) 
have distinct shell macros that can be overriden -- they just default to 
whatever buildshell is if not.


The problem is that the *install-time* default shell can't be overriden 
at all.


-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Tue, Mar 26, 2019 at 4:15 PM Adam Williamson
 wrote:
>
> On Tue, 2019-03-26 at 09:08 -0500, mcatanz...@gnome.org wrote:
> > On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?=
> >  wrote:
> > > [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
> > > /bin/sh
> >
> > How can this discussion still be ongoing? Why not just change it to
> > /bin/bash and move on?
>
> Japheth Cleaver explained why in response to me a couple of days ago:
> apparently changing it would also change the shell used for some
> scriptlets...

He also posted this link:
http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Adam Williamson]

On Tue, 2019-03-26 at 09:08 -0500, mcatanz...@gnome.org wrote:
> On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?= 
>  wrote:
> > [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
> > /bin/sh
> 
> How can this discussion still be ongoing? Why not just change it to 
> /bin/bash and move on?

Japheth Cleaver explained why in response to me a couple of days ago:
apparently changing it would also change the shell used for some
scriptlets...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[mcatanzaro]

On Thu, Mar 21, 2019 at 5:05 PM, Tomasz =?UTF-8?b?S8WCb2N6a28=?= 
 wrote:

[tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
/bin/sh


How can this discussion still be ongoing? Why not just change it to 
/bin/bash and move on?


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-26 14:16, Dridi Boukelmoune a écrit :


I was rather thinking about all the damage made on the java ecosystem
by the introduction of maven but fair enough.


maven, while not ideal, is just a reflection of the Java community 
values SUN impulsed. I still remember when maven was introduced.


We (Fedora) were enthusiastic. As problematic as maven is, the 
alternatives (osgi, manual classpaths in shell scripts) were much worse. 
Which tells a lot about Java community values :(


Regards,

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Tue, Mar 26, 2019 at 1:24 PM Nicolas Mailhot
 wrote:
>
> Le 2019-03-26 12:29, Dridi Boukelmoune a écrit :
> > On Tue, Mar 26, 2019 at 8:43 AM Nicolas Mailhot
> >  wrote:
> >>
> >> Le 2019-03-25 22:47, Japheth Cleaver a écrit :
> >> > If you can take a one-time hit to
> >> > remove bashisms and get a 25-40% improvement,
> >>
> >> CPU time is cheap, packager time is not. Exchanging CPU time for "you
> >> all should learn to write POSIX-only shell scripts" would be an awful
> >> deal. The Java part of Fedora is slowly imploding right now because a
> >> lot of people pushed their complexity on packagers, and the packagers
> >> could not cope. The Fedora target should be to help packagers achieve
> >> more with less work, not achieve less with more work.
> >
> > I think the Java ecosystem is before all imploding because of build
> > tools
> > promoting a quadratic complexity of dependencies in a "community" not
> > bothered to maintain compatibility in libraries and as a result the
> > other
> > side of the community coin not updating dependencies they consume
> > unless they feel like they need it.
>
> It's an ecosystem where tools do not orient devs towards easy to
> integrate choices. It's imploding because someone decided a long time
> ago @SUN not to bother optimising the integrator/packager time,and all
> the efforts to change the Java community values since have failed (and
> someone should look hard @RH why it has not leveraged its stake in
> JBoss/OpenJDK to improve the situation).
>
> Another victim of those choices and Java ecosystem values was Oracle
> that found itself unable to release timely JDK security fixes when it
> had to in the first years after it bought SUN, the integrator chain it
> bought from SUN was that much broken.

I was rather thinking about all the damage made on the java ecosystem
by the introduction of maven but fair enough.

> Packager time is not cheap, it's not inexhaustible, it runs out. Wasting
> it on bashisms is not smart.

As I said, I don't care if Fedora wants to have bash as the default
shell, especially as the default RPM script interpreter.

It's specifically /bin/sh being bash that bothers me as a Fedora end-user.


Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Tue, 26 Mar 2019 at 12:32, Nicolas Mailhot 
wrote:
[..]

> Packager time is not cheap, it's not inexhaustible, it runs out. Wasting
> it on bashisms is not smart.
>

I like that conclusion because it is way better than all what I wrote.
Thanks Nicolas for those two final sentences :)

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-26 12:29, Dridi Boukelmoune a écrit :

On Tue, Mar 26, 2019 at 8:43 AM Nicolas Mailhot
 wrote:


Le 2019-03-25 22:47, Japheth Cleaver a écrit :
> If you can take a one-time hit to
> remove bashisms and get a 25-40% improvement,

CPU time is cheap, packager time is not. Exchanging CPU time for "you
all should learn to write POSIX-only shell scripts" would be an awful
deal. The Java part of Fedora is slowly imploding right now because a
lot of people pushed their complexity on packagers, and the packagers
could not cope. The Fedora target should be to help packagers achieve
more with less work, not achieve less with more work.


I think the Java ecosystem is before all imploding because of build 
tools

promoting a quadratic complexity of dependencies in a "community" not
bothered to maintain compatibility in libraries and as a result the 
other

side of the community coin not updating dependencies they consume
unless they feel like they need it.


It's an ecosystem where tools do not orient devs towards easy to 
integrate choices. It's imploding because someone decided a long time 
ago @SUN not to bother optimising the integrator/packager time,and all 
the efforts to change the Java community values since have failed (and 
someone should look hard @RH why it has not leveraged its stake in 
JBoss/OpenJDK to improve the situation).


Another victim of those choices and Java ecosystem values was Oracle 
that found itself unable to release timely JDK security fixes when it 
had to in the first years after it bought SUN, the integrator chain it 
bought from SUN was that much broken.


Packager time is not cheap, it's not inexhaustible, it runs out. Wasting 
it on bashisms is not smart.


--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Tue, 26 Mar 2019 at 08:57, Jonathan Wakely 
wrote:
[..]

> >What does this 42 means in this case? It means that during whole gcc build
> >are repeated 42 times some subset of *autoconf tests*. How it was possible
> >to loose that?!? 樂
> >gcc is quite monolithic and it should have only one configure.ac. Yes,
>
> Why?
>

Really?
Really do you want me to answer on the question "why there is no any sense
repeat 42 times some tests on the source code configuration stage?" ??

> The GCC tree contains lots of quite separate subsystems, which are
> maintained semi-independently by different groups of people. It makes
> sense for me to maintain libstdc++-v3/configure.ac and
> libstdc++-v3/acinclude.m4 without worrying how my changes affect the
> rest of the tree.
>

Decades ago on the begging of the gcc development it was only handful
developers around which been able to build gcc because compiler it is still
not easy piece of the code to build.
To help in that process and to minimise those less skilled people mistakes
decision about inclusion of the other projects code into gcc tree has been
made. It was really long time ago an in mean time many tools around gcc has
been developed to guarantee that exact libraries or tools would be present
in for example gcc build environment.
Best example: pkgconfig.
However gcc is like some time capsule and preserved that initial state
quite well straight in the code.
Seems like you do not understand the impact of the fact that some parts of
the gcc code are older than some people reading this list today.

As same as not keeping in po/ directories .pot files is not helping
translators the same is with including other project code is not today
helping in gcc development process. gcc as the project bottleneck is on
those few people who are gcc maintainers. Maintaining code of those
inclusions eats part of that precious man/hour resources.
Everything else can be scaled horizontally.

People like Jakub are thinking that they are helping other people (and by
this saving some resources) by spending time on updating and committing
.pot files changes, and as well syncing other projects code in gcc tree and
they are right .. they are *only thinking* that it is still truth.
If may I one more time quote one of my beloved Latin sentences: "errare
humanum est perseverare autem diabolicum".
Which can be translated to "to err is human, but to *persist* in error (out
of pride) is diabolical".

All about I'm asking is just wake up and look around because few things
have changed in last decades.
Consequence of those separations will be that some source code
configurations test will be repeated .. so "do we need faster /bin/sh or
not?" (if moving to meson would be to difficult).

Look on clang/llvm. People maintaining those projects already divided some
nonpolitical trees and they are keeping some parts separately. The same was
with Xorg.
So are those Xorg or clang developers are wrong about dividing and
deliberately forcing to repeat many times some configuration stage tests?

Answer on that question is "No, they are not" because chance to have
situation that some of those (sub)projects tests  will be executed in
parallel on some well designed packages build machinery are now quite high.

Simple sometimes on solving some issues trying to crush them in straight
confrontation is not the best way to win.
Current gcc state in many areas are really bad because within nonpolitical
tree some separations already have been made which make whole build process
not scaleable. This is quite easy to spot on that graph which I've decided
to post here. In other words gcc source code tree is already in some "brain
split" state between be nonpolitical tree and something which is not.
This not my decision about which one model for gcc should be *chosen*.
What worries me id that I can only bet that some decisions about internal
separations within gcc tree had some more aesthetic than technical
background/context.

As you see whole topic of the gcc has many sometimes orthogonal (sometimes
not) aspects. To reach healthy state bore than few things simultaneously
needs to be carefully balanced. Only part of that topic is related to
/bin/sh and that part stretches waaay beyond gcc.

I'll try to hold my finger away from this topic because I think that I've
already managed to put enough solid context around more than few things. If
someone still do not understand what is discussed after pouring here such
bucket of facts it not my problem. I've done all what I can and able to
convince few people.
My understanding is that now it is more or less *only* matter of time to
make some decisions and I'm not going to press on anyone.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Tue, Mar 26, 2019 at 8:43 AM Nicolas Mailhot
 wrote:
>
> Le 2019-03-25 22:47, Japheth Cleaver a écrit :
> > If you can take a one-time hit to
> > remove bashisms and get a 25-40% improvement,
>
> CPU time is cheap, packager time is not. Exchanging CPU time for "you
> all should learn to write POSIX-only shell scripts" would be an awful
> deal. The Java part of Fedora is slowly imploding right now because a
> lot of people pushed their complexity on packagers, and the packagers
> could not cope. The Fedora target should be to help packagers achieve
> more with less work, not achieve less with more work.

I think the Java ecosystem is before all imploding because of build tools
promoting a quadratic complexity of dependencies in a "community" not
bothered to maintain compatibility in libraries and as a result the other
side of the community coin not updating dependencies they consume
unless they feel like they need it.

That creates a little bit of friction with Fedora guidelines and the First
principle when massive leaf packages (aka applications) try to update
while they "share" some of their dependencies with other major leaf
packages.


> POSIX is dead as a shell compatibility target. You want to replace bash
> with something faster, by all means do it. With something that includes
> the GNU extensions like pushd/popd that most packagers expect today.
>
> Less-capable shells are faster is about as useful as argument than
> "rewriting the shell parts of spec files in Go/python/rust/C/whatever
> would be faster".
>
> And, actually, if you wanted to make builds faster, working with
> upstreams to replace their makefiles with something more modern is
> likely to give better results. GNU Make is known to be slow and to
> consume an awful lot of shell bits.
>
> --
> Nicolas Mailhot
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Tue, 26 Mar 2019 at 07:52, Nicolas Mailhot 
wrote:
[..]

> CPU time is cheap, packager time is not. Exchanging CPU time for "you
> all should learn to write POSIX-only shell scripts" would be an awful
> deal.


I need more time. A year for the starter and I can pay good price ..
Where I can make such deal?
Do you know where is the nearest ATM where I can buy more time?

The Java part of Fedora is slowly imploding right now because a
> lot of people pushed their complexity on packagers, and the packagers
> could not cope. The Fedora target should be to help packagers achieve
> more with less work, not achieve less with more work.
>

Problem with that kind of proving is that you are using *analogy*.
Many people are completely unaware that analogy *newer was, isn't and never
will be* any kind of tool of proving something. That what learn us LOGIC as
general methodology of dealing with problems.
Analogy is the only tool of *improving ideas transport process* from brain
A to brain B. Only this and nothing more.

If you want to prove something you must use *only* pure facts and counter
facts.
Most difficult part of the proving process is distinguishing what is the
relevant fact and what is not and sometimes what is the fact and what isn't.

As long as you 'java argument' has nothing to do with not even POXIS sh per
se but with *PERFORMANCE of the /bin/sh* please go somewhere else.
One more time: we are not discussing POSIX sh compliance topic.
One more time: it is *only coincidence that fastest available now sh
interpreter it is not bash and all those fastest are 100% POSIX sh **compliant
ONLY!!!*

All those fastest sh interpreters sh are not faster because they are better
designed in some common of the code parts.
No. They are faster because they are *simpler and smaller*. Many people
even now are investing serious man/hours resources to make some of those
interpreters even smaller and by this *faster*. If you want to improve bash
speed you must strip down many parts of its code. After that bash would be
nothing more than just ~ksh.
If you want to bet on which one sh interpreter will be fastest before
executing actual tests just look on the size of the binary. Looking from
that angle current x86_64 bash it is 1494360 and mksh it is 342800 bytes.
Do you see now that .. small .. "difference"?

With using sh is the same as with using C++ (if may I use analogy to
only *encircle
*some important* aspect* of the discussion and to focus your attention on
the exact part of the topic).
If you want to have fastest possible C++ code by definition you must forget
using C++ exceptions and exactly the same here is with abandoning bashisms
when /bin/sh is used.
Is that clear now?
Question only is: "do you care about performance or not?"
Everything else after vocalising the answer would be nothing more than just
pure consequence of the *decision* .. because dropping or not bashisms
still is matter of the *choice*.
However even not making conscious decision about that question would be the
decision.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

>
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Jonathan Wakely]

On 25/03/19 22:40 +, Tomasz Kłoczko wrote:

On Mon, 25 Mar 2019 at 21:17, Zbigniew Jędrzejewski-Szmek 
wrote:


On Mon, Mar 25, 2019 at 08:18:34PM +, Tomasz Kłoczko wrote:
> Switching to other than bash sh interpreter allow reduce total gcc
package
> build time by ~5%.

OK. But that just shows that it is — possibly — worth to switch the gcc
build
to a different shell, by working with gcc upstream. Nothing should be
extrapolated from this to other packages and in particular to their
spec files.



Zbyszek .. please .. you are wrong about that I'm using here extrapolation.
I've posted only small fragment of what kind of data I'm collecting in my
infrastructure.

gcc has own set of issues which I've exposed here partially in last 2-3
weeks.
All those issues are way more important than what is used as /bin/sh.
Really.


So why are you bringing them up in a thread about /bin/sh?



Speaking about gcc build improvements. Just one number:

[tkloczko@barrel gcc-9.0.1-20190312]$ find . -name configure.ac | wc -l
42

Funny. Looks like current Fedora gcc poses "The answer to the ultimate
question of life, the universe and everything" 

What does this 42 means in this case? It means that during whole gcc build
are repeated 42 times some subset of *autoconf tests*. How it was possible
to loose that?!? 樂
gcc is quite monolithic and it should have only one configure.ac. Yes,


Why?

The GCC tree contains lots of quite separate subsystems, which are
maintained semi-independently by different groups of people. It makes
sense for me to maintain libstdc++-v3/configure.ac and
libstdc++-v3/acinclude.m4 without worrying how my changes affect the
rest of the tree.


am/ac can handle multiple gettext domains in single build tree so even this
is not the obstacle. How to do this? Just peak on gimp, gtk3 and many other
(however those two are kind of out of book examples).
IMO by such transition to single ac it should be possible to shorten gcc
build time (guessing a bit) by another 10% (optimistically).
However IMO it would be better move gcc build framework to meson (because
as I wrote few days ago ac/am/lt is effectively dead by now by how it is
maintained by GNU people and moving those tools maintenance outside GNU
project domain will be not easy task).
Just in case: cmake as ac/am/lt replacement is not IMO good because it is
yet another project in which coding process maintainers are trying to solve
famine problems on Earth.
cmake is written in C++ and additionally its C++ code uses exceptions and
RTT (only this says that says something .. not so good).
Speaking about cmake. Recently trying to fix some cmake issues I found that
cmake build using boostrap and cmake themselves does not produce the same
results because looks like cmake developers are not using cmake to develop
cmake which is really hilarious


This seems like a rambling irrelevant tangent. Please try to stick to
the point so your emails are a sensible length.


Moving to meson (with all necessary tests in one place fired one time only)
probably should shorten gcc build time by another few percents (maybe even
more). I'm almost sure that with not so big effort in 2-3 man/weeks it
would be possible to reduce gcc build time by at least 1/3 (totally).
Is it worth to divert some people resources to do that? IMO definitely yes
as sooner or later gcc build should allow speedup whole gcc development
process (during last New Year I've started gcc build on my old laptop and
it took on it almost 25h).
However I'm fully aware that on top of moving to meson it will be another
chunk of man/hours resources and this could be painful for Jakub as it will
be necessary to sort out all those SIGSEGVs in gcc test suite and stop
doing his gcc magic


What SIGSEGVs? Do you mean SIGABRT, raised by assert() or abort(),
which is how the tests work?


(Jakub don't take this personally please .. I'm really joking)


Maybe don't bother saying it then.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-25 22:47, Japheth Cleaver a écrit :

If you can take a one-time hit to
remove bashisms and get a 25-40% improvement,


CPU time is cheap, packager time is not. Exchanging CPU time for "you 
all should learn to write POSIX-only shell scripts" would be an awful 
deal. The Java part of Fedora is slowly imploding right now because a 
lot of people pushed their complexity on packagers, and the packagers 
could not cope. The Fedora target should be to help packagers achieve 
more with less work, not achieve less with more work.


POSIX is dead as a shell compatibility target. You want to replace bash 
with something faster, by all means do it. With something that includes 
the GNU extensions like pushd/popd that most packagers expect today.


Less-capable shells are faster is about as useful as argument than 
"rewriting the shell parts of spec files in Go/python/rust/C/whatever 
would be faster".


And, actually, if you wanted to make builds faster, working with 
upstreams to replace their makefiles with something more modern is 
likely to give better results. GNU Make is known to be slow and to 
consume an awful lot of shell bits.


--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Mon, 25 Mar 2019 at 21:57, Japheth Cleaver 
wrote:
[..]

> It feels like there's been this vast movement to try to remove every last
> bit of shell from Fedora whenever possible, and I really don't understand
> the aversion.
>

In most of the cases it is nothing more than some form of NIH syndrome (
https://en.wikipedia.org/wiki/Not_invented_here)

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Mon, 25 Mar 2019 at 21:17, Zbigniew Jędrzejewski-Szmek 
wrote:

> On Mon, Mar 25, 2019 at 08:18:34PM +, Tomasz Kłoczko wrote:
> > Switching to other than bash sh interpreter allow reduce total gcc
> package
> > build time by ~5%.
>
> OK. But that just shows that it is — possibly — worth to switch the gcc
> build
> to a different shell, by working with gcc upstream. Nothing should be
> extrapolated from this to other packages and in particular to their
> spec files.
>

Zbyszek .. please .. you are wrong about that I'm using here extrapolation.
I've posted only small fragment of what kind of data I'm collecting in my
infrastructure.

gcc has own set of issues which I've exposed here partially in last 2-3
weeks.
All those issues are way more important than what is used as /bin/sh.
Really.

Speaking about gcc build improvements. Just one number:

[tkloczko@barrel gcc-9.0.1-20190312]$ find . -name configure.ac | wc -l
42

Funny. Looks like current Fedora gcc poses "The answer to the ultimate
question of life, the universe and everything" 

What does this 42 means in this case? It means that during whole gcc build
are repeated 42 times some subset of *autoconf tests*. How it was possible
to loose that?!? 樂
gcc is quite monolithic and it should have only one configure.ac. Yes,
am/ac can handle multiple gettext domains in single build tree so even this
is not the obstacle. How to do this? Just peak on gimp, gtk3 and many other
(however those two are kind of out of book examples).
IMO by such transition to single ac it should be possible to shorten gcc
build time (guessing a bit) by another 10% (optimistically).
However IMO it would be better move gcc build framework to meson (because
as I wrote few days ago ac/am/lt is effectively dead by now by how it is
maintained by GNU people and moving those tools maintenance outside GNU
project domain will be not easy task).
Just in case: cmake as ac/am/lt replacement is not IMO good because it is
yet another project in which coding process maintainers are trying to solve
famine problems on Earth.
cmake is written in C++ and additionally its C++ code uses exceptions and
RTT (only this says that says something .. not so good).
Speaking about cmake. Recently trying to fix some cmake issues I found that
cmake build using boostrap and cmake themselves does not produce the same
results because looks like cmake developers are not using cmake to develop
cmake which is really hilarious

Moving to meson (with all necessary tests in one place fired one time only)
probably should shorten gcc build time by another few percents (maybe even
more). I'm almost sure that with not so big effort in 2-3 man/weeks it
would be possible to reduce gcc build time by at least 1/3 (totally).
Is it worth to divert some people resources to do that? IMO definitely yes
as sooner or later gcc build should allow speedup whole gcc development
process (during last New Year I've started gcc build on my old laptop and
it took on it almost 25h).
However I'm fully aware that on top of moving to meson it will be another
chunk of man/hours resources and this could be painful for Jakub as it will
be necessary to sort out all those SIGSEGVs in gcc test suite and stop
doing his gcc magic
(Jakub don't take this personally please .. I'm really joking)

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/25/2019 2:10 PM, Zbigniew Jędrzejewski-Szmek wrote:

On Mon, Mar 25, 2019 at 08:18:34PM +, Tomasz Kłoczko wrote:

Switching to other than bash sh interpreter allow reduce total gcc package
build time by ~5%.

OK. But that just shows that it is — possibly — worth to switch the gcc build
to a different shell, by working with gcc upstream. Nothing should be
extrapolated from this to other packages and in particular to their
spec files.


But why not? It's obvious that anything with a lot of forking and 
subshelling in it will be improved. Back when ash was part of the 
distribution I regularly got 30-35% concurrency improvements on 
resource-strained machines when my service had an unavoidable fork or 
three in it. (Looking at you, qmail.) I mean, nothing about 
https://wiki.ubuntu.com/DashAsBinSh has actually /changed/ in all these 
years, really.


It feels like there's been this vast movement to try to remove every 
last bit of shell from Fedora whenever possible, and I really don't 
understand the aversion. True, sometimes the the best answer to a 
dilemma about how to improve XYZ is "mu - let's remove XYZ", but a 
slavish devotion to that has some significant drawbacks. (It was 
mentioned above that Fedora found a much better way to deal with boot 
times, and I'd have to disagree. If you can take a one-time hit to 
remove bashisms and get a 25-40% improvement, that seems a far better 
deal than the years of misery and Linux civil wars the alternative 
precipitated.)


Transactions are nice, but they're not everywhere, and won't solve every 
issue a .spec might have. And that's certainly not necessarily the case 
for non-Fedora code. Given that shell won't ever really completely go 
away, there's nothing wrong with evaluating ways to increase 
compatibility, quality, and efficiency in our execution choices.


Part of that begins with ensuring mechanisms are in there for local SAs 
to hook useful decisions into. And making this more explicit allows both 
the RPM format and Fedora's relation to it to be better defined.


-jc


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Zbigniew Jędrzejewski-Szmek]

On Mon, Mar 25, 2019 at 08:18:34PM +, Tomasz Kłoczko wrote:
> Switching to other than bash sh interpreter allow reduce total gcc package
> build time by ~5%.

OK. But that just shows that it is — possibly — worth to switch the gcc build
to a different shell, by working with gcc upstream. Nothing should be
extrapolated from this to other packages and in particular to their
spec files.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

On Mon, 25 Mar 2019 at 20:47, Stephen John Smoogen  wrote:
[..]

> [Also when giving one graph for one type of build, could you also give a
> similar graph showing how it looks with dash or ksh or whatever you used?]
>

Graph for the gcc will be exactly the same. Only overall time will be
shorter.
ksh93 is not the good replacement candidate. I've tested it with gcc and
using it causes that whole build freezes at some point. I've started my
testy from ksh and this is why I've took next on the alphabetic list which
was mksh.
Maybe it is only some issue with Fedora version and latest ksh93 is fixed
.. really I don't care about that as better POSIX sh alternatives are
around.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 16:20, Tomasz Kłoczko 
wrote:

> [.all the comments.]
>
> All replies are between "who cares?", "it is holy war/waste of time" to
> something like "be standards compliant is important" .. this thread is
> hilarious 
> I'm observing all the comments under my post and (with full respect to all
> of you guys) looks like all of you guys are *wrong* why sticking to POSIX
> sh is important 
>
> *Literally all of you lost one very important fact that bash is not
> fastest sh interpreter which is possible to use and only kind of
> coincidence is that all those fastest are almost puritanically POSIX sh
> compliant!!! *
>
>
For future threads, could you actually put this information in your first
post? You did not put in enough info and so everyone is going to bring up a
dozen different reasons for what you are aiming at. I am saying this
because this information would have made it clearer where you wanted it and
why you wanted it... which I think are:

1. Performance. Groups which do regular mass rebuilds of software find the
CPU usage of bash during builds to be making things slower. While 5% over
one build is not a lot, over 10,000 builds or similar it does. [You are the
3rd person or group I have run into the last month which are doing regular
mass rebuilds of parts of Fedora for their customers or own needs.]
2. Security. Why is bash using an old copy of readline and how does it keep
up with CVE's that affect it
3. Spec file conformity. If bash needs to use its readline, then it should
1) list it as bundled and 2) please have some note for the various groups
who have to do these rebuilds and audits to know why.

Does that cover the points you would like to be covered?

[Also when giving one graph for one type of build, could you also give a
similar graph showing how it looks with dash or ksh or whatever you used?]


-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Tomasz Kłoczko]

[.all the comments.]

All replies are between "who cares?", "it is holy war/waste of time" to
something like "be standards compliant is important" .. this thread is
hilarious 
I'm observing all the comments under my post and (with full respect to all
of you guys) looks like all of you guys are *wrong* why sticking to POSIX
sh is important 

*Literally all of you lost one very important fact that bash is not fastest
sh interpreter which is possible to use and only kind of coincidence is
that all those fastest are almost puritanically POSIX sh compliant!!! *

OK, so about what kind of performance difference in context of exact
numbers we are talking about?
I've tested building gcc using mksh as /bin/sh.
Only one detail: on doing that test you will find small obstacle that you
will be not able to measure whole build time because in the %install of the
gcc.spec is used pushd/popd.
*Warning*: on looking on whole gcc %install *please* remember wearing
welder mask because without it may hurt your brain (Yes, it is yet another
my small pebble with name "badly maintained" thrown in direction of the gcc
because it forces to do so many things outside pure "make install").

Checked my zabbix and here is the graph with CPU usage made during building
gcc on 24 core box (48 CPUs with MT) when bash is used as /bin/sh:
[image: image.png]
As you can see on the graph ~+60% of whole gcc build time hangs on single
thread/core serialised processes. Quite significant part of that time is
spend in *running sh scripts*. Only in second half of the build is possible
fully saturate potential of that particular HW (on running test suite).
Switching to other than bash sh interpreter allow reduce total gcc package
build time by ~5%.
On my 24 physical CPU core HW it is about 10 minutes.

Guys just try to have look around and try to estimate how many Fedora
packages are using autoconf/libtool and other POSIX sh based build
frameworks or using /bin/sh.
As all those frameworks are using sh mostly they are running without
parallelism how fast is /bin/sh is crucial.

No .. be POSIX sh compliant it is not about any kind of holly war!!! It is
pure about performance and for people like me quite often *time spend
waiting until package X build will be finished*. Some people cursing that
waiting and calling it "PCtology" (waiting on the front of the PC until
something will be finished).

BTW readline and bash: is there any particular reason why Fedora bash still
is statically linked with readline generated out of the  own copy of
readline code? That not changed from more than decade (if not +13y) so it
must be some reason 樂
IIRC readline is relatively high on the list of most frequently affected
packages by various CVEs so in case of any new issue with readline someone
must remember about that because ..

[tkloczko@domek SPECS.fedora]$ grep bundled bash.spec
[tkloczko@domek SPECS.fedora]$

Someone must remember about that (somehow) because bash is used as /bin/sh
which and is used almost everywhere(tm),  and by this attacking bash still
is quite popular vector of many security related attacks.
Simpler /bin/sh -> lower probability of many security related issues from
that angle.

Do you see now the subject in proper light?

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Mon, Mar 25, 2019 at 4:57 PM Japheth Cleaver  wrote:
>
> On 3/25/2019 8:02 AM, Adam Williamson wrote:
>
> On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
>
> And since RPM appears to be configurable for the
> default interpreter, have it use /usr/bin/bash by default.
>
> TBH, it seems to me reasonable that we just do this.
>
> If our position is that we actually expect Fedora package scriptlets to
> be executed by bash and don't think it's a problem if they don't work
> when executed by some other shell, why not this make this clear and
> explicit in this way instead of having the default be sh, but then tell
> people sh must be bash?
>
> One caveat: I'm not sure current RPM allows this to be configured separately 
> for builds versus installs.
>
> There's a %_buildshell macro as of 2.4.101, but this only references 
> %prep/%build/%install/%check/etc...
> https://github.com/rpm-software-management/rpm/blob/ba85c95963f9b62f237c0442f6b5aca3e355fa83/macros.in#L801
>
> https://github.com/rpm-software-management/rpm/blob/ff4b9111aeba01dd025dd133ce617fb80f7398a0/lib/rpmscript.c#L408
> ...seems like the default for unspecified %pre/%post/etc scriptlets might 
> fall to the compiled-in /bin/sh. And the configuration of *that* might be the 
> rejected proposal at 
> http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877

Interesting in a sad way, thanks for looking that up!

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Mon, Mar 25, 2019 at 4:27 PM Stephen John Smoogen  wrote:
>
> On Mon, 25 Mar 2019 at 11:18, Kevin Fenzi  wrote:
> >
> > On 3/25/19 8:02 AM, Adam Williamson wrote:
> > > On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
> > >> And since RPM appears to be configurable for the
> > >> default interpreter, have it use /usr/bin/bash by default.
> > >
> > > TBH, it seems to me reasonable that we just do this.
> > >
> > > If our position is that we actually expect Fedora package scriptlets to
> > > be executed by bash and don't think it's a problem if they don't work
> > > when executed by some other shell, why not this make this clear and
> > > explicit in this way instead of having the default be sh, but then tell
> > > people sh must be bash?
> >
> > Doesn't bash behave slightly differently when invoked as 'sh' ?
> >
> > Long ago it used to, but I don't know if thats still the case...
> >
>
> It will interpret some things differently but does not shut off things
> like pushd/popd and $() and various other things that are either
> considered non-POSIX

But but... $() is in POSIX sh! Don't refrain from using that one! :D

It's rather things like custom `set -...` options, custom builtins,
associative arrays... Bash in POSIX mode doesn't enforce a strict
POSIX sh syntax:

$ cat test.sh
a=(a b c)
echo ${#a[@]}
$ bash test.sh
3
$ bash --posix test.sh
3
$ dash test.sh
test.sh: 1: test.sh: Syntax error: "(" unexpected

> Man page:
>
>   If  bash  is  invoked  with  the name sh, it tries to mimic the startup
>behavior of historical versions of sh as  closely  as  possible,  while
>conforming  to the POSIX standard as well.  When invoked as an interac‐
>tive login shell, or a non-interactive shell with the  --login  option,
>it  first  attempts  to read and execute commands from /etc/profile and
>~/.profile, in that order.  The  --noprofile  option  may  be  used  to
>inhibit  this  behavior.  When invoked as an interactive shell with the
>name sh, bash looks for the variable ENV, expands its value  if  it  is
>defined,  and uses the expanded value as the name of a file to read and
>execute.  Since a shell invoked as sh does not attempt to read and exe‐
>cute  commands from any other startup files, the --rcfile option has no
>effect.  A non-interactive shell invoked with  the  name  sh  does  not
>attempt  to  read  any  other  startup files.  When invoked as sh, bash
>enters posix mode after the startup files are read.
>
>When bash is started in posix mode, as with the  --posix  command  line
>option, it follows the POSIX standard for startup files.  In this mode,
>interactive shells expand the ENV variable and commands  are  read  and
>executed  from  the  file  whose  name is the expanded value.  No other
>startup files are read.
>
> 
> The argument of where posix conformity ends and where adding features begins.
>
>
> > kevin
> >
> >
> >
> > ___
> > devel mailing list -- devel@lists.fedoraproject.org
> > To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: 
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>
>
>
> --
> Stephen J Smoogen.
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Steven Bonneville]

On Mon, Mar 25, 2019 at 10:27 AM 
wrote:

 On Mon, 25 Mar 2019 08:16:43 -0700 Kevin Fenzi  wrote:

> On 3/25/19 8:02 AM, Adam Williamson wrote:
> > On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
> >> And since RPM appears to be configurable for the
> >> default interpreter, have it use /usr/bin/bash by default.
> >=20
> > TBH, it seems to me reasonable that we just do this.
> >=20
> > If our position is that we actually expect Fedora package scriptlets to=
>
> > be executed by bash and don't think it's a problem if they don't work
> > when executed by some other shell, why not this make this clear and
> > explicit in this way instead of having the default be sh, but then tell=
>
> > people sh must be bash?
>
> Doesn't bash behave slightly differently when invoked as 'sh' ?
>
> Long ago it used to, but I don't know if thats still the case...
>

Yes, it enters POSIX mode after startup.  From 'info bash':

6.11 Bash POSIX Mode
> 
>
> Starting Bash with the '--posix' command-line option or executing 'set
> -o posix' while Bash is running will cause Bash to conform more closely
> to the POSIX standard by changing the behavior to match that specified
> by POSIX in areas where the Bash default differs.
>
>When invoked as 'sh', Bash enters POSIX mode after reading the
> startup files.
>

Details of the differences follow in that section.

  -- Steve

-- 
Steven Bonneville 
Principal Technical Curriculum Architect
Red Hat | Red Hat Training   Phone: +1-612-638-0507
gpg: 1024D/221D06FF  68B1 3E66 A351 6485 B9AF  24D8 3DF5 B50B 221D 06FF
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/25/2019 8:02 AM, Adam Williamson wrote:

On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:

And since RPM appears to be configurable for the
default interpreter, have it use /usr/bin/bash by default.

TBH, it seems to me reasonable that we just do this.

If our position is that we actually expect Fedora package scriptlets to
be executed by bash and don't think it's a problem if they don't work
when executed by some other shell, why not this make this clear and
explicit in this way instead of having the default be sh, but then tell
people sh must be bash?


One caveat: I'm not sure current RPM allows this to be configured 
separately for builds versus installs.


There's a %_buildshell macro as of 2.4.101, but this only references 
%prep/%build/%install/%check/etc...
https://github.com/rpm-software-management/rpm/blob/ba85c95963f9b62f237c0442f6b5aca3e355fa83/macros.in#L801 



https://github.com/rpm-software-management/rpm/blob/ff4b9111aeba01dd025dd133ce617fb80f7398a0/lib/rpmscript.c#L408 

...seems like the default for unspecified %pre/%post/etc scriptlets 
might fall to the compiled-in /bin/sh. And the configuration of *that* 
might be the rejected proposal at 
http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877


-jc

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[R P Herrold]

On Mon, 25 Mar 2019, Kevin Fenzi wrote:

> > explicit in this way instead of having the default be sh, but then tell
> > people sh must be bash?
> 
> Doesn't bash behave slightly differently when invoked as 'sh' ?

bash behaviour has changed [1] over time --- /bin/sh is fixed 
in behaviour

It is pretty clear that feaping creaturitis is in play with 
the bash developer -- an addition (recently) of an inbuilt 
'seq' primitive was a surprise to me; lots of 'undocumented' 
behaviours discussed, and some 'promoted' to being documented.  
May one rely on undocumented behaviours?

-- Russ herrold

1. https://tiswww.case.edu/php/chet/bash/CHANGES

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 11:18, Kevin Fenzi  wrote:
>
> On 3/25/19 8:02 AM, Adam Williamson wrote:
> > On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
> >> And since RPM appears to be configurable for the
> >> default interpreter, have it use /usr/bin/bash by default.
> >
> > TBH, it seems to me reasonable that we just do this.
> >
> > If our position is that we actually expect Fedora package scriptlets to
> > be executed by bash and don't think it's a problem if they don't work
> > when executed by some other shell, why not this make this clear and
> > explicit in this way instead of having the default be sh, but then tell
> > people sh must be bash?
>
> Doesn't bash behave slightly differently when invoked as 'sh' ?
>
> Long ago it used to, but I don't know if thats still the case...
>

It will interpret some things differently but does not shut off things
like pushd/popd and $() and various other things that are either
considered non-POSIX

Man page:

  If  bash  is  invoked  with  the name sh, it tries to mimic the startup
   behavior of historical versions of sh as  closely  as  possible,  while
   conforming  to the POSIX standard as well.  When invoked as an interac‐
   tive login shell, or a non-interactive shell with the  --login  option,
   it  first  attempts  to read and execute commands from /etc/profile and
   ~/.profile, in that order.  The  --noprofile  option  may  be  used  to
   inhibit  this  behavior.  When invoked as an interactive shell with the
   name sh, bash looks for the variable ENV, expands its value  if  it  is
   defined,  and uses the expanded value as the name of a file to read and
   execute.  Since a shell invoked as sh does not attempt to read and exe‐
   cute  commands from any other startup files, the --rcfile option has no
   effect.  A non-interactive shell invoked with  the  name  sh  does  not
   attempt  to  read  any  other  startup files.  When invoked as sh, bash
   enters posix mode after the startup files are read.

   When bash is started in posix mode, as with the  --posix  command  line
   option, it follows the POSIX standard for startup files.  In this mode,
   interactive shells expand the ENV variable and commands  are  read  and
   executed  from  the  file  whose  name is the expanded value.  No other
   startup files are read.


The argument of where posix conformity ends and where adding features begins.


> kevin
>
>
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Kevin Fenzi]

On 3/25/19 8:02 AM, Adam Williamson wrote:
> On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
>> And since RPM appears to be configurable for the
>> default interpreter, have it use /usr/bin/bash by default.
> 
> TBH, it seems to me reasonable that we just do this.
> 
> If our position is that we actually expect Fedora package scriptlets to
> be executed by bash and don't think it's a problem if they don't work
> when executed by some other shell, why not this make this clear and
> explicit in this way instead of having the default be sh, but then tell
> people sh must be bash?

Doesn't bash behave slightly differently when invoked as 'sh' ?

Long ago it used to, but I don't know if thats still the case...

kevin





signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Adam Williamson]

On Mon, 2019-03-25 at 12:59 +0100, Dridi Boukelmoune wrote:
> And since RPM appears to be configurable for the
> default interpreter, have it use /usr/bin/bash by default.

TBH, it seems to me reasonable that we just do this.

If our position is that we actually expect Fedora package scriptlets to
be executed by bash and don't think it's a problem if they don't work
when executed by some other shell, why not this make this clear and
explicit in this way instead of having the default be sh, but then tell
people sh must be bash?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Florian Weimer]

* Stephen John Smoogen:

> My very hazy memory of UsrMove was that one of the arguments was that
> we were behind some other distros on this, and once again not "First".

Huh.  That surprises me.

> I think the issue is that many of us  look at the GNU/Linux ecosystem
> in different ways. There is what is in existence now with the majority
> of Linux being Android phones, and the majority of installed GNU/Linux
> being old Debian releases running on lightbulbs and other embedded
> hardware. However none of that is first, and being compliant with 10
> year old software is easy.. just never fix anything.  We are very much
> not compliant with those.

Debian has those multi-arch paths, though.  It really helps them with
qemu-user, and it also simplifies cross-toolchains because they can
use native libraries.  I assume this makes Debian a much more
attractive target as a developer workstation for targets where you
can't build natively—but I could be mistaken.

> There is the middle road, where you look and see a large number of
> Ubuntu being used in the cloud or in containers or whatever the hyped
> on technology of last month was. We are somewhat compliant with
> these.. but not really.

I don't think there's a significant difference between Ubuntu and
Debian in these matters at present, just the usual version skew
between packages.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/25/2019 5:12 AM, Florian Weimer wrote:

Fedora is so different from other GNU/Linux systems these days, so I'm
not sure if *any* recommendation to encourage portability (at the cost
of convenience to Fedora developers or users) makes sense anymore.
___


If this is true (and I'm not saying I disagree), then it feels like it's 
high time that "Fedora" (as indicated) be spun towards something 
distinctly and clearly /downstream/ from a process that *does* encourage 
portability, so that Red Hat ecosystem users with those desires in mind 
have a place to attempt to manifest them.


-jc
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Zbigniew Jędrzejewski-Szmek]

On Mon, Mar 25, 2019 at 02:06:31PM +0100, Dridi Boukelmoune wrote:
> On Mon, Mar 25, 2019 at 1:12 PM Florian Weimer  wrote:
> >
> > * Dridi Boukelmoune:
> >
> > > This is the kind of spec that leads to spoiled upstreams putting
> > > /bin/sh in shebangs and scratching their heads when they get bug
> > > reports for stricter systems...
> > >
> > > I'd be happier if Fedora was not part of the problem and maintainers
> > > were encouraged to figure out the correct shebang (and when in doubt
> > > use /usr/bin/bash).
> >
> > If you want more compatibility, you definitely can't use /usr/bin/bash.
> 
> Or /bin/bash, it is good to be reminded that not everyone did the UsrMove...
> 
> > Fedora is so different from other GNU/Linux systems these days, so I'm
> > not sure if *any* recommendation to encourage portability (at the cost
> > of convenience to Fedora developers or users) makes sense anymore.
> 
> For me it's not so much about the defaults. When I take a script that
> says #!/bin/sh and it turns out to contain bashisms (not even talking
> about GNU extensions in  coreutils programs) it's a needless pain in
> the backside.
> 
> If were somehow able to switch /bin/sh to dash and see where things
> blow up, we could gradually switch those to bash, not necessarily
> rewrite them to be more portable.
We *could*, but what would be the point? We might just as well mandate
that all tabs should be converted to spaces (with some specific number
per tab), and fight the uphill battle of fixing every file. Fedora
scriptlets are only used in Fedora, and we only use bash, and we don't
plan to switch to anything else, and like with tabs vs. spaces, even
if we achieved the goal of posix spec-files, it would be *zero* benefit
to our users. Seriously, no one would even notice.

In the past, Debian had the idea of switching to dash because dash was
faster and they were using sysvinit and had lots of scripts executed
at startup. We have a *much* better solution to this — let's not run
scripts at all. Systemd did away with a lot of init files, and in the
rpm land, we're in the processing of removing a large chunk of our
scriptlets and replacing them with a few filetriggers. This effect of
this is actually noticable by users and has other benefits apart from
speed. Any effort put into this is much better spent than tweaking
syntax details of scriptlets.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 09:19, Florian Weimer  wrote:
>
> * Stephen John Smoogen:
>
> > On Mon, 25 Mar 2019 at 08:13, Florian Weimer  wrote:
> >>
> >> * Dridi Boukelmoune:
> >>
> >> > This is the kind of spec that leads to spoiled upstreams putting
> >> > /bin/sh in shebangs and scratching their heads when they get bug
> >> > reports for stricter systems...
> >> >
> >> > I'd be happier if Fedora was not part of the problem and maintainers
> >> > were encouraged to figure out the correct shebang (and when in doubt
> >> > use /usr/bin/bash).
> >>
> >> If you want more compatibility, you definitely can't use /usr/bin/bash.
> >>
> >> Fedora is so different from other GNU/Linux systems these days, so I'm
> >> not sure if *any* recommendation to encourage portability (at the cost
> >> of convenience to Fedora developers or users) makes sense anymore.
> >
> > This statement constantly confuses me. Every release we are told we
> > are too different from other GNU/Linux systems. That drives changes
> > which are supposed to make us more compatible, and yet at the next
> > release we are again where we started from.
>
> Hmm.  I haven't seen that, at least not for low-level matters.  (In
> the compiler flags discussion, it's about *not* changing defaults from
> GNU upstream.)
>
> We have implemented UsrMove and we don't use multi-arch paths for ELF
> objects, which makes use incompatible with a fairly large chunk of the
> rest of the GNU/Linux ecosystem.  Our Java packaging does not use the
> Class-Path: manifest attribute, and packages do not consistently use
> /usr/share/java for storing JAR files.
>
> So at the lower levels, there is not much we can do to improve
> compatibility.  Perhaps we can adopt multi-arch paths, and just hope
> that the rest of the world will eventually implement mandatory UsrMove
> as well.

My very hazy memory of UsrMove was that one of the arguments was that
we were behind some other distros on this, and once again not "First".

I think the issue is that many of us  look at the GNU/Linux ecosystem
in different ways. There is what is in existence now with the majority
of Linux being Android phones, and the majority of installed GNU/Linux
being old Debian releases running on lightbulbs and other embedded
hardware. However none of that is first, and being compliant with 10
year old software is easy.. just never fix anything.  We are very much
not compliant with those.

There is the middle road, where you look and see a large number of
Ubuntu being used in the cloud or in containers or whatever the hyped
on technology of last month was. We are somewhat compliant with
these.. but not really.

And finally there is the vast (possibly infinite) amount of GNU/Linux
which doesn't exist yet. And that is what people keep saying we aren't
compliant with as soon as it comes into existence. I would say that
the reason we don't use Java Class-Path is because Java is for old
people.. and not 'sexy' anymore for the next-gen people. Multi-arch is
for old people. UsrMove is what everyone 'should' have done.. but
didn't. When KlaFooBar Linux comes out with PDP-11-128 bit virtual
machines which make functional-object-shard-hyperledger-containers
possible.. we will be tacking as quickly to making the distro to
incorporate that. [It will probably not be something as sexy as that..
128 bit PDP-11 is just too much for this universe. The FOSHC paradigm
will probably be done in 80-128-86 instead.]




--
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Florian Weimer]

* Stephen John Smoogen:

> On Mon, 25 Mar 2019 at 08:13, Florian Weimer  wrote:
>>
>> * Dridi Boukelmoune:
>>
>> > This is the kind of spec that leads to spoiled upstreams putting
>> > /bin/sh in shebangs and scratching their heads when they get bug
>> > reports for stricter systems...
>> >
>> > I'd be happier if Fedora was not part of the problem and maintainers
>> > were encouraged to figure out the correct shebang (and when in doubt
>> > use /usr/bin/bash).
>>
>> If you want more compatibility, you definitely can't use /usr/bin/bash.
>>
>> Fedora is so different from other GNU/Linux systems these days, so I'm
>> not sure if *any* recommendation to encourage portability (at the cost
>> of convenience to Fedora developers or users) makes sense anymore.
>
> This statement constantly confuses me. Every release we are told we
> are too different from other GNU/Linux systems. That drives changes
> which are supposed to make us more compatible, and yet at the next
> release we are again where we started from.

Hmm.  I haven't seen that, at least not for low-level matters.  (In
the compiler flags discussion, it's about *not* changing defaults from
GNU upstream.)

We have implemented UsrMove and we don't use multi-arch paths for ELF
objects, which makes use incompatible with a fairly large chunk of the
rest of the GNU/Linux ecosystem.  Our Java packaging does not use the
Class-Path: manifest attribute, and packages do not consistently use
/usr/share/java for storing JAR files.

So at the lower levels, there is not much we can do to improve
compatibility.  Perhaps we can adopt multi-arch paths, and just hope
that the rest of the world will eventually implement mandatory UsrMove
as well.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

> > > Fedora is based on GNU tools versus strict POSIX compliant ones. As
> > > such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is
> > > /bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless
> > > specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to
> > > be used, packagers may rely on tools to act as the upstream GNU tools
> > > in their spec files.

> So please write the spec you want to see.

Good point! Here is a variation on your proposal:

> Fedora is based on GNU tools versus strict POSIX compliant ones. As
> such, package maintainers can expect RPM build steps and scriptlets
> to be run with bash and that core programs come with GNU extensions.
> This means that unless specified elsewhere that a 'bashism', 'gawkism',
> 'gcc-ism' or other GNUism is not to be used, they may rely on tools to
> act as the upstream GNU tools in their spec files.
>
> When shipping scripts from upstream, care should be taken to check
> whether they rely on such GNUisms and report portability bugs upstream.

Funny enough, people often tell me not to use $(...) and switch to
`...` because the former "is" a bashism (spoiler alert, it isn't) and
as a result the only thing reported to me in this regard is always a
false positive :(

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Mon, Mar 25, 2019 at 1:12 PM Florian Weimer  wrote:
>
> * Dridi Boukelmoune:
>
> > This is the kind of spec that leads to spoiled upstreams putting
> > /bin/sh in shebangs and scratching their heads when they get bug
> > reports for stricter systems...
> >
> > I'd be happier if Fedora was not part of the problem and maintainers
> > were encouraged to figure out the correct shebang (and when in doubt
> > use /usr/bin/bash).
>
> If you want more compatibility, you definitely can't use /usr/bin/bash.

Or /bin/bash, it is good to be reminded that not everyone did the UsrMove...

> Fedora is so different from other GNU/Linux systems these days, so I'm
> not sure if *any* recommendation to encourage portability (at the cost
> of convenience to Fedora developers or users) makes sense anymore.

For me it's not so much about the defaults. When I take a script that
says #!/bin/sh and it turns out to contain bashisms (not even talking
about GNU extensions in  coreutils programs) it's a needless pain in
the backside.

If were somehow able to switch /bin/sh to dash and see where things
blow up, we could gradually switch those to bash, not necessarily
rewrite them to be more portable.

For example, our RPM configuration can definitely be Fedora-specific
and require /usr/bin/bash as the default interpreter. But when a
random script is started as an executable, via its shebang, the
shebang should be accurate and blow up when it isn't.

Bash will indeed run in POSIX mode when it is invoked as sh, but it
doesn't prevent bashisms.

Again, not beating up Fedora or bash, they are both my favorites in
their own categories, but since this topic came up it was a good
opportunity to complain :-/

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 08:13, Florian Weimer  wrote:
>
> * Dridi Boukelmoune:
>
> > This is the kind of spec that leads to spoiled upstreams putting
> > /bin/sh in shebangs and scratching their heads when they get bug
> > reports for stricter systems...
> >
> > I'd be happier if Fedora was not part of the problem and maintainers
> > were encouraged to figure out the correct shebang (and when in doubt
> > use /usr/bin/bash).
>
> If you want more compatibility, you definitely can't use /usr/bin/bash.
>
> Fedora is so different from other GNU/Linux systems these days, so I'm
> not sure if *any* recommendation to encourage portability (at the cost
> of convenience to Fedora developers or users) makes sense anymore.

This statement constantly confuses me. Every release we are told we
are too different from other GNU/Linux systems. That drives changes
which are supposed to make us more compatible, and yet at the next
release we are again where we started from. And yet every time people
try to find out what these differences are, the ones which seem to
cause the most 'this is a complete incompatibility!!!' are shed paint
colour differences versus anything substantial.

> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Florian Weimer]

* Dridi Boukelmoune:

> This is the kind of spec that leads to spoiled upstreams putting
> /bin/sh in shebangs and scratching their heads when they get bug
> reports for stricter systems...
>
> I'd be happier if Fedora was not part of the problem and maintainers
> were encouraged to figure out the correct shebang (and when in doubt
> use /usr/bin/bash).

If you want more compatibility, you definitely can't use /usr/bin/bash.

Fedora is so different from other GNU/Linux systems these days, so I'm
not sure if *any* recommendation to encourage portability (at the cost
of convenience to Fedora developers or users) makes sense anymore.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 08:02, Dridi Boukelmoune
 wrote:
>
> > Try 1 at specification:
> >
> > Fedora is based on GNU tools versus strict POSIX compliant ones. As
> > such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is
> > /bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless
> > specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to
> > be used, packagers may rely on tools to act as the upstream GNU tools
> > in their spec files.
>
> This is the kind of spec that leads to spoiled upstreams putting
> /bin/sh in shebangs and scratching their heads when they get bug
> reports for stricter systems...
>
> I'd be happier if Fedora was not part of the problem and maintainers
> were encouraged to figure out the correct shebang (and when in doubt
> use /usr/bin/bash). And since RPM appears to be configurable for the
> default interpreter, have it use /usr/bin/bash by default.
>
> Dridi

So please write the spec you want to see.


> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

> Try 1 at specification:
>
> Fedora is based on GNU tools versus strict POSIX compliant ones. As
> such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is
> /bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless
> specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to
> be used, packagers may rely on tools to act as the upstream GNU tools
> in their spec files.

This is the kind of spec that leads to spoiled upstreams putting
/bin/sh in shebangs and scratching their heads when they get bug
reports for stricter systems...

I'd be happier if Fedora was not part of the problem and maintainers
were encouraged to figure out the correct shebang (and when in doubt
use /usr/bin/bash). And since RPM appears to be configurable for the
default interpreter, have it use /usr/bin/bash by default.

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Stephen John Smoogen]

On Mon, 25 Mar 2019 at 06:34, Japheth Cleaver  wrote:
>
> On 3/25/2019 2:46 AM, Nicolas Mailhot wrote:
> > Le 2019-03-25 09:53, Jan Pokorný a écrit :
> >
> >> Good point, and that's something capable of making upstream
> >> maintenance cumbersome at times (sed is a common pet peeve),
> >> but that's an order of magnitude more demanding level when it
> >> comes to portability, and with Fedora settled firmly just around
> >> GNU approaches and extensions, there's hardly a pressing need for
> >> the spec files to come anywhere close (but if so, the restrictions
> >> should not be limited to shell interpreter alone as remarked,
> >> since POSIX compliance is a wider topic).
> >
> > More accurately, what is the point of wasting energy on making a Linux
> > system POSIX-only today? POSIX was a useful tool to make proprietary
> > unixes more or less compatible with one another. The situation has
> > changed since. Linux has taken over most of the marketplace. That
> > means the common compat layer you need to target to replace it with
> > something else, is whatever major Linux distributions agree on, and
> > that includes all the GNU tools with all their non-POSIX extensions.
>
> The original point was on the execution shell, not external commands
> being run through it. Those can always vary according to the needs of
> the .spec writer, which is why Requires(pre/post) exist. (Using perl's
> broader compatibility to get around sed oddities springs to mind.) It's
> true that it's always good to strive for maximal compatibility whenever
> possible, but that's slightly orthogonal here.
>
> It's clear that there are GNU/Linux systems that do simply use other
> Bourne shells by default, and users who would like to do so
> individually. Removing unnecessary bashisms would be very nice, but a
> sensible, coherent specification would at least be a good start.

Try 1 at specification:

Fedora is based on GNU tools versus strict POSIX compliant ones. As
such, packagers can expect that /bin/sh is /bin/bash, /bin/awk is
/bin/gawk, /bin/cc is /bin/gcc ad naseum. This means that unless
specified elsewhere that a 'bashism', 'gawkism', 'gcc-ism' is not to
be used, packagers may rely on tools to act as the upstream GNU tools
in their spec files.



-- 
Stephen J Smoogen.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/25/2019 2:46 AM, Nicolas Mailhot wrote:

Le 2019-03-25 09:53, Jan Pokorný a écrit :


Good point, and that's something capable of making upstream
maintenance cumbersome at times (sed is a common pet peeve),
but that's an order of magnitude more demanding level when it
comes to portability, and with Fedora settled firmly just around
GNU approaches and extensions, there's hardly a pressing need for
the spec files to come anywhere close (but if so, the restrictions
should not be limited to shell interpreter alone as remarked,
since POSIX compliance is a wider topic).


More accurately, what is the point of wasting energy on making a Linux 
system POSIX-only today? POSIX was a useful tool to make proprietary 
unixes more or less compatible with one another. The situation has 
changed since. Linux has taken over most of the marketplace. That 
means the common compat layer you need to target to replace it with 
something else, is whatever major Linux distributions agree on, and 
that includes all the GNU tools with all their non-POSIX extensions.


The original point was on the execution shell, not external commands 
being run through it. Those can always vary according to the needs of 
the .spec writer, which is why Requires(pre/post) exist. (Using perl's 
broader compatibility to get around sed oddities springs to mind.) It's 
true that it's always good to strive for maximal compatibility whenever 
possible, but that's slightly orthogonal here.


It's clear that there are GNU/Linux systems that do simply use other 
Bourne shells by default, and users who would like to do so 
individually. Removing unnecessary bashisms would be very nice, but a 
sensible, coherent specification would at least be a good start.


-jc

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Nicolas Mailhot]

Le 2019-03-25 09:53, Jan Pokorný a écrit :


Good point, and that's something capable of making upstream
maintenance cumbersome at times (sed is a common pet peeve),
but that's an order of magnitude more demanding level when it
comes to portability, and with Fedora settled firmly just around
GNU approaches and extensions, there's hardly a pressing need for
the spec files to come anywhere close (but if so, the restrictions
should not be limited to shell interpreter alone as remarked,
since POSIX compliance is a wider topic).


More accurately, what is the point of wasting energy on making a Linux 
system POSIX-only today? POSIX was a useful tool to make proprietary 
unixes more or less compatible with one another. The situation has 
changed since. Linux has taken over most of the marketplace. That means 
the common compat layer you need to target to replace it with something 
else, is whatever major Linux distributions agree on, and that includes 
all the GNU tools with all their non-POSIX extensions.


That's why Microsoft created WSL instead of just fixing its POSIX 
subsystem. POSIX is no longer sufficient to gain significant market 
traction. Users want their pushds/popds.


Regards,

--
Nicolas Mailhot
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Jan Pokorný]

On 22/03/19 19:17 +0100, Dridi Boukelmoune wrote:
> On Fri, Mar 22, 2019 at 6:04 PM Japheth Cleaver  
> wrote:
>> IMO the situation that we're in now ("Assume you're running in
>> bash, but called as -/bin/sh") is a worst-of-both-worlds middle
>> ground, somewhat akin to mandating webpages be written in IE Quirks
>> Mode for all time. It's neither pedantically correct, nor flexible
>> for users and downstreams. And the resolution from all of this last
>> time remains lacking in the guarantees that an independent spec
>> should have:
> 
> The result is what we call bashisms where people put /bin/sh in their
> shebangs but don't realize they are using GNU extensions, and that
> goes for "standard" programs like awk too where people are not aware
> of the differences regarding what's on their machine and what POSIX
> mandates.

Good point, and that's something capable of making upstream
maintenance cumbersome at times (sed is a common pet peeve),
but that's an order of magnitude more demanding level when it
comes to portability, and with Fedora settled firmly just around
GNU approaches and extensions, there's hardly a pressing need for
the spec files to come anywhere close (but if so, the restrictions
should not be limited to shell interpreter alone as remarked,
since POSIX compliance is a wider topic).

> So with both a default non-POSIX-compliant shell (by default, I'm
> aware) and non-POSIX-compliant options to the basic commands it takes
> more effort to write portable shell.
> 
> To be clear, bash is my favorite shell and the one I use, but I'd
> prefer that running it as sh or /bin/sh would be enough to enter
> POSIX mode.

-- 
Jan (Poki)


pgpvqAxMo6aKZ.pgp
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Miroslav Suchý]

Dne 22. 03. 19 v 18:03 Japheth Cleaver napsal(a):
> RPM should IMHO indicate scriptlets are to be written in Bourne shell, give a 
> 'SHOULD'-level recommendation for
> POSIX-correctness, and provide a mechanism for distros to override that 
> default.

BTW any maintainer can easily indicate that by:

%post -p /usr/bin/bash
BASH SCRIPTLET

Miroslav
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Owen Taylor]

On Fri, Mar 22, 2019 at 1:04 PM Japheth Cleaver  wrote:
>
> On 3/21/2019 3:23 PM, Richard W.M. Jones wrote:
>
> So what?  On Fedora /bin/sh is bash, and bash is a fine shell.
>
> All this nonsense of using dash for /bin/sh on Debian is IMO a
> pointless bunch of make-work.
>
> Fedora has certainly made a lot of make-work projects over the last decade, 
> under arguably more questionable reasoning.

Assuming that Fedora *has* had make-work projects, that's not a
justification to create more of them!

> IMO the situation that we're in now ("Assume you're running in bash, but 
> called as -/bin/sh") is a worst-of-both-worlds middle ground, somewhat akin 
> to mandating webpages be written in IE Quirks Mode for all time. It's neither 
> pedantically correct, nor flexible for users and downstreams. And the 
> resolution from all of this last time remains lacking in the guarantees that 
> an independent spec should have:

Any change here would need some strong justification - and allowing
users or derived distributions to replace /bin/sh with non-bash is not
something that I think there's any interest in. An argument could be
made for spec-file portability, but considering the amount of
Fedora-specific macros that we *want* packagers to use, that's not
obviously that convincing either.

Owen
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

On Fri, Mar 22, 2019 at 6:04 PM Japheth Cleaver  wrote:
>
> On 3/21/2019 3:23 PM, Richard W.M. Jones wrote:
>
>
> So what?  On Fedora /bin/sh is bash, and bash is a fine shell.
>
> All this nonsense of using dash for /bin/sh on Debian is IMO a
> pointless bunch of make-work.
>
> Fedora has certainly made a lot of make-work projects over the last decade, 
> under arguably more questionable reasoning.
>
> IMO the situation that we're in now ("Assume you're running in bash, but 
> called as -/bin/sh") is a worst-of-both-worlds middle ground, somewhat akin 
> to mandating webpages be written in IE Quirks Mode for all time. It's neither 
> pedantically correct, nor flexible for users and downstreams. And the 
> resolution from all of this last time remains lacking in the guarantees that 
> an independent spec should have:

The result is what we call bashisms where people put /bin/sh in their
shebangs but don't realize they are using GNU extensions, and that
goes for "standard" programs like awk too where people are not aware
of the differences regarding what's on their machine and what POSIX
mandates.

So with both a default non-POSIX-compliant shell (by default, I'm
aware) and non-POSIX-compliant options to the basic commands it takes
more effort to write portable shell.

To be clear, bash is my favorite shell and the one I use, but I'd prefer that
running it as sh or /bin/sh would be enough to enter POSIX mode.

> http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877
> https://pagure.io/packaging-committee/issue/184
> https://bugzilla.redhat.com/show_bug.cgi?id=850706
> via https://lists.fedoraproject.org/pipermail/devel/2014-October/202998.html
>
> RPM should IMHO indicate scriptlets are to be written in Bourne shell, give a 
> 'SHOULD'-level recommendation for POSIX-correctness, and provide a mechanism 
> for distros to override that default. And If Fedora's packaging guidelines 
> are going to continue to state that bashisms are completely OK, then it 
> should use that override mechanism to set the shell to "/bin/bash" explicitly.

Maybe there is a macro like %_buildshell that should explicitly refer
to /usr/bin/bash when the default scriplet interpreter is used.

At this point trying to make dash own /bin/sh would be downright
impossible within the lifespan of a Fedora release (maybe the extended
development time for f31 could, but it was already allocated for
something else).

My 2 cents...


Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Japheth Cleaver]

On 3/21/2019 3:23 PM, Richard W.M. Jones wrote:


So what?  On Fedora /bin/sh is bash, and bash is a fine shell.

All this nonsense of using dash for /bin/sh on Debian is IMO a
pointless bunch of make-work.

Fedora has certainly made a lot of make-work projects over the last 
decade, under arguably more questionable reasoning.


IMO the situation that we're in now ("Assume you're running in bash, but 
called as -/bin/sh") is a worst-of-both-worlds middle ground, somewhat 
akin to mandating webpages be written in IE Quirks Mode for all time. 
It's neither pedantically correct, nor flexible for users and 
downstreams. And the resolution from all of this last time remains 
lacking in the guarantees that an independent spec should have:


http://web.archive.org/web/20150821020837/http://rpm.org/ticket/877
https://pagure.io/packaging-committee/issue/184
https://bugzilla.redhat.com/show_bug.cgi?id=850706
via https://lists.fedoraproject.org/pipermail/devel/2014-October/202998.html

RPM should IMHO indicate scriptlets are to be written in Bourne shell, 
give a 'SHOULD'-level recommendation for POSIX-correctness, and provide 
a mechanism for distros to override that default. And If Fedora's 
packaging guidelines are going to continue to state that bashisms are 
completely OK, then it should use that override mechanism to set the 
shell to "/bin/bash" explicitly.



-jc

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

> This is fine, because /bin/sh is a symbolic link to /usr/bin/bash on Fedora.
>
> I always use pushd/popd in my packages.

It was also in the python packaging guidelines for py2+py3 builds.

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Vitaly Zaitsev]

Hello, Tomasz Kłoczko.

Thu, 21 Mar 2019 22:05:55 + you wrote:

> Looks like many Fedora packagers forgot that ..
> /bin/sh

This is fine, because /bin/sh is a symbolic link to /usr/bin/bash on Fedora.

I always use pushd/popd in my packages.

--
Sincerely,
 Vitaly Zaitsev (vit...@easycoding.org)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Dridi Boukelmoune]

> So what?  On Fedora /bin/sh is bash, and bash is a fine shell.

I don't disagree.

> All this nonsense of using dash for /bin/sh on Debian is IMO a
> pointless bunch of make-work.

I may complain about Debian on every occasion but I think dash is a
much more sensible choice for /bin/sh, and that doesn't prevent bash
from being the default interactive login shell for users (where dash
becomes IMHO less relevant).

Dridi
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Panu Matilainen]

On 3/22/19 12:23 AM, Richard W.M. Jones wrote:

On Thu, Mar 21, 2019 at 10:05:55PM +, Tomasz Kłoczko wrote:

Just FTR:

[tkloczko@domek SPECS.fedora]$ egrep -w "popd|pushd" * -l| wc -l
2843

Looks like many Fedora packagers forgot that ..

[tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
/bin/sh


So what?  On Fedora /bin/sh is bash, and bash is a fine shell.

All this nonsense of using dash for /bin/sh on Debian is IMO a
pointless bunch of make-work.


Truly.

It's also actually documented,
https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#_default_shell 
says:



In Fedora, all scriptlets can safely assume they are running under the bash 
shell unless a different language has been specified.


It talks about scriptlets, but both scriptlets and build scripts invoke 
the very same /bin/sh so the guarantee holds for both.


- Panu -
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: More than 10% of all Fedora spec files are not POSIX sh compliant

[Richard W.M. Jones]

On Thu, Mar 21, 2019 at 10:05:55PM +, Tomasz Kłoczko wrote:
> Just FTR:
> 
> [tkloczko@domek SPECS.fedora]$ egrep -w "popd|pushd" * -l| wc -l
> 2843
> 
> Looks like many Fedora packagers forgot that ..
> 
> [tkloczko@domek SPECS.fedora]$ rpm -E %_buildshell
> /bin/sh

So what?  On Fedora /bin/sh is bash, and bash is a fine shell.

All this nonsense of using dash for /bin/sh on Debian is IMO a
pointless bunch of make-work.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org