Re: [gentoo-dev] POSIX shell and portable
On Tue, 06 Nov 2007 07:40:20 + Roy Marples [EMAIL PROTECTED] wrote: On Tue, 2007-11-06 at 07:12 +, Ciaran McCreesh wrote: Except it won't, because ebuilds require bash regardless of which package manager is being used. If you want to change that you'll have to rewrite the entire tree. Az once said near enough the same thing about baselayout. And that's your view, your entitled to it, but it is not my view. Change a little bit here, a little bit there. Slowly does it. It's not a view. It's a simple fact. Yes, I know that a fair chunk of the tree will need a re-write, just in the same way that the init scripts got a re-write. It will take time, it will not happen magically over night. To think overwise is foolish :) How many lines of code are in baselayout? How many in the tree? Pushing for non-bash for ebuilds is pointless. The cost of using bash is tiny; the cost of not using bash is huge. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] POSIX shell and portable
On Tue, 2007-11-06 at 08:03 +, Ciaran McCreesh wrote:
On Tue, 06 Nov 2007 07:40:20 +
Roy Marples [EMAIL PROTECTED] wrote:
On Tue, 2007-11-06 at 07:12 +, Ciaran McCreesh wrote:
Except it won't, because ebuilds require bash regardless of which
package manager is being used. If you want to change that you'll
have to rewrite the entire tree.
Az once said near enough the same thing about baselayout. And that's
your view, your entitled to it, but it is not my view. Change a little
bit here, a little bit there. Slowly does it.
It's not a view. It's a simple fact.
It's my considered opinion that it's a view.
You are free to call it what you like.
Yes, I know that a fair chunk of the tree will need a re-write, just
in the same way that the init scripts got a re-write. It will take
time, it will not happen magically over night. To think overwise is
foolish :)
How many lines of code are in baselayout? How many in the tree?
Pushing for non-bash for ebuilds is pointless. The cost of using bash is
tiny; the cost of not using bash is huge.
Size of baselayout compared to the tree is small vs huge. But unlike
baselayout, the ebuilds themselves should be relatively easy as they
don't normally use bash specific features [1]. The real work is in the
eclasses which make extensive use of bash specific features, such as
arrays. A quick look at the dir shows that there's probably a similar
number of eclasses to the number of init scripts installed by ebuilds.
[1] The one expection being ${var//foo/var} which is used a fair bit. It
could also be argued that versionator should be used more which oddly
enough should also reduce the use of this bashism.
Thanks
Roy
--
[EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Mon, 2007-11-05 at 23:18 +, Roy Marples wrote:
On Mon, 2007-11-05 at 16:21 -0400, Mike Frysinger wrote:
we want the installed environment to be portable, not the build
environment.
i do not see any benefit from forcing the build environment to be pure
POSIX
compliant and i see many many detrimental problems.
Oh I don't know. Imagine how cool it would be for starting a new port.
1) Install PM
2) Wang on a portage tree
3) emerge ready to go
Obviously it's not quite that simple as portage requires python and
pkgcore require python, paludis requires tr1-whatever libs - but that's
the restriction of the PM used. Maybe one day Gentoo will have a PM that
doesn't require any of that and is just written in C and sh, using POSIX
libc where it can.
But enough pipe dreaming :)
Stop dreaming: Some very rudimentary thing like this already exists:
To bootstrap an alt/prefix instance on AIX, HPUX, whatever-unix-without-
sufficient-GNU-userland, we're using some prefix-launcher[1].
This simply is a package fetcher/patcher/builder/installer, where a
single package is defined by some .build-script, which is written in
plain bourne shell. You might (should) find some gentoo ideas if you
look inside it ;)
Now to install {python, bash, (prefix-)portage, wget, patch, diffutils,
findutils, gcc, etc.} one single command[2] (think emerge system) does
it all, just requiring GNU make, /bin/sh being bourne shell, some ansi-c
compiler, some (non-GNU) tar, gunzip, (non-GNU) sed, (non-GNU) whatever.
So the only non-native (not installed by default) thing required is GNU
make, which is available as binary package without any dependencies for
each unix I've seen.
[1] http://sourceforge.net/projects/prefix-launcher/
[2] http://prefix-launcher.wiki.sourceforge.net/
/haubi/
--
Michael Haubenwallner
Gentoo on a different level
--
[EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
While I still have access to the [EMAIL PROTECTED] email, I'll respond here. On Mon, 2007-11-05 at 10:22 +0100, Michael Haubenwallner wrote: On Sat, 2007-11-03 at 00:47 +, Roy Marples wrote: As it seems too few people really accept your suggestion, I feel it's time for me to chime in too, although I don't know what exactly POSIX-sh standard defines. Agreed, but (speaking for alt/prefix): Alt/prefix is designed to (mainly) work without superuser access on the target machine, which may also be Solaris, AIX, HP-UX and the like. /bin/sh on such a machine is not POSIX-shell, but old bourne-shell (unfortunately with bugs often). And it is _impossible_ to have sysadmins to get /bin/sh a POSIX-Shell nor to have that bugs fixed. But yes, on most machines there is /bin/ksh, which IMHO is POSIX compliant (maybe also with non-fixable bugs). Although I do not know yet for which _installed_ scripts it'd be really useful to have them non-bash in alt/prefix, I appreciate the discussion. To see benefits for alt/prefix too, it _might_ require that discussion going from requiring /bin/sh being POSIX-sh towards being bourne-shell... Actually you missed the mark completely. Nothing in the tree itself specifies what shell to use - instead it's the package manager. So the PM on Gentoo/Linux/FreeBSD *could* be /bin/sh and on the systems where /bin/sh is not possible to change to a POSIX compliant shell then it can still use /bin/bash or wherever it's installed. This also applies to the userland tools. If the ebuild or eclass *has* to use the GNU variants then it should either adjust $PATH so that it finds them first, or it prefixes them all with g, like it does on Gentoo/FreeBSD. None of this is technically challenging in itself, it's just that the key people who would have to do the work to make this possible have already given a flat out no. It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. More embeddable. More BSDable. More Linuxable - bash isn't the only linux shell, there are plently of others. More (generic) unix-able. Exactly so :) Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Mon, 2007-11-05 at 10:13 +, Roy Marples wrote: While I still have access to the [EMAIL PROTECTED] email, I'll respond here. On Mon, 2007-11-05 at 10:22 +0100, Michael Haubenwallner wrote: On Sat, 2007-11-03 at 00:47 +, Roy Marples wrote: As it seems too few people really accept your suggestion, I feel it's time for me to chime in too, although I don't know what exactly POSIX-sh standard defines. Agreed, but (speaking for alt/prefix): Alt/prefix is designed to (mainly) work without superuser access on the target machine, which may also be Solaris, AIX, HP-UX and the like. /bin/sh on such a machine is not POSIX-shell, but old bourne-shell (unfortunately with bugs often). And it is _impossible_ to have sysadmins to get /bin/sh a POSIX-Shell nor to have that bugs fixed. But yes, on most machines there is /bin/ksh, which IMHO is POSIX compliant (maybe also with non-fixable bugs). Although I do not know yet for which _installed_ scripts it'd be really useful to have them non-bash in alt/prefix, I appreciate the discussion. To see benefits for alt/prefix too, it _might_ require that discussion going from requiring /bin/sh being POSIX-sh towards being bourne-shell... Actually you missed the mark completely. Nothing in the tree itself specifies what shell to use - instead it's the package manager. So the PM on Gentoo/Linux/FreeBSD *could* be /bin/sh and on the systems where /bin/sh is not possible to change to a POSIX compliant shell then it can still use /bin/bash or wherever it's installed. So have the installed scripts to not require bash is another topic ? Ok then: Given that we want to have the tree more generic unix-able: What is the benefit from having the tree being POSIX- but not bourne-shell compatible, so one still needs bash on Solaris/AIX/HP-UX ? Because I'd say those three are the biggest substitutes for unix, while I'd call *BSD and Linux just unix derivates (although with enhancements)... This also applies to the userland tools. If the ebuild or eclass *has* to use the GNU variants then it should either adjust $PATH so that it finds them first, or it prefixes them all with g, like it does on Gentoo/FreeBSD. None of this is technically challenging in itself, it's just that the key people who would have to do the work to make this possible have already given a flat out no. In the early prefix days I had some portage enhancement, providing a wrapper-function around all coreutils/findutils/diffutils/grep/others, trying to find a GNU implementation for them. And if not found, try to map some args to the native ones (xargs -r fex - although didn't work as shell-function). But then we decided to always provide USERLAND=GNU in prefix and this portage patch was thrown away. It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. More embeddable. More BSDable. More Linuxable - bash isn't the only linux shell, there are plently of others. More (generic) unix-able. Exactly so :) Not really as long as not being bourne shell compatible like autoconf's configure. I won't trust to have a POSIX shell like /bin/ksh everywhere, but /bin/sh only, which usually is just a bourne shell on unix. /haubi/ -- Michael Haubenwallner Gentoo on a different level -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Monday 05 November 2007, Michael Haubenwallner wrote: On Mon, 2007-11-05 at 10:13 +, Roy Marples wrote: While I still have access to the [EMAIL PROTECTED] email, I'll respond here. On Mon, 2007-11-05 at 10:22 +0100, Michael Haubenwallner wrote: On Sat, 2007-11-03 at 00:47 +, Roy Marples wrote: As it seems too few people really accept your suggestion, I feel it's time for me to chime in too, although I don't know what exactly POSIX-sh standard defines. Agreed, but (speaking for alt/prefix): Alt/prefix is designed to (mainly) work without superuser access on the target machine, which may also be Solaris, AIX, HP-UX and the like. /bin/sh on such a machine is not POSIX-shell, but old bourne-shell (unfortunately with bugs often). And it is _impossible_ to have sysadmins to get /bin/sh a POSIX-Shell nor to have that bugs fixed. But yes, on most machines there is /bin/ksh, which IMHO is POSIX compliant (maybe also with non-fixable bugs). Although I do not know yet for which _installed_ scripts it'd be really useful to have them non-bash in alt/prefix, I appreciate the discussion. To see benefits for alt/prefix too, it _might_ require that discussion going from requiring /bin/sh being POSIX-sh towards being bourne-shell... Actually you missed the mark completely. Nothing in the tree itself specifies what shell to use - instead it's the package manager. So the PM on Gentoo/Linux/FreeBSD *could* be /bin/sh and on the systems where /bin/sh is not possible to change to a POSIX compliant shell then it can still use /bin/bash or wherever it's installed. So have the installed scripts to not require bash is another topic ? yes, and generally that's a baked topic. if your script is /bin/sh, then it must be POSIX compliant. if your script is /bin/bash, then you're encouraged to convert it to POSIX /bin/sh. but this is because the *runtime* environment is generally a lot more restricted than that of the *buildtime* environment. runtime implies a lot leaner requirements (think binary-only systems, embedded systems, production systems, etc...) than that of a development system (which requires everything in order to compile). Ok then: Given that we want to have the tree more generic unix-able: What is the benefit from having the tree being POSIX- but not bourne-shell compatible, so one still needs bash on Solaris/AIX/HP-UX ? Because I'd say those three are the biggest substitutes for unix, while I'd call *BSD and Linux just unix derivates (although with enhancements)... we want the installed environment to be portable, not the build environment. i do not see any benefit from forcing the build environment to be pure POSIX compliant and i see many many detrimental problems. -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] POSIX shell and portable
On Mon, 2007-11-05 at 14:21 +0100, Michael Haubenwallner wrote: Actually you missed the mark completely. Nothing in the tree itself specifies what shell to use - instead it's the package manager. So the PM on Gentoo/Linux/FreeBSD *could* be /bin/sh and on the systems where /bin/sh is not possible to change to a POSIX compliant shell then it can still use /bin/bash or wherever it's installed. So have the installed scripts to not require bash is another topic ? No, it's a valid topic. Either the profile could hook src_unpack or the ebuild could call a function to do this sed -e '1 s,^#!/bin/sh,#!/path/to/bash,' Either for all files in CONTENTS or all arguments passed. Ok then: Given that we want to have the tree more generic unix-able: What is the benefit from having the tree being POSIX- but not bourne-shell compatible, so one still needs bash on Solaris/AIX/HP-UX ? Because I'd say those three are the biggest substitutes for unix, while I'd call *BSD and Linux just unix derivates (although with enhancements)... The benefit is that we're not locked into any one toolset. This makes development of the tree more attractive to non Linux developers I would say. This also applies to the userland tools. If the ebuild or eclass *has* to use the GNU variants then it should either adjust $PATH so that it finds them first, or it prefixes them all with g, like it does on Gentoo/FreeBSD. None of this is technically challenging in itself, it's just that the key people who would have to do the work to make this possible have already given a flat out no. In the early prefix days I had some portage enhancement, providing a wrapper-function around all coreutils/findutils/diffutils/grep/others, trying to find a GNU implementation for them. And if not found, try to map some args to the native ones (xargs -r fex - although didn't work as shell-function). But then we decided to always provide USERLAND=GNU in prefix and this portage patch was thrown away. I dislike wrappers. The maintainers of revdep-rebuild say the same thing and I'm sure others would as well. An alternative would be to say have a list of ebuilds that don't require the GNU toolset (via an eclass or the ebuild itself) in a profile and slowly update the ebuilds and the lists when we can make them work with the desired userlands. If it requires gratuitous use of extensions then maybe the package itself should be patched upstream instead of us having to write overly complex ebuilds. Probably not the best idea for this, but workable. More (generic) unix-able. Exactly so :) Not really as long as not being bourne shell compatible like autoconf's configure. I won't trust to have a POSIX shell like /bin/ksh everywhere, but /bin/sh only, which usually is just a bourne shell on unix. As I said above, portage could change this. Think of it as an #ifdef :) Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On 05-11-2007 20:32:09 +, Roy Marples wrote: More (generic) unix-able. Exactly so :) Not really as long as not being bourne shell compatible like autoconf's configure. I won't trust to have a POSIX shell like /bin/ksh everywhere, but /bin/sh only, which usually is just a bourne shell on unix. As I said above, portage could change this. Think of it as an #ifdef :) We already (have to) do this, and it is not as trivial as you think. Though, as Mike pointed out, we sort of all agree that scripts outside of portage should be POSIX sh. -- Fabian Groffen Gentoo on a different level -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Monday 05 November 2007, Roy Marples wrote: On Mon, 2007-11-05 at 14:21 +0100, Michael Haubenwallner wrote: Actually you missed the mark completely. Nothing in the tree itself specifies what shell to use - instead it's the package manager. So the PM on Gentoo/Linux/FreeBSD *could* be /bin/sh and on the systems where /bin/sh is not possible to change to a POSIX compliant shell then it can still use /bin/bash or wherever it's installed. So have the installed scripts to not require bash is another topic ? No, it's a valid topic. he didnt say it was invalid, just a different topic Either the profile could hook src_unpack or the ebuild could call a function to do this sed -e '1 s,^#!/bin/sh,#!/path/to/bash,' fix the packages i say -mike signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] POSIX shell and portable
On Mon, 2007-11-05 at 16:21 -0400, Mike Frysinger wrote: we want the installed environment to be portable, not the build environment. i do not see any benefit from forcing the build environment to be pure POSIX compliant and i see many many detrimental problems. Oh I don't know. Imagine how cool it would be for starting a new port. 1) Install PM 2) Wang on a portage tree 3) emerge ready to go Obviously it's not quite that simple as portage requires python and pkgcore require python, paludis requires tr1-whatever libs - but that's the restriction of the PM used. Maybe one day Gentoo will have a PM that doesn't require any of that and is just written in C and sh, using POSIX libc where it can. But enough pipe dreaming :) Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Mon, 05 Nov 2007 23:18:43 + Roy Marples [EMAIL PROTECTED] wrote: paludis requires tr1-whatever libs 4.1 ships those, so you don't need to do anything there. Maybe one day Gentoo will have a PM that doesn't require any of that and is just written in C and sh, using POSIX libc where it can. Except it won't, because ebuilds require bash regardless of which package manager is being used. If you want to change that you'll have to rewrite the entire tree. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] POSIX shell and portable
On Tue, 2007-11-06 at 07:12 +, Ciaran McCreesh wrote: Except it won't, because ebuilds require bash regardless of which package manager is being used. If you want to change that you'll have to rewrite the entire tree. Az once said near enough the same thing about baselayout. And that's your view, your entitled to it, but it is not my view. Change a little bit here, a little bit there. Slowly does it. Yes, I know that a fair chunk of the tree will need a re-write, just in the same way that the init scripts got a re-write. It will take time, it will not happen magically over night. To think overwise is foolish :) But if the default shell remains bash but the PM had enough knobs on it so this could be changed (maybe per ebuild) in /etc then work *could* start. Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On 02-11-2007 17:35:08 +, Roy Marples wrote: I don't see them as inferior. I see them as more portable and less confusing. Please stop calling it more portable. The shell code you see in configure can in a way be called portable. Your POSIX compliant stuff isn't. In fact, by stating #!/bin/sh you actually make the code useless on a number of platforms, where it would have been working fine if there just were #!/bin/bash there. It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. -- Fabian Groffen Gentoo on a different level -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Sat, 2007-11-03 at 01:19 +0100, Fabian Groffen wrote: On 02-11-2007 17:35:08 +, Roy Marples wrote: I don't see them as inferior. I see them as more portable and less confusing. Please stop calling it more portable. But is it more portable as then then works across more than one shell. The shell code you see in configure can in a way be called portable. Your POSIX compliant stuff isn't. Sure it is - it should work on a shell that claims POSIX compliance. In fact, by stating #!/bin/sh you actually make the code useless on a number of platforms, where it would have been working fine if there just were #!/bin/bash there. Then the issue is to fix their sh so it follows POSIX compliance. As soon as a dash, bb or FreeBSD sh issue is found where it deviates from POSIX but it works on bash a lot of people say dash bug, therefore invalid It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. More embeddable. More BSDable. More Linuxable - bash isn't the only linux shell, there are plently of others. Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Sat, 2007-11-03 at 01:19 +0100, Fabian Groffen wrote: On 02-11-2007 17:35:08 +, Roy Marples wrote: I don't see them as inferior. I see them as more portable and less confusing. Please stop calling it more portable. The shell code you see in configure can in a way be called portable. Your POSIX compliant stuff isn't. In fact, by stating #!/bin/sh you actually make the code useless on a number of platforms, where it would have been working fine if there just were #!/bin/bash there. It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. Me again. If the shell isn't infact POSIX then you could always patch everything to change #!/bin/sh to #!/bin/bash and install bash or remove /bin/sh and link it to bash. As a lot of programs out there in the tree that are runtime (not ebuilds or init scripts) depend on /bin/sh being a POSIX shell. Thanks Roy -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Sat, 2007-11-03 at 01:19 +0100, Fabian Groffen wrote: On 02-11-2007 17:35:08 +, Roy Marples wrote: I don't see them as inferior. I see them as more portable and less confusing. Please stop calling it more portable. The shell code you see in configure can in a way be called portable. Your POSIX compliant stuff isn't. It is - in the sense the run in more shells. Roy's POSIX compliant stuff runs on at least a handful different shells. bash scripts runs on only one. (busybox ash can run some bash things but mostly they don't work) bash itself is portable but that is another story. -nc -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] POSIX shell and portable
On Sat, 2007-11-03 at 01:19 +0100, Fabian Groffen wrote: Please stop calling it more portable. The shell code you see in configure can in a way be called portable. Your POSIX compliant stuff isn't. In fact, by stating #!/bin/sh you actually make the code useless on a number of platforms, where it would have been working fine if there just were #!/bin/bash there. It seems to me that you actually mean more FreeBSD-able or something, which is a high price to pay for a relatively small part of Gentoo as a whole. Another way of looking at it is that you're forcing specific tools on people, where I am asking people to use standard POSIX tools. I guess it's because I'm an Engineer and you probably aren't. If the tool isn't up to the job, then fix the tool. If the tool doesn't claim any standards compliance then feel free to change it. Thanks Roy -- [EMAIL PROTECTED] mailing list
