Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 16 Feb 2007 16:44:15 -0700
"Daniel Robbins" <[EMAIL PROTECTED]> wrote:

> For speed, there are a *lot* of changes/improvements that could be
> made.

Yes, re-writing the backend in C whilst keeping /etc/init.d/
and /etc/conf.d/ as shell scripts is a good start :)

> What I would like to see is the ability to get to a login prompt
> before startup is actually completed. Have all the non-essential
> startup stuff run in the background.
> 
> Yes, this would require a sophisticated system since you could log in
> and start something that requires something that has not yet been
> started, and it would need to understand and deal with this
> appropriately and gracefully.
> 
> But I think it would be cool.
> 
> The perception of speed is based on how long you have to wait before
> you can do stuff.

Oh, that's easy. I can do this

RC_LOGIN_SERVICES="foo bar"

and rc will wait until sysinit, boot have completed and then wait for
foo AND bar to complete.

The side effect is at the point we login no service einfo stuff is
shown.

The obvious downside is that the user needs to state which services are
required for logging in, but they would know that much better than us.

Of course, that relies on RC_PARALLEL_STARTUP being set.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 16 Feb 2007 22:20:54 +0100
Paul de Vrieze <[EMAIL PROTECTED]> wrote:
> If that's what you want, don't use bash in the first place. I would
> agree that using bash for parsing is a pain in the but Daniel is
> right in that you're not going to be able to maintain posix
> compatibility. If you find an acceptable way to add the functionality
> to the network configuration files, it is ok, but sacrificing
> usability over an unmaintainable improvement doesn't work.

Provided that /bin/sh is /bin/bash then we can maintain the config_*
directive. 
> 
> If you want to speed up boot, the dependency generation is probably
> what's eating most time.

And that was solved in baselayout-1.13 where we already have the dep
resolver in C.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Daniel Robbins]

For speed, there are a *lot* of changes/improvements that could be made.

What I would like to see is the ability to get to a login prompt
before startup is actually completed. Have all the non-essential
startup stuff run in the background.

Yes, this would require a sophisticated system since you could log in
and start something that requires something that has not yet been
started, and it would need to understand and deal with this
appropriately and gracefully.

But I think it would be cool.

The perception of speed is based on how long you have to wait before
you can do stuff.

-Daniel

On 2/16/07, Paul de Vrieze <[EMAIL PROTECTED]> wrote:

On Friday 09 February 2007, Roy Marples wrote:
> On Thu, 8 Feb 2007 14:49:57 -0700
>
> "Daniel Robbins" <[EMAIL PROTECTED]> wrote:
> > In other words:
> >
> > busybox + single rcS file = fastest and simplest, smallest, best for
> > very small filesystems, not as flexible
> >
> > bash + gentoo baselayout = most flexible, biggest, slower, best for
> > feature-rich systems
> >
> > busybox + gentoo baselayout = ?
>
> FreeBSD sh + Gentoo baselayout = cold boot in around 4 seconds
> Going to multi-user from single user after a boot is under 2 seconds
> (times measured from when init starts rc - the difference is probably
> because the all my local mounts are still mounted)
>
> I have this running on a 2Ghz P4 Laptop right now. Admittedly, no
> network scripts are started expect for the loopback interface, but all
> default scripts + openvpn, ssh, dnsmasq, metalog and vixie-cron are
> started.
>
> Ladies and gentlemen, this has always been about one thing - speed.
> Ever since I got my 300Mhz Sparc64 to play around with FreeBSD, I've
> realised that baselayout + bash is just too damn slow.
>
> I think that's worth it.

If that's what you want, don't use bash in the first place. I would agree that
using bash for parsing is a pain in the but Daniel is right in that you're
not going to be able to maintain posix compatibility. If you find an
acceptable way to add the functionality to the network configuration files,
it is ok, but sacrificing usability over an unmaintainable improvement
doesn't work.

If you want to speed up boot, the dependency generation is probably what's
eating most time.

Paul

--
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Paul de Vrieze]

On Friday 09 February 2007, Roy Marples wrote:
> On Thu, 8 Feb 2007 14:49:57 -0700
>
> "Daniel Robbins" <[EMAIL PROTECTED]> wrote:
> > In other words:
> >
> > busybox + single rcS file = fastest and simplest, smallest, best for
> > very small filesystems, not as flexible
> >
> > bash + gentoo baselayout = most flexible, biggest, slower, best for
> > feature-rich systems
> >
> > busybox + gentoo baselayout = ?
>
> FreeBSD sh + Gentoo baselayout = cold boot in around 4 seconds
> Going to multi-user from single user after a boot is under 2 seconds
> (times measured from when init starts rc - the difference is probably
> because the all my local mounts are still mounted)
>
> I have this running on a 2Ghz P4 Laptop right now. Admittedly, no
> network scripts are started expect for the loopback interface, but all
> default scripts + openvpn, ssh, dnsmasq, metalog and vixie-cron are
> started.
>
> Ladies and gentlemen, this has always been about one thing - speed.
> Ever since I got my 300Mhz Sparc64 to play around with FreeBSD, I've
> realised that baselayout + bash is just too damn slow.
>
> I think that's worth it.

If that's what you want, don't use bash in the first place. I would agree that 
using bash for parsing is a pain in the but Daniel is right in that you're 
not going to be able to maintain posix compatibility. If you find an 
acceptable way to add the functionality to the network configuration files, 
it is ok, but sacrificing usability over an unmaintainable improvement 
doesn't work. 

If you want to speed up boot, the dependency generation is probably what's 
eating most time.

Paul

-- 
Paul de Vrieze
Gentoo Developer
Mail: [EMAIL PROTECTED]
Homepage: http://www.devrieze.net


pgpS9X5o4MDYx.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Harald van Dijk]

On Fri, Feb 09, 2007 at 11:49:05AM +, Roy Marples wrote:
> On Fri, 9 Feb 2007 12:05:52 +0100
> Harald van Dijk <[EMAIL PROTECTED]> wrote:
> > > One issue is that any valid character is allowed in WPAPSK,
> > > including ;
> > 
> > Good point, but excluding newline, right? I can't try it, but rt2500's
> > own settings file format does not allow for newline in WPAPSK.
> 
> No it doesn't, but the actual specification for IEEE 802.11 allows ANY
> character in SSID and keys - newlines, tabs, ;, etc.

Okay, thanks for the info. The only easy alternative I can think of is
using printf "%b", but IIRC printf is not universally supported yet, so
yeah, eval is probably the only thing that will offer all the needed
options. I still hope you can get some decent sanity checks for it.

> Just because the rt2500 proprietary config doesn't allow it doesn't
> mean we shouldn't either.

Proprietary?


pgp512noaYxaT.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 9 Feb 2007 12:05:52 +0100
Harald van Dijk <[EMAIL PROTECTED]> wrote:
> > One issue is that any valid character is allowed in WPAPSK,
> > including ;
> 
> Good point, but excluding newline, right? I can't try it, but rt2500's
> own settings file format does not allow for newline in WPAPSK.

No it doesn't, but the actual specification for IEEE 802.11 allows ANY
character in SSID and keys - newlines, tabs, ;, etc.

Just because the rt2500 proprietary config doesn't allow it doesn't
mean we shouldn't either.

Roy
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Harald van Dijk]

On Fri, Feb 09, 2007 at 10:17:21AM +, Roy Marples wrote:
> On Fri, 9 Feb 2007 10:51:29 +0100
> Harald van Dijk <[EMAIL PROTECTED]> wrote:
> > That works with your ; approach too, as well as with the
> > newline-separated approach:
> > 
> > replace="
> > 4d 1280 768 24
> > 5c 1400 1050 16
> > "
> > 
> > IFS=';
> > '
> > #set -f
> > set -- ${replace}
> > #set +f
> > unset IFS
> > 
> > for x ; do
> >   
> > done
> 
> Yes, but consider this
> 
> iwpriv_ESSID="
>   set AuthMode=WPAPSK;
>   set EncrypType=TKIP;
>   set WPAPSK=yourpasskey
> "
> 
> One issue is that any valid character is allowed in WPAPSK, including ;

Good point, but excluding newline, right? I can't try it, but rt2500's
own settings file format does not allow for newline in WPAPSK. So you
could simply only use newline as the separator whenever ; is allowed.

> $ foo="'one '\'' two' \
>'three ; four' \
>   'five \ six' \
> "  
> $ eval set -- "${foo}" ; for x in "$@" ; do echo $x; done
> one ' two
> three ; four
> five \ six
> $
> 
> Demonstrates that we can achieve multi-line arrays like so, without the
> need to set shell opts or alter IFS

Yeah, you just use eval. :) It has its uses, but personally I would
avoid it where possible because it's so easy to get wrong. Just try it
with

iwpriv_ESSID="
set\ AuthMode=WPAPSK
set\ EncrypType=TKIP
set\ WPAPSK=pass;key
"

Failing to properly escape the ; is a user error, but it becomes
difficult for baselayout to handle that error properly. Similarly, the
backslash-newline combinations are easily left out when you're already
editing a string. The format itself I have no problems with, though, so
if you have a way to nicely deal with accidental bad input, perhaps
using some simple scans, please go for it.

(Since the configuration files are only writable by root, I'll ignore
the possibility of intentional bad input here.)


pgpVARxqQ8aGi.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 9 Feb 2007 10:51:29 +0100
Harald van Dijk <[EMAIL PROTECTED]> wrote:
> That works with your ; approach too, as well as with the
> newline-separated approach:
> 
> replace="
> 4d 1280 768 24
> 5c 1400 1050 16
> "
> 
> IFS=';
> '
> #set -f
> set -- ${replace}
> #set +f
> unset IFS
> 
> for x ; do
>   
> done

Yes, but consider this

iwpriv_ESSID="
set AuthMode=WPAPSK;
set EncrypType=TKIP;
set WPAPSK=yourpasskey
"

One issue is that any valid character is allowed in WPAPSK, including ;

$ foo="'one '\'' two' \
 'three ; four' \
'five \ six' \
"  
$ eval set -- "${foo}" ; for x in "$@" ; do echo $x; done
one ' two
three ; four
five \ six
$

Demonstrates that we can achieve multi-line arrays like so, without the
need to set shell opts or alter IFS

Thanks

Roy
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Harald van Dijk]

On Fri, Feb 09, 2007 at 09:29:04AM +, Roy Marples wrote:
> On Fri, 09 Feb 2007 01:03:04 -0800
> Donnie Berkholz <[EMAIL PROTECTED]> wrote:
> 
> > Jakub Moc wrote:
> > > Mike Frysinger napsal(a):
> > >> On Thursday 08 February 2007, Roy Marples wrote:
> > >>> The actual scripts themselves can be re-worked if they need to be
> > >>> - this problem only when the arrays are used in config files.
> > >> i guess my point was i think we really need to be consistent
> > >> here ... either arrays are OK for init.d scripts or they're not OK
> > >>
> > >> did you get a chance to see how hard it would be to integrate the
> > >> bash array code ?
> > > 
> > > sys-apps/855resolution and sys-apps/915resolution use arrays in
> > > config files, and there's not much way around it ATM if you want to
> > > replace multiple BIOS modes...
> > > 
> > > 
> > > # to replace more than one mode do something like this
> > > #replace=(
> > > # "4d 1280 768 24"
> > > # "5c 1400 1050 16"
> > > #)
> > 
> > How about this?
> > 
> > replace="
> > '4d 1280 768 24'
> > '5c 1400 1050 16'
> > "
> > 
> 
> Actually, that may work better than my delimited with ; approach.
> We could then do
> 
> eval set -- "${replace}"
> for x in "$@" ; do
>  
> done

That works with your ; approach too, as well as with the
newline-separated approach:

replace="
4d 1280 768 24
5c 1400 1050 16
"

IFS=';
'
#set -f
set -- ${replace}
#set +f
unset IFS

for x ; do
  
done


pgprrqWQ2pIf0.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 09 Feb 2007 17:27:58 +0900
Georgi Georgiev <[EMAIL PROTECTED]> wrote:

> Quoting Roy Marples <[EMAIL PROTECTED]>:
> >
> > FreeBSD sh + Gentoo baselayout = cold boot in around 4 seconds
> > Going to multi-user from single user after a boot is under 2 seconds
> > (times measured from when init starts rc - the difference is
> > probably because the all my local mounts are still mounted)
> >
> > I have this running on a 2Ghz P4 Laptop right now. Admittedly, no
> > network scripts are started expect for the loopback interface, but
> > all default scripts + openvpn, ssh, dnsmasq, metalog and vixie-cron
> > are started.
> 
> And what are the numbers with bash? Just curious.

About a second or two slower.
ie not much difference at all in the grand scheme of things. But that's
to be expected as most of the gain has come from moving existing bash
code to C - except for the init script's themselves.

Also note that when this gets released at some point, Linux will be
slower as it has to load udev. FreeBSD has loaded devfs and done its
magic before init, so there will always be some skew as to which is
faster/better.

But that's why *I* am doing it.

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Thu, 8 Feb 2007 20:27:06 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> On Thursday 08 February 2007, Ned Ludd wrote:
> > Please read over what's been talked about elsewhere in this thread.
> > He is not trying to break existing functionality at all. Only
> > extend it to be posix aware (additionally)
> 
> erm, no ... our code is a superset of POSIX, so technically yes he is
> breaking existing functionality and doing the quite opposite of
> extending -mike

True, but remember that this only applies for when /bin/sh isn't bash.
So this will mainly affect FreeBSD and embedded, but it at least gives
them a better chance of working from the get-go if baselayout at least
supports it.

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Fri, 09 Feb 2007 01:03:04 -0800
Donnie Berkholz <[EMAIL PROTECTED]> wrote:

> Jakub Moc wrote:
> > Mike Frysinger napsal(a):
> >> On Thursday 08 February 2007, Roy Marples wrote:
> >>> The actual scripts themselves can be re-worked if they need to be
> >>> - this problem only when the arrays are used in config files.
> >> i guess my point was i think we really need to be consistent
> >> here ... either arrays are OK for init.d scripts or they're not OK
> >>
> >> did you get a chance to see how hard it would be to integrate the
> >> bash array code ?
> > 
> > sys-apps/855resolution and sys-apps/915resolution use arrays in
> > config files, and there's not much way around it ATM if you want to
> > replace multiple BIOS modes...
> > 
> > 
> > # to replace more than one mode do something like this
> > #replace=(
> > #   "4d 1280 768 24"
> > #   "5c 1400 1050 16"
> > #)
> 
> How about this?
> 
> replace="
>   '4d 1280 768 24'
>   '5c 1400 1050 16'
> "
> 

Actually, that may work better than my delimited with ; approach.
We could then do

eval set -- "${replace}"
for x in "$@" ; do
 
done

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Donnie Berkholz]

Jakub Moc wrote:
> Mike Frysinger napsal(a):
>> On Thursday 08 February 2007, Roy Marples wrote:
>>> The actual scripts themselves can be re-worked if they need to be -
>>> this problem only when the arrays are used in config files.
>> i guess my point was i think we really need to be consistent here ... either 
>> arrays are OK for init.d scripts or they're not OK
>>
>> did you get a chance to see how hard it would be to integrate the bash array 
>> code ?
> 
> sys-apps/855resolution and sys-apps/915resolution use arrays in config
> files, and there's not much way around it ATM if you want to replace
> multiple BIOS modes...
> 
> 
> # to replace more than one mode do something like this
> #replace=(
> # "4d 1280 768 24"
> # "5c 1400 1050 16"
> #)

How about this?

replace="
'4d 1280 768 24'
'5c 1400 1050 16'
"



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Network configuration and bash

[Georgi Georgiev]

Quoting Roy Marples <[EMAIL PROTECTED]>:


FreeBSD sh + Gentoo baselayout = cold boot in around 4 seconds
Going to multi-user from single user after a boot is under 2 seconds
(times measured from when init starts rc - the difference is probably
because the all my local mounts are still mounted)

I have this running on a 2Ghz P4 Laptop right now. Admittedly, no
network scripts are started expect for the loopback interface, but all
default scripts + openvpn, ssh, dnsmasq, metalog and vixie-cron are
started.


And what are the numbers with bash? Just curious.



This message was sent using IMP, the Internet Messaging Program.


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Jakub Moc]

Mike Frysinger napsal(a):
> On Thursday 08 February 2007, Roy Marples wrote:
>> The actual scripts themselves can be re-worked if they need to be -
>> this problem only when the arrays are used in config files.
> 
> i guess my point was i think we really need to be consistent here ... either 
> arrays are OK for init.d scripts or they're not OK
> 
> did you get a chance to see how hard it would be to integrate the bash array 
> code ?

sys-apps/855resolution and sys-apps/915resolution use arrays in config
files, and there's not much way around it ATM if you want to replace
multiple BIOS modes...


# to replace more than one mode do something like this
#replace=(
#   "4d 1280 768 24"
#   "5c 1400 1050 16"
#)


-- 
Best regards,

 Jakub Moc
 mailto:[EMAIL PROTECTED]
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Thursday 08 February 2007, Ned Ludd wrote:
> Please read over what's been talked about elsewhere in this thread. He
> is not trying to break existing functionality at all. Only extend it to
> be posix aware (additionally)

erm, no ... our code is a superset of POSIX, so technically yes he is breaking 
existing functionality and doing the quite opposite of extending
-mike


pgplxEo0WIAJC.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Martin Jackson]

He's not going to waste someone else's time, and as he said there will be 
compatibility with current configuration files, I don't think there's any 
downside to users.




FWIW, speaking as a user, I value stability over speed.  But if I have a 
promise of stability (i.e. my current configs will still work), I have 
no problem with Roy scratching his speed itch. :)


Thanks,
Marty
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Diego 'Flameeyes' Pettenò]

On Friday 09 February 2007, Ned Ludd wrote:
> baselayout is only about a half of a meg these days and probably
> getting smaller/faster with the addition of the multicall rc/runscript
> work he has been doing.
>
> Adding bash also requires ncurses which in turn mostly requires having
> a c++ aware compiler or using the nocxx,minimal flags. Even with those
> flags enabled I'm seeing 3M going to ncurses+bash. So I can for sure
> see the benefits.
>
> Also for a moment lets stop and think. Some XYZ update breaks
> ncurses/bash. Supporting this gives us a nice alternative way to still
> boot our boxes for rescue using ash or another shell which might not
> have such big deps.

From where I stand I can see Ned's point just fine: I'm interested in both 
having a sane baselayout that doesn't break on bash upgrade (I've seen the 
breakage with 3.1, 3.2.. I also masked bash 3.1 while Roy fixed baselayout 
for that version), and in a baselayout that can run on medium embedded 
systems (and not just "for fun", trust me), so I wouldn't dismiss Roy's work 
as "unneeded" and/or not useful to anyone.

He's not going to waste someone else's time, and as he said there will be 
compatibility with current configuration files, I don't think there's any 
downside to users.

-- 
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, Sound, ALSA, PAM, KDE, CJK, Ruby ...


pgpg13D9POXYc.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Thu, 8 Feb 2007 14:49:57 -0700
"Daniel Robbins" <[EMAIL PROTECTED]> wrote:
> In other words:
> 
> busybox + single rcS file = fastest and simplest, smallest, best for
> very small filesystems, not as flexible
> 
> bash + gentoo baselayout = most flexible, biggest, slower, best for
> feature-rich systems
> 
> busybox + gentoo baselayout = ?

FreeBSD sh + Gentoo baselayout = cold boot in around 4 seconds
Going to multi-user from single user after a boot is under 2 seconds
(times measured from when init starts rc - the difference is probably
because the all my local mounts are still mounted)

I have this running on a 2Ghz P4 Laptop right now. Admittedly, no
network scripts are started expect for the loopback interface, but all
default scripts + openvpn, ssh, dnsmasq, metalog and vixie-cron are
started.

Ladies and gentlemen, this has always been about one thing - speed.
Ever since I got my 300Mhz Sparc64 to play around with FreeBSD, I've
realised that baselayout + bash is just too damn slow.

I think that's worth it.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ned Ludd]

On Thu, 2007-02-08 at 14:49 -0700, Daniel Robbins wrote:
> On 2/8/07, Ned Ludd <[EMAIL PROTECTED]> wrote:
> > As somebody that's had to hand write many of those kinds of scripts. A
> > single rcS is not very ideal. Our init scripts are in fact mostly usable
> > by busybox. Granted there are a few special special cases, but then Roy
> > is offering to update those for free. One of the larger problems really
> > boils down to many packages provide default init.d scripts and these
> > expect the existing baselayout only. That will be a bigger feat to deal
> > with later on down the road.
> 
> Developers will then need to test their init.d scripts to ensure that
> they are compatible with busybox. This is asking a lot of work of
> people just so you can use Gentoo's initscripts for something they are
> not really ideal for. 

I don't think anybody can/would expect that. They don't test for
hardened or uClibc now before stabilizing a pkg. What would be nice 
to see is if a maintainer is offered a posix compliant init.d script
that they merge it or allow those to be merged for a pkg they maintain
as long as it does not degrade functionality.

> Any time a script is updated a new rev of a
> package is required, and this does impact users and will cause
> packages to be rebuilt when a user does "emerge -u". So I think this
> should be weighed against the potential benefits of baselayout +
> busybox.

busybox is not Roys underlying goal as far as I understand. I think he 
simply mentioned it as an example of another group who wishes to unify 
efforts and have an interest in getting away from arrays where feasible.

> If you are targetting something smaller than 32MB, then maybe busybox
> is appropriate. But you are trying to go really small, then you
> probably don't want all the extra junk in our initscripts. And if you
> are _not_ trying to go really small, then put bash in your filesystem,
> not busybox, and the initscripts will work. If bash isn't fast enough
> from a boot time perspective, then the gentoo initscripts certainly
> aren't going to be fast enough either.
> 
> In other words:
> 
> busybox + single rcS file = fastest and simplest, smallest, best for
> very small filesystems, not as flexible
> 
> bash + gentoo baselayout = most flexible, biggest, slower, best for
> feature-rich systems
> 

> busybox + gentoo baselayout = ?

It's been done in the past by end users. Before there were only about 4 
changes needed to make it work. That all changed when bash arrays were 
introduced.

> I think that in 99 out of 100 cases, if you have room for baselayout,
> then you probably have room for bash too. And in 99 out of 100 cases,
> if you can deal with the load time of baselayout, then you can deal
> with the overhead that might be incurred from having bash.
> 
> I'm just pointing out that it's not an obviously good combination. In
> the grand scheme of things, maybe it's not a great use of developer
> resources. Or, maybe I'm wrong and it is a great idea.

His time and resources. His "itch"

> Personally I think that "baselayout + busybox" may be cool, but adding
> an aftermarket sunroof to your car can be cool too. But that doesn't
> mean it's worth the effort :)

I don't think those who are not interested in this will be burden by 
any extra effort. Worst case is maybe getting a bug assigned to you 
which offers a posix replacement/update for the default init.d a pkg 
you maintain might provide.

> Really, it's hard for me to imagine many scenarios where you really
> need the flexibility of baselayout but can't squeeze in bash. And I
> have a pretty good imagination.

baselayout is only about a half of a meg these days and probably
getting smaller/faster with the addition of the multicall rc/runscript 
work he has been doing.

Adding bash also requires ncurses which in turn mostly requires having
a c++ aware compiler or using the nocxx,minimal flags. Even with those 
flags enabled I'm seeing 3M going to ncurses+bash. So I can for sure 
see the benefits.

Also for a moment lets stop and think. Some XYZ update breaks 
ncurses/bash. Supporting this gives us a nice alternative way to still 
boot our boxes for rescue using ash or another shell which might not 
have such big deps.


-- 
Ned Ludd <[EMAIL PROTECTED]>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Daniel Robbins]

On 2/8/07, Ned Ludd <[EMAIL PROTECTED]> wrote:

As somebody that's had to hand write many of those kinds of scripts. A
single rcS is not very ideal. Our init scripts are in fact mostly usable
by busybox. Granted there are a few special special cases, but then Roy
is offering to update those for free. One of the larger problems really
boils down to many packages provide default init.d scripts and these
expect the existing baselayout only. That will be a bigger feat to deal
with later on down the road.


Developers will then need to test their init.d scripts to ensure that
they are compatible with busybox. This is asking a lot of work of
people just so you can use Gentoo's initscripts for something they are
not really ideal for. Any time a script is updated a new rev of a
package is required, and this does impact users and will cause
packages to be rebuilt when a user does "emerge -u". So I think this
should be weighed against the potential benefits of baselayout +
busybox.

If you are targetting something smaller than 32MB, then maybe busybox
is appropriate. But you are trying to go really small, then you
probably don't want all the extra junk in our initscripts. And if you
are _not_ trying to go really small, then put bash in your filesystem,
not busybox, and the initscripts will work. If bash isn't fast enough
from a boot time perspective, then the gentoo initscripts certainly
aren't going to be fast enough either.

In other words:

busybox + single rcS file = fastest and simplest, smallest, best for
very small filesystems, not as flexible

bash + gentoo baselayout = most flexible, biggest, slower, best for
feature-rich systems

busybox + gentoo baselayout = ?

I think that in 99 out of 100 cases, if you have room for baselayout,
then you probably have room for bash too. And in 99 out of 100 cases,
if you can deal with the load time of baselayout, then you can deal
with the overhead that might be incurred from having bash.

I'm just pointing out that it's not an obviously good combination. In
the grand scheme of things, maybe it's not a great use of developer
resources. Or, maybe I'm wrong and it is a great idea.

Personally I think that "baselayout + busybox" may be cool, but adding
an aftermarket sunroof to your car can be cool too. But that doesn't
mean it's worth the effort :)

Really, it's hard for me to imagine many scenarios where you really
need the flexibility of baselayout but can't squeeze in bash. And I
have a pretty good imagination.

-Daniel
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ned Ludd]

On Thu, 2007-02-08 at 13:23 -0700, Daniel Robbins wrote:
> I sort of missed this conversation, so apologies in advance if this
> has already been covered, but wanted to say that gentoo's initscripts
> are generally not suited for embedded systems.
> 
> So making baselayout busybox-compatible doesn't seem to be worth the
> disruption and headaches it would cause. 

Please read over what's been talked about elsewhere in this thread. He
is not trying to break existing functionality at all. Only extend it to
be posix aware (additionally) 

> It would be disruptive for
> gentoo developers who would need to be extra-careful in maintaining
> their initscripts to ensure busybox compatibility. Not to mention the
> potential disruption for users.

There is no reason this has to be disruptive to the users who don't care
about this functionality.

> If you are building an embedded system using busybox, then generally
> you will want a single /etc/init.d/rcS script that starts all the
> stuff you need.

As somebody that's had to hand write many of those kinds of scripts. A
single rcS is not very ideal. Our init scripts are in fact mostly usable
by busybox. Granted there are a few special special cases, but then Roy
is offering to update those for free. One of the larger problems really
boils down to many packages provide default init.d scripts and these
expect the existing baselayout only. That will be a bigger feat to deal
with later on down the road.

> -Daniel
> 
> On 2/8/07, Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > On Thursday 08 February 2007, Roy Marples wrote:
> > > Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > > > On Wednesday 07 February 2007, Roy Marples wrote:
> > > > > In the current code I'm running it's only the network stuff that
> > > > > uses arrays. If you're thinking about /sbin/functions.sh, well that
> > > > > can stay as bash as it's not used by baselayout anymore.
> > > >
> > > > some init.d scripts use arrays as well
> > >
> > > Do we know which ones?
> >
> > grep for it :p
> > netmount for sure right now
> >
> > > The actual scripts themselves can be re-worked if they need to be -
> > > this problem only when the arrays are used in config files.
> >
> > i guess my point was i think we really need to be consistent here ... either
> > arrays are OK for init.d scripts or they're not OK
> >
> > did you get a chance to see how hard it would be to integrate the bash array
> > code ?
> > -mike
> >
> >
-- 
Ned Ludd <[EMAIL PROTECTED]>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Thursday 08 February 2007, Roy Marples wrote:
> Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > did you get a chance to see how hard it would be to integrate the
> > bash array code ?
>
> Integrate into what? You mean integrate into other shells?

mmm i thought you were looking to parse the config files directly from 
baselayout and not via shell, so i was referring to integrating arrays into 
baselayout
-mike


pgp6VXSWIz5rl.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Daniel Robbins]

I sort of missed this conversation, so apologies in advance if this
has already been covered, but wanted to say that gentoo's initscripts
are generally not suited for embedded systems.

So making baselayout busybox-compatible doesn't seem to be worth the
disruption and headaches it would cause. It would be disruptive for
gentoo developers who would need to be extra-careful in maintaining
their initscripts to ensure busybox compatibility. Not to mention the
potential disruption for users.

If you are building an embedded system using busybox, then generally
you will want a single /etc/init.d/rcS script that starts all the
stuff you need.

-Daniel

On 2/8/07, Mike Frysinger <[EMAIL PROTECTED]> wrote:

On Thursday 08 February 2007, Roy Marples wrote:
> Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > On Wednesday 07 February 2007, Roy Marples wrote:
> > > In the current code I'm running it's only the network stuff that
> > > uses arrays. If you're thinking about /sbin/functions.sh, well that
> > > can stay as bash as it's not used by baselayout anymore.
> >
> > some init.d scripts use arrays as well
>
> Do we know which ones?

grep for it :p
netmount for sure right now

> The actual scripts themselves can be re-worked if they need to be -
> this problem only when the arrays are used in config files.

i guess my point was i think we really need to be consistent here ... either
arrays are OK for init.d scripts or they're not OK

did you get a chance to see how hard it would be to integrate the bash array
code ?
-mike



--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Thu, 8 Feb 2007 13:01:08 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > > some init.d scripts use arrays as well
> >
> > Do we know which ones?
> 
> grep for it :p
> netmount for sure right now

Well, netmount is baselayout, so that will kinda be done by default :)

> i guess my point was i think we really need to be consistent here ...
> either arrays are OK for init.d scripts or they're not OK

I say no, as I'd like eventually anything in /etc/init.d,conf.d to be
useable by any shell. It's not especially hard to make existing scripts
in the tree work either. I'm happy to do this myself.

> did you get a chance to see how hard it would be to integrate the
> bash array code ?

Integrate into what? You mean integrate into other shells?

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Thursday 08 February 2007, Roy Marples wrote:
> Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > On Wednesday 07 February 2007, Roy Marples wrote:
> > > In the current code I'm running it's only the network stuff that
> > > uses arrays. If you're thinking about /sbin/functions.sh, well that
> > > can stay as bash as it's not used by baselayout anymore.
> >
> > some init.d scripts use arrays as well
>
> Do we know which ones?

grep for it :p
netmount for sure right now

> The actual scripts themselves can be re-worked if they need to be -
> this problem only when the arrays are used in config files.

i guess my point was i think we really need to be consistent here ... either 
arrays are OK for init.d scripts or they're not OK

did you get a chance to see how hard it would be to integrate the bash array 
code ?
-mike


pgpc7BSebhvSu.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Wed, 7 Feb 2007 23:42:14 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> On Wednesday 07 February 2007, Roy Marples wrote:
> > In the current code I'm running it's only the network stuff that
> > uses arrays. If you're thinking about /sbin/functions.sh, well that
> > can stay as bash as it's not used by baselayout anymore.
> 
> some init.d scripts use arrays as well
> -mike

Do we know which ones?
The actual scripts themselves can be re-worked if they need to be -
this problem only when the arrays are used in config files.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Wednesday 07 February 2007, Roy Marples wrote:
> In the current code I'm running it's only the network stuff that uses
> arrays. If you're thinking about /sbin/functions.sh, well that can stay
> as bash as it's not used by baselayout anymore.

some init.d scripts use arrays as well
-mike


pgpimDq3QEneT.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Francesco Riosa]

Roy Marples ha scritto:

Hi List

As some of you may be aware, I've started work on baselayout-2 which is
basically re-tooling it in C. One of the side goals is to eliminate the
need for using bash. You'll be pleased to know that it's working well
enough to boot Gentoo/FreeBSD.


[...]

what about to keep compatibility with at least this form:
ifconfig_eth0=(
  "10.1.1.{1,2} netmask 255.255.255.0"
)

and maybe hooks to external code to be executed
{pre,post}ip_link_set
{pre,post}ip_addr_{add,del}
etc...

A fast look at Bash code show that i should be maintenable

~/tmp/bash-3.2 $ wc braces.* array*
  630  2208 14019 braces.c
  932  2843 19720 array.c
  111   498  3817 array.h
  837  2896 20407 arrayfunc.c
   58   300  2289 arrayfunc.h
 2568  8745 60252 total

Regards,
Francesco
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Wed, 7 Feb 2007 01:03:56 -0800
"Robin H. Johnson" <[EMAIL PROTECTED]> wrote:

> On Tue, Feb 06, 2007 at 11:09:11PM +, Roy Marples wrote:
> > On Tue, 6 Feb 2007 21:14:49 +
> > Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
> > > Why? What's wrong with requiring a shell that supports various
> > > features beyond what POSIX specifies? Granted, choice of shell is
> > > good, but not if it's at the expense of functionality or ease of
> > > use.
> > H, just how many features should a config file have beyond the
> > setting of variables?
> I make reasonable use of some of the advanced stuff already:
> http://www.orbis-terrarum.net/~robbat2/conf.d-multihomed
> 

OK, so aside from arrays, you're using foo=${!bar} which could be
re-written as

eval foo=\"\$${bar}\"

That of course is your choice.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Robin H. Johnson]

On Tue, Feb 06, 2007 at 11:09:11PM +, Roy Marples wrote:
> On Tue, 6 Feb 2007 21:14:49 +
> Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
> > Why? What's wrong with requiring a shell that supports various
> > features beyond what POSIX specifies? Granted, choice of shell is
> > good, but not if it's at the expense of functionality or ease of use.
> H, just how many features should a config file have beyond the
> setting of variables?
I make reasonable use of some of the advanced stuff already:
http://www.orbis-terrarum.net/~robbat2/conf.d-multihomed

-- 
Robin Hugh Johnson
Gentoo Linux Developer
E-Mail : [EMAIL PROTECTED]
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85


pgpU3IyoJ7lQL.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 22:27:46 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> On Tuesday 06 February 2007, Roy Marples wrote:
> > No, I'm just interesting in killing bash array requirements in the
> > network config file.
> 
> you need to kill them everywhere then ... network config isnt the
> only file that utilizes arrays
> -mike

In the current code I'm running it's only the network stuff that uses
arrays. If you're thinking about /sbin/functions.sh, well that can stay
as bash as it's not used by baselayout anymore.

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Alin Năstac]

Roy Marples wrote:
> This email is about network configuration. Before I joined Gentoo,
> network configuration was done in bash arrays like so (note, that the
> variable name was changed in baselayout-1.11)
>
> ifconfig_eth0=(
>   "10.1.1.1 netmask 255.255.255.0"
>   "10.1.1.2 netmask 255.255.255.0"
> )
>
> This is all well and good, but only bash and zsh can use it. Infact I'm
> guilty as I embraced arrays and used them as the basis for our entire
> network scripts. They have also served us well, but as they are also
> array based they will have to be re-written too.
>
> So, to free baselayout of forcing bash down our throats I/we am/are
> looking at re-writing our network setup, including configuration.
>   
How about leaving baselayout depending on bash and make baselayout-lite
do whatever you want instead?
Such an upgrade will be a major pita for our entire user base, unless of
course you make conf.d/net syntax transition as easy as running
dispatch-conf.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Network configuration and bash

[Luca Barbato]

Roy Marples wrote:
> Hi List
> 
> As some of you may be aware, I've started work on baselayout-2 which is
> basically re-tooling it in C. One of the side goals is to eliminate the
> need for using bash. You'll be pleased to know that it's working well
> enough to boot Gentoo/FreeBSD.

Hm...

> So, to free baselayout of forcing bash down our throats I/we am/are
> looking at re-writing our network setup, including configuration.

What about adding array support in the C retool?

What about adding this particular feature in busybox and just use it?

lu - against having a completely pointless transition...
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Jakub Moc]

Roy Marples napsal(a):
> This email is about network configuration. Before I joined Gentoo,
> network configuration was done in bash arrays like so (note, that the
> variable name was changed in baselayout-1.11)
> 
> ifconfig_eth0=(
>   "10.1.1.1 netmask 255.255.255.0"
>   "10.1.1.2 netmask 255.255.255.0"
> )
> 
> This is all well and good, but only bash and zsh can use it. Infact I'm
> guilty as I embraced arrays and used them as the basis for our entire
> network scripts. They have also served us well, but as they are also
> array based they will have to be re-written too.

To be honest, I hate the idea of rewriting the network config; being
able to specify things like

config_eth0=( "10.1.1.{1..10}/24 brd +" )

(yes, it really expands correctly and works!) is extremely useful and
something you don't find in any other distro out there last time I've
checked. You can assign tons of IP address on a single line, or just
with a couple of lines if you need different subnets, instead of going
thru the boring and error-prone task of specifying every single address
separately.

Why kill such useful features just to get rid of arrays and bash?

Kevin F. Quinn napsal(a):
> Surely you would provide choice by providing different baselayout
> packages; one tailored for embedded systems that only have busybox,
> one for general purpose use, etc.  That's what the virtual is for,
> isn't it?

+1


-- 
Best regards,

 Jakub Moc
 mailto:[EMAIL PROTECTED]
 GPG signature:
 http://subkeys.pgp.net:11371/pks/lookup?op=get&search=0xCEBA3D9E
 Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95  B30F 8717 D5FD CEBA 3D9E

 ... still no signature   ;)



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Network configuration and bash

[Graham Murray]

Roy Marples <[EMAIL PROTECTED]> writes:

> H, just how many features should a config file have beyond the
> setting of variables?

In the case of networking, the ability to define the functions for the
various hooks. In most systems these will not be needed, but where
policy routing etc is used the hooks need to be defined somewhere, but
not necessarily the configuration file as they are at the moment. 
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Tuesday 06 February 2007, Roy Marples wrote:
> No, I'm just interesting in killing bash array requirements in the
> network config file.

you need to kill them everywhere then ... network config isnt the only file 
that utilizes arrays
-mike


pgpj1mYCh71a3.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Daniel Barkalow]

On Tue, 6 Feb 2007, Roy Marples wrote:

> This email is about network configuration. Before I joined Gentoo,
> network configuration was done in bash arrays like so (note, that the
> variable name was changed in baselayout-1.11)
> 
> ifconfig_eth0=(
>   "10.1.1.1 netmask 255.255.255.0"
>   "10.1.1.2 netmask 255.255.255.0"
> )
> 
> This is all well and good, but only bash and zsh can use it.

That's not true. Only bash and zsh can *source* it. Aside from the fact 
that most other shells don't have any way of storing the results of 
reading such a file in an accessible way, that format is easy enough to 
parse with a bit of sed.

> Who's got any bright ideas for a new config then? Lets brain storm!

If it's necessary to change at all, I'd vote for sections in square 
brackets, applying to everything until the next square bracket, and lines 
with a key, followed by a ':' or '=', followed by optional whitespace, 
followed by any number of values, optionally in quotes with 
backslash-escaped quotes and backslashes, separated by whitespace, with 
the value lists for duplicate keys being concatenated.

It's not hard to parse (assuming, again, that you have some way to store 
the results), users coming from Windows can write it as "ini" files, users 
who like Java can write it as resource files, and old-school Unix types 
can write RFC822 headers. And they're all mutually comprehensible if the 
parser is just reasonably lenient.

-Daniel
*This .sig left intentionally blank*
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 19:39:03 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> On Tuesday 06 February 2007, Roy Marples wrote:
> > Hopefully I've justified this enough :)
> 
> justified what ?  this thread started off killing bash array
> requirements in the network config file and now it looks like your
> killing the shell everywhere ?

No, I'm just interesting in killing bash array requirements in the
network config file.
Once that is done, then you can use whatever shell you wish as far as
the init scripts allow, which will hopefully be all of them in portage.

> to be honest, unless the new code is really good, it's still going to
> be not suitable for real embedded systems ... sure, i run Gentoo on
> all my embedded machines, but lets be honest, these arent even close
> to real embedded -mike

I cannot argue that as I have no embedded systems of which to speak.
However, some users and devs who use embedded systems have expressed an
interest so I'll let them argue as they will.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Kevin F. Quinn]

On Tue, 6 Feb 2007 23:26:32 +
Roy Marples <[EMAIL PROTECTED]> wrote:

> On Tue, 6 Feb 2007 21:28:04 +
> Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
> 
> > I think it's more that you're expected to justify *why* the bash
> > requirement is so bad, given the cost of changing.
> 
> 1) Lack of choice.
> Gentoo is all about giving the user choice. baselayout even supports
> other init systems when requested.

Surely you would provide choice by providing different baselayout
packages; one tailored for embedded systems that only have busybox, one
for general purpose use, etc.  That's what the virtual is for, isn't it?

> 2) Speed.
> Bash is one of the slowest shells around for looping.
> However, it also requires less forking due to it's nice built-ins.
> This does of course only work with bash and not other shells.

Restricting everything to 'sh' I think is more likely to slow things
down than anything else.  Apart from the forking issue that you
mention, builtins are different - '[' for example is about 30% slower
than '[[' in bash (which is what's implementing sh on Gentoo Linux).  I
wonder how much time would be saved on Gentoo Linux by replacing [ with
[[ throughout the init scripts; maybe a percentage point?

If there really is a big speed penalty from using bash on BSD compared
to the native shell, wouldn't it be better to supply a Gentoo/FreeBSD
baselayout?  They don't have to be completely independent; smart use of
the vcs would allow you to share scripts between baselayout branches,
with specific variants where it makes a difference.

> 3) What's the cost of *not* changing away from bash?
> I would say that bash is the best shell around in terms of features
> and ease of use, however that is not without cost. That cost is new
> bash versions consistently breaking baselayout, ebuilds and configure
> files.

w.r.t new bash versions, we should certainly be conservative in marking
new versions stable.  It's worth noting the breakage isn't always
100% the fault of bash. The recent problem with '=~' and quoting for
example is down to glibc behaving differently to everyone else's libc
when it comes to accepting quoted characters for the regex interfaces
(a point where the POSIX standard is open to interpretation).

> 4) Size.
> Because bash has all these nice features it's large, hence unsuitable
> for low memory or low disk space environments.

But you only get one bash text image in memory at any one time (~825k).
So space isn't a real issue, except on small-memory systems.

> 5) I'm *just* talking about config files here.
> If users want to run bash, that's fine and I won't stop them. They can
> also use bash in their init script if they so wish as I plan to
> support something like so
> 
> depend() {
>shell bash
> }

Making that sort of requirement explicit is a good idea.  I wouldn't use
'depend()', as in init scripts it's quite cleanly only to do with the
order of services.  You could make it an option to runscript:

#!/bin/runscript --shell=/bin/bash

or something along those lines - the shebang is clearly all about how
the script is executed, and the shell used falls nicely into that.

> And voila, problem solved. Of course, that's just an idea I just had.
> However, I also think that baselayout provided services should not
> require bash for the above reasons, hence the need for a new config.

I think the argument for conf.d files is better than that for init.d
scripts; you could have multiple baselayout setups that share conf.d
file formats.

-- 
Kevin F. Quinn


signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Ned Ludd]

On Tue, 2007-02-06 at 19:42 -0500, Mike Frysinger wrote:
> On Tuesday 06 February 2007, Kevin F. Quinn wrote:
> > You need to define what shell (or subset) you want to parse it.  'sh'
> > itself varies from platform to platform.
> 
> our standard has always ("always" is relative here; let's say "current") been 
> the bash superset of POSIX ... if a request comes up where someone wants to 
> change code because it breaks in their shell but the code in question is 
> POSIX compliant, the answer is simple: blow me
> 
> if the request is reasonable like stop using some bashism in favor of the 
> POSIX form and the change isnt invasive, then sure we'll generally make the 
> change

/me likes the direction Roy is heading with this.

> -mike
-- 
Ned Ludd <[EMAIL PROTECTED]>
Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Auty]

Or perhaps,
	Something a little more explicit might work?  In instances such as the 
ifconfig lines, it's to create eth0 aliases (such as eth0:0), so perhaps 
that could look like:


ifconfig_eth0:0 = "10.1.1.1 netmask 255.255.255.0"
ifconfig_eth0:1 = "10.1.1.2 netmask 255.255.255.0"

For uses that really do require a list (such as modules), something 
like:

modules = "wpa_supplicant", "ifconfig"

or redefinition:

modules = "wpa_supplicant"
modules = "ifconfig"

might work?  If the comma separated syntax is acceptable, it may be that 
brackets around it are acceptable too, in which case the bash syntax is 
fine.
	If the normal shells will still require *some* form of processing to 
deal with the list, could they not be given processing to deal with 
lists that bash already has builtin?

Mike  5:)
--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Tuesday 06 February 2007, Kevin F. Quinn wrote:
> You need to define what shell (or subset) you want to parse it.  'sh'
> itself varies from platform to platform.

our standard has always ("always" is relative here; let's say "current") been 
the bash superset of POSIX ... if a request comes up where someone wants to 
change code because it breaks in their shell but the code in question is 
POSIX compliant, the answer is simple: blow me

if the request is reasonable like stop using some bashism in favor of the 
POSIX form and the change isnt invasive, then sure we'll generally make the 
change
-mike


pgpG0S2atZLMK.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Mike Frysinger]

On Tuesday 06 February 2007, Roy Marples wrote:
> Hopefully I've justified this enough :)

justified what ?  this thread started off killing bash array requirements in 
the network config file and now it looks like your killing the shell 
everywhere ?

to be honest, unless the new code is really good, it's still going to be not 
suitable for real embedded systems ... sure, i run Gentoo on all my embedded 
machines, but lets be honest, these arent even close to real embedded
-mike


pgpIvD4ML3jyD.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Kevin F. Quinn]

On Tue, 6 Feb 2007 19:09:26 +
Roy Marples <[EMAIL PROTECTED]> wrote:

> The stuff that handles our networking maybe written in A.N.
> Other-Language (Mrs.), but keeping /etc/conf.d/net readable by a shell
> script does have advantages.

You need to define what shell (or subset) you want to parse it.  'sh'
itself varies from platform to platform.  The one we have is a softlink
to bash so doesn't make any difference for Gentoo Linux except for
limiting what can be written.  I just tried variable redirection for
example (which can be used to implement pseudo-arrays without using
eval) - I was surprised it works in sh here - dunno if it works in BSD
sh (doesn't on busybox).  What you have on FreeBSD may be different from
what's on Solaris.

Perhaps busybox sh might be a practical set to choose, for basic posix
compliance.


You could simply do something like:

ifconfig_eth0="\
  10.1.1.1 netmask 255.255.255.0;\
  10.1.1.2 netmask 255.255.255.0"

which means standard shell interpretation doesn't lose information, even
if it's actually normally parsed by something else (chose ';' as a
separator since ':' is used in ipv6 strings).


It seems to me that the problem you're trying to solve, is the
implementation of baselayout on restricted systems.  Reducing all
init.d/conf.d and so on to a common denominator for everyone isn't
necessarily the best way forward.  A different approach could be to
provide more than one baselayout; one for large systems, where
expecting to have bash available isn't such a big deal, and one for
limited systems, restricted to busybox-standard sh.  Actually I kinda
assumed that's what baselayout-lite was all about...

-- 
Kevin F. Quinn


signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 21:28:04 +
Ciaran McCreesh <[EMAIL PROTECTED]> wrote:

> I think it's more that you're expected to justify *why* the bash
> requirement is so bad, given the cost of changing.

1) Lack of choice.
Gentoo is all about giving the user choice. baselayout even supports
other init systems when requested.

2) Speed.
Bash is one of the slowest shells around for looping.
However, it also requires less forking due to it's nice built-ins. This
does of course only work with bash and not other shells.

3) What's the cost of *not* changing away from bash?
I would say that bash is the best shell around in terms of features and
ease of use, however that is not without cost. That cost is new bash
versions consistently breaking baselayout, ebuilds and configure files.

4) Size.
Because bash has all these nice features it's large, hence unsuitable
for low memory or low disk space environments.

5) I'm *just* talking about config files here.
If users want to run bash, that's fine and I won't stop them. They can
also use bash in their init script if they so wish as I plan to support
something like so

depend() {
   shell bash
}

And voila, problem solved. Of course, that's just an idea I just had.
However, I also think that baselayout provided services should not
require bash for the above reasons, hence the need for a new config.

Hopefully I've justified this enough :)

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 21:14:49 +
Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
> Why? What's wrong with requiring a shell that supports various
> features beyond what POSIX specifies? Granted, choice of shell is
> good, but not if it's at the expense of functionality or ease of use.

H, just how many features should a config file have beyond the
setting of variables?
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ciaran McCreesh]

On Tue, 6 Feb 2007 21:11:31 + Roy Marples <[EMAIL PROTECTED]>
wrote:
| >Now, this email isn't about the merits of bash, nor the fact that
| >it's in base system profile so we can use it anyway, blah blah blah.
| >embedded has a vested interest in not using bash and I have a
| >personal interest as Gentoo/FreeBSD on Sparc64 takes a very long
| >time to boot.
| 
| I'm assuming you did't read that bit :)

I think it's more that you're expected to justify *why* the bash
requirement is so bad, given the cost of changing.

-- 
Ciaran McCreesh
Mail: ciaranm at ciaranm.org
Web : http://ciaranm.org/
Paludis, the secure package manager : http://paludis.pioto.org/



signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Olivier Crete]

On Tue, 2007-06-02 at 21:11 +, Roy Marples wrote:
> On Tue, 06 Feb 2007 16:03:50 -0500
> Olivier Crete <[EMAIL PROTECTED]> wrote:
> > What so wrong with bash?
> 
> Unsuited to an init system that wants to work everywhere, like embedded
> systems.
> 
> Also, being tied to one shell causes problems when that shell breaks.
> Witness baselayout problems regarding bash-3.0, 3.1 and 3.2
> 
> Lastly, in my first email I said 
> >Now, this email isn't about the merits of bash, nor the fact that it's
> >in base system profile so we can use it anyway, blah blah blah.
> >embedded has a vested interest in not using bash and I have a personal
> >interest as Gentoo/FreeBSD on Sparc64 takes a very long time to boot.
> 
> I'm assuming you did't read that bit :)

Actually I did, but I really dont see what kind of embedded system would
be complex enough to require the Gentoo init system, yet not enough to
use bash.

And I don't see why g/fbsd sparc64 is problematic? Or is it just that
you have a 1995 vintage system and you want to run Gentoo on it?

-- 
Olivier Crête
[EMAIL PROTECTED]
Gentoo Developer

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ciaran McCreesh]

On Tue, 6 Feb 2007 21:04:59 + Roy Marples <[EMAIL PROTECTED]>
wrote:
| On Tue, 6 Feb 2007 20:58:52 +
| Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
| > | Right, and bash arrays are not shell
| > | http://www.opengroup.org/onlinepubs/009695399/toc.htm
| > 
| > Sure they're shell. They're just not POSIX.
| 
| Maybe I should have been more clear.
| 
| Anything in /etc/conf.d/ should be able to be read by a POSIX
| compliant shell. This means no arrays.

Why? What's wrong with requiring a shell that supports various features
beyond what POSIX specifies? Granted, choice of shell is good, but not
if it's at the expense of functionality or ease of use.

-- 
Ciaran McCreesh
Mail: ciaranm at ciaranm.org
Web : http://ciaranm.org/
Paludis, the secure package manager : http://paludis.pioto.org/



signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 06 Feb 2007 16:03:50 -0500
Olivier Crete <[EMAIL PROTECTED]> wrote:
> What so wrong with bash?

Unsuited to an init system that wants to work everywhere, like embedded
systems.

Also, being tied to one shell causes problems when that shell breaks.
Witness baselayout problems regarding bash-3.0, 3.1 and 3.2

Lastly, in my first email I said 
>Now, this email isn't about the merits of bash, nor the fact that it's
>in base system profile so we can use it anyway, blah blah blah.
>embedded has a vested interest in not using bash and I have a personal
>interest as Gentoo/FreeBSD on Sparc64 takes a very long time to boot.

I'm assuming you did't read that bit :)

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 20:58:52 +
Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
> | Right, and bash arrays are not shell
> | http://www.opengroup.org/onlinepubs/009695399/toc.htm
> 
> Sure they're shell. They're just not POSIX.

Maybe I should have been more clear.

Anything in /etc/conf.d/ should be able to be read by a POSIX compliant
shell. This means no arrays.

At this point in time, the only thing that uses arrays
in /etc/conf.d/net - or any other bashisms or non POSIX shell stuff is
out net config.

Hence the need to change it.
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Olivier Crete]

On Tue, 2007-06-02 at 20:34 +, Roy Marples wrote:
> > Keeping it as is has the advantage that an
> > upgrade/downgrade cycle wouldn't change much in functionality based on
> > config, which is pretty good (ie, backwards compatibility).  In this
> > case, I'm not sure legacy is all that bad, simply because it's
> > expressive and concise and easily understood :)
> 
> So you're saying we patch other shells to support bash arrays?

What so wrong with bash? Apart from the fact that its not included in
the unmodified base system of FreeBSD? Last I checked, we add lots of
Gentooish stuff already. And that its not friendly to tiny embedded
systems, which dont use a complex and powerful init system like Gentoo's
(but prefer very simple ash scripts).

-- 
Olivier Crête
[EMAIL PROTECTED]
Gentoo Developer

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ciaran McCreesh]

On Tue, 6 Feb 2007 20:31:03 + Roy Marples <[EMAIL PROTECTED]>
wrote:
| On Tue, 6 Feb 2007 19:27:15 +
| Ciaran McCreesh <[EMAIL PROTECTED]> wrote:
| 
| > On Tue, 6 Feb 2007 19:09:26 + Roy Marples <[EMAIL PROTECTED]>
| > wrote:
| > | The stuff that handles our networking maybe written in A.N.
| > | Other-Language (Mrs.), but keeping /etc/conf.d/net readable by a
| > shell | script does have advantages.
| > 
| > He didn't say "make it not readable by a shell". He was suggesting
| > making it in some format that is readable both by C and the shell --
| > that is to say, a subset of shell.
| 
| Right, and bash arrays are not shell
| http://www.opengroup.org/onlinepubs/009695399/toc.htm

Sure they're shell. They're just not POSIX.

-- 
Ciaran McCreesh
Mail: ciaranm at ciaranm.org
Web : http://ciaranm.org/
Paludis, the secure package manager : http://paludis.pioto.org/



signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 06 Feb 2007 14:18:43 -0500
Seemant Kulleen <[EMAIL PROTECTED]> wrote:

> Forgive me if this is just noise, but I just wanted to say I agree
> fully with ferdy.  As I was reading Roy's email, and I looked at the
> net config sample he had in there, I thought "well, what's actually
> wrong with this?"

Cannot be parsed by shells other than bash or zsh


> Keeping it as is has the advantage that an
> upgrade/downgrade cycle wouldn't change much in functionality based on
> config, which is pretty good (ie, backwards compatibility).  In this
> case, I'm not sure legacy is all that bad, simply because it's
> expressive and concise and easily understood :)

So you're saying we patch other shells to support bash arrays?
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Diego 'Flameeyes' Pettenò]

On Tuesday 06 February 2007, Michael Hanselmann wrote:
> XML! Actually, no. For me, libconfig[1] turned out to be very easy to
> work with. Its config file format is easy to write by hand and the
> parser resides in the library.

On a simialr note there's libconfuse[1], which uses one of the most common 
formats available: ini-like :)

[1] http://www.nongnu.org/confuse/
-- 
Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/
Gentoo/Alt lead, Gentoo/FreeBSD, Video, Sound, ALSA, PAM, KDE, CJK, Ruby ...


pgpYWPdBzIKuX.pgp
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Seemant Kulleen]

Forgive me if this is just noise, but I just wanted to say I agree fully
with ferdy.  As I was reading Roy's email, and I looked at the net
config sample he had in there, I thought "well, what's actually wrong
with this?"  Keeping it as is has the advantage that an
upgrade/downgrade cycle wouldn't change much in functionality based on
config, which is pretty good (ie, backwards compatibility).  In this
case, I'm not sure legacy is all that bad, simply because it's
expressive and concise and easily understood :)

thanks,
-- 
Seemant Kulleen
Developer, Gentoo Linux

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Ciaran McCreesh]

On Tue, 6 Feb 2007 19:09:26 + Roy Marples <[EMAIL PROTECTED]>
wrote:
| The stuff that handles our networking maybe written in A.N.
| Other-Language (Mrs.), but keeping /etc/conf.d/net readable by a shell
| script does have advantages.

He didn't say "make it not readable by a shell". He was suggesting
making it in some format that is readable both by C and the shell --
that is to say, a subset of shell.

-- 
Ciaran McCreesh
Mail: ciaranm at ciaranm.org
Web : http://ciaranm.org/
Paludis, the secure package manager : http://paludis.pioto.org/



signature.asc
Description: PGP signature

Re: [gentoo-dev] Network configuration and bash

[Roy Marples]

On Tue, 6 Feb 2007 19:27:40 +0100
"Fernando J. Pereda" <[EMAIL PROTECTED]> wrote:

> On Tue, Feb 06, 2007 at 06:17:16PM +, Roy Marples wrote:
> > Hi List
> 
> Hi Roy,
> 
> > [snip]
> >
> > So, to free baselayout of forcing bash down our throats I/we am/are
> > looking at re-writing our network setup, including configuration.
> > 
> > Who's got any bright ideas for a new config then? Lets brain storm!
> 
> It is a good format and easy to write and parse in C, why change it ?
> 
> Just tell people to stop treating config files like bash scripts and
> everything will be fine.
> 
> Or am I missing something?

Well, the stuff in /etc/init.d/ and /etc/conf.d will stay as shell
scripts - just not bash specific scripts.

The stuff that handles our networking maybe written in A.N.
Other-Language (Mrs.), but keeping /etc/conf.d/net readable by a shell
script does have advantages.

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Network configuration and bash

[Fernando J. Pereda]

On Tue, Feb 06, 2007 at 06:17:16PM +, Roy Marples wrote:
> Hi List

Hi Roy,

> [snip]
>
> So, to free baselayout of forcing bash down our throats I/we am/are
> looking at re-writing our network setup, including configuration.
> 
> Who's got any bright ideas for a new config then? Lets brain storm!

It is a good format and easy to write and parse in C, why change it ?

Just tell people to stop treating config files like bash scripts and
everything will be fine.

Or am I missing something?

- ferdy

-- 
Fernando J. Pereda Garcimartín
Gentoo Developer (Alpha,net-mail,mutt,git)
20BB BDC3 761A 4781 E6ED  ED0B 0A48 5B0C 60BD 28D4


pgpVCA2ayOize.pgp
Description: PGP signature

[gentoo-dev] Network configuration and bash

[Roy Marples]

Hi List

As some of you may be aware, I've started work on baselayout-2 which is
basically re-tooling it in C. One of the side goals is to eliminate the
need for using bash. You'll be pleased to know that it's working well
enough to boot Gentoo/FreeBSD.

Now, this email isn't about the merits of bash, nor the fact that it's
in base system profile so we can use it anyway, blah blah blah.
embedded has a vested interest in not using bash and I have a personal
interest as Gentoo/FreeBSD on Sparc64 takes a very long time to boot.

This email is about network configuration. Before I joined Gentoo,
network configuration was done in bash arrays like so (note, that the
variable name was changed in baselayout-1.11)

ifconfig_eth0=(
  "10.1.1.1 netmask 255.255.255.0"
  "10.1.1.2 netmask 255.255.255.0"
)

This is all well and good, but only bash and zsh can use it. Infact I'm
guilty as I embraced arrays and used them as the basis for our entire
network scripts. They have also served us well, but as they are also
array based they will have to be re-written too.

So, to free baselayout of forcing bash down our throats I/we am/are
looking at re-writing our network setup, including configuration.

Who's got any bright ideas for a new config then? Lets brain storm!

Thanks

Roy
-- 
gentoo-dev@gentoo.org mailing list