Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-15 Thread Jamie Lokier
Pádraig Brady wrote:
  The $(( ... )) construct is standard POSIX shell syntax, see
  http://www.opengroup.org/onlinepubs/95399/utilities/xcu_chap02.html#tag_02_06_04
  
  Bash supports $[ ... ] as an alternate syntax for the same thing.
  Perhaps you were thinking of that.
 
 I think the misconception that $(( ... )) is a bashism is caused by
 the wrong highlighting defaults chosen by vim.

I think the misconception is because traditional unix bourne shells
don't implement that construct.  I just tried it on a few machines,
and it failed on 4 of them.  Admittedly, the only up to date one is
running Solaris 10; the others are older unixes that you're unlikely
to build Linux on.

-- Jamie
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-15 Thread Måns Rullgård
Jamie Lokier ja...@shareable.org writes:

 Pádraig Brady wrote:
  The $(( ... )) construct is standard POSIX shell syntax, see
  http://www.opengroup.org/onlinepubs/95399/utilities/xcu_chap02.html#tag_02_06_04
  
  Bash supports $[ ... ] as an alternate syntax for the same thing.
  Perhaps you were thinking of that.
 
 I think the misconception that $(( ... )) is a bashism is caused by
 the wrong highlighting defaults chosen by vim.

 I think the misconception is because traditional unix bourne shells
 don't implement that construct.  I just tried it on a few machines,
 and it failed on 4 of them.  Admittedly, the only up to date one is
 running Solaris 10; the others are older unixes that you're unlikely
 to build Linux on.

On Solaris, /usr/xpg4/bin/sh is usually a POSIX-compliant shell.  I
don't know how long it has been there.

-- 
Måns Rullgård
m...@mansr.com
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-15 Thread Rob Landley
On Tuesday 13 January 2009 20:51:16 Jamie Lokier wrote:
 Paul Mundt wrote:
  This happens in a lot of places, like embedded gentoo ports, where almost
  all of the work is sent across distcc to a cross-compilation machine. In
  systems that use package management, it is done on the host through
  emulation, or painfully cross-compiled.

 Ah yes, I remember using embedded Gentoo.

 95% of the time in ./configure scripts, 5% in compilations.

With SMP becoming commonplace, expect this to become the norm everywhere.  
Once you get to around quad processor, any C program with a ./configure step 
is probably going to take longer to configure than to compile.  (Of course C++ 
manages to remain slow enough that autoconf isn't so obvious a bottleneck.)

 And this is on x86!  I dread to think how slow it gets on something
 slow.

My friend Mark's been experimenting with the amazon cloud thing, feeding in 
an image with a qemu instance and distcc+cross-compiler, and running builds 
under that.  Renting an 8-way ~2.5 ghz server with 7 gigabytes of ram and 1.6 
terabytes of disk is 80 cents/hour through them plus another few cents/day for 
bandwidth and persistent storage and such.  That's likely to get cheaper as 
time goes on.

We're still planning to buy a build server of our own to have something in-
house, but for running nightly builds it's almost to the point where 
depreciation on the hardware is more than buying time from a server farm.  
Just _one_ of those 8-way servers is enough hardware to build an entire distro 
in an hour or so.

What this really allows us to do is experiment with how parallel can we get 
our build?  Because renting ten 8-way servers in a cluster is $8/hour, and 
distcc already scales trivially over that.  Down the road what Firmware Linux 
is working towards is multiple qemu instances running in parallel with a 
central instance distributing builds to each one, so each can do its own 
./configure in parallel, distribute compilation to the distccd instances as it 
has stuff to compile, and then package up the resulting binary into one of 
those portage tarballs and send it back to the central node to install on a 
network mount that the lot of 'em can mount as build context, so the packages 
can get their dependencies right.  (You don't want your build taking place in 
a network mount, but your OS being on one you never write to isn't so bad as 
long as you have local storage to build in.)

We'll probably leverage the heck out of Portage for this, and might wind up 
modifying it heavily.  Dunno yet.  (We can even force dependencies on portage 
so it doesn't need to calculate 'em, the central node can do that and then say 
you have these packages, _build_...)

But yeah, hobbyists with a laptop, network access, and a monthly budget of $20 
can do cluster builds these days.

Rob

P.S.  I still hope autoconf dies off and the world wakes up and moves away 
from that.  And from makefiles for that matter.  But in the meantime, I can 
work around it with enough effort.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-13 Thread Jamie Lokier
Paul Mundt wrote:
 This happens in a lot of places, like embedded gentoo ports, where almost
 all of the work is sent across distcc to a cross-compilation machine. In
 systems that use package management, it is done on the host through
 emulation, or painfully cross-compiled.

Ah yes, I remember using embedded Gentoo.

95% of the time in ./configure scripts, 5% in compilations.

And this is on x86!  I dread to think how slow it gets on something
slow.

-- Jamie
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Paul Mundt
On Sun, Jan 11, 2009 at 09:36:58PM -0600, Mark A. Miller wrote:
 Actually, something that has amused me during this discussion, is that
 right now, the latest stable Perl (5.8.8) does not compile correctly
 on a uclibc host, which is typically what you want for embedded
 systems, which is why you'd bother to cross compile. (Albeit I was
 doing this natively under QEMU with a gcc/uclibc toolchain).
 
 I'll have a patch submitted upstream shortly, but basically the
 hints/linux.sh assumes *obviously* you're linking to glibc. I've made
 that less libc dependent, looking for either glibc or uclibc.
 
There are plenty that ship with glibc, too. What you want for embedded
systems depends entirely on the application for the device, not general
hand-wavy assertions. We (Renesas) ship BSPs on both precisely because of
this reason, and eglibc will probably factor in at some point later on
too.

 So without patching Perl, by adding it to the kernel configuration,
 it's broken being able to compile the kernel on most embedded
 architectures.
 
This again has nothing to do with the kernel and everything to do with
your distribution. I use perl on uclibc natively just fine, it is
possible there are patches that have not been merged upstream, but this
is again an entirely separate issue.

You seem to be confusing the fact that people who build distributions and
people who use them are one and the same, whereas most embedded
developers are going to be using pre-built distributions provided with
their reference hardware, and locking it down during productization. The
fact you are doing a distribution aimed at embedded devices is nice, but
do not try to push off problems you run in to that have a reasonable
expectation of working everywhere else on to the kernel community as
something we ought to care about. 

If you need to use a different libc on your platform, yes, you will have
to update packages for this. This used to be true for gcc and other
packages as well, but those were all fixed over time. The fact perl still
stands out despite there being patches available is simply an indicator
that folks working in that area haven't been very proactive in getting
their changes merged upstream. Tough.

This is now entirely off-topic and has nothing to do with the kernel any
more. Please take this to the uclibc or perl lists instead.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Peter Korsgaard
 Mark == Mark A Miller m...@mirell.org writes:

 Mark And for H. Peter Anvin, before you refer to such uses as compiling the
 Mark kernel under a native environment as a piece of art, please be aware
 Mark that the mainstream embedded development environment, buildroot, is
 Mark also attempting to utilize QEMU for a sanity check on the
 Mark environment.

That's for verifying that the rootfs'es actually work, not for
building stuff.

-- 
Bye, Peter Korsgaard
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Mark A. Miller
On Mon, Jan 12, 2009 at 2:20 AM, Paul Mundt let...@linux-sh.org wrote:
 On Sun, Jan 11, 2009 at 09:36:58PM -0600, Mark A. Miller wrote:
 Actually, something that has amused me during this discussion, is that
 right now, the latest stable Perl (5.8.8) does not compile correctly
 on a uclibc host, which is typically what you want for embedded
 systems, which is why you'd bother to cross compile. (Albeit I was
 doing this natively under QEMU with a gcc/uclibc toolchain).

 I'll have a patch submitted upstream shortly, but basically the
 hints/linux.sh assumes *obviously* you're linking to glibc. I've made
 that less libc dependent, looking for either glibc or uclibc.

 There are plenty that ship with glibc, too. What you want for embedded
 systems depends entirely on the application for the device, not general
 hand-wavy assertions. We (Renesas) ship BSPs on both precisely because of
 this reason, and eglibc will probably factor in at some point later on
 too.

 So without patching Perl, by adding it to the kernel configuration,
 it's broken being able to compile the kernel on most embedded
 architectures.

 This again has nothing to do with the kernel and everything to do with
 your distribution. I use perl on uclibc natively just fine, it is
 possible there are patches that have not been merged upstream, but this
 is again an entirely separate issue.

 You seem to be confusing the fact that people who build distributions and
 people who use them are one and the same, whereas most embedded
 developers are going to be using pre-built distributions provided with
 their reference hardware, and locking it down during productization. The
 fact you are doing a distribution aimed at embedded devices is nice, but
 do not try to push off problems you run in to that have a reasonable
 expectation of working everywhere else on to the kernel community as
 something we ought to care about.

 If you need to use a different libc on your platform, yes, you will have
 to update packages for this. This used to be true for gcc and other
 packages as well, but those were all fixed over time. The fact perl still
 stands out despite there being patches available is simply an indicator
 that folks working in that area haven't been very proactive in getting
 their changes merged upstream. Tough.

 This is now entirely off-topic and has nothing to do with the kernel any
 more. Please take this to the uclibc or perl lists instead.


Paul:
I initially wrote a rather details response to your e-mail. But
instead, I shall quote a previous e-mail of yours:

 I will repeat again that no one has provided a
 _single_ reason for why they are unable to provide perl within their
 constrained environment. Until that happens, this entire thread is a
 joke.

And I did so. And you have disregarded it. That makes me question the
logic of your fervent vehemence against such Perl is perhaps not a
good idea in the kernel build infrastructure people like myself.

Thanks.

-- 
Mark A. Miller
m...@mirell.org
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Paul Mundt
On Mon, Jan 12, 2009 at 03:18:53AM -0600, Mark A. Miller wrote:
 On Mon, Jan 12, 2009 at 2:20 AM, Paul Mundt let...@linux-sh.org wrote:
 
 Paul:
 I initially wrote a rather details response to your e-mail. But
 instead, I shall quote a previous e-mail of yours:
 
  I will repeat again that no one has provided a
  _single_ reason for why they are unable to provide perl within their
  constrained environment. Until that happens, this entire thread is a
  joke.
 
 And I did so. And you have disregarded it. That makes me question the
 logic of your fervent vehemence against such Perl is perhaps not a
 good idea in the kernel build infrastructure people like myself.
 
You have done no such thing. You have provided an example as to why you
personally find perl objectionable, and in your previous mail you even
noted that you have patches for perl to fix the build issues, so there is
no fundamental reason why you are _unable_ to provide perl in your
environment. It all comes down to the fact you don't want to be bothered
to put the effort in to getting perl setup in your environment, which
quite frankly is no one's problem but your own.

Personally I think perl (and python for that matter) is a terrible
language and I wouldn't use it for anything of consequence, but again,
that is my personal opinion and has nothing to do with regards to the
build system and whether it was the better tool for the job as perceived
by the people that elected to implemented infrastructure using it. I
choose not to use it for my own projects, but I have no qualms with those
that do.

The kernel does not need to provide justification for every change that
goes on as long as there is a reasonable attempt not to break things for
other people. The onus is (and always has been) on you to demonstrate why
perl is an unreasonable dependency to push on embedded developers, and
you have failed utterly at demonstrating this in any sort of coherent
fashion.

I will repeat, there has not been a single coherent argument against what
makes perl inherently incapable of being supported. Every single thing
you have presented as a rebuttal has been your personal preference, which
in this case is simply irrelevant.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Mark A. Miller
On Mon, Jan 12, 2009 at 3:41 AM, Paul Mundt let...@linux-sh.org wrote:

 I will repeat, there has not been a single coherent argument against what
 makes perl inherently incapable of being supported.

You're right, this thread is worthless with that particular mindset,
Paul. Not, perhaps that the tool in question is brittle, and prone to
potentially break on more architectures than the current make and C
code infrastructure, no, your stance is, unless Perl *cannot* run on
that particular architecture and environment, it has a valid place in
the kernel because it was chosen by certain developers.

And you're right, I did patch around Perl in order to get it to build
under a MIPSEL uclibc environment.

But yes, this particular thread with you *is* worthless, because it's
an argument who's stance is not worth fighting against because of it's
flawed premise.

-- 
Mark A. Miller
m...@mirell.org
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Sam Ravnborg
On Sun, Jan 11, 2009 at 11:50:31PM -0600, Mark A. Miller wrote:
 On Sun, Jan 11, 2009 at 11:35 PM, Sam Ravnborg s...@ravnborg.org wrote:
  There are several other packages which are broken for embedded
  architectures, which I will hopefully attempt to fix by submitting patches
  upstream. But this is why we should be cautious about including new tools
  for compiling the kernel. Sam Ravnborg was correct in that a C program to 
  do
  the work would be the proper way. But by not addressing a currently 
  existing
  problem with an adequate replacement with something that does not exist
  currently, seems faulty.
 
  Why are make headers_install such a crucial thing for your
  embedded environmnet?
 
 Sanity check. If the environment cannot replicate itself, then
 something has been faulty in the cross-compiling stage, that was used
 to propagate a native environment for the target architecture.

So you actually build your target toolchain on your target?

Sam
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Paul Mundt
On Mon, Jan 12, 2009 at 04:03:32AM -0600, Mark A. Miller wrote:
 On Mon, Jan 12, 2009 at 3:41 AM, Paul Mundt let...@linux-sh.org wrote:
  I will repeat, there has not been a single coherent argument against what
  makes perl inherently incapable of being supported.
 
 You're right, this thread is worthless with that particular mindset,
 Paul. Not, perhaps that the tool in question is brittle, and prone to
 potentially break on more architectures than the current make and C
 code infrastructure, no, your stance is, unless Perl *cannot* run on
 that particular architecture and environment, it has a valid place in
 the kernel because it was chosen by certain developers.
 
Nonsense. I singled out that point because that was the one you were
replying to in the first place. I itemized the objections in this thread
earlier on and attempted to indicate why they were not applicable in this
context, and asked people to add to it if anything had been overlooked.
If you want to play semantics, do it somewhere else.

If the tool is brittle and constantly breaking, we will see bug reports,
and re-evaluate the support position. This hasn't happened to date in the
context of the kernel build system, so there is no point in even bringing
this up.

Anyways, given that you haven't contributed anything to the kernel and
are therefore perhaps unfamiliar with how things work, I attempted to
show you why the kernel made the decision it did and what it would take
to change that. You have from the beginning only wanted to focus on idle
semantics and refused to re-evaluate your own position on what precisely
it is you find to be problematic in the first place.

So, with that, I am done with this thread, and it seems the key takeaways
from this entire thing has only been a few new lines in my killfile.
It's regrettable you didn't get anything else out of this thread, though
I think both the kernel and embedded linux will survive.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Paul Mundt
On Mon, Jan 12, 2009 at 11:18:03AM +0100, Sam Ravnborg wrote:
 On Sun, Jan 11, 2009 at 11:50:31PM -0600, Mark A. Miller wrote:
  On Sun, Jan 11, 2009 at 11:35 PM, Sam Ravnborg s...@ravnborg.org wrote:
   There are several other packages which are broken for embedded
   architectures, which I will hopefully attempt to fix by submitting 
   patches
   upstream. But this is why we should be cautious about including new tools
   for compiling the kernel. Sam Ravnborg was correct in that a C program 
   to do
   the work would be the proper way. But by not addressing a currently 
   existing
   problem with an adequate replacement with something that does not exist
   currently, seems faulty.
  
   Why are make headers_install such a crucial thing for your
   embedded environmnet?
  
  Sanity check. If the environment cannot replicate itself, then
  something has been faulty in the cross-compiling stage, that was used
  to propagate a native environment for the target architecture.
 
 So you actually build your target toolchain on your target?
 
This happens in a lot of places, like embedded gentoo ports, where almost
all of the work is sent across distcc to a cross-compilation machine. In
systems that use package management, it is done on the host through
emulation, or painfully cross-compiled.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Alexander Neundorf
On Monday 12 January 2009 11:22:47 you wrote:
...
 entire environment, QEMU allows it nicely with distcc at a reasonable
 speed. (Albeit there is no distconfigure, but that's entirely an
 unrelated tanget of muck and despair and rants against configure, but
 we're not going there...)

I'd be interested in hearing your issues with configure for cross compiling 
right ? 
I added cross compiling support to cmake, so I'm interested to see whether we 
did it better :-)

Alex
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-12 Thread Rob Landley
On Monday 12 January 2009 02:27:32 Peter Korsgaard wrote:
  Mark == Mark A Miller m...@mirell.org writes:

  Mark And for H. Peter Anvin, before you refer to such uses as compiling
 the Mark kernel under a native environment as a piece of art, please be
 aware Mark that the mainstream embedded development environment,
 buildroot, is Mark also attempting to utilize QEMU for a sanity check on
 the Mark environment.

 That's for verifying that the rootfs'es actually work, not for
 building stuff.

Not in my case.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread Bernd Petrovitsch
On Son, 2009-01-04 at 11:23 +0100, Leon Woestenberg wrote:
[...]
 On Sun, Jan 4, 2009 at 4:06 AM, Paul Mundt let...@linux-sh.org wrote:
[...]

I'm ignoring the cross-compile perl issue - haven't tried it for
years.

 5. Tool *version* dependency is hard to get right. When cross-building
 30 software packages all requiring native perl, we probably need to
 build a few versions of perl (native), one for each set of packages.

perl is IMHO special (and quite different to others - including
especially autotools): perl5 is used widely enough so that one somewhat
recent version should cover all of 30 software packages.
The hard part are the CPAN modules and their versions which are really a
PITA.
As long as you don't use modules from CPAN, perl5 should be specific
enough.

Bernd
-- 
Firmix Software GmbH   http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
  Embedded Linux Development and Services

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread Mark A. Miller
On Sun, Jan 11, 2009 at 6:45 AM, Bernd Petrovitsch be...@firmix.at wrote:

 On Son, 2009-01-04 at 11:23 +0100, Leon Woestenberg wrote:
 [...]
  On Sun, Jan 4, 2009 at 4:06 AM, Paul Mundt let...@linux-sh.org wrote:
 [...]

 I'm ignoring the cross-compile perl issue - haven't tried it for
 years.

  5. Tool *version* dependency is hard to get right. When cross-building
  30 software packages all requiring native perl, we probably need to
  build a few versions of perl (native), one for each set of packages.

 perl is IMHO special (and quite different to others - including
 especially autotools): perl5 is used widely enough so that one somewhat
 recent version should cover all of 30 software packages.
 The hard part are the CPAN modules and their versions which are really a
 PITA.
 As long as you don't use modules from CPAN, perl5 should be specific
 enough.

Bernd
 --
 Firmix Software GmbH   http://www.firmix.at/
 mobil: +43 664 4416156 fax: +43 1 7890849-55
  Embedded Linux Development and Services

Actually, something that has amused me during this discussion, is that
right now, the latest stable Perl (5.8.8) does not compile correctly
on a uclibc host, which is typically what you want for embedded
systems, which is why you'd bother to cross compile. (Albeit I was
doing this natively under QEMU with a gcc/uclibc toolchain).

I'll have a patch submitted upstream shortly, but basically the
hints/linux.sh assumes *obviously* you're linking to glibc. I've made
that less libc dependent, looking for either glibc or uclibc.

So without patching Perl, by adding it to the kernel configuration,
it's broken being able to compile the kernel on most embedded
architectures.

And for H. Peter Anvin, before you refer to such uses as compiling the
kernel under a native environment as a piece of art, please be aware
that the mainstream embedded development environment, buildroot, is
also attempting to utilize QEMU for a sanity check on the
environment.

There are several other packages which are broken for embedded
architectures, which I will hopefully attempt to fix by submitting
patches upstream. But this is why we should be cautious about
including new tools for compiling the kernel. Sam Ravnborg was correct
in that a C program to do the work would be the proper way. But by not
addressing a currently existing problem with an adequate replacement
with something that does not exist currently, seems faulty.
--
Mark A. Miller
m...@mirell.org
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread H. Peter Anvin
Mark A. Miller wrote:
 
 Actually, something that has amused me during this discussion, is that
 right now, the latest stable Perl (5.8.8) does not compile correctly
 on a uclibc host...


The latest stable Perl is 5.10.0, and the latest of the 5.8 series is 5.8.9.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread Mark A. Miller
On Sun, Jan 11, 2009 at 11:11 PM, H. Peter Anvin h...@zytor.com wrote:
 Mark A. Miller wrote:

 Actually, something that has amused me during this discussion, is that
 right now, the latest stable Perl (5.8.8) does not compile correctly
 on a uclibc host...


 The latest stable Perl is 5.10.0, and the latest of the 5.8 series is 5.8.9.

-hpa

 --
 H. Peter Anvin, Intel Open Source Technology Center
 I work for Intel.  I don't speak on their behalf.


My mistake. However,
http://perl5.git.perl.org/perl.git/blob_plain/HEAD:/hints/linux.sh
still has the issue, specifically:

if test -L /lib/libc.so.6; then
libc=`ls -l /lib/libc.so.6 | awk '{print $NF}'`
libc=/lib/$libc
fi

So, my version was incorrect, yet the problem still exists. I've got a
patch, need to submit it, yet Perl *does not compile* on a uclibc
target *as is*.

And this is why we should avoid adding new tools to build the kernel,
because they introduce yet more break points, as such.

Thanks.

-- 
Mark A. Miller
m...@mirell.org

My greatest strength, I guess it would be my humility. My greatest
weakness, it's possible that I'm a little too awesome - Barack Obama
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread Sam Ravnborg
 There are several other packages which are broken for embedded
 architectures, which I will hopefully attempt to fix by submitting patches
 upstream. But this is why we should be cautious about including new tools
 for compiling the kernel. Sam Ravnborg was correct in that a C program to do
 the work would be the proper way. But by not addressing a currently existing
 problem with an adequate replacement with something that does not exist
 currently, seems faulty.

Why are make headers_install such a crucial thing for your
embedded environmnet?

I would assume that if this of such improtance then there is also
someone to step up and contribute a C version of it.

Sam
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-11 Thread Mark A. Miller
On Sun, Jan 11, 2009 at 11:35 PM, Sam Ravnborg s...@ravnborg.org wrote:
 There are several other packages which are broken for embedded
 architectures, which I will hopefully attempt to fix by submitting patches
 upstream. But this is why we should be cautious about including new tools
 for compiling the kernel. Sam Ravnborg was correct in that a C program to do
 the work would be the proper way. But by not addressing a currently existing
 problem with an adequate replacement with something that does not exist
 currently, seems faulty.

 Why are make headers_install such a crucial thing for your
 embedded environmnet?

Sanity check. If the environment cannot replicate itself, then
something has been faulty in the cross-compiling stage, that was used
to propagate a native environment for the target architecture.

 I would assume that if this of such improtance then there is also
 someone to step up and contribute a C version of it.

You've already dismissed a shell version to correct the issue, in
hopes of a possible C version. It would be nice, I'm not capable of
doing it personally, but a solution already exists to unbreak the
kernel build. If you're unwilling to merge the current patches, then
feel free to claim that this doesn't break anything on current
architectures, but it's incorrect, due to Perl not even compiling as
is currently on a native uclibc environment.

I look forward to what other tools will be introduced to break yet
more architectures until the kernel cannot be compiled unless on an
i686+glibc architecture.

Sam

-- 
Mark A. Miller
m...@mirell.org
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-08 Thread klaasjan gm
Rob and to whom it may concern,

 I didn't discover this topic independently.  Somebody pinged me about it on
 freenode back in February, and several other people sent me private email
 about it, and it's been previously raised on several other mailing lists (such
 as busybox and uclibc ones).

 Unfortunately, most of the embedded developers I know aren't subscribed to
 linux-kernel.

FWIW,

 Hopefully the cc: to linux-embedded is helping get more embedded guys involved
 in the discussion than just me. :)

Having some experience with cross-compiling the kernel, and some of the basic
toolsets:
I'm with you on this one (and testing the waters at linux-embedded
while I'm at it)

Regards,
Klaasjan
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-08 Thread Mike Frysinger
On Sun, Jan 4, 2009 at 05:23, Leon Woestenberg wrote:
 On Sun, Jan 4, 2009 at 4:06 AM, Paul Mundt let...@linux-sh.org wrote:
 Let's look at the rationale presented so far in this thread:

2 - Cross-compiling perl is hard.

 2 is not hard.

i dont know where you're getting this mythical version of perl, but
anything beyond micro-perl is a pita.  and micro-perl is practically
worthless.  maybe for kernel/
-mike
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-04 Thread Sam Ravnborg
On Sat, Jan 03, 2009 at 07:45:34PM -0600, Rob Landley wrote:
  With respect to your three patches the plan is to:
  - add the updated timeconst patch to kbuild-next
  - add the updated cpu-feature patch to kbuild-next
 
  - the patch touching headers_install will not be merged.
The way forward for headers_install is to combine the
unifdef feature and the header modifications.
 
 Since you're turning down an existing patch in favor of a theoretical patch, 
 I 
 assume you have plans to do this yourself?

If noone else beats me I will do so - yes.
 
And this must be in a single program that can process
all headers in one go so the install process becomes so fast
that we do not worry about if it was done before or not.
Then we can avoid all the .* files in the directory
where we isntall the headers.
 
 What if they run out of disk space halfway through writing a file and thus it 
 creates a short file (or a 0 length file where the dentry was created but no 
 blocks could be allocated for the write)?

Then they fail and make will know. Then may leave a file or 100
but it still failed. At next run everything will be done right
assuming the culprint has been fixed.

 I can try to make the shell version more readable, and more powerful.  It's 
 already noticeably faster than the perl version.  I have no objections to 
 making unifdef do all of this, I just haven't got any interest either.

I have no interest in merging a shell version.
I clearly expressed above that we need a _single_ program doing
all of the preparations and we do not need a reimplmentatio of the
current headers_install.
I also explained why.

Sam
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-04 Thread Leon Woestenberg
Hello,

On Sun, Jan 4, 2009 at 4:06 AM, Paul Mundt let...@linux-sh.org wrote:
 Let's look at the rationale presented so far in this thread:

1 - Being able to build the kernel natively on a constrained
target is useful, regardless of whether it is being used for
regression/stress testing or for headers installation or whatever
else.

2 - Cross-compiling perl is hard.

3 - Some oddly constrained target distributions manage to ship
with an entire toolchain yet fail to provide any implementation
of perl.

4 - It wasn't required before.

 If there is anything I missed, feel free to add it to the list. It was
 difficult to extract even those 4 from the ranting.


2 is not hard.

5. Tool *version* dependency is hard to get right. When cross-building
30 software packages all requiring native perl, we probably need to
build a few versions of perl (native), one for each set of packages.



Regards,
-- 
Leon
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Wolfgang Denk
Dear Paul Mundt,

In message 20090102095023.ga28...@linux-sh.org you wrote:

 Your main reasons against inclusion of perl seem to be that there is no
 realistic expectation for target systems that will be self-hosting will
 have perl included, or the inherent complexity in maintaining a coherent
 cross compiling environment. Both of these things are issues with your
 own environment, and in no way are these representative of the embedded
 development community at large.

I'm  not  sure  how  representative  for  the  embedded  development
community at large your statement is.

Just to add a data point to the statistice, I do agree with Rob's
opinion about needing Per to build the Linux kernel.

All other arguments aside, natively compiling the Linux kernel  on  a
target  board, especially with root file system mounted over NFS, has
always been an excellent stress test (you may even call it regression
test) for Linux running on some target hardware.

Losing this ability just because  some  scripts  are  implemented  in
language  FOO when plain shell will do as well is something we should
try to avoid.

Of course, YMMV.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH, MD: Wolfgang Denk  Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de
Include the success of others in your dreams for your own success.
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Friday 02 January 2009 10:04:08 Matthieu CASTET wrote:
 Rob Landley a écrit :
  On Friday 02 January 2009 03:26:37 Arkadiusz Miskiewicz wrote:
  On Friday 02 of January 2009, Rob Landley wrote:
 
  Heh,
 
  I believe all three scripts run under dash and busybox ash.  (The
  timeconst.sh one needs 64 bit math which dash only provides on 64 bit
  hosts, which is a regression from Red Hat 9 in 2003 by the way.

 With dash 0.5.4-12 (from debian sid), I seems I got the 64 bit math for
 32 bit hosts :
 $ uname -m
 i686
 $ dash -c 'echo $((132))'
 4294967296

Cool.

The relatively recent 32 bit image I have lying around for testing purposes 
is xubuntu 7.10, and when dash was first introduced into ubuntu it had 
_buckets_ of bugs.  (If you backgrounded a task with  and then hit ctrl-z on 
the command line, it would suspend the backgrounded task.  It was Not Ready 
for Prime Time in a big way.)  Lack of 64 bit math could easily be one more.  
(It _is_ a regression vs Red Hat 9.)

The dash in ubuntu 8.10 seems to have a lot of the more obvious problems 
worked out.  Good to know. :)

That said, while testing the new round of patches against various shells and 
making it reproduce the full set of time constants that the old perl script 
kept cached values for (24, 32, 48, 64, 100, 122, 128, 200, 250, 256, 300, 
512, 1000, 1024, and 1200 hz), I found a bug.

The first patch is  miscalculating USEC_TO_HZ_ADJ32 for 24 HZ and 122 HZ.  All 
the other values are fine.)  It's an integer overflow.  The GCD of 24 and 
100 is 8, so that's 17 significant bits with a shift of 47... which is 
exactly 64 bits, but the math is _signed_, so it goes boing.

For the record, the reason I can't just pregenerate all these suckers on a 
system that's got an arbitrary precision calculator (ala dc) and then just 
ship the resulting header files (more or less the what the first version of 
that first patch did) is that some architectures (arm omap and and arm at91) 
allow you to enter arbitrary HZ values in kconfig.  (Their help text says that 
in many cases values that aren't powers of two won't work, but nothing 
enforces this.)  So if we didn't have the capability to dynamically generate 
these, you could enter a .config value that would break the build.

I'd be willing to use dc in the script if A) it was mentioned in SUSv3 (it's 
not, although bc is), B) the version of dc in busybox wasn't crap (it's 
floating point rather than arbitrary precision, and doesn't implement the left 
shift operator).  The reason I'm not looking more closely at what SUSv3 has to 
say about bc is that A) it claims to be an entire programming language, which 
is definitely overkill for this B) busybox hasn't bothered to implement it so 
it can't be all that widely used anymore.

I'll fix this and resubmit, it just wasn't ready last night.  (If the merge 
window is closing soon I could resubmit the other two with Sam's suggestions 
and resubmit this one next time 'round, but it was only a couple days to write 
in the first place, once I finally figured out what the perl version was 
trying to _do_...)

I believe ADJ32 is the only operation with any potential to overflow.  The 
pathological case for SHIFT is HZ 1, which for USEC conversions would give a 
shift around 52 (32 significant bits plus 20 bits to divide by 100), but 
MUL32 still wouldn't overflow because the shift loop stops when it finds 32 
significant bits, and any larger HZ value would produce a smaller shift.  The 
problem with ADJ32 is it uses the MUL32 shift value, so a small $TO (24 HZ) 
with a big $FROM (100 USEC, 19 signficant bits) and a small Greatest 
Common Denominator (in this case 8) can overflow 64 bits.  Pathological case 
is still HZ 1.  (Or any other smallish prime number.)  If I make that work, 
everything else has to.

So anyway, it's not _arbitrary_ precision math.  It's more like 32+20+20=72 
bits, and I can probably fake that pretty easily by breaking a couple of 
operations into two stages...

Fallen a bit behind on the thread since I noticed this and went off to code, 
I'll try to catch up later today.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Friday 02 January 2009 12:01:34 Sam Ravnborg wrote:
 But the serie rased anohter topic: shall we ban use of perl
 for generating a kernel.

I dunno about ban, but every time somebody adds perl to the hot path of 
the kernel build it breaks my build system, and I write a removal patch 
anyway.  I have to maintain them anyway, so I might as well try to push 'em 
upstream.  (I posted the first patch in this series twice before, once for 25 
and then an updated version to the linux-embedded list for .26.)

I didn't discover this topic independently.  Somebody pinged me about it on 
freenode back in February, and several other people sent me private email 
about it, and it's been previously raised on several other mailing lists (such 
as busybox and uclibc ones).

Unfortunately, most of the embedded developers I know aren't subscribed to 
linux-kernel.  (You either do kernel development, or you do everything else.  
It's not really feasible to keep up with the guts of the kernel and uClibc and 
busybox and gcc and qemu and the current offerings of three different hardware 
vendors and whatever userspace application the board's supposed to run and 
your build system and what INSANE things your EEdiot electrical engineer 
decided to miswire this time and fighting off marketing's desire to switch 
everything over to WinCE because they can get their entire advertising budget 
subsidized and there's a trade show next week we're not READY for...  Not only 
can none of 'em read a meaningful subset of linux-kernel anymore, but if you 
disappear into your own little niche for nine months, when you pop back up the 
kernel's all different and sometimes even the patch submission policy's 
migrated a bit.  Heck, I'm three months behind reading the LWN kernel page 
myself, and that's no substitute for kernel-traffic, RIP...)

Hopefully the cc: to linux-embedded is helping get more embedded guys involved 
in the discussion than just me. :)

 And this is what primary is discussed and the outcome of
 that discussion will not prevent patches that stands on their
 own to be applied.

The best way to get a patch applied is always for that patch to stand on its 
own merits.  Larger agendas are secondary.

Whether or not the kernel decides on a policy of keeping perl out of the 
kernel build's hot path, I still need these patches for my own use, and plan 
to keep coming up with them and submitting them.  I haven't removed ones that 
haven't broken my build yet, but just because I'm not using md today doesn't 
mean I won't _start_.  (And if enough other people keep poking me about the 
kernel build I can tackle 'em to please them.  I actually _do_ know some 
embedded developers using raid for network attached storage and video servers 
and such...)

   Sam

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread H. Peter Anvin
Sam Ravnborg wrote:
 With respect to your three patches the plan is to:
 - add the updated timeconst patch to kbuild-next

If you add this, you take the responsibility for the breakages that will
occur.  The reason his patch is simpler is because he removes the
arbitrary-precision arithmetic, and simply hopes that the system
utilities that he uses uses an integer size which happens to be big enough.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Leon Woestenberg
Hello all,

On Fri, Jan 2, 2009 at 9:07 AM, Rob Landley r...@landley.net wrote:
 Before 2.6.25 (specifically git bdc807871d58285737d50dc6163d0feb72cb0dc2 )
 building a Linux kernel never required perl to be installed on the build
 system.  (Various development and debugging scripts were written in perl and
 python and such, but they weren't involved in actually building a kernel.)
 Building a kernel before 2.6.25 could be done with a minimal system built from
 gcc, binutils, bash, make, busybox, uClibc, and the Linux kernel, and nothing
 else.  (Embedded developers creating clean cross compile environments that


I agree with Rob that the amount of required dependencies should be
kept to a minimum.

If we only use 0.5% of a certain language (or: dependent package),
then rather implement
that 0.5% in the existing language.

Dependencies very quickly become dependency hell. If A requires B,
then A also inherits all
(future) requirements of B, etc. etc.

In my daily software development work with Linux and GNU software in
general, 10% of it is spent fighting/removing these extremely thin
or false depencies, so that it is usuable in embedded devices.


Regards,
-- 
Leon
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Robert Hancock

Rob Landley wrote:
For the record, the reason I can't just pregenerate all these suckers on a 
system that's got an arbitrary precision calculator (ala dc) and then just 
ship the resulting header files (more or less the what the first version of 
that first patch did) is that some architectures (arm omap and and arm at91) 
allow you to enter arbitrary HZ values in kconfig.  (Their help text says that 
in many cases values that aren't powers of two won't work, but nothing 
enforces this.)  So if we didn't have the capability to dynamically generate 
these, you could enter a .config value that would break the build.


Is there a good reason that these archs allow you enter arbitrary HZ 
values? The use case for using custom HZ values at all nowadays seems 
fairly low now that dynticks is around (if that arch supports it 
anyway), let alone being able to specify wierd obscure values for it. 
Especially if nothing can ensure that all values it allows will actually 
result in a functional kernel..

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Friday 02 January 2009 10:04:08 Matthieu CASTET wrote:
 Rob Landley a écrit :
  On Friday 02 January 2009 03:26:37 Arkadiusz Miskiewicz wrote:
  On Friday 02 of January 2009, Rob Landley wrote:
 
  Heh,
 
  I believe all three scripts run under dash and busybox ash.  (The
  timeconst.sh one needs 64 bit math which dash only provides on 64 bit
  hosts, which is a regression from Red Hat 9 in 2003 by the way.

 With dash 0.5.4-12 (from debian sid), I seems I got the 64 bit math for
 32 bit hosts :
 $ uname -m
 i686
 $ dash -c 'echo $((132))'
 4294967296


 Matthieu

Alas, my attempt to install a 32 bit version of xubuntu 8.10 under qemu hung 
at Scanning files: 15%, and has been there for an hour now.  I'll have to 
take your word for it.  (All three scripts work fine under 64 bit dash.)

I encountered one bug in busybox, which I pinged that list about, but 
otherwise busybox ash works on 'em all too.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Friday 02 January 2009 13:27:45 H. Peter Anvin wrote:
 Sam Ravnborg wrote:
  Hi Wookey.
 
  Given the
  simplicitly of these patches I can't see any reason not to put them
  in
 
  Please do NOT do the mistake and think this the same thing.
 
  Rob's patch simplyfy the timecost stuff - and will be applied on
  this merit alone assuming comments will be addressed.
 
  But the serie rased anohter topic: shall we ban use of perl
  for generating a kernel.
  And this is what primary is discussed and the outcome of
  that discussion will not prevent patches that stands on their
  own to be applied.

 My personal opinion on this is that this is ridiculous.  Given that you
 need gcc, binutils, make etc. to build the kernel,

I believe Intel's icc builds the kernel, and tinycc previously built a subset 
of the kernel.  The pcc and llvm/clang projects are getting close to being 
able to build the kernel.  Ever since c99 came out, lots of gcc-isms with c99 
equivalents have been swapped over, most of the rest is testing.

 and this is more than
 inherent, you have to have a pretty bloody strangely constrained system
 to disallow Perl, which is as close to a standard Unix utility you can
 get without making it into SuS.

Please show me A) the standard perl implements, B) the second implementation 
of that standard ala IETF guidelines.

 The only *real* motivation I have seen for this is a system that as far
 I can tell is nothing other than a toy, specifically designed to show
 off how little you need to build the kernel.  In other words, it's not a
 practical application, it's a show-off art piece.

I'm glad you think my Firmware Linux project is a work of art, but if you'd 
like to hear directly from my users I can ask them to complain at you in 
person, if you like.  I'm not sure what that would prove, though.

When cross compiling, it's good to have as constrained an environment as 
possible, because otherwise bits of the host system leak into the target 
system.  If you don't tightly control your cross compiling environment, it 
won't work.  That's just about an axiom in embedded development.

I know every single dependency my system has.  I can list them, explicitly.  I 
did this because it's very _USEFUL_ in this context.

Add perl scripts that call cpan, and this is no longer true.

   -hpa

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread David Brownell
On Saturday 03 January 2009, Robert Hancock wrote:
 Rob Landley wrote:
  ... some architectures (arm omap and and arm at91) 
  allow you to enter arbitrary HZ values in kconfig.  (Their help text says 
  that 
  in many cases values that aren't powers of two won't work, but nothing 
  enforces this.)
 
 Is there a good reason that these archs allow you enter arbitrary HZ 
 values?

Power-of-two can be desirable when using a 32 KiHz oscillator, because
other values accumulate rounding errors ... you can't make 100 Hz, or
250 Hz, or 300 Hz, or 1000 Hz, by a binary division of 32 KiHz.

Other values were supported to help work around stupid software making
bad assumptions about HZ.  IMO, enforcing power-of-two would be better;
that software breaks with dyntick anyway, and needs fixing.


 The use case for using custom HZ values at all nowadays seems  
 fairly low now that dynticks is around (if that arch supports it 
 anyway),

A better argument would be that GENERIC_TIME exists (and works
on OMAP and AT91), which avoids some flavors of rounding error.
ISTR those CONFIG_HZ options predate GENERIC_TIME support.

However, the issue remains that most kernel times are measured in
jiffies not ktime_t -- they're easier and more efficient, all
those 64-bit multiplies can hurt on ARM (32-bit, non-GHz) -- so
it's still good to be able to ensure that jiffies-centric logic
won't always be inserting easily avoidable errors.

- Dave
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Saturday 03 January 2009 17:03:11 H. Peter Anvin wrote:
 Leon Woestenberg wrote:
  I agree with Rob that the amount of required dependencies should be
  kept to a minimum.
 
  If we only use 0.5% of a certain language (or: dependent package),
  then rather implement that 0.5% in the existing language.
 
  Dependencies very quickly become dependency hell. If A requires B,
  then A also inherits all (future) requirements of B, etc. etc.
 
  In my daily software development work with Linux and GNU software in
  general, 10% of it is spent fighting/removing these extremely thin
  or false depencies, so that it is usuable in embedded devices.

 First of all, I largely consider this a joke.

Yes, I've noticed this.  The fact multiple other people do _not_ consider this 
a joke doesn't seem to have sunk in yet.

 All real-life embedded
 kernel builds take place on hosted platforms; anything else seems to be
 done just because it can be done, as a kind of show-off art project.
 Cute, but hardly worth impeding the rest of the kernel community for.

 We're not talking about general platform dependencies here, but build
 dependencies for the kernel.  A platform that can build the kernel is
 not a small platform.

The kernel didn't need perl to build until 2.6.25.  For 17 years, this 
dependency was not required.  You added it, in a way that affected even 
allnoconfig, for no obvious gain.

 Second of all, these patches are not fullworthy replacements.  The
 original patch using bc had less dependencies, but bc failed on some
 platforms, mysteriously.

So rather than debugging it, you rewrote it in perl.  Much less potential 
mysterious behavior there.

 The new patches have *more* environmental
 dependencies than that ever did.

Could you please be a little more specific?

 Third, if someone actually cares to do it right, I have a smallish
 bignum library at http://git.zytor.com/?p=lib/pbn.git;a=summary that
 might be a starting point.

This doesn't _need_ bignum support.  It maxes out around 72 bits and the 
_result_ can't use more than about $SHIFT bits because you're dividing by the 
amount you shifted, so just chop off the bottom 32 bits, do a normal 64 bit 
division on the top (it has to fit), and then do the same division on the 
appropriate shifted remainder, and combine the results.  This is easy because 
when the shift _is_ 32 bits or more, the bottom 32 bits all have to be zeroes 
so you don't even have to mask and add, just shift the remainder left 32 bits 
so you can continue the divide.

Pulling out perl isn't always a good alternative to thinking about the 
problem.

   -hpa

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread H. Peter Anvin
Rob Landley wrote:
 
 The new patches have *more* environmental
 dependencies than that ever did.
 
 Could you please be a little more specific?
 

In this case, you're assuming that every version of every shell this is
going to get involved with is going to do math correctly with the
requisite precision, which is nowhere guaranteed, I'm pretty sure.

 Third, if someone actually cares to do it right, I have a smallish
 bignum library at http://git.zytor.com/?p=lib/pbn.git;a=summary that
 might be a starting point.
 
 This doesn't _need_ bignum support.  It maxes out around 72 bits and the 
 _result_ can't use more than about $SHIFT bits because you're dividing by the 
 amount you shifted, so just chop off the bottom 32 bits, do a normal 64 bit 
 division on the top (it has to fit), and then do the same division on the 
 appropriate shifted remainder, and combine the results.  This is easy because 
 when the shift _is_ 32 bits or more, the bottom 32 bits all have to be zeroes 
 so you don't even have to mask and add, just shift the remainder left 32 bits 
 so you can continue the divide.
 
 Pulling out perl isn't always a good alternative to thinking about the 
 problem.
 

Neither is open-coding a bignum operation instead of relying on an
existing, validated implementation.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Friday 02 January 2009 08:04:09 Theodore Tso wrote:
 Sounds like though modulo dealing with 64-bit arithmetic, your patches
 are mostly dash/POSIX.2 comformant, so you're probably mostly good on
 that front once you address the 32/64-bit issues.  I'd also suggest
 explicitly add a reminder to the shell scripts' comments to avoid
 bashisms for maximum portability, to remind developers in the future
 who might try to change the shell scripts to watch out for portability
 issues.

I changed the scripts to start with #!/bin/sh and retested under dash.

If scripts say #!/bin/sh when they actually need bash, or say #!/bin/bash when 
they work with dash, that should probably be fixed.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Jamie Lokier
Rob Landley wrote:
 This doesn't _need_ bignum support.  It maxes out around 72 bits and
 the _result_ can't use more than about $SHIFT bits because you're
 dividing by the amount you shifted, so just chop off the bottom 32
 bits, do a normal 64 bit division on the top (it has to fit), and
 then do the same division on the appropriate shifted remainder, and
 combine the results.  This is easy because when the shift _is_ 32
 bits or more, the bottom 32 bits all have to be zeroes so you don't
 even have to mask and add, just shift the remainder left 32 bits so
 you can continue the divide.
 
 Pulling out perl isn't always a good alternative to thinking about
 the problem.

Related query:

Does the Perl script being replaced use 64-bit arithmetic?  Because
many Perl installations only do 32-bit arithmetic.

If the Perl version works in 32-bit arithmetic, why does the shell
version not do the same thing?

-- Jamie
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread H. Peter Anvin
Jamie Lokier wrote:
 
 Related query:
 
 Does the Perl script being replaced use 64-bit arithmetic?  Because
 many Perl installations only do 32-bit arithmetic.
 
 If the Perl version works in 32-bit arithmetic, why does the shell
 version not do the same thing?
 

The Perl version uses Math::BigInt, a Perl standard module (with a
canned-values fallback for ancient or minimal Perl installations) to do
arbitrary precision arithmetic.

The original version also produced constants that could be used with
64-bit values, but since gcc doesn't support 128-bit arithmetic on
32-bit platforms (gcc *does* support 128-bit arithmetic on 64-bit
platforms) we didn't end up using it and removed them, although the code
to generate them can still be activated.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Markus Heidelberg
Rob Landley, 04.01.2009:
 On Saturday 03 January 2009 18:37:12 Leon Woestenberg wrote:
  My argument on thin dependencies indeed mostly holds for run-time
  dependencies (to reduce size) but also for build dependency (to reduce
  complexity)*.
 
 I usually just point to the gnucash 1.6 release as where this sort of thing 
 leads if you ignore it long enough:
 http://lwn.net/2001/0614/
 
 These days, a more modern example is the way that after even the gentoo folks 
 gave up on trying to build openoffice (and shipped prebuilt binaries of it in 
 their build everything from source code OS), Open Office's own developers 
 described that project profoundly sick and stagnating ( 
 http://developers.slashdot.org/article.pl?sid=08/12/28/0124230 ).

Now that you mention this the second time, I have to ask where you have
this information from. Since I use Gentoo, I was always able to compile
OpenOffice (version 1, 2 and now 3) myself.

At the same time, it was always possible to use prebuilt packages as an
alternative - the same way as it is possible for a few other packages
(Firefox, Thunderbird, Seamonkey, maybe more). But AFAIK compiling from
source is still the preferred method.

Markus

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Saturday 03 January 2009 21:38:13 Markus Heidelberg wrote:
 Rob Landley, 04.01.2009:
 Now that you mention this the second time, I have to ask where you have
 this information from. Since I use Gentoo, I was always able to compile
 OpenOffice (version 1, 2 and now 3) myself.

The gentoo panel last OLS.  (Either a BOF or a tutorial, I don't remember 
which.)  It's still _possible_ to build it from source, but they created a 
separate openoffice-bin package for the sole purpose of _not_ compiling it 
from source, and it's what they recommend installing.

 At the same time, it was always possible to use prebuilt packages as an
 alternative - the same way as it is possible for a few other packages
 (Firefox, Thunderbird, Seamonkey, maybe more). But AFAIK compiling from
 source is still the preferred method.

Apparently not for Open Office.

First hit googling gentoo openoffice install:
http://grokdoc.net/index.php/Gentoo-OpenOffice.org

The next two hits are bug tracker entries, and the one after that:
http://www.linuxforums.org/forum/gentoo-linux-help/71086-installing-
openoffice-question.html

Contains this cut and paste from emerge output:

These are the packages that would be merged, in order:

Calculating dependencies
!!! All ebuilds that could satisfy openoffice have been masked.
!!! One of the following masked packages is required to complete your request:
- app-office/openoffice-2.0.4_rc1-r1 (masked by: package.mask, package.mask, 
~am d64 keyword)
# 2005/10/24 Simon Stelling bl...@gentoo.org
# Don't even try to compile openoffice-2.x, it won't work.

But not to go too far down this rathole, I'm just using openoffice as an 
example here.  If you want to talk about it more, take it off list please.

 Markus

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-03 Thread Rob Landley
On Saturday 03 January 2009 20:14:44 H. Peter Anvin wrote:
 Rob Landley wrote:
  The new patches have *more* environmental
  dependencies than that ever did.
 
  Could you please be a little more specific?

 In this case, you're assuming that every version of every shell this is
 going to get involved with is going to do math correctly with the
 requisite precision, which is nowhere guaranteed, I'm pretty sure.

Well, SUSv3 requires that the shell support signed long arithmetic:
http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04

And the LP64 standard says that on 64 bit systems, long must be 64 bit:
http://www.unix.org/whitepapers/64bit.html

Now the potential weakness there is that on 32 bit systems, shells might only 
support 32 bit math instead of 64 bit math.  (You'll notice I have a test for 
that.)  However, bash has supported 64 bit math on 32 bit systems since at 
least 2003.  (I keep a Red Hat 9 test image around because it had 50% market 
share when it shipped, so the majority of old Linux systems still in use 
_are_ RH9 or similar.  It also has the oldest gcc version the kernel still 
claims to build under.)  Busybox ash can also support 64 bit math on 32 bit 
hosts, and I just confirmed that the dash in the 32 bit xubuntu 8.10 supports 
64 bit math as well.

(It would also be possible to do a similar overflow avoiding trick to do the 
lot entirely in 32 bit math, but given that the three main shells in use on 
Linux _do_ support 64 bit math on 32 bit hosts and are _required_ to support 
it on 64 bit hosts according to SUSv3, the extra complexity really doesn't 
seem worth it.)

  Third, if someone actually cares to do it right, I have a smallish
  bignum library at http://git.zytor.com/?p=lib/pbn.git;a=summary that
  might be a starting point.
 
  This doesn't _need_ bignum support.  It maxes out around 72 bits and the
  _result_ can't use more than about $SHIFT bits because you're dividing by
  the amount you shifted, so just chop off the bottom 32 bits, do a normal
  64 bit division on the top (it has to fit), and then do the same division
  on the appropriate shifted remainder, and combine the results.  This is
  easy because when the shift _is_ 32 bits or more, the bottom 32 bits all
  have to be zeroes so you don't even have to mask and add, just shift the
  remainder left 32 bits so you can continue the divide.
 
  Pulling out perl isn't always a good alternative to thinking about the
  problem.

 Neither is open-coding a bignum operation instead of relying on an
 existing, validated implementation.

Implementing something by hand isn't _always_ a good alternative, sure.  That 
would be the thinking about the problem part.  In this instance, avoiding 
overflow is trivial.  (If 1-1 didn't wrap around, it wouldn't even need the 
if statement.)

I'm curious, would the existing, validated implementation in this instance 
be the perl implementation, or the library you wrote and pointed me to above 
as a potential replacement for it?

I suppose there's a certain amount of style in accusing me of reinventing the 
wheel while pointing me at your reinvention of the wheel.  (Are you aiming to 
replace Gnu's gmplib.org, the perhaps the BSD licensed one in openssh?  
Dropbear uses Libtommath.  A quick google for C open source big number 
libraries also found Libimath, MPI, NTL, BigDigits, decNumber, and MPI.  The 
last time I personally wrote my own arbitrary precision math package from 
scratch was in 1998, and that was in Java, so I'm a little rusty...)  But I 
don't personally consider avoiding the need for an arbitrary precision math 
library to be the same as reimplementing one.

   -hpa

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Arkadiusz Miskiewicz
On Friday 02 of January 2009, Rob Landley wrote:
 Before 2.6.25 (specifically git bdc807871d58285737d50dc6163d0feb72cb0dc2 )
 building a Linux kernel never required perl to be installed on the build
 system.  (Various development and debugging scripts were written in perl
 and python and such, but they weren't involved in actually building a
 kernel.) Building a kernel before 2.6.25 could be done with a minimal
 system built from gcc, binutils, bash, make, busybox, uClibc, and the Linux
 kernel, and nothing else.

And now bash is going to be required... while some distros don't need/have 
bash. /bin/sh should be enough.

Heh,
-- 
Arkadiusz MiśkiewiczPLD/Linux Team
arekm / maven.plhttp://ftp.pld-linux.org/
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Christoph Hellwig
On Fri, Jan 02, 2009 at 10:26:37AM +0100, Arkadiusz Miskiewicz wrote:
 On Friday 02 of January 2009, Rob Landley wrote:
  Before 2.6.25 (specifically git bdc807871d58285737d50dc6163d0feb72cb0dc2 )
  building a Linux kernel never required perl to be installed on the build
  system.  (Various development and debugging scripts were written in perl
  and python and such, but they weren't involved in actually building a
  kernel.) Building a kernel before 2.6.25 could be done with a minimal
  system built from gcc, binutils, bash, make, busybox, uClibc, and the Linux
  kernel, and nothing else.
 
 And now bash is going to be required... while some distros don't need/have 
 bash. /bin/sh should be enough.

*nod*  bash is in many ways a worse requirement than perl.  strict posix
/bin/sh + awk + sed would be nicest, but if that's too much work perl
seems reasonable.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Mark Miller


On Jan 2, 2009, at 3:26 AM, Arkadiusz Miskiewicz wrote:


On Friday 02 of January 2009, Rob Landley wrote:
Before 2.6.25 (specifically git  
bdc807871d58285737d50dc6163d0feb72cb0dc2 )
building a Linux kernel never required perl to be installed on the  
build
system.  (Various development and debugging scripts were written in  
perl

and python and such, but they weren't involved in actually building a
kernel.) Building a kernel before 2.6.25 could be done with a minimal
system built from gcc, binutils, bash, make, busybox, uClibc, and  
the Linux

kernel, and nothing else.


And now bash is going to be required... while some distros don't  
need/have

bash. /bin/sh should be enough.


Which distros only have /bin/sh which do not have Perl? I'm honestly  
curious.



--
Arkadiusz MiśkiewiczPLD/Linux Team
arekm / maven.plhttp://ftp.pld-linux.org/


--
Mark Miller
m...@mirell.org




--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Alejandro Mery
Christoph Hellwig escribió:
 On Fri, Jan 02, 2009 at 10:26:37AM +0100, Arkadiusz Miskiewicz wrote:
   
 On Friday 02 of January 2009, Rob Landley wrote:
 
 Before 2.6.25 (specifically git bdc807871d58285737d50dc6163d0feb72cb0dc2 )
 building a Linux kernel never required perl to be installed on the build
 system.  (Various development and debugging scripts were written in perl
 and python and such, but they weren't involved in actually building a
 kernel.) Building a kernel before 2.6.25 could be done with a minimal
 system built from gcc, binutils, bash, make, busybox, uClibc, and the Linux
 kernel, and nothing else.
   
 And now bash is going to be required... while some distros don't need/have 
 bash. /bin/sh should be enough.
 

 *nod*  bash is in many ways a worse requirement than perl.  strict posix
 /bin/sh + awk + sed would be nicest, but if that's too much work perl
 seems reasonable.
well, bash is not worse as bash is trivial to cross-compile to run on a
constrained sandbox and perl is a nightmare, but I agree bash should be
avoided too.

I think the $(( ... )) bash-ism can be replaced with a simple .c helper toy.

Thank Rob for reopening the topic.

Alejandro Mery



smime.p7s
Description: S/MIME Cryptographic Signature


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Paul Mundt
On Fri, Jan 02, 2009 at 04:32:42AM -0600, Mark Miller wrote:
 On Jan 2, 2009, at 3:50 AM, Paul Mundt wrote:
 Misguided rhetoric aside, what does this actually accomplish? If folks
 add meaningful tools in to the kernel that require python, and it is
 generally regarded as being fairly ubiquitous, I can't imagine there
 being any substantiated objections against it.
 
 And if the said meaningful tools introduce complex dependencies, then  
 there should be an open discussion as to why exactly we need those  
 tools as opposed to a simpler implementation.
 
Complex is relative, something that is fairly ubiquitious can hardly be
labelled as complex, regardless of whether historically people have had
issues with that dependency in certain spaces. In any event, simplifying
things is always good, but this open discussion thing is pure fancy,
since when was the kernel a democracy?

 Your main reasons against inclusion of perl seem to be that there is
 no realistic expectation for target systems that will be self-hosting
 will have perl included, or the inherent complexity in maintaining a
 coherent cross compiling environment. Both of these things are issues
 with your own environment, and in no way are these representative of
 the  embedded development community at large.
 
 I feel that if you attempted to look for discussions on cross- 
 compiling perl, you will meet with a variety of complaints on what a  
 nightmare it is to do so in a sandboxed environment.
 
I've had to deal with cross compiling perl for over a decade, in all of
its various forms, in all manner of embedded applications, so please tell
someone who cares. There are various options around this headache today,
as there have been for ages (though the options these days are rather
less painful), and if you felt like attempting to look for discussions on
those rather than trying to push this silly matter perhaps you might come
up with something.

The key thing you hit on is that there are a variety of complaints, and
that is all they have ever been. If your system is so constrained, you
shouldn't be doing builds on it in the first place. You certainly won't
be able to build anything in your crippled environment outside of some
simple applications anyways, this isn't a valid technical reason for
keeping the kernel a policy stranglehold.

 Now with that out of the way, this entire series fundamentally fails
 to convert half of the perl scripts shipped with the kernel today,
 some that are required for build depending on Kconfig options,
 
[snip]

 From what I can tell, it allows one to fully build the Linux kernel  
 without Perl.

Wrong, re-read what I said and try again.

 Ignoring the compile-time dependencies that you overlooked, what you
 define as development and debugging scripts are still an integral  
 part of the system, unless you are trying to argue that embedded
 developers have no interest in things like checkstack due to the
 trouble of trying to get perl built.
 
 Do I need that to compile a kernel? No.
 
Compile-time dependencies you do, yes.

 Perl was not required to build the Linux kernel. Now it is. It adds  
 another dependency to the Linux kernel. Requiring bash is far less a  
 dependency that Perl is.
 
Perhaps so, but that is largely irrelevant. Moving off of the bash
dependency is a lot more trivial than moving off of the perl one. The
kernel bashisms are fairly marginal, and if someone were to do the leg
work for that, it might even be accepted. Getting rid of perl is an
uphill battle for no visible gain. People will continue to write and add
scripts both in bash and perl on a routine basis regardless.

 The kernel is and always has been about using the right tool for the  
 job, not a matter of dictating what tools you must use in order to
 accomplish a task you are interested in. Common sense does apply here
 though, so this might be a more daunting task for some than others.
 
 How is Perl a better tool for the job than what currently bash and  
 other standard utilities already offer?
 
How is bash a better tool for than job than what sed and posix shell
already offer? Yes, we can reduce our dependencies to the bare minimum,
but that is not constructive for the folks that are actually writing the
scripts in the first place.

Likewise, this is not even a real problem in the embedded developer
demographic, the only place this is a problem is in specially stripped
distributions or people that don't want to go through the pain of cross
compiling perl. None of which is of any concern.

If people are going to write useful things that are reasonably expected
to be standard on build machines, there is no reason to restrict what
tools they are permitted to use. If you have a personal vendetta against
something that is fairly standard, that is entirely your own personal
problem, you can choose to deal with it or patch out of tree for your own
crippled environment (assuming patch isn't too much of an obscure
dependency).
--
To 

Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Rob Landley
On Friday 02 January 2009 03:26:37 Arkadiusz Miskiewicz wrote:
 On Friday 02 of January 2009, Rob Landley wrote:
  Before 2.6.25 (specifically git bdc807871d58285737d50dc6163d0feb72cb0dc2
  ) building a Linux kernel never required perl to be installed on the
  build system.  (Various development and debugging scripts were written in
  perl and python and such, but they weren't involved in actually building
  a kernel.) Building a kernel before 2.6.25 could be done with a minimal
  system built from gcc, binutils, bash, make, busybox, uClibc, and the
  Linux kernel, and nothing else.

 And now bash is going to be required... while some distros don't need/have
 bash. /bin/sh should be enough.

 Heh,

I believe all three scripts run under dash and busybox ash.  (The timeconst.sh 
one needs 64 bit math which dash only provides on 64 bit hosts, which is a 
regression from Red Hat 9 in 2003 by the way.  Busybox ash can also provide 64 
bit math on 32 bit hosts, and the script should run with that just fine if you 
haven't got bash and that's what your sh in the path is.)

The makefiles execute those scripts via CONFIG_SHELL, not via the #!/blah line 
at the start, so it's largely irrelevant what gets put there anyway.  If you 
haven't got bash installed it'll use sh, which should work with dash on a 64 
bit host or with busybox ash.  (That's why that one file has a test to make 
sure 64 bit math _does_ work.  The only Linux development environment I'm 
aware of where that test would trigger is if you use a 32 bit ubuntu and go 
out of your way to _uninstall_ bash.  Even Cygwin uses bash.)

Beyond that, find linux -type f | xargs grep bin/bash | wc comes up with 38 
instances (admittedly half of 'em in Documentation, but lots in scripts, and 
makefiles, and defconfigs, at least one hardwired in C code.)  So this would 
not a _new_ dependency.

By the way, what Linux distros install a compiler toolchain but not bash?  I'm 
curious.  (Even after Ubuntu moved #!/bin/sh to point to dash, it still 
installs bash as part of the default environment, even if you don't install 
development tools.)  You've built the kernel on this system before?

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Rob Landley
On Friday 02 January 2009 04:16:53 Alejandro Mery wrote:
 Christoph Hellwig escribió:
  On Fri, Jan 02, 2009 at 10:26:37AM +0100, Arkadiusz Miskiewicz wrote:
  On Friday 02 of January 2009, Rob Landley wrote:
  Before 2.6.25 (specifically git
  bdc807871d58285737d50dc6163d0feb72cb0dc2 ) building a Linux kernel
  never required perl to be installed on the build system.  (Various
  development and debugging scripts were written in perl and python and
  such, but they weren't involved in actually building a kernel.)
  Building a kernel before 2.6.25 could be done with a minimal system
  built from gcc, binutils, bash, make, busybox, uClibc, and the Linux
  kernel, and nothing else.
 
  And now bash is going to be required... while some distros don't
  need/have bash. /bin/sh should be enough.
 
  *nod*  bash is in many ways a worse requirement than perl.  strict posix
  /bin/sh + awk + sed would be nicest, but if that's too much work perl
  seems reasonable.

 well, bash is not worse as bash is trivial to cross-compile to run on a
 constrained sandbox and perl is a nightmare, but I agree bash should be
 avoided too.

 I think the $(( ... )) bash-ism can be replaced with a simple .c helper
 toy.

No, $[ ] is the bashism, $(( )) is susv3:
http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04

I intentionally switched from $[ ] to $(( )) to make dash work.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Mark Miller

On Jan 2, 2009, at 4:57 AM, Paul Mundt wrote:


On Fri, Jan 02, 2009 at 04:32:42AM -0600, Mark Miller wrote:

On Jan 2, 2009, at 3:50 AM, Paul Mundt wrote:
Misguided rhetoric aside, what does this actually accomplish? If  
folks

add meaningful tools in to the kernel that require python, and it is
generally regarded as being fairly ubiquitous, I can't imagine there
being any substantiated objections against it.


And if the said meaningful tools introduce complex dependencies, then
there should be an open discussion as to why exactly we need those
tools as opposed to a simpler implementation.

Complex is relative, something that is fairly ubiquitious can hardly  
be
labelled as complex, regardless of whether historically people have  
had
issues with that dependency in certain spaces. In any event,  
simplifying

things is always good, but this open discussion thing is pure fancy,
since when was the kernel a democracy?


I'm ignoring the bait.


Your main reasons against inclusion of perl seem to be that there is
no realistic expectation for target systems that will be self- 
hosting

will have perl included, or the inherent complexity in maintaining a
coherent cross compiling environment. Both of these things are  
issues

with your own environment, and in no way are these representative of
the  embedded development community at large.


I feel that if you attempted to look for discussions on cross-
compiling perl, you will meet with a variety of complaints on what a
nightmare it is to do so in a sandboxed environment.

I've had to deal with cross compiling perl for over a decade, in all  
of
its various forms, in all manner of embedded applications, so please  
tell

someone who cares.


Ignoring this as well.


There are various options around this headache today,
as there have been for ages (though the options these days are rather
less painful), and if you felt like attempting to look for  
discussions on
those rather than trying to push this silly matter perhaps you might  
come

up with something.


And ignoring this too.

The key thing you hit on is that there are a variety of complaints,  
and

that is all they have ever been. If your system is so constrained, you
shouldn't be doing builds on it in the first place. You certainly  
won't

be able to build anything in your crippled environment outside of some
simple applications anyways, this isn't a valid technical reason for
keeping the kernel a policy stranglehold.


I merely don't like seeing another dependency added when there's no  
logical reason to do it, other than why not. And there have been  
reasons given for the not. Your reply, seems merely to be, Because.




Now with that out of the way, this entire series fundamentally fails
to convert half of the perl scripts shipped with the kernel today,
some that are required for build depending on Kconfig options,



[snip]


From what I can tell, it allows one to fully build the Linux kernel
without Perl.


Wrong, re-read what I said and try again.


I have done so. And I restate, without these patches, Perl is required  
*for any install*. Your logic seems to be, if Perl is required for any  
part of doing something with the Linux kernel, then there is no reason  
it should not be allowed in all parts.


With this logic, soon the Linux kernel will require so many  
dependencies, that even a minimal system to rebuild itself under  
itself will comprise 200MB for stripped binaries. Right now, it's down  
to about 20ish.


With that, you also require more expertise in maintaing these variety  
of tools. You have given no logical reason on why we need to add more  
complexity to the kernel, when previous tools have managed to deal  
with this task for over ten years without requiring Perl.



Ignoring the compile-time dependencies that you overlooked, what you
define as development and debugging scripts are still an integral
part of the system, unless you are trying to argue that embedded
developers have no interest in things like checkstack due to the
trouble of trying to get perl built.


Do I need that to compile a kernel? No.


Compile-time dependencies you do, yes.


Perl was not required to build the Linux kernel. Now it is. It adds
another dependency to the Linux kernel. Requiring bash is far less a
dependency that Perl is.


Perhaps so, but that is largely irrelevant. Moving off of the bash
dependency is a lot more trivial than moving off of the perl one. The
kernel bashisms are fairly marginal, and if someone were to do the leg
work for that, it might even be accepted. Getting rid of perl is an
uphill battle for no visible gain. People will continue to write and  
add

scripts both in bash and perl on a routine basis regardless.


The kernel is and always has been about using the right tool for the
job, not a matter of dictating what tools you must use in order to
accomplish a task you are interested in. Common sense does apply  
here

though, so this might be a more daunting task for 

Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Rob Landley
On Friday 02 January 2009 03:50:23 Paul Mundt wrote:
 On Fri, Jan 02, 2009 at 02:07:28AM -0600, Rob Landley wrote:
  The perl checkin for 2.6.25 was the camel's nose under the tent flap, and
  since then two more instances of perl have shown up in the core kernel
  build. This patch series removes the three required to build a basic
  kernel for qemu for the targets I've tested (arm, mips, powerpc, sparc,
  m68k, sh4, and of course x86 and x86-64), replacing them with shell
  scripts.

 Misguided rhetoric aside, what does this actually accomplish? If folks
 add meaningful tools in to the kernel that require python, and it is
 generally regarded as being fairly ubiquitous, I can't imagine there
 being any substantiated objections against it.

I think bloat-o-meter is a marvelous tool, and I'm a big fan of python.  But I 
don't think you shouldn't have to run that to compile a kernel either, largely 
because not needing it for the first 17 years or so implied living without 
this requirement could be done, sustainably even.

There's a difference between a development workstation and a dedicated build 
system.  Requiring you to install X11 plus qt on the headless build server 
cranking out nightly snapshots in order to run the configure stage of the 
kernel build would be silly.  But this is not an argument for ripping out 
make xconfig from the kernel.

Spot the difference?

 Your main reasons against inclusion of perl seem to be that there is no
 realistic expectation for target systems that will be self-hosting will
 have perl included, or the inherent complexity in maintaining a coherent
 cross compiling environment.

I'm saying it's a major new environmental dependency that went in fairly 
recently and largely without comment, and it causes real world headaches for 
real people, of which I am only one.

If you don't think environmental dependencies are a problem, I welcome you to 
attempt to build open office.  (Even the gentoo guys gave up on that one and 
just started shipping a prebuilt binary.)

I think large amounts of complexity start with small amounts of complexity 
that grow.  Complexity is inevitable, but there should be a _reason_ for 
increases in it.

 Both of these things are issues with your
 own environment, and in no way are these representative of the embedded
 development community at large.

 Now with that out of the way, this entire series fundamentally fails to
 convert half of the perl scripts shipped with the kernel today, some that
 are required for build depending on Kconfig options, and others that are
 simply useful tools for self-hosted environments.

I didn't say the job was finished.  These are just the ones I've already 
personally hit, and thus A) needed to rewrite to build the kernel in my build 
environment, B) have a handy test case for.

 Simply converting a
 couple of scripts over you find objectionable is certainly fine if there
 is a real benefit in doing so, but this doesn't actually accomplish your
 goal of removing the perl dependency.

A) It's a start.

B) It works for me, and builds the .configs I've personally needed so far.

 Ignoring the compile-time dependencies that you overlooked, what you
 define as development and debugging scripts are still an integral part
 of the system, unless you are trying to argue that embedded developers
 have no interest in things like checkstack due to the trouble of trying
 to get perl built.

Coming up with new patches and modifying the source is a different use for 
source code than going ./configure; make; make install.  This is true for 
most open source software, I'd expect.

Or are you implying that eclipse or Emacs are such great IDEs that being able 
to build outside of a GUI isn't interesting?  The ability to build within an 
IDE should be allowed to preclude the ability to build without one?

 Until you can post a series that converts all of scripts/*.pl in its
 entirety, you have failed to address the fundamental reason why perl is
 used in the first place.

Never start anything unless you can finish it all in one go, eh?

Last I heard the kernel guys tend to frown on people who wander off in their 
own corner for a year and then dump a massive rewrite on them.  They seem to 
prefer the incremental trail of breadcrumbs approach.  Release early, 
release often, incorporate feedback, keep at it.

Or am I wrong?

 Trying to toss bizarre policy statements around
 regarding things you personally find objectionable without any coherent
 technical argument to the contrary is of no constructive use whatsoever.

Complexity is a cost, environmental dependencies are a form of complexity, if 
the benefit isn't worth the cost (or you can get the benefit without the cost) 
then you shouldn't pay the cost.

I was unaware this was a controversial attitude?

 The kernel is and always has been about using the right tool for the job,
 not a matter of dictating what tools you must use in order to accomplish
 a task you are 

Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Rob Landley
On Friday 02 January 2009 03:49:34 Christoph Hellwig wrote:
 On Fri, Jan 02, 2009 at 10:26:37AM +0100, Arkadiusz Miskiewicz wrote:
  On Friday 02 of January 2009, Rob Landley wrote:
   Before 2.6.25 (specifically git
   bdc807871d58285737d50dc6163d0feb72cb0dc2 ) building a Linux kernel
   never required perl to be installed on the build system.  (Various
   development and debugging scripts were written in perl and python and
   such, but they weren't involved in actually building a kernel.)
   Building a kernel before 2.6.25 could be done with a minimal system
   built from gcc, binutils, bash, make, busybox, uClibc, and the Linux
   kernel, and nothing else.
 
  And now bash is going to be required... while some distros don't
  need/have bash. /bin/sh should be enough.

 *nod*  bash is in many ways a worse requirement than perl.  strict posix
 /bin/sh + awk + sed would be nicest, but if that's too much work perl
 seems reasonable.

The scripts should work with dash (modulo the one that needs 64 bit math, 
which dash only provides on a 64 bit host), or with busybox ash (which can 
provide 64 bit math on 32 bit hosts just like bash can).  I'll explicitly 
retest both of those when I respin the patches in the strikemorning/strike 
afternoon.

(And yes I thought about writing my own arbitrary precision arithmetic shell 
functions, but it really didn't seem worth the complexity since the only 32 
bit Linux distros I've seen that install dash also install bash by default.  I 
just put in a test for 32 bit math so it can spot it and fail, on the off 
chance you're running a 32 bit host with dash after explicitly uninstalling 
bash.  All the embedded 32 bit ones that try to act as development 
environments use busybox ash, or more often just install bash.)

That said, how is bash _worse_ than perl?  (Where's the second implementation 
of perl?  Even Python had jython, but perl has... what?  The attempt to rebase 
on Parrot went down in flames...)

If the argument is that depending on a specific shell implementation is as 
bad as depending on the one and only implementation of perl, that argument I 
can at least follow, even if it doesn't actually apply in this case.  But 
where does worse come in?

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Theodore Tso
On Fri, Jan 02, 2009 at 06:56:31AM -0600, Rob Landley wrote: 
 That said, how is bash _worse_ than perl?  (Where's the second
 implementation of perl?  Even Python had jython, but perl
 has... what?  The attempt to rebase on Parrot went down in
 flames...)

(1) bash implies POSIX extensions; perl is actually quite portable.

(2) There are distributions that install with perl by default but not
bash; they use dash for speed reasons.

Sounds like though modulo dealing with 64-bit arithmetic, your patches
are mostly dash/POSIX.2 comformant, so you're probably mostly good on
that front once you address the 32/64-bit issues.  I'd also suggest
explicitly add a reminder to the shell scripts' comments to avoid
bashisms for maximum portability, to remind developers in the future
who might try to change the shell scripts to watch out for portability
issues.

- Ted

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Matthieu CASTET
Rob Landley a écrit :
 On Friday 02 January 2009 03:26:37 Arkadiusz Miskiewicz wrote:
 On Friday 02 of January 2009, Rob Landley wrote:

 Heh,
 
 I believe all three scripts run under dash and busybox ash.  (The 
 timeconst.sh 
 one needs 64 bit math which dash only provides on 64 bit hosts, which is a 
 regression from Red Hat 9 in 2003 by the way.
With dash 0.5.4-12 (from debian sid), I seems I got the 64 bit math for
32 bit hosts :
$ uname -m
i686
$ dash -c 'echo $((132))'
4294967296


Matthieu
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Wookey
On 2009-01-02 18:50 +0900, Paul Mundt wrote:
 Your main reasons against inclusion of perl seem to be that there is no
 realistic expectation for target systems that will be self-hosting will
 have perl included, or the inherent complexity in maintaining a coherent
 cross compiling environment. Both of these things are issues with your
 own environment, and in no way are these representative of the embedded
 development community at large.

It may well be true that most embedded people cross-build kernels and
use native perl on a fat build box, but there are plenty of situations
where being able to build kernels without perl is useful. Given the
simplicitly of these patches I can't see any reason not to put them
in, and appreciate Rob's work on this.

And if cross-building perl is really easy, as some in this thread
claim, can someone fix the Debian packages to do it, because that
would be really useful (it appears to be of similar complexity to
cross-building gcc, requiring a 2-stage self-referential build). 

Wookey
-- 
Principal hats:  Balloonz - Toby Churchill - Aleph One - Debian
http://wookware.org/
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread Sam Ravnborg
Hi Wookey.

 Given the
 simplicitly of these patches I can't see any reason not to put them
 in

Please do NOT do the mistake and think this the same thing.

Rob's patch simplyfy the timecost stuff - and will be applied on
this merit alone assuming comments will be addressed.

But the serie rased anohter topic: shall we ban use of perl
for generating a kernel.
And this is what primary is discussed and the outcome of
that discussion will not prevent patches that stands on their
own to be applied.

Sam
--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: PATCH [0/3]: Simplify the kernel build by removing perl.

2009-01-02 Thread H. Peter Anvin
Sam Ravnborg wrote:
 Hi Wookey.
 
 Given the
 simplicitly of these patches I can't see any reason not to put them
 in
 
 Please do NOT do the mistake and think this the same thing.
 
 Rob's patch simplyfy the timecost stuff - and will be applied on
 this merit alone assuming comments will be addressed.
 
 But the serie rased anohter topic: shall we ban use of perl
 for generating a kernel.
 And this is what primary is discussed and the outcome of
 that discussion will not prevent patches that stands on their
 own to be applied.
 

My personal opinion on this is that this is ridiculous.  Given that you
need gcc, binutils, make etc. to build the kernel, and this is more than
inherent, you have to have a pretty bloody strangely constrained system
to disallow Perl, which is as close to a standard Unix utility you can
get without making it into SuS.

The only *real* motivation I have seen for this is a system that as far
I can tell is nothing other than a toy, specifically designed to show
off how little you need to build the kernel.  In other words, it's not a
practical application, it's a show-off art piece.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line unsubscribe linux-embedded in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html