Re: rescue/ broke cross compiles
On Tue, Jul 01, 2003 at 12:07:26PM -0700, David O'Brien wrote: On Mon, Jun 30, 2003 at 05:35:16PM -0700, Marcel Moolenaar wrote: Build tools are most of the time so small or trivial (gcc is probably the exception, before that perl probably was) that building them again is lost in creating the rescue bits itself. Please no, don't pessimize the build even more than it absolutely has to be. release/Makefile had a similar issue when I've been working on a cross support for it (both cross-release and cross-platform). The release.5 still has to run the build-tools target again as it has no idea what tools contribute to the build-tools list. Another pessimization is with release.3, where we do a second buildworld pass with SUBDIR_OVERRIDE, but this pessimization is for a good reason. Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software Ltd, [EMAIL PROTECTED] FreeBSD committer pgp0.pgp Description: PGP signature
Re: rescue/ broke cross compiles
On Mon, Jun 30, 2003 at 05:35:16PM -0700, Marcel Moolenaar wrote: Build tools are most of the time so small or trivial (gcc is probably the exception, before that perl probably was) that building them again is lost in creating the rescue bits itself. Please no, don't pessimize the build even more than it absolutely has to be. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rescue/ broke cross compiles
On Tue, Jul 01, 2003 at 01:23:53AM +0300, Ruslan Ermilov wrote: Hi there! As seen by the latest series of tinderbox failures, the rescue/ stuff breaks cross compiles. The problem is that some bits like bin/sh have the so-called build tools. These are small utilities not normally visible in the world except during the build stage. As such, make buildworld builds them in the native host's environment (using the host compiler, headers, libraries, and binutils). The /rescue should have such a target too (build-tools), that would in effect call the build-tools targets in all makefiles that have it, e.g. bin/sh/Makefile. I'm the first to admit my Make-foo is lacking. I'm not sure I understand why /rescue needs build-tools bits. Can you help enlighten me? -gordon pgp0.pgp Description: PGP signature
Re: rescue/ broke cross compiles
On Mon, Jun 30, 2003 at 03:28:20PM -0700, Gordon Tetlow wrote: On Tue, Jul 01, 2003 at 01:23:53AM +0300, Ruslan Ermilov wrote: Hi there! As seen by the latest series of tinderbox failures, the rescue/ stuff breaks cross compiles. The problem is that some bits like bin/sh have the so-called build tools. These are small utilities not normally visible in the world except during the build stage. As such, make buildworld builds them in the native host's environment (using the host compiler, headers, libraries, and binutils). The /rescue should have such a target too (build-tools), that would in effect call the build-tools targets in all makefiles that have it, e.g. bin/sh/Makefile. I'm the first to admit my Make-foo is lacking. I'm not sure I understand why /rescue needs build-tools bits. Can you help enlighten me? Since you create a seperate object tree for rescue, you need to go through the same phases as a world does. That way tools (like build-tools) will be compiled against the right headers and linked against the right libraries (in both cases those of the machine on which the tool runs). Unfortunately, it's not that simple. There's a deliberate phase ordering that makes sure that we don't pick up cross-tools before we're ready for them. Since rescue is built *after* the cross- tools are installed, you'll have a hard time resolving the phase ordering problem. That's why ru@ suggested to add a build-tools target. That way you populate the seperate tree in sync with the phases of a world, thereby avoiding the phase ordering problem. FYI, -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rescue/ broke cross compiles
On Mon, Jun 30, 2003 at 03:52:06PM -0700, Marcel Moolenaar wrote: Since you create a seperate object tree for rescue, you need to go through the same phases as a world does. That way tools (like build-tools) will be compiled against the right headers and linked against the right libraries (in both cases those of the machine on which the tool runs). Unfortunately, it's not that simple. There's a deliberate phase ordering that makes sure that we don't pick up cross-tools before we're ready for them. Since rescue is built *after* the cross- tools are installed, you'll have a hard time resolving the phase ordering problem. That's why ru@ suggested to add a build-tools target. That way you populate the seperate tree in sync with the phases of a world, thereby avoiding the phase ordering problem. Is there a way to leverage the existing build-tools so we don't have to do extra compiling that isn't necessary? -gordon pgp0.pgp Description: PGP signature
Re: rescue/ broke cross compiles
On Mon, Jun 30, 2003 at 04:54:02PM -0700, Gordon Tetlow wrote: That's why ru@ suggested to add a build-tools target. That way you populate the seperate tree in sync with the phases of a world, thereby avoiding the phase ordering problem. Is there a way to leverage the existing build-tools so we don't have to do extra compiling that isn't necessary? Build tools are most of the time so small or trivial (gcc is probably the exception, before that perl probably was) that building them again is lost in creating the rescue bits itself. We could possibly copy the object directory of those tools that have build tools, but if there are paths embedded in generated scripts, we have to regenerate them anyway. What about this: rebuild the build tools to get things sorted out and working and then look if we can optimize? -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rescue/ broke cross compiles
Marcel Moolenaar wrote:
On Mon, Jun 30, 2003 at 04:54:02PM -0700, Gordon Tetlow wrote:
That's why ru@ suggested to add a build-tools target. That way you
populate the seperate tree in sync with the phases of a world,
thereby avoiding the phase ordering problem.
Is there a way to leverage the existing build-tools so we don't have
to do extra compiling that isn't necessary?
Build tools are most of the time so small or trivial (gcc is
probably the exception, before that perl probably was) that
building them again is lost in creating the rescue bits itself.
We could possibly copy the object directory of those tools that
have build tools, but if there are paths embedded in generated
scripts, we have to regenerate them anyway.
What about this: rebuild the build tools to get things sorted
out and working and then look if we can optimize?
Looking through the build tools for /bin/sh, it's definitely
not worth the effort to try copying build tools around.
Although it sounds easy to add a build-tools target to
handle this, I'm not sure I see exactly how to do this.
Any suggestions?
For the longer term, perhaps it would be desirable to
simply eliminate as many of the build-tools as possible?
For example, the attached is a pretty close substitute for
mkinit.c in the /bin/sh build. It's crude, but it seems to work
and eliminates the need to compile mkinit at build time.
I'll see if I can scrape together something similar for
the other /bin/sh tools.
Tim Kientzle
#!/bin/sh
echo /*
echo * This file was generated by the mkinit program.
echo */
echo
echo '#include shell.h'
echo '#include mystring.h'
cat $@ | grep '^INCLUDE' | sed -e s/INCLUDE/#include/
echo
echo
echo
cat $@ | sed -n -e '/^#define/ s/#define //p' | grep -v '\\$' | egrep -v '^[A-Z_]+\('
| awk '{print #undef ,$1; print #define,$0; }'
echo
echo
for f in $@
do
cat $f | sed -n -e '/^MKINIT$/,/^}/ p' -e '/^MKINIT / s/^MKINIT/extern/p' | grep
-v '^MKINIT$'
echo
done
echo
echo
echo /*
echo * Initialization code.
echo */
echo
echo void
echo init() {
for f in $@
do
echo /* from $f: */
cat $f | sed -n -e '/^INIT/,/^}/ p' | sed -e 's/INIT //' | \
awk '{print,$0;}' OFS=''
echo
done
echo }
echo
echo
echo
echo /*
echo * This routine is called when an error or an interrupt occurs in an
echo * interactive shell and control is returned to the main command loop.
echo */
echo
echo void
echo reset() {
for f in $@
do
echo /* from $f: */
cat $f | sed -n -e '/^RESET/,/^}/ p' | sed -e 's/RESET //' | \
awk '{print,$0;}' OFS=''
echo
done
echo }
echo
echo
echo
echo /*
echo * This routine is called to initialize the shell to run a shell procedure.
echo */
echo
echo void
echo initshellproc() {
for f in $@
do
echo /* from $f: */
cat $f | sed -n -e '/^SHELLPROC/,/^}/ p' | sed -e 's/SHELLPROC //' | \
awk '{print,$0;}' OFS=''
echo
done
echo }
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rescue/ broke cross compiles
On Mon, Jun 30, 2003 at 08:10:02PM -0700, Tim Kientzle wrote: Looking through the build tools for /bin/sh, it's definitely not worth the effort to try copying build tools around. Although it sounds easy to add a build-tools target to handle this, I'm not sure I see exactly how to do this. Any suggestions? Add a build-tools target to the Makefile in rescue and have it recurse to the tools that have a build-tools target. Of course rescue needs to be added to Makefile.inc For the longer term, perhaps it would be desirable to simply eliminate as many of the build-tools as possible? Elimination can be good as a way to remove gratuitous complexity, but gratuitous elimination as a way to remove complexity is probably not the way to go. For example, the attached is a pretty close substitute for mkinit.c in the /bin/sh build. It's crude, but it seems to work and eliminates the need to compile mkinit at build time. In general I think that the more portable the build tool, the better. If the shell script is not gross or overly ugly compared to the C program, then replacing the latter may not be a bad idea. I leave this for other to decide, unless there's only 1 build tool we need to handle for rescue and we can solve our problem by using the shell script instead of adding make logic. -- Marcel Moolenaar USPA: A-39004 [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: rescue/ broke cross compiles
Marcel Moolenaar wrote:
On Mon, Jun 30, 2003 at 08:10:02PM -0700, Tim Kientzle wrote:
In general I think that the more portable the build tool, the better.
If the shell script is not gross or overly ugly compared to the C
program, then replacing the latter may not be a bad idea.
The attached diff replaces two of the build tools for /bin/sh
with equivalent shell scripts (using grep/sed/awk). I
personally find the awk versions clearer than the C versions.
I apologize that these have not been very well tested, but they
were written in quite a rush. For the most part, the output
of these is exactly the same as that from the C programs.
The few differences look functionally equivalent to me, but
I've not had time to test it very carefully.
The remaining 'mksyntax' also looks pretty easy to replace with
a shell script, at which point, /bin/sh will require no
build-tools target. Then, /rescue will also not need a buld-tools
target (assuming everyone is willing to drop csh from
/rescue).
Unfortunately, it's late, I haven't had dinner, and I have
a lot of paying work piling up. sigh Someone else may
have to finish this, as I won't be able to get back to it
for a few days.
Tim Kientzle
Index: Makefile
===
RCS file: /usr/cvs/FreeBSD-CVS/src/bin/sh/Makefile,v
retrieving revision 1.40
diff -u -r1.40 Makefile
--- Makefile2 May 2003 06:26:32 - 1.40
+++ Makefile1 Jul 2003 05:25:08 -
@@ -31,7 +31,7 @@
mksyntax mksyntax.o
CLEANFILES+= ${GENSRCS} ${GENHDRS}
-build-tools: mkinit mknodes mksyntax
+build-tools: mksyntax
.ORDER: builtins.c builtins.h
builtins.c builtins.h: mkbuiltins builtins.def
@@ -39,20 +39,18 @@
init.c: mkinit alias.c eval.c exec.c input.c jobs.c options.c parser.c \
redir.c trap.c var.c
- ./mkinit ${.ALLSRC:S/^mkinit$//}
+ sh ${.CURDIR}/mkinit ${.ALLSRC:S/^mkinit$//}
# XXX this is just to stop the default .c rule being used, so that the
# intermediate object has a fixed name.
# XXX we have a default .c rule, but no default .o rule.
.o:
${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
-mkinit: mkinit.o
-mknodes: mknodes.o
mksyntax: mksyntax.o
.ORDER: nodes.c nodes.h
nodes.c nodes.h: mknodes nodetypes nodes.c.pat
- ./mknodes ${.CURDIR}/nodetypes ${.CURDIR}/nodes.c.pat
+ sh ${.CURDIR}/mknodes ${.CURDIR}/nodetypes ${.CURDIR}/nodes.c.pat
.ORDER: syntax.c syntax.h
syntax.c syntax.h: mksyntax
--- /dev/null Mon Jun 30 22:15:03 2003
+++ mkinit Mon Jun 30 22:25:04 2003
@@ -0,0 +1,91 @@
+#!/bin/sh
+
+( # to send output to init.c
+
+echo /*
+echo * This file was generated by the mkinit program.
+echo */
+echo
+echo '#include shell.h'
+echo '#include mystring.h'
+
+cat $@ | grep '^INCLUDE' | sed -e s/INCLUDE/#include/
+
+echo
+echo
+echo
+
+cat $@ | sed -n -e '/^#define/ s/#define //p' | grep -v '\\$' | egrep -v '^[A-Z_]+\('
| awk '{print #undef ,$1; print #define,$0; }'
+
+echo
+echo
+
+for f in $@
+do
+cat $f | sed -n -e '/^MKINIT$/,/^}/ p' -e '/^MKINIT / s/^MKINIT/extern/p' | grep
-v '^MKINIT$'
+echo
+done
+
+echo
+echo
+echo /*
+echo * Initialization code.
+echo */
+
+echo
+echo void
+echo init() {
+
+for f in $@
+do
+echo /* from $f: */
+cat $f | sed -n -e '/^INIT/,/^}/ p' | sed -e 's/INIT //' | \
+awk '{print ,$0;}' OFS=''
+echo
+done
+
+echo }
+
+echo
+echo
+echo
+
+echo /*
+echo * This routine is called when an error or an interrupt occurs in an
+echo * interactive shell and control is returned to the main command loop.
+echo */
+echo
+echo void
+echo reset() {
+
+for f in $@
+do
+echo /* from $f: */
+cat $f | sed -n -e '/^RESET/,/^}/ p' | sed -e 's/RESET //' | \
+awk '{print ,$0;}' OFS=''
+echo
+done
+echo }
+
+echo
+echo
+echo
+echo /*
+echo * This routine is called to initialize the shell to run a shell procedure.
+echo */
+echo
+echo void
+echo initshellproc() {
+
+
+for f in $@
+do
+echo /* from $f: */
+cat $f | sed -n -e '/^SHELLPROC/,/^}/ p' | sed -e 's/SHELLPROC //' | \
+awk '{print ,$0;}' OFS=''
+echo
+done
+echo }
+
+
+) init.c
--- /dev/null Mon Jun 30 22:15:03 2003
+++ mknodes Mon Jun 30 22:20:58 2003
@@ -0,0 +1,227 @@
+#!/bin/sh
+
+NODEFILE=$1
+PATFILE=$2
+
+#
+# The awk scripts are driven by the 'nodetypes' file format,
+# which is basically just a set of C structure definitions.
+#
+# Here's an example entry from nodetypes:
+# NSEMI nbinary
+#type int
+#ch1 nodeptr
+#ch2 nodeptr
+#
+# This says that an 'NSEMI' node uses the 'nbinary' structure;
+# the following lines define the structure. Another example:
+#
+# NWHILE nbinary
+#
+# This says that an 'NWHILE' node uses the 'nbinary' structure,
+# but doesn't define the structure (since it's defined elsewhere).
+# Correlating all of this is much simpler with AWK's associative
+# arrays than with the old C version.
+#
+
+
