Mattia Barbon recently implemented the capability to group multiple
dynamic PMCs into a single library. It took me a while, but the correct
way of using it finally percolated through my thick skull. One remaining
problem is that the build process is very platform-dependent. This
patch doesn't fix that, but it does eliminate the gmake dependency.
Another problem is that you have to specifically write Makefile rules to
build your group of dynamic PMCs into a library, and that is very
difficult to do portably.
This patch introduces something that feels suspiciously like libtool,
despite the fact that libtool has never been very kind to me. But for
now I am targeting this only at the dynamic PMC generation problem; this
solution could be expanded to ease porting of other parts of the build
procedure, but I think other people are already working on that.
The patch adds an additional target to config/gen/makefiles.pl: instead
of just converting config/gen/makefiles/dynclasses.in to
dynclasses/Makefile, it also converts
config/gen/makefiles/dynclasses.pl.in to dynclasses/build.pl, and
changes that Makefile to call build.pl to do all the real work. It is
thus able to pick up config/init/data.pl's notions of all of the ${cc},
${ld}, etc. definitions, but leaves the description of which PMCs to
build with the original Makefile (which probably isn't the greatest
place, but I'm trying to change as little as possible.)
My guess is that this will not immediately cause dynamic PMCs to start
working on the platforms where they do not currently work, but it should
make it easier to get them to work. It also implements a new pmclass
attribute in .pmc files (only meaningful for dynamic PMCs): C<group
GROUPNAME>, which will get automatically picked up by the new
dynclasses/build.pl to generate a single shared library out of all PMCs
with the same group tag. So to implement two new dynamic PMCs
'mylangPmc1' and 'mylangPmc2', you would:
* Implement the .pmc files, and include 'group mylang' in their pmclass lines
* Add mylangPmc1 and mylangPmc2 to config/gen/makefiles/dynclasses.in
* Re-run Configure.pl
That is the same procedure as is currently used to implement independent
dynamic PMCs right now, except for the addition of the 'group mylang'
tag.
I am not committing this patch directly because I know that other people
are currently actively working on the dynamic PMC stuff and the build
system, and I didn't want to step on anyone's toes. Note that build.pl
is NOT a general build tool, although it covers everything needed for
the dynclasses/ directory. At the moment, it doesn't even bother to do
dependency analysis for the grouped PMCs, although it does for all of
the rest. Still, this patch gets stuff working that currently doesn't
exist, and doesn't break anything that currently works AFAIK. I fully
expect (and hope) that it will be replaced by something more general
someday. But I'd rather not wait for that day, having first-hand
experience with how much "fun" it is to get partial linking of dynamic
libraries working on multiple platforms.
Index: config/gen/makefiles.pl
===================================================================
RCS file: /cvs/public/parrot/config/gen/makefiles.pl,v
retrieving revision 1.34
diff -u -r1.34 makefiles.pl
--- config/gen/makefiles.pl 19 Jun 2004 09:33:09 -0000 1.34
+++ config/gen/makefiles.pl 5 Sep 2004 22:28:23 -0000
@@ -81,6 +81,8 @@
commentType => '#', replace_slashes => 1);
genfile('config/gen/makefiles/dynclasses.in', 'dynclasses/Makefile',
commentType => '#', replace_slashes => 1);
+ genfile('config/gen/makefiles/dynclasses.pl.in', 'dynclasses/build.pl',
+ commentType => '#', replace_slashes => 0);
genfile('config/gen/makefiles/dynoplibs.in', 'dynoplibs/Makefile',
commentType => '#', replace_slashes => 1);
genfile('config/gen/makefiles/parrot_compiler.in',
'languages/parrot_compiler/Makefile',
Index: classes/pmc2c2.pl
===================================================================
RCS file: /cvs/public/parrot/classes/pmc2c2.pl,v
retrieving revision 1.16
diff -u -r1.16 pmc2c2.pl
--- classes/pmc2c2.pl 22 Aug 2004 09:15:51 -0000 1.16
+++ classes/pmc2c2.pl 5 Sep 2004 22:28:24 -0000
@@ -135,12 +135,6 @@
Used with C<abstract>: No C<class_init> code is generated.
-=item C<dynpmc>
-
-The class is a dynamic class. These have a special C<class_init>
-routine suitable for dynamic loading at runtime. See the F<dynclasses>
-directory for an example.
-
=item C<const_too>
Classes with this flag get 2 vtables and 2 enums, one pair with
@@ -164,6 +158,18 @@
library
ref
+=item C<dynpmc>
+
+The class is a dynamic class. These have a special C<class_init>
+routine suitable for dynamic loading at runtime. See the F<dynclasses>
+directory for an example.
+
+=item C<group GROUP>
+
+The class is part of a group of interrelated PMCs that should be
+compiled together into a single shared library of the given name. Only
+valid for dynamic PMCs.
+
=back
=item 3.
@@ -318,7 +324,7 @@
my $c = shift;
$$c =~ s/^(.*?^\s*)pmclass ([\w]*)//ms;
my ($pre, $classname) = ($1, $2);
- my %has_value = ( does => 1, extends => 1 );
+ my %has_value = ( does => 1, extends => 1, group => 1 );
my %flags;
# look through the pmc declaration header for flags such as noinit
Index: config/init/data.pl
===================================================================
RCS file: /cvs/public/parrot/config/init/data.pl,v
retrieving revision 1.31
diff -u -r1.31 data.pl
--- config/init/data.pl 10 Jul 2004 07:13:43 -0000 1.31
+++ config/init/data.pl 5 Sep 2004 22:28:24 -0000
@@ -32,6 +32,7 @@
package Configure::Data;
use Config;
use Data::Dumper;
+ use FindBin; # see build_dir
# We need a Glossary somewhere!
@@ -40,6 +41,8 @@
optimize => $optimize ? $Config{optimize} : '',
verbose => $verbose,
+ build_dir => $FindBin::Bin,
+
# Compiler -- used to turn .c files into object files.
# (Usually cc or cl, or something like that.)
cc => $Config{cc},
Index: config/gen/makefiles/dynclasses.in
===================================================================
RCS file: /cvs/public/parrot/config/gen/makefiles/dynclasses.in,v
retrieving revision 1.5
diff -u -r1.5 dynclasses.in
--- config/gen/makefiles/dynclasses.in 25 Apr 2004 10:47:41 -0000 1.5
+++ config/gen/makefiles/dynclasses.in 5 Sep 2004 22:28:24 -0000
@@ -1,41 +1,27 @@
-#
-# sample Makefile
-#
-LD = ${ld}
-LD_SHARED = ${ld_shared}
PERL = ${perl}
RM_F = ${rm_f}
SO = ${so}
-CFLAGS = ${ccflags} ${cc_debug} ${ccwarn} ${cc_hasjit} ${cg_flag} ${gc_flag}
-
# add your dynamic pmcs here
-all: foo$(SO) subproxy$(SO) \
-tclobject$(SO) tclstring$(SO) tclint$(SO) tclfloat$(SO) \
-tcllist$(SO) tclarray$(SO)
-
-.SUFFIXES: .pmc .c $(SO)
-
-# preserve .c if needed
-.PRECIOUS: %.c
-
-%.c : %.pmc
- $(PERL) ..${slash}classes${slash}pmc2c2.pl --dump $<
- $(PERL) ..${slash}classes${slash}pmc2c2.pl --c $<
-
-%$(SO) : %.c
- $(LD) $(CFLAGS) $(LD_SHARED) $(LD_SHARED_FLAGS) $(LDFLAGS) \
- ${cc_o_out}$@ \
- -I..${slash}include -I..${slash}classes \
- -L..${slash}blib${slash}lib -lparrot $<
- $(PERL) -MFile::Copy=cp -e ${PQ}cp q|$@|, q|../runtime/parrot/dynext/$@|${PQ}
+PMCS = foo subproxy \
+tclobject tclstring tclint tclfloat \
+tcllist tclarray \
+match matchrange
+
+BUILD = ${perl} build.pl
+
+all :
+ $(BUILD) generate $(PMCS)
+ $(BUILD) compile $(PMCS)
+ $(BUILD) linklibs $(PMCS)
+ $(BUILD) copy --destination=../runtime/parrot/dynext $(PMCS)
clean :
- $(RM_F) *.c *.h *$(SO) *.dump
+ $(RM_F) *.c *.h *$(SO) *.dump lib-*
realclean: clean
- $(RM_F) Makefile
+ $(RM_F) Makefile build.pl
distclean: realclean
Index: config/gen/makefiles/dynclasses.pl.in
===================================================================
RCS file: config/gen/makefiles/dynclasses.pl.in
diff -N config/gen/makefiles/dynclasses.pl.in
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ config/gen/makefiles/dynclasses.pl.in 6 Sep 2004 23:05:50 -0000
@@ -0,0 +1,188 @@
+use strict;
+use File::Copy qw(copy move);
+
+# Config stuff
+our $CC = "${cc} -c";
+our $LD = "${ld}";
+our $LD_SHARED = "${ld_shared}";
+our $LD_SHARED_FLAGS = "${ld_shared_flags}";
+our $LDFLAGS = "${ldflags}";
+our $PERL = "${perl}";
+our $SO = "${so}";
+our $O = "${o}";
+our $CFLAGS = "${ccflags} ${cc_debug} ${ccwarn} ${cc_hasjit} ${cg_flag} ${gc_flag}";
+our $PMC2C = "$PERL ${build_dir}${slash}classes${slash}pmc2c2.pl";
+
+# Actual commands
+sub compile_shared_cmd {
+ my ($target, $source) = @_;
+ "$LD $CFLAGS $LD_SHARED $LD_SHARED_FLAGS $LDFLAGS " .
+ "${cc_o_out}" . $target . " " .
+ "-I${build_dir}${slash}include -I${build_dir}${slash}classes " .
+ "-L${build_dir}${slash}blib${slash}lib -lparrot " .
+ $source;
+};
+
+sub compile_cmd {
+ my ($target, $source) = @_;
+ "$CC $CFLAGS " .
+ "${cc_o_out}" . $target . " " .
+ "-I${build_dir}${slash}include -I${build_dir}${slash}classes " .
+ $source;
+};
+
+sub partial_link_cmd {
+ my ($target, @sources) = @_;
+ "$LD $CFLAGS $LD_SHARED $LD_SHARED_FLAGS $LDFLAGS ".
+ "${cc_o_out}" . $target . " " .
+ join(" ", @sources);
+}
+
+our $NOW = time;
+
+################### MAIN PROGRAM ################
+
+my ($mode, @pmcs) = @ARGV;
+
+if ($mode eq 'generate') {
+ # Convert X.pmc -> X.dump and X.c and also create any lib-GROUP.c files
+
+ generate_dump($_) foreach (@pmcs);
+ generate_c($_) foreach (@pmcs);
+
+ my ($group_files, $pmc_group) = gather_groups(@pmcs);
+
+ while (my ($group, $pmcs) = each %$group_files) {
+ my $pmcfiles = join(" ", map { "$_.pmc" } @$pmcs);
+ run("$PMC2C --library $group --c $pmcfiles")
+ or die "pmc2c library creation failed ($?)\n";
+ }
+} elsif ($mode eq 'compile') {
+ my ($group_files, $pmc_group) = gather_groups(@pmcs);
+
+ my @grouped_pmcs = grep { exists $pmc_group->{$_} } @pmcs;
+ my @ungrouped_pmcs = grep { ! exists $pmc_group->{$_} } @pmcs;
+
+ # Convert X.c -> X.so for all non-grouped X.c
+ compile_shared($_) foreach (@ungrouped_pmcs);
+
+ # Convert X.c -> X.o for all grouped X.c
+ compile($_) foreach (@grouped_pmcs);
+
+ # lib-GROUP.c
+ for my $group (keys %$group_files) {
+ compile("$group", "lib-$group")
+ or die "compile $group.c failed ($?)\n";
+ }
+} elsif ($mode eq 'linklibs') {
+ my ($group_files, $pmc_group) = gather_groups(@pmcs);
+
+ # Convert lib-GROUP.so + A.so + B.so ... -> GROUP.so
+ while (my ($group, $pmcs) = each %$group_files) {
+ partial_link($group, "lib-$group", @$pmcs)
+ or die "partial link of $group failed ($?)\n";
+ }
+} elsif ($mode eq 'copy') {
+ # Copy *.so -> destination, where destination is the first
+ # argument, given as --destination=DIRECTORY
+ shift(@pmcs) =~ /--destination=(.*)/
+ or die "copy command requires destination";
+ my $dest = $1;
+
+ my ($group_files, $pmc_group) = gather_groups(@pmcs);
+
+ foreach (@pmcs, keys %$group_files) {
+ copy("$_$SO", $dest) or die "Copy $_$SO failed ($?)\n";
+ }
+} else {
+ die "invalid command '$mode'\nmust be one of generate, compile, linklibs, or
copy\n";
+}
+
+sub run {
+ print join(" ", @_), "\n";
+ return system(@_) == 0;
+}
+
+sub gather_groups {
+ my %group_files;
+ my %pmc_group;
+ for my $pmc (@_) {
+ our $class;
+ require "$pmc.dump";
+ my $group = $class->{flags}{group}
+ or next;
+ ($group) = keys %$group;
+ $pmc_group{$pmc} = $group;
+ push @{ $group_files{$group} }, $pmc;
+ }
+
+ return (\%group_files, \%pmc_group);
+}
+
+sub modtime {
+ my $ago = (-M shift);
+ if (defined $ago) {
+ return $NOW - $ago;
+ } else {
+ return;
+ }
+}
+
+sub needs_build {
+ my ($target, @sources) = @_;
+ my $target_mod = modtime($target)
+ or return 1;
+ for my $source (@sources) {
+ return 1 if modtime($source) >= $target_mod;
+ }
+ return 0;
+}
+
+sub generate_dump {
+ my ($pmc) = @_;
+
+ if (needs_build("$pmc.dump", "$pmc.pmc")) {
+ run("$PMC2C --dump $pmc.pmc")
+ or die "pmc2c dump failed ($?)\n";
+ } else {
+ print "$pmc.dump is up to date\n";
+ }
+}
+
+sub generate_c {
+ my ($pmc) = @_;
+
+ if (needs_build("$pmc.c", "$pmc.pmc")) {
+ run("$PMC2C --c $pmc.pmc")
+ or die "pmc2c code generation failed ($?)\n";
+ }
+}
+
+sub compile {
+ my ($src_stem, $dest_stem) = @_;
+ $dest_stem ||= $src_stem;
+ if (needs_build("$dest_stem$O", "$src_stem.c")) {
+ run(compile_cmd("$dest_stem$O", "$src_stem.c"))
+ or die "compile $src_stem.c failed ($?)\n";
+ }
+}
+
+sub compile_shared {
+ my ($src_stem, $dest_stem) = @_;
+ $dest_stem ||= $src_stem;
+ if (needs_build("$dest_stem$SO", "$src_stem.c")) {
+ run(compile_shared_cmd("$dest_stem$SO", "$src_stem.c"))
+ or die "compile $src_stem.c failed ($?)\n";
+ } else {
+ print "$dest_stem$SO is up to date\n";
+ }
+}
+
+sub partial_link {
+ my ($group, @stems) = @_;
+ my @sources = map { "$_$O" } @stems;
+ if (needs_build("$group$SO", @sources)) {
+ run(partial_link_cmd("$group$SO", @sources))
+ or die "partial link $group$SO failed ($?)\n";
+ }
+}
