On Thu Nov 30 19:25:49 2006, [EMAIL PROTECTED] wrote:
> On Thu Nov 30 18:34:48 2006, [EMAIL PROTECTED] wrote:
>
> I fixed the problem with GetOptions() -- but there are additional
> problems that will take time to
> diagnose. So my patches are not yet ready for prime time.
>
I'm attaching a file, mypmc2c.pl, that fixes the problems I had with
GetOptions() in my recent
patch submission for tools/build/pmc2c.pl.
BUT ... today it began to dawn on me that I have seriously misunderstood some
of the
problems we face in testing Parrot's build tools and other parts of the
distribution.
In retrospect, I'm not surprised that problems with GetOptions() would cause my
version of
pmc2c.pl to fail. As I explained in an earlier post, my objective was to take
as much of the
functionality in the current pmc2c.pl out and place it in a module,
Parrot::Pmc2c::Utils,
thereby making that functionality more amenable to testing and coverage
analysis. And my
test suite reflected that. But the one bit of functionality that I could *not*
extract from
pmc2c.pl was the options processing. And that's where the problem lay.
But not the *only* problem. After I fixed the options processing, I was still
having 'make'
failures with my version of pmc2c.pl. Worse, most of the seven files in my
test suite were
starting to fail as well -- when previously they had been passing with flying
colors!
That was Thursday. Then last night my tests started to pass again -- even
though I had
made no further changes to pmc2c.pl and no significant changes to the tests and
none at all
to Parrot::Pmc2c::Utils. Ever more surprising was that at least once, when I
edited Makefile to
substitute 'mypmc2c.pl' for 'pmc2c.pl', 'make' started to succeed. I excitedly
asked Coke,
with whom I was talking on #parrot, to double-check. I did a 'make clean' to
start fresh and
reconfirm the good news -- only to have 'make' once again no longer succeed
and, worse
yet, *all* my test files start to fail.
It was at that point that I began to realize that by going through the cycle of
'make clean',
'Configure.pl', 'make' and 'make test' interspersed with runs of my test suite,
I was *changing
the environment in which my tests were running*. And the major determinant of
whether my
tests and pmc2c.pl would pass or not was the stage in the cycle at which I was
running the
tests.
>From Nov 11 until last night, I had not rebuilt Parrot on my machine (though I
>had done
frequent 'svn update's). So all during that time my tests for functions such
as find_file() and
read_dump() were passing because they were finding files which had been created
by calling
'make' on Nov 11. But once I called 'make clean', many of those files were no
longer there,
so the tests of the Utils.pm functions which looked for them necessarily
failed. When I called
Configure.pl and make once again, those files were regenerated and the tests of
the Utils.pm
functions which look for them began to pass anew.
This was most startlingly evident at a point where I called 'make clean', then
called 'prove t/
tools/pmc2cutils/*.t' and *all* my tests began to fail with a message
indicating that package
Parrot::PMC could not be located. It turned out that this was a correct
failure, because
Parrot::PMC is *not* included in the svn distribution; it's automatically
generated by the
operation of Configure.pl -- which I had not yet called! Once I called
Configure.PL, that
package got created, which meant that Parrot::Pmc2c could 'use' it
successfully, which in turn
meant that Parrot::Pmc2c::Utils could load as well. But many of my tests
continued to fail,
and now I understood why: because I had only called Configure.PL but had not
yet called
make, many of the files which my tests were searching for did not yet exist!
In most of the code I've written, either for CPAN or $job(s), I've taken pains
to make sure that
any files created during, and solely for, the testing process are not left on
the user's machine
after testing and installation. And most of the test suites I've written make
no assumptions
about the presence or absence of files on the user's machine at the onset of
testing. A well
written test suite should give idempotent results.
This approach does not suffice in this case. The functionality in pmc2c.pl
(whether in the
script itself or extracted to a separate package) is strictly dependent on the
point in the
'make' process where it is called. Call the subroutines embodying that
functionality too early
(i.e., before calling Configure.pl or before calling make) and you will
necessarily get failures.
But call them, in a sense, too late (once you have already successfully called
make) and you
get spurious successes, because then you're just finding or re-creating files
which already
exist. The conditions prevailing at different points militate against
idempotent results.
Tests of Parrot's build tools are not tests of Parrot itself. Once Parrot's
make call has
succeeded, testing the components of the build tools is irrelevant. Hence,
including tests of
the build tools in the suite of tests called by 'make test' is also irrelevant.
The build tools can
only be validly tested in a 'pre-make' environment. But that raises the
question: How far
'pre-make': a pre-Configure.pl environment, or a post-Configure.pl, pre-make
environment
(or some combination of the two)?
Which boils down to a more fundamental question: What exactly are we trying to
accomplish
via testing of Parrot's build tools?
Ladies and gentlemen, the floor is open for discussion. Thank you for reading
this far.
kid51
#! perl
# Copyright (C) 2001-2006, The Perl Foundation.
# $Id: pmc2c.pl 15160 2006-11-07 14:44:10Z particle $
use strict;
use warnings;
use FindBin;
use lib "$FindBin::Bin/../..";
use lib "$FindBin::Bin/../../lib";
use Getopt::Long;
use Data::Dumper;
use Parrot::Pmc2c::Utils;
my (%opt, @include);
my %action;
GetOptions(
"include=s" => [EMAIL PROTECTED],
"vtable" => \$action{default},
"dump" => \$action{dump},
"c|gen-c" => \$action{gen_c},
"tree" => \$action{tree},
"no-body" => \$opt{nobody},
"no-lines" => \$opt{nolines},
"debug+" => \$opt{debug},
"verbose+" => \$opt{verbose},
"library=s" => \$opt{library},
) or exit 1;
if ( 0 == grep { $action{$_} } keys %action ) {
die "No action specified!\n";
}
my @args = @ARGV;
my $self = Parrot::Pmc2c::Utils->new( {
include => [EMAIL PROTECTED],
opt => \%opt,
args => [EMAIL PROTECTED],
} );
if ( $action{default} ) {
my $dump_file = $self->dump_default("$FindBin::Bin/../../vtable.tbl");
exit;
}
if ( $action{dump} ) {
$self->dump_pmc();
exit;
}
if ( $action{tree} ) {
$self->print_tree( {
depth => 0,
} );
exit;
}
if ( $action{gen_c} ) {
$self->gen_c();
exit;
}
=head1 NAME
tools/build/pmc2c.pl - PMC definition to C compiler
=head1 SYNOPSIS
=head2 Options used in Parrot F<Makefile>
Create F<src/pmc/foo.dump>:
% perl tools/build/pmc2c.pl --dump src/pmc/foo.pmc ...
Create F<vtable.dump>:
% perl tools/build/pmc2c.pl --vtable
Create F<src/pmc/foo.c> and C<pmc_foo.h> from F<src/pmc/foo.dump>:
% perl tools/build/pmc2c.pl -c src/pmc/foo.pmc ...
=head2 Other Options
Print a class tree for the specified PMCs:
% perl tools/build/pmc2c.pl --tree src/pmc/*.pmc
Create fooX.c and pmc_fooX.h from fooX.dump files, also create libfoo.c
containing the initialization function for all fooX PMCs.
% perl tools/build/pmc2c.pl --library libfoo -c \
src/pmc/foo1.pmc src/pmc/foo2.pmc ...
=head1 DESCRIPTION
The job of the PMC compiler is to take .pmc files and create C files
that can be compiled for use with the Parrot interpreter.
=head1 COMMAND-LINE OPTIONS
=over 4
=item C<--debug>
Increase debug level
=item C<--verbose>
Increase verbose level
=item C<--no-lines>
Omit source line info
=item C<--no-body>
Emit an empty body in the dump. This may be useful for debugging.
=item C<--include=/path/to/pmc>
Specify include path where to find PMCs.
=item C<--library=libname>
Specify the library name. This will create F<E<lt>libnameE<gt>.c> and
F<pmc_E<lt>libnameE<gt>.h>. The initialization function will be named
after libname and will initialize all PMCs in the library.
=back
=head2 Internals
To see the internal data structures please run:
% perl tools/build/pmc2c.pl --c --debug --debug sarray.pmc | less
=head2 Compiling PMCs
First, the program determines the names of the .c and .h files from
the basename of the .pmc file (e.g. F<perlint.pmc> -> F<perlint.c> and
F<perlint.h>).
Next, the file is searched for C</pmclass \w*/> which attempts to find the
class being declared.
Once the class is found, all of its superclasses are scanned and their
methods added to the methods of the current PMC. PMCs default to
inheriting from 'default'. Only single inheritance is supported.
Once the superclass is determined, it is processed and its method names
are extracted and saved.
Next, each method body is processed with various directives (see below)
getting replaced by their appropriate values.
Finally, the .c and .h files are generated. The appropriate base class
header files are included.
If the C<noinit> flag was used, then no init function is generated.
Otherwise, one is generated which sets up the vtable and enters it into
the C<vtables> array.
The .c file is generated by appending the functions after the various
directives have been replaced.
=head2 PMC File Syntax
The basic syntax of a PMC file is
=over 4
=item 1.
A preamble, consisting of code to be copied directly to the .c file
=item 2.
The C<pmclass> declaration:
pmclass PMCNAME [flags] {
where C<flags> are:
=over 4
=item C<extends PMCPARENT>
All methods not defined in PMCNAME are inherited from the PMCPARENT class.
If no parent class is defined, methods from F<default.pmc> are used.
=item C<abstract>
This class cannot be instantiated. Abstract classes are shown with lower
case class names in the class tree.
=item C<noinit>
Used with C<abstract>: No C<class_init> code is generated.
=item C<const_too>
Classes with this flag get 2 vtables and 2 enums, one pair with
read/write set methods, and one with read-only set methods.
=item C<need_ext>
The class needs a C<PMC_EXT> structure. For instance, any class using
C<PMC_data> will have C<need_ext>.
=item C<does interface>
The class 'does' the given interfaces (the collection of methods
which the class implements).
The default is "scalar". Other currently used interfaces are:
array : container PMC with numerically-keyed elements
event : PMC that can be used with event queue
hash : container PMC with string-keyed elements
library : PMC that corresponds to a dynamic library
ref : PMC that references another PMC
string : PMC that behaves similarly to the base string type
boolean : PMC that does true/false only.
integer : PMC that behaves similarly to the base int type
float : PMC that behaves similarly to the base number type
scalar : (only used by the sample src/dynpmc/foo.pmc)
This is not a canonical list, but merely a snapshot of what's in use.
=item C<dynpmc>
The class is a dynamic class. These have a special C<class_init>
routine suitable for dynamic loading at runtime. See the F<src/dynpmc>
directory for an example.
=item C<group GROUP>
The class is part of a group of interrelated PMCs that should be
compiled together into a single shared library of the given name. Only
valid for dynamic PMCs.
=item C<lib LIB>
The class needs an external library.
=item C<hll HLL>
The High level language this PMC corresponds to.
=item C<maps Type>
The basic parrot PMC type that this PMC correspond to for C<.HLL>
usage. For example:
pmcclass TclInt hll Tcl maps Integer
allows this PMC to automatically be used when autoboxing C<I> registers to PMCs.
Requires the C<hll> flag.
=back
=item 3.
A list of vtable method implementations
=item 4.
The final close C<}>
=back
=head2 Method Body Substitutions
The vtable method bodies can use the following substitutions:
=over 4
=item C<SELF>
Converted to the current PMC object of type C<PMC *>.
=item C<INTERP>
Converted to the interpreter object.
=item C<OtherClass.SELF.method(a,b,c)>
Calls the static vtable method 'method' in C<OtherClass>.
=item C<SELF.method(a,b,c)>
Calls the vtable method 'method' using the static type of C<SELF> (in
other words, calls another method defined in the same file).
=item C<DYNSELF.method(a,b,c)>
Calls the vtable method 'method' using the dynamic type of C<SELF>.
=item C<DYNSELF(a,b,c)>
Same as above, but calls the current method.
=item C<OtherClass.SUPER(a,b,c)>
Calls the overridden implementation of the current method in
C<OtherClass>.
=item C<SUPER(a,b,c)>
Calls the overridden implementation of the current method in the nearest
superclass, using the static type of C<SELF>.
=item C<DYNSUPER(a,b,c)>
As above, but uses the actual dynamic type of C<SELF>.
=back
=head1 AUTHOR
Leopold Toetsch.
Cleaned up by Matt Diephouse.
Many thanks to the author of F<pmc2c.pl>, many useful code pieces got
reused.
=cut
__END__
# Local Variables:
# mode: cperl
# cperl-indent-level: 4
# fill-column: 100
# End:
# vim: expandtab shiftwidth=4:
# %opt = map { $_ => 0 } qw(nobody nolines debug verbose);
