Be afraid. Be very afraid.
I have now committed the following change to bleedperl (#22302).
It moves Perl from using byacc to bison, and simplifies a lot of things
along the way. It may well have broken VMS and EBCDIC platforms, since
I'm not in a position to be able to test these.
Here's how it has changed:
old behaviour:
when perly.y was edited, the Pumpking would run
make run_byacc
which would generate perly.h and perly.h, then apply a barrage
of shell, sed and perl scripts plus a patch, to fix them up, and
to build VMS-specific versions of them in vms/. The Pumpking would
then have to regenerate the patch, which almost certainly wouldn't
apply cleanly on the next fix to perly.y.
When Perl was installed, most systems would then use the perly.[ch]
as is; on VMS systems the versions in the vms/ subdir would be used to
overwrite the versions in the top-level directory. On EBCDIC systems,
the Makefile.SH script would attempt to run yacc or bison plus
sed-type stuff to build EBCDIC-friendly versions of perly.[ch],
ignoring the ones bundled with Perl.
new behaviour:
when perly.y is edited, the Pumpking should run
make regen_perly
which calls the perl script regen_perly.pl, which runs bison over
perly.y then extracts stuff from its output to create perly.h,
perly.act and perly.tab. Note that perly.c does *not* get changed;
this is now a static file. It #includes perly.tab and perly.act which
contain the perly.y-dependent tables and actions.
When Perl is installed, all platforms should just the perly.* files
as is: no fixups, no clever tricks, no nuthink. EBCDIC systems
should be able to use the perly.* file pre-generated on an ASCII
system because values returned by yylex are now fed through
NATIVE_TO_ASCII() before being passed onto yyparse,
Some notes:
Pumpkings: Currently it requires bison 1.875 to rebuild perly.*, although
this restriction has more to do with a lack of testing on my part of other
versions, rather than any intrinsic defect. Probably 1.35 onwards will
work.
I was intending to use the clever --no-parser option of bison to suck out
the dependent bits of perly.c into seaparate files; however it turns
out that on newer versions of bison this option is just silently ignored!!
An email to bugs-bison elicited zero response.
Now that perly.c isn't auto-generated, future hacks to it should be a lot
easier.
I've included selected highlights from the change below; basically I've
stripped out all the perly.* changes since they're enormous.
Dave.
--
To collect all the latest movies, simply place an unprotected ftp server
on the Internet, and wait for the disk to fill....
Change 22302 by [EMAIL PROTECTED] on 2004/02/14 02:06:53
Switch from byacc to bison and simplify the perly.* regeneration
process
Affected files ...
... //depot/perl/MANIFEST#1142 edit
... //depot/perl/Makefile.SH#296 edit
... //depot/perl/Porting/pumpkin.pod#55 edit
... //depot/perl/embedvar.h#183 edit
... //depot/perl/intrpvar.h#137 edit
... //depot/perl/op.c#603 edit
... //depot/perl/perl.h#550 edit
... //depot/perl/perlapi.h#105 edit
... //depot/perl/perly.act#1 add
... //depot/perl/perly.c#60 edit
... //depot/perl/perly.fixer#15 delete
... //depot/perl/perly.h#15 edit
... //depot/perl/perly.tab#1 add
... //depot/perl/perly.y#53 edit
... //depot/perl/perly_c.diff#24 delete
... //depot/perl/perlyline.pl#2 delete
... //depot/perl/regen_perly.pl#1 add
... //depot/perl/sv.c#718 edit
... //depot/perl/toke.c#493 edit
... //depot/perl/vms/descrip_mms.template#74 edit
... //depot/perl/vms/perly_c.vms#52 delete
... //depot/perl/vms/perly_h.vms#14 delete
... //depot/perl/vms/vms_yfix.pl#10 delete
Differences ...
==== //depot/perl/Makefile.SH#296 (text) ====
@@ -154,15 +154,6 @@
nonxs_list="$nonxs_list ext/$f/pm_to_blib"
done
-# Handle the usage of different yaccs in posix-bc (During Configure we
-# us yacc for perly.y and byacc for a2p.y. The makefiles must use the
-# same configuration for run_byacc!):
-case "$osname" in
- posix-bc)
- byacc=$yacc
- ;;
-esac
-
echo "Extracting Makefile (with variable substitutions)"
$spitshell >Makefile <<!GROK!THIS!
# Makefile.SH
@@ -170,10 +161,7 @@
# be lost the next time you run Configure.
# Makefile is used to generate $firstmakefile. The only difference
# is that $firstmakefile has the dependencies filled in at the end.
-#
-#
-# I now supply perly.c with the kits, so don't remake perly.c without byacc
-BYACC = $byacc
+
CC = $cc
LD = $ld
@@ -826,28 +814,18 @@
# I now supply perly.c with the kits, so the following section is
-# used only if you force byacc to run by saying
-# make run_byacc
-# Since we patch up the byacc output, the perly.fixer script needs
-# to run with precisely the same version of byacc as I use. You
-# normally shouldn't remake perly.[ch].
+# used only if you force bison to run by saying
+# make regen_perly
+# You normally shouldn't remake perly.[ch].
-.PHONY: check_byacc run_byacc
+.PHONY: regen_perly
-check_byacc:
- @$(BYACC) -V 2>&1 | grep 'version 1\.8\.2'
+run_byacc:
+ @echo "run_byacc is obsolete; try 'make regen_perly' instead"
-run_byacc: FORCE check_byacc
- $(BYACC) -d perly.y
- -chmod 664 perly.c perly.h
- sh $(shellflags) ./perly.fixer y.tab.c perly.c
- sed -e 's/fprintf *( *stderr *,/PerlIO_printf(Perl_debug_log,/g' \
- -e 's/y\.tab/perly/g' perly.c >perly.tmp && mv perly.tmp perly.c
- sed -e '/^extern YYSTYPE yy/D' y.tab.h >yh.tmp && mv yh.tmp y.tab.h
- cmp -s y.tab.h perly.h && rm -f y.tab.h || mv y.tab.h perly.h
- perl -i.old perlyline.pl perly.c
- chmod 664 vms/perly_c.vms vms/perly_h.vms
- perl vms/vms_yfix.pl perly.c perly.h vms/perly_c.vms vms/perly_h.vms
+# this outputs perly.act and perly.tab
+regen_perly:
+ perl regen_perly.pl
# We don't want to regenerate perly.c and perly.h, but they might
# appear out-of-date after a patch is applied or a new distribution is
@@ -858,11 +836,6 @@
perly.h: perly.y
[EMAIL PROTECTED] -c true
-PERLYVMS = vms/perly_c.vms vms/perly_h.vms
-
-$(PERLYVMS): perly.c perly.h vms/vms_yfix.pl
- perl vms/vms_yfix.pl perly.c perly.h vms/perly_c.vms vms/perly_h.vms
-
# No compat3.sym here since and including the 5.004_50.
# No interp.sym since 5.005_03.
SYM = global.sym globvar.sym perlio.sym pp.sym
@@ -904,7 +877,7 @@
regen_pods: FORCE
-cd pod; $(LDLIBPTH) $(MAKE) regen_pods
-regen_all: $(PERLYVMS) regen regen_pods
+regen_all: regen regen_pods
.PHONY: manisort manicheck
@@ -1279,44 +1252,6 @@
echo "This is an EBCDIC system, checking if any parser files need regenerating."
>&2
case "$osname" in
os390|posix-bc)
- rm -f y.tab.c y.tab.h
- # yacc must be a reentrant ("pure") Bison in BS2000 Posix!
- yacc -d perly.y >/dev/null 2>&1
- if cmp -s y.tab.c perly.c; then
- rm -f y.tab.c
- else
- echo "perly.y -> perly.c" >&2
- mv -f y.tab.c perly.c
- chmod u+w perly.c
- sed -e '/^#include "perl\.h"/a\
-\
-#define yydebug PL_yydebug\
-#define yynerrs PL_yynerrs\
-#define yyerrflag PL_yyerrflag\
-#define yychar PL_yychar\
-#define yyval PL_yyval\
-#define yylval PL_yylval' \
- -e '/YYSTYPE *yyval;/D' \
- -e '/YYSTYPE *yylval;/D' \
- -e '/int yychar,/,/yynerrs;/D' \
- -e 's/int yydebug = 0;/yydebug = 0;/' \
- -e 's/[^_]realloc(/PerlMem_realloc(/g' \
- -e 's/fprintf *( *stderr *,/PerlIO_printf(Perl_debug_log,/g' \
- -e 's/y\.tab/perly/g' perly.c >perly.tmp && mv perly.tmp perly.c
- xxx="$xxx perly.c"
- fi
- case "$osname:$usethreads" in
- os390:define)
- sed -e '[EMAIL PROTECTED] int yychar, yyerrflag;@/* extern int yychar,
yyerrflag; */@' perly.c > perly.tmp && mv perly.tmp perly.c
- ;;
- esac
- if cmp -s y.tab.h perly.h; then
- rm -f y.tab.h
- else
- echo "perly.y -> perly.h" >&2
- mv -f y.tab.h perly.h
- xxx="$xxx perly.h"
- fi
if cd x2p
then
rm -f y.tab.c y.tab.h
==== //depot/perl/Porting/pumpkin.pod#55 (text) ====
@@ -414,7 +414,6 @@
keywords.pl
myconfig
opcode.pl
- perly.fixer
t/TEST
t/*/*.t
*.SH
@@ -478,51 +477,27 @@
patch with a promise to quickly issue a follow-up that handles those
directories.
-=head2 make run_byacc
+=head2 make regen_perly
-If you have byacc-1.8.2 (available from CPAN as
-http://www.cpan.org/src/misc/perl-byacc1.8.2.tar.gz), and if there have
-been changes to F<perly.y>, you can regenerate the F<perly.c> file. The
-run_byacc makefile target does this by running byacc and then applying
-some patches so that byacc dynamically allocates space, rather than
-having fixed limits. This patch is handled by the F<perly.fixer>
-script. Depending on the nature of the changes to F<perly.y>, you may
-or may not have to hand-edit the patch to apply correctly. If you do,
-you should include the edited patch in the new distribution. (If you
-have byacc-1.9, the patch won't apply cleanly, notably changes to the printf
-output statements. F<perly.fixer> could be fixed to detect this.)
+If perly.y has been edited, it is nessary to run this target to rebuild
+perly.h, perl.act and perl.tab. In fact this target just runs the Perl
+script regen_perly.pl. Note that perl.c is I<not> rebuilt; this is just a
+plain static file now.
-If C<perly.c> or C<perly.h> changes, make sure you run C<perl vms/vms_yfix.pl>
-to update the corresponding VMS files. The run_byacc target in the Unix
-Makefile takes care of this. See also L<VMS-specific updates>.
+This target relies on you having Bison installed on your system. Running
+the target will tell you if you haven't got the right version, and if so,
+where to get the right one. Or if you prefer, you could hack
+regen_perly.pl to work with your version of Bison. The important things
+are that the regexes can still extract out the right chunks of the Bison
+output into perly.act and perl.tab, and that the contents of those two
+files, plus perly.h, are functionally equivalent to those produced by the
+supported version of Bison.
-Some additional notes from Larry on this:
-
-Don't forget to regenerate perly_c.diff.
-
- byacc -d perly.y
- mv y.tab.c perly.c
- patch perly.c <perly_c.diff
- # manually apply any failed hunks
- diff -u perly.c.orig perly.c >perly_c.diff
-
-One chunk of lines that often fails begins with
-
- #line 29 "perly.y"
-
-and ends one line before
+Note that in the old days, you had to do C<make run_byacc> instead.
- #define YYERRCODE 256
-
-This only happens when you add or remove a token type. I suppose this
-could be automated, but it doesn't happen very often nowadays.
-
-Larry
-
=head2 make regen_all
-This target takes care of the PERLYVMS, regen_headers, and regen_pods
-targets.
+This target takes care of the regen_headers, and regen_pods targets.
=head2 make regen_headers
@@ -630,10 +605,6 @@
=head2 VMS-specific updates
-If you have changed F<perly.y> or F<perly.c>, then you most probably want
-to update F<vms/perly_{h,c}.vms> by running C<perl vms/vms_yfix.pl>, or
-by running `make regen_all` which will run that script for you.
-
The Perl revision number appears as "perl5" in configure.com.
It is courteous to update that if necessary.
==== //depot/perl/intrpvar.h#137 (text) ====
@@ -397,13 +397,14 @@
PERLVAR(Ilast_swash_tmps, U8 *)
PERLVAR(Ilast_swash_slen, STRLEN)
-/* perly.c globals */
-PERLVAR(Iyydebug, int)
-PERLVAR(Iyynerrs, int)
-PERLVAR(Iyyerrflag, int)
-PERLVAR(Iyychar, int)
-PERLVAR(Iyyval, YYSTYPE)
-PERLVAR(Iyylval, YYSTYPE)
+/* ex perly.c globals. Now just placeholders for BINCOMPAT in 5.8.x
+ * XXX delete for 5.9.x */
+PERLVAR(IyydebugBINCOMPAT, int)
+PERLVAR(IyynerrsBINCOMPAT, int)
+PERLVAR(IyyerrflagBINCOMPAT, int)
+PERLVAR(IyycharBINCOMPAT, int)
+PERLVAR(IyyvalBINCOMPAT, YYSTYPE)
+PERLVAR(IyylvalBINCOMPAT, YYSTYPE)
PERLVARI(Iglob_index, int, 0)
PERLVAR(Isrand_called, bool)
==== //depot/perl/op.c#603 (text) ====
@@ -1770,7 +1770,10 @@
{
int retval = PL_savestack_ix;
/* If there were syntax errors, don't try to start a block */
- if (PL_yynerrs) return retval;
+ /* XXX DAPM 13-Feb-04. This symbol no longer gloabl. Think of a better
+ * way
+ * if (PL_yynerrs) return retval;
+ */
pad_block_start(full);
SAVEHINTS();
@@ -1794,7 +1797,10 @@
int needblockscope = PL_hints & HINT_BLOCK_SCOPE;
OP* retval = scalarseq(seq);
/* If there were syntax errors, don't try to close a block */
- if (PL_yynerrs) return retval;
+ /* XXX DAPM 13-Feb-04. This symbol no longer gloabl. Think of a better
+ * way
+ * if (PL_yynerrs) return retval;
+ */
LEAVE_SCOPE(floor);
PL_compiling.op_private = (U8)(PL_hints & HINT_PRIVATE_MASK);
if (needblockscope)
==== //depot/perl/perl.h#550 (text) ====
@@ -11,6 +11,12 @@
#ifndef H_PERL
#define H_PERL 1
+/* XXX DAPM tmp - always do this now - probably nedd to remove all trace
+ * of the define at some pooint. Feb 04 */
+
+#define USE_PURE_BISON 1
+
+
#ifdef PERL_FOR_X2P
/*
* This file is being used for x2p stuff.
@@ -2546,9 +2552,6 @@
#ifdef DEBUGGING
-# undef YYDEBUG
-# define YYDEBUG 1
-
# define DEBUG_p_TEST DEBUG_p_TEST_
# define DEBUG_s_TEST DEBUG_s_TEST_
# define DEBUG_l_TEST DEBUG_l_TEST_
@@ -2727,8 +2730,6 @@
#define PERL_MAGIC_ext '~' /* Available for use by extensions */
-#define YYMAXDEPTH 300
-
#ifndef assert /* <assert.h> might have been included somehow */
#ifdef DEBUGGING
#define assert(what) PERL_DEB( { \
==== //depot/perl/perly.y#53 (text) ====
@@ -1,6 +1,6 @@
/* perly.y
*
- * Copyright (c) 1991-2002, Larry Wall
+ * Copyright (c) 1991-2002, 2003, 2004 Larry Wall
*
* You may distribute under the terms of either the GNU General Public
* License or the Artistic License, as specified in the README file.
@@ -12,45 +12,12 @@
* All that is gold does not glitter, not all those who wander are lost.'
*/
-%{
-#include "EXTERN.h"
-#define PERL_IN_PERLY_C
-#include "perl.h"
-#ifdef EBCDIC
-#undef YYDEBUG
-#endif
-#define dep() deprecate("\"do\" to call subroutines")
+/* Make the parser re-entrant. */
-/* stuff included here to make perly_c.diff apply better */
-
-#define yydebug PL_yydebug
-#define yynerrs PL_yynerrs
-#define yyerrflag PL_yyerrflag
-#define yychar PL_yychar
-#define yyval PL_yyval
-#define yylval PL_yylval
-
-struct ysv {
- short* yyss;
- YYSTYPE* yyvs;
- int oldyydebug;
- int oldyynerrs;
- int oldyyerrflag;
- int oldyychar;
- YYSTYPE oldyyval;
- YYSTYPE oldyylval;
-};
+%pure_parser
-static void yydestruct(pTHX_ void *ptr);
-
-%}
-
%start prog
-%{
-#if 0 /* get this from perly.h instead */
-%}
-
%union {
I32 ival;
char *pval;
@@ -58,16 +25,6 @@
GV *gvval;
}
-%{
-#endif /* 0 */
-
-#ifdef USE_PURE_BISON
-#define YYLEX_PARAM (&yychar)
-#define yylex yylex_r
-#endif
-
-%}
-
%token <ival> '{'
%token <opval> WORD METHOD FUNCMETH THING PMFUNC PRIVATEREF
@@ -144,9 +101,6 @@
progstart:
{
-#if defined(YYDEBUG) && defined(DEBUGGING)
- yydebug = (DEBUG_p_TEST);
-#endif
PL_expect = XSTATE; $$ = block_start(TRUE);
}
;
@@ -792,13 +746,3 @@
| PRIVATEREF
{ $$ = $1; }
;
-
-%% /* PROGRAM */
-
-/* more stuff added to make perly_c.diff easier to apply */
-
-#ifdef yyparse
-#undef yyparse
-#endif
-#define yyparse() Perl_yyparse(pTHX)
-
==== //depot/perl/sv.c#718 (text) ====
@@ -11468,14 +11468,6 @@
PL_last_swash_tmps = (U8*)NULL;
PL_last_swash_slen = 0;
- /* perly.c globals */
- PL_yydebug = proto_perl->Iyydebug;
- PL_yynerrs = proto_perl->Iyynerrs;
- PL_yyerrflag = proto_perl->Iyyerrflag;
- PL_yychar = proto_perl->Iyychar;
- PL_yyval = proto_perl->Iyyval;
- PL_yylval = proto_perl->Iyylval;
-
PL_glob_index = proto_perl->Iglob_index;
PL_srand_called = proto_perl->Isrand_called;
PL_hash_seed = proto_perl->Ihash_seed;
==== //depot/perl/toke.c#493 (text) ====
@@ -2189,6 +2189,11 @@
Perl_croak(aTHX_ "panic: YYMAXLEVEL");
r = Perl_yylex(aTHX);
+# ifdef EBCDIC
+ if (r >= 0 && r < 255) {
+ r = NATIVE_TO_ASCII(r);
+ }
+# endif
if (yyactlevel > 0)
yyactlevel--;
==== //depot/perl/vms/descrip_mms.template#74 (text) ====
@@ -1224,12 +1224,6 @@
$(INSTPERL) regcomp.pl
$(INSTPERL) warnings.pl
-# VMS uses modified perly.[ch] with tags for globaldefs if using DEC compiler
-perly.c : [.vms]perly_c.vms
- Copy/NoConfirm/Log $(MMS$SOURCE) $(MMS$TARGET)
-perly.h : [.vms]perly_h.vms
- Copy/NoConfirm/Log $(MMS$SOURCE) $(MMS$TARGET)
-
.ifdef LINK_ONLY
.else
perly$(O) : perly.c, perly.h, $(h)
