This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU M4 source repository".
http://git.sv.gnu.org/gitweb/?p=m4.git;a=commitdiff;h=ca0ae275710ba45df36fd076b72605345293b54b The branch, branch-1_4 has been updated via ca0ae275710ba45df36fd076b72605345293b54b (commit) from c011c3f82cd89ca40907ace48f8f59215ea600cd (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ca0ae275710ba45df36fd076b72605345293b54b Author: Eric Blake <[EMAIL PROTECTED]> Date: Mon Dec 3 11:53:45 2007 -0700 Stage 20: make m4wrap obey POSIX fifo ordering. * src/m4.h (enum token_chain_type): Add CHAIN_LOC. (struct token_chain): Add u_l member. (wrap_args): New prototype. * src/input.c (push_wrapup_init, push_wrapup_finish): Rewrite to guarantee a FIFO chain in the wrapup stack. (pop_input, peek_input, next_char_1): Support location link. (next_char): Add parameter. (init_macro_token, init_argv_token): Require user to consume empty input. (skip_line, match_input): Adjust callers. (next_token): Always consume first character. * src/macro.c (arg_text): Tighten assertion. (wrap_args): New method. * src/builtin.c (m4_m4wrap): Use it. (define_macro): Issue warning when ignoring builtin token during macro definition. * doc/m4.texinfo (M4wrap, Location, Incompatibilities) (Improved m4wrap): Adjust examples to corrected behavior. * NEWS: Document this fix. (cherry picked from commit f7f45337fa1bfba9512841e8d3d2251359944681) Signed-off-by: Eric Blake <[EMAIL PROTECTED]> ----------------------------------------------------------------------- Summary of changes: ChangeLog | 28 ++++++++++++ NEWS | 4 ++ doc/m4.texinfo | 49 ++++++++++++++-------- src/builtin.c | 26 ++++++----- src/input.c | 126 +++++++++++++++++++++++++++++++++++-------------------- src/m4.h | 10 ++++- src/macro.c | 68 ++++++++++++++++++++++++++++-- 7 files changed, 230 insertions(+), 81 deletions(-) diff --git a/ChangeLog b/ChangeLog index 3c4e2ad..49f9b80 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,31 @@ +2008-03-19 Eric Blake <[EMAIL PROTECTED]> + + Stage 20: make m4wrap obey POSIX fifo ordering. + Improve input engine to support location changes within symbol + chains, then convert m4wrap to always build symbol chain. Also, + avoid wasted peek at start of next_token, for fewer ungetc calls. + Memory impact: none. + Speed impact: noticeable improvement, from fewer function calls. + * src/m4.h (enum token_chain_type): Add CHAIN_LOC. + (struct token_chain): Add u_l member. + (wrap_args): New prototype. + * src/input.c (push_wrapup_init, push_wrapup_finish): Rewrite to + guarantee a FIFO chain in the wrapup stack. + (pop_input, peek_input, next_char_1): Support location link. + (next_char): Add parameter. + (init_macro_token, init_argv_token): Require user to consume empty + input. + (skip_line, match_input): Adjust callers. + (next_token): Always consume first character. + * src/macro.c (arg_text): Tighten assertion. + (wrap_args): New method. + * src/builtin.c (m4_m4wrap): Use it. + (define_macro): Issue warning when ignoring builtin token during + macro definition. + * doc/m4.texinfo (M4wrap, Location, Incompatibilities) + (Improved m4wrap): Adjust examples to corrected behavior. + * NEWS: Document this fix. + 2008-03-17 Eric Blake <[EMAIL PROTECTED]> Update for fresh bootstrap. diff --git a/NEWS b/NEWS index 7dc3aba..53f7282 100644 --- a/NEWS +++ b/NEWS @@ -8,6 +8,10 @@ Foundation, Inc. ** Fix regression introduced in 1.4.10b where using `builtin' or `indir' to perform nested `shift' calls triggered an assertion failure. +** Fix the `m4wrap' builtin to accumulate wrapped text in FIFO order, as + required by POSIX. The manual mentions a way to restore the LIFO order + present in earlier GNU M4 versions. + ** Enhance the `ifdef', `ifelse', and `shift' builtins, as well as all user macros, to transparently handle builtin tokens generated by `defn'. diff --git a/doc/m4.texinfo b/doc/m4.texinfo index f0fbb96..67f1765 100644 --- a/doc/m4.texinfo +++ b/doc/m4.texinfo @@ -4582,6 +4582,9 @@ Stores @var{string} in a safe place, to be reread when end of input is reached. As a @acronym{GNU} extension, additional arguments are concatenated with a space to the @var{string}. +Successive invocations of @code{m4wrap} accumulate saved text in +first-in, first-out order, as required by @acronym{POSIX}. + The expansion of @code{m4wrap} is void. The macro @code{m4wrap} is recognized only with parameters. @end deffn @@ -4601,18 +4604,27 @@ This is the first and last normal input line. The saved input is only reread when the end of normal input is seen, and not if @code{m4exit} is used to exit @code{m4}. [EMAIL PROTECTED] FIXME: this contradicts POSIX, which requires that "If the [EMAIL PROTECTED] m4wrap macro is used multiple times, the arguments specified [EMAIL PROTECTED] shall be processed in the order in which the m4wrap macros were [EMAIL PROTECTED] processed." -It is safe to call @code{m4wrap} from saved text, but then the order in -which the saved text is reread is undefined. If @code{m4wrap} is not used -recursively, the saved pieces of text are reread in the opposite order -in which they were saved (LIFO---last in, first out). However, this -behavior is likely to change in a future release, to match [EMAIL PROTECTED], so you should not depend on this order. - -It is possible to emulate @acronym{POSIX} behavior even +It is safe to call @code{m4wrap} from wrapped text, where all the +recursively wrapped text is deferred until the current wrapped text is +exhausted. As of M4 1.4.11, when @code{m4wrap} is not used recursively, +the saved pieces of text are reread in the same order in which they were +saved (FIFO---first in, first out), as required by @acronym{POSIX}. + [EMAIL PROTECTED] +m4wrap(`1 +') [EMAIL PROTECTED] +m4wrap(`2', `3 +') [EMAIL PROTECTED] +^D [EMAIL PROTECTED] [EMAIL PROTECTED] 3 [EMAIL PROTECTED] example + +However, earlier versions had reverse ordering (LIFO---last in, first +out), as this behavior is more like the semantics of the C function [EMAIL PROTECTED] It is possible to emulate @acronym{POSIX} behavior even with older versions of @acronym{GNU} M4 by including the file @[EMAIL PROTECTED]/@/examples/@/wrapfifo.m4} from the distribution: @@ -4688,13 +4700,13 @@ Invocations of @code{m4wrap} at the same recursion level are concatenated and rescanned as usual: @example -define(`aa', `AA +define(`ab', `AB ') @result{} -m4wrap(`a')m4wrap(`a') +m4wrap(`a')m4wrap(`b') @result{} ^D [EMAIL PROTECTED] [EMAIL PROTECTED] @end example @noindent @@ -6613,9 +6625,9 @@ m4wrap(`__line__ ') @result{} ^D [EMAIL PROTECTED] @result{}6 @result{}6 [EMAIL PROTECTED] @end example The @[EMAIL PROTECTED] macro behaves like @samp{$0} in shell @@ -7057,7 +7069,8 @@ of @samp{-} on the command line. @item @acronym{POSIX} requires @code{m4wrap} (@pxref{M4wrap}) to act in FIFO -(first-in, first-out) order, but @acronym{GNU} @code{m4} currently uses +(first-in, first-out) order, and most other implementations obey this. +However, versions of @acronym{GNU} @code{m4} earlier than 1.4.11 used LIFO order. Furthermore, @acronym{POSIX} states that only the first argument to @code{m4wrap} is saved for later evaluation, but @acronym{GNU} @code{m4} saves and processes all arguments, with output @@ -7745,8 +7758,8 @@ builtin(`m4wrap', ``'define(`bar', ``$0:'-$1-$*-$#-')bar(`a', `b') ') @result{} ^D [EMAIL PROTECTED]:-a-a,b-2- @result{}m4wrap0:---0- [EMAIL PROTECTED]:-a-a,b-2- @end example Additionally, the computation of @code{_m4wrap_level} and creation of diff --git a/src/builtin.c b/src/builtin.c index a441c4c..b5541cf 100644 --- a/src/builtin.c +++ b/src/builtin.c @@ -667,7 +667,14 @@ define_macro (int argc, macro_arguments *argv, symbol_lookup mode) switch (arg_type (argv, 2)) { + case TOKEN_COMP: + m4_warn (0, me, _("cannot concatenate builtins")); + /* TODO fall through instead. */ + break; + case TOKEN_TEXT: + /* TODO flatten TOKEN_COMP value, or support concatenation of + builtins in definitions. */ define_user_macro (ARG (1), ARG_LEN (1), ARG (2), mode); break; @@ -1608,25 +1615,20 @@ m4_m4exit (struct obstack *obs, int argc, macro_arguments *argv) exit (exit_code); } -/*-------------------------------------------------------------------------. -| Save the argument text until EOF has been seen, allowing for user | -| specified cleanup action. GNU version saves all arguments, the standard | -| version only the first. | -`-------------------------------------------------------------------------*/ +/*-----------------------------------------------------------------. +| Save the argument text in FIFO order until EOF has been seen, | +| allowing for user specified cleanup action. Extra arguments are | +| saved when not in POSIX mode. | +`-----------------------------------------------------------------*/ static void m4_m4wrap (struct obstack *obs, int argc, macro_arguments *argv) { if (bad_argc (ARG (0), argc, 1, -1)) return; - obs = push_wrapup_init (); - if (no_gnu_extensions) - obstack_grow (obs, ARG (1), ARG_LEN (1)); - else - /* TODO - allow builtins, rather than always flattening. */ - arg_print (obs, argv, 1, NULL, true, " ", NULL, false); - push_wrapup_finish (); + wrap_args (argv); } + /* Enable tracing of all specified macros, or all, if none is specified. Tracing is disabled by default, when a macro is defined. This can be diff --git a/src/input.c b/src/input.c index b8784d0..86db704 100644 --- a/src/input.c +++ b/src/input.c @@ -529,12 +529,36 @@ struct obstack * push_wrapup_init (void) { input_block *i; - i = (input_block *) obstack_alloc (wrapup_stack, sizeof *i); - i->prev = wsp; - i->type = INPUT_STRING; - i->file = current_file; - i->line = current_line; - wsp = i; + token_chain *chain; + + assert (obstack_object_size (wrapup_stack) == 0); + if (wsp) + { + i = wsp; + assert (i->type == INPUT_CHAIN && i->u.u_c.end + && i->u.u_c.end->type != CHAIN_LOC); + } + else + { + i = (input_block *) obstack_alloc (wrapup_stack, sizeof *i); + i->prev = wsp; + i->file = current_file; + i->line = current_line; + i->type = INPUT_CHAIN; + i->u.u_c.chain = i->u.u_c.end = NULL; + wsp = i; + } + chain = (token_chain *) obstack_alloc (wrapup_stack, sizeof *chain); + if (i->u.u_c.end) + i->u.u_c.end->next = chain; + else + i->u.u_c.chain = chain; + i->u.u_c.end = chain; + chain->next = NULL; + chain->type = CHAIN_LOC; + chain->quote_age = 0; + chain->u.u_l.file = current_file; + chain->u.u_l.line = current_line; return wrapup_stack; } @@ -545,17 +569,7 @@ push_wrapup_init (void) void push_wrapup_finish (void) { - input_block *i = wsp; - if (obstack_object_size (wrapup_stack) == 0) - { - wsp = i->prev; - obstack_free (wrapup_stack, i); - } - else - { - i->u.u_s.len = obstack_object_size (wrapup_stack); - i->u.u_s.str = (char *) obstack_finish (wrapup_stack); - } + make_text_link (wrapup_stack, &wsp->u.u_c.chain, &wsp->u.u_c.end); } @@ -610,6 +624,8 @@ pop_input (bool cleanup) return false; arg_adjust_refcount (chain->u.u_a.argv, false); break; + case CHAIN_LOC: + return false; default: assert (!"pop_input"); abort (); @@ -837,6 +853,8 @@ peek_input (bool allow_argv) chain->u.u_a.comma = true; push_string_finish (); return peek_input (allow_argv); + case CHAIN_LOC: + break; default: assert (!"peek_input"); abort (); @@ -863,16 +881,18 @@ peek_input (bool allow_argv) | string, so factor that out into a macro for speed. If | | ALLOW_QUOTE, and the current input matches the current quote age, | | return CHAR_QUOTE and leave consumption of data for | -| append_quote_token. | +| append_quote_token; otherwise, if ALLOW_ARGV and the current input | +| matches an argv reference with the correct quoting, return | +| CHAR_ARGV and leave consuption of data for init_argv_token. | `-------------------------------------------------------------------*/ -#define next_char(AQ) \ +#define next_char(AQ, AA) \ (isp && isp->type == INPUT_STRING && isp->u.u_s.len && !input_change \ ? (isp->u.u_s.len--, to_uchar (*isp->u.u_s.str++)) \ - : next_char_1 (AQ)) + : next_char_1 (AQ, AA)) static int -next_char_1 (bool allow_quote) +next_char_1 (bool allow_quote, bool allow_argv) { int ch; token_chain *chain; @@ -929,6 +949,7 @@ next_char_1 (bool allow_quote) chain = isp->u.u_c.chain; while (chain) { + unsigned int argc; if (allow_quote && chain->quote_age == current_quote_age) return CHAR_QUOTE; switch (chain->type) @@ -949,7 +970,8 @@ next_char_1 (bool allow_quote) return CHAR_MACRO; break; case CHAIN_ARGV: - if (chain->u.u_a.index == arg_argc (chain->u.u_a.argv)) + argc = arg_argc (chain->u.u_a.argv); + if (chain->u.u_a.index == argc) { arg_adjust_refcount (chain->u.u_a.argv, false); break; @@ -959,6 +981,12 @@ next_char_1 (bool allow_quote) chain->u.u_a.comma = false; return ','; } + /* Only return a reference if the quoting is correct + and the reference has more than one argument + left. */ + if (allow_argv && chain->quote_age == current_quote_age + && chain->u.u_a.quotes && chain->u.u_a.index + 1 < argc) + return CHAR_ARGV; /* Rather than directly parse argv here, we push another input block containing the next unparsed argument from argv. */ @@ -970,7 +998,13 @@ next_char_1 (bool allow_quote) chain->u.u_a.index++; chain->u.u_a.comma = true; push_string_finish (); - return next_char_1 (allow_quote); + return next_char_1 (allow_quote, allow_argv); + case CHAIN_LOC: + isp->file = chain->u.u_l.file; + isp->line = chain->u.u_l.line; + input_change = true; + isp->u.u_c.chain = chain->next; + return next_char_1 (allow_quote, allow_argv); default: assert (!"next_char_1"); abort (); @@ -1002,7 +1036,7 @@ skip_line (const char *name) const char *file = current_file; int line = current_line; - while ((ch = next_char (false)) != CHAR_EOF && ch != '\n') + while ((ch = next_char (false, false)) != CHAR_EOF && ch != '\n') ; if (ch == CHAR_EOF) /* current_file changed to "" if we see CHAR_EOF, use the @@ -1028,25 +1062,28 @@ skip_line (const char *name) static void init_macro_token (token_data *td) { - int ch = next_char (false); - assert (ch == CHAR_MACRO); - if (td) - TOKEN_DATA_TYPE (td) = TOKEN_FUNC; + token_chain *chain; + if (isp->type == INPUT_MACRO) { assert (isp->u.func); if (td) - TOKEN_DATA_FUNC (td) = isp->u.func; + { + TOKEN_DATA_TYPE (td) = TOKEN_FUNC; + TOKEN_DATA_FUNC (td) = isp->u.func; + } isp->u.func = NULL; } else { - token_chain *chain; assert (isp->type == INPUT_CHAIN); chain = isp->u.u_c.chain; assert (!chain->quote_age && chain->type == CHAIN_FUNC && chain->u.func); if (td) - TOKEN_DATA_FUNC (td) = chain->u.func; + { + TOKEN_DATA_TYPE (td) = TOKEN_FUNC; + TOKEN_DATA_FUNC (td) = chain->u.func; + } chain->u.func = NULL; } } @@ -1108,9 +1145,9 @@ init_argv_token (struct obstack *obs, token_data *td) { token_chain *src_chain; token_chain *chain; - int ch = next_char (true); + int ch; - assert (ch == CHAR_QUOTE && TOKEN_DATA_TYPE (td) == TOKEN_VOID + assert (TOKEN_DATA_TYPE (td) == TOKEN_VOID && isp->type == INPUT_CHAIN && isp->u.u_c.chain->type == CHAIN_ARGV && obs && obstack_object_size (obs) == 0); @@ -1146,7 +1183,7 @@ init_argv_token (struct obstack *obs, token_data *td) decreased once the final element is parsed. */ assert (*curr_comm.str1 != ',' && *curr_comm.str1 != ')' && *curr_comm.str1 != *curr_quote.str1); - ch = peek_input (false); + ch = peek_input (true); if (ch != ',' && ch != ')') { isp->u.u_c.chain = src_chain; @@ -1181,14 +1218,14 @@ match_input (const char *s, bool consume) if (s[1] == '\0') { if (consume) - next_char (false); + next_char (false, false); return true; /* short match */ } - next_char (false); + next_char (false, false); for (n = 1, t = s++; (ch = peek_input (false)) == to_uchar (*s++); ) { - next_char (false); + next_char (false, false); n++; if (*s == '\0') /* long match */ { @@ -1556,15 +1593,13 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, if (!line) line = &dummy; - /* Can't consume character until after CHAR_MACRO is handled. */ TOKEN_DATA_TYPE (td) = TOKEN_VOID; - ch = peek_input (allow_argv && current_quote_age); + ch = next_char (false, allow_argv && current_quote_age); if (ch == CHAR_EOF) { #ifdef DEBUG_INPUT xfprintf (stderr, "next_token -> EOF\n"); #endif /* DEBUG_INPUT */ - next_char (false); return TOKEN_EOF; } if (ch == CHAR_MACRO) @@ -1588,7 +1623,6 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, return TOKEN_ARGV; } - next_char (false); /* Consume character we already peeked at. */ file = current_file; *line = current_line; if (MATCH (ch, curr_comm.str1, true)) @@ -1598,7 +1632,7 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, obstack_grow (obs_td, curr_comm.str1, curr_comm.len1); while (1) { - ch = next_char (false); + ch = next_char (false, false); if (ch == CHAR_EOF) /* Current_file changed to "" if we see CHAR_EOF, use the previous value we stored earlier. */ @@ -1629,7 +1663,7 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, && (isalnum (ch) || ch == '_')) { obstack_1grow (&token_stack, ch); - next_char (false); + next_char (false, false); } type = TOKEN_WORD; } @@ -1652,7 +1686,7 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, obstack_blank (&token_stack, -1); break; } - next_char (false); + next_char (false, false); } obstack_1grow (&token_stack, '\0'); @@ -1697,7 +1731,7 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, type = TOKEN_STRING; while (1) { - ch = next_char (obs != NULL && current_quote_age); + ch = next_char (obs != NULL && current_quote_age, false); if (ch == CHAR_EOF) /* Current_file changed to "" if we see CHAR_EOF, use the previous value we stored earlier. */ @@ -1721,7 +1755,7 @@ next_token (token_data *td, int *line, struct obstack *obs, bool allow_argv, xfprintf (stderr, "next_token -> MACDEF (%s)\n", bp->name); #endif - ch = next_char (false); + ch = next_char (false, false); MATCH (ch, curr_quote.str2, true); return TOKEN_MACDEF; } diff --git a/src/m4.h b/src/m4.h index ef45359..3e7fc76 100644 --- a/src/m4.h +++ b/src/m4.h @@ -283,7 +283,8 @@ enum token_chain_type { CHAIN_STR, /* Link contains a string, u.u_s is valid. */ CHAIN_FUNC, /* Builtin function definition, u.func is valid. */ - CHAIN_ARGV /* Link contains a $@ reference, u.u_a is valid. */ + CHAIN_ARGV, /* Link contains a $@ reference, u.u_a is valid. */ + CHAIN_LOC /* Link contains location of m4wrap, u.u_l is valid. */ }; /* Composite tokens are built of a linked list of chains. Each link @@ -315,6 +316,12 @@ struct token_chain const string_pair *quotes; /* NULL for $*, quotes for [EMAIL PROTECTED] */ } u_a; + struct + { + const char *file; /* File where subsequent links originate. */ + int line; /* Line where subsequent links originate. */ + } + u_l; } u; }; @@ -508,6 +515,7 @@ void push_arg (struct obstack *, macro_arguments *, unsigned int); void push_arg_quote (struct obstack *, macro_arguments *, unsigned int, const string_pair *); void push_args (struct obstack *, macro_arguments *, bool, bool); +void wrap_args (macro_arguments *); /* Grab the text at argv index I. Assumes macro_argument *argv is in scope, and aborts if the argument is not text. */ diff --git a/src/macro.c b/src/macro.c index f794d86..6a6a90c 100644 --- a/src/macro.c +++ b/src/macro.c @@ -946,11 +946,8 @@ arg_text (macro_arguments *argv, unsigned int index) case CHAIN_STR: obstack_grow (obs, chain->u.u_s.str, chain->u.u_s.len); break; - case CHAIN_FUNC: - /* TODO concatenate builtins. */ - assert (!"implemented"); - abort (); case CHAIN_ARGV: + assert (!chain->u.u_a.has_func || argv->flatten); arg_print (obs, chain->u.u_a.argv, chain->u.u_a.index, quote_cache (NULL, chain->quote_age, chain->u.u_a.quotes), @@ -1515,3 +1512,66 @@ push_args (struct obstack *obs, macro_arguments *argv, bool skip, bool quote) if (push_token (token, -1, argv->inuse)) arg_mark (argv); } + +/* Push arguments from ARGV, which can include builtins, onto the wrap + stack for later rescanning. If GNU extensions are disabled, only + the first argument is pushed; otherwise, all arguments are pushed + and separated with a space. */ +void +wrap_args (macro_arguments *argv) +{ + int i; + struct obstack *obs; + token_data *token; + token_chain *chain; + + if ((argv->argc == 2 || no_gnu_extensions) && arg_empty (argv, 1)) + return; + + obs = push_wrapup_init (); + for (i = 1; i < (no_gnu_extensions ? 2 : argv->argc); i++) + { + if (i != 1) + obstack_1grow (obs, ' '); + token = arg_token (argv, i, NULL, false); + switch (TOKEN_DATA_TYPE (token)) + { + case TOKEN_TEXT: + obstack_grow (obs, TOKEN_DATA_TEXT (token), TOKEN_DATA_LEN (token)); + break; + case TOKEN_FUNC: + /* TODO allow builtins through m4wrap. */ + assert (false); + case TOKEN_COMP: + chain = token->u.u_c.chain; + while (chain) + { + switch (chain->type) + { + case CHAIN_STR: + obstack_grow (obs, chain->u.u_s.str, chain->u.u_s.len); + break; + case CHAIN_FUNC: + /* TODO allow builtins through m4wrap. */ + assert (false); + break; + case CHAIN_ARGV: + arg_print (obs, chain->u.u_a.argv, chain->u.u_a.index, + quote_cache (NULL, chain->quote_age, + chain->u.u_a.quotes), + chain->u.u_a.flatten, NULL, NULL, false); + break; + default: + assert (!"wrap_args"); + abort (); + } + chain = chain->next; + } + break; + default: + assert (!"wrap_args"); + abort (); + } + } + push_wrapup_finish (); +} hooks/post-receive -- GNU M4 source repository
