[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Zack Weinberg]

Follow-up Comment #7, sr #110403 (project autoconf):

OK, I've verified that this is a bug in OmniOS /bin/sh.

OmniOS /bin/sh reads scripts in 8192-byte chunks.  If the last line of a
here-document straddles a chunk boundary, and the last character of that line
is not a backslash, but the last character of the first chunk _is_ a
backslash, and the first character of the second chunk is a dollar sign, then
the here-document will be written out without a final newline.  (I'm not sure
whether the boundary has to fall within the last line of the here-document, or
whether the first character of the second chunk has to be a dollar sign.  I
know \# doesn't work, though.)

The attached file t-001fa2.sh.gz is a self-contained reproducer for the bug. 
If you uncompress it and run it with a shell that has this bug, it will print
something like this:


cmp: EOF on /tmp/tmp.LCyimJxG8v after byte 1, in line 1
+ xxd /tmp/tmp.q5iPccsLOn
: 240a $.
+ xxd /tmp/tmp.LCyimJxG8v
: 24   $
+ exit 1
+ rm -f /tmp/tmp.q5iPccsLOn /tmp/tmp.LCyimJxG8v


(Note that the script does not have a `#!` line.  Also, you need to have `xxd`
for it to work as intended, but both invocations of this utility are after the
critical part of the code, so you can safely change them to any other hex-dump
program you may happen to have around without breaking the test.  I have also
attached the program I used to generate the test script, which may be easier
to tinker with.)

(The name t-001fa2 refers to the number of padding characters in the script,
before the actual code.)

> I wonder why autoconf-2.69b does not exhibit the issue

I believe this was pure luck.  In your testing, autoconf-2.69b happened not to
generate a script that had \$, inside a here document, at exactly the right
offset.  I don't see any reason why this couldn't have happened with _any_
version of Autoconf.

It's also possible that the bug was recently introduced -- did you patch
IP65-66-246-71.simplesystems.org recently?  I wasn't able to reproduce the bug
with any of /bin/sh, /bin/ksh, or /usr/xpg4/bin/sh on heritage Sun Solaris 10
or 11.



Now we know what's wrong, the question is what to do about it.  Of course
OmniOS should be notified.  I looked at https://omniosce.org/ a bit and was
not able to figure out how to report the bug.  (It says that development
happens on Github, and I found a link to what _seems_ to be the repo where
development happens, namely https://github.com/omniosorg/illumos-omnios -- but
that repo has no open issues at all, so I'm guessing that is _not_ the place
to file bug reports.)

It is not feasible to make Autoconf avoid ever writing out backslashes inside
here-documents.  Generation of config.status, in particular, involves using
here-documents to emit shell code, in which backslashes are often necessary.

What we _could_ do is add a test for this sh bug to _AS_DETECT_BETTER_SHELL. 
It would be a little fiddly but not difficult.  I hesitate, though, because of
how sensitive this test would be to unrelated modifications to the shell --
for instance, if someone happens to decide tomorrow that OmniOS sh should read
files in 16kB chunks instead of 8kB chunks, a test that only probed the 8kB
boundary would be invalidated.  How many different offsets do we need to try
before we have confidence that the bug is not present?

cc:ing several other autoconf maintainers for their thoughts.

(file #50563, file #50564)
___

Additional Item Attachment:

File name: t-001fa2.sh.gz Size:0 KB


File name: test-gen.pySize:1 KB




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Bob Friesenhahn]

Follow-up Comment #6, sr #110403 (project autoconf):

I wonder why autoconf-2.69b does not exhibit the issue, and in fact I have not
seen any other issues with configure scripts on this system?  This system has
been one of my core development systems for quite some time now.

Note that this system has a config.site script at the default location, but
that is normal for my systems.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Zack Weinberg]

Follow-up Comment #5, sr #110403 (project autoconf):

I just ran the autoconf testsuite myself on this system, using the tip of
branch-2.70.  I do *not* see AC_TYPE_INTMAX_T fail, but I *do* see two other
failures which appear to have the same root cause: config.status files with a
missing line break at the end of a here-document in the parent configure
script.


SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL##  ##
## M4sh Initialization. ##


and


test -n "$AWK" || AWK=awk# The default lists apply if the user does not
specify any file.


In Bourne shell syntax, `#` only begins a comment if it is the *beginning* of
a shell word, so both of these turn into syntax errors.

To my surprise, this is *not* an intermittent fault.  It happens every single
time I run the configure script for one of the tests that fails for me. 
However, it doesn't happen if I run `bash ./configure` instead of
`./configure`, and it goes away if I add a single blank line near the top of
the offending configure script.  

I think what we're looking at here is a bug in OmniOS /bin/sh, in which
here-documents lose their final \n character if they are at particular offsets
within a shell script.  I'll see if I can come up with a precise test for that
hypothesis.

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Bob Friesenhahn]

Follow-up Comment #4, sr #110403 (project autoconf):

I just verified that this test passes for autoconf-2.69b.  This is the test
result using autoconf-2.69b, which is what I had tested on this system
previously:

## - ##
## Test results. ##
## - ##

474 tests behaved as expected.
49 tests were skipped.


___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Bob Friesenhahn]

Follow-up Comment #3, sr #110403 (project autoconf):

The external host name appears to be "IP65-66-246-71.simplesystems.org". 
Sorry for not making it more memorable in DNS.

Previously we determined that the ksh93-based shell behaves poorly if
/usr/xpg4/bin is in the path.  So I removed it from my path.

My path setting is
/usr/local/bin:/opt/DTT/bin:/opt/local/bin:/opt/local/sbin:/usr/gnu/bin:/usr/bin:/usr/sbin:/usr/ucb

The compiler used comes from /opt/local/bin/gcc

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Zack Weinberg]

Follow-up Comment #2, sr #110403 (project autoconf):

Thanks for the testsuite directory attachment.  The problem is clearer now:


# Set up the scripts for CONFIG_HEADERS section.
# No need to generate them if there are no CONFIG_HEADERS.
# This happens for instance with `./config.status Makefile'.
if test -n "$CONFIG_HEADERS"; then
cat >"$ac_tmp/defines.awk" <<\_ACAWK ||
BEGIN {
D["PACKAGE_NAME"]=" \"\""
...
}
{ print }
_ACAWK  as_fn_error $? "could not setup config headers machinery" "$LINENO" 5
fi # test -n "$CONFIG_HEADERS"


In a correctly generated config.status, the `as_fn_error` would not be on the
same line as `_ACAWK`.  However, the code that generates this part of
config.status hasn't changed in years either, and the corresponding piece of
the configure script in this test is exactly as it ought to be:


{ print }
_ACAWK
_ACEOF
cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
  as_fn_error $? "could not setup config headers machinery" "$LINENO" 5
fi # test -n "$CONFIG_HEADERS"


So I'm still suspicious of some kind of intermittent fault in the shell
executing this configure script.

> As a reminder, you have login access to this system.

I seem to have lost the message in which you told me the *name* of this
system, could you tell me what it is again?

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Bob Friesenhahn]

Additional Item Attachment, sr #110403 (project autoconf):

File name: autoconf-2.70-test537.tar.gz   Size:905 KB
   




___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Zack Weinberg]

Update of sr #110403 (project autoconf):

  Status:None => Need Info  

___

Follow-up Comment #1:

This is bizarre; nothing has changed in a very long time that would plausibly
cause this failure.  AC_TYPE_INTMAX_T hasn't been changed *at all* since it
was added, the core logic to check for missing types hasn't been changed since
2008, and the logic to check for `long long' hasn't been changed since 2012. 
And AC_TYPE_INTMAX_T is almost exactly the same as AC_TYPE_UINTMAX_T, but the
test for that didn't fail.

Could you please try running `make check TESTSUITEFLAGS=537` over and over
again, at least ten times, and report whether it fails every single time? 
Could you please also tar up the contents of tests/testsuite.dir/537, from a
run where it failed, and attach that to this bug report?

___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/

[sr #110403] autoconf-2.70 AC_TYPE_INTMAX_T test failure under OmniOS

[Bob Friesenhahn]

URL:
  

 Summary: autoconf-2.70 AC_TYPE_INTMAX_T test failure under
OmniOS
 Project: Autoconf
Submitted by: bfriesen
Submitted on: Wed 16 Dec 2020 02:40:22 PM CST
Category: None
Priority: 5 - Normal
Severity: 3 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
Operating System: Solaris

___

Details:

While running the autoconf-2.70 test suite under OmniOS I encountered a new
failure that I have not seen before.  This failure did not occur with at least
some of the release candidates.

A testsuite.log file is attached.



___

File Attachments:


---
Date: Wed 16 Dec 2020 02:40:22 PM CST  Name: testsuite.log.gz  Size: 19KiB  
By: bfriesen



___

Reply to this item at:

  

___
  Message sent via Savannah
  https://savannah.gnu.org/