Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Hello, James McCoy wrote: > What about just disabling the Perl bindings on kfreebsd-any for now? If you'd be happy to do that, yes please. But keeping the bug open. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Fri, Nov 04, 2016 at 01:39:35PM +, Steven Chamberlain wrote: > Hi, > > If you've no objection, I may build vim on the kfreebsd-* porterboxes > with DEB_BUILD_OPTIONS=nocheck, and binNMU the result. What about just disabling the Perl bindings on kfreebsd-any for now? Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Hi, If you've no objection, I may build vim on the kfreebsd-* porterboxes with DEB_BUILD_OPTIONS=nocheck, and binNMU the result. The problem would still be there, and it would be RC-severity if kfreebsd were part of the stable release, but it's currently not. Doing this would fix debootstrap of sid in the meantime. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Control: tag -1 help On Sun, Oct 16, 2016 at 12:27:13AM +0100, Steven Chamberlain wrote: > Hi! > > I was about finished for the night, but then I thought to recompile > option.c this way: > > $ cd src/vim-gtk > $ touch option.c > $ make PERL_CFLAGS="$(perl -MExtUtils::Embed -e ccopts) -g -O0 > -fno-omit-frame-pointer" > | gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK -pthread > -I/usr/include/gtk-2.0 -I/usr/lib/i386-kfreebsd-gnu/gtk-2.0/include > -I/usr/include/gio-unix-2.0/ -I/usr/include/cairo -I/usr/include/pango-1.0 > -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 > -I/usr/include/libpng16 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 > -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 > -I/usr/include/glib-2.0 -I/usr/lib/i386-kfreebsd-gnu/glib-2.0/include > -I/usr/include/freetype2 -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 > -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe > -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 > -I/usr/lib/i386-kfreebsd-gnu/perl/5.24/CORE -g -O0 -fno-omit-frame-pointer > -o objects/option.o option.c > > (all $*_CFLAGS were undefined except for PERL_CFLAGS, plus my additions) > and I finally got a backtrace from gdb! > > | #0 0x0810d45b in parse_cino (buf=0x82eebe0) at misc1.c:6955 6955 for (p = buf->b_p_cino; *p; ) Yeah, that's the same stack I was seeing all yesterday while bisecting. buf->b_p_cino shouldn't be NULL at this point. Taking a step back, there are two issues here. First, something has changed in the toolset (gcc, Perl, libc, etc.) that causes 7.4.963 (i.e., the last Vim package built on kfreebsd) to no longer be able to run properly anymore. This appears to be related to the -D_LARGEFILE_SOURCE/-D_FILE_OFFSET_BITS=64 defines. In 7.4.963, the explicit use of those was isolated to the if_perl* modules, so that's the only part of Vim that can trigger the crash. You can see the crash by running the tests of any of the Vim package variants which include the language bindings (vim-nox, vim-gtk, vim-athena, or vim-gnome). Alternatively, just run something like: ./src/vim -u NONE -i NONE --cmd 'perl $curbuf->Set($curline, "Crash")' Second, the impact of the problem became broader in Vim patch 7.4.1065 because the PERL_CFLAGS are now applied to option.c, thus touching code that is used in much more of Vim. At that point, we now can see the crash simply from trying to run Vim. That's the stack you hit. Considering that this _only_ exhibits on kfreebsd-* (and maybe alpha), I don't think it's a Vim problem. I also don't know how to go further with this. Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
In case it is helpful to know this later: 1561 100852 vim 0.016031 NAMI "/home/stevenc/vim-8.0.0022/src/vim-gtk/po" 1561 100852 vim 0.016033 RET __getcwd 0 1561 100852 vim 0.016065 CALL break(0x136b000) 1561 100852 vim 0.016071 RET break 0 1561 100852 vim 0.016110 CALL gettimeofday(0xbfbfe548,0) 1561 100852 vim 0.016113 RET gettimeofday 0 1561 100852 vim 0.016251 CALL stat(0x134bf98,0xbfbfe3b0) 1561 100852 vim 0.016256 NAMI "/usr/share/vim/vim80" 1561 100852 vim 0.016269 RET stat -1 errno 2 No such file or directory 1561 100852 vim 0.016273 CALL stat(0x134bf98,0xbfbfe3b0) 1561 100852 vim 0.016276 NAMI "/usr/share/vim/runtime" 1561 100852 vim 0.016283 RET stat -1 errno 2 No such file or directory 1561 100852 vim 0.016313 CALL ioctl(0x1,0x402c7413,0xbfbfe4a4) 1561 100852 vim 0.016317 RET ioctl 0 1561 100852 vim 0.016364 PSIG SIGSEGV SIG_DFL code=SEGV_MAPERR 1561 100852 vim 0.016370 NAMI "vim.core" the ioctl there is TIOCGETA (type 0x74, number 0x13) and it is reading 0x2c = 44 bytes (struct termios) to a stack variable, for fd 0x1 (stdout). That part looks okay; the segfault happens sometime after it. The only code that would seem to use that ioctl is os_unix.c's mch_settmode() which normally does a tcgetattr followed by a tcsetattr. This is a wild guess, but maybe the segfault is in that function? (HAVE_TERMIOS_H is defined here on kfreebsd-i386). Nothing here seems related to large file support, though. Are you sure the crashed triggered by large file support is really the same issue? (Does the end of the ktrace look like the above?) Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
James McCoy wrote: > Lo and behold, configuring without --enable-perlinterp but adding > "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" to PERL_CFLAGS in > src/auto/config.mk reproduces the issue. [...] > Which begs the question, why does configure think the defines aren't > necessary? That seems to not be the issue. It is just testing if lseeko() is declared already, without explicitly asking for -D_LARGEFILE_SOURCE. The result on both kfreebsd and linux is that it is not needed to use -D_LARGEFILE_SOURCE at all. So the linux arches already build option.c with -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64, and most other sources without. I wonder what are the implications. -D_FILE_OFFSET_BITS=64 has some effects on 32-bit arches, but not on 64-bit where relevant types are 64 bits long anyway. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
James McCoy wrote: > And why does Perl's build think they are necessary, advertising them as > required to build against Perl? Even in Linux architectures (amd64 and armhf for example) Perl advertises -D_LARGEFILE_SOURCE as a necessary compiler flag: $ perl -MExtUtils::Embed -e ccopts -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/x86_64-linux-gnu/perl/5.20/CORE According to feature_test_macros(7) it should not be necessary and is now discouraged; but it reads like _LARGEFILE_SOURCE should be defined by default anyway. Maybe that is the issue we have on kfreebsd-i386. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Sat, Oct 15, 2016 at 03:41:07PM +0100, Steven Chamberlain wrote: > Steven Chamberlain wrote: > > If I override that flag with -fno-wrapv: > > > > + $(CCC) $(LUA_CFLAGS) $(PERL_CFLAGS) -fno-wrapv $(PYTHON_CFLAGS) > > $(PYTHON3_CFLAGS) $(RUBY_CFLAGS) $(TCL_CFLAGS) -o $@ option.c > > > > then it no longer segfaults, and all vim-gtk tests pass :) > > Or, I possibly mis-compiled it without Perl bindings. I'd better > double-check this first. -fno-wrapv doesn't seem to have an effect for me. Playing around with the other flags, it actually seems to be the large file support flags. configure determines that they aren't generally needed, so they aren't part of CFLAGS for most of the source files. However, PERL_CFLAGS does contain them. Lo and behold, configuring without --enable-perlinterp but adding "-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64" to PERL_CFLAGS in src/auto/config.mk reproduces the issue. Which begs the question, why does configure think the defines aren't necessary? Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Steven Chamberlain wrote: > If I override that flag with -fno-wrapv: > > + $(CCC) $(LUA_CFLAGS) $(PERL_CFLAGS) -fno-wrapv $(PYTHON_CFLAGS) > $(PYTHON3_CFLAGS) $(RUBY_CFLAGS) $(TCL_CFLAGS) -o $@ option.c > > then it no longer segfaults, and all vim-gtk tests pass :) Or, I possibly mis-compiled it without Perl bindings. I'd better double-check this first. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Hi! James McCoy wrote: > That pattern continued and Vim patch 7.4.1065[0] appears to be what > broke Vim for kFreeBSD. Thanks for narrowing it down to that patch. Most of it only relates to --enable-perlinterp=dynamic and not to --enable-perlinterp[=yes] so the changes are mostly no-ops... What actually seems to make a difference is: src/Makefile: - $(CCC) -o $@ option.c + $(CCC) $(LUA_CFLAGS) $(PERL_CFLAGS) $(PYTHON_CFLAGS) $(PYTHON3_CFLAGS) $(RUBY_CFLAGS) -o $@ option.c where S["PERL_CFLAGS"]=" -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/i386-kfreebsd-gnu/perl/5.24/CORE " and -fwrapv significantly changes the code generated in objects/option.o. If I override that flag with -fno-wrapv: + $(CCC) $(LUA_CFLAGS) $(PERL_CFLAGS) -fno-wrapv $(PYTHON_CFLAGS) $(PYTHON3_CFLAGS) $(RUBY_CFLAGS) $(TCL_CFLAGS) -o $@ option.c then it no longer segfaults, and all vim-gtk tests pass :) So maybe there is a signed integer overflow in option.c (not necessarily in code related to Perl at all). But the linux-i386 build compiles option.c with -fwrapv, and yet it does not segfault; I'm not sure why. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Thu, Oct 13, 2016 at 09:08:17PM -0400, James McCoy wrote: > On Thu, Oct 13, 2016 at 08:27:12PM -0400, James McCoy wrote: > > On Thu, Oct 13, 2016 at 07:37:13PM -0400, James McCoy wrote: > > > However, I was currently looking into a different test failure whose > > > symptom is that ":set modified" doesn't actually do anything. > > > > Interesting. This has something to do with enabling the language > > bindings. When compiling with the language bindings, ":set modified" > > doesn't change the option. > > Similarly, the segfault only happens when the Perl bindings are enabled. I decided to do a little bisecting to see when running ./src/vim -u NONE -i NONE -N --cmd 'set modified' --cmd 'if | q! | else | cq | endif' would start breaking, using HEAD as known bad and 7.4.963 as known good. Interestingly, the first step saw Vim outright crash when it started up. However, the core was actually usable by gdb and also went away when compiling without --enable-perlinterp! That pattern continued and Vim patch 7.4.1065[0] appears to be what broke Vim for kFreeBSD. [0]: https://github.com/vim/vim/releases/tag/v7.4.1065 Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Thu, Oct 13, 2016 at 08:27:12PM -0400, James McCoy wrote: > On Thu, Oct 13, 2016 at 07:37:13PM -0400, James McCoy wrote: > > However, I was currently looking into a different test failure whose > > symptom is that ":set modified" doesn't actually do anything. > > Interesting. This has something to do with enabling the language > bindings. When compiling with the language bindings, ":set modified" > doesn't change the option. Similarly, the segfault only happens when the Perl bindings are enabled. Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Fri, Oct 14, 2016 at 12:08:46AM +0100, Steven Chamberlain wrote: > Hi, > > James McCoy wrote: > > I've solved the fifo test failure, and now am able to see the segfault > > on fischer. However, gdb has been pretty useless for me on kfreebsd. > > > > Is there something else that I should try using instead? > > I would usually enable core dumps and look at those in gdb first, but > most of the time ktrace explains better what led up to the crash. > > I usually use `ktrace -di -- executable` and `kdump -EHf ktrace.out`. > Please show me the output of that, if you can reproduce the crash that > way. Thanks for the ktrace pointer. That's what helped me fix the fifo problem. However, I was currently looking into a different test failure whose symptom is that ":set modified" doesn't actually do anything. Being able to step through the code would make that a bit easier, but I guess I can add printfs. Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Steven Chamberlain wrote: > I didn't see a segfault yet on fischer, only this: Never mind, I can reproduce it with ~/vim-8.0.0022/src/vim-gtk/po$ ktrace -di -- ../vim -u NONE -e -X -S check.vim -c "if error == 0 | q | endif" -c cq af.po I'm reading over the kdump output now. I'm not sure what's with gdb. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
Hi, James McCoy wrote: > I've solved the fifo test failure, and now am able to see the segfault > on fischer. However, gdb has been pretty useless for me on kfreebsd. > > Is there something else that I should try using instead? I would usually enable core dumps and look at those in gdb first, but most of the time ktrace explains better what led up to the crash. I usually use `ktrace -di -- executable` and `kdump -EHf ktrace.out`. Please show me the output of that, if you can reproduce the crash that way. Thanks, Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Sat, Oct 01, 2016 at 08:33:54PM -0400, James McCoy wrote: > On Sat, Oct 01, 2016 at 10:48:47PM +0100, Steven Chamberlain wrote: > > But now, a new error is being seen on kfreebsd and interestingly, also > > on linux-alpha: > > > > | Found errors in Test_tagcase(): > > | Caught exception in Test_tagcase(): Vim(edit):E37: No write since last > > change (add ! to override) @ function RunTheTest[9]..Test_tagcase, line 3 > > | TEST FAILURE > > There's also a segfault on kfreebsd-i386 and a failure in > test_startup_utf8 (Test_read_fifo_utf8) on kfreebsd-*. I've solved the fifo test failure, and now am able to see the segfault on fischer. However, gdb has been pretty useless for me on kfreebsd. Is there something else that I should try using instead? Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
On Sat, Oct 01, 2016 at 10:48:47PM +0100, Steven Chamberlain wrote: > Hi, Hi! > The error is slightly different now. > > Test_communicate was fixed upstream: > | 1621 7.4.2154 Test_communicate() fails sometimes Well, worked around. That just marks the test as flaky, so it'll get retried once. :) > But now, a new error is being seen on kfreebsd and interestingly, also > on linux-alpha: > > | Found errors in Test_tagcase(): > | Caught exception in Test_tagcase(): Vim(edit):E37: No write since last > change (add ! to override) @ function RunTheTest[9]..Test_tagcase, line 3 > | TEST FAILURE There's also a segfault on kfreebsd-i386 and a failure in test_startup_utf8 (Test_read_fifo_utf8) on kfreebsd-*. I was just trying to reproduce the segfault yesterday on fischer, but only got the fifo failure. Haven't had a chance to look into that one more. Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Re: Bug#827319: vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed
retitle 827319 vim: FTBFS[alpha,kfreebsd-*]: Test_tagcase() failed found 827319 src:vim/2:7.4.2330-1 thanks Hi, The error is slightly different now. Test_communicate was fixed upstream: | 1621 7.4.2154 Test_communicate() fails sometimes But now, a new error is being seen on kfreebsd and interestingly, also on linux-alpha: | Found errors in Test_tagcase(): | Caught exception in Test_tagcase(): Vim(edit):E37: No write since last change (add ! to override) @ function RunTheTest[9]..Test_tagcase, line 3 | TEST FAILURE The regression appeared somewhere between 2:7.4.1829-1 and 2:7.4.2330-1. Regards, -- Steven Chamberlain ste...@pyro.eu.org signature.asc Description: Digital signature