On 12/29/15, Karel Gardas <[email protected]> wrote: > On Sun, Dec 27, 2015 at 10:23 PM, Richard Hipp <[email protected]> wrote: >> However, one side effect of the Git clone effort is that the >> fast-import logic of Fossil has been improved. It should run faster >> now, and use a lot less disk space. Can you please try your import >> again and let us know what happens? > > Hello, so testing 09d3e7ebf0 on OpenBSD current as of Nov 2015. > AMD64/Xeon E3-1220v3. The import of bitrig repository still crashes > fossil with: > > $ time git fast-export --all | $HOME/sfw/fossil-head/bin/fossil import > --git ../bitrig2.fossil > Rebuilding repository meta-data...
The "fossil import" command has completely read and parsed and inserted all of the repository content. It is now running a "fossil rebuild". That is not strictly necessary. It would work if we just stopped prior to running the rebuild. Nevertheless, it shouldn't segfault. There must be something in the "fossil import" logic that is causing heap corruption. I added a new command-line option to "fossil import" to cause it to skip the "rebuild" phase. Please recompile using the latest trunk version and run with the --no-rebuild option. Then make a copy of the finished bitrig2.fossil file and run "fossil rebuild --compress copy.fossil" on the copy to see if *that* fails. My guess is that the rebuild step is stumbling over some kind of heap corruption that the import phase left behind. Doing the import in two phases, as suggested in the previous paragraph, do not cure this problem, but if it works it does help to localize it. > Segmentation fault (core dumped) > 1232m32.58s real 937m14.44s user 94m15.63s system > > as you can see it's either a little bit slower or crashes later. > Anyway, backtrace looks: > > $ gdb /home/karel/sfw/fossil-head/bin/fossil fossil.core > GNU gdb 6.3 > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-unknown-openbsd5.8"... > Core was generated by `fossil'. > Program terminated with signal 11, Segmentation fault. > Loaded symbols for /home/karel/sfw/fossil-head/bin/fossil > Reading symbols from /usr/lib/libssl.so.37.1...done. > Loaded symbols for /usr/lib/libssl.so.37.1 > Reading symbols from /usr/lib/libcrypto.so.36.1...done. > Loaded symbols for /usr/lib/libcrypto.so.36.1 > Reading symbols from /usr/lib/libz.so.5.0...done. > Loaded symbols for /usr/lib/libz.so.5.0 > Reading symbols from /usr/lib/libm.so.9.0...done. > Loaded symbols for /usr/lib/libm.so.9.0 > Reading symbols from /usr/lib/libfuse.so.1.1...done. > Loaded symbols for /usr/lib/libfuse.so.1.1 > Reading symbols from /usr/lib/libc.so.84.0...done. > Loaded symbols for /usr/lib/libc.so.84.0 > Reading symbols from /usr/libexec/ld.so...done. > Loaded symbols for /usr/libexec/ld.so > #0 0x000012c4ca367ce8 in next_token (p=0x7f7ffffdc270, > pLen=0x7f7ffffdc298) at manifest.c:292 > 292 manifest.c: No such file or directory. > in manifest.c > (gdb) where > #0 0x000012c4ca367ce8 in next_token (p=0x7f7ffffdc270, > pLen=0x7f7ffffdc298) at manifest.c:292 > #1 0x000012c4ca3695b1 in manifest_parse (pContent=Variable "pContent" > is not available. > ) at manifest.c:446 > #2 0x000012c4ca36b35b in manifest_crosslink (rid=305814, > pContent=0x7f7ffffdc560, flags=0) at manifest.c:1811 > #3 0x000012c4ca37d303 in rebuild_step (rid=305814, size=14, > pBase=0x7f7ffffdc560) at rebuild.c:255 > #4 0x000012c4ca37dec2 in rebuild_db (randomize=Variable "randomize" > is not available. > ) at rebuild.c:399 > #5 0x000012c4ca357d36 in import_cmd () at import.c:1668 > #6 0x000012c4ca36746e in main (argc=0, argv=0x0) at main.c:804 > > > The fossil is probably using its own version of SQLite: > $ ldd /home/karel/sfw/fossil-head/bin/fossil > /home/karel/sfw/fossil-head/bin/fossil: > Start End Type Open Ref GrpRef Name > 0000167fb1c00000 0000167fb2269000 exe 1 0 0 > /home/karel/sfw/fossil-head/bin/fossil > 00001682a2c5d000 00001682a30b7000 rlib 0 1 0 > /usr/lib/libssl.so.37.1 > 0000168266f30000 00001682674fe000 rlib 0 2 0 > /usr/lib/libcrypto.so.36.1 > 00001682558fa000 0000168255d0f000 rlib 0 1 0 > /usr/lib/libz.so.5.0 > 0000168203935000 0000168203d5d000 rlib 0 1 0 > /usr/lib/libm.so.9.0 > 000016822c0db000 000016822c4e4000 rlib 0 1 0 > /usr/lib/libfuse.so.1.1 > 000016828c833000 000016828ccfc000 rlib 0 1 0 > /usr/lib/libc.so.84.0 > 00001681e3100000 00001681e3100000 rtld 0 1 0 > /usr/libexec/ld.so > > Perhaps the issue is in git 2.6.2? On Solaris 11 I'm testing with very > old 1.7.9.2, so perhaps fast export changed in a way fossil is not > able to survive? I'll see if I can test with the same git on Solaris > box or with older git on OpenBSD box. Once done I'll report back. > > Thanks! Karel > _______________________________________________ > fossil-users mailing list > [email protected] > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > -- D. Richard Hipp [email protected] _______________________________________________ fossil-users mailing list [email protected] http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
