Re: Tags break when enabling large file support
Haakon Riiser wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. Is there another way to get large file support that works? There's no ./configure argument to enable it, so I just added the above macro definition flag in CFLAGS. The problem with not having large file support is not that I need to edit files larger than 2 GiB, but that ^X^F doesn't expand large files, probably because stat() fails with EOVERFLOW. By the way, this problem was encountered on the following system: OS: Slackware Linux 10.2 Libc: 2.3.5 GCC: several versions: 3.3.6, 3.4.6, 4.1.1 Vim: several versions, including 7.0.0 and 7.0.101 I'm using gvim (currently 7.0.101 but soon I'll compile the recently-published patches) and, without any compiler defines other than those set by configure according to the configure flags shown on my howto page http://users.skynet.be/antoine.mechelynck/vim/compunix.htm I'm currently editing a file with 1,118,641 lines and 33,705,005 bytes of data (and growing). No probs with accessing the help in that same instance of gvim. Of course there's still some margin between 33 Meg and 2 G. I'm on SuSE Linux 9.3 with gcc 3.3.5 20050117 (prerelease) (SuSE Linux) and glibc 2.3.4. -D_FILE_OFFSET_BITS=64 is set on my gcc command-line without me having done anything special to get it. Best regards, Tony. P.S. :version VIM - Vi IMproved 7.0 (2006 May 7, compiled Sep 13 2006 00:24:04) Included patches: 1-101 Compiled by [EMAIL PROTECTED] Huge version with GTK2-GNOME GUI. Features included (+) or not (-): +arabic +autocmd +balloon_eval +browse ++builtin_terms +byte_offset +cindent +clientserver +clipboard +cmdline_compl +cmdline_hist +cmdline_info +comments +cryptv +cscope +cursorshape +dialog_con_gui +diff +digraphs +dnd -ebcdic +emacs_tags +eval +ex_extra +extra_search +farsi +file_in_path +find_in_path +folding -footer +fork() +gettext -hangul_input +iconv +insert_expand +jumplist +keymap +langmap +libcall +linebreak +lispindent +listcmds +localmap +menu +mksession +modify_fname +mouse +mouseshape +mouse_dec +mouse_gpm -mouse_jsbterm +mouse_netterm +mouse_xterm +multi_byte +multi_lang -mzscheme +netbeans_intg -osfiletype +path_extra +perl +postscript +printer +profile +python +quickfix +reltime +rightleft +ruby +scrollbind +signs +smartindent -sniff +statusline -sun_workshop +syntax +tag_binary +tag_old_static -tag_any_white +tcl +terminfo +termresponse +textobjects +title +toolbar +user_commands +vertsplit +virtualedit +visual +visualextra +viminfo +vreplace +wildignore +wildmenu +windows +writebackup +X11 -xfontset +xim +xsmp_interact +xterm_clipboard -xterm_save system vimrc file: $VIM/vimrc user vimrc file: $HOME/.vimrc user exrc file: $HOME/.exrc system gvimrc file: $VIM/gvimrc user gvimrc file: $HOME/.gvimrc system menu file: $VIMRUNTIME/menu.vim fall-back for $VIM: /usr/local/share/vim Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -I/opt/gnome/include/gtk-2.0 -I/opt/gnome/lib/gtk-2.0/include -I/usr/X11R6/include -I/opt/gnome/include/atk-1.0 -I/opt/gnome/include/pango-1.0 -I/usr/include/freetype2 -I/usr/include/freetype2/config -I/opt/gnome/include/glib-2.0 -I/opt/gnome/lib/glib-2.0/include -DORBIT2=1 -pthread -DXTHREADS -D_REENTRANT -DXUSE_MTSAFE_API -I/usr/include/libart-2.0 -I/usr/include/libxml2 -I/opt/gnome/include/libgnomeui-2.0 -I/opt/gnome/include/libgnome-2.0 -I/opt/gnome/include/libgnomecanvas-2.0 -I/opt/gnome/include/gtk-2.0 -I/opt/gnome/include/gconf/2 -I/opt/gnome/include/libbonoboui-2.0 -I/opt/gnome/include/glib-2.0 -I/opt/gnome/lib/glib-2.0/include -I/opt/gnome/include/orbit-2.0 -I/opt/gnome/include/libbonobo-2.0 -I/opt/gnome/include/gnome-vfs-2.0 -I/opt/gnome/lib/gnome-vfs-2.0/include -I/opt/gnome/include/bonobo-activation-2.0 -I/opt/gnome/include/pango-1.0 -I/usr/include/freetype2 -I/opt/gnome/lib/gtk-2.0/include -I/usr/X11R6/include -I/opt/gnome/include/atk-1.0 -I/usr/include/freetype2/config -O2 -fno-strength-reduce -Wall -I/usr/X11R6/include -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/lib/perl5/5.8.6/i586-linux-thread-multi/CORE -I/usr/include/python2.4 -pthread -I/usr/include -D_LARGEFILE64_SOURCE=1 -I/usr/lib/ruby/1.8/i686-linux Linking: gcc -L/opt/gnome/lib -L/usr/X11R6/lib -rdynamic -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.8.6/i586-linux-thread-multi/CORE -L/usr/local/lib -o vim -L/opt/gnome/lib -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgdk_pixbuf-2.0 -lpangoxft-1.0 -lpangox-1.0 -lpango-1.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -L/opt/gnome/lib -L/usr/X11R6/lib -lgnomeui-2 -lbonoboui-2 -lxml2 -lz -lgnomecanvas-2 -lgnome-2 -lpopt -lart_lgpl_2 -lpangoft2-1.0 -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0
Re: Tags break when enabling large file support
On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote:
After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything
involving tags break, including the help system. Typing :h,
or pressing ^] to jump to a tag, causes Vim to get caught in an
infinite loop.
Is there another way to get large file support that works?
There's no ./configure argument to enable it, so I just added
the above macro definition flag in CFLAGS. The problem with
not having large file support is not that I need to edit files
larger than 2 GiB, but that ^X^F doesn't expand large files,
probably because stat() fails with EOVERFLOW.
By the way, this problem was encountered on the following system:
OS: Slackware Linux 10.2
Libc: 2.3.5
GCC: several versions: 3.3.6, 3.4.6, 4.1.1
Vim: several versions, including 7.0.0 and 7.0.101
I also get infinite loop when I build vim with -D_FILE_OFFSET_BITS=64
on FedoraCode 3.
Applying 'strace -p' shows that vim is stuck searching for
tag, see the trace below. Apparently some read() in vim
does not have check for == 0 return value.
Yakov
[ here vim searches for tags in other file ... then ...]
open(/home/lerner/.vim/doc/tags, O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2506, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fbf000
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
_llseek(3, 0, [2506], SEEK_END) = 0
_llseek(3, 0, [0], SEEK_SET)= 0
_llseek(3, 0, [0], SEEK_SET)= 0
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
infinite .
Re: Tags break when enabling large file support
Yakov Lerner wrote:
On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote:
After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything
involving tags break, including the help system. Typing :h,
or pressing ^] to jump to a tag, causes Vim to get caught in an
infinite loop.
Is there another way to get large file support that works?
There's no ./configure argument to enable it, so I just added
the above macro definition flag in CFLAGS. The problem with
not having large file support is not that I need to edit files
larger than 2 GiB, but that ^X^F doesn't expand large files,
probably because stat() fails with EOVERFLOW.
By the way, this problem was encountered on the following system:
OS: Slackware Linux 10.2
Libc: 2.3.5
GCC: several versions: 3.3.6, 3.4.6, 4.1.1
Vim: several versions, including 7.0.0 and 7.0.101
I also get infinite loop when I build vim with -D_FILE_OFFSET_BITS=64
on FedoraCode 3.
Applying 'strace -p' shows that vim is stuck searching for
tag, see the trace below. Apparently some read() in vim
does not have check for == 0 return value.
Yakov
[ here vim searches for tags in other file ... then ...]
open(/home/lerner/.vim/doc/tags, O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2506, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fbf000
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
_llseek(3, 0, [2506], SEEK_END) = 0
_llseek(3, 0, [0], SEEK_SET)= 0
_llseek(3, 0, [0], SEEK_SET)= 0
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
infinite .
Can you find out where in the code this happens?
I suspect the 64 bit library functions work different from the normal
functions. Perhaps fgets(), since that's what is being used to read the
tags file.
--
Lose weight, NEVER Diet again with
The Invisible Weight Loss Patch
(spam e-mail)
/// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\download, build and distribute -- http://www.A-A-P.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Tags break when enabling large file support
On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote:
Yakov Lerner wrote:
On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote:
After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything
involving tags break, including the help system. Typing :h,
or pressing ^] to jump to a tag, causes Vim to get caught in an
infinite loop.
Is there another way to get large file support that works?
There's no ./configure argument to enable it, so I just added
the above macro definition flag in CFLAGS. The problem with
not having large file support is not that I need to edit files
larger than 2 GiB, but that ^X^F doesn't expand large files,
probably because stat() fails with EOVERFLOW.
By the way, this problem was encountered on the following system:
OS: Slackware Linux 10.2
Libc: 2.3.5
GCC: several versions: 3.3.6, 3.4.6, 4.1.1
Vim: several versions, including 7.0.0 and 7.0.101
I also get infinite loop when I build vim with -D_FILE_OFFSET_BITS=64
on FedoraCode 3.
Applying 'strace -p' shows that vim is stuck searching for
tag, see the trace below. Apparently some read() in vim
does not have check for == 0 return value.
Yakov
[ here vim searches for tags in other file ... then ...]
open(/home/lerner/.vim/doc/tags, O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2506, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0xb7fbf000
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
_llseek(3, 0, [2506], SEEK_END) = 0
_llseek(3, 0, [0], SEEK_SET)= 0
_llseek(3, 0, [0], SEEK_SET)= 0
read(3, :Brightness\txterm16.txt\t/*:Brigh..., 4096) = 2506
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
read(3, , 4096) = 0
infinite .
Can you find out where in the code this happens?
I suspect the 64 bit library functions work different from the normal
functions. Perhaps fgets(), since that's what is being used to read the
tags file.
In situation where fgets() returns NULL, find_tags() enters
infinite loop as follows.
(gdb) where
#0 0xe410 in ?? ()
#1 0xbff7bab0 in ?? ()
#2 0x0015 in ?? ()
#3 0xb7f7b000 in ?? ()
#4 0x00235573 in __write_nocancel () from /lib/tls/libc.so.6
#5 0x001dc57f in _IO_new_file_write () from /lib/tls/libc.so.6
#6 0x001db02b in _IO_new_do_write () from /lib/tls/libc.so.6
#7 0x001dba70 in _IO_new_file_overflow () from /lib/tls/libc.so.6
#8 0x001dd1ca in __overflow () from /lib/tls/libc.so.6
#9 0x001d351f in puts () from /lib/tls/libc.so.6
#10 0x080c2bcc in vim_fgets (
buf=0x8244a78
xterm16fg_GroupName\txterm16.txt\t/*xterm16fg_GroupName*\n,
size=512,
fp=0x827c1f8) at fileio.c:5853
#11 0x08177494 in find_tags (pat=0x8244568 [EMAIL PROTECTED],
num_matches=0x14, matchesp=0x14, flags=40,
mincount=1, buf_ffname=0x0) at tag.c:1625
#12 0x08178b35 in do_tag (tag=0x81f1168 [EMAIL PROTECTED], type=1, count=1,
forceit=0, verbose=1)
at tag.c:548
#13 0x080a3b08 in ex_help (eap=0x827d1b0) at ex_cmds.c:5519
#14 0x080b1e72 in do_one_cmd (cmdlinep=0xbff7ce08, sourcing=0,
cstack=0xbff7ce80,
fgetline=0x80c1828 getexline, cookie=0x0) at ex_docmd.c:2616
#15 0x080b3082 in do_cmdline (cmdline=0x0, getline=0x80c1828
getexline, cookie=0x0, flags=0)
at ex_docmd.c:1098
#16 0x08113620 in nv_colon (cap=0xbff7d220) at normal.c:5150
#17 0x08116d2f in normal_cmd (oap=0xbff7d290, toplevel=1) at normal.c:1137
#18 0x080decee in main_loop (cmdwin=0, noexmode=0) at main.c:1154
#19 0x080e2077 in main (argc=20, argv=0x14) at main.c:934
(gdb) b tag.c:1626
Breakpoint 1 at 0x8177494: file tag.c, line 1626.
(gdb) cont
Continuing.
Breakpoint 1, find_tags (pat=0x8244568 [EMAIL PROTECTED], num_matches=0x1,
matchesp=0x1, flags=40,
mincount=1, buf_ffname=0x0) at tag.c:1626
1626if (!eof search_info.curr_offset != 0)
(gdb) n
1625eof = tag_fgets(lbuf, LSIZE, fp);
(gdb) n
1626if (!eof search_info.curr_offset != 0)
(gdb) n
1651search_info.match_offset = ftell(fp);
(gdb) n
1650state = TS_SKIP_BACK;
(gdb) n
1651search_info.match_offset = ftell(fp);
(gdb) n
1652search_info.curr_offset =
search_info.curr_offset_used;
(gdb) n
1860continue;
(gdb) n
1652search_info.curr_offset =
search_info.curr_offset_used;
(gdb) n
1562line_breakcheck(); /* check for CTRL-C typed */
(gdb) n
1564if ((flags TAG_INS_COMP)) /* Double brackets for gcc */
(gdb) n
1566if (got_int || compl_interrupted)
(gdb) n
1576if (mincount == TAG_MANY match_count = TAG_MANY)
(gdb) n
1582if (get_it_again)
(gdb) n
1588if
Re: Tags break when enabling large file support
Yakov Lerner wrote: On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. [...] Can you find out where in the code this happens? I suspect the 64 bit library functions work different from the normal functions. Perhaps fgets(), since that's what is being used to read the tags file. In situation where fgets() returns NULL, find_tags() enters infinite loop as follows. Sorry, this doesn't show me where the loop actually is. Please compile without the optimizer, otherwise code reordering confuses the debugger. At least tell us if Vim hangs in tag_fgets() or in find_tags(). If it's in tag_fgets() it's probably a bug in the library. If it's in find_tags() then please try to find out why using 64 bit support makes a difference. -- MONK: ... and the Lord spake, saying, First shalt thou take out the Holy Pin, then shalt thou count to three, no more, no less. Three shalt be the number thou shalt count, and the number of the counting shalt be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thou foe, who being naughty in my sight, shall snuff it. Monty Python and the Holy Grail PYTHON (MONTY) PICTURES LTD /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
Re: Tags break when enabling large file support
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote:
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote:
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote:
On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote:
Yakov Lerner wrote:
On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote:
After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything
involving tags break, including the help system. Typing :h,
or pressing ^] to jump to a tag, causes Vim to get caught in an
infinite loop.
[...]
Can you find out where in the code this happens?
I suspect the 64 bit library functions work different from the
normal
functions. Perhaps fgets(), since that's what is being used to read
the
tags file.
In situation where fgets() returns NULL, find_tags() enters
infinite loop as follows.
Sorry, this doesn't show me where the loop actually is. Please compile
without the optimizer, otherwise code reordering confuses the debugger.
At least tell us if Vim hangs in tag_fgets() or in find_tags().
The loop is is inside find_tags(), and it procees around following line
numbers:
1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566
1576 1582 1588 1601 1603 1604 1616 1619 1621 1625
and infinite loop repeats. Look at this:
gdb print print search_info.curr_offset
$2 = 5258708303049
And additionally:
(gdb) print search_info
$3 = {low_offset = 0, high_offset = 10526964844947, curr_offset =
5258708303049,
curr_offset_used = 5258708303049, match_offset = 10763188046282,
low_char = 0, high_char = 120}
But ftell(fp) prints sane value:
(gdb) print ftell(fp)
$4 = 2506
And filesize is OK, 2506
Yakov
Re: Tags break when enabling large file support
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote: Yakov Lerner wrote: On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. [...] Can you find out where in the code this happens? I suspect the 64 bit library functions work different from the normal functions. Perhaps fgets(), since that's what is being used to read the tags file. In situation where fgets() returns NULL, find_tags() enters infinite loop as follows. Sorry, this doesn't show me where the loop actually is. Please compile without the optimizer, otherwise code reordering confuses the debugger. At least tell us if Vim hangs in tag_fgets() or in find_tags(). The loop is is inside find_tags(), and it procees around following line numbers: 1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566 1576 1582 1588 1601 1603 1604 1616 1619 1621 1625 and infinite loop repeats. Look at this: gdb print print search_info.curr_offset $2 = 5258708303049 I cannot guess where this loop first gets beyond end of file (filesize=2506), but once it gets beyond, we are missing check that search_info.*offset''s are beyond end of file ? Maybe add such check somewhere ? I don't know where to add the check. Yakov
Re: Tags break when enabling large file support
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote: Yakov Lerner wrote: On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. [...] Can you find out where in the code this happens? I suspect the 64 bit library functions work different from the normal functions. Perhaps fgets(), since that's what is being used to read the tags file. In situation where fgets() returns NULL, find_tags() enters infinite loop as follows. Sorry, this doesn't show me where the loop actually is. Please compile without the optimizer, otherwise code reordering confuses the debugger. At least tell us if Vim hangs in tag_fgets() or in find_tags(). The loop is is inside find_tags(), and it procees around following line numbers: 1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566 1576 1582 1588 1601 1603 1604 1616 1619 1621 1625 and infinite loop repeats. Look at this: gdb print print search_info.curr_offset $2 = 5258708303049 I cannot guess where this loop first gets beyond end of file (filesize=2506), but once it gets beyond, we are missing check that search_info.*offset''s are beyond end of file ? Maybe add such check somewhere ? I don't know where to add the check. Yakov It seems I found the culprint. It is line 1651, search_info.match_offset = ftell(fp); (gdb) p ftell(fp) $11 = 2506 (gdb) n 1652search_info.curr_offset = search_info.curr_offset_used; (gdb) p search_info.curr_offset $12 = 5258708302025 (gdb) Yakov
Re: Tags break when enabling large file support
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote: Yakov Lerner wrote: On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. [...] Can you find out where in the code this happens? I suspect the 64 bit library functions work different from the normal functions. Perhaps fgets(), since that's what is being used to read the tags file. In situation where fgets() returns NULL, find_tags() enters infinite loop as follows. Sorry, this doesn't show me where the loop actually is. Please compile without the optimizer, otherwise code reordering confuses the debugger. At least tell us if Vim hangs in tag_fgets() or in find_tags(). The loop is is inside find_tags(), and it procees around following line numbers: 1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566 1576 1582 1588 1601 1603 1604 1616 1619 1621 1625 and infinite loop repeats. Look at this: gdb print print search_info.curr_offset $2 = 5258708303049 I cannot guess where this loop first gets beyond end of file (filesize=2506), but once it gets beyond, we are missing check that search_info.*offset''s are beyond end of file ? Maybe add such check somewhere ? I don't know where to add the check. Yakov It seems I found the culprint. It is line 1651, search_info.match_offset = ftell(fp); (gdb) p ftell(fp) $11 = 2506 (gdb) n 1652search_info.curr_offset = search_info.curr_offset_used; (gdb) p search_info.curr_offset $12 = 5258708302025 (gdb) Ok, this must be gcc bug. It bites even without optimization. Rewriting lines 'search_info.curr_offset = ftell(fp)' fixes the problem if I rewrite them into this: long ltmp; ltmp = ftell(fp); search_info.curr_offset = ltmp; NB that ftell() returns long. Type of search_info.curr_offset is long long when -D_FILE_OFFSET_BITS=64. Simpe asignment search_info.curr_offset = ftell(fp) puts trash into lhs. ftell(fp) by itself returns correct number. this is gcc 3.4.2 Fedora3 Yakov
Re: Tags break when enabling large file support
On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Yakov Lerner [EMAIL PROTECTED] wrote: On 9/14/06, Bram Moolenaar [EMAIL PROTECTED] wrote: Yakov Lerner wrote: On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote: After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything involving tags break, including the help system. Typing :h, or pressing ^] to jump to a tag, causes Vim to get caught in an infinite loop. [...] Can you find out where in the code this happens? I suspect the 64 bit library functions work different from the normal functions. Perhaps fgets(), since that's what is being used to read the tags file. In situation where fgets() returns NULL, find_tags() enters infinite loop as follows. Sorry, this doesn't show me where the loop actually is. Please compile without the optimizer, otherwise code reordering confuses the debugger. At least tell us if Vim hangs in tag_fgets() or in find_tags(). The loop is is inside find_tags(), and it procees around following line numbers: 1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566 1576 1582 1588 1601 1603 1604 1616 1619 1621 1625 and infinite loop repeats. Look at this: gdb print print search_info.curr_offset $2 = 5258708303049 I cannot guess where this loop first gets beyond end of file (filesize=2506), but once it gets beyond, we are missing check that search_info.*offset''s are beyond end of file ? Maybe add such check somewhere ? I don't know where to add the check. Yakov It seems I found the culprint. It is line 1651, search_info.match_offset = ftell(fp); (gdb) p ftell(fp) $11 = 2506 (gdb) n 1652search_info.curr_offset = search_info.curr_offset_used; (gdb) p search_info.curr_offset $12 = 5258708302025 (gdb) Thi screenshot did not who intormation rght. It shoudl look (gdb) b tag.c:1651 Breakpoint 1 at 0x81a5032: file tag.c, line 1651. (gdb) cont Continuing. Breakpoint 1, find_tags (pat=0x827b538 [EMAIL PROTECTED], num_matches=0xbff0696c, matchesp=0xbff06968, flags=40, mincount=1, buf_ffname=0x0) at tag.c:1651 1651search_info.match_offset = ftell(fp); (gdb) n 1652search_info.curr_offset = search_info.curr_offset_used; (gdb) p search_info.match_offset $1 = 10763188046282 (gdb) p ftell(fp) $2 = 2506 (gdb)
Re: Tags break when enabling large file support
Haakon Riiser wrote: [A.J.Mechelynck] -D_FILE_OFFSET_BITS=64 is set on my gcc command-line without me having done anything special to get it. Strange; it doesn't do that on my system. Does it work if you invoke ./configure without any extra arguments? By the way, I noticed that your build also sets _LARGEFILE_SOURCE. I didn't think this macro was needed, since all it does it enable two extra functions, ftello() and fseeko(), which I assumed would not make any difference in the tags issue. Turns out that I was wrong -- defining _LARGEFILE_SOURCE somehow fixed the problem. Configure with no extra arguments will build a Normal version without Gnome and with no interpreters. Now that million-line file I spoke about is in UTF-8 and contains CJK characters mixed in with English text. A Normal-version Vim couldn't edit it because it would have no multi-byte editing support. Hmm... Which version are you using? - tiny, small, normal, large, big, huge? (I have huge) - no GUI, Motif, Athena, NextAW, GTK1, GTK1 with Gnome, GTK2, GTK2 with Gnome? Or MacOsX without X11? And in the latter case, Motorola, Intel or Universal binary? (I have GTK2 with Gnome on Linux i386) I suppose (rightly or wrongly) that one of Huge, GTK2 and Gnome (or something that is implied by them) is what turns on -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 Best regards, Tony.
Re: Tags break when enabling large file support
Bram Moolenaar wrote:
Yakov Lerner wrote:
On 9/14/06, Haakon Riiser [EMAIL PROTECTED] wrote:
After recompiling Vim with -D_FILE_OFFSET_BITS=64, everything
involving tags break, including the help system. Typing :h,
or pressing ^] to jump to a tag, causes Vim to get caught in an
infinite loop.
[...]
Can you find out where in the code this happens?
I suspect the 64 bit library functions work different from the
normal functions. Perhaps fgets(), since that's what is
being used to read the tags file.
In situation where fgets() returns NULL, find_tags() enters
infinite loop as follows.
Sorry, this doesn't show me where the loop actually is. Please compile
without the optimizer, otherwise code reordering confuses the debugger.
At least tell us if Vim hangs in tag_fgets() or in find_tags().
The loop is is inside find_tags(), and it procees around following line
numbers:
1625 1626 1642 1647 1650 1651 1652 1653 1562 1564 1566
1576 1582 1588 1601 1603 1604 1616 1619 1621 1625
and infinite loop repeats. Look at this:
gdb print print search_info.curr_offset
$2 = 5258708303049
The difference must be related to fseek(). I don't see where
search_info.curr_offset is first initialized, though.
The problem must be that either ftello() doesn't return the right
position or fseeko() doesn't position the read pointer properly. I
can't think of another reason.
And additionally:
(gdb) print search_info
$3 = {low_offset = 0, high_offset = 10526964844947, curr_offset = 5258708303049,
curr_offset_used = 5258708303049, match_offset = 10763188046282,
low_char = 0, high_char = 120}
This suggests that an unsigned number overflows and is turned into an
int somewhere. I can't guess where and I can't reproduce this. Perhaps
someone can think of a workaround and try it out.
One thing you can do: check that off_t is signed. You should have seen
compiler warnings then.
See Haakon's reply to my first post near the top of this thread: turning
on -D_LARGEFILE_SOURCE defines ftello() and fseeko(); it also makes the
bug disappear.
Best regards,
Tony.
Re: Tags break when enabling large file support
Haakon Riiser wrote: [A.J.Mechelynck] Configure with no extra arguments will build a Normal version without Gnome and with no interpreters. Now that million-line file I spoke about is in UTF-8 and contains CJK characters mixed in with English text. A Normal-version Vim couldn't edit it because it would have no multi-byte editing support. Hmm... Which version are you using? - tiny, small, normal, large, big, huge? (I have huge) - no GUI, Motif, Athena, NextAW, GTK1, GTK1 with Gnome, GTK2, GTK2 with Gnome? Or MacOsX without X11? And in the latter case, Motorola, Intel or Universal binary? (I have GTK2 with Gnome on Linux i386) Normal I guess. These are the configure options I use: --disable-acl --disable-gpm --disable-nls --disable-netbeans --disable-xsmp --disable-xsmp-interact --with-x=no --enable-gui=no Hm, yes; IIUC this would build a Normal version without GUI and without X11 (i.e. no access to the clipboard). So I suppose there is something in my Huge GTK2/Gnome2 gvim (with, as it turns out, all interpreters except MzScheme) which turns on those two large-file defines, and later on on the gcc command-line also -D_LARGEFILE64_SOURCE=1 Best regards, Tony.
Re: Tags break when enabling large file support
Yakov Lerner wrote: Surprisingly, changing ftell() to ftello() does not change anything. On systems that support ftello() ftell() is defined to use ftello(). Casting ftell() to (off_t) doesn't change anything. Funnily, casting ftell to (long) makes the problem go away (as it ftell does not already return long, it does): 1628 search_info.curr_offset = (long)ftell(fp); 1644 search_info.curr_offset = (long)ftell(fp); 1651 search_info.match_offset = (long)ftell(fp); This won't work correctly when off_t is bigger than long, e.g. long long. New developments: It looks like -D_FILE_OFFSET_BITS=64 alone is wrong because it's not complete. Correct build settings are: gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE This makes problem go away without any changes in the sources. So gcc has an option to break your code? That can't be right. 3. Thou shalt not use Google as a verb. http://www.thechurchofgoogle.org/Scripture/10_Commandments.html One of the (many) false statements found on the internet. Google objects to using googling for searching in general, it's OK to use for searching with Google. -- Veni, Vidi, VW -- I came, I saw, I drove around in a little car. /// Bram Moolenaar -- [EMAIL PROTECTED] -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\download, build and distribute -- http://www.A-A-P.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org///
