Re: A performance question (patch included)

2007-05-26 Thread Yakov Lerner
On 5/25/07, Charles E Campbell Jr [EMAIL PROTECTED] wrote: John Beckett wrote: A.J.Mechelynck wrote: What about a different function to return, say, the number of 1K blocks (or the number of times 2^n bytes, with a parameter passed to the function) that a file uses? Yes, that's a much

Re: A performance question (patch included)

2007-05-25 Thread Charles E Campbell Jr
John Beckett wrote: A.J.Mechelynck wrote: What about a different function to return, say, the number of 1K blocks (or the number of times 2^n bytes, with a parameter passed to the function) that a file uses? Yes, that's a much more general and better idea. Since there's probably not much

Re: A performance question (patch included)

2007-05-25 Thread Charles E Campbell Jr
A.J.Mechelynck wrote: I'm not sure what varnumber_T means: will st.stsize (the dividend) be wide enough to avoid losing bits on the left? varnumber_T is int (long if an sizeof(int) = 3). st.stsize 's size depends on whether 32bit or 64bit integers are available. So, its possible to lose

Re: A performance question (patch included)

2007-05-25 Thread Charles E Campbell Jr
A.J.Mechelynck wrote: Yes, yes, but before the division, will it be able to hold the file size? (sorry, I meant st.st_size) Will mch_stat (at line 10134, one line before the context of your patch) be able to return huge file sizes? mch_stat is variously defined, depending on o/s. Under

Re: A performance question (patch included)

2007-05-25 Thread A.J.Mechelynck
Charles E Campbell Jr wrote: A.J.Mechelynck wrote: I'm not sure what varnumber_T means: will st.stsize (the dividend) be wide enough to avoid losing bits on the left? varnumber_T is int (long if an sizeof(int) = 3). st.stsize 's size depends on whether 32bit or 64bit integers are available.

Re: A performance question (patch included)

2007-05-25 Thread A.J.Mechelynck
Yakov Lerner wrote: [...] stat() on Linux has 32-bit st_size field (off_t is 32-bit). There is stat64() syscall which uses 'struct stat64' structure where st_size is 64-bit. By defining __USE_LARGEFILE64 at compile-time, stat() is redirected to stat64(). I don't know whether default Linux vim

Re: A performance question (utility included)

2007-05-25 Thread John Beckett
Charles E Campbell Jr wrote: I've attached a patch to vim 7.1 which extends getfsize() As I've mentioned, I think further testing will be needed before patching Vim for 64-bit file lengths. Here is a possible interim workaround to allow Dr.Chip's LargeFile.vim script to accurately detect

Re: A performance question (patch included)

2007-05-25 Thread John Beckett
Charles E Campbell Jr wrote: I'm also under the impression that ls itself uses fstat(), so its not likely to be any more informative. That's likely on some systems, but 'ls -l' gives correct results for files over 4GB on Fedora Core 6 using x86-32. John

Re: A performance question

2007-05-25 Thread Yakov Lerner
On 5/25/07, John Beckett [EMAIL PROTECTED] wrote: A.J.Mechelynck wrote: What about a different function to return, say, the number of 1K blocks (or the number of times 2^n bytes, with a parameter passed to the function) that a file uses? Yes, that's a much more general and better idea.

Re: A performance question

2007-05-25 Thread Yakov Lerner
On 5/25/07, Yongwei Wu [EMAIL PROTECTED] wrote: On 24/05/07, Robert M Robinson [EMAIL PROTECTED] wrote: On Wed, 23 May 2007, fREW wrote: |Someone recently was emailing the list about looking at a small |section of DNA with vim as text and it was a number of gigs. I think |he ended up using

Re: A performance question

2007-05-25 Thread fREW
On 5/25/07, Yakov Lerner [EMAIL PROTECTED] wrote: On 5/25/07, Yongwei Wu [EMAIL PROTECTED] wrote: On 24/05/07, Robert M Robinson [EMAIL PROTECTED] wrote: On Wed, 23 May 2007, fREW wrote: |Someone recently was emailing the list about looking at a small |section of DNA with vim as text

Re: A performance question

2007-05-25 Thread Robert M Robinson
No, I implied vim has more uses than any one person could possibly imagine. I also meant any question like Why would anyone want ...? really just means I can't imagine wanting , so if that isn't what you meant to say you might want to rephrase your question. I would ask why anyone would

Re: A performance question

2007-05-25 Thread John Beckett
Yakov Lerner wrote: 9-digit number can still be larger than 2^32-1, or than 2^31-1. Just for the record: 2^30 = 1,073,741,824 So 999,999,999 (largest 9-digit number) won't overflow a 32-bit signed integer. John

Re: A performance question

2007-05-24 Thread John Beckett
Charles E Campbell Jr wrote: Sounds like the filesize is getting stored in a 32bit signed number, and overflowing. Yes, definitely. Please let me know what getfsize() is actually returning The return value is the bit pattern for the low 32 bits of the true 64-bit file size:

Re: A performance question

2007-05-24 Thread John Beckett
Yongwei Wu wrote: Even FAT32 supports files much larger than 4GB. Not true. FAT32 supports files up to 4 GB. Sorry I shot my mouth off there - I realised my blunder about ten minutes after sending. I haven't actually used a FAT32 partition for over ten years, and was confusing the maximum

Re: A performance question

2007-05-24 Thread John Beckett
panshizhu wrote: Yes, but on all systems, vim script could not take 64-bit integers I know that. My proposal is for a new Vim script function: islargefile({fname}, {limit}) which would return nonzero if the size of the file is greater than the 32-bit signed {limit} argument. Vim could

Re: A performance question

2007-05-24 Thread A.J.Mechelynck
John Beckett wrote: Charles E Campbell Jr wrote: Sounds like the filesize is getting stored in a 32bit signed number, and overflowing. Yes, definitely. Please let me know what getfsize() is actually returning The return value is the bit pattern for the low 32 bits of the true 64-bit file

Re: A performance question

2007-05-24 Thread A.J.Mechelynck
John Beckett wrote: Yongwei Wu wrote: Even FAT32 supports files much larger than 4GB. Not true. FAT32 supports files up to 4 GB. Sorry I shot my mouth off there - I realised my blunder about ten minutes after sending. I haven't actually used a FAT32 partition for over ten years, and was

Re: A performance question

2007-05-24 Thread John Beckett
A.J.Mechelynck wrote: What about a different function to return, say, the number of 1K blocks (or the number of times 2^n bytes, with a parameter passed to the function) that a file uses? Yes, that's a much more general and better idea. Since there's probably not much need for this, I think

Re: A performance question

2007-05-23 Thread Peter Palm
Op woensdag 23 mei 2007, schreef fREW: Another thing that might help with speed that was mentioned a month or so ago is the following script specifically aimed at increasing speed for large files: http://www.vim.org/scripts/script.php?script_id=1506. Indeed, among other things, this disables

Re: A performance question

2007-05-23 Thread John Beckett
Peter Palm wrote: http://www.vim.org/scripts/script.php?script_id=1506. Indeed, among other things, this disables the swap file for 'large' files, which should really speed up things. I was going to report the following issue to vim-dev after I got a chance to investigate it a little

Re: A performance question

2007-05-23 Thread panshizhu
John Beckett [EMAIL PROTECTED] 写于 2007-05-23 18:39:22: The result was really ugly. The script failed to notice that 3GB was large because the Vim function getfsize(f) returned a negative number. I haven't checked getfsize() on 32-bit Linux yet, nor am I sufficiently patient to try opening

Re: A performance question

2007-05-23 Thread John Beckett
panshizhu wrote: As far as I know, Windows does not support files larger than 4GB. So its okay to use unsigned 32-bit for filesize in windows. It's not as bad as that! Even FAT32 supports files much larger than 4GB. The Win32 API includes function _stati64() to get a 64-bit file size (the API

Re: A performance question

2007-05-23 Thread panshizhu
John Beckett [EMAIL PROTECTED] 写于 2007-05-23 19:32:25: On many systems, the calculation could use 64-bit integers. John Yes, but on all systems, vim script could not take 64-bit integers: see eval.txt line 38: 1.1 Variable types ~ *E712* There are

Re: A performance question

2007-05-23 Thread Charles E Campbell Jr
Robert M Robinson wrote: That brings me to my question. I have noticed that when editing large files (millions of lines), deleting a large number of lines (say, hundreds of thousands to millions) takes an unbelieveably long time in VIM--at least on my systems. This struck me as so odd, I

Re: A performance question

2007-05-23 Thread Charles E Campbell Jr
John Beckett wrote: Peter Palm wrote: http://www.vim.org/scripts/script.php?script_id=1506. Indeed, among other things, this disables the swap file for 'large' files, which should really speed up things. I was going to report the following issue to vim-dev after I got a chance to

Re: A performance question

2007-05-23 Thread Robert Maxwell Robinson
In that case, I'll have to thank Bram for fixing my problem before I even asked him to do so! Thanks Gary, when I get a chance I'll download vim 7. To those of you who provided links to work-around scripts etc., thank you for your help. If any of you are having trouble with large files I'd

Re: A performance question

2007-05-23 Thread A.J.Mechelynck
Robert Maxwell Robinson wrote: In that case, I'll have to thank Bram for fixing my problem before I even asked him to do so! Thanks Gary, when I get a chance I'll download vim 7. To those of you who provided links to work-around scripts etc., thank you for your help. If any of you are

Re: A performance question

2007-05-23 Thread panshizhu
Charles E Campbell Jr [EMAIL PROTECTED] 写于 2007-05-23 21:38:27: Sounds like the filesize is getting stored in a 32bit signed number, and overflowing. Is the negative number -1 (that would mean file can't be found)? If not, then perhaps that fact could be used to extend the LargeFile's

Re: A performance question

2007-05-23 Thread Yongwei Wu
On 23/05/07, John Beckett [EMAIL PROTECTED] wrote: panshizhu wrote: As far as I know, Windows does not support files larger than 4GB. So its okay to use unsigned 32-bit for filesize in windows. It's not as bad as that! Even FAT32 supports files much larger than 4GB. Not true. FAT32

Re: A performance question

2007-05-23 Thread panshizhu
Yongwei Wu [EMAIL PROTECTED] 写于 2007-05-24 11:28:06: Who really want to edit TEXT files as large as that? I cannot think of scenarios other than log files. Maybe Vim does not fit in this role. Best regards, Yongwei -- Yes it fits in this role, and frankly speaking this was the reason I

Re: A performance question

2007-05-23 Thread fREW
On 5/23/07, Yongwei Wu [EMAIL PROTECTED] wrote: On 23/05/07, John Beckett [EMAIL PROTECTED] wrote: panshizhu wrote: As far as I know, Windows does not support files larger than 4GB. So its okay to use unsigned 32-bit for filesize in windows. It's not as bad as that! Even FAT32 supports

A performance question

2007-05-22 Thread Robert M Robinson
First, thanks very much for creating VIM! I have been using it on Linux systems for years, and now use it via cygwin at home as well. I vastly prefer VIM to EMACS, especially at home. I learned vi on a VAX/VMS system long ago (a friend of mine had ported it), when our computer science

Re: A performance question

2007-05-22 Thread A.J.Mechelynck
Robert M Robinson wrote: First, thanks very much for creating VIM! I have been using it on Linux systems for years, and now use it via cygwin at home as well. I vastly prefer VIM to EMACS, especially at home. I learned vi on a VAX/VMS system long ago (a friend of mine had ported it), when

Re: A performance question

2007-05-22 Thread Tim Chase
That brings me to my question. I have noticed that when editing large files (millions of lines), deleting a large number of lines (say, hundreds of thousands to millions) takes an unbelieveably long time in VIM--at least on my systems. The issue of editing large files comes up occasionally. A

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
Thanks, Tim. I'll look at the options you recommended--and those you didn't, so I may not need to ask next time. :) Cheers, Max On Tue, 22 May 2007, Tim Chase wrote: The issue of editing large files comes up occasionally. A few settings can be tweaked to vastly improve performance.

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
Well, I don't mean to. :set says this: -- autoindent helplang=en scroll=11 t_Sb=Esc[4%dm backspace=2 history=50 ttyfast t_Sf=Esc[3%dm cscopetag hlsearchttymouse=xterm cscopeverbose ruler viminfo='20,50

Re: A performance question

2007-05-22 Thread Tim Chase
Do you have syntax highlighting enabled? That can really slow vim down. Well, I don't mean to. :set says this: It can be toggled via :syntax on and :syntax off To see what flavor of syntax highlighting you currently have, you can query the 'syntax' setting:

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
I just tried deleting 1133093 lines of a 1133093+1133409 line file, after typing :syntax off. It took about 3 minutes. Max On Tue, 22 May 2007, Tim Chase wrote: Do you have syntax highlighting enabled? That can really slow vim down. Well, I don't mean to. :set says this: It can be

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
:set syntax? replies syntax=. I don't think it's syntax highlighting. I've used that with C and Prolog code before; I gave it up because it was too slow. I'm editing text output from one of my programs; truncating the output of a day-long run to match a run in progress for testing purposes,

Re: A performance question

2007-05-22 Thread Andy Wokula
A.J.Mechelynck schrieb: Robert M Robinson wrote: First, thanks very much for creating VIM! I have been using it on Linux systems for years, and now use it via cygwin at home as well. I vastly prefer VIM to EMACS, especially at home. I learned vi on a VAX/VMS system long ago (a friend of

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
Thanks, Andy; the black hole register is a new idea to me. Unfortunately, :.,$d _ to the black hole register appears to take the same amount of time as :.,$d itself. set undolevels=-1 speeds it up, but set undolevels=0 does not; this suggests to me that the problem isn't related to how

Re: A performance question

2007-05-22 Thread Gary Johnson
On 2007-05-22, Robert Maxwell Robinson [EMAIL PROTECTED] wrote: :set undolevels=-1 caused my test to run in less than 15 sec, with no other options fiddled with. Thanks Tim, now I have a work-around! Now, does having the undo facility available _necessarily_ mean deleting a large

Re: A performance question

2007-05-22 Thread Robert Maxwell Robinson
Hmm, interesting. I've noticed before that the CPU is pegged when I'm deleting, but I don't think my machine's behavior is due to CPU load; the machine has two CPUs, I'm typically the only (serious) user, as top has confirmed is the case now, and I get the same behavior whether I'm running

Re: A performance question

2007-05-22 Thread Gary Johnson
On 2007-05-22, Robert Maxwell Robinson [EMAIL PROTECTED] wrote: Hmm, interesting. I've noticed before that the CPU is pegged when I'm deleting, but I don't think my machine's behavior is due to CPU load; the machine has two CPUs, I'm typically the only (serious) user, as top has

Re: A performance question

2007-05-22 Thread fREW
On 5/22/07, Gary Johnson [EMAIL PROTECTED] wrote: On 2007-05-22, Robert Maxwell Robinson [EMAIL PROTECTED] wrote: Hmm, interesting. I've noticed before that the CPU is pegged when I'm deleting, but I don't think my machine's behavior is due to CPU load; the machine has two CPUs, I'm

Re: A performance question

2007-05-22 Thread panshizhu
AFAIK Vim 7 has a different way of handling undo levels. Have you tried with Vim 6 instead? I had used Vim 6 to edit a text file (3Gbytes) and do things within seconds. -- Sincerely, Pan, Shi Zhu. ext: 2606 Robert Maxwell Robinson [EMAIL PROTECTED] 写于 2007-05-23 05:59:20: :set undolevels=-1