Ramel Eshed wrote: > > > 1) Actually, there is a bug here which is not exactly what I've > > > described earlier. The issue is that :call setqflist([]), instead of > > > adding one more list after the last list, will clear the next list and > > > delete the ones after. For example: let's say I used :grep 4 times so > > > I have now 4 lists. Now, if I do :colder 3, the first list becomes the > > > current list. :call setqflist([]) will empty list 2, and delete lists > > > 3 and 4. > > > > We do remove newer lists when adding a new list before the end, but in > > this case it should not happen. > I'm not sure I understand. Is removing all the newer lists are the > expected behavior? why? and why is it different here?
Removing newer lists is what happens when inserting a list. But you are not inserting here, you are clearing it. > > > 2) Although quickfix performance got much better, I'm afraid there is > > > more to do: > > > a) It takes me about 9 seconds(!) to add 68,505 entries to the qf > > > list (using simple :grep command. Time was measured from after the > > > grep command output finished, of course). Yegappan's test takes > > > only 2 seconds because all the results are from the same file. Try > > > to add the loop variable to the file name in his test and see what > > > happens... > > > > > > This is the profiler log of the search I did: > > > > > > % cumulative self self total > > > time seconds seconds calls ms/call ms/call name > > > 40.63 1.69 1.69 68510 0.02 0.02 buf_valid > > > 21.88 2.60 0.91 68505 0.01 0.02 > > > buflist_findname_stat > > > 12.38 3.12 0.52 127664549 0.00 0.00 otherfile_buf > > > 4.81 3.32 0.20 4345 0.05 0.05 buflist_findnr > > > > Well, when adding an entry a buffer is created, so that we keep track of > > the actual name when doing ":cd". And it avoids duplicates. > > buf_valid() is often needed to check if an autocommand didn't delete a > > buffer. Both of these are hard to get rid of. > This can really slow down Vim. I'm not sure if my test cases here are > realistic, but I'm sure that if you keep Vim session active for enough > time you'll probably get thousands of unlisted buffers which will slow > down everything. > In this case, if there is nothing else can be done (like using another > data structure for qf entries) I think that command for deleting > quickfix list is necessary. As I mentioned earlier, :call > setqflist([], 'r') is not good enough and it also doesn't free the > associated buffers. > Maybe a simple mechanism for search, which does not use hidden marks, > should be considered. When you're starting a search command you don't > really know how many results you'll get so the last thing you want to > do is to add 100000 results to the qf list. You need to be able to > review the results, maybe filter them, and then add them to the > quickfix if needed (in many cases it is not needed, like when you only > want to review the code without changing anything). Actually, this is > what I've done in the Agrep plugin.. The only way to avoid creating buffers is by intentionally dropping some functionality and only keeping the file name. Perhaps we can still expand it to a full path (that's also not a cheap operation, but at least it's only once per entry). The file name would be turned into a buffer only when jumping to the location then. I suppose this is a property of the quickfix list. Perhaps with a function like "setqfflag()"? Would not work with commands like :cfile though. > > > b) I remember you did 2 things in order to avoid the line > > > adjustments when it's not necessary: check first if the buffer has a > > > quickfix entry and don't call line adjustment when adding a line at > > > the end. > > > > > > I still see that adding to a buffer when there are many qf entries > > > is very slow. After I had the 68505 results of the :grep command, > > > adding ~84000 lines to a buffer (from channel output) became really > > > slow. This is the profiler log: > > > > How are the lines added? > The lines are added by using 'out_io': 'buffer'. Without having many > (4000) unlisted buffers adding 80000 lines takes about 1.2 seconds. > The unlisted buffers slow this down to ~6 seconds. If, in addition, I > add an out_cb to this job which calls setbufvar() it can take even 15 > seconds. OK, so it's not adding the lines that's slow, but checking every time whether the buffer pointer we have is still valid. > > > Each sample counts as 0.01 seconds. > > > % cumulative self self total > > > time seconds seconds calls s/call s/call name > > > 31.46 6.98 6.98 149100 0.00 0.00 buf_valid > > > 29.20 13.46 6.48 80590 0.00 0.00 buflist_findpat > > > 26.66 19.38 5.92 84950 0.00 0.00 buflist_findnr > > > 3.56 20.17 0.79 68507 0.00 0.00 > > > buflist_findname_stat > > > 2.28 20.67 0.51 127673186 0.00 0.00 otherfile_buf > > > > > > It seems like checking the buffer each time alone takes a lot of time. > > > Is there any way to optimize the above functions? > > > Also, why I still see all these buffer functions even though all the > > > lines were added at the end of the buffer? > > > > If you use 'errorformat' it will locate the file name and find out what > > buffer has that file name. > > > I'm not using this in this case. Ehm, you do get those file names added as buffers, thus that must happen somewhere. Anyway, it seems that your problem is that you use thousands of buffers and that's something Vim wasn't prepared for. -- hundred-and-one symptoms of being an internet addict: 80. At parties, you introduce your spouse as your "service provider." /// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org /// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
