On Sunday, May 8, 2016 at 8:50:30 PM UTC+3, Ramel Eshed wrote: > On Sunday, May 8, 2016 at 2:26:05 PM UTC+3, Bram Moolenaar wrote: > > Ramel Eshed wrote: > > > > > On Sunday, May 8, 2016 at 8:59:24 AM UTC+3, Bram Moolenaar wrote: > > > > Ramel Eshed wrote: > > > > > > > > > Thanks to Bram and his recent work on channels, I have a preliminary > > > > > version of an asynchronous grep plugin which lets you work with the > > > > > available results while grep is still running. I would like to take > > > > > advantage of this relatively short script to raise some issues and > > > > > thoughts I have. The plugin can be found at > > > > > https://github.com/ramele/agrep. > > > > > > > > > > My original idea was to use the quickfix list and add each match to it > > > > > from the out callback. I had some issues with it, so temporarily (I > > > > > hope..) I’m using my own window to display the results (“agrep > > > > > window”). This is the default now and it should demonstrate what this > > > > > plugin should do eventually. In order to reproduce the quickfix > > > > > issues, you'll need to tell agrep to use quickfix list instead of > > > > > agrep window (:let qgrep_use_qf=1). You can install the plugin as is > > > > > or load it with $vim -u NONE -N -S <path to agrep.vim> > > > > > > > > > > 1. Stability: > > > > > Currently (7.4.1819) when :Agrep returns high number of results > > > > > (~2800. So we get similar number of callbacks in a time frame of ~2:45 > > > > > seconds) Vim crashes with: > > > > > > > > > > Caught deadly signal SEGV > > > > > Vim: Finished. > > > > > Segmentation fault (core dumped) > > > > > > > > What system are you using? If on Windows, what compiler? > > > > > > > This plugin is for Linux. I'm using Ubuntu 16.04 (and tested it also on > > > RHEL 5.5). > > > > > > > > This is more likely to happen when using the quickfix list. But it > > > > > happens sometimes with agrep window as well. I never see this when I > > > > > change the out_cb to only process the msg without adding the results > > > > > to neither agrep window or qf list (-with lines 108-118 commented > > > > > out). > > > > > > > > > > 2. Quickfix issues > > > > > a. As long as there are not too many matches and the quickfix > > > > > window is closed you can fire up a search and start jumping to the > > > > > available matches while the search is still running using the quickfix > > > > > navigation commands (:cn, for example). The problem is when the > > > > > quickfix window is opened while the search is active – here there is a > > > > > serious performance issue, and Vim hangs for a while. You can see this > > > > > by running :copen and then :Agrep. Choose a search that will generate > > > > > many results (> 1000). > > > > > > > > You could use the channel log to find out where Vim spends time. You > > > > may need to add more log statements in the code and recompile. > > > > > > > I think that this is a quickfix issue rather than channels. I don't see > > > that problem when running with agrep window or even with quickfix without > > > the qf window opened (there is also the other issue (see #b below) but > > > this one is much more serious). The channels code does exactly the same, > > > no matter which mode we're using to display the results. > > > > > > > > b. As the matches rate becomes higher, Vim will become > > > > > unresponsive. It may happen for short intervals or it can hangs for a > > > > > while (for very high rate). By “rate” I simply mean to > > > > > number_of_matches / sec. To compare, the performance is much better > > > > > with the agrep window. > > > > > > > > > > As I understand (and please correct me if I’m wrong) when using > > > > > setqflist() to create a new list or add items to the current list, Vim > > > > > reads all the buffers (as an unlisted buffers) in order to set the > > > > > hidden marks. I think that this is the reason of the slowness I’m > > > > > experiencing. If I'm right, maybe the way Vim loads the buffers of the > > > > > qf list should be changed so buffers will be loaded only when they're > > > > > been accessed and not when they're added to the list. What do you > > > > > think? > > > > > > > > Then it would also be slow if you add all the matches at once. Does > > > > that happen? > > > > > > > You're right, I forgot to mention that. It is noticeable also when adding > > > many results at once (you can see this by running :Agrepsetqf after you > > > have a long list of results). > > > > > > > > 3. Changing other buffers: > > > > > As far as I know, there is no nice way to change buffers other than > > > > > the current one. There is the getbufline() function, but the symmetry > > > > > is broken since there is no setbufline(). I found a discussion about > > > > > this from 10 years ago > > > > > (http://vim.1045645.n5.nabble.com/missing-setbufline-td1155970.html) > > > > > but I'm not sure if Bram added it to his TODO list or not. In this > > > > > script I'm using the move_to_buf() and back_from_buf() functions. The > > > > > problems with this approach are: > > > > > > > > > > - It is ugly. > > > > > - Poor performance (it is slow, you can see the cursor jumping > > > > > between windows, it can cause Vim to hang). > > > > > - Difficult to change hidden buffers or buffers in other tab pages. > > > > > - Is has side effects - It seems to break my undo tree and I'm sure > > > > > that there are more. > > > > > > > > > > I'm not very much experienced with external plugins (like Python) > > > > > which can change any buffer, but it seems that when using a channel > > > > > with "out_io": "buffer" the performance is much better. Is it possible > > > > > to implement such a function based on how it's done there? > > > > > > > > > > I think I'll stop here for now. I'm looking forward to get your > > > > > feedback. > > > > > > > > Thanks. I'm quite busy the coming days, but perhaps someone else can > > > > have a go at it. > > > > > > > Thanks Bram! > > > > I fixed two related problems: processes hanging in "defunc" state and > > not redirecting stdout to /dev/null. Probably won't have time to check > > out the other problems this week. > > > > -- > > A M00se once bit my sister ... > > "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES > > LTD > > > > /// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \\\ > > /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ > > \\\ an exciting new programming language -- http://www.Zimbu.org /// > > \\\ help me help AIDS victims -- http://ICCF-Holland.org /// > > It didn't help. I still get the SEGV error.
Hi Bram, I'm not sure what exactly you've done in the last few patches, but it seems like the SEGV issue is solved now, Thanks. Unfortunately, I have a different issue now when using the gui; Sometimes, when I perform a large search (~9000 results) Vim hangs and never restored. I enabled the channel log and I see the following Netbeans messages even though I'm not using the Netbeans protocol: ERR on 1: messageFromNetbeans(): Cannot read from channel, will close it soon When running Vim in a terminal everything works fine, but for some reason I still see errors in the channel's log. The terminal error is: ERR on 1: channel_select_check(): Cannot read from channel, will close it soon Any idea what went wrong? The gui issue doesn't seem deterministic, so I can't give a way to reproduce it, but you can download my plugin and play with it a bit (It takes me no more than 10 searches in the linux kernel code base until I see the issue). Thanks, Ramel -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
