Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
skywind3000 wrote： > Bram Moolenaar wrote: > > > Skywind wrote: > > > > > Bram Moolenaar wrote： > > > > > This will not break the plugins, and will make ch_read better. > > > > > It could be used in a timer for congestion control manually. > > > > > > > > That changes what you get back, that is a bit strange. > > > > How about adding a function that returns whether there is something to > > > > read? Could call it ch_canread(). > > > > > > > > > > I think "ch_canread()" is a nice design ! > > > > Please try out patch 8.0.0103, it should no longer get stuck. > > > > After trying 8.0.0103 several times on windows/mac/debian(via ssh), > I believe stuck issue is fixed. > > however, cpu usage is still high with this test (exceeds 92%), and I can feel > the lag between a keystroke and screen update, especially through a ssh > terminal. > > At first I suppose my network bandwidth is fully filled with UI/quickfix > update messages. After removing the ":cbottom" lines, I find vim is still > very busy (cpu usage exceeds 90%) even if UI is not updated in the job > callback. -- I just tried "ch_read()" in a 10Hz timer without any callback (both "callback", "out_cb" and "err_cb" is unset) and :caddexpr what I readed from ch_read() to quickfix. I got these on quickfix window: --- || [python output] benchjob|8328| this is line 8328 benchjob|19927| this is line 19927 benchjob|31094| this is line 31094 benchjob|42038| this is line 42038 benchjob|53045| this is line 53045 benchjob|64103| this is line 64103 benchjob|75220| this is line 75220 It appears many data have been missed when I am calling ch_read. I suppose I can read all the data, one by one from ch_read if I don't set the callbacks. -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Bram Moolenaar wrote: > Skywind wrote: > > > Bram Moolenaar wrote： > > > > This will not break the plugins, and will make ch_read better. > > > > It could be used in a timer for congestion control manually. > > > > > > That changes what you get back, that is a bit strange. > > > How about adding a function that returns whether there is something to > > > read? Could call it ch_canread(). > > > > > > > I think "ch_canread()" is a nice design ! > > Please try out patch 8.0.0103, it should no longer get stuck. > After trying 8.0.0103 several times on windows/mac/debian(via ssh), I believe stuck issue is fixed. however, cpu usage is still high with this test (exceeds 92%), and I can feel the lag between a keystroke and screen update, especially through a ssh terminal. At first I suppose my network bandwidth is fully filled with UI/quickfix update messages. After removing the ":cbottom" lines, I find vim is still very busy (cpu usage exceeds 90%) even if UI is not updated in the job callback. -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Skywind wrote: > Bram Moolenaar wrote： > > > This will not break the plugins, and will make ch_read better. > > > It could be used in a timer for congestion control manually. > > > > That changes what you get back, that is a bit strange. > > How about adding a function that returns whether there is something to > > read? Could call it ch_canread(). > > > > I think "ch_canread()" is a nice design ! Please try out patch 8.0.0103, it should no longer get stuck. -- hundred-and-one symptoms of being an internet addict: 51. You put a pillow case over your laptop so your lover doesn't see it while you are pretending to catch your breath. /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Bram Moolenaar wrote： > > This will not break the plugins, and will make ch_read better. > > It could be used in a timer for congestion control manually. > > That changes what you get back, that is a bit strange. > How about adding a function that returns whether there is something to > read? Could call it ch_canread(). > I think "ch_canread()" is a nice design ! -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Skywind wrote:
> What about provide an option "nl" in ch_read to make it better ?
>
> :call ch_read(mych, {"timeout": 0, "nl":1})
>
> returns "abc\n" for a line "abc"
> returns "\n" for an empty line
> returns "" for not enough data
>
> This will not break the plugins, and will make ch_read better.
> It could be used in a timer for congestion control manually.
That changes what you get back, that is a bit strange.
How about adding a function that returns whether there is something to
read? Could call it ch_canread().
--
The real
trick is
this: to
keep the
lines as
short as
possible
and keep
the size
the same
yet free
from the
need for
hyphena-
Dammit!! (Matthew Winn)
/// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Skywind wrote:
> Thanks for the patch,
>
> I wrote a test :
> (benchjob.vim for starting job, benchjob.py for printing 8 lines)
>
> ---
> benchjob.vim:
>
> function! MyCallback(job, text)
> " make cpu a little busy
> for i in range(100)
> for j in range(10)
> cbottom
> endfor
> endfor
> let text = iconv(a:text, "gbk", )
> caddexpr a:text
> cbottom
> endfunc
>
> copen 8
> cexpr "[python output]"
> wincmd k
>
> let cmd = ['/usr/bin/python', 'benchjob.py']
> let opt = {"out_cb": "MyCallback", "out_io": "pipe", "err_io": "out"}
>
> let job = job_start(cmd, opt)
>
>
>
> --
> benchjob.py:
>
> #! /usr/bin/env python2
> import sys
>
> for i in xrange(8):
> sys.stdout.write('benchjob:%s: this is line %d\n'%(i, i))
> sys.stdout.flush()
>
>
> I can move cursor with the latest version (8.0.101), gui doesn't freeze as
> the older versions. But I also find some issues:
>
> 1. Although I can move cursor while benchjob.py is running, but I
> nearly can't input any thing, when I type a single character in insert
> mode, I can see the character appears in the screen, cursor moves
> right and then cursor rewinds to the head of the line, and then redraw
> the whole line. (delete the whole line at first, then display them
> again). It seems gui is very busy, especially accessing a remote vim
> from terminal.
It's hard to avoid this, since Vim is indeed 100% busy.
> 2. After a while, the output in quickfix stopped (benchjob.py isn't finished
> yet), until I move cursor or type something in vim, and it resumes and
> updates a few lines in quickfix and stops again until I type something again.
> If I don't touch my keyboard, the the output will pause forever.
That sounds like a bug. I suppose that when no key is available the
loop doesn't go back to checking for messages.
--
hundred-and-one symptoms of being an internet addict:
48. You get a tatoo that says "This body best viewed with Netscape 3.1 or
higher."
/// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
What about provide an option "nl" in ch_read to make it better ?
:call ch_read(mych, {"timeout": 0, "nl":1})
returns "abc\n" for a line "abc"
returns "\n" for an empty line
returns "" for not enough data
This will not break the plugins, and will make ch_read better.
It could be used in a timer for congestion control manually.
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Thanks for the patch,
I wrote a test :
(benchjob.vim for starting job, benchjob.py for printing 8 lines)
---
benchjob.vim:
function! MyCallback(job, text)
" make cpu a little busy
for i in range(100)
for j in range(10)
cbottom
endfor
endfor
let text = iconv(a:text, "gbk", )
caddexpr a:text
cbottom
endfunc
copen 8
cexpr "[python output]"
wincmd k
let cmd = ['/usr/bin/python', 'benchjob.py']
let opt = {"out_cb": "MyCallback", "out_io": "pipe", "err_io": "out"}
let job = job_start(cmd, opt)
--
benchjob.py:
#! /usr/bin/env python2
import sys
for i in xrange(8):
sys.stdout.write('benchjob:%s: this is line %d\n'%(i, i))
sys.stdout.flush()
I can move cursor with the latest version (8.0.101), gui doesn't freeze as the
older versions. But I also find some issues:
1. Although I can move cursor while benchjob.py is running, but I nearly can't
input any thing, when I type a single character in insert mode, I can see the
character appears in the screen, cursor moves right and then cursor rewinds to
the head of the line, and then redraw the whole line. (delete the whole line at
first, then display them again). It seems gui is very busy, especially
accessing a remote vim from terminal.
2. After a while, the output in quickfix stopped (benchjob.py isn't finished
yet), until I move cursor or type something in vim, and it resumes and updates
a few lines in quickfix and stops again until I type something again. If I
don't touch my keyboard, the the output will pause forever.
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to vim_dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
Skywind wrote: > The callback of "out_cb" is convenience to use, but has performance issue: > Gui gets freezed if the background job continues outputing massive data (eg, > grep a high frequency word "to" on the documents root, or some crazy stl > errors). > > Vim is busy in receiving the data and invoking "out_cb" which cause gui no > responsed for a very noticeable period. > > The practice way to prevent gui freeze is introducing a flow control for > background process by reading the fixed number of lines from the channel on a > timer (eg, read 50 lines from stdout each 100ms interval). > > If the grep output is faster then the reader in the timer, system pipe will > be full and "invoking write()" will block the child process until the system > pipe buffer has free space again (some data have been readed out by vim). > > In order to process the job output more smoothly, I tried to use this > approach but when I am ready to read data from the channel in a timer, It is > confused that both "\n" (empty line) and "not enough data" return "" from > ch_read(). > > If I want to read out at most 100 lines from a channel in an interval, how > can I tell if there is no data at the moment or it's just a "\n" from child > process ? > > No clue to decide whether to continue reading or just break out. > > When I am using "(pipe object).readline()" from python's subprocess module, > "abc\n" means a line "abc", "\n" means an empty line and "" means not enough > data. It's very simple without any ambiguity. > > But when I am using "ch_read()" in vim. "abc" means a line "abc", and both > (empty line) and (no message) return "". > > Is it a "logic fallacy" of "ch_read()" ?? Can you please try out 8.0.0097 using out_cb and check that it works better? -- The future isn't what it used to be. /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: "ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
skywind3000 wrote： > The callback of "out_cb" is convenience to use, but has performance issue: > Gui gets freezed if the background job continues outputing massive data (eg, > grep a high frequency word "to" on the documents root, or some crazy stl > errors). > > Vim is busy in receiving the data and invoking "out_cb" which cause gui no > responsed for a very noticeable period. > > The practice way to prevent gui freeze is introducing a flow control for > background process by reading the fixed number of lines from the channel on a > timer (eg, read 50 lines from stdout each 100ms interval). > > If the grep output is faster then the reader in the timer, system pipe will > be full and "invoking write()" will block the child process until the system > pipe buffer has free space again (some data have been readed out by vim). > > In order to process the job output more smoothly, I tried to use this > approach but when I am ready to read data from the channel in a timer, It is > confused that both "\n" (empty line) and "not enough data" return "" from > ch_read(). > > If I want to read out at most 100 lines from a channel in an interval, how > can I tell if there is no data at the moment or it's just a "\n" from child > process ? > > No clue to decide whether to continue reading or just break out. > > When I am using "(pipe object).readline()" from python's subprocess module, > "abc\n" means a line "abc", "\n" means an empty line and "" means not enough > data. It's very simple without any ambiguity. > > But when I am using "ch_read()" in vim. "abc" means a line "abc", and both > (empty line) and (no message) return "". > > Is it a "logic fallacy" of "ch_read()" ?? btw, the "ch_read()" referenced before is invoked with "timeout=0" in "NL" mode. -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
"ch_read()" couldn't tell the difference between "not enough message" and "a empty line"
The callback of "out_cb" is convenience to use, but has performance issue: Gui gets freezed if the background job continues outputing massive data (eg, grep a high frequency word "to" on the documents root, or some crazy stl errors). Vim is busy in receiving the data and invoking "out_cb" which cause gui no responsed for a very noticeable period. The practice way to prevent gui freeze is introducing a flow control for background process by reading the fixed number of lines from the channel on a timer (eg, read 50 lines from stdout each 100ms interval). If the grep output is faster then the reader in the timer, system pipe will be full and "invoking write()" will block the child process until the system pipe buffer has free space again (some data have been readed out by vim). In order to process the job output more smoothly, I tried to use this approach but when I am ready to read data from the channel in a timer, It is confused that both "\n" (empty line) and "not enough data" return "" from ch_read(). If I want to read out at most 100 lines from a channel in an interval, how can I tell if there is no data at the moment or it's just a "\n" from child process ? No clue to decide whether to continue reading or just break out. When I am using "(pipe object).readline()" from python's subprocess module, "abc\n" means a line "abc", "\n" means an empty line and "" means not enough data. It's very simple without any ambiguity. But when I am using "ch_read()" in vim. "abc" means a line "abc", and both (empty line) and (no message) return "". Is it a "logic fallacy" of "ch_read()" ?? -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
