Re: 6k buffer bug in shells
Chet Ramey wrote: There have been several interesting discussions on this topic, most centered on Linux: https://groups.google.com/d/msg/linux.kernel/PYgS2MyNQfw/orRVLXtinOwJ http://lists.gnu.org/archive/html/bug-readline/2012-07/msg4.html http://lists.gnu.org/archive/html/bug-bash/2012-04/msg00065.html http://lists.gnu.org/archive/html/bug-readline/2013-07/msg00013.html Seems most of those were 5-6 years ago. Might explain why I can't reproduce it, though I'm pasting into a remote TTY window in each case. (one SecureCRT, other xterm) -- w/about 290899 chars pasted. Only case I've seen of pasted chars being eaten is if the source code was indented with TAB... then various autocomplete prompts swallow chars randomly based on number of matching autocomplete entries. including the following comment: "So, after quite a lot of extra hours spent on this, we were able to track MOST of the breakage to this commit: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f23eb2b2b28547fc70df82dd5049eb39bec5ba12; Margarita Manterola and Maximiliano Curia did most of the work; here is the final summary of their findings (the subsequent thread is interesting as well): https://lkml.org/lkml/2013/7/25/205 The question is how to deal with large amounts of data in a situation where the tty modes keep getting changed after each newline, while preserving acceptable behavior for the usual case of cooked mode. Chet
Re: 6k buffer bug in shells
On 2/2/18 3:19 PM, Chet Ramey wrote: > On 2/2/18 9:50 AM, Sten Westerback wrote: >> Hi >> >> After trying to figure out what causes a weird bug, which i can see with >> lots of shells and connection types (local terminal on linux, ssh + >> numerous shells on AIX and putty (linux & windows)+numerous shells on AIX) >> i though i would ask makers of my favorite shell. There have been several interesting discussions on this topic, most centered on Linux: https://groups.google.com/d/msg/linux.kernel/PYgS2MyNQfw/orRVLXtinOwJ http://lists.gnu.org/archive/html/bug-readline/2012-07/msg4.html http://lists.gnu.org/archive/html/bug-bash/2012-04/msg00065.html http://lists.gnu.org/archive/html/bug-readline/2013-07/msg00013.html including the following comment: "So, after quite a lot of extra hours spent on this, we were able to track MOST of the breakage to this commit: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f23eb2b2b28547fc70df82dd5049eb39bec5ba12; Margarita Manterola and Maximiliano Curia did most of the work; here is the final summary of their findings (the subsequent thread is interesting as well): https://lkml.org/lkml/2013/7/25/205 The question is how to deal with large amounts of data in a situation where the tty modes keep getting changed after each newline, while preserving acceptable behavior for the usual case of cooked mode. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: 6k buffer bug in shells
On 2/2/18 9:50 AM, Sten Westerback wrote: > Hi > > After trying to figure out what causes a weird bug, which i can see with > lots of shells and connection types (local terminal on linux, ssh + > numerous shells on AIX and putty (linux & windows)+numerous shells on AIX) > i though i would ask makers of my favorite shell. > > The problem is that when i paste in longer texts instead of just typing > the result isn't as expected. It seems that after a certain amount of > characters the shell drops a few characters but then continues. The number > of characters dropped depends on the line with (with 50 it's 4, with 64 it > was 12 etc). And with slightly larger amounts of text there is also a > pause before it continues while one would expect the flow to be continous > with a simple task like this, It's not quite as simple as you think. If you're pasting into an interactive shell without turning off line editing, you have to deal with the kernel turning character-at-a-time mode on and off for each line (and the single-character reads line editing requires). In the past, we've found that doing this causes characters to be discarded. This is part of the problem that `bracketed paste' mode was intended to address. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
6k buffer bug in shells
Hi After trying to figure out what causes a weird bug, which i can see with lots of shells and connection types (local terminal on linux, ssh + numerous shells on AIX and putty (linux & windows)+numerous shells on AIX) i though i would ask makers of my favorite shell. The problem is that when i paste in longer texts instead of just typing the result isn't as expected. It seems that after a certain amount of characters the shell drops a few characters but then continues. The number of characters dropped depends on the line with (with 50 it's 4, with 64 it was 12 etc). And with slightly larger amounts of text there is also a pause before it continues while one would expect the flow to be continous with a simple task like this, Here is a simple file which when copied to clipboard and then pasted into a shell should produce the obvious text in /tmp/chartestRESULT.txt... but it just doesn't always. Here a result file sample with error shown at row starting with 3JJJ and 5WWW :: I have tried workarounds such as: - breaking into 6 k blocks to append to file - replacing cat >> file .. << homedocwith echo " ... \n ... \n ... \n" >> file - term=dumbwhich i first thought fixed it.. but later problem appeared. I didn't yet test to change terminal on all ssh levels. It seems that the shell or something involved, such as clipboard functionality or ssh protocol, doesn't like to be flooded by a couple of MB of text. Hope you can point me towards a fix or work-around. What i'm actually attempting is pasting in of Base64 encoded data, broken into 64 byte rows, as a simple way to transfer small binary files from a workstation to a host that is behind a longer chain of ssh sessions. This works perfectly for smaller files but then i noticed this problem. Here example of failing rows from row 140 forward: Input MUEOSDFqQMHIwCY9GQwKx2iiFQ/CFQb7d/pH0IWmdqs9PEHr4ZNPBnt4x9AzWl0ijz9nPlG7d1hd2MFZ /fpgXXdTXS8/D4yxCq0++IvzZJEdPKHjm4zaOdGY8fimhUbMngxadrEfp8rsQmP51/AFgujz8GdCu4DZ Rk60GmzrIBuZ8OODeOY2z+jZE01xEj5hYvl0FE0m8HLlKS2I5Ju9xWbWdCf0qXw9blL5XGb2/XvavTWT HRIJdtsKFK5jMUaTfDMPJidvp2Di6omIJc3Q8V/bisv9Juj3jAKBDZR+oc2SR6+C+MTqyzfMpWvTNp9P Result: MUEOSDFqQMHIwCY9GQwKx2iiFQ/CFQb7d/pH0IWmdqs9PEHr4ZNPBnt4x9AzWl0ijz9nPlG7d1hd2MFZ /fpgXXdTXS8/D4yxCq0++IvzZJEdPKHjm4zaOdGY8fimhUbMngxadrEfp8rsQmP51/AFgujz8GdCu4DZ Rk60GmzrIBuZ8OOD HRIJdtsKFK5jMUaTfDMPJidvp2Di6omIJc3Q8V/bisv9Juj3jAKBDZR+oc2SR6+C+MTqyzfMpWvTNp9P So something dropped the eOY2z+jZE01xEj5hYvl0FE0m8HLlKS2I5Ju9xWbWdCf0qXw9blL5XGb2/XvavTWT In this case after 141*81 + 64 = 11485 characters on 142nd row. Environment: 1: Redhat linux 2.6.32-696.6.3.el6.x86_64 #1 SMP Fri Jun 30 13:24:18 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux GNU Bash-4.1 GNOME Terminal 2.31.3 Environment 2: AIX 7.1.5.0 TL05 Putty Development snapshot 2017-06-21.d618974 (Windows 64 bit) - Sten Ellei edellä ole toisin mainittu: / Unless stated otherwise above: Oy IBM Finland Ab PL 265, 00101 Helsinki, Finland Business ID, Y-tunnus: 0195876-3 Registered in Finland cat > /tmp/chartestRESULT.txt << THEEND 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2
