Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Thu, Nov 6, 2014 at 3:15 PM, Chet Ramey chet.ra...@case.edu wrote: There is no standard ipc except pipes across the range of systems bash runs on. thanks. good to know, before I open the flood gate of git commits ;-) can I assume that this includes named pipes? cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/07/2014 12:23 PM, Piotr Grzybowski wrote: On Thu, Nov 6, 2014 at 3:15 PM, Chet Ramey chet.ra...@case.edu wrote: There is no standard ipc except pipes across the range of systems bash runs on. thanks. good to know, before I open the flood gate of git commits ;-) can I assume that this includes named pipes? Named pipes (aka fifos) are notorious for being buggy on Cygwin and non-available on mingw. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Fri, Nov 7, 2014 at 1:26 PM, Eric Blake ebl...@redhat.com wrote: Named pipes (aka fifos) are notorious for being buggy on Cygwin and non-available on mingw. thanks Eric. good news. so this means, that shared history can only be implemented using regular files. is that right? cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Guys thanks so much for taking this discussion further than just the reported bug that prevented the existing workaround. I’ve often considered putting this work in myself but wasn’t sure what the reason was for this not being in bash already and if there was an architectural difficulty with implementing it. It sounds more like that it just wasn’t considered as an option rather than there is a problem implementing it. Given that ksh only ever worked this way (but its shared history performed flawlessly)l, I would think that the mechanism they used and particularly how they handled the locking is a good place to start with. GJ On 7 Nov 2014, at 11:29 pm, Piotr Grzybowski narsil...@gmail.com wrote: On Fri, Nov 7, 2014 at 1:26 PM, Eric Blake ebl...@redhat.com wrote: Named pipes (aka fifos) are notorious for being buggy on Cygwin and non-available on mingw. thanks Eric. good news. so this means, that shared history can only be implemented using regular files. is that right? cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Good point Graham, definitely good place to start. If you are interested we can discuss further, privately. Mainly because I want to say some nasty things about implementing sharing and locking resources withouth ipc. :)) cheers, pg On Fri, Nov 7, 2014 at 1:38 PM, Graham Jones your-name-h...@grahamjones.org wrote: Guys thanks so much for taking this discussion further than just the reported bug that prevented the existing workaround. I’ve often considered putting this work in myself but wasn’t sure what the reason was for this not being in bash already and if there was an architectural difficulty with implementing it. It sounds more like that it just wasn’t considered as an option rather than there is a problem implementing it. Given that ksh only ever worked this way (but its shared history performed flawlessly)l, I would think that the mechanism they used and particularly how they handled the locking is a good place to start with. GJ On 7 Nov 2014, at 11:29 pm, Piotr Grzybowski narsil...@gmail.com wrote: On Fri, Nov 7, 2014 at 1:26 PM, Eric Blake ebl...@redhat.com wrote: Named pipes (aka fifos) are notorious for being buggy on Cygwin and non-available on mingw. thanks Eric. good news. so this means, that shared history can only be implemented using regular files. is that right? cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Thu, Nov 6, 2014 at 3:28 AM, Chet Ramey chet.ra...@case.edu wrote: In theory, one could change the functions in history.c and histfile.c to change the in-memory history list to one that uses a file, possibly with mmap(). I'd be happy to look at contributed code to do this [..] would you accept a solution like this: 1. all running instances of bash with history support on, share history via shmem segment (some care as to its size should be taken) 2. $HISTFILE in every bash points to history file where every bash writes history in chronological (depending on settings) order 3. $HISTFILE is only read on startup, if there are no other bash instances running 4. sharing of history and synchronisation between instances is realised via standard ipc I have no idea how zsh does it, maybe it should be checked first. cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Thu, Nov 06, 2014 at 11:19:55AM +0100, Piotr Grzybowski wrote: would you accept a solution like this: 1. all running instances of bash with history support on, share history via shmem segment (some care as to its size should be taken) ... As long as it is off by default. This is the worst feature of ksh, and one of the primary reasons I switched from ksh to bash. Doing ESC k ENTER in ksh with an account to which multiple human beings have shared access (like root) is playing Russian roulette. You never know what you're going to end up executing. It's truly, truly terrible. (And yes, there are systems where root's default shell is ksh. AIX is one of them, or at least it was, back in 1993-1995 when I worked with it extensively.)
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/6/14, 5:19 AM, Piotr Grzybowski wrote: On Thu, Nov 6, 2014 at 3:28 AM, Chet Ramey chet.ra...@case.edu wrote: In theory, one could change the functions in history.c and histfile.c to change the in-memory history list to one that uses a file, possibly with mmap(). I'd be happy to look at contributed code to do this [..] would you accept a solution like this: 1. all running instances of bash with history support on, share history via shmem segment (some care as to its size should be taken) 2. $HISTFILE in every bash points to history file where every bash writes history in chronological (depending on settings) order You could use mmap, but the synchronization between different shells writing to the same mmaped segment is a problem that has to be solved. 3. $HISTFILE is only read on startup, if there are no other bash instances running 4. sharing of history and synchronisation between instances is realised via standard ipc There is no standard ipc except pipes across the range of systems bash runs on. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/4/14, 5:16 PM, Piotr Grzybowski wrote: Chet: for reasons unexplained calls to read_history_range at history.def:219 219 result = read_history_range (filename, history_lines_in_file, -1); return more and more records (77824 is above my HISTFILESIZE): 1: history_lines_in_file = 77824 the loop at histfile.c:269 in read_history_range is the one that hangs. code at commit ca6a2ba40c709c2b45a56e49d21d0dfc66e21974 Date: Sun Oct 5 19:12:20 2014 -0400 I still can't reproduce it on OS X 10.10: (1)$ ./bash --rcfile bashrc (2)$ echo $BASH_VERSION 4.3.30(22)-release (2)$ ls -l $HISTFILE -rw--- 1 chet staff 402 Nov 5 08:49 history-file (2)$ wc -l $HISTFILE 20 (2)$ echo $HISTSIZE $HISTFILESIZE 50 50 (2)$ for (( i = 0; i 30 ; i++ )); do history -a ; history -n ; done (2)$ wc -l $HISTFILE 25 (2)$ tail $HISTFILE ls ls exit PROMPT_COMMAND='history -a ; history -n' PROMPT_COMMAND='history -a ; history -n' echo $BASH_VERSION ls -l $HISTFILE wc -l $HISTFILE echo $HISTSIZE $HISTFILESIZE for (( i = 0; i 30 ; i++ )); do history -a ; history -n ; done (2)$ i=0; while [ $i -lt 30 ]; do history -a ; history -n; let i++; done (2)$ wc -l $HISTFILE 28 (2)$ tail $HISTFILE PROMPT_COMMAND='history -a ; history -n' PROMPT_COMMAND='history -a ; history -n' echo $BASH_VERSION ls -l $HISTFILE wc -l $HISTFILE echo $HISTSIZE $HISTFILESIZE for (( i = 0; i 30 ; i++ )); do history -a ; history -n ; done wc -l $HISTFILE tail $HISTFILE i=0; while [ $i -lt 30 ]; do history -a ; history -n; let i++; done There are issues with keeping track of the number of lines currently in the history file, but those occur at the boundary conditions where the number of commands in the history list approaches $HISTFILESIZE. That doesn't explain your results. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Wed, Nov 5, 2014 at 2:57 PM, Chet Ramey chet.ra...@case.edu wrote: I still can't reproduce it on OS X 10.10: [..] not only I cannot reproduce it at the moment, but also I have reached the following state (user: root): # cat .bash_history ls -al #1415144153 ls -al .bash_history #1415144154 pwd #1415144159 rm .bash_history #1415144161 cd #1415144163 pwd #1415144165 rm .bash_history #1415144171 i=0; while [ $i -lt 64 ]; do history -a; history -n; let i++; done; # ls -alh .bash_history -rw--- 1 root wheel 9.0M Nov 5 22:59 .bash_history # du -sh .bash_history 9.0M.bash_history # hexdump -C .bash_history 6c 73 20 2d 61 6c 20 0a 23 31 34 31 35 31 34 34 |ls -al .#1415144| 0010 31 35 33 0a 6c 73 20 2d 61 6c 20 2e 62 61 73 68 |153.ls -al .bash| 0020 5f 68 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 |_history .#14151| 0030 34 34 31 35 34 0a 70 77 64 0a 23 31 34 31 35 31 |44154.pwd.#14151| 0040 34 34 31 35 39 0a 72 6d 20 2e 62 61 73 68 5f 68 |44159.rm .bash_h| 0050 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 34 34 |istory .#1415144| 0060 31 36 31 0a 63 64 0a 23 31 34 31 35 31 34 34 31 |161.cd.#14151441| 0070 36 33 0a 70 77 64 0a 23 31 34 31 35 31 34 34 31 |63.pwd.#14151441| 0080 36 35 0a 72 6d 20 2e 62 61 73 68 5f 68 69 73 74 |65.rm .bash_hist| 0090 6f 72 79 20 0a 23 31 34 31 35 31 34 34 31 37 31 |ory .#1415144171| 00a0 0a 69 3d 30 3b 20 77 68 69 6c 65 20 5b 20 24 69 |.i=0; while [ $i| 00b0 20 2d 6c 74 20 36 34 20 5d 3b 20 64 6f 20 68 69 | -lt 64 ]; do hi| 00c0 73 74 6f 72 79 20 2d 61 3b 20 68 69 73 74 6f 72 |story -a; histor| 00d0 79 20 2d 6e 3b 20 6c 65 74 20 69 2b 2b 3b 20 64 |y -n; let i++; d| 00e0 6f 6e 65 3b 0a 00 00 00 00 00 00 00 00 00 00 00 |one;| 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 00902b60 which puzzles me a bit. Now I lost even all my while-loop commands! :) Graham: did you actually logout and removed the history file, and therefore started fresh? Sorry I guess I lost track of what is going on here. cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 6 Nov 2014, at 9:11 am, Piotr Grzybowski narsil...@gmail.com wrote: On Wed, Nov 5, 2014 at 2:57 PM, Chet Ramey chet.ra...@case.edu wrote: I still can't reproduce it on OS X 10.10: [..] not only I cannot reproduce it at the moment, but also I have reached the following state (user: root): # cat .bash_history ls -al #1415144153 ls -al .bash_history #1415144154 pwd #1415144159 rm .bash_history #1415144161 cd #1415144163 pwd #1415144165 rm .bash_history #1415144171 i=0; while [ $i -lt 64 ]; do history -a; history -n; let i++; done; # ls -alh .bash_history -rw--- 1 root wheel 9.0M Nov 5 22:59 .bash_history # du -sh .bash_history 9.0M.bash_history # hexdump -C .bash_history 6c 73 20 2d 61 6c 20 0a 23 31 34 31 35 31 34 34 |ls -al .#1415144| 0010 31 35 33 0a 6c 73 20 2d 61 6c 20 2e 62 61 73 68 |153.ls -al .bash| 0020 5f 68 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 |_history .#14151| 0030 34 34 31 35 34 0a 70 77 64 0a 23 31 34 31 35 31 |44154.pwd.#14151| 0040 34 34 31 35 39 0a 72 6d 20 2e 62 61 73 68 5f 68 |44159.rm .bash_h| 0050 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 34 34 |istory .#1415144| 0060 31 36 31 0a 63 64 0a 23 31 34 31 35 31 34 34 31 |161.cd.#14151441| 0070 36 33 0a 70 77 64 0a 23 31 34 31 35 31 34 34 31 |63.pwd.#14151441| 0080 36 35 0a 72 6d 20 2e 62 61 73 68 5f 68 69 73 74 |65.rm .bash_hist| 0090 6f 72 79 20 0a 23 31 34 31 35 31 34 34 31 37 31 |ory .#1415144171| 00a0 0a 69 3d 30 3b 20 77 68 69 6c 65 20 5b 20 24 69 |.i=0; while [ $i| 00b0 20 2d 6c 74 20 36 34 20 5d 3b 20 64 6f 20 68 69 | -lt 64 ]; do hi| 00c0 73 74 6f 72 79 20 2d 61 3b 20 68 69 73 74 6f 72 |story -a; histor| 00d0 79 20 2d 6e 3b 20 6c 65 74 20 69 2b 2b 3b 20 64 |y -n; let i++; d| 00e0 6f 6e 65 3b 0a 00 00 00 00 00 00 00 00 00 00 00 |one;| 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 00902b60 which puzzles me a bit. Now I lost even all my while-loop commands! :) Graham: did you actually logout and removed the history file, and therefore started fresh? Sorry I guess I lost track of what is going on here. I did. We in terms of “logout”, I had a singe terminal session open. I unlinked the history file and then ctrl-Dd that last session to close it. When I created a new one, it was created and empty. cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Just out of curiosity, this mechanism is obviously something of a work around to give bash a single shared history in the same style as ksh had. Has there ever been any thought of providing a mechanism to support a single history session in a more native way? I know that zsh has the setopt share_history option to provide this (as I’ve tried this before finding I can’t get on with zsh for other reasons). Is there something about the way the history works in bash that makes it difficult to implement? On 6 Nov 2014, at 9:11 am, Piotr Grzybowski narsil...@gmail.com wrote: On Wed, Nov 5, 2014 at 2:57 PM, Chet Ramey chet.ra...@case.edu wrote: I still can't reproduce it on OS X 10.10: [..] not only I cannot reproduce it at the moment, but also I have reached the following state (user: root): # cat .bash_history ls -al #1415144153 ls -al .bash_history #1415144154 pwd #1415144159 rm .bash_history #1415144161 cd #1415144163 pwd #1415144165 rm .bash_history #1415144171 i=0; while [ $i -lt 64 ]; do history -a; history -n; let i++; done; # ls -alh .bash_history -rw--- 1 root wheel 9.0M Nov 5 22:59 .bash_history # du -sh .bash_history 9.0M.bash_history # hexdump -C .bash_history 6c 73 20 2d 61 6c 20 0a 23 31 34 31 35 31 34 34 |ls -al .#1415144| 0010 31 35 33 0a 6c 73 20 2d 61 6c 20 2e 62 61 73 68 |153.ls -al .bash| 0020 5f 68 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 |_history .#14151| 0030 34 34 31 35 34 0a 70 77 64 0a 23 31 34 31 35 31 |44154.pwd.#14151| 0040 34 34 31 35 39 0a 72 6d 20 2e 62 61 73 68 5f 68 |44159.rm .bash_h| 0050 69 73 74 6f 72 79 20 0a 23 31 34 31 35 31 34 34 |istory .#1415144| 0060 31 36 31 0a 63 64 0a 23 31 34 31 35 31 34 34 31 |161.cd.#14151441| 0070 36 33 0a 70 77 64 0a 23 31 34 31 35 31 34 34 31 |63.pwd.#14151441| 0080 36 35 0a 72 6d 20 2e 62 61 73 68 5f 68 69 73 74 |65.rm .bash_hist| 0090 6f 72 79 20 0a 23 31 34 31 35 31 34 34 31 37 31 |ory .#1415144171| 00a0 0a 69 3d 30 3b 20 77 68 69 6c 65 20 5b 20 24 69 |.i=0; while [ $i| 00b0 20 2d 6c 74 20 36 34 20 5d 3b 20 64 6f 20 68 69 | -lt 64 ]; do hi| 00c0 73 74 6f 72 79 20 2d 61 3b 20 68 69 73 74 6f 72 |story -a; histor| 00d0 79 20 2d 6e 3b 20 6c 65 74 20 69 2b 2b 3b 20 64 |y -n; let i++; d| 00e0 6f 6e 65 3b 0a 00 00 00 00 00 00 00 00 00 00 00 |one;| 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 || * 00902b60 which puzzles me a bit. Now I lost even all my while-loop commands! :) Graham: did you actually logout and removed the history file, and therefore started fresh? Sorry I guess I lost track of what is going on here. cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/4/14 5:16 PM, Piotr Grzybowski wrote: Chet: for reasons unexplained calls to read_history_range at history.def:219 219 result = read_history_range (filename, history_lines_in_file, -1); return more and more records (77824 is above my HISTFILESIZE): There is an issue with this particular, relatively infrequent, set of circumstances: 1. HISTFILESIZE HISTSIZE 2. Actual size of the history file HISTSIZE 3. history list contains HISTSIZE entries 4. the user executes a sequence of `history -a; history -n'. Even `history -a; history -r' would probably prevent this, at the cost of more malloc/free operations. In this case, history_lines_in_file (which, despite its name, is supposed to keep track of how many lines bash has read from the history file) is calculated incorrectly. The result is that bash reads way too many lines from the history file each time. There are a couple of ways to fix it: one that uses facilities the history library provides today and one that changes the history library to expose the values more directly. One will appear in the next snapshot. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/5/14 8:39 PM, Graham Jones wrote: Just out of curiosity, this mechanism is obviously something of a work around to give bash a single shared history in the same style as ksh had. Has there ever been any thought of providing a mechanism to support a single history session in a more native way? I know that zsh has the setopt share_history option to provide this (as I’ve tried this before finding I can’t get on with zsh for other reasons). Is there something about the way the history works in bash that makes it difficult to implement? In theory, one could change the functions in history.c and histfile.c to change the in-memory history list to one that uses a file, possibly with mmap(). I'd be happy to look at contributed code to do this, but implementing it is not high on the list. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Thanks Chet. Couple of notes: 1) I’m not setting HISTFILESIZE, only HISTSIZE. HISTFILESIZE is getting set for me based on my choice of HISTSIZE (though I assume this is something that bash does that you know about). 2) As mentioned in the initial bug report, if I set PROMPT_COMMAND='history -a; history -r’ in my profile, my terminal was immediately hanging on initialisation. Since reseting the history file though, this no longer seems to be the case. On 6 Nov 2014, at 1:25 pm, Chet Ramey chet.ra...@case.edu wrote: On 11/4/14 5:16 PM, Piotr Grzybowski wrote: Chet: for reasons unexplained calls to read_history_range at history.def:219 219 result = read_history_range (filename, history_lines_in_file, -1); return more and more records (77824 is above my HISTFILESIZE): There is an issue with this particular, relatively infrequent, set of circumstances: 1. HISTFILESIZE HISTSIZE 2. Actual size of the history file HISTSIZE 3. history list contains HISTSIZE entries 4. the user executes a sequence of `history -a; history -n'. Even `history -a; history -r' would probably prevent this, at the cost of more malloc/free operations. In this case, history_lines_in_file (which, despite its name, is supposed to keep track of how many lines bash has read from the history file) is calculated incorrectly. The result is that bash reads way too many lines from the history file each time. There are a couple of ways to fix it: one that uses facilities the history library provides today and one that changes the history library to expose the values more directly. One will appear in the next snapshot. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Nov 6, 2014 3:25 AM, Chet Ramey chet.ra...@case.edu wrote: On 11/4/14 5:16 PM, Piotr Grzybowski wrote: Chet: for reasons unexplained calls to read_history_range at [..] There is an issue with this particular, relatively infrequent, set of circumstances [..] Thanks a lot Chet, seems that this was exactly what happened, although, at the moment, I cannot reproduce it. I can git revert to my previous .bash_history :) and then try again. It also would seem that what Graham sees is something differrent. There are a couple of ways to fix it: one that uses facilities the history library provides today and one that changes the history library to expose the values more directly. One will appear in the next snapshot. thanks a lot! pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 11/3/14 5:08 PM, Graham Jones wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) This trace looks pretty reasonable. Maybe you could temporarily move your history file to some other name and see if you can reproduce this behavior starting fresh with an empty history file. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Hi, reproducible on mac os x 10.6.8 # uname -a Darwin 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 # bash --version GNU bash, version 4.3.24(1)-release (x86_64-apple-darwin10.8.0) Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Graham: could you please send the output of these commands: # cat .bash_history | wc -l # echo $HISTFILESIZE # echo $HISTSIZE it obviously happens when you are almost at the level of $HISTFILESIZE with `cat .bash_history | wc -l`. cheers, pg On Tue, Nov 4, 2014 at 2:54 PM, Chet Ramey chet.ra...@case.edu wrote: On 11/3/14 5:08 PM, Graham Jones wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) This trace looks pretty reasonable. Maybe you could temporarily move your history file to some other name and see if you can reproduce this behavior starting fresh with an empty history file. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 5 Nov 2014, at 4:49 am, Piotr Grzybowski narsil...@gmail.com wrote: Hi, reproducible on mac os x 10.6.8 Really? Very interesting. I assume that it’s not reproducible with the version of bash that ships with 10.6.8 however? This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Graham: could you please send the output of these commands: # cat .bash_history | wc -l # echo $HISTFILESIZE # echo $HISTSIZE it obviously happens when you are almost at the level of $HISTFILESIZE with `cat .bash_history | wc -l`. Not at all. I keep my history for a long time and have an arbitrarily large HISTSIZE. However, after the recent Yosemite upgrade, histories were truncated, so it’s relatively empty and nowhere near the level of HISTFILESIZE after the 20 or so returns that I enter reproduce it (and I’m not sure that just hitting enter even puts anything in the history). Here are the details you wanted: graham@zebedee:~$ set | grep HIST HISTFILE=/Users/graham/.bash_history HISTFILESIZE=50 HISTSIZE=50 graham@zebedee:~$ wc -l $HISTFILE 21 /Users/graham/.bash_history cheers, pg On Tue, Nov 4, 2014 at 2:54 PM, Chet Ramey chet.ra...@case.edu wrote: On 11/3/14 5:08 PM, Graham Jones wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) This trace looks pretty reasonable. Maybe you could temporarily move your history file to some other name and see if you can reproduce this behavior starting fresh with an empty history file. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Hi Graham, On Tue, Nov 4, 2014 at 10:19 PM, Graham Jones your-name-h...@grahamjones.org wrote: Really? Very interesting. I assume that it’s not reproducible with the version of bash that ships with 10.6.8 however? not exactly the version shipped with 10.6.8, more like installed by hand: # which bash /opt/local/bin//bash Not at all. I keep my history for a long time and have an arbitrarily large HISTSIZE. However, after the recent Yosemite upgrade, histories were truncated, so it’s relatively empty and nowhere near the level of HISTFILESIZE after the 20 or so returns that I enter reproduce it (and I’m not sure that just hitting enter even puts anything in the history). well, thats strange, but it means that this also should do the trick, without setting PROMPT_COMMAND: # i=0; while [ $i -lt 25 ]; do history -a; history -n; let i++; done; it actually does for me, anything above 24 in the upper limit of the above loop triggers the issue (no PROMPT_COMMAND set anywhere). at the same time, the same loop does nothing out of the ordinary on linux/i386 with bash--version: #bash --version GNU bash, version 4.3.30(1)-release (i686-pc-linux-gnu) Here are the details you wanted: [..] thanks, nice HISTSIZE btw ;-) cheers, pg
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 5 Nov 2014, at 12:54 am, Chet Ramey chet.ra...@case.edu wrote: On 11/3/14 5:08 PM, Graham Jones wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) This trace looks pretty reasonable. Maybe you could temporarily move your history file to some other name and see if you can reproduce this behavior starting fresh with an empty history file. As you may have seen from the other email, the history file is relatively small. I did just remove it and start afresh, and the issue still occurs. Whilst attempting to reproduce it, I noticed that is doesn’t reoccur immediately. In my first test, I deleted the history file and closing that session (the only one). When I created a single terminal session, I could su to root and hit return a large number of times without it reproducing the error. I need created another terminal session and (without suing) after issue just a single command, the next command on the root terminal exhibited the behaviour. I retried a similar experiment, again after removing the history file, and again it wasn’t immediately reproducible. This time I didn’t create a second session but left the terminal along for 3 or 4 minutes. When I next entered a command the behaviour has returned. In both cases the line count in $HISTFILE was low = ~40. Now after performing these to experiments I just tried to create a new session and found my $HISTFILE owned by root (or at least permission denied, I didn’t actually check the file mode and owner before chmoding it). This seems strange given I never log in as root only su so presumably the first command executed must always be by my own user. After chmoding it, I notice this: graham@zebedee:~$ wc -l $HISTFILE 51 /Users/graham/.bash_history I have obviously not entered that many commands! Immediately after I go to check the contents and it’s owned by root again graham@zebedee:~$ wc -l $HISTFILE wc: /Users/graham/.bash_history: open: Permission denied graham@zebedee:~$ l $HISTFILE -rw--- 1 root staff 150 4 Nov 21:36 /Users/graham/.bash_history graham@zebedee:~$ ps aux |grep bash graham 99600 0.0 0.0 2432772648 s000 S+9:40pm 0:00.00 grep bash graham 99555 0.0 0.5 2492596 39076 s000 S 9:36pm 0:00.49 -bash But yet there is only my bash running non-priveledge. Now I was expecting a hole in the file, but when I finally get to see the contents, I have 500,000 ls commands in there (one of my test commands from above) graham@zebedee:~$ wc -l $HISTFILE wc: /Users/graham/.bash_history: open: Permission denied graham@zebedee:~$ sudo chown graham $HISTFILE Password: graham@zebedee:~$ sort $HISTFILE | uniq -c | sort -n 1 sudo chown graham $HISTFILE 1 vi $HISTFILE 50 ls Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On Tue, Nov 4, 2014 at 10:44 PM, Graham Jones your-name-h...@grahamjones.org wrote: Now I was expecting a hole in the file, but when I finally get to see the contents, I have 500,000 ls commands in there (one of my test commands from above) just for the record, now I have $HISTSIZE i=0; while [ $i -lt 25 ]; do history -a; history -n; let i++; done; commands in .bash_history :) cheers, pg P.S. pity I really needed those ;-)
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Chet: for reasons unexplained calls to read_history_range at history.def:219 219 result = read_history_range (filename, history_lines_in_file, -1); return more and more records (77824 is above my HISTFILESIZE): 1: history_lines_in_file = 77824 the loop at histfile.c:269 in read_history_range is the one that hangs. code at commit ca6a2ba40c709c2b45a56e49d21d0dfc66e21974 Date: Sun Oct 5 19:12:20 2014 -0400 cheers, pg On Tue, Nov 4, 2014 at 2:54 PM, Chet Ramey chet.ra...@case.edu wrote: On 11/3/14 5:08 PM, Graham Jones wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) This trace looks pretty reasonable. Maybe you could temporarily move your history file to some other name and see if you can reproduce this behavior starting fresh with an empty history file. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) lldb backtrace: (lldb) process attach --pid 5955 Process 5955 stopped Executable module set to /usr/local/bin/bash. Architecture set to: x86_64h-apple-macosx. (lldb) bt * thread #1: tid = 0x169339, 0x0001020421c0 bash`add_history + 160, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP * frame #0: 0x0001020421c0 bash`add_history + 160 frame #1: 0x00010204565c bash`read_history_range + 540 frame #2: 0x000102009719 bash`history_builtin + 1433 frame #3: 0x000101fbda9e bash`execute_builtin + 590 frame #4: 0x000101fbb941 bash`execute_command_internal + 17457 frame #5: 0x000101fb8c44 bash`execute_command_internal + 5940 frame #6: 0x000102006104 bash`parse_and_execute + 916 frame #7: 0x000101fa8da6 bash`execute_variable_command + 374 frame #8: 0x000101fa5ef6 bash`parse_command + 86 frame #9: 0x000101fa5e5f bash`read_command + 223 frame #10: 0x000101fa5c84 bash`reader_loop + 468 frame #11: 0x000101fa506c bash`main + 6620 frame #12: 0x7fff886f15c9 libdyld.dylib`start + 1 frame #13: 0x7fff886f15c9 libdyld.dylib`start + 1 (lldb) thread list Process 5955 stopped * thread #1: tid = 0x169339, 0x0001020421c0 bash`add_history + 160, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP dtruss -a: root@zebedee:~# dtruss -p 5955 -a Invalid connection: com.apple.coresymbolicationd PID/THRD RELATIVE ELAPSDCPU SYSCALL(args) = return 5955/0x169339: 57713743 114 2 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051859 0 5955/0x169339: 57723064 71 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051859 0 5955/0x169339: 59167979 61 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051861 0 5955/0x169339: 59177993 63 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051861 0 5955/0x169339: 60645254 62 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 60655370 6 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 60635499 60 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 62072822 87 3 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051864 0 5955/0x169339: 63511141 62 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 63520959 88 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 63531242 7 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 65032419 61 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051867 0 5955/0x169339: 65042389 6 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051867 0 5955/0x169339: 66134657 65 2 madvise(0x103442000, 0x4D3000, 0x9) = 0 0 5955/0x169339: 66135208 554550 madvise(0x103442000, 0x4D3000, 0x7) = 0 0 5955/0x169339: 66135269 31 27 stat64(/var/mail/root\0, 0x7FFF5DC5C618, 0x7) = -1 Err#2 5955/0x169339: 66135303 4 1 sigprocmask(0x1, 0x7FFF5DC5C6BC, 0x7FFF5DC5C6B8)= 0x0 0 5955/0x169339: 66135308 5 2 ioctl(0xFF, 0x4004667A, 0x7FFF5DC5C60C) = 0 0 5955/0x169339: 66135318 11 9 ioctl(0xFF, 0x80047476, 0x7FFF5DC5C68C) = 0 0 5955/0x169339: 66135320 3 0 sigprocmask(0x3, 0x7FFF5DC5C6B8, 0x0) = 0x0 0 5955/0x169339: 66135322 3 0 sigaction(0x2, 0x7FFF5DC5C688, 0x7FFF5DC5C6B0) = 0 0 5955/0x169339: 66135351 3 1 ioctl(0x0, 0x40087468, 0x7FFF5DC5C638) = 0 0 5955/0x169339: 66135353 3 2 ioctl(0x0, 0x80087467, 0x7FFF5DC5C638) = 0 0 5955/0x169339: 66135354 2 0 ioctl(0x0, 0x40487413, 0x7FFF5DC5C640) = 0 0 5955/0x169339: 66135359 5 3 ioctl(0x0, 0x80487415, 0x7FFF5DC5C640) = 0 0 5955/0x169339: 66135361 2 0 sigprocmask(0x1, 0x102073E14, 0x102073E18) = 0x0 0 5955/0x169339: 66135362 2 0 sigaction(0x2, 0x7FFF5DC5C638, 0x7FFF5DC5C698) = 0 0 5955/0x169339: 66135362 2 0 sigaction(0xF, 0x7FFF5DC5C638, 0x7FFF5DC5C698) = 0 0 5955/0x169339: 66135363 2 0 sigaction(0x1, 0x7FFF5DC5C638, 0x7FFF5DC5C698) = 0 0 5955/0x169339: 66135364 1 0 sigaction(0x3, 0x7FFF5DC5C638, 0x7FFF5DC5C698) = 0 0 5955/0x169339: 66135364 2 0 sigaction(0x3, 0x7FFF5DC5C638, 0x7FFF5DC5C678) = 0 0 5955/0x169339: 66135365 1 0 sigaction(0xE, 0x7FFF5DC5C638,
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Resend with missing stats for truss after ctrl-C On 4 Nov 2014, at 9:08 am, Graham Jones your-name-h...@grahamjones.org wrote: These are for: bash --version GNU bash, version 4.3.30(1)-release (x86_64-apple-darwin14.0.0) lldb backtrace: (lldb) process attach --pid 5955 Process 5955 stopped Executable module set to /usr/local/bin/bash. Architecture set to: x86_64h-apple-macosx. (lldb) bt * thread #1: tid = 0x169339, 0x0001020421c0 bash`add_history + 160, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP * frame #0: 0x0001020421c0 bash`add_history + 160 frame #1: 0x00010204565c bash`read_history_range + 540 frame #2: 0x000102009719 bash`history_builtin + 1433 frame #3: 0x000101fbda9e bash`execute_builtin + 590 frame #4: 0x000101fbb941 bash`execute_command_internal + 17457 frame #5: 0x000101fb8c44 bash`execute_command_internal + 5940 frame #6: 0x000102006104 bash`parse_and_execute + 916 frame #7: 0x000101fa8da6 bash`execute_variable_command + 374 frame #8: 0x000101fa5ef6 bash`parse_command + 86 frame #9: 0x000101fa5e5f bash`read_command + 223 frame #10: 0x000101fa5c84 bash`reader_loop + 468 frame #11: 0x000101fa506c bash`main + 6620 frame #12: 0x7fff886f15c9 libdyld.dylib`start + 1 frame #13: 0x7fff886f15c9 libdyld.dylib`start + 1 (lldb) thread list Process 5955 stopped * thread #1: tid = 0x169339, 0x0001020421c0 bash`add_history + 160, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP dtruss -a: root@zebedee:~# dtruss -p 5955 -a Invalid connection: com.apple.coresymbolicationd PID/THRD RELATIVE ELAPSDCPU SYSCALL(args) = return 5955/0x169339: 57713743 114 2 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051859 0 5955/0x169339: 57723064 71 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051859 0 5955/0x169339: 59167979 61 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051861 0 5955/0x169339: 59177993 63 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051861 0 5955/0x169339: 60645254 62 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 60655370 6 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 60635499 60 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051862 0 5955/0x169339: 62072822 87 3 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051864 0 5955/0x169339: 63511141 62 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 63520959 88 1 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 63531242 7 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051865 0 5955/0x169339: 65032419 61 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051867 0 5955/0x169339: 65042389 6 0 gettimeofday(0x7FFF5DC5CAA0, 0x0, 0x4D27BB) = 1415051867 0 5955/0x169339: 66134657 65 2 madvise(0x103442000, 0x4D3000, 0x9) = 0 0 5955/0x169339: 66135208 554550 madvise(0x103442000, 0x4D3000, 0x7) = 0 0 5955/0x169339: 66135269 31 27 stat64(/var/mail/root\0, 0x7FFF5DC5C618, 0x7) = -1 Err#2 5955/0x169339: 66135303 4 1 sigprocmask(0x1, 0x7FFF5DC5C6BC, 0x7FFF5DC5C6B8) = 0x0 0 5955/0x169339: 66135308 5 2 ioctl(0xFF, 0x4004667A, 0x7FFF5DC5C60C) = 0 0 5955/0x169339: 66135318 11 9 ioctl(0xFF, 0x80047476, 0x7FFF5DC5C68C) = 0 0 5955/0x169339: 66135320 3 0 sigprocmask(0x3, 0x7FFF5DC5C6B8, 0x0) = 0x0 0 5955/0x169339: 66135322 3 0 sigaction(0x2, 0x7FFF5DC5C688, 0x7FFF5DC5C6B0)= 0 0 5955/0x169339: 66135351 3 1 ioctl(0x0, 0x40087468, 0x7FFF5DC5C638)= 0 0 5955/0x169339: 66135353 3 2 ioctl(0x0, 0x80087467, 0x7FFF5DC5C638)= 0 0 5955/0x169339: 66135354 2 0 ioctl(0x0, 0x40487413, 0x7FFF5DC5C640)= 0 0 5955/0x169339: 66135359 5 3 ioctl(0x0, 0x80487415, 0x7FFF5DC5C640)= 0 0 5955/0x169339: 66135361 2 0 sigprocmask(0x1, 0x102073E14, 0x102073E18)= 0x0 0 5955/0x169339: 66135362 2 0 sigaction(0x2, 0x7FFF5DC5C638, 0x7FFF5DC5C698)= 0 0 5955/0x169339: 66135362 2 0 sigaction(0xF, 0x7FFF5DC5C638, 0x7FFF5DC5C698)= 0 0 5955/0x169339: 66135363 2 0 sigaction(0x1, 0x7FFF5DC5C638, 0x7FFF5DC5C698)= 0 0 5955/0x169339: 66135364 1 0 sigaction(0x3,
PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
Since upgrading to OS X 10.10 Yosemite and thus bash 3.2.53 use of the PROMPT_COMMAND='history -a; history -n' workaround to get a single ksh-style history causes the shell to become less and less responsive until it permanently hangs. The problem is somewhat intermitant in terms of how quickly it starts, but it's more pronounced after a su - to root (or probably any su). It can be readily reproduced by: 1) Putting the above command in your .bashrc file 2) Start a new terminal session 3) su - 4) Hit return around 20-40 times (optionally entering commands) 5) Watch as the prompt returns more and more slowly, then never at all, and PS reports ~98% CPU usage for bash. For a complete description, please see this stack exchange post: http://apple.stackexchange.com/questions/153361/prompt-command-history-a-history-n-changed-broken-in-yosemite-bash-3-2 http://apple.stackexchange.com/questions/153361/prompt-command-history-a-history-n-changed-broken-in-yosemite-bash-3-2
Re: PROMPT_COMMAND='history -a; history -n' causes shell hang in OX 10.10 Yosemite / bash 3.2.53
On 10/29/14, 1:31 AM, Graham Jones wrote: Since upgrading to OS X 10.10 Yosemite and thus bash 3.2.53 use of the PROMPT_COMMAND='history -a; history -n' workaround to get a single ksh-style history causes the shell to become less and less responsive until it permanently hangs. I wasn't able to reproduce this on 10.10, but there are a lot of variables and potential environmental dependencies here. What would be useful is some kind of backtrace (from lldb or gdb) or a system call trace (from dtruss or ktrace) that shows what bash is doing. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
