Re: zsh crash in recent -current
That makes perfect sense. I'll rebuild it without this flag. On Thu, 14 Mar 2019 at 12:21, Geoff Wing wrote: > > On Thursday 2019-03-14 10:57 +, ci4...@gmail.com output: > :Well, after installing the unstripped zsh+modules and ncurses, I no > :longer get zsh any crashes. Plus, as I mentioned, there was some > :jemalloc updates a couple of days ago. Hence, no idea. > > Hi, > if you ended up configuring with --enable-zsh-mem (as you mentioned trying in > a previous message) then you should not be using jemalloc at all but zsh's > mem routines - in which case it may avoid any use-after-free issues or > otherwise which jemalloc may expose. > > Regards, > Geoff --
Re: zsh crash in recent -current
On Thursday 2019-03-14 10:57 +, ci4...@gmail.com output: :Well, after installing the unstripped zsh+modules and ncurses, I no :longer get zsh any crashes. Plus, as I mentioned, there was some :jemalloc updates a couple of days ago. Hence, no idea. Hi, if you ended up configuring with --enable-zsh-mem (as you mentioned trying in a previous message) then you should not be using jemalloc at all but zsh's mem routines - in which case it may avoid any use-after-free issues or otherwise which jemalloc may expose. Regards, Geoff
Re: zsh crash in recent -current
Well, after installing the unstripped zsh+modules and ncurses, I no longer get zsh any crashes. Plus, as I mentioned, there was some jemalloc updates a couple of days ago. Hence, no idea. On Thu, 14 Mar 2019 at 05:23, matthew green wrote: > > > (while none I have ever seen actually do, the > > malloc implementation is free to retrn the memory to the kernel, and > > remove it from the process's address space). > > FWIW, both old and new jemalloc are capable of doing this. :-) > > > .mrg. --
re: zsh crash in recent -current
> (while none I have ever seen actually do, the > malloc implementation is free to retrn the memory to the kernel, and > remove it from the process's address space). FWIW, both old and new jemalloc are capable of doing this. :-) .mrg.
Re: zsh crash in recent -current
Date:Wed, 13 Mar 2019 10:37:41 -0700
From:Brian Buhrow
Message-ID: <201903131737.x2dhbfd8001...@lothlorien.nfbcal.org>
| Given this code fragment and the discussion you raise
| about it, allow me to ask what perhaps is a naive question. If the sample
| you quote is incorrect, what is the correct way to accomplish the same
| task?
The basic principle is that once memory is freed it no longer belongs
to the application (while none I have ever seen actually do, the
malloc implementation is free to retrn the memory to the kernel, and
remove it from the process's address space).
So, everything one wants from the memory block needs to be done before
the free() happens.
In cases of muti-threaded applications, including the kernel,
the right answer depends upon just how locking gets done, what
gets locked when, and for how long (need to make sure that the
list isn't being changed by some other thread, or deal with the
possibility that it is, if we cannot reasonably prevent it - in
the worst case, exit the loop and restart from the beginning each
time something happens that requires releasing a lock on the
list itself).
But for a simple single threaded userland application, something
like:
| } for (list_ptr = list_head; list_ptr != NULL; list_ptr = next) {
next = list_ptr->nxt;
| } /* do stuff on list */
| } if (element_should_be_deleted) {
| } /* with testing for NULLs added but not shown here */
| } list_ptr->prev->nxt = list_ptr->nxt;
| } list_ptr->nxt->prev = list_ptr->prev;
| } free(list_ptr);
| } }
| } }
There are plenty of other (valid) ways to write it of course, but all of
them require that after that "free()" we never touch *list_ptr (aka list_ptr->)
until list_ptr has been changed to refer to something that has not been free'd.
kre
Re: zsh crash in recent -current
On Wed, Mar 13, 2019 at 11:32:05AM +, Chavdar Ivanov wrote: > Setting INSTALL_UNSTRIPPED=yes didn't work for me for zsh and ncurses, > I still had to find all the '-s' flags in the install command > invocations. Perhaps I am doing something wrong. Still, I've gor now > zsh with the debug options and all modules plus ncurses unstripped. > Now to build the OS debug sets. > Oh ncurses I was going to mention that some time ago I did link our native curses against libefence which caught a few memory issues. -- Brett Lymn -- Sent from my NetBSD device. "We are were wolves", "You mean werewolves?", "No we were wolves, now we are something else entirely", "Oh"
Re: zsh crash in recent -current
hello Robert. Given this code fragment and the discussion you raise
about it, allow me to ask what perhaps is a naive question. If the sample
you quote is incorrect, what is the correct way to accomplish the same
task?
-thanks
-Brian
On Mar 13, 6:27pm, Robert Elz wrote:
} Subject: Re: zsh crash in recent -current
} Date:Wed, 13 Mar 2019 10:06:42 +
} From:Chavdar Ivanov
} Message-ID:
}
} | I saw the one with the trashed history as well.
} |
} | I don't think it is zsh's problem, though. As I mentioned above, I've
} | used v5.7 since it came out without any problems until perhaps 3-4
} | days ago.
}
} I would guess that maybe there is code like this
}
} for (list_ptr = list_head; list_ptr != NULL; list_ptr = list_ptr->nxt)
} {
} /* do stuff on list */
} if (element_should_be_deleted) {
} /* with testing for NULLs added but not shown here */
} list_ptr->prev->nxt = list_ptr->nxt;
} list_ptr->nxt->prev = list_ptr->prev;
} free(list_ptr);
} }
} }
}
} which will "work" perfectly wih most versions of malloc, as
} that free does not change anything in the memory that has been
} freed, but will collapse in a giant heap if free() scribbles
} over the memory as part of deleting things, which some of the
} dumps that various people have shown on this (and similar) issues
} looks to be what is happening (the scribbling - it is deliberate
} to expose bugs like this one).
}
} Code like the above is easy to write, and most of the time works fine
} (and would have worked with the previous malloc) but will die
} big time when the arena is scrambled (not just zeroed, usually).
}
} Someone should look for something like this in the areas of zsh
} that are crashing, and other programs.
}
} This is far more likely than the new malloc being broken, and just
} only happening to hit a few programs, and is more likely than some
} random memory corruption that simply has never been noticed until
} now.
}
} kre
}
}
>-- End of excerpt from Robert Elz
Re: zsh crash in recent -current
s/gor/got/. On the other hand there were some jemalloc changes overnight, so I will update the system first and then try to get zsh crashes. On Wed, 13 Mar 2019 at 11:32, Chavdar Ivanov wrote: > > Setting INSTALL_UNSTRIPPED=yes didn't work for me for zsh and ncurses, > I still had to find all the '-s' flags in the install command > invocations. Perhaps I am doing something wrong. Still, I've gor now > zsh with the debug options and all modules plus ncurses unstripped. > Now to build the OS debug sets. > > On Wed, 13 Mar 2019 at 10:23, Chavdar Ivanov wrote: > > > > Thanks. One has to read the manuals from time to time... > > > > On Wed, 13 Mar 2019 at 10:21, Patrick Welche wrote: > > > > > > On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > > > > I saw the one with the trashed history as well. > > > > > > > > I don't think it is zsh's problem, though. As I mentioned above, I've > > > > used v5.7 since it came out without any problems until perhaps 3-4 > > > > days ago. > > > > > > > > I tried to build zsh with debug (adding --enable-zsh-debug > > > > --enable-zsh-mem --enable-zsh-mem-debug --enable-zsh-mem-warning > > > > --enable-zsh-secure-free --enable-zsh-heap-debug > > > > --enable-zsh-hash-debug to the makefile), but I still get a stripped > > > > executable, no doubt I miss some pkgsrc environment variable. If I try > > > > to build it within the work folder with gmake, it refuses to build the > > > > curses.so module, which is one of the failing ones. > > > > > > INSTALL_UNSTRIPPED=yes > > > > > > ? > > > > > > Cheers, > > > > > > Patrick > > > > > > > > -- > > > > > > -- > --
Re: zsh crash in recent -current
Setting INSTALL_UNSTRIPPED=yes didn't work for me for zsh and ncurses, I still had to find all the '-s' flags in the install command invocations. Perhaps I am doing something wrong. Still, I've gor now zsh with the debug options and all modules plus ncurses unstripped. Now to build the OS debug sets. On Wed, 13 Mar 2019 at 10:23, Chavdar Ivanov wrote: > > Thanks. One has to read the manuals from time to time... > > On Wed, 13 Mar 2019 at 10:21, Patrick Welche wrote: > > > > On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > > > I saw the one with the trashed history as well. > > > > > > I don't think it is zsh's problem, though. As I mentioned above, I've > > > used v5.7 since it came out without any problems until perhaps 3-4 > > > days ago. > > > > > > I tried to build zsh with debug (adding --enable-zsh-debug > > > --enable-zsh-mem --enable-zsh-mem-debug --enable-zsh-mem-warning > > > --enable-zsh-secure-free --enable-zsh-heap-debug > > > --enable-zsh-hash-debug to the makefile), but I still get a stripped > > > executable, no doubt I miss some pkgsrc environment variable. If I try > > > to build it within the work folder with gmake, it refuses to build the > > > curses.so module, which is one of the failing ones. > > > > INSTALL_UNSTRIPPED=yes > > > > ? > > > > Cheers, > > > > Patrick > > > > -- > --
Re: zsh crash in recent -current
Date:Wed, 13 Mar 2019 10:06:42 +
From:Chavdar Ivanov
Message-ID:
| I saw the one with the trashed history as well.
|
| I don't think it is zsh's problem, though. As I mentioned above, I've
| used v5.7 since it came out without any problems until perhaps 3-4
| days ago.
I would guess that maybe there is code like this
for (list_ptr = list_head; list_ptr != NULL; list_ptr = list_ptr->nxt)
{
/* do stuff on list */
if (element_should_be_deleted) {
/* with testing for NULLs added but not shown here */
list_ptr->prev->nxt = list_ptr->nxt;
list_ptr->nxt->prev = list_ptr->prev;
free(list_ptr);
}
}
which will "work" perfectly wih most versions of malloc, as
that free does not change anything in the memory that has been
freed, but will collapse in a giant heap if free() scribbles
over the memory as part of deleting things, which some of the
dumps that various people have shown on this (and similar) issues
looks to be what is happening (the scribbling - it is deliberate
to expose bugs like this one).
Code like the above is easy to write, and most of the time works fine
(and would have worked with the previous malloc) but will die
big time when the arena is scrambled (not just zeroed, usually).
Someone should look for something like this in the areas of zsh
that are crashing, and other programs.
This is far more likely than the new malloc being broken, and just
only happening to hit a few programs, and is more likely than some
random memory corruption that simply has never been noticed until
now.
kre
Re: zsh crash in recent -current
Thanks. One has to read the manuals from time to time... On Wed, 13 Mar 2019 at 10:21, Patrick Welche wrote: > > On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > > I saw the one with the trashed history as well. > > > > I don't think it is zsh's problem, though. As I mentioned above, I've > > used v5.7 since it came out without any problems until perhaps 3-4 > > days ago. > > > > I tried to build zsh with debug (adding --enable-zsh-debug > > --enable-zsh-mem --enable-zsh-mem-debug --enable-zsh-mem-warning > > --enable-zsh-secure-free --enable-zsh-heap-debug > > --enable-zsh-hash-debug to the makefile), but I still get a stripped > > executable, no doubt I miss some pkgsrc environment variable. If I try > > to build it within the work folder with gmake, it refuses to build the > > curses.so module, which is one of the failing ones. > > INSTALL_UNSTRIPPED=yes > > ? > > Cheers, > > Patrick --
Re: zsh crash in recent -current
On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > I saw the one with the trashed history as well. > > I don't think it is zsh's problem, though. As I mentioned above, I've > used v5.7 since it came out without any problems until perhaps 3-4 > days ago. > > I tried to build zsh with debug (adding --enable-zsh-debug > --enable-zsh-mem --enable-zsh-mem-debug --enable-zsh-mem-warning > --enable-zsh-secure-free --enable-zsh-heap-debug > --enable-zsh-hash-debug to the makefile), but I still get a stripped > executable, no doubt I miss some pkgsrc environment variable. If I try > to build it within the work folder with gmake, it refuses to build the > curses.so module, which is one of the failing ones. INSTALL_UNSTRIPPED=yes ? Cheers, Patrick
Re: zsh crash in recent -current
OK, I understand. I should carry on using it to see if it will break again and perhaps get something useful. On Wed, 13 Mar 2019 at 10:09, Martin Husemann wrote: > > On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > > I saw the one with the trashed history as well. > > > > I don't think it is zsh's problem, though. As I mentioned above, I've > > used v5.7 since it came out without any problems until perhaps 3-4 > > days ago. > > It still is very likely a zsh bug, only old jemalloc (and new one w/o > debugging enabled) is more forgiving. > > Martin --
Re: zsh crash in recent -current
On Wed, Mar 13, 2019 at 10:06:42AM +, Chavdar Ivanov wrote: > I saw the one with the trashed history as well. > > I don't think it is zsh's problem, though. As I mentioned above, I've > used v5.7 since it came out without any problems until perhaps 3-4 > days ago. It still is very likely a zsh bug, only old jemalloc (and new one w/o debugging enabled) is more forgiving. Martin
Re: zsh crash in recent -current
I saw the one with the trashed history as well. I don't think it is zsh's problem, though. As I mentioned above, I've used v5.7 since it came out without any problems until perhaps 3-4 days ago. I tried to build zsh with debug (adding --enable-zsh-debug --enable-zsh-mem --enable-zsh-mem-debug --enable-zsh-mem-warning --enable-zsh-secure-free --enable-zsh-heap-debug --enable-zsh-hash-debug to the makefile), but I still get a stripped executable, no doubt I miss some pkgsrc environment variable. If I try to build it within the work folder with gmake, it refuses to build the curses.so module, which is one of the failing ones. On Tue, 12 Mar 2019 at 21:58, Thomas Klausner wrote: > > On Tue, Mar 12, 2019 at 03:33:26PM +, Chavdar Ivanov wrote: > > On amd64 -curent from yesterday (and a couple of days earlier) I > > started to get zsh crashes when tab-completing (files, directories, > > packages), similar to > > I see lots of crashes with zsh too. > > Some happen in completion, sometimes when I press enter on a command > line, sometimes the history gets trashed (lots of weird characters > turn up when I press 'up') and the shell dies soon after. > > I think there are at least two different bugs in zsh here. > Thomas --
Re: zsh crash in recent -current
On Tue, Mar 12, 2019 at 03:33:26PM +, Chavdar Ivanov wrote: > On amd64 -curent from yesterday (and a couple of days earlier) I > started to get zsh crashes when tab-completing (files, directories, > packages), similar to I see lots of crashes with zsh too. Some happen in completion, sometimes when I press enter on a command line, sometimes the history gets trashed (lots of weird characters turn up when I press 'up') and the shell dies soon after. I think there are at least two different bugs in zsh here. Thomas
Re: zsh crash in recent -current
On Tue, Mar 12, 2019 at 03:33:26PM +, Chavdar Ivanov wrote: > Hi, > > On amd64 -curent from yesterday (and a couple of days earlier) I > started to get zsh crashes when tab-completing (files, directories, > packages), similar to > . > Core was generated by `zsh'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7cf050211540 in permmatches () from > /usr/pkg/lib/zsh/5.7/zsh/complete.so > (gdb) bt > #0 0x7cf050211540 in permmatches () from > /usr/pkg/lib/zsh/5.7/zsh/complete.so > #1 0x7cf050208b7c in ?? () from /usr/pkg/lib/zsh/5.7/zsh/complete.so > #2 0x004550d5 in getstrvalue () > #3 0x00456085 in ?? () > #4 0x00456f67 in getindex () > #5 0x00457557 in fetchvalue () > #6 0x00471da6 in ?? () > #7 0x00476ab7 in prefork () > #8 0x00476f42 in singsub () > #9 0x0042057c in evalcond () > #10 0x00421f49 in ?? () > #11 0x0042bbb8 in ?? () > #12 0x00429a88 in execlist () > (several repeats of the above snippet). This is likely the more strict jemalloc debug code in libc finding a memory corruption issue. Try building zsh with debug info (and maybe also install debug sets to get symbol information for libc). Martin
Re: zsh crash in recent -current
In article , Chavdar Ivanov wrote: >Hi, > >On amd64 -curent from yesterday (and a couple of days earlier) I >started to get zsh crashes when tab-completing (files, directories, >packages), similar to >. >Core was generated by `zsh'. >Program terminated with signal SIGSEGV, Segmentation fault. >#0 0x7cf050211540 in permmatches () from >/usr/pkg/lib/zsh/5.7/zsh/complete.so >(gdb) bt >#0 0x7cf050211540 in permmatches () from >/usr/pkg/lib/zsh/5.7/zsh/complete.so >#1 0x7cf050208b7c in ?? () from /usr/pkg/lib/zsh/5.7/zsh/complete.so >#2 0x004550d5 in getstrvalue () >#3 0x00456085 in ?? () >#4 0x00456f67 in getindex () >#5 0x00457557 in fetchvalue () >#6 0x00471da6 in ?? () >#7 0x00476ab7 in prefork () >#8 0x00476f42 in singsub () >#9 0x0042057c in evalcond () >#10 0x00421f49 in ?? () >#11 0x0042bbb8 in ?? () >#12 0x00429a88 in execlist () > (several repeats of the above snippet). > >Rebuilding zsh did not solve it. I have been usiing the latest version >- 5.7 - since it became available in pkgsrc, under -current, and have >never had problems before; the user zsh environment hasn't changed for >a while. It is not impossiible that some of the files in this >environment is damaged, so I will recreate it from scratch and retry, >but wanted to mention it in case someone else has seen similar. You've been probably bitten by jemalloc, and I am guessing it is probably some memory corruption... You could try linking with -lgnumalloc and see if you can get it working again. christos
zsh crash in recent -current
Hi, On amd64 -curent from yesterday (and a couple of days earlier) I started to get zsh crashes when tab-completing (files, directories, packages), similar to . Core was generated by `zsh'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7cf050211540 in permmatches () from /usr/pkg/lib/zsh/5.7/zsh/complete.so (gdb) bt #0 0x7cf050211540 in permmatches () from /usr/pkg/lib/zsh/5.7/zsh/complete.so #1 0x7cf050208b7c in ?? () from /usr/pkg/lib/zsh/5.7/zsh/complete.so #2 0x004550d5 in getstrvalue () #3 0x00456085 in ?? () #4 0x00456f67 in getindex () #5 0x00457557 in fetchvalue () #6 0x00471da6 in ?? () #7 0x00476ab7 in prefork () #8 0x00476f42 in singsub () #9 0x0042057c in evalcond () #10 0x00421f49 in ?? () #11 0x0042bbb8 in ?? () #12 0x00429a88 in execlist () (several repeats of the above snippet). Rebuilding zsh did not solve it. I have been usiing the latest version - 5.7 - since it became available in pkgsrc, under -current, and have never had problems before; the user zsh environment hasn't changed for a while. It is not impossiible that some of the files in this environment is damaged, so I will recreate it from scratch and retry, but wanted to mention it in case someone else has seen similar. --
