[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-29 Thread Carlos O'Donell
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #8 from Carlos O'Donell  ---
(In reply to Mark Wielaard from comment #7)
> (In reply to Carlos O'Donell from comment #5)
> > (In reply to Mark Wielaard from comment #3)
> > > What is a "detached thread" according to you?  There is no concept of
> > > detached threads in valgrind, all threads are under control of valgrind 
> > > and
> > > valgrind makes sure that only one (kernel) thread is scheduled to run at 
> > > any
> > > one time. See
> > > https://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads
> > 
> > If you use ptrace under the hood to stop a pthread detached thread, and
> > therefore terminate it, then yes, there will be nothing else using the
> > streams.
> > 
> > If that's the case, and we feel comfortable with that as a position then I
> > can flip this back to glibc and discuss *always* shutting down streams via
> > __libc_freeres regardless of their state?
> 
> We don't use ptrace under the hood, we control all kernel threads. From the
> perspective of glibc valgrind provides the kernel interface. Valgrind core
> doesn't use glibc itself. From the perspective of valgrind glibc is just
> like any other code the process executes. With some exceptions, we intercept
> some glibc functions, most specifically __libc_freeres. That function is
> called after the process exits (from glibc's perspective). We should
> probably make sure there are no other users of __libc_freeres, or if there
> are, that they use it in the same way. So it seems a good idea to put this
> back to glibc.

There are other users of __libc_freeres(), particularly mtrace, which frees
everything in a destructors, but that problably makes such code incompatible
with valgrind and that's OK.

Let me note, that even if we fix it in glibc, that is to say that we decree
that all streams, buffered or unbuffered, are freed, there is going to be old
glibc's  and applications that use stderr (unbuffered) and allocate a buffer
(possibly a 20 year old glibc bug) that is not freed.

I leave it up to you to decide if you want to supress an unfreed stderr that
will always show up when using stderr. I'll see about fixing it in glibc.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-29 Thread Mark Wielaard
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #7 from Mark Wielaard  ---
(In reply to Carlos O'Donell from comment #5)
> (In reply to Mark Wielaard from comment #3)
> > What is a "detached thread" according to you?  There is no concept of
> > detached threads in valgrind, all threads are under control of valgrind and
> > valgrind makes sure that only one (kernel) thread is scheduled to run at any
> > one time. See
> > https://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads
> 
> If you use ptrace under the hood to stop a pthread detached thread, and
> therefore terminate it, then yes, there will be nothing else using the
> streams.
> 
> If that's the case, and we feel comfortable with that as a position then I
> can flip this back to glibc and discuss *always* shutting down streams via
> __libc_freeres regardless of their state?

We don't use ptrace under the hood, we control all kernel threads. From the
perspective of glibc valgrind provides the kernel interface. Valgrind core
doesn't use glibc itself. From the perspective of valgrind glibc is just like
any other code the process executes. With some exceptions, we intercept some
glibc functions, most specifically __libc_freeres. That function is called
after the process exits (from glibc's perspective). We should probably make
sure there are no other users of __libc_freeres, or if there are, that they use
it in the same way. So it seems a good idea to put this back to glibc.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-25 Thread Carlos O'Donell
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #6 from Carlos O'Donell  ---
(In reply to Carlos O'Donell from comment #5)
> (In reply to Mark Wielaard from comment #3)
> > What is a "detached thread" according to you?  There is no concept of
> > detached threads in valgrind, all threads are under control of valgrind and
> > valgrind makes sure that only one (kernel) thread is scheduled to run at any
> > one time. See
> > https://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads
> 
> If you use ptrace under the hood to stop a pthread detached thread, and
> therefore terminate it, then yes, there will be nothing else using the
> streams.
> 
> If that's the case, and we feel comfortable with that as a position then I
> can flip this back to glibc and discuss *always* shutting down streams via
> __libc_freeres regardless of their state?

Even if we did enhance glibc, there is a need for all existing distributions
that have unbuffered stderr which won't be freed by __libc_freeres().

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-25 Thread Carlos O'Donell
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #5 from Carlos O'Donell  ---
(In reply to Mark Wielaard from comment #3)
> What is a "detached thread" according to you?  There is no concept of
> detached threads in valgrind, all threads are under control of valgrind and
> valgrind makes sure that only one (kernel) thread is scheduled to run at any
> one time. See
> https://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads

If you use ptrace under the hood to stop a pthread detached thread, and
therefore terminate it, then yes, there will be nothing else using the streams.

If that's the case, and we feel comfortable with that as a position then I can
flip this back to glibc and discuss *always* shutting down streams via
__libc_freeres regardless of their state?

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-25 Thread Tom Hughes
https://bugs.kde.org/show_bug.cgi?id=421931

Tom Hughes  changed:

   What|Removed |Added

 CC||t...@compton.nu

--- Comment #4 from Tom Hughes  ---
A detached thread in pthreads is one that is no longer being supervised and
which will vanish when it completes rather than waiting to be reaped (joined in
pthreads terminology) by another thread.

The situation under valgrind is as you say rather different though in that even
if pthreads has forgotten about the thread valgrind hasn't.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-25 Thread Mark Wielaard
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #3 from Mark Wielaard  ---
(In reply to Carlos O'Donell from comment #2)
> (In reply to Mark Wielaard from comment #1)
> > Which process shutdown steps do you expect to run during or after
> > __libc_freeres is called?
> 
> Any detached thread may write to the stream until the kernel kills the task.
> 
> > __libc_freeres as used by valgrind should be the last "user code" running,
> > no other threads should be scheduled. It is called after all user threads
> > are "dead". And after __libc_freeres has ran only the valgrind (tool) code
> > will run (which doesn't use glibc code).
> 
> How do you wait for detached threads?
> 
> > So at least from the perspective of valgrind __libc_freeres can release all
> > resources.
> 
> So from your perspective at this point, even detached threads, can expect to
> have the streams shutdown?

What is a "detached thread" according to you?  There is no concept of detached
threads in valgrind, all threads are under control of valgrind and valgrind
makes sure that only one (kernel) thread is scheduled to run at any one time.
See https://valgrind.org/docs/manual/manual-core.html#manual-core.pthreads

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-25 Thread Carlos O'Donell
https://bugs.kde.org/show_bug.cgi?id=421931

--- Comment #2 from Carlos O'Donell  ---
(In reply to Mark Wielaard from comment #1)
> Which process shutdown steps do you expect to run during or after
> __libc_freeres is called?

Any detached thread may write to the stream until the kernel kills the task.

> __libc_freeres as used by valgrind should be the last "user code" running,
> no other threads should be scheduled. It is called after all user threads
> are "dead". And after __libc_freeres has ran only the valgrind (tool) code
> will run (which doesn't use glibc code).

How do you wait for detached threads?

> So at least from the perspective of valgrind __libc_freeres can release all
> resources.

So from your perspective at this point, even detached threads, can expect to
have the streams shutdown?

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-23 Thread Mark Wielaard
https://bugs.kde.org/show_bug.cgi?id=421931

Mark Wielaard  changed:

   What|Removed |Added

 CC||m...@klomp.org

--- Comment #1 from Mark Wielaard  ---
Which process shutdown steps do you expect to run during or after
__libc_freeres is called?

__libc_freeres as used by valgrind should be the last "user code" running, no
other threads should be scheduled. It is called after all user threads are
"dead". And after __libc_freeres has ran only the valgrind (tool) code will run
(which doesn't use glibc code).

So at least from the perspective of valgrind __libc_freeres can release all
resources.

-- 
You are receiving this mail because:
You are watching all bug changes.

[valgrind] [Bug 421931] Enhance valgrind to supress unfreed stderr.

2020-05-22 Thread Carlos O'Donell
https://bugs.kde.org/show_bug.cgi?id=421931

Carlos O'Donell  changed:

   What|Removed |Added

   See Also||https://sourceware.org/bugz
   ||illa/show_bug.cgi?id=26022

-- 
You are receiving this mail because:
You are watching all bug changes.