Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Ignacy Gawędzki]

On Wed, Nov 30, 2016 at 06:40:01AM +0100, thus spake Waldemar Brodkorb:
> Hi Ignacy,

Hi Waldemar,

> Ignacy Gawędzki wrote,
> 
> > Since librt and libpthread are now integrated into libc, including
> > unwind-resume and unwind-forcedunwind implementations of unwind code
> > makes no sense.  Only unwind-forcedunwind is now included with
> > functions hidden to avoid them overriding the ones from libgcc_s.
> 
> I tested the patch and I think I will push it in the next days.

Nice.

> Sorry that it took a while, but the removal of the test suite
> took a while. Now the test suite is compiled as a normal software
> package and not with the initial gcc.

That's definitely the right way to go.

> Therefore no regressions seen with your patch.

I'm happy to hear that. =)

> Any other news to the patch?

No.  We've been using it on uClibc-ng 1.0.18 in-house for weeks
without any noticeable problem.

Regards,

Ignacy

-- 
Ignacy Gawędzki
R Engineer
Green Communications
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Waldemar Brodkorb]

Hi Ignacy,
Ignacy Gawędzki wrote,

> Since librt and libpthread are now integrated into libc, including
> unwind-resume and unwind-forcedunwind implementations of unwind code
> makes no sense.  Only unwind-forcedunwind is now included with
> functions hidden to avoid them overriding the ones from libgcc_s.

I tested the patch and I think I will push it in the next days.

Sorry that it took a while, but the removal of the test suite
took a while. Now the test suite is compiled as a normal software
package and not with the initial gcc. 
Therefore no regressions seen with your patch.

Any other news to the patch?

best regards
 Waldemar
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Ignacy Gawędzki]

On Mon, Oct 17, 2016 at 03:40:04PM +0200, thus spake Ignacy Gawedzki:
> On Sun, Oct 16, 2016 at 10:01:30PM +0200, thus spake Waldemar Brodkorb:
> > > As I said, I used d4d4f37 and not strictly origin/master.  But I
> > > otherwise used the command line you gave me, including --libc-source.
> > 
> > But 17ad14e5780533db90171e16b95dbeda4e81ffb0 contains a workaround
> > for exactly the seen problem.
> >  
> > > I will re-run the tests with origin/master and my patch on top of it.
> 
> I found a few interesting things.  The test binaries are compiled by
> openadk with the "initial" GCC and not the "final" one.  The
> difference is that the implementation of the unwinder code is then
> taken from libgcc.a.  I get the same behavior if I explicitly add this
> library to the list of objects to link into the test binary.  With a
> final version of GCC, the implementation is taken from libgcc_s.so as
> it should and the behavior is maybe not ideal but to me it looks far
> better (is that because the initial GCC doesn't really implement any
> unwinder code?).

BTW, the initial GCC is built with --disable-threads.  Could that
possibly be the reason some of the tests in nptl fail?

-- 
Ignacy Gawędzki
R Engineer
Green Communications
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Ignacy Gawędzki]

On Sun, Oct 16, 2016 at 10:01:30PM +0200, thus spake Waldemar Brodkorb:
> > As I said, I used d4d4f37 and not strictly origin/master.  But I
> > otherwise used the command line you gave me, including --libc-source.
> 
> But 17ad14e5780533db90171e16b95dbeda4e81ffb0 contains a workaround
> for exactly the seen problem.
>  
> > I will re-run the tests with origin/master and my patch on top of it.

I found a few interesting things.  The test binaries are compiled by
openadk with the "initial" GCC and not the "final" one.  The
difference is that the implementation of the unwinder code is then
taken from libgcc.a.  I get the same behavior if I explicitly add this
library to the list of objects to link into the test binary.  With a
final version of GCC, the implementation is taken from libgcc_s.so as
it should and the behavior is maybe not ideal but to me it looks far
better (is that because the initial GCC doesn't really implement any
unwinder code?).

The reason why this works without my patch is that the implementation
of the unwinder code is then linked dynamically from libc.so, which,
as I've said previously, is not supposed to be used by code from
outside the lib itself.

I haven't had time yet to look into openadk in order to understand why
it's not using the final GCC to build the tests.  Do you have any
idea?

-- 
Ignacy Gawędzki
R Engineer
Green Communications
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Waldemar Brodkorb]

Hi,
Ignacy Gawędzki wrote,

> On Sun, Oct 16, 2016 at 05:45:40PM +0200, thus spake Waldemar Brodkorb:
> > I tested again. With uClibc-ng master i get exactly 3 errors.
> > Tried three times.
> 
> I was using d4d4f37 and my patch applied on top of it.
> 
> > 
> > With your patch I get sometime 11 or 12 errors. Seems a timeout
> > issue. 
> > 
> > What version of Qemu are you using?
> > 
> > qemu-system-mips -version   
> >  
> > QEMU emulator version 2.6.1, Copyright (c) 2003-2008 Fabrice Bellard
> 
> I'm using 2.5.0.
> 
> > 
> > We recently had issues with the testsuite build without the full gcc
> > and I thinking of separating the testsuite to its own project.
> > 
> > OpenADK does not change between runs of embedded-test as long as
> > you don't use --update.
> 
> Yes, but I ran my tests in two copies of embedded-test.git, and most
> probably the second one cloned a more recent version of openadk.
> 
> > Are you sure you use uClibc-ng fresh checkout and referencing the
> > directory with --libc-source? 1.0.18 release would have 11/12
> > errors, which is the fallback when no special directory is used.
> 
> As I said, I used d4d4f37 and not strictly origin/master.  But I
> otherwise used the command line you gave me, including --libc-source.

But 17ad14e5780533db90171e16b95dbeda4e81ffb0 contains a workaround
for exactly the seen problem.
 
> I will re-run the tests with origin/master and my patch on top of it.

Okay, thanks.
 Waldemar
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Ignacy Gawędzki]

On Sun, Oct 16, 2016 at 05:45:40PM +0200, thus spake Waldemar Brodkorb:
> I tested again. With uClibc-ng master i get exactly 3 errors.
> Tried three times.

I was using d4d4f37 and my patch applied on top of it.

> 
> With your patch I get sometime 11 or 12 errors. Seems a timeout
> issue. 
> 
> What version of Qemu are you using?
> 
> qemu-system-mips -version 
>
> QEMU emulator version 2.6.1, Copyright (c) 2003-2008 Fabrice Bellard

I'm using 2.5.0.

> 
> We recently had issues with the testsuite build without the full gcc
> and I thinking of separating the testsuite to its own project.
> 
> OpenADK does not change between runs of embedded-test as long as
> you don't use --update.

Yes, but I ran my tests in two copies of embedded-test.git, and most
probably the second one cloned a more recent version of openadk.

> Are you sure you use uClibc-ng fresh checkout and referencing the
> directory with --libc-source? 1.0.18 release would have 11/12
> errors, which is the fallback when no special directory is used.

As I said, I used d4d4f37 and not strictly origin/master.  But I
otherwise used the command line you gave me, including --libc-source.

I will re-run the tests with origin/master and my patch on top of it.

Cheers,

Ignacy

-- 
Ignacy Gawędzki
R Engineer
Green Communications
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Waldemar Brodkorb]

Hi,
Ignacy Gawędzki wrote,

> On Thu, Oct 13, 2016 at 09:19:19PM +0200, thus spake Waldemar Brodkorb:
> > If you want you can try it with my embedded-test script (you need
> > mksh and qemu installed):
> > git clone git://git.embedded-test.org/git/embedded-test
> > cd embedded-test
> > git clone git://git.uclibc-ng.org/git/uclibc-ng
> > mksh embedded-test.sh --arch=mipsel --libc=uclibc-ng \
> >   --libc-source=uclibc-ng --test=libc
> > 
> > Only 3 tests should fail.
> > Then apply your patch and run it again, it will automatically
> > do a complete clean build.
> 
> Okay I did this several times yesterday.  On the first attempts, 11
> tests failed without my patch vs. 12 with it.  The additional test
> that failed with my patch was nptl/tst-cancelx10.  I started
> investigating what could possibly go wrong and tried several things,
> from running the binary with qemu-mipsel to logging into the vm used
> for tests and running the test from there.  It seemed at first that
> with my patch the binary got killed with SIGABRT.  I re-ran
> embedded-test.sh several times in the process and at some point
> noticed that the SHA-1 for openadk is not the same.  Since that
> moment, I re-ran embedded-test.sh with a pristine setup in both cases
> (with and without my patch).  Now in both cases I have 11 failed tests
> and nptl/tst-cancelx10 doesn't fail.
> 
> So I'm wondering whether anything in openadk could have changed in the
> meantime that makes the tests work well in both cases.
> 
> Could you please run the tests again yourself to check this?

I tested again. With uClibc-ng master i get exactly 3 errors.
Tried three times.

With your patch I get sometime 11 or 12 errors. Seems a timeout
issue. 

What version of Qemu are you using?

qemu-system-mips -version   
 
QEMU emulator version 2.6.1, Copyright (c) 2003-2008 Fabrice Bellard

We recently had issues with the testsuite build without the full gcc
and I thinking of separating the testsuite to its own project.

OpenADK does not change between runs of embedded-test as long as
you don't use --update.

Are you sure you use uClibc-ng fresh checkout and referencing the
directory with --libc-source? 1.0.18 release would have 11/12
errors, which is the fallback when no special directory is used.

best regards
 Waldemar
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v4 1/1] libpthread: Fix inclusion of unwind code.

[Ignacy Gawędzki]

On Thu, Oct 13, 2016 at 09:19:19PM +0200, thus spake Waldemar Brodkorb:
> If you want you can try it with my embedded-test script (you need
> mksh and qemu installed):
> git clone git://git.embedded-test.org/git/embedded-test
> cd embedded-test
> git clone git://git.uclibc-ng.org/git/uclibc-ng
> mksh embedded-test.sh --arch=mipsel --libc=uclibc-ng \
>   --libc-source=uclibc-ng --test=libc
> 
> Only 3 tests should fail.
> Then apply your patch and run it again, it will automatically
> do a complete clean build.

Okay I did this several times yesterday.  On the first attempts, 11
tests failed without my patch vs. 12 with it.  The additional test
that failed with my patch was nptl/tst-cancelx10.  I started
investigating what could possibly go wrong and tried several things,
from running the binary with qemu-mipsel to logging into the vm used
for tests and running the test from there.  It seemed at first that
with my patch the binary got killed with SIGABRT.  I re-ran
embedded-test.sh several times in the process and at some point
noticed that the SHA-1 for openadk is not the same.  Since that
moment, I re-ran embedded-test.sh with a pristine setup in both cases
(with and without my patch).  Now in both cases I have 11 failed tests
and nptl/tst-cancelx10 doesn't fail.

So I'm wondering whether anything in openadk could have changed in the
meantime that makes the tests work well in both cases.

Could you please run the tests again yourself to check this?

Cheers,

Ignacy

-- 
Ignacy Gawędzki
R Engineer
Green Communications
___
devel mailing list
devel@uclibc-ng.org
http://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Re: [uclibc-ng-devel] [RFC PATCH v2 0/1] libpthread/nptl: Use arch-dependent unwind-forcedunwind.c if available

[Ignacy Gawędzki]

On Tue, Oct 04, 2016 at 06:46:22AM +0200, thus spake Waldemar Brodkorb:
> Hi Ignacy,
> Ignacy Gawędzki wrote,
> 
> > On Mon, Oct 03, 2016 at 06:13:45PM +0200, thus spake Waldemar Brodkorb:
> > > Hi Ignacy,
> > > Ignacy Gawędzki wrote,
> > > 
> > > > On Mon, Oct 03, 2016 at 12:26:19PM +0200, thus spake Ignacy Gawedzki:
> > > > > On Sat, Oct 01, 2016 at 01:22:37PM +0200, thus spake Waldemar 
> > > > > Brodkorb:
> > > > > > Can you please try with 1.0.18? I can not reproduce the issue with
> > > > > > 1.0.18.
> > > > > > 
> > > > > > I just get "caught" with the test app.
> > > > > > 
> > > > > > May be it was accidentally fixed in the release.
> > > > > 
> > > > > I don't get the uncaught exception with 1.0.18 indeed.  This is
> > > > > certainly due to the way libpthread is now integrated into libc.  Now
> > > > > the version of Unwind_Resume that is called is always the one from
> > > > > libgcc_1.so, even when using threads.  I'm affraid that this is not
> > > > > the right thing to do, because the wrapping code for Unwind_Resume in
> > > > > libpthread was there for some reason.
> > > > 
> > > > Okay, I've explored that problem a bit more today and I have
> > > > additional information.  It seems that the Unwind_* functions are
> > > > provided in the libc for local use and are not supposed to be visible
> > > > for the dynamic linker.  Compare that with GNU libc where the Unwind_*
> > > > symbols are local.
> > > > 
> > > > Right now it also seems that none of uClibc's code actually uses these
> > > > functions, at least in my case, so the incorrect Unwind_Resume inside
> > > > uClibc is harmless.
> > > 
> > > But isn't pthread_cancel using pthread_cancel_init from
> > > libpthread/nptl/sysdeps/pthread/unwind-forcedunwind.c?
> > 
> > Yes, it looks like it does.  But at least _Unwind_Resume doesn't seem
> > to be used at all.
> > 
> > > > The problem I was having with v1.0.17 was that by linking with
> > > > libpthread, the Unwind_Resume wrapper code in that library was
> > > > overriding the implementation in libgcc_s that was supposed to be
> > > > called directly.
> > > 
> > > I still do not understand the issue.
> > > As far as I know, the Unwind* functions in uClibc and GNU libc is
> > > wrapper code around gcc libgcc_s.so.1 which is loaded via dlopen
> > > at runtime.
> > 
> > Yes, they are, but the problem is with the way the ARM unwinder works.
> > Obviously, it doesn't expect to have one additional frame added by a
> > call to _Unwind_Resume.  This is why in the case of the ARM
> > architecture, a specially-crafted version of the wrapper around
> > _Unwind_Resume is provided which doesn't create a new frame.
> > 
> > > I once asked about this on the libc mailinglist:
> > > https://sourceware.org/ml/libc-help/2014-07/msg0.html
> > >  
> > > > Now that libpthread (and librt for that matter) are integrated into
> > > > libc, it seems the problem is gone, because the libgcc_s
> > > > implementation is not overridden by uClibc's.  But this happens only
> > > > because libgcc_s.so is listed before libc.so in the .dynamic section,
> > > > and if it were not the case anymore, for any reason, the problem would
> > > > reemerge immediately.
> > > > 
> > > > You can force the problem to re-emerge by simply preloading libc.so
> > > > using LD_PRELOAD.  This makes uClibc's incorrect implementation of
> > > > Unwind_Resume to be called and the execution of the example code
> > > > aborts instead of outputting "caught".
> > > > 
> > > > Please find attached the updated patch for v1.0.18.  This is more like
> > > > a quick fix, since a proper fix would most probably be to make those
> > > > conflicting symbols local in the resulting .so.
> > > 
> > > But what exactly is fixed by the patch then?
> > 
> > The patch makes _Unwind_Resume in libpthread/nptl/ be implemented by
> > sysdeps/unix/sysv/linux/arm/unwind-forcedunwind.c in the case of ARM
> > and by sysdeps/pthread/unwind-forcedunwind.c otherwise.  So if
> > libc.so's _Unwind_Resume ever takes over libgcc_s.so's, or if the
> > wrapper around _Unwind_Resume is used in the future, it will simply
> > work transparently.
> > 
> > The way this is achieved is inspired by how it's already done for
> > librt with unwind-resume.c (see sysdeps/pthread/rt-unwind-resume.c).
> > 
> > > Is it still a ARM specific issue?
> > 
> > Yes it is.
> > 
> > > Could you take a look at glibc commit:
> > > commit 46abb64d6287d09100b147d062f6810066389b7e
> > > Author: Roland McGrath 
> > > Date:   Mon Jan 5 14:01:49 2015 -0800
> > > 
> > > ARM: Consolidate with generic unwinder wrapper code
> > > 
> > > I think it would be a better solution to sync with glibc and
> > > get rid of the special ARM specific versions.
> > 
> > I you look closely at this commit, it's exactly what I'm talking
> > about: it's providing an ARM-specific implementation of _Unwind_Resume
> > in assembly.  For other architectures, the generic wrapper is used.
> > 
> > Now I agree