Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-15 Thread Stephan
After looking at the code, I´m pretty certain it´s the JavaScriptCore
JIT compiler* who generates some assembly on a heap segment which then
calls library functions with a misaligned stack pointer. The version
of webkit-gtk which I built from a recent pkgsrc release is 2.4.8. The
best thing to do first is trying the current version (2.10) of
webkit-gtk I guess.


* 
https://trac.webkit.org/browser/trunk/Source/JavaScriptCore/assembler/ARMAssembler.h


2015-10-14 7:56 GMT+02:00 Stephan :
> 2015-10-13 21:34 GMT+02:00 Nick Hudson :
>> On 10/13/15 17:58, Stephan wrote:
>>
>>>
>>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>>> /usr/pkg/lib/libglib-2.0.so.0
>>> (gdb) i r $r12
>>> r120x7fffb8c8   2147465416
>>>
>>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>>> /usr/pkg/lib/libglib-2.0.so.0
>>> (gdb) i r $r12
>>> r120x7fffb870   2147465328
>>>
>>> Contrary, sp is broken in the non-working case:
>>>
>>> (gdb) i r $r12
>>> r120x7fffa414   2147460116
>>>
>>> Unfortunately, the call trace is incomplete in that case:
>>
>> r13 is sp.
>
> Sure. When I use gdb, I set a breakpoint on . In practise,
> the breakpoint hits after the stack frame was set up by that function,
> e.g. when sp was already lowered to allocate space for local variables
> (or the stack pointer was potentially aligned like gcc on x86 does).
> On gcc/ARM, ip (r12) gets a copy of sp on the first instruction of the
> function prologue. That´s why I monitored r12 (another solution could
> be to break on *function+0, but I haven´t tested that).
>
>>
>> debug symbols (compile flag -g) will help a lot here.
>>
>
> Uff... ATM it looks like Webkit is at fault. It took 2 days to compile
> on the RPI2 so it could take a long time to get the symbols ready.
> Also the last thing I´ve seen yesterday was that the frame where gdb
> stopped being able to follow the stack was special. Basically
> speaking, I think the condition that makes this crash happen is when a
> new tab/page is supposed to open from a JavaScript event - so it´s
> likely that the JS interpreter is on the call trace. The code to the
> said frame where gdb stops was executable memory (rwx) residing in an
> [anon] region. I´m not sure whether symbols can help in this case nor
> what else can... ;)
>
>>
>> Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-14 Thread Stephan
Just a side note: The official Pi folks have been working on
optimizations specific to the Pi / armv6 architecture:

http://blog.barisione.org/2014-09/rpi-browser/

Among these are "JavaScript JIT fixes for ARMv6". Perhaps that version
works better (also in terms of performace and stability), though I
don´t know where the source code is. Fixes solving our issues might
even be included in a more recent version of webkit-gtk, so it might
make sense to check that first.


2015-10-14 7:56 GMT+02:00 Stephan :
> 2015-10-13 21:34 GMT+02:00 Nick Hudson :
>> On 10/13/15 17:58, Stephan wrote:
>>
>>>
>>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>>> /usr/pkg/lib/libglib-2.0.so.0
>>> (gdb) i r $r12
>>> r120x7fffb8c8   2147465416
>>>
>>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>>> /usr/pkg/lib/libglib-2.0.so.0
>>> (gdb) i r $r12
>>> r120x7fffb870   2147465328
>>>
>>> Contrary, sp is broken in the non-working case:
>>>
>>> (gdb) i r $r12
>>> r120x7fffa414   2147460116
>>>
>>> Unfortunately, the call trace is incomplete in that case:
>>
>> r13 is sp.
>
> Sure. When I use gdb, I set a breakpoint on . In practise,
> the breakpoint hits after the stack frame was set up by that function,
> e.g. when sp was already lowered to allocate space for local variables
> (or the stack pointer was potentially aligned like gcc on x86 does).
> On gcc/ARM, ip (r12) gets a copy of sp on the first instruction of the
> function prologue. That´s why I monitored r12 (another solution could
> be to break on *function+0, but I haven´t tested that).
>
>>
>> debug symbols (compile flag -g) will help a lot here.
>>
>
> Uff... ATM it looks like Webkit is at fault. It took 2 days to compile
> on the RPI2 so it could take a long time to get the symbols ready.
> Also the last thing I´ve seen yesterday was that the frame where gdb
> stopped being able to follow the stack was special. Basically
> speaking, I think the condition that makes this crash happen is when a
> new tab/page is supposed to open from a JavaScript event - so it´s
> likely that the JS interpreter is on the call trace. The code to the
> said frame where gdb stops was executable memory (rwx) residing in an
> [anon] region. I´m not sure whether symbols can help in this case nor
> what else can... ;)
>
>>
>> Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-13 Thread Stephan
2015-10-13 21:34 GMT+02:00 Nick Hudson :
> On 10/13/15 17:58, Stephan wrote:
>
>>
>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>> /usr/pkg/lib/libglib-2.0.so.0
>> (gdb) i r $r12
>> r120x7fffb8c8   2147465416
>>
>> Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from
>> /usr/pkg/lib/libglib-2.0.so.0
>> (gdb) i r $r12
>> r120x7fffb870   2147465328
>>
>> Contrary, sp is broken in the non-working case:
>>
>> (gdb) i r $r12
>> r120x7fffa414   2147460116
>>
>> Unfortunately, the call trace is incomplete in that case:
>
> r13 is sp.

Sure. When I use gdb, I set a breakpoint on . In practise,
the breakpoint hits after the stack frame was set up by that function,
e.g. when sp was already lowered to allocate space for local variables
(or the stack pointer was potentially aligned like gcc on x86 does).
On gcc/ARM, ip (r12) gets a copy of sp on the first instruction of the
function prologue. That´s why I monitored r12 (another solution could
be to break on *function+0, but I haven´t tested that).

>
> debug symbols (compile flag -g) will help a lot here.
>

Uff... ATM it looks like Webkit is at fault. It took 2 days to compile
on the RPI2 so it could take a long time to get the symbols ready.
Also the last thing I´ve seen yesterday was that the frame where gdb
stopped being able to follow the stack was special. Basically
speaking, I think the condition that makes this crash happen is when a
new tab/page is supposed to open from a JavaScript event - so it´s
likely that the JS interpreter is on the call trace. The code to the
said frame where gdb stops was executable memory (rwx) residing in an
[anon] region. I´m not sure whether symbols can help in this case nor
what else can... ;)

>
> Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-13 Thread Nick Hudson

On 05/31/15 18:07, Stephan wrote:

Hi folks,

I am currently testing some applications on the RPI 2. Some work
pretty well, others not yet. As for webkit-gtk based browsers, I am
experiencing crashes from time to time.

One problem that occurs often seems to be related to g_dpgettext2 ()
from glib2. The top of the stack looks like this:

(gdb) bt
#0  0x636f7452 in ?? ()
#1  0x45ff3fa8 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
#2  0x42ad6030 in gtk_stock_lookup () from /usr/pkg/lib/libgtk-x11-2.0.so.0
#3  0x42987b98 in gtk_action_set_stock_id () from 
/usr/pkg/lib/libgtk-x11-2.0.so.0
#4  0x45f55cfc in g_object_set_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#5  0x45f5642c in g_object_set () from /usr/pkg/lib/libgobject-2.0.so.0
#6  0x4298a27c in gtk_action_group_add_actions_full () from 
/usr/pkg/lib/libgtk-x11-2.0.so.0
#7  0x4298a388 in gtk_action_group_add_actions () from 
/usr/pkg/lib/libgtk-x11-2.0.so.0
#8  0x0004238c in ?? ()
#9  0x45f5322c in g_object_new_internal () from /usr/pkg/lib/libgobject-2.0.so.0
#10 0x45f5587c in g_object_new_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#11 0x45f55a24 in g_object_new () from /usr/pkg/lib/libgobject-2.0.so.0
#12 0x00043ff0 in ephy_window_new_with_chrome ()
#13 0x0003ac94 in ephy_shell_new_tab_full ()
#14 0x0003f81c in ?? ()
#15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from 
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#16 0x45f4e070 in g_closure_invoke () from /usr/pkg/lib/libgobject-2.0.so.0
#17 0x45f6154c in signal_emit_unlocked_R () from 
/usr/pkg/lib/libgobject-2.0.so.0
#18 0x45f69278 in g_signal_emit_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#19 0x45f69cac in g_signal_emit_by_name () from /usr/pkg/lib/libgobject-2.0.so.0
#20 0x409d9074 in 
WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction const&) 
() from /usr/pkg/lib/libwebkitgtk-1.0.so.0
...


I'm pretty sure the problem is that somewhere in the call stack the
sp isn't 8-byte aligned and the alloca in g_dpgettext2 falls over this

Move up the frames doing up; info frame (or similar)

Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-13 Thread Stephan
Thanks all for your input. Nick was right that the stack pointer needs
to be aligned on a 8-byte boundary. I was totally unaware that the
calling convention requires this. The g_dpgettext2() function runs
successfully a couple of times when epiphany launches. It is then
called with an appropriately aligned stack pointer:

Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
(gdb) i r $r12
r120x7fffb8c8   2147465416

Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
(gdb) i r $r12
r120x7fffb870   2147465328

Contrary, sp is broken in the non-working case:

(gdb) i r $r12
r120x7fffa414   2147460116

Unfortunately, the call trace is incomplete in that case:

#35 0x40c42178 in
WebCore::EventTarget::dispatchEvent(WTF::PassRefPtr,
int&) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
#36 0x4147fd84 in
WebCore::jsNodePrototypeFunctionDispatchEvent(JSC::ExecState*) () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#37 0x43fc81a4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

The only thing I can say just now is that sp is already misaligned in
frame 36 when it comes out of this obfuscated frame 37. I set a
separate breakpoint as it seems not too easy to get the value of sp
for each frame on a full call trace:


Breakpoint 1, 0x4147fc94 in
WebCore::jsNodePrototypeFunctionDispatchEvent(JSC::ExecState*) () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
(gdb) bt
#0  0x4147fc94 in
WebCore::jsNodePrototypeFunctionDispatchEvent(JSC::ExecState*) () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#1  0x50add124 in ?? ()

(gdb) i r $r12
r120x7fffb88c 2147465356


This is not too easy :)

2015-10-13 8:17 GMT+00:00 Nick Hudson :
> On 05/31/15 18:07, Stephan wrote:
>>
>> Hi folks,
>>
>> I am currently testing some applications on the RPI 2. Some work
>> pretty well, others not yet. As for webkit-gtk based browsers, I am
>> experiencing crashes from time to time.
>>
>> One problem that occurs often seems to be related to g_dpgettext2 ()
>> from glib2. The top of the stack looks like this:
>>
>> (gdb) bt
>> #0  0x636f7452 in ?? ()
>> #1  0x45ff3fa8 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
>> #2  0x42ad6030 in gtk_stock_lookup () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>> #3  0x42987b98 in gtk_action_set_stock_id () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>> #4  0x45f55cfc in g_object_set_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #5  0x45f5642c in g_object_set () from /usr/pkg/lib/libgobject-2.0.so.0
>> #6  0x4298a27c in gtk_action_group_add_actions_full () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>> #7  0x4298a388 in gtk_action_group_add_actions () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>> #8  0x0004238c in ?? ()
>> #9  0x45f5322c in g_object_new_internal () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #10 0x45f5587c in g_object_new_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #11 0x45f55a24 in g_object_new () from /usr/pkg/lib/libgobject-2.0.so.0
>> #12 0x00043ff0 in ephy_window_new_with_chrome ()
>> #13 0x0003ac94 in ephy_shell_new_tab_full ()
>> #14 0x0003f81c in ?? ()
>> #15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
>> /usr/pkg/lib/libwebkitgtk-1.0.so.0
>> #16 0x45f4e070 in g_closure_invoke () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #17 0x45f6154c in signal_emit_unlocked_R () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #18 0x45f69278 in g_signal_emit_valist () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #19 0x45f69cac in g_signal_emit_by_name () from
>> /usr/pkg/lib/libgobject-2.0.so.0
>> #20 0x409d9074 in
>> WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
>> const&) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
>> ...
>
>
> I'm pretty sure the problem is that somewhere in the call stack the
> sp isn't 8-byte aligned and the alloca in g_dpgettext2 falls over this
>
> Move up the frames doing up; info frame (or similar)
>
> Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-13 Thread Nick Hudson

On 10/13/15 17:58, Stephan wrote:



Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
(gdb) i r $r12
r120x7fffb8c8   2147465416

Breakpoint 1, 0x46213ff4 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
(gdb) i r $r12
r120x7fffb870   2147465328

Contrary, sp is broken in the non-working case:

(gdb) i r $r12
r120x7fffa414   2147460116

Unfortunately, the call trace is incomplete in that case:

r13 is sp.

debug symbols (compile flag -g) will help a lot here.


Nick


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-10 Thread Rhialto
On Thu 08 Oct 2015 at 17:59:22 +, Stephan wrote:
> IMHO it would be valid to pass the value of sp as the first argument
> (mov r0, sp). However, the way how r0 is computed looks quite
> obfuscated to me:
> 
>0x4621402c : mov r3, sp=> ?
>0x46214030 : lsr r7, r3, #3   => ?
>0x46214034 : lsl r4, r7, #3   => ?
...
>0x46214040 : mov r0, r4  => ?

I don't really know any ARM assembly but it seems that the value from sp
is shifted right 3 bits and then left again. In effect clearing the low
3 bits, or aligning to 8. Which by itself is a very sensible thing to do
for an allocator.

Unfortunately (since it rounds down) it can have the effect of
allocating the memory lower than sp, which is bound to fail at some
point. Probably the value should also go back into sp.

There is also an alloca in libc: cvs/src/lib/libc/arch/arm/gen/alloca.S.
It uses a similar shifting trick but it uses it to round up the
allocation size to multiples of 8, and it does end up with the same
value in r0 and sp.

-Olaf.
-- 
___ Olaf 'Rhialto' Seibert  -- The Doctor: No, 'eureka' is Greek for
\X/ rhialto/at/xs4all.nl-- 'this bath is too hot.'


pgpFCM8YmMK22.pgp
Description: PGP signature


Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-10-06 Thread Stephan
Folks,

more on this topic:

This

-8<--
(gdb) x/5i 0x45fd3efc
   0x45fd3efc:  add r12, pc, #1048576   ; 0x10
   0x45fd3f00:  add r12, r12, #4096 ; 0x1000
   0x45fd3f04:  ldr pc, [r12, #1652]!   ; 0x674
   0x45fd3f08:  add r12, pc, #1048576   ; 0x10
   0x45fd3f0c:  add r12, r12, #4096 ; 0x1000
---

is not garbage. It's kind of a link table (plt) inside of glib2. It
makes the first 2 calls in g_dpgettext2() go to strlen() and the third
go to memcpy(). memcpy() saves the r0 and lr regisers at the very
beginning and pops them into r0 and pc at a later time in order to
return to the caller. However somewhere in the function, the stack
data seems to become corrupted and a junk value is loaded into pc,
leading to the crash. Either its value was overwritten or sp is
pointing to the wrong place.


(gdb) x/50i 0x45e80f98
   0x45e80f98 : push {r0, lr}=> SAVE REGISTERS
   0x45e80f9c : subs r2, r2, #4
   0x45e80fa0 : blt 0x45e81028 
   0x45e80fa4 : ands r12, r0, #3
   0x45e80fa8 : bne 0x45e81050 
   0x45e80fac : ands r12, r1, #3
   0x45e80fb0 : bne 0x45e81080 
   0x45e80fb4 : subs r2, r2, #8
   0x45e80fb8 : blt 0x45e81008 
   0x45e80fbc : subs r2, r2, #20
   0x45e80fc0 : blt 0x45e80ff4 
   0x45e80fc4 : push {r4} ; (str r4, [sp, #-4]!)
   0x45e80fc8 : ldm r1!, {r3, r4, r12, lr}
   0x45e80fcc : stmia r0!, {r3, r4, r12, lr}
   0x45e80fd0 : ldm r1!, {r3, r4, r12, lr}
   0x45e80fd4 : stmia r0!, {r3, r4, r12, lr}
   0x45e80fd8 : subs r2, r2, #32
   0x45e80fdc : bge 0x45e80fc8 
   0x45e80fe0 : cmn r2, #16
   0x45e80fe4 : ldmge r1!, {r3, r4, r12, lr}
   0x45e80fe8 : stmiage r0!, {r3, r4, r12, lr}
   0x45e80fec : subge r2, r2, #16
   0x45e80ff0 : pop {r4} ; (ldr r4, [sp], #4)
   0x45e80ff4 : adds r2, r2, #20
   0x45e80ff8 : ldmge r1!, {r3, r12, lr}
   0x45e80ffc : stmiage r0!, {r3, r12, lr}
   0x45e81000 : subsge r2, r2, #12
   0x45e81004 : bge 0x45e80ff8 
   0x45e81008 : adds r2, r2, #8
   0x45e8100c : blt 0x45e81028 
   0x45e81010 : subs r2, r2, #4
   0x45e81014 : ldrlt r3, [r1], #4
   0x45e81018 : strlt r3, [r0], #4
   0x45e8101c : ldmge r1!, {r3, r12}
   0x45e81020 : stmiage r0!, {r3, r12}
   0x45e81024 : subge r2, r2, #4
   0x45e81028 : adds r2, r2, #4
   0x45e8102c : popeq {r0, pc}
   0x45e81030 : cmp r2, #2
   0x45e81034 : ldrb r3, [r1], #1
   0x45e81038 : strb r3, [r0], #1
   0x45e8103c : ldrbge r3, [r1], #1
   0x45e81040 : strbge r3, [r0], #1
   0x45e81044 : ldrbgt r3, [r1], #1
   0x45e81048 : strbgt r3, [r0], #1
=> 0x45e8104c : pop {r0, pc}   ==> CRASH: PC
gets corrupted value
   0x45e81050 : rsb r12, r12, #4

Further investigation is to come ;)

2015-05-31 19:05 GMT+00:00 Jared McNeill :
> I saw the same issue a while back, don't remember exactly which version of
> glib2 it was. I LD_PRELOAD'd a version of g_dpgettext2 that always returned
> NULL and it made the crashes go away. g_dpgettext2 uses alloca internally,
> so I increased stack size and the crashes went away again.
>
> https://git.gnome.org/browse/glib/tree/glib/ggettext.c#n271
>
>
> On Sun, 31 May 2015, Stephan wrote:
>
>>
>> That does not help unfortunately - just tested again with ulimit -s
>> unlimited. I dont think we are hitting the stack boundary
>> because sp is still inside the stack region outlined by pmap. Have you
>> seen the very same issue as I do?
>>
>> Am 31.05.2015 19:48 schrieb "Jared McNeill" :
>>   Default stack size limits on evbarm are too low for webkit-gtk. Bump
>> it up with ulimit -s and the g_dpgettext2
>>   problem should go away.
>>
>>
>>
>>   On Sun, 31 May 2015, Stephan wrote:
>>
>> Hi folks,
>>
>> I am currently testing some applications on the RPI 2. Some
>> work
>> pretty well, others not yet. As for webkit-gtk based browsers,
>> I am
>> experiencing crashes from time to time.
>>
>> One problem that occurs often seems to be related to
>> g_dpgettext2 ()
>> from glib2. The top of the stack looks like this:
>>
>> (gdb) bt
>> #0  0x636f7452 in ?? ()
>> #1  0x45ff3fa8 in g_dpgettext2 () from
>> /usr/pkg/lib/libglib-2.0.so.0
>> #2  0x42ad6030 in gtk_stock_lookup () from
>> /usr/pkg/lib/libgtk-x11-2.0.so.0
>> #3  0x42987b98 in 

Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-05-31 Thread Jared McNeill
Default stack size limits on evbarm are too low for webkit-gtk. Bump it up 
with ulimit -s and the g_dpgettext2 problem should go away.




On Sun, 31 May 2015, Stephan wrote:


Hi folks,

I am currently testing some applications on the RPI 2. Some work
pretty well, others not yet. As for webkit-gtk based browsers, I am
experiencing crashes from time to time.

One problem that occurs often seems to be related to g_dpgettext2 ()
from glib2. The top of the stack looks like this:

(gdb) bt
#0  0x636f7452 in ?? ()
#1  0x45ff3fa8 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
#2  0x42ad6030 in gtk_stock_lookup () from /usr/pkg/lib/libgtk-x11-2.0.so.0
#3  0x42987b98 in gtk_action_set_stock_id () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#4  0x45f55cfc in g_object_set_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#5  0x45f5642c in g_object_set () from /usr/pkg/lib/libgobject-2.0.so.0
#6  0x4298a27c in gtk_action_group_add_actions_full () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#7  0x4298a388 in gtk_action_group_add_actions () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#8  0x0004238c in ?? ()
#9  0x45f5322c in g_object_new_internal () from /usr/pkg/lib/libgobject-2.0.so.0
#10 0x45f5587c in g_object_new_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#11 0x45f55a24 in g_object_new () from /usr/pkg/lib/libgobject-2.0.so.0
#12 0x00043ff0 in ephy_window_new_with_chrome ()
#13 0x0003ac94 in ephy_shell_new_tab_full ()
#14 0x0003f81c in ?? ()
#15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#16 0x45f4e070 in g_closure_invoke () from /usr/pkg/lib/libgobject-2.0.so.0
#17 0x45f6154c in signal_emit_unlocked_R () from
/usr/pkg/lib/libgobject-2.0.so.0
#18 0x45f69278 in g_signal_emit_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#19 0x45f69cac in g_signal_emit_by_name () from /usr/pkg/lib/libgobject-2.0.so.0
#20 0x409d9074 in
WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
const) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
...

The call to the topmost frame happens from 0x45ff3fa4 in the said
funtion. The disassembly from within gdb loos like this:

(gdb) disass g_dpgettext2
Dump of assembler code for function g_dpgettext2:
  0x45ff3f44 +0: mov r12, sp
  0x45ff3f48 +4: push{r4, r5, r6, r7, r8, r9, r10, r11, r12, lr, pc}
  0x45ff3f4c +8: sub r11, r12, #4
  0x45ff3f50 +12:sub sp, sp, #12
  0x45ff3f54 +16:mov r9, r0
  0x45ff3f58 +20:mov r0, r1
  0x45ff3f5c +24:mov r6, r2
  0x45ff3f60 +28:str r1, [r11, #-48] ; 0x30
  0x45ff3f64 +32:bl  0x45fd3f44
  0x45ff3f68 +36:mov r5, r0
  0x45ff3f6c +40:mov r0, r6
  0x45ff3f70 +44:bl  0x45fd3f44
  0x45ff3f74 +48:add r10, r5, #1
  0x45ff3f78 +52:add r8, r0, #1
  0x45ff3f7c +56:add r3, r8, r10
  0x45ff3f80 +60:add r3, r3, #14
  0x45ff3f84 +64:bic r3, r3, #7
  0x45ff3f88 +68:sub sp, sp, r3
  0x45ff3f8c +72:mov r3, sp
  0x45ff3f90 +76:lsr r7, r3, #3
  0x45ff3f94 +80:lsl r4, r7, #3
  0x45ff3f98 +84:ldr r1, [r11, #-48] ; 0x30
  0x45ff3f9c +88:mov r2, r5
  0x45ff3fa0 +92:mov r0, r4
  0x45ff3fa4 +96:bl  0x45fd3efc
  0x45ff3fa8 +100:   mov r3, #4
  0x45ff3fac +104:   mov r2, r8
  0x45ff3fb0 +108:   strbr3, [r5, r7, lsl #3]
  ...

Now the jump target actually seems to be some random memory:

(gdb) info symbol 0x45fd3efc
No symbol matches 0x45fd3efc.

At least it is mapped to the text section of libglib2. It decodes like
follows which just looks like random garbage:

(gdb) x/5i 0x45fd3efc
  0x45fd3efc:  add r12, pc, #1048576   ; 0x10
  0x45fd3f00:  add r12, r12, #4096 ; 0x1000
  0x45fd3f04:  ldr pc, [r12, #1652]!   ; 0x674
  0x45fd3f08:  add r12, pc, #1048576   ; 0x10
  0x45fd3f0c:  add r12, r12, #4096 ; 0x1000

The disassembly of g_dpgettext2() from objdump looks weird as well:

00033f44 g_dpgettext2:
  33f44:   e1a0c00dmov ip, sp
  33f48:   e92ddff0push{r4, r5, r6, r7, r8, r9, sl,
fp, ip, lr, pc}
  33f4c:   e24cb004sub fp, ip, #4
  33f50:   e24dd00csub sp, sp, #12
  33f54:   e1a09000mov r9, r0
  33f58:   e1a1mov r0, r1
  33f5c:   e1a06002mov r6, r2
  33f60:   e50b1030str r1, [fp, #-48]  ; 0x30
  33f64:   ebff7ff6bl  13f44 g_mem_chunk_new-0x7c
  33f68:   e1a05000mov r5, r0
  33f6c:   e1a6mov r0, r6
  33f70:   ebff7ff3bl  13f44 g_mem_chunk_new-0x7c
  33f74:   e285a001add sl, r5, #1
  33f78:   e2808001add r8, r0, #1
  33f7c:   e088300aadd r3, r8, sl
  33f80:   e283300eadd r3, r3, #14
  33f84:   e3c33007bic r3, r3, #7
  33f88:   e04dd003sub sp, sp, r3
  33f8c:   e1a0300dmov r3, sp
  33f90: 

Re: Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-05-31 Thread Jared McNeill
I saw the same issue a while back, don't remember exactly which version of 
glib2 it was. I LD_PRELOAD'd a version of g_dpgettext2 that always 
returned NULL and it made the crashes go away. g_dpgettext2 uses alloca 
internally, so I increased stack size and the crashes went away again.


https://git.gnome.org/browse/glib/tree/glib/ggettext.c#n271

On Sun, 31 May 2015, Stephan wrote:



That does not help unfortunately - just tested again with ulimit -s unlimited. 
I dont think we are hitting the stack boundary
because sp is still inside the stack region outlined by pmap. Have you seen the 
very same issue as I do?

Am 31.05.2015 19:48 schrieb Jared McNeill jmcne...@invisible.ca:
  Default stack size limits on evbarm are too low for webkit-gtk. Bump it 
up with ulimit -s and the g_dpgettext2
  problem should go away.



  On Sun, 31 May 2015, Stephan wrote:

Hi folks,

I am currently testing some applications on the RPI 2. Some work
pretty well, others not yet. As for webkit-gtk based browsers, I am
experiencing crashes from time to time.

One problem that occurs often seems to be related to g_dpgettext2 ()
from glib2. The top of the stack looks like this:

(gdb) bt
#0  0x636f7452 in ?? ()
#1  0x45ff3fa8 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
#2  0x42ad6030 in gtk_stock_lookup () from 
/usr/pkg/lib/libgtk-x11-2.0.so.0
#3  0x42987b98 in gtk_action_set_stock_id () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#4  0x45f55cfc in g_object_set_valist () from 
/usr/pkg/lib/libgobject-2.0.so.0
#5  0x45f5642c in g_object_set () from 
/usr/pkg/lib/libgobject-2.0.so.0
#6  0x4298a27c in gtk_action_group_add_actions_full () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#7  0x4298a388 in gtk_action_group_add_actions () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#8  0x0004238c in ?? ()
#9  0x45f5322c in g_object_new_internal () from 
/usr/pkg/lib/libgobject-2.0.so.0
#10 0x45f5587c in g_object_new_valist () from 
/usr/pkg/lib/libgobject-2.0.so.0
#11 0x45f55a24 in g_object_new () from 
/usr/pkg/lib/libgobject-2.0.so.0
#12 0x00043ff0 in ephy_window_new_with_chrome ()
#13 0x0003ac94 in ephy_shell_new_tab_full ()
#14 0x0003f81c in ?? ()
#15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#16 0x45f4e070 in g_closure_invoke () from 
/usr/pkg/lib/libgobject-2.0.so.0
#17 0x45f6154c in signal_emit_unlocked_R () from
/usr/pkg/lib/libgobject-2.0.so.0
#18 0x45f69278 in g_signal_emit_valist () from 
/usr/pkg/lib/libgobject-2.0.so.0
#19 0x45f69cac in g_signal_emit_by_name () from 
/usr/pkg/lib/libgobject-2.0.so.0
#20 0x409d9074 in

WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
const) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
...

The call to the topmost frame happens from 0x45ff3fa4 in the said
funtion. The disassembly from within gdb loos like this:

(gdb) disass g_dpgettext2
Dump of assembler code for function g_dpgettext2:
  0x45ff3f44 +0:     mov     r12, sp
  0x45ff3f48 +4:     push    {r4, r5, r6, r7, r8, r9, r10, r11, 
r12, lr, pc}
  0x45ff3f4c +8:     sub     r11, r12, #4
  0x45ff3f50 +12:    sub     sp, sp, #12
  0x45ff3f54 +16:    mov     r9, r0
  0x45ff3f58 +20:    mov     r0, r1
  0x45ff3f5c +24:    mov     r6, r2
  0x45ff3f60 +28:    str     r1, [r11, #-48] ; 0x30
  0x45ff3f64 +32:    bl      0x45fd3f44
  0x45ff3f68 +36:    mov     r5, r0
  0x45ff3f6c +40:    mov     r0, r6
  0x45ff3f70 +44:    bl      0x45fd3f44
  0x45ff3f74 +48:    add     r10, r5, #1
  0x45ff3f78 +52:    add     r8, r0, #1
  0x45ff3f7c +56:    add     r3, r8, r10
  0x45ff3f80 +60:    add     r3, r3, #14
  0x45ff3f84 +64:    bic     r3, r3, #7
  0x45ff3f88 +68:    sub     sp, sp, r3
  0x45ff3f8c +72:    mov     r3, sp
  0x45ff3f90 +76:    lsr     r7, r3, #3
  0x45ff3f94 +80:    lsl     r4, r7, #3
  0x45ff3f98 +84:    ldr     r1, [r11, #-48] ; 0x30
  0x45ff3f9c +88:    mov     r2, r5
  0x45ff3fa0 +92:    mov     r0, r4
  0x45ff3fa4 +96:    bl      0x45fd3efc
  0x45ff3fa8 +100:   mov     r3, #4
  0x45ff3fac +104:   mov     r2, r8
  0x45ff3fb0 +108:   strb    r3, [r5, r7, lsl #3]
  ...

Now the jump target actually seems to be some random memory:


Debugging Epiphany/Midori (webkit-gtk based) on earmv6hf (RPI 2)

2015-05-31 Thread Stephan
Hi folks,

I am currently testing some applications on the RPI 2. Some work
pretty well, others not yet. As for webkit-gtk based browsers, I am
experiencing crashes from time to time.

One problem that occurs often seems to be related to g_dpgettext2 ()
from glib2. The top of the stack looks like this:

(gdb) bt
#0  0x636f7452 in ?? ()
#1  0x45ff3fa8 in g_dpgettext2 () from /usr/pkg/lib/libglib-2.0.so.0
#2  0x42ad6030 in gtk_stock_lookup () from /usr/pkg/lib/libgtk-x11-2.0.so.0
#3  0x42987b98 in gtk_action_set_stock_id () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#4  0x45f55cfc in g_object_set_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#5  0x45f5642c in g_object_set () from /usr/pkg/lib/libgobject-2.0.so.0
#6  0x4298a27c in gtk_action_group_add_actions_full () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#7  0x4298a388 in gtk_action_group_add_actions () from
/usr/pkg/lib/libgtk-x11-2.0.so.0
#8  0x0004238c in ?? ()
#9  0x45f5322c in g_object_new_internal () from /usr/pkg/lib/libgobject-2.0.so.0
#10 0x45f5587c in g_object_new_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#11 0x45f55a24 in g_object_new () from /usr/pkg/lib/libgobject-2.0.so.0
#12 0x00043ff0 in ephy_window_new_with_chrome ()
#13 0x0003ac94 in ephy_shell_new_tab_full ()
#14 0x0003f81c in ?? ()
#15 0x40a090e8 in webkit_marshal_OBJECT__OBJECT () from
/usr/pkg/lib/libwebkitgtk-1.0.so.0
#16 0x45f4e070 in g_closure_invoke () from /usr/pkg/lib/libgobject-2.0.so.0
#17 0x45f6154c in signal_emit_unlocked_R () from
/usr/pkg/lib/libgobject-2.0.so.0
#18 0x45f69278 in g_signal_emit_valist () from /usr/pkg/lib/libgobject-2.0.so.0
#19 0x45f69cac in g_signal_emit_by_name () from /usr/pkg/lib/libgobject-2.0.so.0
#20 0x409d9074 in
WebKit::FrameLoaderClient::dispatchCreatePage(WebCore::NavigationAction
const) () from /usr/pkg/lib/libwebkitgtk-1.0.so.0
...

The call to the topmost frame happens from 0x45ff3fa4 in the said
funtion. The disassembly from within gdb loos like this:

(gdb) disass g_dpgettext2
Dump of assembler code for function g_dpgettext2:
   0x45ff3f44 +0: mov r12, sp
   0x45ff3f48 +4: push{r4, r5, r6, r7, r8, r9, r10, r11, r12, lr, pc}
   0x45ff3f4c +8: sub r11, r12, #4
   0x45ff3f50 +12:sub sp, sp, #12
   0x45ff3f54 +16:mov r9, r0
   0x45ff3f58 +20:mov r0, r1
   0x45ff3f5c +24:mov r6, r2
   0x45ff3f60 +28:str r1, [r11, #-48] ; 0x30
   0x45ff3f64 +32:bl  0x45fd3f44
   0x45ff3f68 +36:mov r5, r0
   0x45ff3f6c +40:mov r0, r6
   0x45ff3f70 +44:bl  0x45fd3f44
   0x45ff3f74 +48:add r10, r5, #1
   0x45ff3f78 +52:add r8, r0, #1
   0x45ff3f7c +56:add r3, r8, r10
   0x45ff3f80 +60:add r3, r3, #14
   0x45ff3f84 +64:bic r3, r3, #7
   0x45ff3f88 +68:sub sp, sp, r3
   0x45ff3f8c +72:mov r3, sp
   0x45ff3f90 +76:lsr r7, r3, #3
   0x45ff3f94 +80:lsl r4, r7, #3
   0x45ff3f98 +84:ldr r1, [r11, #-48] ; 0x30
   0x45ff3f9c +88:mov r2, r5
   0x45ff3fa0 +92:mov r0, r4
   0x45ff3fa4 +96:bl  0x45fd3efc
   0x45ff3fa8 +100:   mov r3, #4
   0x45ff3fac +104:   mov r2, r8
   0x45ff3fb0 +108:   strbr3, [r5, r7, lsl #3]
   ...

Now the jump target actually seems to be some random memory:

(gdb) info symbol 0x45fd3efc
No symbol matches 0x45fd3efc.

At least it is mapped to the text section of libglib2. It decodes like
follows which just looks like random garbage:

(gdb) x/5i 0x45fd3efc
   0x45fd3efc:  add r12, pc, #1048576   ; 0x10
   0x45fd3f00:  add r12, r12, #4096 ; 0x1000
   0x45fd3f04:  ldr pc, [r12, #1652]!   ; 0x674
   0x45fd3f08:  add r12, pc, #1048576   ; 0x10
   0x45fd3f0c:  add r12, r12, #4096 ; 0x1000

The disassembly of g_dpgettext2() from objdump looks weird as well:

00033f44 g_dpgettext2:
   33f44:   e1a0c00dmov ip, sp
   33f48:   e92ddff0push{r4, r5, r6, r7, r8, r9, sl,
fp, ip, lr, pc}
   33f4c:   e24cb004sub fp, ip, #4
   33f50:   e24dd00csub sp, sp, #12
   33f54:   e1a09000mov r9, r0
   33f58:   e1a1mov r0, r1
   33f5c:   e1a06002mov r6, r2
   33f60:   e50b1030str r1, [fp, #-48]  ; 0x30
   33f64:   ebff7ff6bl  13f44 g_mem_chunk_new-0x7c
   33f68:   e1a05000mov r5, r0
   33f6c:   e1a6mov r0, r6
   33f70:   ebff7ff3bl  13f44 g_mem_chunk_new-0x7c
   33f74:   e285a001add sl, r5, #1
   33f78:   e2808001add r8, r0, #1
   33f7c:   e088300aadd r3, r8, sl
   33f80:   e283300eadd r3, r3, #14
   33f84:   e3c33007bic r3, r3, #7
   33f88:   e04dd003sub sp, sp, r3
   33f8c:   e1a0300dmov r3, sp
   33f90:   e1a071a3lsr r7, r3, #3
   33f94:   e1a04187lsl r4, r7, #3
   33f98:   e51b1030