Re: gcc: optimizations, and stack traces
Le 09/02/2018 à 13:32, Joerg Sonnenberger a écrit : On Fri, Feb 09, 2018 at 11:23:17AM +0100, Maxime Villard wrote: It implies that if a bug occurs _before_ these two instructions are executed, we have a %rbp that points to the _previous_ function, the one we got called from. And therefore, GDB does not display the current function (where the bug actually happened), but displays its caller. This analysis is wrong. GDB will first of all look for frame annotation data, i.e. .eh_frame or the corresponding .debug_frame. Only if it can't find such annotation will it fall back to guessing from the function itself. We default to building .eh_frame for all binaries, but I'm not completely sure if GCC will create async unwind tables by default. I'll have to re-check the GDB code, but that the previous function was displayed and not the current one is the conclusion I came to back then. Will verify tomorrow. Maxime
Re: gcc: optimizations, and stack traces
Le 09/02/2018 à 12:13, Valery Ushakov a écrit : [Summoning Krister] On Fri, Feb 09, 2018 at 11:23:17 +0100, Maxime Villard wrote: There are also several cases where functions in the call tree can disappear from the backtrace. In the following call tree: A -> B -> C -> D (and D panics) if, in B, GCC put the two instructions after the instruction that calls C, the backtrace will be: A -> C -> D This can make a bug completely undebuggable. Does gcc actually generates code like that? I thought that it can delay frame pointer creation, but only until it needs to make a nested call, to C in your example, (as in the sample I showed in another mail to this thread). Mmh, now I'm not so sure about this. Wait a minute, I'll re-give a look and try to understand what I was doing. Maxime
Re: gcc: optimizations, and stack traces
On Fri, Feb 09, 2018 at 11:23:17AM +0100, Maxime Villard wrote: > It implies that if a bug occurs _before_ these two instructions are executed, > we have a %rbp that points to the _previous_ function, the one we got called > from. And therefore, GDB does not display the current function (where the bug > actually happened), but displays its caller. This analysis is wrong. GDB will first of all look for frame annotation data, i.e. .eh_frame or the corresponding .debug_frame. Only if it can't find such annotation will it fall back to guessing from the function itself. We default to building .eh_frame for all binaries, but I'm not completely sure if GCC will create async unwind tables by default. Joerg
Re: gcc: optimizations, and stack traces
[Summoning Krister] On Fri, Feb 09, 2018 at 11:23:17 +0100, Maxime Villard wrote: > There are also several cases where functions in the call tree can disappear > from the backtrace. In the following call tree: > > A -> B -> C -> D (and D panics) > > if, in B, GCC put the two instructions after the instruction that calls C, > the backtrace will be: > > A -> C -> D > > This can make a bug completely undebuggable. Does gcc actually generates code like that? I thought that it can delay frame pointer creation, but only until it needs to make a nested call, to C in your example, (as in the sample I showed in another mail to this thread). -uwe
Re: gcc: optimizations, and stack traces
Le 09/02/2018 à 12:08, Valery Ushakov a écrit :
On Fri, Feb 09, 2018 at 11:38:47 +0100, Martin Husemann wrote:
On Fri, Feb 09, 2018 at 11:23:17AM +0100, Maxime Villard wrote:
When I spotted this several months ago (while developing Live
Kernel ASLR), I tried to look for GCC options that say "optimize
with -O2, but keep the stack trace intact". I couldn't find one,
and the only thing I ended up doing was disabling -O2 in the
makefiles.
-fno-omit-frame-pointer?
That won't help.
`-O' also turns on `-fomit-frame-pointer' on machines where doing
so does not interfere with debugging.
so it's not turned off in the first place. The problem is that some
of the later optimization passes may push frame pointer setup to some
place later in function. E.g. on -7
void
kernfs_get_rrootdev(void)
{
static int tried = 0;
if (tried) {
/* Already did it once. */
return;
}
tried = 1;
if (rootdev == NODEV)
return;
rrootdev = devsw_blk2chr(rootdev);
if (rrootdev != NODEV)
return;
rrootdev = NODEV;
printf("kernfs_get_rrootdev: no raw root device\n");
}
is compiled to
c068f81b :
c068f81b: mov0xc0fc6b40,%eax
c068f820: test %eax,%eax
c068f822: jnec068f867
c068f824: movl $0x1,0xc0fc6b40
c068f82e: mov0xc0fde0b8,%edx
c068f834: mov0xc0fde0bc,%eax
c068f839: mov%edx,%ecx
c068f83b: and%eax,%ecx
c068f83d: cmp$0x,%ecx
c068f840: je c068f867
-> c068f842: push %ebp
-> c068f843: mov%esp,%ebp
c068f845: sub$0x8,%esp
c068f848: mov%edx,(%esp)
c068f84b: mov%eax,0x4(%esp)
c068f84f: call c091ce52
Yes, exactly. -fno-omit-frame-pointer doesn't change anything here, GCC
does not omit the frame pointer but moves the instructions a little later
in the function.
So we need to find a say to keep the two instructions at the beginning...
Maxime
Re: gcc: optimizations, and stack traces
On Fri, Feb 09, 2018 at 11:38:47 +0100, Martin Husemann wrote:
> On Fri, Feb 09, 2018 at 11:23:17AM +0100, Maxime Villard wrote:
>
> > When I spotted this several months ago (while developing Live
> > Kernel ASLR), I tried to look for GCC options that say "optimize
> > with -O2, but keep the stack trace intact". I couldn't find one,
> > and the only thing I ended up doing was disabling -O2 in the
> > makefiles.
>
> -fno-omit-frame-pointer?
That won't help.
`-O' also turns on `-fomit-frame-pointer' on machines where doing
so does not interfere with debugging.
so it's not turned off in the first place. The problem is that some
of the later optimization passes may push frame pointer setup to some
place later in function. E.g. on -7
void
kernfs_get_rrootdev(void)
{
static int tried = 0;
if (tried) {
/* Already did it once. */
return;
}
tried = 1;
if (rootdev == NODEV)
return;
rrootdev = devsw_blk2chr(rootdev);
if (rrootdev != NODEV)
return;
rrootdev = NODEV;
printf("kernfs_get_rrootdev: no raw root device\n");
}
is compiled to
c068f81b :
c068f81b: mov0xc0fc6b40,%eax
c068f820: test %eax,%eax
c068f822: jnec068f867
c068f824: movl $0x1,0xc0fc6b40
c068f82e: mov0xc0fde0b8,%edx
c068f834: mov0xc0fde0bc,%eax
c068f839: mov%edx,%ecx
c068f83b: and%eax,%ecx
c068f83d: cmp$0x,%ecx
c068f840: je c068f867
-> c068f842: push %ebp
-> c068f843: mov%esp,%ebp
c068f845: sub$0x8,%esp
c068f848: mov%edx,(%esp)
c068f84b: mov%eax,0x4(%esp)
c068f84f: call c091ce52
So the "tried" check and the first "rootdev" check happen before the
frame pointer is set up.
-uwe
Re: gcc: optimizations, and stack traces
On Fri, Feb 09, 2018 at 11:23:17AM +0100, Maxime Villard wrote: > When I spotted this several months ago (while developing Live Kernel ASLR), I > tried to look for GCC options that say "optimize with -O2, but keep the stack > trace intact". I couldn't find one, and the only thing I ended up doing was > disabling -O2 in the makefiles. -fno-ommit-frame-pointer? Martin
gcc: optimizations, and stack traces
An issue I spotted a few months ago, but PR/52560 just reminded me about it.
Basically, in order to get a backtrace, GDB reads the %rbp register. At the
beginning of each function, GCC inserts the two following instructions:
pushq %rbp
movq%rsp,%rbp
Therefore, at any given point, a program can read %rbp, and unwrap the call
tree by iterating this way (approximately):
uint64_t *rbp = read_rbp();
uint64_t rip;
while (1) {
if (rbp == NULL)
break; /* End of the chain */
rip = *(rbp + 1);
find_function_from_rip(rip); /* whatever */
rbp = (uint64_t *)*ptr;
}
But the thing is, when a program is compiled with -O2, GCC can move the two
aforementioned instructions, and put them somewhere else in a function. That
is to say, the beginning of the function may not contain the two instructions.
Currently the x86 kernels (so, both i386 and amd64) are built with -O2.
It implies that if a bug occurs _before_ these two instructions are executed,
we have a %rbp that points to the _previous_ function, the one we got called
from. And therefore, GDB does not display the current function (where the bug
actually happened), but displays its caller.
There are also several cases where functions in the call tree can disappear
from the backtrace. In the following call tree:
A -> B -> C -> D (and D panics)
if, in B, GCC put the two instructions after the instruction that calls C,
the backtrace will be:
A -> C -> D
This can make a bug completely undebuggable.
When I spotted this several months ago (while developing Live Kernel ASLR), I
tried to look for GCC options that say "optimize with -O2, but keep the stack
trace intact". I couldn't find one, and the only thing I ended up doing was
disabling -O2 in the makefiles.
Now, it doesn't mean that I didn't miss it.
Is someone interested in investigating this? Not being able to get correct
backtraces is a real problem for us.
Maxime
Re: Merging ugen into the usb stack
On Thu, 8 Feb 2018, Martin Husemann wrote: On Thu, Feb 08, 2018 at 05:22:18PM +0100, Wolfgang Solfrank wrote: Hi, with the attached diffs I'm able to attach my debug board like this: ugen0 at uhub5 port 1 configuration 1 interface 0 ugen0: OpenMoko (0x1457) Debug Board for Neo1973 (0x5118), rev 2.00/5.00, addr 3 uftdi0 at uhub5 port 1 configuration 1 interface 1 uftdi0: OpenMoko (0x1457) Debug Board for Neo1973 (0x5118), rev 2.00/5.00, addr 3 ucom0 at uftdi0 portno 2 Very cool! Seems to work for me as well: root file system type: ffs kern.module.path=/stand/i386/8.99.12/modules ugen0 at uhub0 port 1 configuration 1 interface 0 ugen0: FTDI (0x9e88) SheevaPlug JTAGKey FT2232D B (0x9e8f), rev 2.00/5.00, addr 2 uftdi0 at uhub0 port 1 configuration 1 interface 1 uftdi0: FTDI (0x9e88) SheevaPlug JTAGKey FT2232D B (0x9e8f), rev 2.00/5.00, addr 2 ucom1 at uftdi0 portno 2 uhub5 at uhub2 port 2: vendor 0451 (0x451) product 1446 (0x1446), class 9/0, rev 1.10/1.10, addr 3 uhub5: 4 ports with 4 removable, self powered uplcom1 at uhub5 port 1 uplcom1: Prolific Technology Inc. (0x67b) USB-Serial Controller (0x2303), rev 1.10/3.00, addr 4 ucom2 at uplcom1 uftdi1 at uhub5 port 3 configuration 1 interface 0 uftdi1: FTDI (0x403) TTL232R-3V3 (0x6001), rev 2.00/6.00, addr 5 ucom3 at uftdi1 portno 1 uftdi2 at uhub5 port 4 configuration 1 interface 0 uftdi2: FTDI (0x403) TTL232R-3V3 (0x6001), rev 2.00/6.00, addr 6 ... I like the aproach, we could even add a few uftdi specific instances of this ugen variant to GENERIC (or usbdevices.config). Can we get this commited? Anyone objecting? It doesn't solve the generic (no pun intended) problem of raw ugen access to arbitrary other devices, e.g. for use by USB over IP http://usbip.sourceforge.net/ Of course, a better approach may be to add detach kernel driver functionality: http://libusb.sourceforge.net/api-1.0/group__dev.html#ga21bd343325f558987ca57e4e281a6d47 -- Stephen
