Re: Oopses and invalid addresses under Hatari

[Eero Tamminen]

Hi,

On 5/3/19 12:04 PM, Michael Schmitz wrote:

Am 02.05.2019 um 09:34 schrieb Eero Tamminen:

...>> And apparently during last schedule() operation in above call stack,

user_inthandler() gets called at 0x4 offset where there's no valid
opcode (as that address is between instructions):
---------------------------------------------------

d user_inthandler

user_inthandler:
$00002a28 : 42a7       clr.l     -(sp)
$00002a2a : 4878 ffff  pea       $ffffffff.w
$00002a2e : 2f00       move.l    d0,-(sp)
...

m $2a2c

00002A2C: ff ff 2f 00 ...
---------------------------------------------------

I assume the invalid "ffff" "opcode" to cause the "line 1111" trap.

Right. And the fact that this address was used as a jump or rts target means something has smashed your stack. I don't see how this address could have got on the stack in any other way.

Since the stack has been corrupted, you can't probably trust the stack contents at all in regards to figuring out the sequence of function calls that got you there.

Hatari backtrace isn't based on CPU stack, so it's not affected by
stack corruption, like Linux kernel's own backtrace would be.

FYI: Hatari backtraces are part of profiling functionality which tracks
things in very brute force way based just on:
- loaded symbol addresses
- PC register address at every executed instruction,
  and type of that instruction

Hatari keeps its own call stack structure.  At every instruction
it does following checks:

* If previous instruction was rts/rte/etc return instruction,
  and PC is currently at subroutine call return address, item is
  popped from call stack (and costs accrued for that function are
  propagated upward in call hierarchy)

* If PC matches address in symbol table, it's pushed to call stack
  if previous instruction was subroutine call (or exception), and
  previous PC address is marked as return address for that.
  Otherwise that address is just marked as e.g. branch to
  the tracked address (for callgraphs)

Because of this approach, Hatari backtrace is always correct in
the sense that code has gone through the indicated functions using
subroutine call instructions, in the indicated order.

It may have gone through also through a lot of other functions
but those don't show up in the call stack if:
* they aren't in the symbol table, e.g. because they're static
* compiler has inlined the function (without adding symbol
  for/at the inlining address)

That's the reason why the function name offsets in the backtraces
can be rather large, but those addresses will still be correct.


(All this tracking is pretty heavy, but on modern computers that's
not really a problem, and it's needed when investigating harder
problems.)

Even if the stack was intact, at the time of rte or schedule the stack has been restored to what it was before the syscall or interrupt happened. Anything on the stack at that point can't reflect kernel call history.

Not sure how to get information about the kernel call history - you might have to save the stack page upon returning from the syscall, before ret_from_syscall restores saved registers from the stack and you take the trap.

Because of the PS_S status register state, kernel trap handler says
it's a bad trap, see [1].


Questions:

* Any idea what could cause wrong offset to user_inthandler()?

* Does schedule(), or rest of system_call(), clean exception stack?

No, that would be impossible to get portable. It's all handled on the system call return path in entry.S, either through ret_from_syscall or resume_userspace.

Ok, so it can run out of stack?

* What kind of size limits there are on different stack sizes?

* Are there e.g. any symbols pointing to stack base addresses
  and stack size limits which I could use in conditional breakpoints?

The kernel task stack pointer is initialized to 8k past the start of the kernel data segment (see head.S). So your stack limit is 8k - beyond that, you're going to run into the rodata segment.

The user stack is set to just below TASK_SIZE initially (fs/exec.c), but the address sp points to looks like it's later changed to just below TASK_UNMAPPED_BASE. Geert or Andreas may know more details.

Thanks, I'll investigate whether I can get something that gives
backtraces when stacks over/underflow.


        - Eero