Re: Toggling between remote KGDB and local DDB within a debugging session

[Conrad Meyer]

On Tue, Apr 19, 2016 at 5:49 AM, Aijaz Baig  wrote:
> I would like to know if there is indeed a way to toggle between gdb
> and ddb while debugging a remote kernel. I am already at the gdb (or
> rather kgdb) prompt. From here how do I switch to local ddb on the
> debugged machine??

Ctrl-c on the serial console.

> When remote remote KGDB is listening and I force a
> panic using 'sysctl debug.kdb.enter=1', it drops into remote KGDB.
> However, when it is NOT listening on the serial port, the local system
> just freezes

Are you sure ddb just doesn't run on the serial port?

> What I want, is to enter ddb on the local machine. Do some debugging
> using it; drop to remote KGDB for things that are best done using
> KGDB, then switch back to local DDB when I'm done.

Yes.  I regularly do this with ctrl-c (gdb->ddb) / "gdb" (ddb->gdb).

Best,
Conrad
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Toggling between remote KGDB and local DDB within a debugging session

[Aijaz Baig]

On Tue, Apr 19, 2016 at 10:17 PM, Conrad Meyer  wrote:
> On Tue, Apr 19, 2016 at 9:35 AM, Aijaz Baig  wrote:
>> On Tue, Apr 19, 2016 at 9:08 PM, Conrad Meyer  wrote:
>>> On Tue, Apr 19, 2016 at 5:49 AM, Aijaz Baig  wrote:
 I would like to know if there is indeed a way to toggle between gdb
 and ddb while debugging a remote kernel. I am already at the gdb (or
 rather kgdb) prompt. From here how do I switch to local ddb on the
 debugged machine??
>>>
>>> Ctrl-c on the serial console.
>> For me I merely see 'Quit' being spat out when I do a ctrl-c
>
> Ctrl-C on the serial console, not in GDB.  It looks like this:
Yes I tried ctrl-c on the serial console (in-fact it even says so at
the ddb promp) but it doesn't work
I find the control still with the kgdb

Has it got anything to do with the fact that I am on a VM? Has anyone
been successful in doing this with VMs?
>
> # sysctl debug.kdb.enter=1
> debug.kdb.enter:KDB: enter: sysctl debug.kdb.enter
> [ thread pid 21907 tid 102340 ]
> Stopped at  kdb_sysctl_enter+0x87:  movq$0,kdb_why
> db> gdb
> (ctrl-c will return control to ddb)
> Switching to gdb back-end
> Received ^C; trying to switch back to ddb.
> using longjmp, hope it works!
> KDB: reentering
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfea79d0e6140
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfea79d0e61f0
> kdb_reenter() at kdb_reenter+0x33/frame 0xfea79d0e6200
> gdb_tx_end() at gdb_tx_end+0x28a/frame 0xfea79d0e6240
> gdb_trap() at gdb_trap+0x1f9/frame 0xfea79d0e6390
> kdb_trap() at kdb_trap+0x169/frame 0xfea79d0e63f0
> trap() at trap+0x71d/frame 0xfea79d0e6600
> calltrap() at calltrap+0x8/frame 0xfea79d0e6600
> --- trap 0x3, rip = 0x8058f177, rsp = 0xfea79d0e66c0, rbp
> = 0xfea79d0e66f0 ---
> kdb_sysctl_enter() at kdb_sysctl_enter+0x87/frame 0xfea79d0e66f0
> sysctl_root() at sysctl_root+0x24a/frame 0xfea79d0e6740
> userland_sysctl() at userland_sysctl+0x1d2/frame 0xfea79d0e67f0
> sys___sysctl() at sys___sysctl+0x74/frame 0xfea79d0e68a0
> amd64_syscall() at amd64_syscall+0x397/frame 0xfea79d0e6ab0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfea79d0e6ab0
> --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x80095ed4a, rsp
> = 0x7fffc948, rbp = 0x7fffc980 ---
> gdb_trap bailing, hopefully back to ddb!
> Switching to ddb back-end
> [ thread pid 21907 tid 102340 ]
> Stopped at  kdb_sysctl_enter+0x87:  movq$0,kdb_why
> db> c
>  0 -> 0
>
> Best,
> Conrad



-- 

Best Regards,
Aijaz Baig
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Toggling between remote KGDB and local DDB within a debugging session

[Conrad Meyer]

On Tue, Apr 19, 2016 at 9:35 AM, Aijaz Baig  wrote:
> On Tue, Apr 19, 2016 at 9:08 PM, Conrad Meyer  wrote:
>> On Tue, Apr 19, 2016 at 5:49 AM, Aijaz Baig  wrote:
>>> I would like to know if there is indeed a way to toggle between gdb
>>> and ddb while debugging a remote kernel. I am already at the gdb (or
>>> rather kgdb) prompt. From here how do I switch to local ddb on the
>>> debugged machine??
>>
>> Ctrl-c on the serial console.
> For me I merely see 'Quit' being spat out when I do a ctrl-c

Ctrl-C on the serial console, not in GDB.  It looks like this:

# sysctl debug.kdb.enter=1
debug.kdb.enter:KDB: enter: sysctl debug.kdb.enter
[ thread pid 21907 tid 102340 ]
Stopped at  kdb_sysctl_enter+0x87:  movq$0,kdb_why
db> gdb
(ctrl-c will return control to ddb)
Switching to gdb back-end
Received ^C; trying to switch back to ddb.
using longjmp, hope it works!
KDB: reentering
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfea79d0e6140
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfea79d0e61f0
kdb_reenter() at kdb_reenter+0x33/frame 0xfea79d0e6200
gdb_tx_end() at gdb_tx_end+0x28a/frame 0xfea79d0e6240
gdb_trap() at gdb_trap+0x1f9/frame 0xfea79d0e6390
kdb_trap() at kdb_trap+0x169/frame 0xfea79d0e63f0
trap() at trap+0x71d/frame 0xfea79d0e6600
calltrap() at calltrap+0x8/frame 0xfea79d0e6600
--- trap 0x3, rip = 0x8058f177, rsp = 0xfea79d0e66c0, rbp
= 0xfea79d0e66f0 ---
kdb_sysctl_enter() at kdb_sysctl_enter+0x87/frame 0xfea79d0e66f0
sysctl_root() at sysctl_root+0x24a/frame 0xfea79d0e6740
userland_sysctl() at userland_sysctl+0x1d2/frame 0xfea79d0e67f0
sys___sysctl() at sys___sysctl+0x74/frame 0xfea79d0e68a0
amd64_syscall() at amd64_syscall+0x397/frame 0xfea79d0e6ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfea79d0e6ab0
--- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x80095ed4a, rsp
= 0x7fffc948, rbp = 0x7fffc980 ---
gdb_trap bailing, hopefully back to ddb!
Switching to ddb back-end
[ thread pid 21907 tid 102340 ]
Stopped at  kdb_sysctl_enter+0x87:  movq$0,kdb_why
db> c
 0 -> 0

Best,
Conrad
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Toggling between remote KGDB and local DDB within a debugging session

[Aijaz Baig]

On Tue, Apr 19, 2016 at 9:08 PM, Conrad Meyer  wrote:
> On Tue, Apr 19, 2016 at 5:49 AM, Aijaz Baig  wrote:
>> I would like to know if there is indeed a way to toggle between gdb
>> and ddb while debugging a remote kernel. I am already at the gdb (or
>> rather kgdb) prompt. From here how do I switch to local ddb on the
>> debugged machine??
>
> Ctrl-c on the serial console.
For me I merely see 'Quit' being spat out when I do a ctrl-c
>
>> When remote remote KGDB is listening and I force a
>> panic using 'sysctl debug.kdb.enter=1', it drops into remote KGDB.
>> However, when it is NOT listening on the serial port, the local system
>> just freezes
>
> Are you sure ddb just doesn't run on the serial port?
For the very first time, a manual panic does indeed bring up the ddb
prompt. However it is only AFTER I've attached kgdb to it, does this
start happening, namely not going back ever to ddb (with 'Quit' being
displayed instead and with the control still with KGDB)
>
>> What I want, is to enter ddb on the local machine. Do some debugging
>> using it; drop to remote KGDB for things that are best done using
>> KGDB, then switch back to local DDB when I'm done.
>
> Yes.  I regularly do this with ctrl-c (gdb->ddb) / "gdb" (ddb->gdb).
If it'd help, I'm using VMs. So my serial console is actually the VM
console. Now just to be on the safe side I tried putting "comconsole"
into '/boot/loader.conf'. However now, with the aforementioned sysctl,
it doesn't drop to ddb even for the very first time unlike previous
trials where. dropping into DDB was smooth as long as KGDB was not
attached to it ever.
>
> Best,
> Conrad

Keen to hear from you

-- 

Best Regards,
Aijaz Baig
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Toggling between remote KGDB and local DDB within a debugging session

[Julian Elischer]

On 19/04/2016 8:49 PM, Aijaz Baig wrote:

Hello

I think the title says it all!! :)

I would like to know if there is indeed a way to toggle between gdb
and ddb while debugging a remote kernel. I am already at the gdb (or
rather kgdb) prompt. From here how do I switch to local ddb on the
debugged machine??
you don't .. at teh moment I think it' s a one way street, but at one 
stage you could "detach"

and it wuld switch back to ddb.. I don't think that works any more..
I've looked at making it work more than once but never got enough of 
an understanding to make it work,
I suspect that it is a case of setting the appropriate word somewhere 
to teh appropriate value.

How to find that location from gdb is the hard part.




My kernel configuration file already contains 'options
BREAK_TO_DEBUGGER' and I have BOTH GDB and DDB configured aka:
options GDB
options DDB

I tried adding 'options KDB_UNATTENDED' but that does not make any difference.

As per the developer's handbook, "Every time you type gdb, the mode
will be toggled between remote GDB and local DDB. In order to force a
next trap immediately, simply type s (step). Your hosting GDB will now
gain control over the target kernel:" Now when you type 'gdb' at the
DDB prompt, KGDB takes over remotely. On continuing at the KGDB
prompt, you arrive back at the debugged machine but it is not longer
under the control of DDB.

My question is, how do I drop to DDB from within a running machine
whose serial ports (albeit virtual ones) are remotely attached to
another machine? When remote remote KGDB is listening and I force a
panic using 'sysctl debug.kdb.enter=1', it drops into remote KGDB.
However, when it is NOT listening on the serial port, the local system
just freezes

What I want, is to enter ddb on the local machine. Do some debugging
using it; drop to remote KGDB for things that are best done using
KGDB, then switch back to local DDB when I'm done.

Is there a way to do that? If yes please do let me know



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Toggling between remote KGDB and local DDB within a debugging session

[Aijaz Baig]

Hello

I think the title says it all!! :)

I would like to know if there is indeed a way to toggle between gdb
and ddb while debugging a remote kernel. I am already at the gdb (or
rather kgdb) prompt. From here how do I switch to local ddb on the
debugged machine??

My kernel configuration file already contains 'options
BREAK_TO_DEBUGGER' and I have BOTH GDB and DDB configured aka:
options GDB
options DDB

I tried adding 'options KDB_UNATTENDED' but that does not make any difference.

As per the developer's handbook, "Every time you type gdb, the mode
will be toggled between remote GDB and local DDB. In order to force a
next trap immediately, simply type s (step). Your hosting GDB will now
gain control over the target kernel:" Now when you type 'gdb' at the
DDB prompt, KGDB takes over remotely. On continuing at the KGDB
prompt, you arrive back at the debugged machine but it is not longer
under the control of DDB.

My question is, how do I drop to DDB from within a running machine
whose serial ports (albeit virtual ones) are remotely attached to
another machine? When remote remote KGDB is listening and I force a
panic using 'sysctl debug.kdb.enter=1', it drops into remote KGDB.
However, when it is NOT listening on the serial port, the local system
just freezes

What I want, is to enter ddb on the local machine. Do some debugging
using it; drop to remote KGDB for things that are best done using
KGDB, then switch back to local DDB when I'm done.

Is there a way to do that? If yes please do let me know

-- 

Best Regards,
Aijaz Baig
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Eugene Grosbein]

On 19.04.2016 17:39, Melissa Jenkins wrote:
> Will do - it behaves the same even without PXE.
> 
> I believe the issue is that kern.proc.pathname does not exist on any of my 
> systems

Have you tried running "sysctl -d kern.proc.pathname" ?


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Edward Tomasz Napierała]

On 0419T0906, Melissa Jenkins wrote:
> I've been trying to get reboot -r to work but get an error that 
> kern.proc.pathname is undefined.  It then drops to single user mode.
> 
> Interestingly I've checked the value of kern.proc.pathname and it appears to 
> be undefined on all the OS boxes we have from 9.3 up to current.  In fact the 
> kern.proc tree doesn't appear to contain anything though it does exist at 
> least on some of the boxes.

The kern.proc.pathname is a weird sysctl.  It's per-process, and it's
impossible to access it via name, only by numeric ID.  So, this is normal.

The fact that reroot doesn't work because of this is not normal, though.
I have no idea why this would fail; I'll investigate.

Also - could you add a PR with all the information you have?  Thanks!

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Melissa Jenkins]

Will do - it behaves the same even without PXE.

I believe the issue is that kern.proc.pathname does not exist on any of my 
systems

Mel

> On 19 Apr 2016, at 11:19, Eugene Grosbein  wrote:
> 
> CCing Edward Tomasz Napierala, "Root Remount" project contact.
> 
> On 19.04.2016 16:42, Melissa Jenkins wrote:
>> My apologies:
>> 
>> [root@test:~]# sysctl -A kern.proc.pathname
>> [root@test:~]# kenv vfs.root.mountfrom
>> nfs:nfsserver:/bootenv/10.3
>> [root@test:~]# kenv vfs.root.mountfrom=zfs:test/root_role
>> vfs.root.mountfrom="zfs:test/root_role"
>> [root@test:~]# reboot -r
>> [root@test:~]# 
>> 
>> Apr 19 09:35:28 test reroot: rerooted by melissa
>> Apr 19 09:35:28 test init: failed to get kern.proc.pathname: No such file or 
>> directory
>> Apr 19 09:35:28 test init: reroot failed; going to single user mode
>> Apr 19 09:35:28 test root: /etc/rc: WARNING: could not store hostuuid in 
>> /etc/hostid.
>> Apr 19 09:35:31 test root: /etc/rc: WARNING: failed to start dev
>> 
>> It actually doesn't do seem to either drop to single user mode or actually 
>> kill anything off.  If you reroot to the existing directory it seems to 
>> panic though I don't have a crash file from this.
>> 
>> The machine has been booted from a vanilla 10.3 PXE boot.  I've then created 
>> a zfs file system on local disks, installed the operating system in it and 
>> would like to 'restart the system into' this new file system.  Normally I'd 
>> use the init_script and init_chroot kenv flags but rerooting seems cleaner 
>> and may allow chroots to work better
> 
> It seems, reroot currently does not work for PXE-booted systems.
> You should fill a PR.
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Eugene Grosbein]

CCing Edward Tomasz Napierala, "Root Remount" project contact.

On 19.04.2016 16:42, Melissa Jenkins wrote:
> My apologies:
> 
>  [root@test:~]# sysctl -A kern.proc.pathname
>  [root@test:~]# kenv vfs.root.mountfrom
> nfs:nfsserver:/bootenv/10.3
>  [root@test:~]# kenv vfs.root.mountfrom=zfs:test/root_role
> vfs.root.mountfrom="zfs:test/root_role"
>  [root@test:~]# reboot -r
>  [root@test:~]# 
> 
> Apr 19 09:35:28 test reroot: rerooted by melissa
> Apr 19 09:35:28 test init: failed to get kern.proc.pathname: No such file or 
> directory
> Apr 19 09:35:28 test init: reroot failed; going to single user mode
> Apr 19 09:35:28 test root: /etc/rc: WARNING: could not store hostuuid in 
> /etc/hostid.
> Apr 19 09:35:31 test root: /etc/rc: WARNING: failed to start dev
> 
> It actually doesn't do seem to either drop to single user mode or actually 
> kill anything off.  If you reroot to the existing directory it seems to panic 
> though I don't have a crash file from this.
> 
> The machine has been booted from a vanilla 10.3 PXE boot.  I've then created 
> a zfs file system on local disks, installed the operating system in it and 
> would like to 'restart the system into' this new file system.  Normally I'd 
> use the init_script and init_chroot kenv flags but rerooting seems cleaner 
> and may allow chroots to work better

It seems, reroot currently does not work for PXE-booted systems.
You should fill a PR.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Melissa Jenkins]

My apologies:

 [root@test:~]# sysctl -A kern.proc.pathname
 [root@test:~]# kenv vfs.root.mountfrom
nfs:nfsserver:/bootenv/10.3
 [root@test:~]# kenv vfs.root.mountfrom=zfs:test/root_role
vfs.root.mountfrom="zfs:test/root_role"
 [root@test:~]# reboot -r
 [root@test:~]# 

Apr 19 09:35:28 test reroot: rerooted by melissa
Apr 19 09:35:28 test init: failed to get kern.proc.pathname: No such file or 
directory
Apr 19 09:35:28 test init: reroot failed; going to single user mode
Apr 19 09:35:28 test root: /etc/rc: WARNING: could not store hostuuid in 
/etc/hostid.
Apr 19 09:35:31 test root: /etc/rc: WARNING: failed to start dev

It actually doesn't do seem to either drop to single user mode or actually kill 
anything off.  If you reroot to the existing directory it seems to panic though 
I don't have a crash file from this.

The machine has been booted from a vanilla 10.3 PXE boot.  I've then created a 
zfs file system on local disks, installed the operating system in it and would 
like to 'restart the system into' this new file system.  Normally I'd use the 
init_script and init_chroot kenv flags but rerooting seems cleaner and may 
allow chroots to work better

Mel
 

> On 19 Apr 2016, at 10:15, Eugene Grosbein  wrote:
> 
> On 19.04.2016 15:06, Melissa Jenkins wrote:
>> I've been trying to get reboot -r to work but get an error that 
>> kern.proc.pathname is undefined.  It then drops to single user mode.
>> 
>> Interestingly I've checked the value of kern.proc.pathname and it appears to 
>> be undefined on all the OS boxes we have from 9.3 up to current.  In fact 
>> the kern.proc tree doesn't appear to contain anything though it does exist 
>> at least on some of the boxes.
>> 
>> I haven't been able to find much about the kern.proc node except the source 
>> code, and there doesn't appear to be anything in the OPTIONS file that needs 
>> to be defined.  
>> 
>> This leaves me feeling like I'm missing something obvious!?
>> 
>> Can somebody please point me in the right direction?
> 
> You should show exact command you type and exact error message.
> 
> 

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: 10.3 and reboot -r (reroot)

[Eugene Grosbein]

On 19.04.2016 15:06, Melissa Jenkins wrote:
> I've been trying to get reboot -r to work but get an error that 
> kern.proc.pathname is undefined.  It then drops to single user mode.
> 
> Interestingly I've checked the value of kern.proc.pathname and it appears to 
> be undefined on all the OS boxes we have from 9.3 up to current.  In fact the 
> kern.proc tree doesn't appear to contain anything though it does exist at 
> least on some of the boxes.
> 
> I haven't been able to find much about the kern.proc node except the source 
> code, and there doesn't appear to be anything in the OPTIONS file that needs 
> to be defined.  
> 
> This leaves me feeling like I'm missing something obvious!?
> 
> Can somebody please point me in the right direction?

You should show exact command you type and exact error message.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

10.3 and reboot -r (reroot)

[Melissa Jenkins]

I've been trying to get reboot -r to work but get an error that 
kern.proc.pathname is undefined.  It then drops to single user mode.

Interestingly I've checked the value of kern.proc.pathname and it appears to be 
undefined on all the OS boxes we have from 9.3 up to current.  In fact the 
kern.proc tree doesn't appear to contain anything though it does exist at least 
on some of the boxes.

I haven't been able to find much about the kern.proc node except the source 
code, and there doesn't appear to be anything in the OPTIONS file that needs to 
be defined.  

This leaves me feeling like I'm missing something obvious!?

Can somebody please point me in the right direction?

Thanks,
Mel
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"