Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

Reviewing my previous work, the two package you need are:

alsa-lib
alsa-utils

apt-get source alsa-utils
cd alsa-utils-1.0.25
export DEB_BUILD_OPTIONS=“nostrip noopt debug"
dpkg-buildpackage -rfakeroot -uc -us
make
make install

or 
dpkg -i libasound2*.deb


or
apt-get install libasound2-dbg
apt-get install libasound2.dev

Anyway, as I said I did this a long time ago, so some of this could be wrong.

Regards,
John




> On Apr 6, 2016, at 6:49 PM, Robert Nelson  wrote:
> 
> 
> 
> On Wed, Apr 6, 2016 at 8:44 PM, Rick Mann  > wrote:
> That looks like something that would be useful to figure out how to do. The 
> only part that's not immediately clear is "rebuild the entire ALSA library 
> and tools with debug symbols. Also, your Kernel must have debug symbols 
> turned on."
> 
> Can I do this with RCN's kernel build script setup?
> 
> yeah, double check your kernel's version (uname -r) grab the "yakbuild"
> 
> https://github.com/RobertCNelson/yakbuild 
> 
> 
> follow the readme.md , picking a version of gcc and 
> setting the kernel tag you want to build..
> 
> Run:
> 
> ./build_kernel.sh
> 
> when menuconfig pop's up, enable CONFIG_DEBUG_INFO...
> 
> Kernel Hacking -> Compile-Time checks and compiler options -> [x] Compile the 
> kernel with debug info "CONFIG_DEBUG_INFO"
> 
> Regards,
> 
> -- 
> Robert Nelson
> https://rcn-ee.com/ 
> 
> -- 
> For more options, visit http://beagleboard.org/discuss 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

I did this years ago. From what I can find, I believe this is how you get this 
done:

sudo apt-get build-dep libasound2
apt-get source libasound2
cd alsa-lib-1.0.22/
export DEB_BUILD_OPTIONS=nostrip,noopt
dpkg-buildpackage -rfakeroot -uc -us

After that, you will have some *.deb files created, which can be installed 
using:

dpkg -i libasound*.deb

The you have to do the same with alsa-tools.


Regards,
John




> On Apr 6, 2016, at 6:44 PM, Rick Mann  wrote:
> 
> That looks like something that would be useful to figure out how to do. The 
> only part that's not immediately clear is "rebuild the entire ALSA library 
> and tools with debug symbols. Also, your Kernel must have debug symbols 
> turned on."
> 
> Can I do this with RCN's kernel build script setup?
> 
> Thanks.
> 
>> On Apr 6, 2016, at 18:05 , John Syne  wrote:
>> 
>> OK, so you have to rebuild the entire ALSA library and tools with debug 
>> symbols. Also, your Kernel must have debug symbols turned on. You will need 
>> to enable various ftrace features with make config. Here is a typical script 
>> I use to capture program flow.
>> 
>> 
>> ===
>> #!/bin/bash
>> set -x
>> 
>> pause() {
>> local dummy
>> read -s -r -p "Press any key to continue..." -n 1 dummy
>> }
>> 
>> echo function_graph > /sys/kernel/debug/tracing/current_tracer
>> #echo function > /sys/kernel/debug/tracing/current_tracer
>> echo 10 > /sys/kernel/debug/tracing/buffer_size_kb
>> echo function-trace > /sys/kernel/debug/tracing/trace_options
>> echo latency-format > /sys/kernel/debug/tracing/trace_options
>> echo graph-time > /sys/kernel/debug/tracing/trace_options
>> echo funcgraph-tail > /sys/kernel/debug/tracing/trace_options
>> echo funcgraph-proc > /sys/kernel/debug/tracing/trace_options
>> echo funcgraph-abstime > /sys/kernel/debug/tracing/trace_options
>> #echo __do_fault > /sys/kernel/debug/tracing/set_graph_function
>> echo 1 > /sys/kernel/debug/tracing/tracing_on
>> 
>> #Insure modprobe statements here if you want to trace the kernel module at 
>> startup 
>> 
>> aplay command
>> 
>> #Uninstall Kernel module here
>> 
>> echo 0 > /sys/kernel/debug/tracing/tracing_on
>> 
>> #cat /sys/kernel/debug/tracing/trace_pipe |grep  | grep -v 
>> "" | grep -v "" > test-trace.txt
>> #cat /sys/kernel/debug/tracing/trace_pipe | sed -n '//p' | sed 
>> '//d' > test-1-trace.txt
>> cat /sys/kernel/debug/tracing/trace_pipe > test-2-trace.txt
>> ===
>> 
>> Place this script in your BBB /home/debian folder, then as root, execute 
>> ./trace.sh
>> 
>> Finally, I index the entire linux kernel using a variation of these 
>> instructions:
>> 
>> https://wiki.eclipse.org/HowTo_use_the_CDT_to_navigate_Linux_kernel_source
>> 
>> To speed things up, you must filter out all the code from other processor 
>> architectures and drivers you don’t use. 
>> 
>> Now I can ctrl click on any term to trace the flow I get from ftrace. For 
>> something this complex, I tend to diagram the flow with a mind map or 
>> something similar. The visual layout helps me understand the framework 
>> architecture. 
>> 
>> Regards,
>> John
>> 
>> 
>> 
>> 
>>> On Apr 6, 2016, at 5:45 PM, Rick Mann  wrote:
>>> 
>>> 
 On Apr 6, 2016, at 16:53 , Peter Hurley  wrote:
 
 If this is a latency problem, ftrace is probably your best bet, but
 function_graph and the other tracers can induce even more latency.
>>> 
>>> Nothing so esoteric. I'm just trying to get the damn thing to work at all. 
>>> It's worked in previous kernels, but not in the latest. Seeing what code 
>>> gets executed will help a lot. But I'm currently stuck with the default 
>>> diagnostics available (e.g. "Unable to set hw params for playback: Invalid 
>>> argument" NOT HELPFUL).
>>> 
>>> In the past, when I spent so much time with printk, I was trying to 
>>> understand how all the modules worked together to actually configure and 
>>> drive the CODEC. I found that to be virtually impossible to do in any 
>>> comprehensive way, not already understanding it. Can't use printk() if you 
>>> don't know where to put it.
>>> 
>>> 
>>> -- 
>>> Rick Mann
>>> rm...@latencyzero.com
>>> 
>>> 
>>> -- 
>>> For more options, visit http://beagleboard.org/discuss
>>> --- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "BeagleBoard" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to beagleboard+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>> 
>> -- 
>> For more options, visit http://beagleboard.org/discuss
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "BeagleBoard" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to beagleboard+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> 
> -- 
> Rick Mann
> 

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[Robert Nelson]

On Wed, Apr 6, 2016 at 8:44 PM, Rick Mann  wrote:

> That looks like something that would be useful to figure out how to do.
> The only part that's not immediately clear is "rebuild the entire ALSA
> library and tools with debug symbols. Also, your Kernel must have debug
> symbols turned on."
>
> Can I do this with RCN's kernel build script setup?
>

yeah, double check your kernel's version (uname -r) grab the "yakbuild"

https://github.com/RobertCNelson/yakbuild

follow the readme.md, picking a version of gcc and setting the kernel tag
you want to build..

Run:

./build_kernel.sh

when menuconfig pop's up, enable CONFIG_DEBUG_INFO...

Kernel Hacking -> Compile-Time checks and compiler options -> [x] Compile
the kernel with debug info "CONFIG_DEBUG_INFO"

Regards,

-- 
Robert Nelson
https://rcn-ee.com/

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[Rick Mann]

That looks like something that would be useful to figure out how to do. The 
only part that's not immediately clear is "rebuild the entire ALSA library and 
tools with debug symbols. Also, your Kernel must have debug symbols turned on."

Can I do this with RCN's kernel build script setup?

Thanks.

> On Apr 6, 2016, at 18:05 , John Syne  wrote:
> 
> OK, so you have to rebuild the entire ALSA library and tools with debug 
> symbols. Also, your Kernel must have debug symbols turned on. You will need 
> to enable various ftrace features with make config. Here is a typical script 
> I use to capture program flow.
> 
> 
> ===
> #!/bin/bash
> set -x
> 
> pause() {
>  local dummy
>  read -s -r -p "Press any key to continue..." -n 1 dummy
> }
> 
> echo function_graph > /sys/kernel/debug/tracing/current_tracer
> #echo function > /sys/kernel/debug/tracing/current_tracer
> echo 10 > /sys/kernel/debug/tracing/buffer_size_kb
> echo function-trace > /sys/kernel/debug/tracing/trace_options
> echo latency-format > /sys/kernel/debug/tracing/trace_options
> echo graph-time > /sys/kernel/debug/tracing/trace_options
> echo funcgraph-tail > /sys/kernel/debug/tracing/trace_options
> echo funcgraph-proc > /sys/kernel/debug/tracing/trace_options
> echo funcgraph-abstime > /sys/kernel/debug/tracing/trace_options
> #echo __do_fault > /sys/kernel/debug/tracing/set_graph_function
> echo 1 > /sys/kernel/debug/tracing/tracing_on
> 
> #Insure modprobe statements here if you want to trace the kernel module at 
> startup 
> 
> aplay command
> 
> #Uninstall Kernel module here
> 
> echo 0 > /sys/kernel/debug/tracing/tracing_on
> 
> #cat /sys/kernel/debug/tracing/trace_pipe |grep  | grep -v 
> "" | grep -v "" > test-trace.txt
> #cat /sys/kernel/debug/tracing/trace_pipe | sed -n '//p' | sed 
> '//d' > test-1-trace.txt
> cat /sys/kernel/debug/tracing/trace_pipe > test-2-trace.txt
> ===
> 
> Place this script in your BBB /home/debian folder, then as root, execute 
> ./trace.sh
> 
> Finally, I index the entire linux kernel using a variation of these 
> instructions:
> 
> https://wiki.eclipse.org/HowTo_use_the_CDT_to_navigate_Linux_kernel_source
> 
> To speed things up, you must filter out all the code from other processor 
> architectures and drivers you don’t use. 
> 
> Now I can ctrl click on any term to trace the flow I get from ftrace. For 
> something this complex, I tend to diagram the flow with a mind map or 
> something similar. The visual layout helps me understand the framework 
> architecture. 
> 
> Regards,
> John
> 
> 
> 
> 
>> On Apr 6, 2016, at 5:45 PM, Rick Mann  wrote:
>> 
>> 
>>> On Apr 6, 2016, at 16:53 , Peter Hurley  wrote:
>>> 
>>> If this is a latency problem, ftrace is probably your best bet, but
>>> function_graph and the other tracers can induce even more latency.
>> 
>> Nothing so esoteric. I'm just trying to get the damn thing to work at all. 
>> It's worked in previous kernels, but not in the latest. Seeing what code 
>> gets executed will help a lot. But I'm currently stuck with the default 
>> diagnostics available (e.g. "Unable to set hw params for playback: Invalid 
>> argument" NOT HELPFUL).
>> 
>> In the past, when I spent so much time with printk, I was trying to 
>> understand how all the modules worked together to actually configure and 
>> drive the CODEC. I found that to be virtually impossible to do in any 
>> comprehensive way, not already understanding it. Can't use printk() if you 
>> don't know where to put it.
>> 
>> 
>> -- 
>> Rick Mann
>> rm...@latencyzero.com
>> 
>> 
>> -- 
>> For more options, visit http://beagleboard.org/discuss
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "BeagleBoard" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to beagleboard+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Rick Mann
rm...@latencyzero.com


-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

OK, so you have to rebuild the entire ALSA library and tools with debug 
symbols. Also, your Kernel must have debug symbols turned on. You will need to 
enable various ftrace features with make config. Here is a typical script I use 
to capture program flow.


===
#!/bin/bash
set -x

pause() {
  local dummy
  read -s -r -p "Press any key to continue..." -n 1 dummy
}

echo function_graph > /sys/kernel/debug/tracing/current_tracer
#echo function > /sys/kernel/debug/tracing/current_tracer
echo 10 > /sys/kernel/debug/tracing/buffer_size_kb
echo function-trace > /sys/kernel/debug/tracing/trace_options
echo latency-format > /sys/kernel/debug/tracing/trace_options
echo graph-time > /sys/kernel/debug/tracing/trace_options
echo funcgraph-tail > /sys/kernel/debug/tracing/trace_options
echo funcgraph-proc > /sys/kernel/debug/tracing/trace_options
echo funcgraph-abstime > /sys/kernel/debug/tracing/trace_options
#echo __do_fault > /sys/kernel/debug/tracing/set_graph_function
echo 1 > /sys/kernel/debug/tracing/tracing_on

#Insure modprobe statements here if you want to trace the kernel module at 
startup 

aplay command

#Uninstall Kernel module here

echo 0 > /sys/kernel/debug/tracing/tracing_on

#cat /sys/kernel/debug/tracing/trace_pipe |grep  | grep -v 
"" | grep -v "" > test-trace.txt
#cat /sys/kernel/debug/tracing/trace_pipe | sed -n '//p' | sed '//d' > test-1-trace.txt
cat /sys/kernel/debug/tracing/trace_pipe > test-2-trace.txt
===

Place this script in your BBB /home/debian folder, then as root, execute 
./trace.sh

Finally, I index the entire linux kernel using a variation of these 
instructions:

https://wiki.eclipse.org/HowTo_use_the_CDT_to_navigate_Linux_kernel_source

To speed things up, you must filter out all the code from other processor 
architectures and drivers you don’t use. 

Now I can ctrl click on any term to trace the flow I get from ftrace. For 
something this complex, I tend to diagram the flow with a mind map or something 
similar. The visual layout helps me understand the framework architecture. 

Regards,
John




> On Apr 6, 2016, at 5:45 PM, Rick Mann  wrote:
> 
> 
>> On Apr 6, 2016, at 16:53 , Peter Hurley  wrote:
>> 
>> If this is a latency problem, ftrace is probably your best bet, but
>> function_graph and the other tracers can induce even more latency.
> 
> Nothing so esoteric. I'm just trying to get the damn thing to work at all. 
> It's worked in previous kernels, but not in the latest. Seeing what code gets 
> executed will help a lot. But I'm currently stuck with the default 
> diagnostics available (e.g. "Unable to set hw params for playback: Invalid 
> argument" NOT HELPFUL).
> 
> In the past, when I spent so much time with printk, I was trying to 
> understand how all the modules worked together to actually configure and 
> drive the CODEC. I found that to be virtually impossible to do in any 
> comprehensive way, not already understanding it. Can't use printk() if you 
> don't know where to put it.
> 
> 
> -- 
> Rick Mann
> rm...@latencyzero.com
> 
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[Rick Mann]

> On Apr 6, 2016, at 16:53 , Peter Hurley  wrote:
> 
> If this is a latency problem, ftrace is probably your best bet, but
> function_graph and the other tracers can induce even more latency.

Nothing so esoteric. I'm just trying to get the damn thing to work at all. It's 
worked in previous kernels, but not in the latest. Seeing what code gets 
executed will help a lot. But I'm currently stuck with the default diagnostics 
available (e.g. "Unable to set hw params for playback: Invalid argument" NOT 
HELPFUL).

In the past, when I spent so much time with printk, I was trying to understand 
how all the modules worked together to actually configure and drive the CODEC. 
I found that to be virtually impossible to do in any comprehensive way, not 
already understanding it. Can't use printk() if you don't know where to put it.


-- 
Rick Mann
rm...@latencyzero.com


-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

> On Apr 6, 2016, at 5:27 PM, 'Mark Lazarewicz' via BeagleBoard 
>  wrote:
> 
> If it's a company project the Lauterbach is well worth the money 
Definitely. Lauterbach is one of my favorite tools and I cannot imagine doing 
some work without it. It is a difficult tool to learn, but it is very powerful. 
I’m sure I don’t use 90% of the features available. Best part, it is so fast 
for a JTAG debugger. 

Regards,
John
> 
> Sent from Yahoo Mail on Android 
> 
> On Wed, Apr 6, 2016 at 5:56 PM, Rick Mann
>  wrote:
> What would I need to get/do to enable me to single-step through kernel code 
> on BBB/G? JTAG header, some kind of interface, and a bunch of software 
> installed on Ubuntu? Can anyone make specific recommendations? Is it even 
> possible?
> 
> Thanks,
> 
> -- 
> Rick Mann
> rm...@latencyzero.com 
> 
> 
> -- 
> For more options, visit http://beagleboard.org/discuss 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com. 
> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> For more options, visit http://beagleboard.org/discuss 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com 
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

['Mark Lazarewicz' via BeagleBoard]

If it's a company project the Lauterbach is well worth the money 

Sent from Yahoo Mail on Android 
 
  On Wed, Apr 6, 2016 at 5:56 PM, Rick Mann wrote:   
What would I need to get/do to enable me to single-step through kernel code on 
BBB/G? JTAG header, some kind of interface, and a bunch of software installed 
on Ubuntu? Can anyone make specific recommendations? Is it even possible?

Thanks,

-- 
Rick Mann
rm...@latencyzero.com


-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
  

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

> On Apr 6, 2016, at 4:53 PM, Peter Hurley  wrote:
> 
> On 04/06/2016 02:56 PM, Rick Mann wrote:
>> What would I need to get/do to enable me to single-step through
>> kernel code on BBB/G? JTAG header, some kind of interface, and a
>> bunch of software installed on Ubuntu? Can anyone make specific
>> recommendations? Is it even possible?
> 
> On 04/06/2016 03:34 PM, Rick Mann wrote:
>> Thanks, John. I've tried printk and other logging, but it's virtually
>> impossible to trace something like audio, which spans many modules.
> 
> If this is a latency problem, ftrace is probably your best bet, but
> function_graph and the other tracers can induce even more latency.
My main purpose for using function_graph option was to understand program flow, 
so latency wasn’t an issue for me. I haven’t had any latency issues with my 
drivers, so I haven’t used the latency tracers that much. Mostly I use them to 
measure interrupt latency of the kernel. Other than that, my Lauterbach 
debugger gives me ns timing resolution in the trace buffer. 
> 
> I've had to write my own latency tracer on occasion to measure
> specific code paths (most recently with UART DMA timeouts from
> softirq latency). I cribbed the basic functionality from the
> irqsoff tracer; let me know if that's something you'd find useful.
> 
> If you really need to single-step, KGDB can be made to work but
> there are some limitations, the most crucial being that kgdboc
> runs over the console so you can't debug earlier than when the
> built-in 8250 driver probes. That's like 3 secs into boot so lots
> of init has already happened.
I’m wondering if late init might work for the driver you want to debug? I 
haven’t tried this, but maybe this is a workaround for this issue. 
> 
> FWIW, I did push an earlycon OMAP bootconsole for v4.6 but -rc2
> is early days, and there's already been one problem with
> MMC block device numbering. But if you're desperate and don't
> have access to JTAG, earlycon does printk() output from even
> before page table setup up to console initialization.
> 
> Regards,
> Peter Hurley
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[Peter Hurley]

On 04/06/2016 02:56 PM, Rick Mann wrote:
> What would I need to get/do to enable me to single-step through
> kernel code on BBB/G? JTAG header, some kind of interface, and a
> bunch of software installed on Ubuntu? Can anyone make specific
> recommendations? Is it even possible?

On 04/06/2016 03:34 PM, Rick Mann wrote:
> Thanks, John. I've tried printk and other logging, but it's virtually
> impossible to trace something like audio, which spans many modules.

If this is a latency problem, ftrace is probably your best bet, but
function_graph and the other tracers can induce even more latency.

I've had to write my own latency tracer on occasion to measure
specific code paths (most recently with UART DMA timeouts from
softirq latency). I cribbed the basic functionality from the
irqsoff tracer; let me know if that's something you'd find useful.

If you really need to single-step, KGDB can be made to work but
there are some limitations, the most crucial being that kgdboc
runs over the console so you can't debug earlier than when the
built-in 8250 driver probes. That's like 3 secs into boot so lots
of init has already happened.

FWIW, I did push an earlycon OMAP bootconsole for v4.6 but -rc2
is early days, and there's already been one problem with
MMC block device numbering. But if you're desperate and don't
have access to JTAG, earlycon does printk() output from even
before page table setup up to console initialization.

Regards,
Peter Hurley

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

For audio, your best solution is ftrace. If you use function_graph option, then 
you can see the hierarchy of the call sequence. You can also use trace_printk 
to display variables at specific locations. 

BTW, Lauterbach is about $5K investment for the base tool with USB3 connection. 
You might be able to get something less expensive on eBay, but I doubt if you 
can get anything less than $1.5. 

Regards,
John




> On Apr 6, 2016, at 3:34 PM, Rick Mann  wrote:
> 
> Thanks, John. I've tried printk and other logging, but it's virtually 
> impossible to trace something like audio, which spans many modules.
> 
> I'll look into Lauterbach.
> 
>> On Apr 6, 2016, at 15:31 , John Syne  wrote:
>> 
>> You can do this with CCSV6 with a USB200 JTAG adapter, but CCSV6 is no 
>> longer kernel aware. CCSV4 was kernel aware, but won’t work with the current 
>> kernels. You can still debug with single stepping, software and hardware 
>> breakpoints, read/write memory, etc. Trying to debug a kernel module without 
>> a kernel aware debugger is very difficult. Since you don’t know the load 
>> address of the KM, it is difficult to set a breakpoint on INIT or probe. 
>> Best to make the driver a kernel builtin so that you can break on the start 
>> of the driver. 
>> 
>> For the most part, most kernel developers use printk to display status at 
>> strategic locations in their driver. Also, ftrace and dynamic trace tools 
>> are very useful. Read Documentation/trace on how to use these tools. They 
>> are very powerful. I have never use KDBG, but that is another solution that 
>> might work.
>> 
>> I use Lauterbach, which is like the gold standard for Kernel debugging and 
>> it is kernel aware and has a trace buffer. 
>> 
>> Regards,
>> John
>> 
>> 
>> 
>> 
>>> On Apr 6, 2016, at 2:56 PM, Rick Mann  wrote:
>>> 
>>> What would I need to get/do to enable me to single-step through kernel code 
>>> on BBB/G? JTAG header, some kind of interface, and a bunch of software 
>>> installed on Ubuntu? Can anyone make specific recommendations? Is it even 
>>> possible?
>>> 
>>> Thanks,
>>> 
>>> -- 
>>> Rick Mann
>>> rm...@latencyzero.com
>>> 
>>> 
>>> -- 
>>> For more options, visit http://beagleboard.org/discuss
>>> --- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "BeagleBoard" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to beagleboard+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>> 
>> -- 
>> For more options, visit http://beagleboard.org/discuss
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "BeagleBoard" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to beagleboard+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> 
> -- 
> Rick Mann
> rm...@latencyzero.com
> 
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[Rick Mann]

Thanks, John. I've tried printk and other logging, but it's virtually 
impossible to trace something like audio, which spans many modules.

I'll look into Lauterbach.

> On Apr 6, 2016, at 15:31 , John Syne  wrote:
> 
> You can do this with CCSV6 with a USB200 JTAG adapter, but CCSV6 is no longer 
> kernel aware. CCSV4 was kernel aware, but won’t work with the current 
> kernels. You can still debug with single stepping, software and hardware 
> breakpoints, read/write memory, etc. Trying to debug a kernel module without 
> a kernel aware debugger is very difficult. Since you don’t know the load 
> address of the KM, it is difficult to set a breakpoint on INIT or probe. Best 
> to make the driver a kernel builtin so that you can break on the start of the 
> driver. 
> 
> For the most part, most kernel developers use printk to display status at 
> strategic locations in their driver. Also, ftrace and dynamic trace tools are 
> very useful. Read Documentation/trace on how to use these tools. They are 
> very powerful. I have never use KDBG, but that is another solution that might 
> work.
> 
> I use Lauterbach, which is like the gold standard for Kernel debugging and it 
> is kernel aware and has a trace buffer. 
> 
> Regards,
> John
> 
> 
> 
> 
>> On Apr 6, 2016, at 2:56 PM, Rick Mann  wrote:
>> 
>> What would I need to get/do to enable me to single-step through kernel code 
>> on BBB/G? JTAG header, some kind of interface, and a bunch of software 
>> installed on Ubuntu? Can anyone make specific recommendations? Is it even 
>> possible?
>> 
>> Thanks,
>> 
>> -- 
>> Rick Mann
>> rm...@latencyzero.com
>> 
>> 
>> -- 
>> For more options, visit http://beagleboard.org/discuss
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "BeagleBoard" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to beagleboard+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
Rick Mann
rm...@latencyzero.com


-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Source-level single-stepping kernel code on BBB?

[John Syne]

You can do this with CCSV6 with a USB200 JTAG adapter, but CCSV6 is no longer 
kernel aware. CCSV4 was kernel aware, but won’t work with the current kernels. 
You can still debug with single stepping, software and hardware breakpoints, 
read/write memory, etc. Trying to debug a kernel module without a kernel aware 
debugger is very difficult. Since you don’t know the load address of the KM, it 
is difficult to set a breakpoint on INIT or probe. Best to make the driver a 
kernel builtin so that you can break on the start of the driver. 

For the most part, most kernel developers use printk to display status at 
strategic locations in their driver. Also, ftrace and dynamic trace tools are 
very useful. Read Documentation/trace on how to use these tools. They are very 
powerful. I have never use KDBG, but that is another solution that might work.

I use Lauterbach, which is like the gold standard for Kernel debugging and it 
is kernel aware and has a trace buffer. 

Regards,
John




> On Apr 6, 2016, at 2:56 PM, Rick Mann  wrote:
> 
> What would I need to get/do to enable me to single-step through kernel code 
> on BBB/G? JTAG header, some kind of interface, and a bunch of software 
> installed on Ubuntu? Can anyone make specific recommendations? Is it even 
> possible?
> 
> Thanks,
> 
> -- 
> Rick Mann
> rm...@latencyzero.com
> 
> 
> -- 
> For more options, visit http://beagleboard.org/discuss
> --- 
> You received this message because you are subscribed to the Google Groups 
> "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to beagleboard+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.