Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

>I am not sure you can do this without providing your own babeltrace
source component in a separate plugin to what is already available. If
the trace is not converted to a useful CTF format.

Yes. The blog post is merely to demonstrate how to use babeltrace. We will
not be dealing with text trace input to babeltrace. I am reading up on how
to create a customized plugin for RTEMS binary trace output for this.
Transporting ASCII files off the target will be a heavy operation hence
this is avoided.

Thanks



On Thu, Jul 5, 2018 at 4:04 AM, Guillaume Champagne <
guillaume.champa...@polymtl.ca> wrote:

> On Tue, 2018-07-03 at 07:50 +0530, Vidushi Vashishth wrote:
> > > 3. Clearly document your modifications in separate commit messages.
> >
> > > 4. Add a Makefile to build it automatically.
> >
> > I will push commits to a fork of the rtems. Have given a link to the
> > repository in the readme.md of the Tracing repo. Will that be okay?
> >
> > > 6. Document the steps involved in the tracing, e.g. trace data
> > generation on the RTEMS target, transfer of the data to the
> > development host, conversion of trace data in format X to Y using
> > tool Z, etc. Document the interfaces between the different steps and
> > what runs on the RTEMS target and what runs on the development host.
> >
> > Done
> >
> > > To get the trace data from the simulator to the development host,
> > you can just dump the data via printf() and parse it on the host.
> > This is slow, but enough for a test scenario.
> >
> > I have tested the babeltrace conversion by saving the console output
> > to a text file on host manually (https://vidushivashishth.github.io/e
> > ighthpost/). Can I use sockets to transport the traces from target to
> > host instead? Will that be feasible? I have already created a client
> > and server side program and tested a text file transfer. This is
> > working.
> >
>
> I tried the example in your blog post which works without `sudo` with
> babeltrace 2.0.0-pre4 on fedora. However, I am not sure the ctf output
> is correct. Each line in the original trace is dumped in a ctf event
> named "string" since the example uses the source `text.dmesg`. This
> source is usually used to parse the output of the dmesg linux tool [1].
>
> string: { }, { str = "0:00:26.7031882952081911  0a010002 [  2/  2]
> > malloc((size_t) 0130)" }
> string: { }, { str = "0:00:26.703324403 136108  0a010002 [  2/  2]
> < malloc => (void*) 0x219be08" }
> [..]
>
> This should be changed to actually split the data into correct CTF
> event field and header, like lttng-ust does:
>
> [18:14:22.611410669] (+0.01652) station12.dorsal.polymtl.ca
> lttng_ust_cyg_profile:func_entry: { cpu_id = 5 }, { vpid = 21117, vtid =
> 21117, procname = "demo" }, { addr = 0x7FF848DB6E21, call_site =
> 0x7FF848DB7856 }
> [..]
>
> I am not sure you can do this without providing your own babeltrace
> source component in a separate plugin to what is already available. If
> the trace is not converted to a useful CTF format, Trace Compass won't
> provide any useful analyses since it won't know what the event or the
> payload means.
>
> [1] http://man7.org/linux/man-pages/man7/babeltrace-source.text.dmesg.7
> .html
>
> Guillaume
>
> > > Either we should use barectf to generate CTF traces in RTEMS, we
> > > should implement our own CTF generator in RTEMS, or we should
> > provide
> > > a converter for running babeltrace on a host (Linux/MacOS/etc) to
> > > convert from RTEMS trace linker format to CTF.
> >
> > I am implementing the last option.
> >
> > On Tue, Jul 3, 2018 at 1:37 AM, Gedare Bloom 
> > wrote:
> > > On Mon, Jul 2, 2018 at 1:27 AM, Sebastian Huber
> > >  wrote:
> > > > Hello Vidushi,
> > > >
> > > > On 29/06/18 09:44, Vidushi Vashishth wrote:
> > > >>
> > > >>
> > > >> >Could you please create a self-contained repository
> > > which
> > > >> contains
> > > >>
> > > >> >* a README
> > > >>
> > > >> >* a simple RTEMS application which runs on a simulator
> > > BSP
> > > >>
> > > >> >* the stuff that makes it possible to view the trace
> > > output
> > > >> (it is not a problem if it doesn't work, but all pieces
> > > should
> > > >> be included)
> > > >>
> > > >> >The repository should not be a clone of some larger
> > > project.
> > > >> It may contain external references as submodules.
> > > >>
> > > >> Okay. Got it. I will update you when its done.
> > > >>
> > > >>
> > > >> Ok, do you have a time estimate for this? Which parts are
> > > missing?
> > > >>
> > > >>
> > > >> Viewing the trace output is buggy right now. I will have to
> > > combine the
> > > >> rest. I will push the required things in the repository by end
> > > of today and
> > > >> notify you.
> > > >>
> > > >
> > > > the stuff you published here
> > > >
> > > > https://github.com/VidushiVashishth/Tracing
> > > >
> > > > yesterday (this is not "by end of today") is not much considering
> > > this the
> >

Re: [GSOC - Tracing] Status Update

[Guillaume Champagne]

On Tue, 2018-07-03 at 07:50 +0530, Vidushi Vashishth wrote:
> > 3. Clearly document your modifications in separate commit messages.
> 
> > 4. Add a Makefile to build it automatically.
> 
> I will push commits to a fork of the rtems. Have given a link to the
> repository in the readme.md of the Tracing repo. Will that be okay?
> 
> > 6. Document the steps involved in the tracing, e.g. trace data
> generation on the RTEMS target, transfer of the data to the
> development host, conversion of trace data in format X to Y using
> tool Z, etc. Document the interfaces between the different steps and
> what runs on the RTEMS target and what runs on the development host.
> 
> Done
> 
> > To get the trace data from the simulator to the development host,
> you can just dump the data via printf() and parse it on the host.
> This is slow, but enough for a test scenario.
> 
> I have tested the babeltrace conversion by saving the console output
> to a text file on host manually (https://vidushivashishth.github.io/e
> ighthpost/). Can I use sockets to transport the traces from target to
> host instead? Will that be feasible? I have already created a client
> and server side program and tested a text file transfer. This is
> working.
> 

I tried the example in your blog post which works without `sudo` with
babeltrace 2.0.0-pre4 on fedora. However, I am not sure the ctf output
is correct. Each line in the original trace is dumped in a ctf event
named "string" since the example uses the source `text.dmesg`. This
source is usually used to parse the output of the dmesg linux tool [1].

string: { }, { str = "0:00:26.7031882952081911  0a010002 [  2/  2]
> malloc((size_t) 0130)" }
string: { }, { str = "0:00:26.703324403 136108  0a010002 [  2/  2]
< malloc => (void*) 0x219be08" }
[..]

This should be changed to actually split the data into correct CTF
event field and header, like lttng-ust does:

[18:14:22.611410669] (+0.01652) station12.dorsal.polymtl.ca 
lttng_ust_cyg_profile:func_entry: { cpu_id = 5 }, { vpid = 21117, vtid = 21117, 
procname = "demo" }, { addr = 0x7FF848DB6E21, call_site = 0x7FF848DB7856 }
[..]

I am not sure you can do this without providing your own babeltrace
source component in a separate plugin to what is already available. If
the trace is not converted to a useful CTF format, Trace Compass won't
provide any useful analyses since it won't know what the event or the
payload means.

[1] http://man7.org/linux/man-pages/man7/babeltrace-source.text.dmesg.7
.html

Guillaume

> > Either we should use barectf to generate CTF traces in RTEMS, we
> > should implement our own CTF generator in RTEMS, or we should
> provide
> > a converter for running babeltrace on a host (Linux/MacOS/etc) to
> > convert from RTEMS trace linker format to CTF.
> 
> I am implementing the last option. 
> 
> On Tue, Jul 3, 2018 at 1:37 AM, Gedare Bloom 
> wrote:
> > On Mon, Jul 2, 2018 at 1:27 AM, Sebastian Huber
> >  wrote:
> > > Hello Vidushi,
> > >
> > > On 29/06/18 09:44, Vidushi Vashishth wrote:
> > >>
> > >>
> > >> >Could you please create a self-contained repository
> > which
> > >> contains
> > >>
> > >> >* a README
> > >>
> > >> >* a simple RTEMS application which runs on a simulator
> > BSP
> > >>
> > >> >* the stuff that makes it possible to view the trace
> > output
> > >> (it is not a problem if it doesn't work, but all pieces
> > should
> > >> be included)
> > >>
> > >> >The repository should not be a clone of some larger
> > project.
> > >> It may contain external references as submodules.
> > >>
> > >> Okay. Got it. I will update you when its done.
> > >>
> > >>
> > >> Ok, do you have a time estimate for this? Which parts are
> > missing?
> > >>
> > >>
> > >> Viewing the trace output is buggy right now. I will have to
> > combine the
> > >> rest. I will push the required things in the repository by end
> > of today and
> > >> notify you.
> > >>
> > >
> > > the stuff you published here
> > >
> > > https://github.com/VidushiVashishth/Tracing
> > >
> > > yesterday (this is not "by end of today") is not much considering
> > this the
> > > 8th week of the GSoC project.
> > >
> > > You seem to have imported cpukit/libmisc/shell/main_rtrace.c and
> > modified
> > > it. The modifications are not visible in the repository history.
> > Why did yo
> > > copy this file out of the RTEMS sources at all?
> > >
> > > There is no Makefile (or similar). You have to read the
> > documentation to
> > > build it. You cannot copy and past the instructions to build it
> > since it
> > > contains your local paths.
> > >
> > > The test data is generated with user input.
> > >
> > > How can I transfer the test data generated on the simulator to my
> > host
> > > development system?
> > >
> > > I asked for a self-contained repository, with"the stuff that
> > makes it
> > > possible to view the trace output". This is missing.
> > >
> > > Could you please:
> > >

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

On 03/07/18 14:50, Vidushi Vashishth wrote:

Hi!

> Ok, thanks, but why did you remove the RTEMS application? This is 
the first step. The repository should contain a sample application 
with a Makefile (or waf or whatever, but no manual steps) to generate 
a sample trace raw data.


I was testing everything on the fileio sample test case which is 
already a part of the rtems repository. I have written the 
instructions to generate the trace in the readme file for this 
testcase. I could create a new application and add its makefile too if 
this seems too lengthy.


Yes, please add an example application which generates some trace data 
and outputs this to the console. Please add also a simple Makefile to 
build it.




> Could you please integrate the content of this blog post in the 
repository and automate the manual steps in it.


Yes I am working on it.

> You can communicate via sockets on some simulator BSPs, however, in 
the scope and state of the current project, I think this is not feasible.


Alright I will stick to what you proposed. I however also have to work 
on transportation mechanisms to transfer trace buffers from the 
target/capture engine to the host. I can begin working on the socket 
approach after I am done with CTF?


We should get first the prototype ready. The prototype should look like 
this from my point of view: sample application -> trace output > convert 
-> display in Trace Compass. Once we have this prototype we can decide 
what can be made better within the scope of the project.


You don't have to output plain text data to the console, you may also 
output XML, some binary format, an uuencoded binary or whatever. If the 
current output makes it difficult for you, then you can change its 
format. This should be discussed on the mailing list. This may end up in 
some RTEMS patches.




> Before you implement anything, could you please first outline what 
you want to do.


The general flow of this approach will be as follows:

 1. Generation of RTEMS trace.
 2. Transportation of the generated trace to the host machine.
 3. Conversion of transported traces to CTF using babeltrace scripts.
 4. Viewing the converted output (CTF).

I had documented this in the readme of the Tracing repository also. 
Does this seem okay?


The steps are all right. Please add also some content to the steps, e.g.

* alternatives
* pros and cons
* what works and what doesn't
* lessons learnt
* tools used in the step
* limitations
* etc.

--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

On 03/07/18 04:20, Vidushi Vashishth wrote:

> 3. Clearly document your modifications in separate commit messages.

> 4. Add a Makefile to build it automatically.

I will push commits to a fork of the rtems. Have given a link to the 
repository in the readme.md of the Tracing repo. Will that be okay?


> 6. Document the steps involved in the tracing, e.g. trace data 
generation on the RTEMS target, transfer of the data to the 
development host, conversion of trace data in format X to Y using tool 
Z, etc. Document the interfaces between the different steps and what 
runs on the RTEMS target and what runs on the development host.


Done


Ok, thanks, but why did you remove the RTEMS application? This is the 
first step. The repository should contain a sample application with a 
Makefile (or waf or whatever, but no manual steps) to generate a sample 
trace raw data.




> To get the trace data from the simulator to the development host, you can just dump 
the data via printf() and parse it on the host. This is slow, but 
enough for a test scenario.


I have tested the babeltrace conversion by saving the console output 
to a text file on host manually 
(https://vidushivashishth.github.io/eighthpost/).


Could you please integrate the content of this blog post in the 
repository and automate the manual steps in it.


Can I use sockets to transport the traces from target to host instead? 
Will that be feasible? I have already created a client and server side 
program and tested a text file transfer. This is working.


You can communicate via sockets on some simulator BSPs, however, in the 
scope and state of the current project, I think this is not feasible.




> Either we should use barectf to generate CTF traces in RTEMS, we
> should implement our own CTF generator in RTEMS, or we should provide
> a converter for running babeltrace on a host (Linux/MacOS/etc) to
> convert from RTEMS trace linker format to CTF.

I am implementing the last option.


Before you implement anything, could you please first outline what you 
want to do.


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

> 3. Clearly document your modifications in separate commit messages.

> 4. Add a Makefile to build it automatically.

I will push commits to a fork of the rtems. Have given a link to the
repository in the readme.md of the Tracing repo. Will that be okay?

> 6. Document the steps involved in the tracing, e.g. trace data generation
on the RTEMS target, transfer of the data to the development host,
conversion of trace data in format X to Y using tool Z, etc. Document the
interfaces between the different steps and what runs on the RTEMS target
and what runs on the development host.

Done

> To get the trace data from the simulator to the development host, you can
just dump the data via printf() and parse it on the host. This is slow, but
enough for a test scenario.

I have tested the babeltrace conversion by saving the console output to a
text file on host manually (https://vidushivashishth.github.io/eighthpost/).
Can I use sockets to transport the traces from target to host instead? Will
that be feasible? I have already created a client and server side program
and tested a text file transfer. This is working.

> Either we should use barectf to generate CTF traces in RTEMS, we
> should implement our own CTF generator in RTEMS, or we should provide
> a converter for running babeltrace on a host (Linux/MacOS/etc) to
> convert from RTEMS trace linker format to CTF.

I am implementing the last option.

On Tue, Jul 3, 2018 at 1:37 AM, Gedare Bloom  wrote:

> On Mon, Jul 2, 2018 at 1:27 AM, Sebastian Huber
>  wrote:
> > Hello Vidushi,
> >
> > On 29/06/18 09:44, Vidushi Vashishth wrote:
> >>
> >>
> >> >Could you please create a self-contained repository which
> >> contains
> >>
> >> >* a README
> >>
> >> >* a simple RTEMS application which runs on a simulator BSP
> >>
> >> >* the stuff that makes it possible to view the trace output
> >> (it is not a problem if it doesn't work, but all pieces should
> >> be included)
> >>
> >> >The repository should not be a clone of some larger project.
> >> It may contain external references as submodules.
> >>
> >> Okay. Got it. I will update you when its done.
> >>
> >>
> >> Ok, do you have a time estimate for this? Which parts are missing?
> >>
> >>
> >> Viewing the trace output is buggy right now. I will have to combine the
> >> rest. I will push the required things in the repository by end of today
> and
> >> notify you.
> >>
> >
> > the stuff you published here
> >
> > https://github.com/VidushiVashishth/Tracing
> >
> > yesterday (this is not "by end of today") is not much considering this
> the
> > 8th week of the GSoC project.
> >
> > You seem to have imported cpukit/libmisc/shell/main_rtrace.c and
> modified
> > it. The modifications are not visible in the repository history. Why did
> yo
> > copy this file out of the RTEMS sources at all?
> >
> > There is no Makefile (or similar). You have to read the documentation to
> > build it. You cannot copy and past the instructions to build it since it
> > contains your local paths.
> >
> > The test data is generated with user input.
> >
> > How can I transfer the test data generated on the simulator to my host
> > development system?
> >
> > I asked for a self-contained repository, with"the stuff that makes it
> > possible to view the trace output". This is missing.
> >
> > Could you please:
> >
> > 1. Not copy sources out of the RTEMS repository. Before you do this, ask
> on
> > the mailing list and explain why have have to do it.
> >
> > 2. If you import things, then import the original files and state the
> > origin.
> >
> > 3. Clearly document your modifications in separate commit messages.
> >
> > 4. Add a Makefile to build it automatically.
> >
> > 5. Add the missing stuff that makes it possible to view the trace
> output. If
> > something is not available yet, then no problem. Just document the
> > interfaces and what it is supposed to do.
> >
> > 6. Document the steps involved in the tracing, e.g. trace data
> generation on
> > the RTEMS target, transfer of the data to the development host,
> conversion
> > of trace data in format X to Y using tool Z, etc. Document the interfaces
> > between the different steps and what runs on the RTEMS target and what
> runs
> > on the development host.
> >
> > To get the trace data from the simulator to the development host, you can
> > just dump the data via printf() and parse it on the host. This is slow,
> but
> > enough for a test scenario.
> >
> > You should be able to do this in a couple of hours.
> >
>
> I'm trying to catch up. I generally agree with all this. Also, I do
> not see any reason ever to try to run babeltrace from RTEMS. I don't
> know if that is being attempted, but it is not a worthwhile effort.
> Either we should use barectf to generate CTF traces in RTEMS, we
> should implement our own CTF generator in RTEMS, or we should provide
> a converter for running babeltrace on a host (Linux/MacO

Re: [GSOC - Tracing] Status Update

[Gedare Bloom]

On Mon, Jul 2, 2018 at 1:27 AM, Sebastian Huber
 wrote:
> Hello Vidushi,
>
> On 29/06/18 09:44, Vidushi Vashishth wrote:
>>
>>
>> >Could you please create a self-contained repository which
>> contains
>>
>> >* a README
>>
>> >* a simple RTEMS application which runs on a simulator BSP
>>
>> >* the stuff that makes it possible to view the trace output
>> (it is not a problem if it doesn't work, but all pieces should
>> be included)
>>
>> >The repository should not be a clone of some larger project.
>> It may contain external references as submodules.
>>
>> Okay. Got it. I will update you when its done.
>>
>>
>> Ok, do you have a time estimate for this? Which parts are missing?
>>
>>
>> Viewing the trace output is buggy right now. I will have to combine the
>> rest. I will push the required things in the repository by end of today and
>> notify you.
>>
>
> the stuff you published here
>
> https://github.com/VidushiVashishth/Tracing
>
> yesterday (this is not "by end of today") is not much considering this the
> 8th week of the GSoC project.
>
> You seem to have imported cpukit/libmisc/shell/main_rtrace.c and modified
> it. The modifications are not visible in the repository history. Why did yo
> copy this file out of the RTEMS sources at all?
>
> There is no Makefile (or similar). You have to read the documentation to
> build it. You cannot copy and past the instructions to build it since it
> contains your local paths.
>
> The test data is generated with user input.
>
> How can I transfer the test data generated on the simulator to my host
> development system?
>
> I asked for a self-contained repository, with"the stuff that makes it
> possible to view the trace output". This is missing.
>
> Could you please:
>
> 1. Not copy sources out of the RTEMS repository. Before you do this, ask on
> the mailing list and explain why have have to do it.
>
> 2. If you import things, then import the original files and state the
> origin.
>
> 3. Clearly document your modifications in separate commit messages.
>
> 4. Add a Makefile to build it automatically.
>
> 5. Add the missing stuff that makes it possible to view the trace output. If
> something is not available yet, then no problem. Just document the
> interfaces and what it is supposed to do.
>
> 6. Document the steps involved in the tracing, e.g. trace data generation on
> the RTEMS target, transfer of the data to the development host, conversion
> of trace data in format X to Y using tool Z, etc. Document the interfaces
> between the different steps and what runs on the RTEMS target and what runs
> on the development host.
>
> To get the trace data from the simulator to the development host, you can
> just dump the data via printf() and parse it on the host. This is slow, but
> enough for a test scenario.
>
> You should be able to do this in a couple of hours.
>

I'm trying to catch up. I generally agree with all this. Also, I do
not see any reason ever to try to run babeltrace from RTEMS. I don't
know if that is being attempted, but it is not a worthwhile effort.
Either we should use barectf to generate CTF traces in RTEMS, we
should implement our own CTF generator in RTEMS, or we should provide
a converter for running babeltrace on a host (Linux/MacOS/etc) to
convert from RTEMS trace linker format to CTF.

Gedare

>
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

Hello Vidushi,

On 29/06/18 09:44, Vidushi Vashishth wrote:


>Could you please create a self-contained repository which contains

>* a README

>* a simple RTEMS application which runs on a simulator BSP

>* the stuff that makes it possible to view the trace output
(it is not a problem if it doesn't work, but all pieces should
be included)

>The repository should not be a clone of some larger project.
It may contain external references as submodules.

Okay. Got it. I will update you when its done.


Ok, do you have a time estimate for this? Which parts are missing?


Viewing the trace output is buggy right now. I will have to combine 
the rest. I will push the required things in the repository by end of 
today and notify you.




the stuff you published here

https://github.com/VidushiVashishth/Tracing

yesterday (this is not "by end of today") is not much considering this 
the 8th week of the GSoC project.


You seem to have imported cpukit/libmisc/shell/main_rtrace.c and 
modified it. The modifications are not visible in the repository 
history. Why did yo copy this file out of the RTEMS sources at all?


There is no Makefile (or similar). You have to read the documentation to 
build it. You cannot copy and past the instructions to build it since it 
contains your local paths.


The test data is generated with user input.

How can I transfer the test data generated on the simulator to my host 
development system?


I asked for a self-contained repository, with"the stuff that makes it 
possible to view the trace output". This is missing.


Could you please:

1. Not copy sources out of the RTEMS repository. Before you do this, ask 
on the mailing list and explain why have have to do it.


2. If you import things, then import the original files and state the 
origin.


3. Clearly document your modifications in separate commit messages.

4. Add a Makefile to build it automatically.

5. Add the missing stuff that makes it possible to view the trace 
output. If something is not available yet, then no problem. Just 
document the interfaces and what it is supposed to do.


6. Document the steps involved in the tracing, e.g. trace data 
generation on the RTEMS target, transfer of the data to the development 
host, conversion of trace data in format X to Y using tool Z, etc. 
Document the interfaces between the different steps and what runs on the 
RTEMS target and what runs on the development host.


To get the trace data from the simulator to the development host, you 
can just dump the data via printf() and parse it on the host. This is 
slow, but enough for a test scenario.


You should be able to do this in a couple of hours.

--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

Hi!

https://github.com/VidushiVashishth/Tracing

This is the required repository. I read up the POSIX API documentation to
understand that nothing of the like of system( ) would work on rtems since
its a single process os.
The main_rtrace.c in the given repository currently consists of this wrong
approach. There are two options that I can do.

1) See if babelltrace C API would work and replace the system call logic
with this.

OR

2) Continue with the barectf approach. PFA the metadata file that I came up
with for this purpose.
A typical trace will consist a single stream. Events inside the stream will
have three fields for name of the function, two input argument values or
one return value (according to whether the event describes the entry or
exit of the function being traced)

Things to be done in this method:
1) Convert the metada file to a yaml configuration file. (almost done)
2) Inspect the header and c file generated by barectf for this configuration
3) Create a trace linker generator which binds these barectf function calls
to tracing.

Checkout the barectf documentation to see how it works:

I will send the yaml configuration too when its done. What do you suggest I
should do?

References:
https://github.com/efficios/barectf/wiki/Using-barectf

Thanks

On Sun, Jul 1, 2018 at 7:12 AM, Chris Johns  wrote:

>
> On 29 Jun 2018, at 5:44 pm, Vidushi Vashishth  wrote:
>
> >Could you please create a self-contained repository which contains
>>>
>>> >* a README
>>>
>>> >* a simple RTEMS application which runs on a simulator BSP
>>>
>>> >* the stuff that makes it possible to view the trace output (it is not
>>> a problem if it doesn't work, but all pieces should be included)
>>>
>>> >The repository should not be a clone of some larger project. It may
>>> contain external references as submodules.
>>>
>>> Okay. Got it. I will update you when its done.
>>>
>>
>> Ok, do you have a time estimate for this? Which parts are missing?
>
>
> Viewing the trace output is buggy right now. I will have to combine the
> rest. I will push the required things in the repository by end of today and
> notify you.
>
>
> Did this happen?
>
> Chris
>


metadata
Description: Binary data
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Chris Johns]

On 29 Jun 2018, at 5:44 pm, Vidushi Vashishth  wrote:

>>> >Could you please create a self-contained repository which contains
>>> 
>>> >* a README
>>> 
>>> >* a simple RTEMS application which runs on a simulator BSP
>>> 
>>> >* the stuff that makes it possible to view the trace output (it is not a 
>>> >problem if it doesn't work, but all pieces should be included)
>>> 
>>> >The repository should not be a clone of some larger project. It may 
>>> >contain external references as submodules.
>>> 
>>> Okay. Got it. I will update you when its done.
>> 
>> Ok, do you have a time estimate for this? Which parts are missing?
> 
> Viewing the trace output is buggy right now. I will have to combine the rest. 
> I will push the required things in the repository by end of today and notify 
> you.  

Did this happen?

Chris___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

>
> >Could you please create a self-contained repository which contains
>>
>> >* a README
>>
>> >* a simple RTEMS application which runs on a simulator BSP
>>
>> >* the stuff that makes it possible to view the trace output (it is not a
>> problem if it doesn't work, but all pieces should be included)
>>
>> >The repository should not be a clone of some larger project. It may
>> contain external references as submodules.
>>
>> Okay. Got it. I will update you when its done.
>>
>
> Ok, do you have a time estimate for this? Which parts are missing?


Viewing the trace output is buggy right now. I will have to combine the
rest. I will push the required things in the repository by end of today and
notify you.

> Is visualization with Trace Compass a goal of this project?
>>
>> No it isn't. Trace generation in CTF is a goal.
>>
>
> What are the consumers of this CTF? Why do we need it? What is the benefit
> for the RTEMS users?


CTF allows traces to be natively generated by any C/C++ application or
system, as well as by bare-metal (hardware) components. CTF is a flexible
tracing format with a range of configurable datatypes. This allows CTF to
describe any binary structure. Also since CTF is a binary trace format it
is faster to write. The post processing (eg Trace Compass) and real time
monitoring tools available with it are valuable.


On Fri, Jun 29, 2018 at 10:35 AM, Sebastian Huber <
sebastian.hu...@embedded-brains.de> wrote:

> On 28/06/18 19:41, Vidushi Vashishth wrote:
>
>> >Could you please create a self-contained repository which contains
>>
>> >* a README
>>
>> >* a simple RTEMS application which runs on a simulator BSP
>>
>> >* the stuff that makes it possible to view the trace output (it is not a
>> problem if it doesn't work, but all pieces should be included)
>>
>> >The repository should not be a clone of some larger project. It may
>> contain external references as submodules.
>>
>> Okay. Got it. I will update you when its done.
>>
>
> Ok, do you have a time estimate for this? Which parts are missing?
>
>
>> >Did you modify babeltrace?
>>
>> No I did not.
>>
>
> Ok, this is good.
>
>
>> >Why do you need a very recent babeltrace for your work?
>>
>> I haven't tested the code with other versions of babeltrace. I cant say
>> why it does not work with others. I ll try to test the code with other
>> versions and identify what's wrong.
>>
>
> If it doesn't work with the babeltrace shipped with standard
> distributions, then this a slight burden to get started for users. It is
> not a high priority issue at the moment.
>
>
>> > Is visualization with Trace Compass a goal of this project?
>>
>> No it isn't. Trace generation in CTF is a goal.
>>
>
> What are the consumers of this CTF? Why do we need it? What is the benefit
> for the RTEMS users?
>
>
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

On 28/06/18 19:41, Vidushi Vashishth wrote:

>Could you please create a self-contained repository which contains

>* a README

>* a simple RTEMS application which runs on a simulator BSP

>* the stuff that makes it possible to view the trace output (it is 
not a problem if it doesn't work, but all pieces should be included)


>The repository should not be a clone of some larger project. It may 
contain external references as submodules.


Okay. Got it. I will update you when its done.


Ok, do you have a time estimate for this? Which parts are missing?



>Did you modify babeltrace?

No I did not.


Ok, this is good.



>Why do you need a very recent babeltrace for your work?

I haven't tested the code with other versions of babeltrace. I cant 
say why it does not work with others. I ll try to test the code with 
other versions and identify what's wrong.


If it doesn't work with the babeltrace shipped with standard 
distributions, then this a slight burden to get started for users. It is 
not a high priority issue at the moment.




> Is visualization with Trace Compass a goal of this project?

No it isn't. Trace generation in CTF is a goal.


What are the consumers of this CTF? Why do we need it? What is the 
benefit for the RTEMS users?


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

>Could you please create a self-contained repository which contains

>* a README

>* a simple RTEMS application which runs on a simulator BSP

>* the stuff that makes it possible to view the trace output (it is not a
problem if it doesn't work, but all pieces should be included)

>The repository should not be a clone of some larger project. It may
contain external references as submodules.

Okay. Got it. I will update you when its done.

>Did you modify babeltrace?

No I did not.

>Why do you need a very recent babeltrace for your work?

I haven't tested the code with other versions of babeltrace. I cant say why
it does not work with others. I ll try to test the code with other versions
and identify what's wrong.

> Is visualization with Trace Compass a goal of this project?

No it isn't. Trace generation in CTF is a goal.

On Thu, Jun 28, 2018 at 11:43 AM, Sebastian Huber <
sebastian.hu...@embedded-brains.de> wrote:

> On 28/06/18 07:42, Vidushi Vashishth wrote:
>
>> >which use case does this address? What is the connection to an RTEMS
>> application?
>> This converts the trace buffering of the fileio sample testcase output to
>> CTF. This pertains to the function tracing testcase. The CTF stream
>> generated records the entry and exit of the calloc, malloc, realloc and
>> free functions and also the values of the arguments and return values.
>>
>
> Could you please create a self-contained repository which contains
>
> * a README
>
> * a simple RTEMS application which runs on a simulator BSP
>
> * the stuff that makes it possible to view the trace output (it is not a
> problem if it doesn't work, but all pieces should be included)
>
> The repository should not be a clone of some larger project. It may
> contain external references as submodules.
>
>
>> >Why do I need sudo to run the commands?
>> The command needs permission to create a directory for the traces. The
>> command did not work without this.
>>
>
> It worked without sudo in my clone. Using sudo for this kind of stuff
> would be a very severe usability issue from my point of view.
>
>
>> >Why do I need your babeltrace? Not being able to use a tool from a
>> standard distribution makes it harder to do RTEMS tracing.
>> I wanted to work on integrating barectf with RTEMS trace linker instead.
>> I documented the approach in: https://vidushivashishth.githu
>> b.io/sixthpost/.
>>
>
> Could you please also post your stuff on the devel@rtems.org mailing
> list. I don't have time to look for blog posts actively.
>
> Did you modify babeltrace?
>
> Why do you need a very recent babeltrace for your work?
>
> I was stuck on deciding the yaml configuration input for the process and
>> sent out an email for this discussion on the mailing list as well. I have
>> however finally created one. I can continue work on the barectf approach if
>> that is okay with you? I will start a new thread on this with the
>> configuration file I have chosen?
>>
>
> I need a demo projects which puts all the pieces together. This is all to
> abstract to me.
>
>
>> >What can I do with a metadata stream and the CTF trace stream?
>> These can be transported to the host and then viewed using Trace Compass.
>>
>> I lost some time trying to figure things out on my own. I will be more
>> frequent in communicating places where I am stuck. Apologies for the same.
>>
>
> Is visualization with Trace Compass a goal of this project?
>
>
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

On 28/06/18 07:42, Vidushi Vashishth wrote:

>which use case does this address? What is the connection to an RTEMS 
application?
This converts the trace buffering of the fileio sample testcase output 
to CTF. This pertains to the function tracing testcase. The CTF stream 
generated records the entry and exit of the calloc, malloc, realloc 
and free functions and also the values of the arguments and return values.


Could you please create a self-contained repository which contains

* a README

* a simple RTEMS application which runs on a simulator BSP

* the stuff that makes it possible to view the trace output (it is not a 
problem if it doesn't work, but all pieces should be included)


The repository should not be a clone of some larger project. It may 
contain external references as submodules.




>Why do I need sudo to run the commands?
The command needs permission to create a directory for the traces. The 
command did not work without this.


It worked without sudo in my clone. Using sudo for this kind of stuff 
would be a very severe usability issue from my point of view.




>Why do I need your babeltrace? Not being able to use a tool from a standard distribution 
makes it harder to do RTEMS tracing.
I wanted to work on integrating barectf with RTEMS trace linker 
instead. I documented the approach in: 
https://vidushivashishth.github.io/sixthpost/.


Could you please also post your stuff on the devel@rtems.org mailing 
list. I don't have time to look for blog posts actively.


Did you modify babeltrace?

Why do you need a very recent babeltrace for your work?

I was stuck on deciding the yaml configuration input for the process 
and sent out an email for this discussion on the mailing list as well. 
I have however finally created one. I can continue work on the barectf 
approach if that is okay with you? I will start a new thread on this 
with the configuration file I have chosen?


I need a demo projects which puts all the pieces together. This is all 
to abstract to me.




>What can I do with a metadata stream and the CTF trace stream?
These can be transported to the host and then viewed using Trace Compass.

I lost some time trying to figure things out on my own. I will be more 
frequent in communicating places where I am stuck. Apologies for the same.


Is visualization with Trace Compass a goal of this project?

--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

>which use case does this address? What is the connection to an RTEMS
application?
This converts the trace buffering of the fileio sample testcase output to
CTF. This pertains to the function tracing testcase. The CTF stream
generated records the entry and exit of the calloc, malloc, realloc and
free functions and also the values of the arguments and return values.

>Why do I need sudo to run the commands?
The command needs permission to create a directory for the traces. The
command did not work without this.

>Why do I need your babeltrace? Not being able to use a tool from a
standard distribution makes it harder to do RTEMS tracing.
I wanted to work on integrating barectf with RTEMS trace linker instead. I
documented the approach in: https://vidushivashishth.github.io/sixthpost/.
I was stuck on deciding the yaml configuration input for the process and
sent out an email for this discussion on the mailing list as well. I have
however finally created one. I can continue work on the barectf approach if
that is okay with you? I will start a new thread on this with the
configuration file I have chosen?

>What can I do with a metadata stream and the CTF trace stream?
These can be transported to the host and then viewed using Trace Compass.

I lost some time trying to figure things out on my own. I will be more
frequent in communicating places where I am stuck. Apologies for the same.


On Thu, Jun 28, 2018 at 10:42 AM, Vidushi Vashishth 
wrote:

> >Run on what? RTEMS?
> yes.
>
> >Are there any RTEMS repos with your work?
> I havent pushed my code since it is throwing errors. Should I push it
> nonetheless for you to see? Or create another repo?
>
> >RTEMS does not support `system()`.
> Does RTEMS support fdopen or popen? I am trying to execute the babeltrace
> command on the shell, from within the c program. I could use babeltrace C
> API instead but Gedare suggested during a meeting that it might not be
> integrable with RTEMS.
>
> >Could please explain how you are integrating `program.c`?
> I have created a new shell utility function in the
> *rtems/cpukit/libmisc/shell/main_rtrace.c*. It saves the raw trace buffer
> to a file and then uses the logic of `program.c`
> to convert it into CTF traces.
>
>
> On Thu, Jun 28, 2018 at 10:07 AM, Chris Johns  wrote:
>
>> On 28/06/2018 13:48, Vidushi Vashishth wrote:
>> >
>> > Sorry for the delay. I had been trying to make the code run.
>>
>> Run on what? RTEMS?
>>
>> > https://github.com/VidushiVashishth/babeltrace/tree/master/Demo
>>
>> Are there any RTEMS repos with your work?
>>
>> > This is a demonstration of the how to use babeltrace to convert rtems
>> traces. I
>> > have integrated the program.c code in the rtems main_rtrace.c file.
>> This is
>> > throwing some errors that I am working on.
>>
>> RTEMS does not support `system()`.
>>
>> Could please explain how you are integrating `program.c`?
>>
>> Thanks
>> Chris
>>
>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

>Run on what? RTEMS?
yes.

>Are there any RTEMS repos with your work?
I havent pushed my code since it is throwing errors. Should I push it
nonetheless for you to see? Or create another repo?

>RTEMS does not support `system()`.
Does RTEMS support fdopen or popen? I am trying to execute the babeltrace
command on the shell, from within the c program. I could use babeltrace C
API instead but Gedare suggested during a meeting that it might not be
integrable with RTEMS.

>Could please explain how you are integrating `program.c`?
I have created a new shell utility function in the
*rtems/cpukit/libmisc/shell/main_rtrace.c*. It saves the raw trace buffer
to a file and then uses the logic of `program.c`
to convert it into CTF traces.


On Thu, Jun 28, 2018 at 10:07 AM, Chris Johns  wrote:

> On 28/06/2018 13:48, Vidushi Vashishth wrote:
> >
> > Sorry for the delay. I had been trying to make the code run.
>
> Run on what? RTEMS?
>
> > https://github.com/VidushiVashishth/babeltrace/tree/master/Demo
>
> Are there any RTEMS repos with your work?
>
> > This is a demonstration of the how to use babeltrace to convert rtems
> traces. I
> > have integrated the program.c code in the rtems main_rtrace.c file. This
> is
> > throwing some errors that I am working on.
>
> RTEMS does not support `system()`.
>
> Could please explain how you are integrating `program.c`?
>
> Thanks
> Chris
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

Hello,

On 28/06/18 05:48, Vidushi Vashishth wrote:

Hi!

Sorry for the delay. I had been trying to make the code run.
https://github.com/VidushiVashishth/babeltrace/tree/master/Demo
This is a demonstration of the how to use babeltrace to convert rtems 
traces.


which use case does this address? What is the connection to an RTEMS 
application?


Why do I need sudo to run the commands?

Using my openSUSE 42.3 babeltrace yields:

    babeltrace run --component=A:source.text.dmesg 
--params='no-extract-timestamp=no' --key=path --value ./rtemtrace 
--component=B:sink.ctf.fs --key=path --value ./ --connect=A:B

    Error parsing options.

    BabelTrace Trace Viewer and Converter 1.2.4"

When I compile the babeltrace clone from you, then I get:

../install/bin/babeltrace run --component=A:source.text.dmesg 
--params='no-extract-timestamp=no' --key=path --value ./rtemtrace 
--component=B:sink.ctf.fs --key=path --value ./ --connect=A:B

ctf.fs sink creating trace in .//rtemtrace-1

Why do I need your babeltrace? Not being able to use a tool from a 
standard distribution makes it harder to do RTEMS tracing.


From your Readme.md:

"These commands will show the generated metadata stream and a ctf trace 
stream."


What can I do with a metadata stream and the CTF trace stream?

At the moment I have absolutely no idea how I can use this stuff to do 
RTEMS tracing. I thought that I can trace an RTEMS application (e.g. 
task switches) and then see some graphical output in a tool like Trace 
Compass (http://tracecompass.org/#screenshots).


Quite frankly, I don't think that you worked 30+ hours per week on 
average for this GSoC project. If you want to pass the next evaluation, 
then you really have to work harder.


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Chris Johns]

On 28/06/2018 13:48, Vidushi Vashishth wrote:
> 
> Sorry for the delay. I had been trying to make the code run.

Run on what? RTEMS?

> https://github.com/VidushiVashishth/babeltrace/tree/master/Demo

Are there any RTEMS repos with your work?

> This is a demonstration of the how to use babeltrace to convert rtems traces. 
> I
> have integrated the program.c code in the rtems main_rtrace.c file. This is
> throwing some errors that I am working on.

RTEMS does not support `system()`.

Could please explain how you are integrating `program.c`?

Thanks
Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Vidushi Vashishth]

Hi!

Sorry for the delay. I had been trying to make the code run.
https://github.com/VidushiVashishth/babeltrace/tree/master/Demo
This is a demonstration of the how to use babeltrace to convert rtems
traces. I have integrated the program.c code in the rtems main_rtrace.c
file. This is throwing some errors that I am working on. I will push the
changes to the main_rtrace.c file in a couple of hours.

Thanks

On Wed, Jun 27, 2018 at 10:31 AM, Sebastian Huber <
sebastian.hu...@embedded-brains.de> wrote:

> Hello Vidushi,
>
> On 22/06/18 03:16, Vidushi Vashishth wrote:
>
>> Progress so far:
>>
>> - Still working on adding rtrace shell capability to convert RTEMS traces
>> into CTF. There has been progress which I will push to (
>> https://github.com/VidushiVashishth/rtems) by end of today.
>>
>
> I don't see an update in this repository from you. Could you please
> publish your work today so that I can try it out.
>
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim,
> 
> Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [GSOC - Tracing] Status Update

[Sebastian Huber]

Hello Vidushi,

On 22/06/18 03:16, Vidushi Vashishth wrote:

Progress so far:

- Still working on adding rtrace shell capability to convert RTEMS 
traces into CTF. There has been progress which I will push to 
(https://github.com/VidushiVashishth/rtems) by end of today.


I don't see an update in this repository from you. Could you please 
publish your work today so that I can try it out.


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[GSOC - Tracing] Status Update

[Vidushi Vashishth]

Hi!

I have updated the weekly status of my work on the GSoC tracking page. Here
is a summary of it:

Progress so far:

- Still working on adding rtrace shell capability to convert RTEMS traces
into CTF. There has been progress which I will push to (
https://github.com/VidushiVashishth/rtems) by end of today.
- I have gone through libdebugger's abstraction of transport interfaces in
preparation of the second phase. I need to deliver a transport mechanism to
transfer buffers to the host in this phase.

Next up:

- Submit mergeable code for CTF integration
- Write a blog on the possible transport mechanisms to be explored (
https://vidushivashishth.github.io/seventhpost/)
- Setup qemu for ARM as working environment to try qemu fat files and file
transfer as a method of transport.

Thanks!
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel