Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-15 Thread Doug Schaefer 
On Fri, 2019-03-15 at 10:56 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,


On Fri, Mar 15, 2019 at 02:17:04PM +, Doug Schaefer wrote:

Hi Geneviève, thanks for this.


For now, I'm focused on libbabeltrace. I think this is something my users will 
like to work with. They're mainly C hacks working on our kernel to start. I 
think they'll also like using the python bindings.


Did you have a look at these projects?


LTTng-analyses : 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lttng_lttng-2Danalyses=DwIFAw=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=1pK_XMBIe71LlYS3RRDwiTMzbShbPSQ_TIc18jjghtM=qbUlWwabJcfJYNVSdT4Bznx-7Nm0zBQ51eZ_QB30veQ=

LTTng-lami (Machine interface spec) : 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_lttng_lami-2Dspec=DwIFAw=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=1pK_XMBIe71LlYS3RRDwiTMzbShbPSQ_TIc18jjghtM=bGKvY6hgxnwYxJJHIqE30Q86cqprybagOZkUEfQV_xM=


It might be a good starting point or example for what can be achieved via the

binding. Keep in mind that these projects are *experimental*.

Cool. Thanks! Everything I'm doing at the moment is experimental so these fit 
right in ☺.



Cheers

___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-15 Thread Jonathan Rajotte-Julien 
Hi Doug,

On Fri, Mar 15, 2019 at 02:17:04PM +, Doug Schaefer wrote:
> Hi Geneviève, thanks for this.
> 
> For now, I'm focused on libbabeltrace. I think this is something my users 
> will like to work with. They're mainly C hacks working on our kernel to 
> start. I think they'll also like using the python bindings.

Did you have a look at these projects?

LTTng-analyses : https://github.com/lttng/lttng-analyses
LTTng-lami (Machine interface spec) : https://github.com/lttng/lami-spec

It might be a good starting point or example for what can be achieved via the
binding. Keep in mind that these projects are *experimental*.

Cheers
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-15 Thread Geneviève Bastien 
Hi Doug,

Welcome to the CTF ecosystem!

FYI, one thing I am working now on TraceCompass is to add support for scripting 
using the Eclipse EASE framework.

In short, one could use python, javascript, ruby or groovy to interact with 
POJOs. We'll provide a simple API to get the trace events (the parsers are 
already available in TraceCompass, so all supported trace types will work), 
fill backend (state systems, segment stores) and create time graph / XY views.

In a few weeks, I should be able to share a prototype for the community to try. 
I hope you can benefit from it.

Regards,

Geneviève


On 2019-03-12 11:23 a.m., Doug Schaefer wrote:
> It should be easy to generate a random one with the class I mentioned below 
> and fill it with 16 million events. BTW, two of the int fields are 5 and 10 
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
> I'll also take a look at the babeltrace code and see what I can see.
>
> BTW, my focus on libbabeltrace is to allow for a full range of tooling for 
> our users using the language of their choice. The python binding is 
> particularly interesting. As we move forward into the new world of IDEs, I 
> can see a node.js binding being interesting as well. And it may even make 
> sense to use it in Java tooling like TraceCompass using JNA. That's the 
> vision at least, but first, baby steps ☺.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:
>> Hi Doug,
>> On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:
>> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
>> My biggest worry is on the read side. Babeltrace blew up on that file. It's 
>> a pretty simple trace (for now) with a single event class with 4 ints and a 
>> sequence of 32-bit unsigned ints which is usually only 2 elements long.
>> This is not I expect. Still eithet jgalar or eep might have a more insight on
>> this. CCing the lttng-dev mailing list.
>> Is there any way to share us a similar trace? Either with a generator or we 
>> can
>> provide a link for you to upload it. The current limit on bugs.lttng.org is a
>> bit too small for such trace.
>> Aside from that, am very pleased with the how easy CTF is to work with. 
>> Looking forward to doing more.
>> Doug.
>> On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:
>> Hi Doug,
>> Great to hear you're coming to a standard! I don't know if trace compass 
>> will scale properly as I don't know what the trace configuration is.
>> I have one of our trace which has 16 million events, the size of each event 
>> is around 32 bytes giving me a 540MB file. My first attempt at simply 
>> writing out the CTF events without a flush ran out of virtual memory. I then 
>> flushed after every event which made each event take 32K. So I found a 
>> middle ground and the resulting stream file is close to the same size.
>> My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
>> each event is 32 bytes, basically flush every 25k events or so.
>> Please keep us posted!
>> Matthew.
>> 
>> From: tracecompass-dev-boun...@eclipse.org 
>>  
>> > > on behalf of Doug Schaefer 
>> mailto:dschae...@blackberry.com>>
>> Sent: Tuesday, March 12, 2019 10:30:15 AM
>> To: tracecompass-...@eclipse.org 
>> Subject: [tracecompass-dev] Scalability
>> Hey gang,
>> We're finally starting to look at converting our custom traces into CTF so 
>> we can leverage tools like TraceCompass and, of course, contribute to it. 
>> One thing I quickly ran into is a scalability issue I'm seeing with 
>> libbabeltrace.
>> I have one of our trace which has 16 million events, the size of each event 
>> is around 32 bytes giving me a 540MB file. My first attempt at simply 
>> writing out the CTF events without a flush ran out of virtual memory. I then 
>> flushed after every event which made each event take 32K. So I found a 
>> middle ground and the resulting stream file is close to the same size.
>> But when I use babeltrace to print it out, I ran out of virtual memory. I 
>> then hand coded a reader and simply adding the trace to the context caused 
>> the memory issue. It really looks like libbabeltrace (version 1.5 from the 
>> Ubuntu 18.04 distro), tries to inflate the events into it's internal 
>> representation for the entire trace. I need to do more investigation to 
>> confirm that.
>> So my question for this list, would TraceCompass do better? Does it have 
>> it's own parsing libraries?
>> Thanks,
>> Doug
>> ___
>> tracecompass-dev mailing list
>> tracecompass-...@eclipse.org 
>> To change your delivery options, retrieve your password, or unsubscribe from 
>> this list, visit
>>

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-15 Thread Doug Schaefer 
Hi Geneviève, thanks for this.

For now, I'm focused on libbabeltrace. I think this is something my users will 
like to work with. They're mainly C hacks working on our kernel to start. I 
think they'll also like using the python bindings.

It would be awesome to have a stack that built on top of that that used the 
same APIs in whatever language. My current thinking is to build a prototype 
that visualizes using D3 in an Electron app, so JavaScript/TypeScript would be 
key there. I'd be happy to help contribute to that. But that will probably be 
in the Babeltrace repo.

I'm still not clear how TraceCompass fits in but will start looking at that 
once I'm off the ground. At this stage of the game, as you can see from my work 
on CDT, whatever solution I plan on adopting has to plug into not only the 
Eclipse IDE, but HTML5 front ends like VS Code or just simple Electron apps or 
web pages as well. It's a pretty exciting new direction. Imagine visualizing 
traces using WebGL. We can do some pretty cool things that users will get a lot 
of value from.

Cheers,
Doug.

On Fri, 2019-03-15 at 09:40 -0400, Geneviève Bastien wrote:

Hi Doug,

Welcome to the CTF ecosystem!

FYI, one thing I am working now on TraceCompass is to add support for scripting 
using the Eclipse EASE framework.

In short, one could use python, javascript, ruby or groovy to interact with 
POJOs. We'll provide a simple API to get the trace events (the parsers are 
already available in TraceCompass, so all supported trace types will work), 
fill backend (state systems, segment stores) and create time graph / XY views.

In a few weeks, I should be able to share a prototype for the community to try. 
I hope you can benefit from it.

Regards,

Geneviève


On 2019-03-12 11:23 a.m., Doug Schaefer wrote:
It should be easy to generate a random one with the class I mentioned below and 
fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits 
respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our 
users using the language of their choice. The python binding is particularly 
interesting. As we move forward into the new world of IDEs, I can see a node.js 
binding being interesting as well. And it may even make sense to use it in Java 
tooling like TraceCompass using JNA. That's the vision at least, but first, 
baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.

My biggest worry is on the read side. Babeltrace blew up on that file. It's a 
pretty simple trace (for now) with a single event class with 4 ints and a 
sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.

Aside from that, am very pleased with the how easy CTF is to work with. Looking 
forward to doing more.

Doug.

On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:

Hi Doug,

Great to hear you're coming to a standard! I don't know if trace compass will 
scale properly as I don't know what the trace configuration is.

I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving me a 540MB file. My first attempt at simply writing out 
the CTF events without a flush ran out of virtual memory. I then flushed after 
every event which made each event take 32K. So I found a middle ground and the 
resulting stream file is close to the same size.

My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
each event is 32 bytes, basically flush every 25k events or so.

Please keep us posted!

Matthew.



From: 
tracecompass-dev-boun...@eclipse.org
 
mailto:tracecompass-dev-boun...@eclipse.org>>
 on behalf of Doug Schaefer 
mailto:dschae...@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-...@eclipse.org

Subject: [tracecompass-dev] Scalability

Hey gang,

We're finally starting to look at converting our custom traces into CTF so we 
can leverage tools like TraceCompass and, of course, contribute to it. One 
thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.

I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving me a 540MB file. My first attempt at simply writing out 
the CTF events without a flush ran out of virtual memory. I

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Philippe Proulx 
On Wed, Mar 13, 2019 at 10:14 AM Doug Schaefer  wrote:
>
> I'll use the patch .
>
> I was somewhat following the CTF Writer API python example which
> implied I could use the original. Is that not the case?

Yes it's the case, but there was a bug!

Phil

>
> Thanks!
> Doug
>
>
> --
> This transmission (including any attachments) may contain confidential 
> information, privileged material (including material protected by the 
> solicitor-client or other applicable privileges), or constitute non-public 
> information. Any use of this information by anyone other than the intended 
> recipient is prohibited. If you have received this transmission in error, 
> please immediately reply to the sender and delete this information from your 
> system. Use, dissemination, distribution, or reproduction of this 
> transmission by unintended recipients is not authorized and may be unlawful.
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Doug Schaefer 
On Wed, 2019-03-13 at 08:59 -0400, Philippe Proulx wrote:
> On Wed, Mar 13, 2019 at 8:46 AM Doug Schaefer <
> dschae...@blackberry.com> wrote:
> 
> Very cool. Would explicitly setting the byte order for each field be
> a short term workaround?
> 
> You found a very specific scenario highlighting this bug.
> 
> Some field types are copied behind the scenes for resolving purposes.
> This is done in bt_ctf_stream_class_add_event_class() and
> bt_ctf_writer_create_stream(). The effective byte order of the
> original
> (`data_length_type` for example) remains unset, while the internal
> copy
> is set. However you use the originals to create fields.
> 
> Without the patch, a temporary fix would be to put all the field type
> references and get the copies (which have the correct byte order)
> using
> bt_ctf_event_class_get_field_by_name() and
> bt_ctf_field_sequence_get_field() _after_ calling
> bt_ctf_writer_create_stream() in your specific case.
> 

I'll use the patch ☺.

I was somewhat following the CTF Writer API python example which
implied I could use the original. Is that not the case?

Thanks!
Doug

___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Philippe Proulx 
On Wed, Mar 13, 2019 at 8:46 AM Doug Schaefer  wrote:
>
> Very cool. Would explicitly setting the byte order for each field be a short 
> term workaround?

You found a very specific scenario highlighting this bug.

Some field types are copied behind the scenes for resolving purposes.
This is done in bt_ctf_stream_class_add_event_class() and
bt_ctf_writer_create_stream(). The effective byte order of the original
(`data_length_type` for example) remains unset, while the internal copy
is set. However you use the originals to create fields.

Without the patch, a temporary fix would be to put all the field type
references and get the copies (which have the correct byte order) using
bt_ctf_event_class_get_field_by_name() and
bt_ctf_field_sequence_get_field() _after_ calling
bt_ctf_writer_create_stream() in your specific case.

Phil

>
> Sent from my BlackBerry - the most secure mobile device - via the Rogers 
> Network
> From: eeppelitel...@gmail.com
> Sent: March 13, 2019 8:30 AM
> To: matthew.khou...@ericsson.com
> Cc: jonathan.rajotte-jul...@efficios.com; dschae...@blackberry.com; 
> tracecompass-...@eclipse.org; lttng-dev@lists.lttng.org
> Subject: Re: [lttng-dev] [tracecompass-dev] Scalability
>
> On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
>  wrote:
> >
> > Hi Doug & Efficios,
> >
> >
> > I suspect the issue is with libbabeltrace-ctf. When opening the trace in 
> > Trace Compass, it failed saying the sequence was 2gb big. When opening in 
> > babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, 
> > I saw 2gb allocations.
>
> Yes, byte order issues. Here's the fix:
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_efficios_babeltrace_pull_103=DwIFaQ=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M=0QPaQrHAx605SVgAPeaZnF264NFCA44uOkcXdbV5dpA=>.
>
> Phil
>
> >
> >
> > On another subject, fixing the trace made it scale well in both babeltrace 
> > and Trace Compass.
> >
> >
> > I don't know why, data_length writes the wrong data. Hope this helps all 
> > parties.
> >
> >
> > BR,
> >
> > Matthew
> >
> > 
> > From: tracecompass-dev-boun...@eclipse.org 
> >  on behalf of Doug Schaefer 
> > 
> > Sent: Tuesday, March 12, 2019 2:50:29 PM
> > To: jonathan.rajotte-jul...@efficios.com
> > Cc: lttng-dev@lists.lttng.org; tracecompass-...@eclipse.org
> > Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
> >
> > I have attached the C file for a generator that creates a trace that 
> > reproduces the memory issue with babeltrace. The same issue happens with a 
> > custom program in the call to add a trace to a context.
> >
> > I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the 
> > version number 1.5.5.
> >
> > Thanks for the help!
> >
> > On Tue, 2019-03-12 at 15:47 +, Doug Schaefer wrote:
> >
> > In the thread on the tracecompass list, I also mentioned that the 
> > babeltrace utility itself ran out of memory when reading the file. That's 
> > actually my biggest worry. I got around the generator issue by flushing 
> > every 100K events.
> >
> > I'll create a quick generator that shows the same issue and post it in a 
> > bit.
> >
> > Doug.
> >
> > On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
> >
> > On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:
> >
> > It should be easy to generate a random one with the class I mentioned below
> >
> > and fill it with 16 million events.
> >
> >
> > Not if the problem is in the generator code ;).
> >
> >
> > BTW, two of the int fields are 5 and 10
> >
> > bits respectively, but I'm not sure that matters (or at least it shouldn't).
> >
> >
> > Make sure to check the endian-ness of those fields.
> >
> >
> > What class? Not sure I see any reference to any.
> >
> >
> > Since you are generating CTF the error can be in the generator code. Please
> >
> > provide either a trace reproducing the issue or a base reproducer. It will 
> > be
> >
> > much easier overall than us trying to reproduce it blindly.
> >
> >
> > Cheers.
> >
> >
> > ___
> >
> > lttng-dev mailing list
> >
> > lttng-dev@lists.lttng.org
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinf

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Doug Schaefer 
Very cool. Would explicitly setting the byte order for each field be a short 
term workaround?

Sent from my BlackBerry - the most secure mobile device - via the Rogers Network
From: eeppelitel...@gmail.com
Sent: March 13, 2019 8:30 AM
To: matthew.khou...@ericsson.com
Cc: jonathan.rajotte-jul...@efficios.com; dschae...@blackberry.com; 
tracecompass-...@eclipse.org; lttng-dev@lists.lttng.org
Subject: Re: [lttng-dev] [tracecompass-dev] Scalability


On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
 wrote:
>
> Hi Doug & Efficios,
>
>
> I suspect the issue is with libbabeltrace-ctf. When opening the trace in 
> Trace Compass, it failed saying the sequence was 2gb big. When opening in 
> babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I 
> saw 2gb allocations.

Yes, byte order issues. Here's the fix:
<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_efficios_babeltrace_pull_103=DwIFaQ=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M=0QPaQrHAx605SVgAPeaZnF264NFCA44uOkcXdbV5dpA=>.

Phil

>
>
> On another subject, fixing the trace made it scale well in both babeltrace 
> and Trace Compass.
>
>
> I don't know why, data_length writes the wrong data. Hope this helps all 
> parties.
>
>
> BR,
>
> Matthew
>
> 
> From: tracecompass-dev-boun...@eclipse.org 
>  on behalf of Doug Schaefer 
> 
> Sent: Tuesday, March 12, 2019 2:50:29 PM
> To: jonathan.rajotte-jul...@efficios.com
> Cc: lttng-dev@lists.lttng.org; tracecompass-...@eclipse.org
> Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
>
> I have attached the C file for a generator that creates a trace that 
> reproduces the memory issue with babeltrace. The same issue happens with a 
> custom program in the call to add a trace to a context.
>
> I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the 
> version number 1.5.5.
>
> Thanks for the help!
>
> On Tue, 2019-03-12 at 15:47 +, Doug Schaefer wrote:
>
> In the thread on the tracecompass list, I also mentioned that the babeltrace 
> utility itself ran out of memory when reading the file. That's actually my 
> biggest worry. I got around the generator issue by flushing every 100K events.
>
> I'll create a quick generator that shows the same issue and post it in a bit.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
>
> On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:
>
> It should be easy to generate a random one with the class I mentioned below
>
> and fill it with 16 million events.
>
>
> Not if the problem is in the generator code ;).
>
>
> BTW, two of the int fields are 5 and 10
>
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
>
> Make sure to check the endian-ness of those fields.
>
>
> What class? Not sure I see any reference to any.
>
>
> Since you are generating CTF the error can be in the generator code. Please
>
> provide either a trace reproducing the issue or a base reproducer. It will be
>
> much easier overall than us trying to reproduce it blindly.
>
>
> Cheers.
>
>
> ___
>
> lttng-dev mailing list
>
> lttng-dev@lists.lttng.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev=DwICAg=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g=
>
>
> -
> This transmission (including any attachments) may contain confidential 
> information, privileged material (including material protected by the 
> solicitor-client or other applicable privileges), or constitute non-public 
> information. Any use of this information by anyone other than the intended 
> recipient is prohibited. If you have received this transmission in error, 
> please immediately reply to the sender and delete this information from your 
> system. Use, dissemination, distribution, or reproduction of this 
> transmission by unintended recipients is not authorized and may be unlawful.
> ___
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev=DwIFaQ=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=mro2UhDCczUi0qeroLQR1acWtiyKnR0DeQlLZcufM9M=0dykfQKNunY0J3PjBwLhAqvF-WUIJ-gk74D-duicVKI=
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Philippe Proulx 
On Wed, Mar 13, 2019 at 8:13 AM Matthew Khouzam
 wrote:
>
> Hi Doug & Efficios,
>
>
> I suspect the issue is with libbabeltrace-ctf. When opening the trace in 
> Trace Compass, it failed saying the sequence was 2gb big. When opening in 
> babeltrace, it OOMed with 2gb allocations. When opening with a hex editor, I 
> saw 2gb allocations.

Yes, byte order issues. Here's the fix:
.

Phil

>
>
> On another subject, fixing the trace made it scale well in both babeltrace 
> and Trace Compass.
>
>
> I don't know why, data_length writes the wrong data. Hope this helps all 
> parties.
>
>
> BR,
>
> Matthew
>
> 
> From: tracecompass-dev-boun...@eclipse.org 
>  on behalf of Doug Schaefer 
> 
> Sent: Tuesday, March 12, 2019 2:50:29 PM
> To: jonathan.rajotte-jul...@efficios.com
> Cc: lttng-dev@lists.lttng.org; tracecompass-...@eclipse.org
> Subject: Re: [tracecompass-dev] [lttng-dev] Scalability
>
> I have attached the C file for a generator that creates a trace that 
> reproduces the memory issue with babeltrace. The same issue happens with a 
> custom program in the call to add a trace to a context.
>
> I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the 
> version number 1.5.5.
>
> Thanks for the help!
>
> On Tue, 2019-03-12 at 15:47 +, Doug Schaefer wrote:
>
> In the thread on the tracecompass list, I also mentioned that the babeltrace 
> utility itself ran out of memory when reading the file. That's actually my 
> biggest worry. I got around the generator issue by flushing every 100K events.
>
> I'll create a quick generator that shows the same issue and post it in a bit.
>
> Doug.
>
> On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:
>
> On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:
>
> It should be easy to generate a random one with the class I mentioned below
>
> and fill it with 16 million events.
>
>
> Not if the problem is in the generator code ;).
>
>
> BTW, two of the int fields are 5 and 10
>
> bits respectively, but I'm not sure that matters (or at least it shouldn't).
>
>
> Make sure to check the endian-ness of those fields.
>
>
> What class? Not sure I see any reference to any.
>
>
> Since you are generating CTF the error can be in the generator code. Please
>
> provide either a trace reproducing the issue or a base reproducer. It will be
>
> much easier overall than us trying to reproduce it blindly.
>
>
> Cheers.
>
>
> ___
>
> lttng-dev mailing list
>
> lttng-dev@lists.lttng.org
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev=DwICAg=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g=
>
>
> -
> This transmission (including any attachments) may contain confidential 
> information, privileged material (including material protected by the 
> solicitor-client or other applicable privileges), or constitute non-public 
> information. Any use of this information by anyone other than the intended 
> recipient is prohibited. If you have received this transmission in error, 
> please immediately reply to the sender and delete this information from your 
> system. Use, dissemination, distribution, or reproduction of this 
> transmission by unintended recipients is not authorized and may be unlawful.
> ___
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-13 Thread Matthew Khouzam 
Hi Doug & Efficios,


I suspect the issue is with libbabeltrace-ctf. When opening the trace in Trace 
Compass, it failed saying the sequence was 2gb big. When opening in babeltrace, 
it OOMed with 2gb allocations. When opening with a hex editor, I saw 2gb 
allocations.


On another subject, fixing the trace made it scale well in both babeltrace and 
Trace Compass.


I don't know why, data_length writes the wrong data. Hope this helps all 
parties.


BR,

Matthew


From: tracecompass-dev-boun...@eclipse.org 
 on behalf of Doug Schaefer 

Sent: Tuesday, March 12, 2019 2:50:29 PM
To: jonathan.rajotte-jul...@efficios.com
Cc: lttng-dev@lists.lttng.org; tracecompass-...@eclipse.org
Subject: Re: [tracecompass-dev] [lttng-dev] Scalability

I have attached the C file for a generator that creates a trace that reproduces 
the memory issue with babeltrace. The same issue happens with a custom program 
in the call to add a trace to a context.

I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the 
version number 1.5.5.

Thanks for the help!

On Tue, 2019-03-12 at 15:47 +, Doug Schaefer wrote:
In the thread on the tracecompass list, I also mentioned that the babeltrace 
utility itself ran out of memory when reading the file. That's actually my 
biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.


___

lttng-dev mailing list

lttng-dev@lists.lttng.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev=DwICAg=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g=


-
This transmission (including any attachments) may contain confidential 
information, privileged material (including material protected by the 
solicitor-client or other applicable privileges), or constitute non-public 
information. Any use of this information by anyone other than the intended 
recipient is prohibited. If you have received this transmission in error, 
please immediately reply to the sender and delete this information from your 
system. Use, dissemination, distribution, or reproduction of this transmission 
by unintended recipients is not authorized and may be unlawful.
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Doug Schaefer 
I have attached the C file for a generator that creates a trace that reproduces 
the memory issue with babeltrace. The same issue happens with a custom program 
in the call to add a trace to a context.

I am using babeltrace that I get from my Ubuntu 18.04 distro. It has the 
version number 1.5.5.

Thanks for the help!

On Tue, 2019-03-12 at 15:47 +, Doug Schaefer wrote:
In the thread on the tracecompass list, I also mentioned that the babeltrace 
utility itself ran out of memory when reading the file. That's actually my 
biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.


___

lttng-dev mailing list

lttng-dev@lists.lttng.org

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.lttng.org_cgi-2Dbin_mailman_listinfo_lttng-2Ddev=DwICAg=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=DWwlFfl5FD5R8-HPY3GfbG6l6m7y_o2gXPER8IkQstk=bPK6X3vufUtEAemGpslNIlIL4UslBsgK-SWnraL-e9g=

-
This transmission (including any attachments) may contain confidential 
information, privileged material (including material protected by the 
solicitor-client or other applicable privileges), or constitute non-public 
information. Any use of this information by anyone other than the intended 
recipient is prohibited. If you have received this transmission in error, 
please immediately reply to the sender and delete this information from your 
system. Use, dissemination, distribution, or reproduction of this transmission 
by unintended recipients is not authorized and may be unlawful.
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#define CHECK(call) \
if ((call) < 0) { \
fprintf(stderr, "failed %s:%d %s\n", __func__, __LINE__, #call); \
exit(1); \
}

#define NULLCHECK(ptr) \
if ((ptr) == NULL) { \
fprintf(stderr, "failed %s:%d %s\n", __func__, __LINE__, #ptr); \
exit(1); \
}

int main(int argc, char *argv[])
{
struct bt_ctf_writer *writer = bt_ctf_writer_create("scaletest.ctf");
NULLCHECK(writer);

struct bt_ctf_clock *clock = bt_ctf_clock_create("simple_clock");
NULLCHECK(clock);
CHECK(bt_ctf_clock_set_description(clock, "simple clock, really"));
CHECK(bt_ctf_writer_add_clock(writer, clock));

struct bt_ctf_stream_class *scale_stream_class = bt_ctf_stream_class_create("scale_stream");
NULLCHECK(scale_stream_class);
CHECK(bt_ctf_stream_class_set_clock(scale_stream_class, clock));

// simple class
struct bt_ctf_event_class *simple_event_class = bt_ctf_event_class_create("simple");
NULLCHECK(simple_event_class);

struct bt_ctf_field_type *cpu_type = bt_ctf_field_type_integer_create(6);
NULLCHECK(cpu_type);
CHECK(bt_ctf_field_type_integer_set_signed(cpu_type, 0));
CHECK(bt_ctf_event_class_add_field(simple_event_class, cpu_type, "cpu"));

struct bt_ctf_field_type *class_type = bt_ctf_field_type_integer_create(5);
NULLCHECK(class_type);
CHECK(bt_ctf_field_type_integer_set_signed(class_type, 0));
CHECK(bt_ctf_event_class_add_field(simple_event_class, class_type, "class"));

struct bt_ctf_field_type *type_type = bt_ctf_field_type_integer_create(10);
NULLCHECK(type_type);
CHECK(bt_ctf_field_type_integer_set_signed(type_type, 0));
CHECK(bt_ctf_event_class_add_field(simple_event_class, type_type, "type"));

struct bt_ctf_field_type *data_length_type = bt_ctf_field_type_integer_create(32);
NULLCHECK(data_length_type);
CHECK(bt_ctf_field_type_integer_set_signed(data_length_type, 0));
CHECK(bt_ctf_event_class_add_field(simple_event_class, data_length_type, "data_length"));

struct bt_ctf_field_type *data_element_type = bt_ctf_field_type_integer_create(32);
NULLCHECK(data_element_type);
CHECK(bt_ctf_field_type_integer_set_signed(data_element_type, 0));
struct bt_ctf_field_type *data_type =

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Bernd Hufmann 
Hi Doug

Great to see your interest this area.

>> BTW, my focus on libbabeltrace is to allow for a full range of tooling for 
>> our users using the language of their choice. The python binding is 
>> particularly interesting. As we move forward into the new world of IDEs, I 
>> can see a node.js binding being interesting as well. And it may even make 
>> sense to use it in Java tooling like TraceCompass using JNA. That's the 
>> vision at least, but first, baby steps ☺.

For the integration with Trace Compass in Eclipse, we chose to implement a CTF 
parser in Java to avoid using JNI/JNA to libbabeltrace. This was for 
performance reasons over JNI/JNA interface. We had implemented that at the 
beginning when Trace Compass supported LTTng version 0.x (prior CTF) and the 
performance was not sufficient enough.

For the integration in next generation IDEs, we are working on defining a Trace 
Server Protocol (TSP) as an interface between a client and server for 
exchanging of trace date for visualization purposes. It’s a similar idea as the 
LSP (Language Server Protocol) or DAP (Debug Server protocol). The TSP 
interface is on a higher level than what a trace reading interface like the CTF 
trace reader in Java or over your suggested JNA interface over libbabeltrace 
would provide.

We have some promising results for a Typescript front-end based on the next 
generation IDE Theia that uses the TSP to a trace server using Trace Compass 
core features as backend. We can already visualize various Trace Compass views. 
It’s still preliminary, but we could demo it if you’re interested.

BR
Bernd


From: tracecompass-dev-boun...@eclipse.org 
 On Behalf Of Doug Schaefer
Sent: March-12-19 11:23 AM
To: tracecompass-...@eclipse.org; jonathan.rajotte-jul...@efficios.com
Cc: lttng-dev@lists.lttng.org
Subject: Re: [tracecompass-dev] Scalability

It should be easy to generate a random one with the class I mentioned below and 
fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits 
respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our 
users using the language of their choice. The python binding is particularly 
interesting. As we move forward into the new world of IDEs, I can see a node.js 
binding being interesting as well. And it may even make sense to use it in Java 
tooling like TraceCompass using JNA. That's the vision at least, but first, 
baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,



On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.



My biggest worry is on the read side. Babeltrace blew up on that file. It's a 
pretty simple trace (for now) with a single event class with 4 ints and a 
sequence of 32-bit unsigned ints which is usually only 2 elements long.



This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.



Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.





Aside from that, am very pleased with the how easy CTF is to work with. Looking 
forward to doing more.



Doug.



On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:



Hi Doug,





Great to hear you're coming to a standard! I don't know if trace compass will 
scale properly as I don't know what the trace configuration is.





I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving me a 540MB file. My first attempt at simply writing out 
the CTF events without a flush ran out of virtual memory. I then flushed after 
every event which made each event take 32K. So I found a middle ground and the 
resulting stream file is close to the same size.







My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
each event is 32 bytes, basically flush every 25k events or so.





Please keep us posted!





Matthew.





From: 
tracecompass-dev-boun...@eclipse.org
 
mailto:tracecompass-dev-boun...@eclipse.org>>
 on behalf of Doug Schaefer 
mailto:dschae...@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-...@eclipse.org

Subject: [tracecompass-dev] Scalability



Hey gang,



We're finally starting to look at converting our custom traces into CTF so we 
can leverage tools like TraceCompass and, of course, contribute to it. One 
thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.



I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Doug Schaefer 
In the thread on the tracecompass list, I also mentioned that the babeltrace 
utility itself ran out of memory when reading the file. That's actually my 
biggest worry. I got around the generator issue by flushing every 100K events.

I'll create a quick generator that shows the same issue and post it in a bit.

Doug.

On Tue, 2019-03-12 at 11:34 -0400, Jonathan Rajotte-Julien wrote:

On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:

It should be easy to generate a random one with the class I mentioned below

and fill it with 16 million events.


Not if the problem is in the generator code ;).


BTW, two of the int fields are 5 and 10

bits respectively, but I'm not sure that matters (or at least it shouldn't).


Make sure to check the endian-ness of those fields.


What class? Not sure I see any reference to any.


Since you are generating CTF the error can be in the generator code. Please

provide either a trace reproducing the issue or a base reproducer. It will be

much easier overall than us trying to reproduce it blindly.


Cheers.

___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Jonathan Rajotte-Julien 
On Tue, Mar 12, 2019 at 03:23:23PM +, Doug Schaefer wrote:
> It should be easy to generate a random one with the class I mentioned below
> and fill it with 16 million events.

Not if the problem is in the generator code ;).

> BTW, two of the int fields are 5 and 10
> bits respectively, but I'm not sure that matters (or at least it shouldn't).

Make sure to check the endian-ness of those fields.

What class? Not sure I see any reference to any.

Since you are generating CTF the error can be in the generator code. Please
provide either a trace reproducing the issue or a base reproducer. It will be
much easier overall than us trying to reproduce it blindly.

Cheers.
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Doug Schaefer 
It should be easy to generate a random one with the class I mentioned below and 
fill it with 16 million events. BTW, two of the int fields are 5 and 10 bits 
respectively, but I'm not sure that matters (or at least it shouldn't).

I'll also take a look at the babeltrace code and see what I can see.

BTW, my focus on libbabeltrace is to allow for a full range of tooling for our 
users using the language of their choice. The python binding is particularly 
interesting. As we move forward into the new world of IDEs, I can see a node.js 
binding being interesting as well. And it may even make sense to use it in Java 
tooling like TraceCompass using JNA. That's the vision at least, but first, 
baby steps ☺.

Doug.

On Tue, 2019-03-12 at 11:03 -0400, Jonathan Rajotte-Julien wrote:

Hi Doug,


On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:

Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.


My biggest worry is on the read side. Babeltrace blew up on that file. It's a 
pretty simple trace (for now) with a single event class with 4 ints and a 
sequence of 32-bit unsigned ints which is usually only 2 elements long.


This is not I expect. Still eithet jgalar or eep might have a more insight on

this. CCing the lttng-dev mailing list.


Is there any way to share us a similar trace? Either with a generator or we can

provide a link for you to upload it. The current limit on bugs.lttng.org is a

bit too small for such trace.



Aside from that, am very pleased with the how easy CTF is to work with. Looking 
forward to doing more.


Doug.


On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:


Hi Doug,



Great to hear you're coming to a standard! I don't know if trace compass will 
scale properly as I don't know what the trace configuration is.



I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving me a 540MB file. My first attempt at simply writing out 
the CTF events without a flush ran out of virtual memory. I then flushed after 
every event which made each event take 32K. So I found a middle ground and the 
resulting stream file is close to the same size.




My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
each event is 32 bytes, basically flush every 25k events or so.



Please keep us posted!



Matthew.




From: 
tracecompass-dev-boun...@eclipse.org
 
mailto:tracecompass-dev-boun...@eclipse.org>>
 on behalf of Doug Schaefer 
mailto:dschae...@blackberry.com>>

Sent: Tuesday, March 12, 2019 10:30:15 AM

To: tracecompass-...@eclipse.org

Subject: [tracecompass-dev] Scalability


Hey gang,


We're finally starting to look at converting our custom traces into CTF so we 
can leverage tools like TraceCompass and, of course, contribute to it. One 
thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.


I have one of our trace which has 16 million events, the size of each event is 
around 32 bytes giving me a 540MB file. My first attempt at simply writing out 
the CTF events without a flush ran out of virtual memory. I then flushed after 
every event which made each event take 32K. So I found a middle ground and the 
resulting stream file is close to the same size.


But when I use babeltrace to print it out, I ran out of virtual memory. I then 
hand coded a reader and simply adding the trace to the context caused the 
memory issue. It really looks like libbabeltrace (version 1.5 from the Ubuntu 
18.04 distro), tries to inflate the events into it's internal representation 
for the entire trace. I need to do more investigation to confirm that.


So my question for this list, would TraceCompass do better? Does it have it's 
own parsing libraries?


Thanks,

Doug


___

tracecompass-dev mailing list

tracecompass-...@eclipse.org

To change your delivery options, retrieve your password, or unsubscribe from 
this list, visit

https://urldefense.proofpoint.com/v2/url?u=https-3A__www.eclipse.org_mailman_listinfo_tracecompass-2Ddev=DwIBAg=yzoHOc_ZK-sxl-kfGNSEvlJYanssXN3q-lhj0sp26wE=NrrbvTHWa2Nbp_kAN0Hl1o3lM1WAwSes64uBjxjNhMc=ZoCx61VE_sHGa6j3DXehbqjX5P1NGEuDtEFaGORCh9k=qi9q1zvDaC9cJCci0y123O-j66M643YwxRJccJCzg_c=



___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Matthew Khouzam 
Welcome Jonathan,


Also, for people seeking great babeltrace support, I feel I should plug the 
lttng-dev mailinglist.


I hope the URL is not censored. 
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev if it is, just 
search for "lttng-dev -- LTTng development list"

lttng-dev Info Page
lists.lttng.org
To see the collection of prior postings to the list, visit the lttng-dev 
Archives.. Using lttng-dev: To post a message to all the list members, send 
email to lttng-dev@lists.lttng.org. You can subscribe to the list, or change 
your existing subscription, in the sections below.




From: Jonathan Rajotte-Julien 
Sent: Tuesday, March 12, 2019 11:03:14 AM
To: Doug Schaefer; tracecompass developer discussions
Cc: Matthew Khouzam; lttng-dev
Subject: Re: [tracecompass-dev] Scalability

Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:
> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
>
> My biggest worry is on the read side. Babeltrace blew up on that file. It's a 
> pretty simple trace (for now) with a single event class with 4 ints and a 
> sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on
this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can
provide a link for you to upload it. The current limit on bugs.lttng.org is a
bit too small for such trace.

>
> Aside from that, am very pleased with the how easy CTF is to work with. 
> Looking forward to doing more.
>
> Doug.
>
> On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:
>
> Hi Doug,
>
>
> Great to hear you're coming to a standard! I don't know if trace compass will 
> scale properly as I don't know what the trace configuration is.
>
>
> > I have one of our trace which has 16 million events, the size of each event 
> > is around 32 bytes giving me a 540MB file. My first attempt at simply 
> > writing out the CTF events without a flush ran out of virtual memory. I 
> > then flushed after every event which made each event take 32K. So I found a 
> > middle ground and the resulting stream file is close to the same size.
>
>
>
> My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
> each event is 32 bytes, basically flush every 25k events or so.
>
>
> Please keep us posted!
>
>
> Matthew.
>
> 
> From: tracecompass-dev-boun...@eclipse.org 
>  on behalf of Doug Schaefer 
> 
> Sent: Tuesday, March 12, 2019 10:30:15 AM
> To: tracecompass-...@eclipse.org
> Subject: [tracecompass-dev] Scalability
>
> Hey gang,
>
> We're finally starting to look at converting our custom traces into CTF so we 
> can leverage tools like TraceCompass and, of course, contribute to it. One 
> thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.
>
> I have one of our trace which has 16 million events, the size of each event 
> is around 32 bytes giving me a 540MB file. My first attempt at simply writing 
> out the CTF events without a flush ran out of virtual memory. I then flushed 
> after every event which made each event take 32K. So I found a middle ground 
> and the resulting stream file is close to the same size.
>
> But when I use babeltrace to print it out, I ran out of virtual memory. I 
> then hand coded a reader and simply adding the trace to the context caused 
> the memory issue. It really looks like libbabeltrace (version 1.5 from the 
> Ubuntu 18.04 distro), tries to inflate the events into it's internal 
> representation for the entire trace. I need to do more investigation to 
> confirm that.
>
> So my question for this list, would TraceCompass do better? Does it have it's 
> own parsing libraries?
>
> Thanks,
> Doug

> ___
> tracecompass-dev mailing list
> tracecompass-...@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from 
> this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev


--
Jonathan Rajotte-Julien
EfficiOS
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [tracecompass-dev] Scalability

2019-03-12 Thread Jonathan Rajotte-Julien 
Hi Doug,

On Tue, Mar 12, 2019 at 02:56:26PM +, Doug Schaefer wrote:
> Thanks, Matthew, I ended up flushing every 100K events and that seemed OK.
> 
> My biggest worry is on the read side. Babeltrace blew up on that file. It's a 
> pretty simple trace (for now) with a single event class with 4 ints and a 
> sequence of 32-bit unsigned ints which is usually only 2 elements long.

This is not I expect. Still eithet jgalar or eep might have a more insight on
this. CCing the lttng-dev mailing list.

Is there any way to share us a similar trace? Either with a generator or we can
provide a link for you to upload it. The current limit on bugs.lttng.org is a
bit too small for such trace.

> 
> Aside from that, am very pleased with the how easy CTF is to work with. 
> Looking forward to doing more.
> 
> Doug.
> 
> On Tue, 2019-03-12 at 14:41 +, Matthew Khouzam wrote:
> 
> Hi Doug,
> 
> 
> Great to hear you're coming to a standard! I don't know if trace compass will 
> scale properly as I don't know what the trace configuration is.
> 
> 
> > I have one of our trace which has 16 million events, the size of each event 
> > is around 32 bytes giving me a 540MB file. My first attempt at simply 
> > writing out the CTF events without a flush ran out of virtual memory. I 
> > then flushed after every event which made each event take 32K. So I found a 
> > middle ground and the resulting stream file is close to the same size.
> 
> 
> 
> My suggestion is to have 1 MB packets. This makes seeking very efficient. If 
> each event is 32 bytes, basically flush every 25k events or so.
> 
> 
> Please keep us posted!
> 
> 
> Matthew.
> 
> 
> From: tracecompass-dev-boun...@eclipse.org 
>  on behalf of Doug Schaefer 
> 
> Sent: Tuesday, March 12, 2019 10:30:15 AM
> To: tracecompass-...@eclipse.org
> Subject: [tracecompass-dev] Scalability
> 
> Hey gang,
> 
> We're finally starting to look at converting our custom traces into CTF so we 
> can leverage tools like TraceCompass and, of course, contribute to it. One 
> thing I quickly ran into is a scalability issue I'm seeing with libbabeltrace.
> 
> I have one of our trace which has 16 million events, the size of each event 
> is around 32 bytes giving me a 540MB file. My first attempt at simply writing 
> out the CTF events without a flush ran out of virtual memory. I then flushed 
> after every event which made each event take 32K. So I found a middle ground 
> and the resulting stream file is close to the same size.
> 
> But when I use babeltrace to print it out, I ran out of virtual memory. I 
> then hand coded a reader and simply adding the trace to the context caused 
> the memory issue. It really looks like libbabeltrace (version 1.5 from the 
> Ubuntu 18.04 distro), tries to inflate the events into it's internal 
> representation for the entire trace. I need to do more investigation to 
> confirm that.
> 
> So my question for this list, would TraceCompass do better? Does it have it's 
> own parsing libraries?
> 
> Thanks,
> Doug

> ___
> tracecompass-dev mailing list
> tracecompass-...@eclipse.org
> To change your delivery options, retrieve your password, or unsubscribe from 
> this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev


-- 
Jonathan Rajotte-Julien
EfficiOS
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev