Oh ... I would also need the size of each stream's sample (including the size of vectors if there are any) ...
About index loading: the way the index was marshalled needed to be changed (but was not) after the change you made to indexing (i.e. making indexes dense). A 3-line patch improves performance quite a lot already. Alignment is already pretty good on my test file (~4s). I've generated a 6G log file with 3 streams, ~650k samples in 1 minute of logical time, reading the streams using the Ruby stream aligner yields ~115x if I don't read the data sample, ~8x if I do. To me, this definitely means that the performance problem is not pocolog's low-level handling but both I/O (I have "images" of 1M, so I/O is actually a big performance problem), typelib demarshalling (there is one thing in this code path that sucks big time currently in the typelib-ruby binding related to demarshalling, did not measure the actual impact of it, though) and the fact that I hit the ruby GC pretty hard by creating 650k samples and 650k intermediate buffers. The point is: as soon as you'll get ruby bindings, you'll have the same issues (GC, typelib demarshalling and I/O will not go away). The only thing that goes away is the creation of 650k intermediate buffers, but that can be removed with very little work already. Now, I do get 8x replay speed when demarshalling the data. That's very far from 0.6. When you talk about bad performance, were you using the replay UI ? For reference: pocolog test.0.log < this is a 5.9G file Stream images [/images] < these are 1M samples 5994 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41 [0:09:59.299] Stream motors [/motors] < these are 100 bytes samples 599708 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41 [0:09:59.706] Stream pose_samples [/pose_samples] < these are 512 bytes samples 59970 samples from Sun 08/06/2014 22:36:42 to Sun 08/06/2014 22:46:41 [0:09:59.690] Sylvain
_______________________________________________ Rock-dev mailing list [email protected] http://www.dfki.de/mailman/cgi-bin/listinfo/rock-dev
