On Thu, Jun 7, 2018 at 12:21 PM Robin Sommer <ro...@icir.org> wrote: > > Hmm, I'm still seeing much larger runtimes on that M57 trace using our > test configuration, even with yesterday's change: > > ; Master, pre-Broker > # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro > 73.72user 7.90system 1:06.53elapsed 122%CPU (0avgtext+0avgdata > 199092maxresident) > > ; Current master > # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro > 2191.59user 1606.69system 12:39.92elapsed 499%CPU (0avgtext+0avgdata > 228400maxresident) > > Bro must still be blocking somewhere when reading from trace files.
Thanks, if you pull master changes [1] again there's likely some improvement. You can also try comparing the timing of: # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro Known::use_host_store=F Known::use_service_store=F Known::use_cert_store=F With that, I get the same timings as from before pre-Broker. At least a good chunk of the difference when using data stores is that, for every query, Bro will immediately block until getting a response back from the data store thread/actor. Not doing this type of synchronous data store access when reading pcaps leads to the possibility of results differing between runs (and I recall such differences being likely from experience in running the unit test suite). - Jon [1] https://github.com/bro/broker/commit/9b56fea4999d4e11a5cd2caaafd934759015fab5 _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
