On Thu, Jun 7, 2018 at 12:21 PM Robin Sommer wrote:
>
> Hmm, I'm still seeing much larger runtimes on that M57 trace using our
> test configuration, even with yesterday's change:
>
> ; Master, pre-Broker
> # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro
> 73.72user 7.90system 1:06.53elapsed 122%CPU (0avgtext+0avgdata
> 199092maxresident)
>
> ; Current master
> # zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro
> 2191.59user 1606.69system 12:39.92elapsed 499%CPU (0avgtext+0avgdata
> 228400maxresident)
>
> Bro must still be blocking somewhere when reading from trace files.
Thanks, if you pull master changes [1] again there's likely some improvement.
You can also try comparing the timing of:
# zcat 2009-M57-day11-18.trace.gz | time bro -r - tests/m57-long.bro
Known::use_host_store=F Known::use_service_store=F
Known::use_cert_store=F
With that, I get the same timings as from before pre-Broker. At least
a good chunk of the difference when using data stores is that, for
every query, Bro will immediately block until getting a response back
from the data store thread/actor. Not doing this type of synchronous
data store access when reading pcaps leads to the possibility of
results differing between runs (and I recall such differences being
likely from experience in running the unit test suite).
- Jon
[1]
https://github.com/bro/broker/commit/9b56fea4999d4e11a5cd2caaafd934759015fab5
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
