[
https://issues.apache.org/jira/browse/METRON-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898498#comment-16898498
]
Dale Richardson commented on METRON-2195:
-----------------------------------------
Performance test with lazy logging added: added as a disabled unit test.
The results of a test run on my machine shown below:
===============================
Total trial time (ms): 22.752827
===============================
Small object 1000 times
DescriptiveStatistics:
n: 1000
min: 0.007333
max: 0.667019
mean: 0.018786147000000003
std dev: 0.023760301192959944
median: 0.014077
skewness: 20.761050581381458
kurtosis: 556.2525388151329
===============================
Total trial time (ms): 2.827201
===============================
Small object 1000 times using lazy logging
DescriptiveStatistics:
n: 1000
min: 4.99E-4
max: 1.607847
mean: 0.002565500000000002
std dev: 0.05084386966162362
median: 7.01E-4
skewness: 31.567931477605043
kurtosis: 997.6667296992453
===============================
Total trial time (ms): 28190.98512
===============================
Large object 1000 times
DescriptiveStatistics:
n: 1000
min: 24.834654
max: 154.110025
mean: 28.185177632
std dev: 5.750345163228156
median: 27.031372
skewness: 13.401082278526083
kurtosis: 253.2784266258804
===============================
Total trial time (ms): 1.086161
===============================
Large object 1000 times using lazy logging
DescriptiveStatistics:
n: 1000
min: 2.34E-4
max: 0.463436
mean: 9.264779999999996E-4
std dev: 0.014714673562929612
median: 3.565E-4
skewness: 31.16228339668676
kurtosis: 979.9505834291792
===============================
Total trial time (ms): 0.415353
===============================
Simple string 1000 times
DescriptiveStatistics:
n: 1000
min: 1.91E-4
max: 0.018793
mean: 2.84338E-4
std dev: 7.675489085935066E-4
median: 2.29E-4
skewness: 19.097139802144284
kurtosis: 403.0772650972191
===============================
Total trial time (ms): 1.076497
===============================
Simple string 1000 times using lazy logging
DescriptiveStatistics:
n: 1000
min: 2.45E-4
max: 0.371249
mean: 8.975510000000001E-4
std dev: 0.012477908060045725
median: 3.885E-4
skewness: 27.454079996752938
kurtosis: 791.6760062810134
===============================
> Add defensive log level checks when constructing logs is expensive
> ------------------------------------------------------------------
>
> Key: METRON-2195
> URL: https://issues.apache.org/jira/browse/METRON-2195
> Project: Metron
> Issue Type: Improvement
> Reporter: Otto Fowler
> Assignee: Dale Richardson
> Priority: Critical
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> There are instances where we log, and output strings for json objects and
> other things that are quite expensive.
> These are done regardless of the log level being enabled and can increase
> performance significantly:
> https://gist.github.com/mmiklavc/7fd6af13bfa0ca05d9b3f4e7806c8d77
> https://github.com/apache/metron/blob/master/metron-platform/metron-writer/metron-writer-storm/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java#L127
> We need to find places where this happens, and employ the best practice check
> for the log level being enabled ( best practice if your parameters or log
> message construction takes a lot of time ).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
