[
https://issues.apache.org/jira/browse/METRON-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16896360#comment-16896360
]
Michael Miklavcic commented on METRON-2195:
-------------------------------------------
{quote}My next question is does anybody have a prioritised list of performance
sensitive areas I should be hunting through to update to the new logging
wrapper?
{quote}
I would look at the writers, for starters. Beyond that, I think it's going to
be a trial and error exercise. Minimally, address the HDFSWriter logging
statement in the original example. One option that's a bit brute force could be
doing a grep in the full code base on LOG.trace( and LOG.debug( and looking for
any obvious complex calls. I just ran this locally and trace appears to be
quite manageable. grep --include *.java -R LOG.trace .|wc -l
50
Debug came up with 264 results, but that's still not egregious to look for low
hanging fruit. I agree with your conclusion that we should not worry as much
about "error" logging, because those are typically already gated by try/catch
blocks.
One additional note about this API - I think we need to be conspicuously clear
that this is a temporary solution and add appropriate javadoc warnings to the
interface that references this ticket. We might also consider adopting the
Hadoop API annotation approach (i.e. @ Unstable), but that's probably overkill
for a private API. The last piece of this should be to create a Jira for
upgrading SLF4J. Here's an example for upgrading JUnit -
https://issues.apache.org/jira/browse/METRON-2037
> Add defensive log level checks when constructing logs is expensive
> ------------------------------------------------------------------
>
> Key: METRON-2195
> URL: https://issues.apache.org/jira/browse/METRON-2195
> Project: Metron
> Issue Type: Improvement
> Reporter: Otto Fowler
> Assignee: Dale Richardson
> Priority: Critical
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> There are instances where we log, and output strings for json objects and
> other things that are quite expensive.
> These are done regardless of the log level being enabled and can increase
> performance significantly:
> https://gist.github.com/mmiklavc/7fd6af13bfa0ca05d9b3f4e7806c8d77
> https://github.com/apache/metron/blob/master/metron-platform/metron-writer/metron-writer-storm/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java#L127
> We need to find places where this happens, and employ the best practice check
> for the log level being enabled ( best practice if your parameters or log
> message construction takes a lot of time ).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
