[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416329#comment-16416329
]
Renan DelValle commented on AURORA-1825:
[~jingc] I see that you closed this issue as Done, can you provide a link to
the review as well as the version this landed in?
Thanks!
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15694453#comment-15694453
]
Mehrdad Nurolahzade commented on AURORA-1825:
-
I defined [AURORA-1825] to further explore this idea.
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692614#comment-15692614
]
Stephan Erb commented on AURORA-1825:
-
Cool! In addition:
*
http://blog.takipi.com/how-to-instantly-improve-your-java-logging-with-7-logback-tweaks/
indicates that we can use {{%logger}} as a much cheaper replacement for the
class/line reflection.
* Switching from STDERR to STDOUT could probably also yield a small performance
improvement. The first one is unbuffered, the second one is buffered.
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691758#comment-15691758
]
Zameer Manji commented on AURORA-1825:
--
Locally I removed the expensive parts of our logback config with:
{noformat}
diff --git c/src/main/resources/logback.xml w/src/main/resources/logback.xml
index 84c175c..6206806 100644
--- c/src/main/resources/logback.xml
+++ w/src/main/resources/logback.xml
@@ -23,7 +23,7 @@ limitations under the License.
System.err
-%.-1level%date{MMdd HH:mm:ss.SSS} [%thread, %class{0}:%line] %message
%xThrowable%n
+%.-1level%date{MMdd HH:mm:ss.SSS} [%thread] %message %xThrowable%n
{noformat}
Before:
{noformat}
Benchmark (numPendingTasks)
(numTasksToDelete) Mode Cnt Score Error Units
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
1000 thrpt 10 2.510 ± 0.557 ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
1 thrpt 10 0.272 ± 0.030 ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
5 thrpt 10 0.053 ± 0.011 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 1000
N/A thrpt 10 2.446 ± 0.698 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 1
N/A thrpt 10 0.246 ± 0.018 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 5
N/A thrpt 10 0.041 ± 0.006 ops/s
{noformat}
After:
{noformat}
Benchmark (numPendingTasks)
(numTasksToDelete) Mode Cnt Score Error Units
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
1000 thrpt 10 8.640 ± 1.431 ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
1 thrpt 10 0.892 ± 0.066 ops/s
StateManagerBenchmarks.DeleteTasksBenchmark.run N/A
5 thrpt 10 0.172 ± 0.010 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 1000
N/A thrpt 10 4.837 ± 1.511 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 1
N/A thrpt 10 0.510 ± 0.315 ops/s
StateManagerBenchmarks.InsertPendingTasksBenchmark.run 5
N/A thrpt 10 0.079 ± 0.052 ops/s
{noformat}
I picked this benchmark because it logs a lot in the critical path.
We could probably fix this problem by removing line number and removing class
name with the logger name. The net result would be no line numbers but way
faster logging.
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691616#comment-15691616
]
Mehrdad Nurolahzade commented on AURORA-1825:
-
Just as a side note:
It would be interesting to see benchmarks on logging with expensive reflection
based patterns like class name or line number removed. I did not find anything
on this on Logback but Log4j documentation, for example, explicitly warns
against using such patterns in performance critical systems:
[https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html]
{quote}WARNING Generating the caller class information is slow. Thus, use
should be avoided unless execution speed is not an issue.{quote}
{quote}WARNING Generating caller location information is extremely slow and
should be avoided unless execution speed is not an issue.{quote}
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691562#comment-15691562
]
Stephan Erb commented on AURORA-1825:
-
I would love to see a benchmark showing that async logging is worthwhile before
we go down that route. Asynchronous logging can be a real pain when debugging
crashes and spurious bugs.
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Assignee: Jing Chen
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AURORA-1825) Enable async logging by default
[
https://issues.apache.org/jira/browse/AURORA-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688506#comment-15688506
]
Zameer Manji commented on AURORA-1825:
--
We could achieve this by changing {{logback.xml}} to use this:
http://logback.qos.ch/manual/appenders.html#AsyncAppender
> Enable async logging by default
> ---
>
> Key: AURORA-1825
> URL: https://issues.apache.org/jira/browse/AURORA-1825
> Project: Aurora
> Issue Type: Task
>Reporter: Zameer Manji
>Priority: Minor
>
> Based on my experience while working on AURORA-1823 and [~StephanErb]'s work
> on logging recently, I think it would be best if we enabled async logging.
> For example if one attempts to parallelize the work inside
> {{StateManagerImpl}} there isn't much benefit because all of the state
> transitions are logged and all of the threads would contend for the lock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
