Kishor Patil created STORM-1215:
-----------------------------------
Summary: Use Async Loggers to avoid locking and logging overhead
Key: STORM-1215
URL: https://issues.apache.org/jira/browse/STORM-1215
Project: Apache Storm
Issue Type: Improvement
Components: storm-core
Reporter: Kishor Patil
Assignee: Kishor Patil
The loggers are synchronous with immediateFlush to disk, making some of the
daemons slow down. In some other cases, nimbus is slow too with submit-lock.
Making loggers asynchronous with no necessity to write to disk on every logger
event would improve cpu resource usage for logging.
{code}
"pool-7-thread-986" #1025 prio=5 os_prio=0 tid=0x00007f0f9628c800 nid=0x1b84
runnable [0x00007f0f0fa2a000]
java.lang.Thread.State: RUNNABLE
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
- locked <0x00000003c00ae520> (a java.io.BufferedOutputStream)
at java.io.PrintStream.write(PrintStream.java:482)
- locked <0x00000003c00ae500> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
- locked <0x00000003c00ae640> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
- locked <0x00000003c00ae500> (a java.io.PrintStream)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:806)
- locked <0x00000003c00ae500> (a java.io.PrintStream)
at
org.apache.logging.log4j.status.StatusConsoleListener.log(StatusConsoleListener.java:81)
at
org.apache.logging.log4j.status.StatusLogger.logMessage(StatusLogger.java:218)
at
org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727)
at
org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716)
at
org.apache.logging.log4j.spi.AbstractLogger.error(AbstractLogger.java:344)
at
org.apache.logging.log4j.core.appender.DefaultErrorHandler.error(DefaultErrorHandler.java:59)
at
org.apache.logging.log4j.core.appender.AbstractAppender.error(AbstractAppender.java:86)
at
org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:116)
at
org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99)
at
org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430)
at
org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409)
at
org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367)
at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112)
at
org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:727)
at
org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:716)
at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:198)
at clojure.tools.logging$eval1$fn__7.invoke(NO_SOURCE_FILE:0)
at clojure.tools.logging.impl$fn__28$G__8__39.invoke(impl.clj:16)
at clojure.tools.logging$log_STAR_.invoke(logging.clj:59)
at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:781)
at clojure.lang.RestFn.invoke(RestFn.java:410)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
