[
https://issues.apache.org/jira/browse/CASSANDRA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-3061:
--------------------------------------
Reviewer: tjake
Fix Version/s: 0.8.5
> Optionally skip log4j configuration
> -----------------------------------
>
> Key: CASSANDRA-3061
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3061
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.8.4
> Reporter: Aaron Morton
> Assignee: Aaron Morton
> Priority: Minor
> Fix For: 0.8.5
>
> Attachments: 0001-3061.patch
>
>
> from this thread
> http://groups.google.com/group/brisk-users/browse_thread/thread/3a18f4679673bea8
> When brisk accesses cassandra classes inside of a Hadoop Task JVM the
> AbstractCassandraDaemon uses a log4j PropertyConfigurator to setup cassandra
> logging. This closes all the existing appenders, including the
> TaskLogAppender for the hadoop task. They are not opened again because they
> are not in the config.
> log4j has Logger Repositories to handle multiple configs in the same process,
> but there is a bit of suck involved in making a RepositorySelector.
> Two examples...
> http://www.mail-archive.com/[email protected]/msg02972.html
> http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2/html/Getting_Started_Guide/logging.log4j.reposelect.html
> Basically all the selector has access to thread local storage, and it looks
> like normally people get the class loader from the current thread. A thread
> will inherit it's class loader from the thread that created it, unless
> otherwise specified.
> We have code in the same thread the uses hadoop and cassandra classes, so
> this could be a dead end.
> As a work around i've added cassandra.log4j.configure JVM param and made the
> AbstractCassandraServer skip the log4j config if it's false. My job completes
> and I can see the cassandra code logging an extra message I put in into the
> Hadoop task log file...
> 2011-08-19 15:56:06,442 WARN
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Metrics system not
> started: Cannot locate configuration: tried
> hadoop-metrics2-maptask.properties, hadoop-metrics2.properties
> 2011-08-19 15:56:06,776 INFO
> org.apache.cassandra.service.AbstractCassandraDaemon: Logging initialized
> externally
> 2011-08-19 15:56:07,332 INFO org.apache.hadoop.mapred.MapTask:
> numReduceTasks: 0
>
> The param has to be passed to the task JVM, so need to modify Haddop
> mapred-site.xml as follows
> <property>
> <name>mapred.child.java.opts</name>
> <value>-Xmx256m -Dcassandra.log4j.configure=false</value>
> <description>
> Tune your mapred jvm arguments for best performance.
> Also see documentation from jvm vendor.
> </description>
> </property>
> It's not pretty but it works. In my extra log4j logging I can see the second
> reset() call is gone.
> Change the to Hadoop TaskLogAppender also stops the NPE but there may also be
> some lost log messages
> https://issues.apache.org/jira/browse/HADOOP-7556
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
