[
https://issues.apache.org/jira/browse/OOZIE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755984#comment-16755984
]
Julia Kinga Marton commented on OOZIE-3135:
-------------------------------------------
Unfortunately Sqoop is using log4j1, so if we configure Log4j2, we will loose
the Sqoop logs.
We need to check if only the tests are affected or real action is affected as
well.
If only the tests are affected, we need to find another approach for fixing
them.
> Configure log4j2 in SqoopMain
> -----------------------------
>
> Key: OOZIE-3135
> URL: https://issues.apache.org/jira/browse/OOZIE-3135
> Project: Oozie
> Issue Type: Sub-task
> Affects Versions: 5.0.0b1
> Reporter: Attila Sasvari
> Assignee: Julia Kinga Marton
> Priority: Major
> Attachments: OOZIE-3135-001.patch
>
>
> In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in
> {{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for
> Oozie). We need to setup log4j accordingly (it is also related to
> HADOOP-12956).
> Without proper configuration in the Sqoop action, we won't be able to get
> external job id-s (SqoopActionExecutor unit tests and real action would be
> also affected).
>
> [The API for Log4j 2 is not compatible with Log4j
> 1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both
> hadoop 2 and hadoop 3 profiles for a while.
> We could use reflection to determine the type of the logger object in
> {{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it,
> but there might be a better way.
> For example we could do something like this:
> - add a new method for configuring log4j2:
> {code}
> private String setUpSqoopLog4J2(final String rootLogLevel) throws
> IOException {
> System.out.println("Setting up log4j2");
> final String logFile = getSqoopLogFile();
> final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
> try (Writer writer = new FileWriter(log4j2Xml))
> {
> final String logj2SettingsXml = "<?xml version=\"1.0\"
> encoding=\"UTF-8\"?>\n" +
> "<Configuration status=\"WARN\">\n" +
> " <Appenders>\n" +
> " <Console name=\"Console\"
> target=\"SYSTEM_OUT\">\n" +
> " <PatternLayout pattern=\"%d{HH:mm:ss.SSS}
> [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " </Console>\n" +
> " <File name=\"File\" fileName=\"" + logFile +
> "\"> \n" +
> " <PatternLayout pattern=\"%d{HH:mm:ss.SSS}
> [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " </File> \n" +
> " </Appenders>\n" +
> " <Loggers>\n" +
> " <Root level=\"" + rootLogLevel.toLowerCase() +
> "\">\n" +
> " <AppenderRef ref=\"Console\"/>\n" +
> " <AppenderRef ref=\"File\"/>\n" +
> " </Root>\n" +
> " </Loggers>\n" +
> "</Configuration>";
> writer.write(logj2SettingsXml);
> }
> System.out.printf("log4j2 configuration file created at %s%n",
> log4j2Xml.getAbsolutePath());
> final LoggerContext context = (LoggerContext)
> LogManager.getContext(false);
> context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2
> reconfiguration
> return logFile;
> }
> {code}
> and call it in the {{run()}} method if the mapreduce client is using slf4j
> for logging:
> {code}
> String logFile;
> // MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j
> accordingly
> if
> (org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
> isAssignableFrom(org.slf4j.Logger.class)) {
> logFile = setUpSqoopLog4J2(rootLogLevel);
> }
> else {
> logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
