[
https://issues.apache.org/jira/browse/OOZIE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16756086#comment-16756086
]
Hadoop QA commented on OOZIE-3135:
----------------------------------
Testing JIRA OOZIE-3135
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
. {color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/sqoop] that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
sharelib/sqoop/findbugs-new.html
. The most important FindBugs errors are:
. At SqoopMain.java:[line 126]: Found reliance on default encoding in
org.apache.oozie.action.hadoop.SqoopMain.setUpSqoopLog4J2(String): new
java.io.FileWriter(File)
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3110
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/986/
> Configure log4j2 in SqoopMain
> -----------------------------
>
> Key: OOZIE-3135
> URL: https://issues.apache.org/jira/browse/OOZIE-3135
> Project: Oozie
> Issue Type: Sub-task
> Affects Versions: 5.0.0b1
> Reporter: Attila Sasvari
> Assignee: Julia Kinga Marton
> Priority: Major
> Attachments: OOZIE-3135-001.patch
>
>
> In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in
> {{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for
> Oozie). We need to setup log4j accordingly (it is also related to
> HADOOP-12956).
> Without proper configuration in the Sqoop action, we won't be able to get
> external job id-s (SqoopActionExecutor unit tests and real action would be
> also affected).
>
> [The API for Log4j 2 is not compatible with Log4j
> 1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both
> hadoop 2 and hadoop 3 profiles for a while.
> We could use reflection to determine the type of the logger object in
> {{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it,
> but there might be a better way.
> For example we could do something like this:
> - add a new method for configuring log4j2:
> {code}
> private String setUpSqoopLog4J2(final String rootLogLevel) throws
> IOException {
> System.out.println("Setting up log4j2");
> final String logFile = getSqoopLogFile();
> final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
> try (Writer writer = new FileWriter(log4j2Xml))
> {
> final String logj2SettingsXml = "<?xml version=\"1.0\"
> encoding=\"UTF-8\"?>\n" +
> "<Configuration status=\"WARN\">\n" +
> " <Appenders>\n" +
> " <Console name=\"Console\"
> target=\"SYSTEM_OUT\">\n" +
> " <PatternLayout pattern=\"%d{HH:mm:ss.SSS}
> [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " </Console>\n" +
> " <File name=\"File\" fileName=\"" + logFile +
> "\"> \n" +
> " <PatternLayout pattern=\"%d{HH:mm:ss.SSS}
> [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " </File> \n" +
> " </Appenders>\n" +
> " <Loggers>\n" +
> " <Root level=\"" + rootLogLevel.toLowerCase() +
> "\">\n" +
> " <AppenderRef ref=\"Console\"/>\n" +
> " <AppenderRef ref=\"File\"/>\n" +
> " </Root>\n" +
> " </Loggers>\n" +
> "</Configuration>";
> writer.write(logj2SettingsXml);
> }
> System.out.printf("log4j2 configuration file created at %s%n",
> log4j2Xml.getAbsolutePath());
> final LoggerContext context = (LoggerContext)
> LogManager.getContext(false);
> context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2
> reconfiguration
> return logFile;
> }
> {code}
> and call it in the {{run()}} method if the mapreduce client is using slf4j
> for logging:
> {code}
> String logFile;
> // MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j
> accordingly
> if
> (org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
> isAssignableFrom(org.slf4j.Logger.class)) {
> logFile = setUpSqoopLog4J2(rootLogLevel);
> }
> else {
> logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
