[
https://issues.apache.org/jira/browse/HADOOP-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16237876#comment-16237876
]
Steve Loughran edited comment on HADOOP-15015 at 11/3/17 4:41 PM:
------------------------------------------------------------------
Patch 001
# main log for normal output. special xml and config ones for debugging at that
level.
# moved all the logging
# Deleted the old boolean keys...nothing downstream was using them.
# Added empty sets to the relevant fields, so we can eliminate all the != null
checks
# Sort things for output by feeding sets through TreeSet before iterating
through the values.
# Assertion errors now include error text.
# Went with the IDE's java 8 migrations to leaner code all round, except for
the most trivial for loops
Example
{code}
Failed tests:
TestMapreduceConfigFields>TestConfigurationFieldsBase.testCompareXmlAgainstConfigurationClass:542
mapred-default.xml has 2 properties missing in interface
org.apache.hadoop.mapreduce.MRJobConfig interface
org.apache.hadoop.mapreduce.MRConfig class
org.apache.hadoop.mapreduce.v2.jobhistory.JHAdminConfig class
org.apache.hadoop.mapred.ShuffleHandler class
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat class
org.apache.hadoop.mapreduce.lib.input.FileInputFormat class
org.apache.hadoop.mapreduce.Job class
org.apache.hadoop.mapreduce.lib.input.NLineInputFormat class
org.apache.hadoop.mapred.JobConf class
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter Entries:
mapreduce.outputcommitter.factory.class
mapreduce.outputcommitter.factory.scheme.s3a expected:<0> but was:<2>
{code}
Getting that list of missing entries is key, as now Jenkins will include its
list of problems in all its failed tests. You shouldn't need to look at the
output logs to debug basic problems.
was (Author: [email protected]):
Patch 001
1. main log for normal output. special xml and config ones for debugging at
that level.
1. moved all the logging
1. Deleted the old boolean keys...nothing downstream was using them.
1. Added empty sets to the relevant fields, so we can eliminate all the != null
checks
1. Sort things for output by feeding sets through TreeSet before iterating
through the values.
1. Assertion errors now include error text.
1. Went with the IDE's java 8 migrations to leaner code all round
Example
{code}
Failed tests:
TestMapreduceConfigFields>TestConfigurationFieldsBase.testCompareXmlAgainstConfigurationClass:542
mapred-default.xml has 2 properties missing in interface
org.apache.hadoop.mapreduce.MRJobConfig interface
org.apache.hadoop.mapreduce.MRConfig class
org.apache.hadoop.mapreduce.v2.jobhistory.JHAdminConfig class
org.apache.hadoop.mapred.ShuffleHandler class
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat class
org.apache.hadoop.mapreduce.lib.input.FileInputFormat class
org.apache.hadoop.mapreduce.Job class
org.apache.hadoop.mapreduce.lib.input.NLineInputFormat class
org.apache.hadoop.mapred.JobConf class
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter Entries:
mapreduce.outputcommitter.factory.class
mapreduce.outputcommitter.factory.scheme.s3a expected:<0> but was:<2>
{code}
Getting that list of missing entries is key, as now Jenkins will include its
list of problems in all its failed tests. You shouldn't need to look at the
output logs to debug basic problems.
> TestConfigurationFieldsBase to use SLF4J for logging
> ----------------------------------------------------
>
> Key: HADOOP-15015
> URL: https://issues.apache.org/jira/browse/HADOOP-15015
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf, test
> Affects Versions: 3.0.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-15015-001.patch
>
>
> {{TestConfigurationFieldsBase}} has a protected "configDebug" field used to
> turn logging on/off
> {code}
> if (configDebug) {
> System.out.println("Field: " + f);
> }
> {code}
> Presumably its there to allow people with code access to debug their classes.
> But if we switch to SLF4J you get controllable logging @ runtime.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
