[jira] [Updated] (MAPREDUCE-7230) TestHSWebApp.testLogsViewSingle fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szilard Nemeth updated MAPREDUCE-7230:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.1.3
3.2.1
3.3.0
Status: Resolved (was: Patch Available)
> TestHSWebApp.testLogsViewSingle fails
> -
>
> Key: MAPREDUCE-7230
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7230
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, test
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: MAPREDUCE-7230-001.patch
>
>
> TestHSWebApp.testLogsViewSingle fails.
> {code}
> [ERROR]
> testLogsViewSingle(org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp)
> Time elapsed: 0.294 s <<< FAILURE!
> Argument(s) are different! Wanted:
> printWriter.write(
> "Logs not available for container_10_0001_01_01. Aggregation may not
> be complete, Check back later or try the nodemanager at localhost:1234"
> );
> -> at
> org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp.testLogsViewSingle(TestHSWebApp.java:234)
> Actual invocations have different arguments:
> printWriter.print(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.echoWithoutEscapeHtml(TextView.java:62)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at java.io.PrintWriter.print(PrintWriter.java:617)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>",
> 0,
> 90
> );
> -> at java.io.PrintWriter.write(PrintWriter.java:473)
> printWriter.println(
>
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.putWithoutEscapeHtml(TextView.java:81)
> printWriter.print(
> " );
> -> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl.printStartTag(HamletImpl.java:273)
> printWriter.write(
> " );
> -> at java.io.PrintWriter.print(PrintWriter.java:603)
> printWriter.write(
> " 0,
> 5
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7230) TestHSWebApp.testLogsViewSingle fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908160#comment-16908160
]
Szilard Nemeth commented on MAPREDUCE-7230:
---
Hi [~Prabhu Joseph]!
Pushed patch to trunk, bracnch-3.2 and branch-3.1!
Thanks for your contribution!
> TestHSWebApp.testLogsViewSingle fails
> -
>
> Key: MAPREDUCE-7230
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7230
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, test
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Attachments: MAPREDUCE-7230-001.patch
>
>
> TestHSWebApp.testLogsViewSingle fails.
> {code}
> [ERROR]
> testLogsViewSingle(org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp)
> Time elapsed: 0.294 s <<< FAILURE!
> Argument(s) are different! Wanted:
> printWriter.write(
> "Logs not available for container_10_0001_01_01. Aggregation may not
> be complete, Check back later or try the nodemanager at localhost:1234"
> );
> -> at
> org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp.testLogsViewSingle(TestHSWebApp.java:234)
> Actual invocations have different arguments:
> printWriter.print(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.echoWithoutEscapeHtml(TextView.java:62)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at java.io.PrintWriter.print(PrintWriter.java:617)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>",
> 0,
> 90
> );
> -> at java.io.PrintWriter.write(PrintWriter.java:473)
> printWriter.println(
>
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.putWithoutEscapeHtml(TextView.java:81)
> printWriter.print(
> " );
> -> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl.printStartTag(HamletImpl.java:273)
> printWriter.write(
> " );
> -> at java.io.PrintWriter.print(PrintWriter.java:603)
> printWriter.write(
> " 0,
> 5
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7230) TestHSWebApp.testLogsViewSingle fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908162#comment-16908162
]
Prabhu Joseph commented on MAPREDUCE-7230:
--
Thanks [~snemeth].
> TestHSWebApp.testLogsViewSingle fails
> -
>
> Key: MAPREDUCE-7230
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7230
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, test
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: MAPREDUCE-7230-001.patch
>
>
> TestHSWebApp.testLogsViewSingle fails.
> {code}
> [ERROR]
> testLogsViewSingle(org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp)
> Time elapsed: 0.294 s <<< FAILURE!
> Argument(s) are different! Wanted:
> printWriter.write(
> "Logs not available for container_10_0001_01_01. Aggregation may not
> be complete, Check back later or try the nodemanager at localhost:1234"
> );
> -> at
> org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp.testLogsViewSingle(TestHSWebApp.java:234)
> Actual invocations have different arguments:
> printWriter.print(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.echoWithoutEscapeHtml(TextView.java:62)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at java.io.PrintWriter.print(PrintWriter.java:617)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>",
> 0,
> 90
> );
> -> at java.io.PrintWriter.write(PrintWriter.java:473)
> printWriter.println(
>
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.putWithoutEscapeHtml(TextView.java:81)
> printWriter.print(
> " );
> -> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl.printStartTag(HamletImpl.java:273)
> printWriter.write(
> " );
> -> at java.io.PrintWriter.print(PrintWriter.java:603)
> printWriter.write(
> " 0,
> 5
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7230) TestHSWebApp.testLogsViewSingle fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-7230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908207#comment-16908207
]
Hudson commented on MAPREDUCE-7230:
---
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17130 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17130/])
MAPREDUCE-7230. TestHSWebApp.testLogsViewSingle fails. Contributed by (snemeth:
rev 3f4f097f1f29249c8ad5d59ccefcddf0a66b2e6b)
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
> TestHSWebApp.testLogsViewSingle fails
> -
>
> Key: MAPREDUCE-7230
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7230
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver, test
>Affects Versions: 3.3.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: MAPREDUCE-7230-001.patch
>
>
> TestHSWebApp.testLogsViewSingle fails.
> {code}
> [ERROR]
> testLogsViewSingle(org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp)
> Time elapsed: 0.294 s <<< FAILURE!
> Argument(s) are different! Wanted:
> printWriter.write(
> "Logs not available for container_10_0001_01_01. Aggregation may not
> be complete, Check back later or try the nodemanager at localhost:1234"
> );
> -> at
> org.apache.hadoop.mapreduce.v2.hs.webapp.TestHSWebApp.testLogsViewSingle(TestHSWebApp.java:234)
> Actual invocations have different arguments:
> printWriter.print(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.echoWithoutEscapeHtml(TextView.java:62)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>"
> );
> -> at java.io.PrintWriter.print(PrintWriter.java:617)
> printWriter.write(
> " "http://www.w3.org/TR/html4/strict.dtd";>",
> 0,
> 90
> );
> -> at java.io.PrintWriter.write(PrintWriter.java:473)
> printWriter.println(
>
> );
> -> at
> org.apache.hadoop.yarn.webapp.view.TextView.putWithoutEscapeHtml(TextView.java:81)
> printWriter.print(
> " );
> -> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl.printStartTag(HamletImpl.java:273)
> printWriter.write(
> " );
> -> at java.io.PrintWriter.print(PrintWriter.java:603)
> printWriter.write(
> " 0,
> 5
> );
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-7168) Add option to not kill already-done map tasks when node becomes unusable
[ https://issues.apache.org/jira/browse/MAPREDUCE-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned MAPREDUCE-7168: -- Assignee: Mikayla Konst > Add option to not kill already-done map tasks when node becomes unusable > > > Key: MAPREDUCE-7168 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7168 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: mrv2 >Affects Versions: 2.9.2 > Environment: Google Compute Engine (Dataproc), Java 8 >Reporter: Mikayla Konst >Assignee: Mikayla Konst >Priority: Minor > Attachments: MAPREDUCE-7168.patch > > > When a node becomes unusable, if there are still reduce tasks running, all > completed map tasks that were run on that node are killed so that they can be > re-run on a different node. This is because the node can no longer serve > shuffle data, so the map task output cannot be fetched by the reducers. > If map tasks do not write their shuffle data locally, killing already-done > map tasks will make the job lose map progress unnecessarily. This change > prevents map progress from being lost when shuffle data is not written > locally by providing a property mapreduce.map.rerun-if-node-unusable that can > be set to false to prevent killing already-done map tasks. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7082) Fix FileInputFormat throw java.lang.ArrayIndexOutOfBoundsException(0)
[
https://issues.apache.org/jira/browse/MAPREDUCE-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908573#comment-16908573
]
Wei-Chiu Chuang commented on MAPREDUCE-7082:
-1 sorry this change breaks shading because it pulls in HDFS library into
compile scope. We have to find a different way. It's too heavy to introduce a
library into classpath just to add an exception class.
> Fix FileInputFormat throw java.lang.ArrayIndexOutOfBoundsException(0)
> -
>
> Key: MAPREDUCE-7082
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7082
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
>Affects Versions: 2.7.1
> Environment: CentOS 7
> Hive 1.2.1
> Hadoop 2.7.1
>Reporter: tartarus
>Priority: Major
> Attachments: MAPREDUCE_7082.001.patch, MAPREDUCE_7082.patch
>
>
> when hdfs is miss block and then MR is create split with FileInputFormat
> then will throw ArrayIndexOutOfBoundsException like this
> {code:java}
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.hadoop.mapred.FileInputFormat.identifyHosts(FileInputFormat.java:708)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplitHostsAndCachedHosts(FileInputFormat.java:675)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:365)
> at
> com.hadoop.mapred.DeprecatedLzoTextInputFormat.getSplits(DeprecatedLzoTextInputFormat.java:129)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:305)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:407)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getCombineSplits(CombineHiveInputFormat.java:408)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:571)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:363)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:355)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:231)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
> {code}
> part code of method {color:#d04437}getSplits(JobConf job, int
> numSplits){color} :
> {code:java}
> if (isSplitable(fs, path)) {
> long blockSize = file.getBlockSize();
> long splitSize = computeSplitSize(goalSize, minSize, blockSize);
> long bytesRemaining = length;
> while (((double) bytesRemaining)/splitSize > SPLIT_SLOP) {
> String[][] splitHosts = getSplitHostsAndCachedHosts(blkLocations,
> length-bytesRemaining, splitSize, clusterMap);
> splits.add(makeSplit(path, length-bytesRemaining, splitSize,
> splitHosts[0], splitHosts[1]));
> bytesRemaining -= splitSize;
> }
> if (bytesRemaining != 0) {
> String[][] splitHosts = getSplitHostsAndCachedHosts(blkLocations, length
> - bytesRemaining, bytesRemaining, clusterMap);
> splits.add(makeSplit(path, length - bytesRemaining, bytesRemaining,
> splitHosts[0], splitHosts[1]));
> }
> } else {
> if (LOG.isDebugEnabled()) {
> // Log only if the file is big enough to be splitted
> if (length > Math.min(file.getBlockSize(), minSize)) {
> LOG.debug("File is not splittable so no parallelization "
> + "is possible: " + file.getPath());
> }
> }
> String[][] splitHosts =
> getSplitHostsAndCachedHosts(blkLocations,0,length,clusterMap);
> splits.add(makeSplit(path, 0, length, splitHosts[0], splitHosts[1]));
> }
> {code}
> part code of method
> {color:#d04437}getSplitHostsAndCachedHosts(BlockLocation[] blkLocations,
> {color}
> {color:#d04437} long offset, long splitSize, NetworkTopology
> clusterMap){color} :
> {code:java}
> allTopos = blkLocations[index].getTopologyPaths();
> // If no topology information is available, just
> // prefix a fakeRack
> if (allTopos.length == 0) {
> allTopos = fakeRacks(blkLocations, index);
> }
> ...
> retur
[jira] [Assigned] (MAPREDUCE-7082) Fix FileInputFormat throw java.lang.ArrayIndexOutOfBoundsException(0)
[
https://issues.apache.org/jira/browse/MAPREDUCE-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang reassigned MAPREDUCE-7082:
--
Assignee: tartarus
> Fix FileInputFormat throw java.lang.ArrayIndexOutOfBoundsException(0)
> -
>
> Key: MAPREDUCE-7082
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7082
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
>Affects Versions: 2.7.1
> Environment: CentOS 7
> Hive 1.2.1
> Hadoop 2.7.1
>Reporter: tartarus
>Assignee: tartarus
>Priority: Major
> Attachments: MAPREDUCE_7082.001.patch, MAPREDUCE_7082.patch
>
>
> when hdfs is miss block and then MR is create split with FileInputFormat
> then will throw ArrayIndexOutOfBoundsException like this
> {code:java}
> java.lang.ArrayIndexOutOfBoundsException: 0
> at
> org.apache.hadoop.mapred.FileInputFormat.identifyHosts(FileInputFormat.java:708)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplitHostsAndCachedHosts(FileInputFormat.java:675)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:365)
> at
> com.hadoop.mapred.DeprecatedLzoTextInputFormat.getSplits(DeprecatedLzoTextInputFormat.java:129)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:305)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:407)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getCombineSplits(CombineHiveInputFormat.java:408)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:571)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:363)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:355)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:231)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
> at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
> {code}
> part code of method {color:#d04437}getSplits(JobConf job, int
> numSplits){color} :
> {code:java}
> if (isSplitable(fs, path)) {
> long blockSize = file.getBlockSize();
> long splitSize = computeSplitSize(goalSize, minSize, blockSize);
> long bytesRemaining = length;
> while (((double) bytesRemaining)/splitSize > SPLIT_SLOP) {
> String[][] splitHosts = getSplitHostsAndCachedHosts(blkLocations,
> length-bytesRemaining, splitSize, clusterMap);
> splits.add(makeSplit(path, length-bytesRemaining, splitSize,
> splitHosts[0], splitHosts[1]));
> bytesRemaining -= splitSize;
> }
> if (bytesRemaining != 0) {
> String[][] splitHosts = getSplitHostsAndCachedHosts(blkLocations, length
> - bytesRemaining, bytesRemaining, clusterMap);
> splits.add(makeSplit(path, length - bytesRemaining, bytesRemaining,
> splitHosts[0], splitHosts[1]));
> }
> } else {
> if (LOG.isDebugEnabled()) {
> // Log only if the file is big enough to be splitted
> if (length > Math.min(file.getBlockSize(), minSize)) {
> LOG.debug("File is not splittable so no parallelization "
> + "is possible: " + file.getPath());
> }
> }
> String[][] splitHosts =
> getSplitHostsAndCachedHosts(blkLocations,0,length,clusterMap);
> splits.add(makeSplit(path, 0, length, splitHosts[0], splitHosts[1]));
> }
> {code}
> part code of method
> {color:#d04437}getSplitHostsAndCachedHosts(BlockLocation[] blkLocations,
> {color}
> {color:#d04437} long offset, long splitSize, NetworkTopology
> clusterMap){color} :
> {code:java}
> allTopos = blkLocations[index].getTopologyPaths();
> // If no topology information is available, just
> // prefix a fakeRack
> if (allTopos.length == 0) {
> allTopos = fakeRacks(blkLocations, index);
> }
> ...
> return new String[][] { identifyHosts(allTopos.length, racksMap),
> new String[0]};
> {code}
> part code of method{color:#d04437} identifyHosts(int replicationFactor,
> Map racksMap) :{color}
> {code:java}
[jira] [Commented] (MAPREDUCE-7082) Fix FileInputFormat throw java.lang.ArrayIndexOutOfBoundsException(0)
[
https://issues.apache.org/jira/browse/MAPREDUCE-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908596#comment-16908596
]
Hadoop QA commented on MAPREDUCE-7082:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 12m
44s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
9s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | MAPREDUCE-7082 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919605/MAPREDUCE_7082.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 971e87e2fcbd 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 5882cf9 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7657/testReport/ |
| Max. process+thread count | 1601 (vs. ulimit of 1) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduc
[jira] [Commented] (MAPREDUCE-7168) Add option to not kill already-done map tasks when node becomes unusable
[
https://issues.apache.org/jira/browse/MAPREDUCE-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908605#comment-16908605
]
Hadoop QA commented on MAPREDUCE-7168:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 34s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange}
hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 6 new +
665 unchanged - 0 fixed = 671 total (was 665) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 16s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
19s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
6s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | MAPREDUCE-7168 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12950263/MAPREDUCE-7168.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 24fe5c50ec4d 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23
09:17:01 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/pe
