[jira] [Commented] (MAPREDUCE-5596) Allow configuring the number of threads used to serve shuffle connections
[ https://issues.apache.org/jira/browse/MAPREDUCE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806782#comment-13806782 ] Jason Lowe commented on MAPREDUCE-5596: --- Thanks, Sandy. Comments on the patch: * maxShuffleThreads is defaulting to the number of available processors instead of twice that amount as documented * MAX_SHUFFLE_THREADS setting has Mapreduce.shuffle.max.threads instead of mapreduce.shuffle.max.threads * Nit: IMHO mentioning Netty in the documentation is unnecessary detail. Allow configuring the number of threads used to serve shuffle connections - Key: MAPREDUCE-5596 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5596 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5596.patch MR1 had mapreduce.tasktracker.http.threads. MR2 always uses the Netty default 2 * Runtime.availableProcessors(). We should make this configurable. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-4980: --- Status: Open (was: Patch Available) Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-4980: --- Status: Patch Available (was: Open) Kicking build. Hoping to get a better run this time now that the build servers have been fixed. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5465) Container killed before hprof dumps profile.out
[ https://issues.apache.org/jira/browse/MAPREDUCE-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806907#comment-13806907 ] Ravi Prakash commented on MAPREDUCE-5465: - And I totally forgot to answer Hitesh's question! Sorry about that. :( NM_SLEEP_DELAY_BEFORE_SIGKILL_MS handles the timeout before a SIGKILL. However at that point in time, a SIGTERM has already been sent once. This can be disruptive too. Container killed before hprof dumps profile.out --- Key: MAPREDUCE-5465 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5465 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mr-am, mrv2 Affects Versions: 2.0.3-alpha Reporter: Radim Kolar Assignee: Ravi Prakash Attachments: MAPREDUCE-5465.patch If there is profiling enabled for mapper or reducer then hprof dumps profile.out at process exit. It is dumped after task signaled to AM that work is finished. AM kills container with finished work without waiting for hprof to finish dumps. If hprof is dumping larger outputs (such as with depth=4 while depth=3 works) , it could not finish dump in time before being killed making entire dump unusable because cpu and heap stats are missing. There needs to be better delay before container is killed if profiling is enabled. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5392) mapred job -history all command throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shinichi Yamashita updated MAPREDUCE-5392:
--
Attachment: MAPREDUCE-5392.patch
mapred job -history all command throws IndexOutOfBoundsException
--
Key: MAPREDUCE-5392
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5392
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0, 2.0.5-alpha, 2.2.0
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Minor
Fix For: 3.0.0
Attachments: MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch, MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch
When I use an all option by mapred job -history comamnd, the following
exceptions are displayed and do not work.
{code}
Exception in thread main java.lang.StringIndexOutOfBoundsException: String
index out of range: -3
at java.lang.String.substring(String.java:1875)
at
org.apache.hadoop.mapreduce.util.HostUtil.convertTrackerNameToHostName(HostUtil.java:49)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.getTaskLogsUrl(HistoryViewer.java:459)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.printAllTaskAttempts(HistoryViewer.java:235)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.print(HistoryViewer.java:117)
at org.apache.hadoop.mapreduce.tools.CLI.viewHistory(CLI.java:472)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:313)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1233)
{code}
This is because a node name recorded in History file is not given tracker_.
Therefore it makes modifications to be able to read History file even if a
node name is not given by tracker_.
In addition, it fixes the URL of displayed task log.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5596) Allow configuring the number of threads used to serve shuffle connections
[ https://issues.apache.org/jira/browse/MAPREDUCE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806958#comment-13806958 ] Sandy Ryza commented on MAPREDUCE-5596: --- Thanks for catching my mistakes, Jason. Uploading a new patch. Regarding mentioning Netty in the doc, I wanted people to know that it was used, meaning that they wouldn't need a thread for each connection. Added a separate sentence to make this more clear. Allow configuring the number of threads used to serve shuffle connections - Key: MAPREDUCE-5596 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5596 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5596.patch MR1 had mapreduce.tasktracker.http.threads. MR2 always uses the Netty default 2 * Runtime.availableProcessors(). We should make this configurable. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5596) Allow configuring the number of threads used to serve shuffle connections
[ https://issues.apache.org/jira/browse/MAPREDUCE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5596: -- Attachment: MAPREDUCE-5596-1.patch Allow configuring the number of threads used to serve shuffle connections - Key: MAPREDUCE-5596 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5596 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5596-1.patch, MAPREDUCE-5596.patch MR1 had mapreduce.tasktracker.http.threads. MR2 always uses the Netty default 2 * Runtime.availableProcessors(). We should make this configurable. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5392) mapred job -history all command throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806967#comment-13806967
]
Hadoop QA commented on MAPREDUCE-5392:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12610591/MAPREDUCE-5392.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs:
org.apache.hadoop.mapreduce.v2.hs.TestJobHistoryServer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4148//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4148//console
This message is automatically generated.
mapred job -history all command throws IndexOutOfBoundsException
--
Key: MAPREDUCE-5392
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5392
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0, 2.0.5-alpha, 2.2.0
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Minor
Fix For: 3.0.0
Attachments: MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch, MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch
When I use an all option by mapred job -history comamnd, the following
exceptions are displayed and do not work.
{code}
Exception in thread main java.lang.StringIndexOutOfBoundsException: String
index out of range: -3
at java.lang.String.substring(String.java:1875)
at
org.apache.hadoop.mapreduce.util.HostUtil.convertTrackerNameToHostName(HostUtil.java:49)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.getTaskLogsUrl(HistoryViewer.java:459)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.printAllTaskAttempts(HistoryViewer.java:235)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.print(HistoryViewer.java:117)
at org.apache.hadoop.mapreduce.tools.CLI.viewHistory(CLI.java:472)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:313)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1233)
{code}
This is because a node name recorded in History file is not given tracker_.
Therefore it makes modifications to be able to read History file even if a
node name is not given by tracker_.
In addition, it fixes the URL of displayed task log.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806963#comment-13806963 ] Robert Kanter commented on MAPREDUCE-4680: -- The test passes locally and the test I added doesn't start a JHS, so it can't be causing this. Job history cleaner should only check timestamps of files in old enough directories --- Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.0.0-alpha Reporter: Sandy Ryza Assignee: Robert Kanter Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch Job history files are stored in /mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806966#comment-13806966 ] Sandy Ryza commented on MAPREDUCE-4680: --- Makes sense. +1, will commit this later today. Job history cleaner should only check timestamps of files in old enough directories --- Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.0.0-alpha Reporter: Sandy Ryza Assignee: Robert Kanter Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch Job history files are stored in /mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5596) Allow configuring the number of threads used to serve shuffle connections
[
https://issues.apache.org/jira/browse/MAPREDUCE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13806989#comment-13806989
]
Hadoop QA commented on MAPREDUCE-5596:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12610596/MAPREDUCE-5596-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4149//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4149//console
This message is automatically generated.
Allow configuring the number of threads used to serve shuffle connections
-
Key: MAPREDUCE-5596
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5596
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: MAPREDUCE-5596-1.patch, MAPREDUCE-5596.patch
MR1 had mapreduce.tasktracker.http.threads. MR2 always uses the Netty
default 2 * Runtime.availableProcessors(). We should make this configurable.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5053) java.lang.InternalError from decompression codec cause reducer to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807072#comment-13807072 ] Sangjin Lee commented on MAPREDUCE-5053: I do not think this made it into 2.0.4 or 2.0.5. I believe the proper fixed version is 2.1.0 for hadoop 2. java.lang.InternalError from decompression codec cause reducer to fail -- Key: MAPREDUCE-5053 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5053 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: trunk, 2.0.3-alpha, 0.23.5 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 0.23.7, 2.0.4-alpha Attachments: MAPREDUCE-5053-1.patch, MAPREDUCE-5053-2.patch, MAPREDUCE-5053-b023-1.patch lz4, snappy, zlib, and lzo Decompressor's only throw java.lang.InternalError. This exception will cause the reducer to fail and bypass all of the fetch failure logic. The decompressing errors should be treated as fetch failures. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-3860: --- Attachment: MAPREDUCE-3860--n3.patch As I see in the logs failures on Linux environment were caused by tests timeouts being too low. Attaching a patch which fixes that. As for failures on Mac env, I see that all M/R jobs failed there. I saw similar issues when running tests without having JAVA_HOME set. Can't find more out of the logs. [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: linux-surefire-reports.tar, mac-surfire-reports.tar, MAPREDUCE-3860--n2.patch, MAPREDUCE-3860--n3.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-4980: --- Attachment: MAPREDUCE-4980--n7.patch Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5597) Missing alternatives in javadocs for deprecated API
Christopher Tubbs created MAPREDUCE-5597: Summary: Missing alternatives in javadocs for deprecated API Key: MAPREDUCE-5597 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5597 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client, documentation, job submission Affects Versions: 2.2.0 Reporter: Christopher Tubbs Deprecated API, such as `new Job()` don't have javadocs explaining what the alternatives are. (It'd also help if the new methods had @since tags to help determine if one could safely use that API on older versions at runtime.) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5392) mapred job -history all command throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807211#comment-13807211
]
Shinichi Yamashita commented on MAPREDUCE-5392:
---
This test passed my local environment and this patch does not change
JobHistoryServer.
mapred job -history all command throws IndexOutOfBoundsException
--
Key: MAPREDUCE-5392
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5392
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0, 2.0.5-alpha, 2.2.0
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Minor
Fix For: 3.0.0
Attachments: MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch, MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch
When I use an all option by mapred job -history comamnd, the following
exceptions are displayed and do not work.
{code}
Exception in thread main java.lang.StringIndexOutOfBoundsException: String
index out of range: -3
at java.lang.String.substring(String.java:1875)
at
org.apache.hadoop.mapreduce.util.HostUtil.convertTrackerNameToHostName(HostUtil.java:49)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.getTaskLogsUrl(HistoryViewer.java:459)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.printAllTaskAttempts(HistoryViewer.java:235)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.print(HistoryViewer.java:117)
at org.apache.hadoop.mapreduce.tools.CLI.viewHistory(CLI.java:472)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:313)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1233)
{code}
This is because a node name recorded in History file is not given tracker_.
Therefore it makes modifications to be able to read History file even if a
node name is not given by tracker_.
In addition, it fixes the URL of displayed task log.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-5598:
-
Affects Version/s: trunk
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: trunk, 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
Robert Kanter created MAPREDUCE-5598:
Summary: TestUserDefinedCounters.testMapReduceJob is flakey
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-5598:
-
Status: Patch Available (was: Open)
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 2.2.0, trunk
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-5598.patch
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-5598:
-
Attachment: MAPREDUCE-5598.patch
The patch modifies the input directory path to include the name of the test (so
its less generic and less likely to be used by another test). It also deletes
the input directory instead of just the input file.
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: trunk, 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-5598.patch
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807263#comment-13807263 ] Jonathan Eagles commented on MAPREDUCE-3860: Mac errors are due to java_home not being set. However, linux with jdk 1.7.0_40 still fails consistently even with the new patch. Will try to attach more logs with details. [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: linux-surefire-reports.tar, mac-surfire-reports.tar, MAPREDUCE-3860--n2.patch, MAPREDUCE-3860--n3.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807283#comment-13807283
]
Hadoop QA commented on MAPREDUCE-5598:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12610688/MAPREDUCE-5598.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4151//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4151//console
This message is automatically generated.
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: trunk, 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-5598.patch
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-5598:
-
Attachment: MAPREDUCE-5598.patch
Reuploading same patch to kick off another build. There was some kind of
memory problem starting the JVM
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: trunk, 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-5598.patch, MAPREDUCE-5598.patch
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5598) TestUserDefinedCounters.testMapReduceJob is flakey
[
https://issues.apache.org/jira/browse/MAPREDUCE-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807411#comment-13807411
]
Hadoop QA commented on MAPREDUCE-5598:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12610698/MAPREDUCE-5598.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4152//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4152//console
This message is automatically generated.
TestUserDefinedCounters.testMapReduceJob is flakey
--
Key: MAPREDUCE-5598
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5598
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: trunk, 2.2.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Attachments: MAPREDUCE-5598.patch, MAPREDUCE-5598.patch
{{TestUserDefinedCounters.testMapReduceJob}} is flakey.
We sometimes see it fail:
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertTrue(Assert.java:27)
at
org.apache.hadoop.mapred.TestUserDefinedCounters.testMapReduceJob(TestUserDefinedCounters.java:113)
{noformat}
Upon investigation, the problem is that the input for the MR job in this test
is at {{System.getProperty(test.build.data, /tmp) + /input}}. If an
earlier test wrote some files there, this test will use them as part of its
input. This can cause all sorts of problems with this test because its not
expecting the additional input data.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (MAPREDUCE-5599) JobHistoryServer unnecessarily copies all jobs on each query
Sandy Ryza created MAPREDUCE-5599: - Summary: JobHistoryServer unnecessarily copies all jobs on each query Key: MAPREDUCE-5599 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5599 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Instead, in CachedHistoryStorage, we should only copy jobs that will be returned. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-4680: -- Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Robert! Job history cleaner should only check timestamps of files in old enough directories --- Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.0.0-alpha Reporter: Sandy Ryza Assignee: Robert Kanter Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch Job history files are stored in /mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-4680: -- Fix Version/s: 2.2.1 Hadoop Flags: Reviewed Job history cleaner should only check timestamps of files in old enough directories --- Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.0.0-alpha Reporter: Sandy Ryza Assignee: Robert Kanter Fix For: 2.2.1 Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch Job history files are stored in /mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4680) Job history cleaner should only check timestamps of files in old enough directories
[ https://issues.apache.org/jira/browse/MAPREDUCE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807514#comment-13807514 ] Hudson commented on MAPREDUCE-4680: --- SUCCESS: Integrated in Hadoop-trunk-Commit #4663 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4663/]) MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1536558) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestJobHistoryUtils.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistory.java Job history cleaner should only check timestamps of files in old enough directories --- Key: MAPREDUCE-4680 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4680 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.0.0-alpha Reporter: Sandy Ryza Assignee: Robert Kanter Fix For: 2.2.1 Attachments: MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch, MAPREDUCE-4680.patch Job history files are stored in /mm/dd folders. Currently, the job history cleaner checks the modification date of each file in every one of these folders to see whether it's past the maximum age. The load on HDFS could be reduced by only checking the ages of files in directories that are old enough, as determined by their name. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5392) mapred job -history all command throws IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/MAPREDUCE-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807531#comment-13807531
]
Zhijie Shen commented on MAPREDUCE-5392:
bq. When I use JHAdminConfig, a package-dependent problem occurs in Maven
Gosh, it may result in cyclic dependency. However, it seems not good to have
two constants that actually point to the same property. How about moving
JHAdminConfig to core project? Not sure about it.
BTW, the test case failure seems to be unrelated. See
{code}
java.net.BindException: Address already in use; For more details see:
http://wiki.apache.org/hadoop/BindException
{code}
Looks like some conflicts in the test environment.
mapred job -history all command throws IndexOutOfBoundsException
--
Key: MAPREDUCE-5392
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5392
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0, 2.0.5-alpha, 2.2.0
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Minor
Fix For: 3.0.0
Attachments: MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch, MAPREDUCE-5392.patch, MAPREDUCE-5392.patch,
MAPREDUCE-5392.patch
When I use an all option by mapred job -history comamnd, the following
exceptions are displayed and do not work.
{code}
Exception in thread main java.lang.StringIndexOutOfBoundsException: String
index out of range: -3
at java.lang.String.substring(String.java:1875)
at
org.apache.hadoop.mapreduce.util.HostUtil.convertTrackerNameToHostName(HostUtil.java:49)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.getTaskLogsUrl(HistoryViewer.java:459)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.printAllTaskAttempts(HistoryViewer.java:235)
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer.print(HistoryViewer.java:117)
at org.apache.hadoop.mapreduce.tools.CLI.viewHistory(CLI.java:472)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:313)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1233)
{code}
This is because a node name recorded in History file is not given tracker_.
Therefore it makes modifications to be able to read History file even if a
node name is not given by tracker_.
In addition, it fixes the URL of displayed task log.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807668#comment-13807668 ] Jonathan Eagles commented on MAPREDUCE-3860: Part of the issue is that the individual tests can't run independently. For example, the following three tests always fail when run by themselves. This type of failure is problematic for jdk 7 since test order is random. This is not an exhaustive list and there are more than this. mvn clean test -Dtest=TestRumenJobTraces#testHadoop20JHParser mvn clean test -Dtest=TestRumenJobTraces#testResourceUsageMetricsWithHadoopLogsAnalyzer mvn clean test -Dtest=TestRumenJobTraces#testTopologyBuilder [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: linux-surefire-reports.tar, mac-surfire-reports.tar, MAPREDUCE-3860--n2.patch, MAPREDUCE-3860--n3.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807680#comment-13807680 ] Jonathan Eagles commented on MAPREDUCE-3860: Even when running the above three tests with jdk6, I am able to make the tests fail. The seem to be assuming some setup not present in the test. [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: linux-surefire-reports.tar, mac-surfire-reports.tar, MAPREDUCE-3860--n2.patch, MAPREDUCE-3860--n3.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807693#comment-13807693 ] Andrey Klochkov commented on MAPREDUCE-3860: Jonathan, thanks for testing this. Can you please attach surefire logs? I'm still missing a possible reason for the failures you see. I just tried to run the 3 commands you mentioned, and all three passed on my osx with jdk7. I'm not trying to use works for me argument, but I can't reproduce this, so logs would be really helpful. [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: linux-surefire-reports.tar, mac-surfire-reports.tar, MAPREDUCE-3860--n2.patch, MAPREDUCE-3860--n3.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
