[jira] [Created] (MAPREDUCE-6380) AggregatedLogDeletionService will throw exception when there are some other directories in remote-log-dir
Zhang Wei created MAPREDUCE-6380: Summary: AggregatedLogDeletionService will throw exception when there are some other directories in remote-log-dir Key: MAPREDUCE-6380 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6380 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Reporter: Zhang Wei Priority: Trivial AggregatedLogDeletionService will throw FileNotFoundException when there are some extraneous directories put in remote-log-dir. The deletion function will try to listStatus against the extraneous-dir + suffix dir. I think it would be better if the function can ignore these directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6379) @taskid@ in yarn.app.mapreduce.am.command-opts can't be replaced by current taskid
[ https://issues.apache.org/jira/browse/MAPREDUCE-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena reassigned MAPREDUCE-6379: --- Assignee: Varun Saxena @taskid@ in yarn.app.mapreduce.am.command-opts can't be replaced by current taskid -- Key: MAPREDUCE-6379 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6379 Project: Hadoop Map/Reduce Issue Type: Bug Components: mr-am Affects Versions: 2.7.0 Reporter: Zhang Wei Assignee: Varun Saxena Priority: Minor In this document, http://hadoop.apache.org/docs/r2.7.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml, we can find the parameter yarn.app.mapreduce.am.command-opts is described as @taskid@ in yarn.app.mapreduce.am.command-opts can be replaced by current taskid. But I checked the code in 2.7.0, there is no logic to replace it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[ https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated MAPREDUCE-6174: -- Status: Patch Available (was: Open) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. --- Key: MAPREDUCE-6174 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.6.0, 3.0.0 Reporter: Eric Payne Assignee: Eric Payne Labels: BB2015-05-RFC Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch, MAPREDUCE-6174.004.patch, MAPREDUCE-6174.005.patch, MAPREDUCE-6174.006.patch, MAPREDUCE-6174.v1.txt Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing similar things with regards to IFile streams. In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are different from 3rd-party implementations, this JIRA will make them subclass a common class (see https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6381) Some of MapReduce Commands opreations should have audit log printed
Bob created MAPREDUCE-6381: -- Summary: Some of MapReduce Commands opreations should have audit log printed Key: MAPREDUCE-6381 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6381 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.7.0 Reporter: Bob Below mapred commands are important operations that should also have audit logs recorded like 'yarn' commands. Mapred commands: mapred job -set-priority job-id priority mapred job -kill-task task-attempt-id mapred job -fail-task task-attempt-id mapred job -kill job-id mapred pipes mapred job -submit job-file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6381) Some of MapReduce Commands opreations should have audit log printed
[
https://issues.apache.org/jira/browse/MAPREDUCE-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bob updated MAPREDUCE-6381:
---
Description:
Below mapred commands are important operations that should also have audit logs
recorded like 'yarn' commands.
*Mapred commands:*
{noformat}
mapred job -set-priority job-id priority
mapred job -kill-task task-attempt-id
mapred job -fail-task task-attempt-id
mapred job -kill job-id
mapred pipes
mapred job -submit job-file
{noformat}
was:
Below mapred commands are important operations that should also have audit logs
recorded like 'yarn' commands.
Mapred commands:
mapred job -set-priority job-id priority
mapred job -kill-task task-attempt-id
mapred job -fail-task task-attempt-id
mapred job -kill job-id
mapred pipes
mapred job -submit job-file
Some of MapReduce Commands opreations should have audit log printed
---
Key: MAPREDUCE-6381
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6381
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: client
Affects Versions: 2.7.0
Reporter: Bob
Below mapred commands are important operations that should also have audit
logs recorded like 'yarn' commands.
*Mapred commands:*
{noformat}
mapred job -set-priority job-id priority
mapred job -kill-task task-attempt-id
mapred job -fail-task task-attempt-id
mapred job -kill job-id
mapred pipes
mapred job -submit job-file
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[ https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6377: - Attachment: (was: MAPREDUCE-6377.000.patch) JHS sorting on state column not working in webUi Key: MAPREDUCE-6377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.7.0 Environment: 2 NM, JHS Reporter: Bibin A Chundatt Assignee: zhihai xu Priority: Minor Attachments: Sorting Issue.png, state_sorted1.pdf, state_sorted2.pdf Steps to reproduce 1. Install and setup HA cluster with JHS 2.Create state in in JHS where few jobs are killed and Success Check sorting State in JHS WebUI Actual = Sorting on state column not working in JHS Expected == Sorting on state column should be working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567430#comment-14567430
]
Eric Payne commented on MAPREDUCE-6174:
---
The checkstyle warning is because the new {{OnDiskMapOutput}} constructor has 8
parameters instead of the checkstyle-approved 7. The deprecated constructor had
10, so my contention is that the new one is going in the right direction.
Combine common stream code into parent class for InMemoryMapOutput and
OnDiskMapOutput.
---
Key: MAPREDUCE-6174
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 3.0.0, 2.6.0
Reporter: Eric Payne
Assignee: Eric Payne
Labels: BB2015-05-RFC
Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch,
MAPREDUCE-6174.004.patch, MAPREDUCE-6174.005.patch, MAPREDUCE-6174.006.patch,
MAPREDUCE-6174.v1.txt
Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing
similar things with regards to IFile streams.
In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are
different from 3rd-party implementations, this JIRA will make them subclass a
common class (see
https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[ https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6377: - Attachment: MAPREDUCE-6377.000.patch JHS sorting on state column not working in webUi Key: MAPREDUCE-6377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.7.0 Environment: 2 NM, JHS Reporter: Bibin A Chundatt Assignee: zhihai xu Priority: Minor Attachments: MAPREDUCE-6377.000.patch, Sorting Issue.png, state_sorted1.pdf, state_sorted2.pdf Steps to reproduce 1. Install and setup HA cluster with JHS 2.Create state in in JHS where few jobs are killed and Success Check sorting State in JHS WebUI Actual = Sorting on state column not working in JHS Expected == Sorting on state column should be working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567285#comment-14567285
]
Hadoop QA commented on MAPREDUCE-6174:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 15s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 50s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 47s | The applied patch generated 1
new checkstyle issues (total was 89, now 88). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 22s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 1m 38s | Tests passed in
hadoop-mapreduce-client-core. |
| | | 40m 6s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736389/MAPREDUCE-6174.006.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 63e3fee |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5760/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
|
| hadoop-mapreduce-client-core test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5760/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5760/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5760/console |
This message was automatically generated.
Combine common stream code into parent class for InMemoryMapOutput and
OnDiskMapOutput.
---
Key: MAPREDUCE-6174
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 3.0.0, 2.6.0
Reporter: Eric Payne
Assignee: Eric Payne
Labels: BB2015-05-RFC
Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch,
MAPREDUCE-6174.004.patch, MAPREDUCE-6174.005.patch, MAPREDUCE-6174.006.patch,
MAPREDUCE-6174.v1.txt
Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing
similar things with regards to IFile streams.
In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are
different from 3rd-party implementations, this JIRA will make them subclass a
common class (see
https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[
https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567614#comment-14567614
]
Hadoop QA commented on MAPREDUCE-6377:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 22s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 28s | The applied patch generated 1
new checkstyle issues (total was 4, now 4). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 51s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 5m 52s | Tests passed in
hadoop-mapreduce-client-hs. |
| | | 42m 12s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736603/MAPREDUCE-6377.000.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 63e3fee |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-hs.txt
|
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5761/console |
This message was automatically generated.
JHS sorting on state column not working in webUi
Key: MAPREDUCE-6377
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: 2.7.0
Environment: 2 NM, JHS
Reporter: Bibin A Chundatt
Assignee: zhihai xu
Priority: Minor
Attachments: MAPREDUCE-6377.000.patch, Sorting Issue.png,
state_sorted1.pdf, state_sorted2.pdf
Steps to reproduce
1. Install and setup HA cluster with JHS
2.Create state in in JHS where few jobs are killed and Success
Check sorting State in JHS WebUI
Actual
=
Sorting on state column not working in JHS
Expected
==
Sorting on state column should be working
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[ https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6377: - Attachment: (was: MAPREDUCE-6377.000.patch) JHS sorting on state column not working in webUi Key: MAPREDUCE-6377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.7.0 Environment: 2 NM, JHS Reporter: Bibin A Chundatt Assignee: zhihai xu Priority: Minor Attachments: Sorting Issue.png, state_sorted1.pdf, state_sorted2.pdf Steps to reproduce 1. Install and setup HA cluster with JHS 2.Create state in in JHS where few jobs are killed and Success Check sorting State in JHS WebUI Actual = Sorting on state column not working in JHS Expected == Sorting on state column should be working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[ https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhihai xu updated MAPREDUCE-6377: - Attachment: MAPREDUCE-6377.000.patch JHS sorting on state column not working in webUi Key: MAPREDUCE-6377 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 2.7.0 Environment: 2 NM, JHS Reporter: Bibin A Chundatt Assignee: zhihai xu Priority: Minor Attachments: MAPREDUCE-6377.000.patch, Sorting Issue.png, state_sorted1.pdf, state_sorted2.pdf Steps to reproduce 1. Install and setup HA cluster with JHS 2.Create state in in JHS where few jobs are killed and Success Check sorting State in JHS WebUI Actual = Sorting on state column not working in JHS Expected == Sorting on state column should be working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567812#comment-14567812
]
Gera Shegalov commented on MAPREDUCE-6174:
--
Hi [~eepayne], thanks for 006. Agreed regarding checkstyle.
I am sorry, I have one remaining tiny nit before committing. The second
deprecated constructor should also delegate for clarity:
{code}
@Deprecated
@VisibleForTesting
OnDiskMapOutput(TaskAttemptID mapId, TaskAttemptID reduceId,
MergeManagerImplK,V merger, long size,
JobConf conf,
MapOutputFile mapOutputFile,
int fetcher, boolean primaryMapOutput,
FileSystem fs, Path outputPath) throws IOException {
this(mapId, merger, size, conf, fetcher, primaryMapOutput, fs, outputPath);
}
{code}
Combine common stream code into parent class for InMemoryMapOutput and
OnDiskMapOutput.
---
Key: MAPREDUCE-6174
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 3.0.0, 2.6.0
Reporter: Eric Payne
Assignee: Eric Payne
Labels: BB2015-05-RFC
Attachments: MAPREDUCE-6174.002.patch, MAPREDUCE-6174.003.patch,
MAPREDUCE-6174.004.patch, MAPREDUCE-6174.005.patch, MAPREDUCE-6174.006.patch,
MAPREDUCE-6174.v1.txt
Per MAPREDUCE-6166, both InMemoryMapOutput and OnDiskMapOutput will be doing
similar things with regards to IFile streams.
In order to make it explicit that InMemoryMapOutput and OnDiskMapOutput are
different from 3rd-party implementations, this JIRA will make them subclass a
common class (see
https://issues.apache.org/jira/browse/MAPREDUCE-6166?focusedCommentId=14223368page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14223368)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6377) JHS sorting on state column not working in webUi
[
https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14567792#comment-14567792
]
Hadoop QA commented on MAPREDUCE-6377:
--
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 42s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 45s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 49s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 28s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 0m 50s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 5m 56s | Tests passed in
hadoop-mapreduce-client-hs. |
| | | 43m 2s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12736617/MAPREDUCE-6377.000.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 63e3fee |
| hadoop-mapreduce-client-hs test log |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/artifact/patchprocess/testrun_hadoop-mapreduce-client-hs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf908.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5762/console |
This message was automatically generated.
JHS sorting on state column not working in webUi
Key: MAPREDUCE-6377
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: 2.7.0
Environment: 2 NM, JHS
Reporter: Bibin A Chundatt
Assignee: zhihai xu
Priority: Minor
Attachments: MAPREDUCE-6377.000.patch, Sorting Issue.png,
state_sorted1.pdf, state_sorted2.pdf
Steps to reproduce
1. Install and setup HA cluster with JHS
2.Create state in in JHS where few jobs are killed and Success
Check sorting State in JHS WebUI
Actual
=
Sorting on state column not working in JHS
Expected
==
Sorting on state column should be working
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-1380) Adaptive Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568572#comment-14568572 ] Jordà Polo commented on MAPREDUCE-1380: --- It's not located in the yarn package because it predates yarn. This patch worked with older versions of Hadoop, but it won't work against newer versions, at least not as it is. Adaptive Scheduler -- Key: MAPREDUCE-1380 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1380 Project: Hadoop Map/Reduce Issue Type: New Feature Affects Versions: 2.4.1 Reporter: Jordà Polo Assignee: Jordà Polo Priority: Minor Attachments: MAPREDUCE-1380-branch-1.2.patch, MAPREDUCE-1380_0.1.patch, MAPREDUCE-1380_1.1.patch, MAPREDUCE-1380_1.1.pdf The Adaptive Scheduler is a pluggable Hadoop scheduler that automatically adjusts the amount of used resources depending on the performance of jobs and on user-defined high-level business goals. Existing Hadoop schedulers are focused on managing large, static clusters in which nodes are added or removed manually. On the other hand, the goal of this scheduler is to improve the integration of Hadoop and the applications that run on top of it with environments that allow a more dynamic provisioning of resources. The current implementation is quite straightforward. Users specify a deadline at job submission time, and the scheduler adjusts the resources to meet that deadline (at the moment, the scheduler can be configured to either minimize or maximize the amount of resources). If multiple jobs are run simultaneously, the scheduler prioritizes them by deadline. Note that the current approach to estimate the completion time of jobs is quite simplistic: it is based on the time it takes to finish each task, so it works well with regular jobs, but there is still room for improvement for unpredictable jobs. The idea is to further integrate it with cloud-like and virtual environments (such as Amazon EC2, Emotive, etc.) so that if, for instance, a job isn't able to meet its deadline, the scheduler automatically requests more resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-1380) Adaptive Scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568577#comment-14568577 ] ericson yang commented on MAPREDUCE-1380: - Is there any codes correspond to the yarn? I want to alter this scheduler to the yarn package. Would you please give me some suggestions? Adaptive Scheduler -- Key: MAPREDUCE-1380 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1380 Project: Hadoop Map/Reduce Issue Type: New Feature Affects Versions: 2.4.1 Reporter: Jordà Polo Assignee: Jordà Polo Priority: Minor Attachments: MAPREDUCE-1380-branch-1.2.patch, MAPREDUCE-1380_0.1.patch, MAPREDUCE-1380_1.1.patch, MAPREDUCE-1380_1.1.pdf The Adaptive Scheduler is a pluggable Hadoop scheduler that automatically adjusts the amount of used resources depending on the performance of jobs and on user-defined high-level business goals. Existing Hadoop schedulers are focused on managing large, static clusters in which nodes are added or removed manually. On the other hand, the goal of this scheduler is to improve the integration of Hadoop and the applications that run on top of it with environments that allow a more dynamic provisioning of resources. The current implementation is quite straightforward. Users specify a deadline at job submission time, and the scheduler adjusts the resources to meet that deadline (at the moment, the scheduler can be configured to either minimize or maximize the amount of resources). If multiple jobs are run simultaneously, the scheduler prioritizes them by deadline. Note that the current approach to estimate the completion time of jobs is quite simplistic: it is based on the time it takes to finish each task, so it works well with regular jobs, but there is still room for improvement for unpredictable jobs. The idea is to further integrate it with cloud-like and virtual environments (such as Amazon EC2, Emotive, etc.) so that if, for instance, a job isn't able to meet its deadline, the scheduler automatically requests more resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
