[jira] [Commented] (YARN-286) Add a YARN ApplicationClassLoader
[ https://issues.apache.org/jira/browse/YARN-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542060#comment-13542060 ] Tom White commented on YARN-286: The patch is needed by MAPREDUCE-1700 for class isolation in MR tasks. Add a YARN ApplicationClassLoader - Key: YARN-286 URL: https://issues.apache.org/jira/browse/YARN-286 Project: Hadoop YARN Issue Type: New Feature Components: applications Affects Versions: 2.0.2-alpha Reporter: Tom White Assignee: Tom White Fix For: 2.0.3-alpha Attachments: YARN-286.patch Add a classloader that provides webapp-style class isolation for use by applications. This is the YARN part of MAPREDUCE-1700 (which was already developed in that JIRA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-304) RM Tracking Links for purged applications needs a long-term solution
Derek Dagit created YARN-304: Summary: RM Tracking Links for purged applications needs a long-term solution Key: YARN-304 URL: https://issues.apache.org/jira/browse/YARN-304 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 0.23.5, 3.0.0 Reporter: Derek Dagit This JIRA is intended to track a proper long-term fix for the issue described in YARN-285. The following is from the original description: As applications complete, the RM tracks their IDs in a completed list. This list is routinely truncated to limit the total number of application remembered by the RM. When a user clicks the History for a job, either the browser is redirected to the application's tracking link obtained from the stored application instance. But when the application has been purged from the RM, an error is displayed. In very busy clusters the rate at which applications complete can cause applications to be purged from the RM's internal list within hours, which breaks the proxy URLs users have saved for their jobs. We would like the RM to provide valid tracking links persist so that users are not frustrated by broken links. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-205) yarn logs throws nullpointerexception for running application
[ https://issues.apache.org/jira/browse/YARN-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542204#comment-13542204 ] Thomas Graves commented on YARN-205: This was happening on any application I just started and was running. I was just running wordcount at the time. What version are you using? I haven't tried it on trunk or branch-2. yarn logs throws nullpointerexception for running application - Key: YARN-205 URL: https://issues.apache.org/jira/browse/YARN-205 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 0.23.4 Reporter: Thomas Graves I started an application and tried to look at the task logs for the takss that had finished via yarn logs. Unfortunately it throws a null pointer exception until the job finished: yarn logs -appOwner tgraves --applicationId application_1352316570518_0002 Exception in thread main java.lang.NullPointerException at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:573) at java.io.DataInputStream.readFully(DataInputStream.java:178) at java.io.DataInputStream.readLong(DataInputStream.java:399) at org.apache.hadoop.io.file.tfile.BCFile$Reader.init(BCFile.java:623) at org.apache.hadoop.io.file.tfile.TFile$Reader.init(TFile.java:803) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.init(AggregatedLogFormat.java:293) at org.apache.hadoop.yarn.logaggregation.LogDumper.dumpAllContainersLogs(LogDumper.java:209) at org.apache.hadoop.yarn.logaggregation.LogDumper.run(LogDumper.java:109) at org.apache.hadoop.yarn.logaggregation.LogDumper.main(LogDumper.java:242) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-205) yarn logs throws nullpointerexception for running application
[ https://issues.apache.org/jira/browse/YARN-205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542209#comment-13542209 ] Jason Lowe commented on YARN-205: - I was not able to reproduce this on a recent 0.23 build. The yarn logs command simply returned nothing while the application was running. After the job finished the same command dumped the logs. yarn logs throws nullpointerexception for running application - Key: YARN-205 URL: https://issues.apache.org/jira/browse/YARN-205 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 0.23.4 Reporter: Thomas Graves I started an application and tried to look at the task logs for the takss that had finished via yarn logs. Unfortunately it throws a null pointer exception until the job finished: yarn logs -appOwner tgraves --applicationId application_1352316570518_0002 Exception in thread main java.lang.NullPointerException at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:573) at java.io.DataInputStream.readFully(DataInputStream.java:178) at java.io.DataInputStream.readLong(DataInputStream.java:399) at org.apache.hadoop.io.file.tfile.BCFile$Reader.init(BCFile.java:623) at org.apache.hadoop.io.file.tfile.TFile$Reader.init(TFile.java:803) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.init(AggregatedLogFormat.java:293) at org.apache.hadoop.yarn.logaggregation.LogDumper.dumpAllContainersLogs(LogDumper.java:209) at org.apache.hadoop.yarn.logaggregation.LogDumper.run(LogDumper.java:109) at org.apache.hadoop.yarn.logaggregation.LogDumper.main(LogDumper.java:242) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-291) Dynamic resource configuration on NM
[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542259#comment-13542259 ] Bikas Saha commented on YARN-291: - Junping, it would be helpful if you attach the patches all together when you are looking for review comments. And leave a comment saying how to go about reviewing the patches. eg. order in which to read the patches and what they contain. If there is 1-1 mapping between the doc and your patches then the doc is sufficient. If not, then a comment explaining the patches would be great. Dynamic resource configuration on NM Key: YARN-291 URL: https://issues.apache.org/jira/browse/YARN-291 Project: Hadoop YARN Issue Type: New Feature Components: nodemanager, scheduler Reporter: Junping Du Assignee: Junping Du Labels: features Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, YARN-291-YARNClientCommandline-04.patch The current Hadoop YARN resource management logic assumes per node resource is static during the lifetime of the NM process. Allowing run-time configuration on per node resource will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized. About more background and design details, please refer: HADOOP-9165. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-300) After YARN-271, fair scheduler can infinite loop and not schedule any application.
[ https://issues.apache.org/jira/browse/YARN-300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-300: Summary: After YARN-271, fair scheduler can infinite loop and not schedule any application. (was: After yarn-271, when yarn.scheduler.fair.max.assign=0, fairscheduler will infinite loop and not schedule any application.) After YARN-271, fair scheduler can infinite loop and not schedule any application. -- Key: YARN-300 URL: https://issues.apache.org/jira/browse/YARN-300 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Reporter: shenhong Labels: None Fix For: 2.0.3-alpha Attachments: YARN-300.patch Original Estimate: 10h Remaining Estimate: 10h After yarn-271, when yarn.scheduler.fair.max.assign=0, when a node was been reserved, fairScheduler will infinite loop and not schedule any application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-300) After YARN-271, fair scheduler can infinite loop and not schedule any application.
[ https://issues.apache.org/jira/browse/YARN-300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-300: Attachment: YARN-300.patch After YARN-271, fair scheduler can infinite loop and not schedule any application. -- Key: YARN-300 URL: https://issues.apache.org/jira/browse/YARN-300 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Reporter: shenhong Labels: None Fix For: 2.0.3-alpha Attachments: YARN-300.patch, YARN-300.patch Original Estimate: 10h Remaining Estimate: 10h After yarn-271, when yarn.scheduler.fair.max.assign=0, when a node was been reserved, fairScheduler will infinite loop and not schedule any application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-291) Dynamic resource configuration on NM
[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542297#comment-13542297 ] Luke Lu commented on YARN-291: -- A simpler and more straightforward alternative would be changing the node resource on RM directly via JMX. No protocol changes would be necessary. Changing resource on NM and propagating the change to RM would lead to nasty race conditions where RM still thinks that an NM has enough resource and schedule new containers on the already downsized NM, which should fail and RM would need to reschedule the containers elsewhere, which is not pretty, especially when large number of NMs are affected, even if all the corner cases are handled. bq. we should do RPC and optionally the web-services to the NM directly. And we should add command line util too. Since the only near term user of the feature is *external* admin processes that prefer not having Hadoop jar dependencies, JMX change alone (a much smaller patch that doesn't impact major production code paths) would suffice. YARN RPC changes can be in a separate JIRA, when YARN itself needs to change per node resources. I'm not sure how useful a YARN CLI interface for this is, given it's impractical for people to manually change per node resources (when number of nodes is greater than 10). OTOH, I'm fine with having everything in, as long as it doesn't delay the inclusion of the basic functionality. Dynamic resource configuration on NM Key: YARN-291 URL: https://issues.apache.org/jira/browse/YARN-291 Project: Hadoop YARN Issue Type: New Feature Components: nodemanager, scheduler Reporter: Junping Du Assignee: Junping Du Labels: features Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, YARN-291-YARNClientCommandline-04.patch The current Hadoop YARN resource management logic assumes per node resource is static during the lifetime of the NM process. Allowing run-time configuration on per node resource will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized. About more background and design details, please refer: HADOOP-9165. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-300) After YARN-271, fair scheduler can infinite loop and not schedule any application.
[ https://issues.apache.org/jira/browse/YARN-300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542298#comment-13542298 ] Sandy Ryza commented on YARN-300: - For YARN-271, Alejandro recommended that it would look best to add the check as the while loop's condition. Uploaded a patch that does that. After YARN-271, fair scheduler can infinite loop and not schedule any application. -- Key: YARN-300 URL: https://issues.apache.org/jira/browse/YARN-300 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Reporter: shenhong Labels: None Fix For: 2.0.3-alpha Attachments: YARN-300.patch, YARN-300.patch Original Estimate: 10h Remaining Estimate: 10h After yarn-271, when yarn.scheduler.fair.max.assign=0, when a node was been reserved, fairScheduler will infinite loop and not schedule any application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-302) Fair scheduler assignmultiple should default to false
[ https://issues.apache.org/jira/browse/YARN-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-302: Attachment: YARN-302.patch Fair scheduler assignmultiple should default to false - Key: YARN-302 URL: https://issues.apache.org/jira/browse/YARN-302 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.2-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-302.patch The MR1 default was false. When true, it results in overloading some machines with many tasks and underutilizing others. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-300) After YARN-271, fair scheduler can infinite loop and not schedule any application.
[
https://issues.apache.org/jira/browse/YARN-300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542320#comment-13542320
]
Hadoop QA commented on YARN-300:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562938/YARN-300.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/268//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/268//console
This message is automatically generated.
After YARN-271, fair scheduler can infinite loop and not schedule any
application.
--
Key: YARN-300
URL: https://issues.apache.org/jira/browse/YARN-300
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager, scheduler
Reporter: shenhong
Labels: None
Fix For: 2.0.3-alpha
Attachments: YARN-300.patch, YARN-300.patch
Original Estimate: 10h
Remaining Estimate: 10h
After yarn-271, when yarn.scheduler.fair.max.assign=0, when a node was been
reserved, fairScheduler will infinite loop and not schedule any application.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-300) After YARN-271, fair scheduler can infinite loop and not schedule any application.
[ https://issues.apache.org/jira/browse/YARN-300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542327#comment-13542327 ] Sandy Ryza commented on YARN-300: - Failing core tests are unrelated. After YARN-271, fair scheduler can infinite loop and not schedule any application. -- Key: YARN-300 URL: https://issues.apache.org/jira/browse/YARN-300 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Reporter: shenhong Labels: None Fix For: 2.0.3-alpha Attachments: YARN-300.patch, YARN-300.patch Original Estimate: 10h Remaining Estimate: 10h After yarn-271, when yarn.scheduler.fair.max.assign=0, when a node was been reserved, fairScheduler will infinite loop and not schedule any application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-301) Fair scheduler throws ConcurrentModificationException when iterating over app's priorities
[
https://issues.apache.org/jira/browse/YARN-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542339#comment-13542339
]
Hadoop QA commented on YARN-301:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562875/YARN-301.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/270//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/270//console
This message is automatically generated.
Fair scheduler throws ConcurrentModificationException when iterating over
app's priorities
--
Key: YARN-301
URL: https://issues.apache.org/jira/browse/YARN-301
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager, scheduler
Affects Versions: 2.0.2-alpha
Reporter: shenhong
Fix For: 2.0.3-alpha
Attachments: YARN-301.patch, YARN-301.patch
In my test cluster, fairscheduler appear to concurrentModificationException
and RM crash, here is the message:
2012-12-30 17:14:17,171 FATAL
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.util.ConcurrentModificationException
at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1100)
at java.util.TreeMap$KeyIterator.next(TreeMap.java:1154)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable.assignContainer(AppSchedulable.java:297)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:181)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:780)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:842)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:98)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:340)
at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-142) Change YARN APIs to throw IOException
[ https://issues.apache.org/jira/browse/YARN-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-142: --- Attachment: YARN-142.2.patch Change YARN APIs to throw IOException - Key: YARN-142 URL: https://issues.apache.org/jira/browse/YARN-142 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 2.0.0-alpha Reporter: Siddharth Seth Assignee: Xuan Gong Priority: Critical Attachments: YARN-142.1.patch, YARN-142.2.patch Ref: MAPREDUCE-4067 All YARN APIs currently throw YarnRemoteException. 1) This cannot be extended in it's current form. 2) The RPC layer can throw IOExceptions. These end up showing up as UndeclaredThrowableExceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-297) Improve hashCode implementations for PB records
[ https://issues.apache.org/jira/browse/YARN-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542375#comment-13542375 ] Xuan Gong commented on YARN-297: So, those java files use small primes,31, and seed value as 1 to generate the hash value: ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationAttemptId.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ContainerId.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/NodeId.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Priority.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java ./hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceRequest.java Improve hashCode implementations for PB records --- Key: YARN-297 URL: https://issues.apache.org/jira/browse/YARN-297 Project: Hadoop YARN Issue Type: Improvement Reporter: Arun C Murthy Assignee: Xuan Gong As [~hsn] pointed out in YARN-2, we use very small primes in all our hashCode implementations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-288) Fair scheduler queue doesn't accept any jobs when ACLs are configured.
[ https://issues.apache.org/jira/browse/YARN-288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-288: Attachment: YARN-288.patch Fair scheduler queue doesn't accept any jobs when ACLs are configured. -- Key: YARN-288 URL: https://issues.apache.org/jira/browse/YARN-288 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.2-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 2.0.3-alpha Attachments: YARN-288.patch If a queue is configured with an ACL for who can submit jobs, no jobs are allowed, even if a user on the list tries. This is caused by using the scheduler thinking the user is yarn, because it calls UserGroupInformation.getCurrentUser() instead of UserGroupInformation.createRemoteUser() with the given user name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-142) Change YARN APIs to throw IOException
[ https://issues.apache.org/jira/browse/YARN-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-142: --- Attachment: YARN-142.2.patch Change YARN APIs to throw IOException - Key: YARN-142 URL: https://issues.apache.org/jira/browse/YARN-142 Project: Hadoop YARN Issue Type: Bug Affects Versions: 0.23.3, 2.0.0-alpha Reporter: Siddharth Seth Assignee: Xuan Gong Priority: Critical Attachments: YARN-142.1.patch, YARN-142.2.patch Ref: MAPREDUCE-4067 All YARN APIs currently throw YarnRemoteException. 1) This cannot be extended in it's current form. 2) The RPC layer can throw IOExceptions. These end up showing up as UndeclaredThrowableExceptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-288) Fair scheduler queue doesn't accept any jobs when ACLs are configured.
[ https://issues.apache.org/jira/browse/YARN-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542422#comment-13542422 ] Sandy Ryza commented on YARN-288: - Test failure is unrelated. Fair scheduler queue doesn't accept any jobs when ACLs are configured. -- Key: YARN-288 URL: https://issues.apache.org/jira/browse/YARN-288 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.2-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 2.0.3-alpha Attachments: YARN-288.patch If a queue is configured with an ACL for who can submit jobs, no jobs are allowed, even if a user on the list tries. This is caused by using the scheduler thinking the user is yarn, because it calls UserGroupInformation.getCurrentUser() instead of UserGroupInformation.createRemoteUser() with the given user name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-293) Node Manager leaks LocalizerRunner object for every Container
[ https://issues.apache.org/jira/browse/YARN-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542488#comment-13542488 ] Jason Lowe commented on YARN-293: - +1, lgtm. Node Manager leaks LocalizerRunner object for every Container -- Key: YARN-293 URL: https://issues.apache.org/jira/browse/YARN-293 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.0.2-alpha, 0.23.3, 2.0.1-alpha Reporter: Devaraj K Assignee: Robert Joseph Evans Priority: Critical Attachments: YARN-293-trunk.txt Node Manager creates a new LocalizerRunner object for every container and puts in ResourceLocalizationService.LocalizerTracker.privLocalizers map but it never removes from the map. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-297) Improve hashCode implementations for PB records
[ https://issues.apache.org/jira/browse/YARN-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542511#comment-13542511 ] Xuan Gong commented on YARN-297: Thanks. Could you kind educate me why we prefer to use bigger prime number in the hashCode function ? Improve hashCode implementations for PB records --- Key: YARN-297 URL: https://issues.apache.org/jira/browse/YARN-297 Project: Hadoop YARN Issue Type: Improvement Reporter: Arun C Murthy Assignee: Xuan Gong As [~hsn] pointed out in YARN-2, we use very small primes in all our hashCode implementations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-293) Node Manager leaks LocalizerRunner object for every Container
[ https://issues.apache.org/jira/browse/YARN-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542548#comment-13542548 ] Hudson commented on YARN-293: - Integrated in Hadoop-trunk-Commit #3163 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3163/]) YARN-293. Node Manager leaks LocalizerRunner object for every Container. Contributed by Robert Joseph Evans (Revision 1428095) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1428095 Files : * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/event/LocalizerEventType.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestResourceLocalizationService.java Node Manager leaks LocalizerRunner object for every Container -- Key: YARN-293 URL: https://issues.apache.org/jira/browse/YARN-293 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.0.2-alpha, 0.23.3, 2.0.1-alpha Reporter: Devaraj K Assignee: Robert Joseph Evans Priority: Critical Fix For: 2.0.3-alpha, 0.23.6 Attachments: YARN-293-trunk.txt Node Manager creates a new LocalizerRunner object for every container and puts in ResourceLocalizationService.LocalizerTracker.privLocalizers map but it never removes from the map. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-50) Implement renewal / cancellation of Delegation Tokens
[
https://issues.apache.org/jira/browse/YARN-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542553#comment-13542553
]
Hadoop QA commented on YARN-50:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562960/YARN50_with_MR4894_forjenkins_2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs
org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesTasks
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobsQuery
org.apache.hadoop.mapreduce.v2.TestNonExistentJob
org.apache.hadoop.mapred.TestMiniMRClientCluster
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs
org.apache.hadoop.mapred.TestJobCounters
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser
org.apache.hadoop.mapreduce.v2.TestRMNMInfo
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.mapred.TestMerge
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapreduce.TestChild
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapreduce.v2.TestMRJobs
org.apache.hadoop.mapreduce.v2.TestMRAppWithCombiner
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.mapreduce.security.TestMRCredentials
org.apache.hadoop.mapred.TestMiniMRBringup
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapred.TestBlockLimits
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapreduce.security.ssl.TestEncryptedShuffle
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapred.TestNetworkedJob
org.apache.hadoop.yarn.client.TestGetGroups
org.apache.hadoop.yarn.client.TestYarnClient
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/272//testReport/
[jira] [Moved] (YARN-306) Job Priority is not changing
[ https://issues.apache.org/jira/browse/YARN-306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla moved MAPREDUCE-4124 to YARN-306: -- Component/s: (was: mrv2) scheduler Affects Version/s: (was: 3.0.0) 2.0.2-alpha Key: YARN-306 (was: MAPREDUCE-4124) Project: Hadoop YARN (was: Hadoop Map/Reduce) Job Priority is not changing - Key: YARN-306 URL: https://issues.apache.org/jira/browse/YARN-306 Project: Hadoop YARN Issue Type: Bug Components: scheduler Affects Versions: 2.0.2-alpha Reporter: Nishan Shetty 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (YARN-306) Job Priority is not changing
[ https://issues.apache.org/jira/browse/YARN-306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla reassigned YARN-306: - Assignee: Karthik Kambatla Job Priority is not changing - Key: YARN-306 URL: https://issues.apache.org/jira/browse/YARN-306 Project: Hadoop YARN Issue Type: Bug Components: scheduler Affects Versions: 2.0.2-alpha Reporter: Nishan Shetty Assignee: Karthik Kambatla 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-306) FIFO scheduler doesn't respect changing job priority
[ https://issues.apache.org/jira/browse/YARN-306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-306: -- Summary: FIFO scheduler doesn't respect changing job priority (was: Job Priority is not changing ) FIFO scheduler doesn't respect changing job priority Key: YARN-306 URL: https://issues.apache.org/jira/browse/YARN-306 Project: Hadoop YARN Issue Type: Bug Components: scheduler Affects Versions: 2.0.2-alpha Reporter: Nishan Shetty Assignee: Karthik Kambatla 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-306) FIFO scheduler doesn't respect changing job priority
[ https://issues.apache.org/jira/browse/YARN-306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542607#comment-13542607 ] Karthik Kambatla commented on YARN-306: --- Changed the summary to reflect that this JIRA addresses only FIFO scheduler, per Bobby's suggestion. FIFO scheduler doesn't respect changing job priority Key: YARN-306 URL: https://issues.apache.org/jira/browse/YARN-306 Project: Hadoop YARN Issue Type: Bug Components: scheduler Affects Versions: 2.0.2-alpha Reporter: Nishan Shetty Assignee: Karthik Kambatla 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-307) NodeManager should log container launch command.
Lohit Vijayarenu created YARN-307: - Summary: NodeManager should log container launch command. Key: YARN-307 URL: https://issues.apache.org/jira/browse/YARN-307 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.0.3-alpha Reporter: Lohit Vijayarenu NodeManager's DefaultContainerExecutor seems to log only path of default container executor script instead of contents of script. It would be good to log the execution command so that one could see what is being launched. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-307) NodeManager should log container launch command.
[
https://issues.apache.org/jira/browse/YARN-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542616#comment-13542616
]
Lohit Vijayarenu commented on YARN-307:
---
For example I am seeing container launch failure without any useful message
like this.
{noformat}
2013-01-03 00:33:49,045 DEBUG
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node's
health-status : true,
2013-01-03 00:33:49,090 WARN
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code
from task is : 1
2013-01-03 00:33:49,090 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
{noformat}
Script seems to exit with exit code of 1. To debug further, I wanted to see the
command being execute, but in the logs I can see only the line as shown below
{noformat}
2013-01-03 00:33:46,591 INFO
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
launchContainer: [bash,
/data/disk2/yarn/local/usercache/hadoop/appcache/application_1357147147433_0011/container_1357147147433_0011_01_01/default_container_executor.sh]
{noformat}
Once task fails, this directory is cleaned up. There seems to be no easy way to
find out why container is failing. It would be good to log contents of
default_container_executor.sh along with the path.
NodeManager should log container launch command.
Key: YARN-307
URL: https://issues.apache.org/jira/browse/YARN-307
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.0.3-alpha
Reporter: Lohit Vijayarenu
NodeManager's DefaultContainerExecutor seems to log only path of default
container executor script instead of contents of script. It would be good to
log the execution command so that one could see what is being launched.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-297) Improve hashCode implementations for PB records
[ https://issues.apache.org/jira/browse/YARN-297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-297: --- Attachment: YARN.297.1.patch Improve hashCode implementations for PB records --- Key: YARN-297 URL: https://issues.apache.org/jira/browse/YARN-297 Project: Hadoop YARN Issue Type: Improvement Reporter: Arun C Murthy Assignee: Xuan Gong Attachments: YARN.297.1.patch As [~hsn] pointed out in YARN-2, we use very small primes in all our hashCode implementations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (YARN-308) Improve documentation about what asks means in AMRMProtocol
Sandy Ryza created YARN-308: --- Summary: Improve documentation about what asks means in AMRMProtocol Key: YARN-308 URL: https://issues.apache.org/jira/browse/YARN-308 Project: Hadoop YARN Issue Type: Bug Components: api, resourcemanager Affects Versions: 2.0.2-alpha Reporter: Sandy Ryza Fix For: 2.0.3-alpha It's unclear to me from reading the javadoc exactly what asks means when the AM sends a heartbeat to the RM. Is the AM supposed to send a list of all resources that it is waiting for? Or just inform the RM about new ones that it wants? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-223) Change processTree interface to work better with native code
[ https://issues.apache.org/jira/browse/YARN-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated YARN-223: --- Attachment: YARN-223-branch-trunk-win.1.patch branch-trunk-win diverged slightly due to committing YARN-234 ahead of this. We have a merge conflict now when trying to merge trunk to branch-trunk-win. I am attaching a separate patch for branch-trunk-win to resolve the merge conflict. Change processTree interface to work better with native code Key: YARN-223 URL: https://issues.apache.org/jira/browse/YARN-223 Project: Hadoop YARN Issue Type: Bug Reporter: Radim Kolar Assignee: Radim Kolar Priority: Critical Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: pstree-update4.txt, pstree-update6.txt, pstree-update6.txt, YARN-223-branch-trunk-win.1.patch Problem is that on every update of processTree new object is required. This is undesired when working with processTree implementation in native code. replace ProcessTree.getProcessTree() with updateProcessTree(). No new object allocation is needed and it simplify application code a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-223) Change processTree interface to work better with native code
[ https://issues.apache.org/jira/browse/YARN-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated YARN-223: --- Fix Version/s: trunk-win Change processTree interface to work better with native code Key: YARN-223 URL: https://issues.apache.org/jira/browse/YARN-223 Project: Hadoop YARN Issue Type: Bug Reporter: Radim Kolar Assignee: Radim Kolar Priority: Critical Fix For: 3.0.0, 2.0.3-alpha, trunk-win, 0.23.6 Attachments: pstree-update4.txt, pstree-update6.txt, pstree-update6.txt, YARN-223-branch-trunk-win.1.patch Problem is that on every update of processTree new object is required. This is undesired when working with processTree implementation in native code. replace ProcessTree.getProcessTree() with updateProcessTree(). No new object allocation is needed and it simplify application code a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-291) Dynamic resource configuration on NM
[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542665#comment-13542665 ] Junping Du commented on YARN-291: - Hi Bikas, Thanks for your comments. The biggest patch (YARN-291-all.patch) is the with all patches together. The doc I attached recently have implement part which have 1-1 mapping to sub patches (01, 01-fix, 02, 03, 04). Hope this will be helpful for review. Thanks! Dynamic resource configuration on NM Key: YARN-291 URL: https://issues.apache.org/jira/browse/YARN-291 Project: Hadoop YARN Issue Type: New Feature Components: nodemanager, scheduler Reporter: Junping Du Assignee: Junping Du Labels: features Attachments: Elastic Resources for YARN-v0.2.pdf, YARN-291-AddClientRMProtocolToSetNodeResource-03.patch, YARN-291-all-v1.patch, YARN-291-core-HeartBeatAndScheduler-01.patch, YARN-291-JMXInterfaceOnNM-02.patch, YARN-291-OnlyUpdateWhenResourceChange-01-fix.patch, YARN-291-YARNClientCommandline-04.patch The current Hadoop YARN resource management logic assumes per node resource is static during the lifetime of the NM process. Allowing run-time configuration on per node resource will give us finer granularity of resource elasticity. This allows Hadoop workloads to coexist with other workloads on the same hardware efficiently, whether or not the environment is virtualized. About more background and design details, please refer: HADOOP-9165. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-223) Change processTree interface to work better with native code
[ https://issues.apache.org/jira/browse/YARN-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542686#comment-13542686 ] Suresh Srinivas commented on YARN-223: -- Chris, thanks for the merged patch. It looks good. I will commit it to branch-trunk-win. Change processTree interface to work better with native code Key: YARN-223 URL: https://issues.apache.org/jira/browse/YARN-223 Project: Hadoop YARN Issue Type: Bug Reporter: Radim Kolar Assignee: Radim Kolar Priority: Critical Fix For: 3.0.0, 2.0.3-alpha, trunk-win, 0.23.6 Attachments: pstree-update4.txt, pstree-update6.txt, pstree-update6.txt, YARN-223-branch-trunk-win.1.patch Problem is that on every update of processTree new object is required. This is undesired when working with processTree implementation in native code. replace ProcessTree.getProcessTree() with updateProcessTree(). No new object allocation is needed and it simplify application code a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-296) Resource Manager throws InvalidStateTransitonException: Invalid event: APP_ACCEPTED at RUNNING for RMAppImpl
[
https://issues.apache.org/jira/browse/YARN-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542702#comment-13542702
]
Devaraj K commented on YARN-296:
It is not coming always, we have observed in the logs when we run some jobs.
Resource Manager throws InvalidStateTransitonException: Invalid event:
APP_ACCEPTED at RUNNING for RMAppImpl
Key: YARN-296
URL: https://issues.apache.org/jira/browse/YARN-296
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 2.0.2-alpha, 2.0.1-alpha
Reporter: Devaraj K
{code:xml}
2012-12-28 11:14:47,671 ERROR
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Can't handle
this event at current state
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event:
APP_ACCEPTED at RUNNING
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:301)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:443)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:528)
at
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.handle(RMAppImpl.java:72)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:405)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher.handle(ResourceManager.java:389)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:126)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:75)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
