from:"Jira"

[jira] [Updated] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-06-04 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

闫昆 updated YARN-514:


Component/s: (was: resourcemanager)

 Delayed store operations should not result in RM unavailability for app 
 submission
 --

 Key: YARN-514
 URL: https://issues.apache.org/jira/browse/YARN-514
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Bikas Saha
Assignee: Zhijie Shen
 Fix For: 2.1.0-beta

 Attachments: YARN-514.1.patch, YARN-514.2.patch, YARN-514.3.patch, 
 YARN-514.4.patch, YARN-514.5.patch, YARN-514.6.patch, YARN-514.7.patch, 
 YARN-514.8.patch


 Currently, app submission is the only store operation performed synchronously 
 because the app must be stored before the request returns with success. This 
 makes the RM susceptible to blocking all client threads on slow store 
 operations, resulting in RM being perceived as unavailable by clients.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-1205) Not able to run sleep job

2013-09-16 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13768885#comment-13768885
 ] 

André Kelpe commented on YARN-1205:
---

Shouldn't junit be included in the tests jar then, instead of having it on the 
cluster itself? Every other program brings the libs it needs with it. Why 
wouldn't that be the case here?

 Not able to run sleep job
 -

 Key: YARN-1205
 URL: https://issues.apache.org/jira/browse/YARN-1205
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.1.1-beta
Reporter: Omkar Vinit Joshi
Priority: Blocker

 Not able to run sleep job on latest trunk
 bin/yarn jar 
 ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT-tests.jar
 output :-
 Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
 java.lang.NoClassDefFoundError: junit/framework/TestCase
   at java.lang.ClassLoader.defineClass1(Native Method)
   at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
   at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
   at 
 java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
   at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
   at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
   at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
   at 
 org.apache.hadoop.test.MapredTestDriver.init(MapredTestDriver.java:60)
   at 
 org.apache.hadoop.test.MapredTestDriver.init(MapredTestDriver.java:54)
   at 
 org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
 Caused by: java.lang.ClassNotFoundException: junit.framework.TestCase
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
   ... 20 more
 An example program must be given as the first argument.
 Valid program names are:

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-1244) Missing yarn queue-cli

2014-08-31 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bartosz Ługowski updated YARN-1244:
---
Attachment: YARN-1244.1.patch

 Missing yarn queue-cli
 --

 Key: YARN-1244
 URL: https://issues.apache.org/jira/browse/YARN-1244
 Project: Hadoop YARN
  Issue Type: New Feature
Reporter: Vinod Kumar Vavilapalli
  Labels: newbie
 Attachments: YARN-1244.1.patch


 We don't have a yarn queue CLI. For now mapred still has one that is 
 working, but we need to move over that functionality to yarn CLI itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-16 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Attachment: YARN-126.patch

Patch with a GNU Readline based implementation of RMAdmin.java.

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 3.0.0
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-17 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Attachment: YARN-126.patch

Please find attached the patch with the Apache license header added.



 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 3.0.0
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch, YARN-126.patch, YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-18 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Attachment: (was: YARN-126.patch)

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 3.0.0
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-18 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Attachment: (was: YARN-126.patch)

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 3.0.0
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-19 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

 Target Version/s: 2.0.4-alpha  (was: 3.0.0)
Affects Version/s: (was: 3.0.0)
   2.0.3-alpha

2.0.3-alpha is affected. 
Not sure that it is my role to update these two fields, please let me know if I 
made a mistake.
Thanks.


 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-21 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Target Version/s: 3.0.0, 0.23.4, 2.0.4-alpha  (was: 2.0.4-alpha)

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-182) Unnecessary Container killed by the ApplicationMaster message for successful containers

2013-04-21 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637580#comment-13637580
 ] 

Rémy SAISSY commented on YARN-182:
--

I also ran into this usability issue. At first glance, the word killed made 
me think that something went wrong.
It would have been much more clear to me if I saw something like this:

Container finished by the ApplicationMaster

or when something was wrong:

Container killed by the ApplicationMaster


 Unnecessary Container killed by the ApplicationMaster message for 
 successful containers
 -

 Key: YARN-182
 URL: https://issues.apache.org/jira/browse/YARN-182
 Project: Hadoop YARN
  Issue Type: Bug
  Components: nodemanager
Affects Versions: 2.0.1-alpha
Reporter: zhengqiu cai
  Labels: hadoop, usability
 Attachments: Log.txt


 I was running wordcount and the resourcemanager web UI shown the status as 
 FINISHED SUCCEEDED, but the log shown Container killed by the 
 ApplicationMaster

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-23 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639513#comment-13639513
 ] 

Rémy SAISSY commented on YARN-126:
--

I removed the GenericOptionsParser because I saw that the yarn distributed 
shell ApplicationMaster.java relies on org.apache.commons.cli.GnuParser instead.
So I though that since an options parser is not something specific to Hadoop, 
it might be a good thing to remove the GenericOptionParser dependency where it 
is possible to favor the commons.cli library instead.
Isn't it the case?



 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-23 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639613#comment-13639613
 ] 

Rémy SAISSY commented on YARN-126:
--

| Ideally I think we should redo all of the hadoop/hdfs/yarn/mapred commands to 
have consistent usage and interfaces but that is a much bigger change.

Yes and the command line options of these tools might be confusing sometimes.
In my daily life I am an IT consultant and when I install/train clients on 
Hadoop, they very often encounter two issues when it comes to using the command 
line:
 - which options are optional for a specific command (-fs in yarn rmadmin?)
 - where does some arguments must be specified (-Dmapreduce.map.java.opts when 
submiting a job on command line using hadoop jar).

Maybe the redo could be done in an umbrella JIRA with several issues, one per 
project?


 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch, YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-24 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-

Attachment: (was: YARN-126.patch)

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-26 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642787#comment-13642787
 ] 

Rémy SAISSY commented on YARN-126:
--

If we need to do it, I can take a crack at this larger change.


 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2013-04-26 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642879#comment-13642879
 ] 

Rémy SAISSY commented on YARN-126:
--

A bird in the hand is worth two in the bush :).


 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (YARN-683) Class MiniYARNCluster not found when starting the minicluster

2013-05-14 Thread JIRA

Rémy SAISSY created YARN-683:


 Summary: Class MiniYARNCluster not found when starting the 
minicluster
 Key: YARN-683
 URL: https://issues.apache.org/jira/browse/YARN-683
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 2.0.4-alpha, 3.0.0
 Environment: MacOSX 10.8.3 - Java 1.6.0_45
Reporter: Rémy SAISSY


Starting the minicluster with the following command line:
bin/hadoop jar 
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.4-alpha-tests.jar 
minicluster -format

Fails for MiniYARNCluster with the following error:

13/05/14 17:06:58 INFO hdfs.MiniDFSCluster: Cluster is active
13/05/14 17:06:58 INFO mapreduce.MiniHadoopClusterManager: Started 
MiniDFSCluster -- namenode on port 55205
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/MiniYARNCluster
at 
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:170)
at 
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:129)
at 
org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:314)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
at 
org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:115)
at 
org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.yarn.server.MiniYARNCluster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 16 more



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (YARN-1442) change yarn minicluster base directory via system property

2013-11-26 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13832673#comment-13832673
 ] 

André Kelpe commented on YARN-1442:
---

any chance that this patch could be applied?

 change yarn minicluster base directory via system property
 --

 Key: YARN-1442
 URL: https://issues.apache.org/jira/browse/YARN-1442
 Project: Hadoop YARN
  Issue Type: New Feature
Affects Versions: 2.2.0
Reporter: André Kelpe
Priority: Minor
 Attachments: HADOOP-10122.patch


 The yarn minicluster used for testing uses the target directory by default. 
 We use gradle for building our projects and we would like to see it using a 
 different directory. This patch makes it possible to use a different 
 directory by setting the yarn.minicluster.directory system property.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2014-12-01 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.2.patch

Patch update.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Fix For: 2.7.0

Attachments: YARN-1621.1.patch, YARN-1621.2.patch

As more applications are moved to YARN, we need generic CLI to list rows of
task attempt ID, container ID, host of container, state of container. Today
if YARN application running in a container does hang, there is no way to find
out more info because a user does not know where each attempt is running in.
For each running application, it is useful to differentiate between
running/succeeded/failed/killed containers.

{code:title=proposed yarn cli}
$ yarn application -list-containers -applicationId appId [-containerState
state of container]
where containerState is optional filter to list container in given state only.
container state can be running/succeeded/killed/failed/all.
A user can specify more than one container state at once e.g. KILLED,FAILED.
task attempt ID container ID host of container state of container
{code}
CLI should work with running application/completed application. If a
container runs many task attempts, all attempts should be shown. That will
likely be the case of Tez container-reuse application.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1723) AMRMClientAsync missing blacklist addition and removal functionality

2014-12-06 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bartosz Ługowski updated YARN-1723:
---
Attachment: YARN-1723.1.patch

Tests are already for AMRMClient.

 AMRMClientAsync missing blacklist addition and removal functionality
 

 Key: YARN-1723
 URL: https://issues.apache.org/jira/browse/YARN-1723
 Project: Hadoop YARN
  Issue Type: Sub-task
Affects Versions: 2.2.0
Reporter: Bikas Saha
 Fix For: 2.7.0

 Attachments: YARN-1723.1.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-08 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.3.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Fix For: 2.7.0

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-11 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14316981#comment-14316981
]

Bartosz Ługowski commented on YARN-1621:

Thanks for comments [~vinodkv] and [~Naganarasimha].

[~vinodkv], I can't rename it to list-containers, because options parser
disallows - char.
[~Naganarasimha], I think it is good idea to move it to yarn container -list,
so we can easily add additional filters only in one place.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-10 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314744#comment-14314744
]

Bartosz Ługowski commented on YARN-1621:

Patch update.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Fix For: 2.7.0

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-10 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.3.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Fix For: 2.7.0

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-10 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.3.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Fix For: 2.7.0

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-15 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362450#comment-14362450
]

Bartosz Ługowski commented on YARN-1621:

Any comments?

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch, YARN-1621.5.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-05 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.5.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-05 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349339#comment-14349339
]

Bartosz Ługowski commented on YARN-1621:

[~Naganarasimha], thanks for review.

1. Done.
2. Done.
3. Good idea, there is no need to repeat application attempt ID for each
container. Example:
{code}ApplicationAttempt-Id: appattempt_1234_0005_01
Total number of containers: 2
Container-IdStart Time Finish Time
StateHost
LOG-URL
container_1234_0005_01_01 Thu Jan 01 01:00:01 +0100 1970 Thu Jan 01
01:00:01 +0100 1970 COMPLETE host1:8881
logURL
container_1234_0005_01_02 Thu Jan 01 01:00:01 +0100 1970 Thu Jan 01
01:00:01 +0100 1970 NEW host2:8882
logURL{code}

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-07 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.5.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-07 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.5.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-25 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-25 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Rebase and run tests again.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-25 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-25 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-31 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-31 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-28 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Rebase and trigger Jenkins again.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-28 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-28 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341609#comment-14341609
]

Bartosz Ługowski commented on YARN-1621:

[~vinodkv], [~Naganarasimha], could you review patch? Thanks.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-22 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14332142#comment-14332142
 ] 

Bartosz Ługowski commented on YARN-1621:


I had extracted {{yarn container}} method from ApplicationCLI to new 
file(ContainerCLI) and changed {{yarn container -list Application Attempt 
ID}} to {{yarn container -list Application ID|Application Attempt ID}}. Now 
container states filtering is done at server side(not at CLI, as suggested 
[~Naganarasimha]), and works for both: applications and application attempts. I 
didn't include changes in {{yarn.cmd}} because it brakes the patch(git can't 
apply it).

 Add CLI to list rows of task attempt ID, container ID, host of container, 
 state of container
 --

 Key: YARN-1621
 URL: https://issues.apache.org/jira/browse/YARN-1621
 Project: Hadoop YARN
  Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
 Fix For: 2.7.0

 Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch, 
 YARN-1621.4.patch


 As more applications are moved to YARN, we need generic CLI to list rows of 
 task attempt ID, container ID, host of container, state of container. Today 
 if YARN application running in a container does hang, there is no way to find 
 out more info because a user does not know where each attempt is running in.
 For each running application, it is useful to differentiate between 
 running/succeeded/failed/killed containers.
  
 {code:title=proposed yarn cli}
 $ yarn application -list-containers -applicationId appId [-containerState 
 state of container]
 where containerState is optional filter to list container in given state only.
 container state can be running/succeeded/killed/failed/all.
 A user can specify more than one container state at once e.g. KILLED,FAILED.
 task attempt ID container ID host of container state of container 
 {code}
 CLI should work with running application/completed application. If a 
 container runs many task attempts, all attempts should be shown. That will 
 likely be the case of Tez container-reuse application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-02-22 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.4.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-24 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378391#comment-14378391
]

Bartosz Ługowski commented on YARN-1621:

Thanks [~Naganarasimha].

Done all, apart of:

{quote}
* May be we can leverage the benifit of passing the states to AHS too, this
will reduce the transfer of data from AHS to the client. ur opinion ?
* If we are incorporating the above point then i feel only only when
appNotFoundInRM we need to query for all states from AHS if not querying for
COMPLETE state would be sufficient.
{quote}
Correct me if I'm wrong, but AHS has only COMPLETE containers, so we need to
query AHS only if states filter is empty(ALL) or contains COMPLETE state.
{quote}
* No test cases for modification of
GetContainersRequestPBImpl/GetContainersRequestProto
{quote}
There are already tests for this in:
org.apache.hadoop.yarn.api.TestPBImplRecords#testGetContainersRequestPBImpl ?
{quote}
* there are some test case failures and findbugs issues reported can you take a
look at it
{quote}
Not related with this patch.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-03-24 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-04-13 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493085#comment-14493085
]

Bartosz Ługowski commented on YARN-1621:

Failed tests are not related with this patch.

Any comments?

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-05-04 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-05-04 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-04-28 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-04-28 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2015-05-10 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-
Attachment: YARN-126.002.patch

Hi Li Lu,
here it is.
Regards,


 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.002.patch, YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-20) More information for yarn.resourcemanager.webapp.address in yarn-default.xml

2015-05-07 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bartosz Ługowski updated YARN-20:
-
Attachment: YARN-20.2.patch

I've changed only space indent.
Uploaded new version(v2) without {code}yarn.http.policy{code} reformatting.

 More information for yarn.resourcemanager.webapp.address in yarn-default.xml
 --

 Key: YARN-20
 URL: https://issues.apache.org/jira/browse/YARN-20
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: documentation, resourcemanager
Affects Versions: 2.0.0-alpha
Reporter: Nemon Lou
Priority: Trivial
  Labels: BB2015-05-TBR, newbie
 Attachments: YARN-20.1.patch, YARN-20.2.patch, YARN-20.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

   The parameter  yarn.resourcemanager.webapp.address in yarn-default.xml  is 
 in host:port format,which is noted in the cluster set up guide 
 (http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html).
   When i read though the code,i find host format is also supported. In 
 host format,the port will be random.
   So we may add more documentation in  yarn-default.xml for easy understood.
   I will submit a patch if it's helpful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3630) YARN should suggest a heartbeat interval for applications

2015-05-12 Thread JIRA

Zoltán Zvara created YARN-3630:
--

 Summary: YARN should suggest a heartbeat interval for applications
 Key: YARN-3630
 URL: https://issues.apache.org/jira/browse/YARN-3630
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: client, resourcemanager, scheduler
Reporter: Zoltán Zvara


It seems currently applications - for example Spark - are not adaptive to RM 
regarding heartbeat intervals. RM should be able to suggest a desired heartbeat 
interval to application.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-3630) YARN should suggest a heartbeat interval for applications

2015-05-12 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Zvara updated YARN-3630:
---
Description: It seems currently applications - for example Spark - are not 
adaptive to RM regarding heartbeat intervals. RM should be able to suggest a 
desired heartbeat interval to applications.  (was: It seems currently 
applications - for example Spark - are not adaptive to RM regarding heartbeat 
intervals. RM should be able to suggest a desired heartbeat interval to 
application.)

 YARN should suggest a heartbeat interval for applications
 -

 Key: YARN-3630
 URL: https://issues.apache.org/jira/browse/YARN-3630
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: client, resourcemanager, scheduler
Reporter: Zoltán Zvara

 It seems currently applications - for example Spark - are not adaptive to RM 
 regarding heartbeat intervals. RM should be able to suggest a desired 
 heartbeat interval to applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-3630) YARN should suggest a heartbeat interval for applications

2015-05-12 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539631#comment-14539631
 ] 

Zoltán Zvara commented on YARN-3630:


I was thinking in a more adaptive solution that takes current load into 
consideration.

Anyway, having a global configuration parameter for desired heartbeat would 
simplify thinks for application.
Any thoughts on implementing something like this?

 YARN should suggest a heartbeat interval for applications
 -

 Key: YARN-3630
 URL: https://issues.apache.org/jira/browse/YARN-3630
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: resourcemanager, scheduler
Affects Versions: 2.7.0
Reporter: Zoltán Zvara
Priority: Minor

 It seems currently applications - for example Spark - are not adaptive to RM 
 regarding heartbeat intervals. RM should be able to suggest a desired 
 heartbeat interval to applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-3630) YARN should suggest a heartbeat interval for applications

2015-05-12 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14539671#comment-14539671
 ] 

Zoltán Zvara commented on YARN-3630:


[~xinxianyin], I would not mind. But I think this issue should have a higher 
priority. For a highly contested and multi-tenant cluster this should be a 
problem.

 YARN should suggest a heartbeat interval for applications
 -

 Key: YARN-3630
 URL: https://issues.apache.org/jira/browse/YARN-3630
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: resourcemanager, scheduler
Affects Versions: 2.7.0
Reporter: Zoltán Zvara
Priority: Minor

 It seems currently applications - for example Spark - are not adaptive to RM 
 regarding heartbeat intervals. RM should be able to suggest a desired 
 heartbeat interval to applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-05-14 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
Labels: BB2015-05-TBR
Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch, YARN-1621.5.patch, YARN-1621.6.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-05-14 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
Labels: BB2015-05-TBR
Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch, YARN-1621.5.patch, YARN-1621.6.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2015-05-17 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-
Attachment: (was: YARN-126.002.patch)

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT

2015-05-17 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rémy SAISSY updated YARN-126:
-
Attachment: YARN-126.002.patch

 yarn rmadmin help message contains reference to hadoop cli and JT
 -

 Key: YARN-126
 URL: https://issues.apache.org/jira/browse/YARN-126
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Affects Versions: 2.0.3-alpha
Reporter: Thomas Graves
Assignee: Rémy SAISSY
  Labels: usability
 Attachments: YARN-126.002.patch, YARN-126.patch


 has option to specify a job tracker and the last line for general command 
 line syntax had bin/hadoop command [genericOptions] [commandOptions]
 ran yarn rmadmin to get usage:
 RMAdmin
 Usage: java RMAdmin
[-refreshQueues]
[-refreshNodes]
[-refreshUserToGroupsMappings]
[-refreshSuperUserGroupsConfiguration]
[-refreshAdminAcls]
[-refreshServiceAcl]
[-help [cmd]]
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-04-11 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: (was: YARN-1621.6.patch)

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-04-11 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bartosz Ługowski updated YARN-1621:
---
Attachment: YARN-1621.6.patch

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-20) More information for yarn.resourcemanager.webapp.address in yarn-default.xml

2015-04-14 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bartosz Ługowski updated YARN-20:
-
Attachment: YARN-20.1.patch

Rebased, https description changed too.

 More information for yarn.resourcemanager.webapp.address in yarn-default.xml
 --

 Key: YARN-20
 URL: https://issues.apache.org/jira/browse/YARN-20
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: documentation, resourcemanager
Affects Versions: 2.0.0-alpha
Reporter: Nemon Lou
Priority: Trivial
 Attachments: YARN-20.1.patch, YARN-20.patch

   Original Estimate: 1h
  Remaining Estimate: 1h

   The parameter  yarn.resourcemanager.webapp.address in yarn-default.xml  is 
 in host:port format,which is noted in the cluster set up guide 
 (http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html).
   When i read though the code,i find host format is also supported. In 
 host format,the port will be random.
   So we may add more documentation in  yarn-default.xml for easy understood.
   I will submit a patch if it's helpful.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-3772) Let AMs to change the name of the application upon RM registration

2015-06-05 Thread JIRA

Zoltán Zvara created YARN-3772:
--

 Summary: Let AMs to change the name of the application upon RM 
registration
 Key: YARN-3772
 URL: https://issues.apache.org/jira/browse/YARN-3772
 Project: Hadoop YARN
  Issue Type: New Feature
  Components: resourcemanager
Reporter: Zoltán Zvara


Many applications like to set their name in their own way with their own API 
internally, but also want to display that name on YARN. Therefore it is not 
always possible to know the name of the application during submission. YARN 
should let AMs to change their name on - at least - the first time they 
register with the RM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-2139) [Umbrella] Support for Disk as a Resource in YARN

2015-06-04 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573468#comment-14573468
 ] 

kassiano josé matteussi commented on YARN-2139:
---

Dears, 

I have studied resource management under Hadoop applications running wrapped in 
Linux containers and I have faced troubles to restrict disk I/O with cgroups 
(bps_write, bps_read). 

Does anybody know if it is possible to do so?

I have heard that limiting I/O with cgroups is restricted to synchronous 
writing (SYNC) and that is why it wouldn't work well with Hadoop + HDFS. Is 
this still true in more recent kernel implementation?

Best Regards,
Kassiano

 [Umbrella] Support for Disk as a Resource in YARN 
 --

 Key: YARN-2139
 URL: https://issues.apache.org/jira/browse/YARN-2139
 Project: Hadoop YARN
  Issue Type: New Feature
Reporter: Wei Yan
 Attachments: Disk_IO_Isolation_Scheduling_3.pdf, 
 Disk_IO_Scheduling_Design_1.pdf, Disk_IO_Scheduling_Design_2.pdf, 
 YARN-2139-prototype-2.patch, YARN-2139-prototype.patch


 YARN should consider disk as another resource for (1) scheduling tasks on 
 nodes, (2) isolation at runtime, (3) spindle locality. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container

2015-07-13 Thread JIRA

[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625207#comment-14625207
]

Bartosz Ługowski commented on YARN-1621:

I would appreciate if anyone could review this patch. Thanks.

Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--

Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
Labels: BB2015-05-TBR
Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch, YARN-1621.5.patch, YARN-1621.6.patch

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-5130) Mark ContainerStatus and NodeReport as evolving

2016-05-27 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-5130:

Attachment: YARN-5130.001.patch

> Mark ContainerStatus and NodeReport as evolving
> ---
>
> Key: YARN-5130
> URL: https://issues.apache.org/jira/browse/YARN-5130
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-5130.001.patch
>
>
> It turns out that slider won't build as the {{ContainerStatus}} and 
> {{NodeReport}} classes have added more abstract methods, so breaking the mock 
> objects.
> While it is everyone's freedom to change things, these classes are both tagged
> {code}
> @Public
> @Stable
> {code}
> Given they aren't stable, can someone mark them as {{@Evolving}}? That way 
> when downstream code breaks, we can be less disappointed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-5130) Mark ContainerStatus and NodeReport as evolving

2016-05-27 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák reassigned YARN-5130:
---

Assignee: Gergely Novák

> Mark ContainerStatus and NodeReport as evolving
> ---
>
> Key: YARN-5130
> URL: https://issues.apache.org/jira/browse/YARN-5130
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
>
> It turns out that slider won't build as the {{ContainerStatus}} and 
> {{NodeReport}} classes have added more abstract methods, so breaking the mock 
> objects.
> While it is everyone's freedom to change things, these classes are both tagged
> {code}
> @Public
> @Stable
> {code}
> Given they aren't stable, can someone mark them as {{@Evolving}}? That way 
> when downstream code breaks, we can be less disappointed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-5130) Mark ContainerStatus and NodeReport as evolving

2016-05-27 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304020#comment-15304020
 ] 

Gergely Novák commented on YARN-5130:
-

Added patch for [~ste...@apache.org]'s suggestion.

> Mark ContainerStatus and NodeReport as evolving
> ---
>
> Key: YARN-5130
> URL: https://issues.apache.org/jira/browse/YARN-5130
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: yarn
>Affects Versions: 2.8.0
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-5130.001.patch
>
>
> It turns out that slider won't build as the {{ContainerStatus}} and 
> {{NodeReport}} classes have added more abstract methods, so breaking the mock 
> objects.
> While it is everyone's freedom to change things, these classes are both tagged
> {code}
> @Public
> @Stable
> {code}
> Given they aren't stable, can someone mark them as {{@Evolving}}? That way 
> when downstream code breaks, we can be less disappointed



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-27 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15352196#comment-15352196
 ] 

Gergely Novák commented on YARN-5227:
-

Thanks for the comments, updated patch #2 with the suggested error message and 
refactor.

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch, YARN-5227.002.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-27 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-5227:

Attachment: YARN-5227.002.patch

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch, YARN-5227.002.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-25 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349709#comment-15349709
 ] 

Gergely Novák commented on YARN-5227:
-

If both the appId and the containerId are specified, the appId will be 
interpreted in the block you quoted and the containerId will be interpreted in 
line 233-241:
{code}
if (containerId == null) {
  containerId = ContainerId.fromString(containerIdStr);
  if (!containerId.getApplicationAttemptId().getApplicationId()
  .equals(appId)) {
System.err.println("The Application:" + appId
+ " does not have the container:" + containerId);
return -1;
  }
}
{code}

I will change the error message in the next patch according to your suggestion.

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
>     URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-5227) yarn logs command: no need to specify -applicaionId when specifying containerId

2016-06-24 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák reassigned YARN-5227:
---

Assignee: Gergely Novák

> yarn logs command: no need to specify -applicaionId when specifying 
> containerId
> ---
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-24 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-5227:

Attachment: YARN-5227.001.patch

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-24 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-5227:

Summary: yarn logs command: no need to specify -applicationId when 
specifying containerId  (was: yarn logs command: no need to specify 
-applicaionId when specifying containerId)

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-24 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348929#comment-15348929
 ] 

Gergely Novák edited comment on YARN-5227 at 6/25/16 12:12 AM:
---

We fail if neither of them is specified ({{appIdStr == null && containerIdStr 
== null}}). Do you mean that the error message should be changed to something 
like "ApplicationId is optional only if the containerId is specified."


was (Author: gergelynovak):
We fail if neither of them is specified ({{appIdStr == null && containerIdStr 
== null}}). 

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-5227) yarn logs command: no need to specify -applicationId when specifying containerId

2016-06-24 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348929#comment-15348929
 ] 

Gergely Novák commented on YARN-5227:
-

We fail if neither of them is specified ({{appIdStr == null && containerIdStr 
== null}}). 

> yarn logs command: no need to specify -applicationId when specifying 
> containerId
> 
>
> Key: YARN-5227
> URL: https://issues.apache.org/jira/browse/YARN-5227
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Jian He
>Assignee: Gergely Novák
> Attachments: YARN-5227.001.patch
>
>
> No need to specify -applicaionId when specifying containerId, because 
> applicationId is retrievable from containerId



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-07 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230307#comment-15230307
 ] 

Gergely Novák commented on YARN-4928:
-

Uploaded a new patch (v.002) that seems to work as originally intended: like in 
HDFS-6189 it uses {{System.getProperty("test.build.data", 
System.getProperty("java.io.tmpdir"))}} as the base directory for 
MiniDFSCluster, and {{/tmp/...}} as (and only as) HDFS path (in accordance with 
[~arpitagarwal]'s comment). So it does not use {{C:\tmp}} on the local file 
system, but still works on Windows too (because {{DFSUtil.isValidName()}} 
checks the HDFS path, not the local path).

[~djp] or [~ste...@apache.org] could you please review?

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
>     URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-07 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230408#comment-15230408
 ] 

Gergely Novák commented on YARN-4928:
-

Sorry, I used on older (incompatible) branch for the patch, v003 now works for 
branch-2.8 and trunk. 

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch, 
> YARN-4928.003.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-07 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4928:

Attachment: YARN-4928.003.patch

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch, 
> YARN-4928.003.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-06 Thread JIRA

Gergely Novák created YARN-4928:
---

 Summary: Some yarn.server.timeline.* tests fail on Windows 
attempting to use a test root path containing a colon
 Key: YARN-4928
 URL: https://issues.apache.org/jira/browse/YARN-4928
 Project: Hadoop YARN
  Issue Type: Bug
  Components: test
Affects Versions: 2.8.0
 Environment: OS: Windows Server 2012
JDK: 1.7.0_79
Reporter: Gergely Novák
Priority: Minor


yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
yarn.server.timeline.TestLogInfo.* fail on Windows, because they are attempting 
to use a test root paths like 
"/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
 which contains a ":" (after the Windows drive letter) and 
DFSUtil.isValidName() does not accept paths containing ":".

This problem is identical to HDFS-6189, so I suggest to use the same approach: 
using "/tmp/..." as test root dir instead of 
System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-06 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4928:

Attachment: YARN-4928.001.patch

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Description: 
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more
{noformat}

This is caused by really not having enough available memory to complete the 
reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
by lowering the number of containers to 2, or the memory to 512 MB) would make 
the test more stable. 

  was:
TestYarnClient.testReservationAPIs sometimes fails with this error:
```
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Description: 
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
{noformat}

This is caused by really not having enough available memory to complete the 
reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
by lowering the number of containers to 2, or the memory to 512 MB) would make 
the test more stable. 

  was:
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Description: 
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
{noformat}

This is caused by really not having enough available memory to complete the 
reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
by lowering the number of containers to 2, or the memory to 512 MB) would make 
the test more stable. 

  was:
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Description: 
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more

at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
{noformat}

This is caused by really not having enough available memory to complete the 
reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
by lowering the number of containers to 2, or the memory to 512 MB) would make 
the test more stable. 

  was:
TestYarnClient.testReservationAPIs sometimes fails with this error:
{noformat}
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Attachment: YARN-4954.001.patch

> TestYarnClient.testReservationAPIs fails on machines with less than 4 GB 
> available memory
> -
>
> Key: YARN-4954
> URL: https://issues.apache.org/jira/browse/YARN-4954
> Project: Hadoop YARN
>  Issue Type: Test
>  Components: test
>Affects Versions: 3.0.0
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4954.001.patch
>
>
> TestYarnClient.testReservationAPIs sometimes fails with this error:
> {noformat}
> java.lang.AssertionError: 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
>  The request cannot be satisfied
>   at 
> org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
>   at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
> Caused by: 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
>  The request cannot be satisfied
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
>   ... 10 more
>   at org.junit.Assert.fail(Assert.java:88)
>   at 
> org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
> {noformat}
> This is caused by really not having enough available memory to complete the 
> reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
> by lowering the number of containers to 2, or the memory to 512 MB) would 
> make the test more stable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4954:

Description: 
TestYarnClient.testReservationAPIs sometimes fails with this error:
```
java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more
```

This is caused by really not having enough available memory to complete the 
reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
by lowering the number of containers to 2, or the memory to 512 MB) would make 
the test more stable. 

  was:
TestYarnClient.testReservationAPIs sometimes fails with this error:
{{java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55

[jira] [Created] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-13 Thread JIRA

Gergely Novák created YARN-4954:
---

 Summary: TestYarnClient.testReservationAPIs fails on machines with 
less than 4 GB available memory
 Key: YARN-4954
 URL: https://issues.apache.org/jira/browse/YARN-4954
 Project: Hadoop YARN
  Issue Type: Test
  Components: test
Affects Versions: 3.0.0
Reporter: Gergely Novák
Assignee: Gergely Novák
Priority: Minor


TestYarnClient.testReservationAPIs sometimes fails with this error:
{{java.lang.AssertionError: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
at 
org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
at 
org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
Caused by: 
org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
 The request cannot be satisfied
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
at 
org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
... 10 more


at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
at 
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
at 
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method

[jira] [Updated] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-08 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4928:

Attachment: YARN-4928.005.patch

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch, 
> YARN-4928.003.patch, YARN-4928.004.patch, YARN-4928.005.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-08 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4928:

Attachment: YARN-4928.004.patch

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch, 
> YARN-4928.003.patch, YARN-4928.004.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-08 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231937#comment-15231937
 ] 

Gergely Novák commented on YARN-4928:
-

Thanks for the reviews. Uploaded v004, the only modification is the import. 

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch, YARN-4928.002.patch, 
> YARN-4928.003.patch, YARN-4928.004.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-4928) Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon

2016-04-06 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228459#comment-15228459
 ] 

Gergely Novák commented on YARN-4928:
-

No, you're right, this actually does use the local file system. HDFS-6189 uses 
a different approach as well, let me update the patch based on that.

> Some yarn.server.timeline.* tests fail on Windows attempting to use a test 
> root path containing a colon
> ---
>
> Key: YARN-4928
> URL: https://issues.apache.org/jira/browse/YARN-4928
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
> Environment: OS: Windows Server 2012
> JDK: 1.7.0_79
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4928.001.patch
>
>
> yarn.server.timeline.TestEntityGroupFSTimelineStore.* and 
> yarn.server.timeline.TestLogInfo.* fail on Windows, because they are 
> attempting to use a test root paths like 
> "/C:/hdp/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/target/test-dir/TestLogInfo",
>  which contains a ":" (after the Windows drive letter) and 
> DFSUtil.isValidName() does not accept paths containing ":".
> This problem is identical to HDFS-6189, so I suggest to use the same 
> approach: using "/tmp/..." as test root dir instead of 
> System.getProperty("test.build.data", System.getProperty("java.io.tmpdir")).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-4680) TimerTasks leak in ATS V1.5 Writer

2016-03-02 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175792#comment-15175792
 ] 

Jakob Stengård commented on YARN-4680:
--

Hi.
What are the symptoms of this issue?

Im having a problem with hiveserver2, which is creating a lot of timer tasks 
named 
"LogFDsCachecleanInActiveFDsTimer" and "LogFDsCacheFlushTimer".

Eventually, hiveserver2 crashes. Could this be related to this bug?

> TimerTasks leak in ATS V1.5 Writer
> --
>
> Key: YARN-4680
> URL: https://issues.apache.org/jira/browse/YARN-4680
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: timelineserver
>Reporter: Xuan Gong
>Assignee: Xuan Gong
> Fix For: 2.8.0
>
> Attachments: YARN-4680.1.patch, YARN-4680.20160108.patch, 
> YARN-4680.20160109.patch, YARN-4680.20160222.patch
>
>
> We have seen TimerTasks leak which could cause application server done (such 
> as oozie server done due to too many active threads)
> Although we have fixed some potentially leak situations in upper application 
> level, such as
> https://issues.apache.org/jira/browse/MAPREDUCE-6618
> https://issues.apache.org/jira/browse/MAPREDUCE-6621, we still can not 
> guarantee that we fixed the issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (YARN-4894) Elide long app names in web UI

2016-04-29 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4894:

Attachment: Screen Shot 2016-04-29 at 09.07.50.png

Which version are you using? On trunk and on 2.7.2 long names don't push the 
other columns right, they get wrapped (like on the attached screenshot).

> Elide long app names in web UI
> --
>
> Key: YARN-4894
> URL: https://issues.apache.org/jira/browse/YARN-4894
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: webapp
>Reporter: Ryan Williams
>Priority: Minor
> Attachments: Screen Shot 2016-04-29 at 09.07.50.png
>
>
> When someone submits an app with a long name, the other columns in the UI get 
> pushed far to the right and require scrolling to see, which makes for an 
> awkward experience.
> !http://f.cl.ly/items/1L2G2U3B0s1U3m060Z42/Screen%20Shot%202016-03-29%20at%201.15.22%20PM.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Assigned] (YARN-4828) Create a pull request template for github

2016-05-17 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák reassigned YARN-4828:
---

Assignee: Gergely Novák  (was: Andras Bokor)

> Create a pull request template for github
> -
>
> Key: YARN-4828
> URL: https://issues.apache.org/jira/browse/YARN-4828
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.0.0-alpha1
> Environment: github
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
>
> We're starting to see PRs appear without any JIRA, explanation etc. These are 
> going to be ignored without them.
> It's possible to [create a PR text 
> template](https://help.github.com/articles/creating-a-pull-request-template-for-your-repository/)
>  under {{.github/PULL_REQUEST_TEMPLATE}}
> We can do such a template, which provides template summary points, such as:
> * which JIRA
> * if against an object store, how did you test it?
> * if its a shell script, how did you test it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-4828) Create a pull request template for github

2016-05-17 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4828:

Attachment: YARN-4828.001.patch

> Create a pull request template for github
> -
>
> Key: YARN-4828
> URL: https://issues.apache.org/jira/browse/YARN-4828
> Project: Hadoop YARN
>  Issue Type: Improvement
>  Components: build
>Affects Versions: 3.0.0-alpha1
> Environment: github
>Reporter: Steve Loughran
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4828.001.patch
>
>
> We're starting to see PRs appear without any JIRA, explanation etc. These are 
> going to be ignored without them.
> It's possible to [create a PR text 
> template](https://help.github.com/articles/creating-a-pull-request-template-for-your-repository/)
>  under {{.github/PULL_REQUEST_TEMPLATE}}
> We can do such a template, which provides template summary points, such as:
> * which JIRA
> * if against an object store, how did you test it?
> * if its a shell script, how did you test it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-4978) The number of javadocs warnings is limited to 100

2016-05-02 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gergely Novák updated YARN-4978:

Attachment: YARN-4978.001.patch

> The number of javadocs warnings is limited to 100 
> --
>
> Key: YARN-4978
> URL: https://issues.apache.org/jira/browse/YARN-4978
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Li Lu
> Attachments: YARN-4978.001.patch
>
>
> We are generating a lot of javadoc warnings with jdk 1.8. Right now the 
> number is limited to 100. Enlarge this limitation can probably reveal more 
> problems in one batch for our javadoc generation process. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-4978) The number of javadocs warnings is limited to 100

2016-05-03 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268724#comment-15268724
 ] 

Gergely Novák commented on YARN-4978:
-

Uploaded patch #001, it increases the limit of the javadoc warnings to 10 000. 

I don't see how this could be unit tested, an example for manual verification 
is using {{mvn javadoc:javadoc}} from 
{{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api}} with JDK 1.8; before the 
patch this gives 100 warnings, after the patch it gives 5406 ones.

> The number of javadocs warnings is limited to 100 
> --
>
> Key: YARN-4978
> URL: https://issues.apache.org/jira/browse/YARN-4978
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Li Lu
> Attachments: YARN-4978.001.patch
>
>
> We are generating a lot of javadoc warnings with jdk 1.8. Right now the 
> number is limited to 100. Enlarge this limitation can probably reveal more 
> problems in one batch for our javadoc generation process. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Updated] (YARN-5025) Container move (relocation) between nodes

2016-05-02 Thread JIRA


 [ 
https://issues.apache.org/jira/browse/YARN-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltán Zvara updated YARN-5025:
---
Attachment: YARN-Container-Move-(Relocation)-Between-Nodes.pdf

Version 0.1.

> Container move (relocation) between nodes
> -
>
> Key: YARN-5025
> URL: https://issues.apache.org/jira/browse/YARN-5025
> Project: Hadoop YARN
>  Issue Type: New Feature
>  Components: nodemanager, resourcemanager, yarn
>Reporter: Zoltán Zvara
> Attachments: YARN-Container-Move-(Relocation)-Between-Nodes.pdf
>
>
> Support for relocating containers has become a must-have requirement for most 
> multi-service applications, since the inevitable concept-drifts make SLAs 
> hard to be satisfied. The relocation and co-location of services (long 
> running containers) can help to reduce bottlenecks in a multi-service 
> cluster, especially where data-intensive, streaming applications interfere.
> See the high-level implementation details in the attached design document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Created] (YARN-5025) Container move (relocation) between nodes

2016-05-02 Thread JIRA

Zoltán Zvara created YARN-5025:
--

 Summary: Container move (relocation) between nodes
 Key: YARN-5025
 URL: https://issues.apache.org/jira/browse/YARN-5025
 Project: Hadoop YARN
  Issue Type: New Feature
  Components: nodemanager, resourcemanager, yarn
Reporter: Zoltán Zvara


Support for relocating containers has become a must-have requirement for most 
multi-service applications, since the inevitable concept-drifts make SLAs hard 
to be satisfied. The relocation and co-location of services (long running 
containers) can help to reduce bottlenecks in a multi-service cluster, 
especially where data-intensive, streaming applications interfere.

See the high-level implementation details in the attached design document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Commented] (YARN-4954) TestYarnClient.testReservationAPIs fails on machines with less than 4 GB available memory

2016-04-19 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247565#comment-15247565
 ] 

Gergely Novák commented on YARN-4954:
-

Tried to reproduce the timeout issues but couldn't. However, it is certain that 
they have nothing to do with my patch and in the subsequent builds they are not 
present.

> TestYarnClient.testReservationAPIs fails on machines with less than 4 GB 
> available memory
> -
>
> Key: YARN-4954
> URL: https://issues.apache.org/jira/browse/YARN-4954
> Project: Hadoop YARN
>  Issue Type: Test
>  Components: test
>Affects Versions: 3.0.0
>Reporter: Gergely Novák
>Assignee: Gergely Novák
>Priority: Minor
> Attachments: YARN-4954.001.patch
>
>
> TestYarnClient.testReservationAPIs sometimes fails with this error:
> {noformat}
> java.lang.AssertionError: 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
>  The request cannot be satisfied
>   at 
> org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:38)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1254)
>   at 
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitReservation(ApplicationClientProtocolPBServiceImpl.java:457)
>   at 
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:515)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:637)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2422)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2418)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2416)
> Caused by: 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException:
>  The request cannot be satisfied
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.IterativePlanner.computeJobAllocation(IterativePlanner.java:151)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.allocateUser(PlanningAlgorithm.java:64)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.PlanningAlgorithm.createReservation(PlanningAlgorithm.java:140)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.TryManyReservationAgents.createReservation(TryManyReservationAgents.java:55)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy.createReservation(AlignedPlannerWithGreedy.java:84)
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitReservation(ClientRMService.java:1237)
>   ... 10 more
>   at org.junit.Assert.fail(Assert.java:88)
>   at 
> org.apache.hadoop.yarn.client.api.impl.TestYarnClient.testReservationAPIs(TestYarnClient.java:1227)
> {noformat}
> This is caused by really not having enough available memory to complete the 
> reservation (4 * 1024 MB). In my opinion lowering the required memory (either 
> by lowering the number of containers to 2, or the memory to 512 MB) would 
> make the test more stable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (YARN-5475) Test failed for TestAggregatedLogFormat on trunk

2016-08-12 Thread JIRA


[ 
https://issues.apache.org/jira/browse/YARN-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418844#comment-15418844
 ] 

Gergely Novák commented on YARN-5475:
-

This is a permission issue. Some observations:
* testReadAcontainerLogs1 only fails if ran after testForCorruptedAggregatedLogs
* TestAggregatedLogFormat's permissions:
** after testForCorruptedAggregatedLogs: rwxr-xr-x 
** after testReadAcontainerLogs1 (if testForCorruptedAggregatedLogs ran): 
*rw-r-*
** after testReadAcontainerLogs1 (if testForCorruptedAggregatedLogs did'n run) 
rwxr-xr-x
* removing the line {{fs.mkdirs(subDir);}} from testReadAcontainerLog() seems 
to solve the issue

> Test failed for TestAggregatedLogFormat on trunk
> 
>
> Key: YARN-5475
> URL: https://issues.apache.org/jira/browse/YARN-5475
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Junping Du
>
> From some jenkins run: 
> https://builds.apache.org/job/PreCommit-YARN-Build/12651/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
> The error message for the test is:
> {noformat}
> Tests run: 3, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.114 sec <<< 
> FAILURE! - in org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat
> testReadAcontainerLogs1(org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat)
>   Time elapsed: 0.012 sec  <<< ERROR!
> java.io.IOException: Unable to create directory : 
> /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/TestAggregatedLogFormat/testReadAcontainerLogs1/srcFiles/application_1_0001/container_1_0001_01_01/subDir
>   at 
> org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat.getOutputStreamWriter(TestAggregatedLogFormat.java:403)
>   at 
> org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat.writeSrcFile(TestAggregatedLogFormat.java:382)
>   at 
> org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat.testReadAcontainerLog(TestAggregatedLogFormat.java:211)
>   at 
> org.apache.hadoop.yarn.logaggregation.TestAggregatedLogFormat.testReadAcontainerLogs1(TestAggregatedLogFormat.java:185)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 195674 matches

Mail list logo