yes... acl's are not enforced.
On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <[email protected]>wrote: > Do you mean that acl are not being enforced? > > Olivier > On 26 Nov 2013 20:18, "Munna" <[email protected]> wrote: > >> Hi Olivier, >> >> have u hold any solution, why user are not identifying their queues? >> >> Thanks >> Munna >> >> >> On Tue, Nov 26, 2013 at 11:17 PM, Munna <[email protected]> wrote: >> >>> *mapred queue -list* result: >>> >>> [user@host ~]$ mapred queue -list >>> 13/11/26 09:37:48 INFO service.AbstractService: >>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. >>> 13/11/26 09:37:48 INFO service.AbstractService: >>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. >>> ====================== >>> Queue Name : exploration >>> Queue State : running >>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>> CurrentCapacity: 0.0 >>> ====================== >>> Queue Name : a >>> Queue State : running >>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>> CurrentCapacity: 0.0 >>> ====================== >>> Queue Name : b >>> Queue State : running >>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0, >>> CurrentCapacity: 0.0 >>> ====================== >>> Queue Name : c >>> Queue State : running >>> Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0, >>> CurrentCapacity: 0.0 >>> ====================== >>> Queue Name : production >>> Queue State : running >>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0, CurrentCapacity: >>> 0.0 >>> >>> and I saw in RM logs, mapred user try to find default queue, logs are >>> below: >>> >>> 2013-11-26 05:22:59,804 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application >>> with id 1 submitted by user mapred >>> 2013-11-26 05:22:59,808 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred >>> IP=10.10.10.1 OPERATION=Submit Application Request >>> TARGET=ClientRMService RESULT=SUCCESS APPID=application_1385472086605_0001 >>> 2013-11-26 05:22:59,828 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: >>> application_1385472086605_0001 State change from NEW to SUBMITTED >>> 2013-11-26 05:22:59,829 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: >>> Registering appattempt_1385472086605_0001_000001 >>> 2013-11-26 05:22:59,830 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: >>> appattempt_1385472086605_0001_000001 State change from NEW to SUBMITTED >>> 2013-11-26 05:22:59,832 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: >>> appattempt_1385472086605_0001_000001 State change from SUBMITTED to FAILED >>> 2013-11-26 05:22:59,836 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: >>> application_1385472086605_0001 State change from SUBMITTED to FAILED >>> 2013-11-26 05:22:59,837 WARN >>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred >>> OPERATION=Application Finished - Failed TARGET=RMAppManager >>> RESULT=FAILURE DESCRIPTION=App failed with state: FAILED >>> PERMISSIONS=Application appattempt_1385472086605_0001_000001 submitted by >>> user mapred to unknown queue: default APPID=application_1385472086605_0001 >>> 2013-11-26 05:22:59,841 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: >>> Removing info for app: application_1385472086605_0001 >>> 2013-11-26 05:22:59,842 INFO >>> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: >>> appId=application_1385472086605_0001,name=select count(*) from >>> crc_exchange_rate(Stage-1),user=mapred,queue=default,state=FAILED,trackingUrl=hostname:8088/proxy/application_1385472086605_0001/,appMasterHost=N/A,startTime=1385472179799,finishTime=1385472179836 >>> >>> >>> >>> On Tue, Nov 26, 2013 at 10:36 PM, Olivier Renault < >>> [email protected]> wrote: >>> >>>> Could you confirm that the queue are running by: >>>> mapred queue -list >>>> >>>> Otherwise what error do you get when submitting a job ? >>>> >>>> Thanks, >>>> Olivier >>>> >>>> >>>> On 26 November 2013 15:33, Munna <[email protected]> wrote: >>>> >>>>> Below configured queue's are showing, then why i cannt able to run the >>>>> job. >>>>> >>>>> [root@host ~]# hadoop queue -showacls >>>>> DEPRECATED: Use of this script to execute mapred command is deprecated. >>>>> Instead use the mapred command for it. >>>>> >>>>> 13/11/26 05:25:43 INFO service.AbstractService: >>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited. >>>>> 13/11/26 05:25:44 INFO service.AbstractService: >>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started. >>>>> Queue acls for user : root >>>>> >>>>> Queue Operations >>>>> ===================== >>>>> root ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> exploration ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> a ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> b ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> c ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> production ADMINISTER_QUEUE,SUBMIT_APPLICATIONS >>>>> >>>>> >>>>> >>>>> On Tue, Nov 26, 2013 at 8:05 PM, Olivier Renault < >>>>> [email protected]> wrote: >>>>> >>>>>> Sorry my mistake, it's : >>>>>> mapred queue -showacls >>>>>> >>>>>> Or you can also use hadoop queue -showacls as suggested by Jitendra >>>>>> >>>>>> Thanks, >>>>>> Olviier >>>>>> >>>>>> >>>>>> On 26 November 2013 14:17, Munna <[email protected]> wrote: >>>>>> >>>>>>> Yes! I have executed same command: >>>>>>> >>>>>>> [user@host ~]$* yarn queue -showacls* >>>>>>> >>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError: queue >>>>>>> Caused by: java.lang.ClassNotFoundException: queue >>>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>>>> at >>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>>>>>> Could not find the main class: queue. Program will exit. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Nov 26, 2013 at 7:25 PM, Olivier Renault < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> it should be : >>>>>>>> yarn queue -showacls >>>>>>>> >>>>>>>> Olivier >>>>>>>> >>>>>>>> >>>>>>>> On 26 November 2013 13:28, Munna <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Olivier, >>>>>>>>> >>>>>>>>> Thank you for your reply. >>>>>>>>> >>>>>>>>> As you said, i ran those commands and i am getting following error >>>>>>>>> message on both the commands. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *[root@host ~]# sudo -u yarn yarn queue -showacls Exception in >>>>>>>>> thread "main" java.lang.NoClassDefFoundError: queueCaused by: >>>>>>>>> java.lang.ClassNotFoundException: queue at >>>>>>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:202) at >>>>>>>>> java.security.AccessController.doPrivileged(Native Method) at >>>>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:190) at >>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:306) at >>>>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>>>>>>>> at >>>>>>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find >>>>>>>>> the >>>>>>>>> main class: queue. Program will exit.* >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Munna >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Nov 26, 2013 at 4:10 PM, Olivier Renault < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Could you maybe send us the output of : >>>>>>>>>> - yarn queue -showacls >>>>>>>>>> - yarn queue -list >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Olivier >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 26 November 2013 05:53, Munna <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I working with Capacity Scheduler on YARN and I have configured >>>>>>>>>>> different queues. I can able to see all the queues on RM ui. But, >>>>>>>>>>> when i >>>>>>>>>>> start to run MR jobs with configured user names(yarn,mapred), i am >>>>>>>>>>> unable >>>>>>>>>>> to run the Jobs and job are suspended. Again i set default as FIFO >>>>>>>>>>> working >>>>>>>>>>> fine. >>>>>>>>>>> >>>>>>>>>>> Can you please help me out sort this issue and configured >>>>>>>>>>> Configurations are given below. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *<property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.queues</name> >>>>>>>>>>> <value>production,exploration</value> </property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.queues</name> >>>>>>>>>>> <value>a,b,c</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.capacity</name> >>>>>>>>>>> <value>100</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.production.capacity</name> >>>>>>>>>>> <value>70</value></property> <property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.capacity</name> >>>>>>>>>>> <value>30</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.a.capacity</name> >>>>>>>>>>> <value>30</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.b.capacity</name> >>>>>>>>>>> <value>30</value></property> <property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.c.capacity</name> >>>>>>>>>>> <value>40</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root. >>>>>>>>>>> production.acl_submit_applications</name> >>>>>>>>>>> <value>yarn,mapred</value></property><property> >>>>>>>>>>> <value>userb</value> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.a.acl_submit_applications</name> >>>>>>>>>>> </property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.b.acl_submit_applications</name> >>>>>>>>>>> <value>userb</value></property><property> >>>>>>>>>>> <name>yarn.scheduler.capacity.root.exploration.c.acl_submit_applications</name> >>>>>>>>>>> <value>userc</value></property></configuration>* >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Regards* >>>>>>>>>>> >>>>>>>>>>> *Munna* >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>> CONFIDENTIALITY NOTICE >>>> NOTICE: This message is intended for the use of the individual or >>>> entity to which it is addressed and may contain information that is >>>> confidential, privileged and exempt from disclosure under applicable law. >>>> If the reader of this message is not the intended recipient, you are hereby >>>> notified that any printing, copying, dissemination, distribution, >>>> disclosure or forwarding of this communication is strictly prohibited. If >>>> you have received this communication in error, please contact the sender >>>> immediately and delete it from your system. Thank You. >>>> >>> >>> >>> >>> -- >>> *Regards* >>> >>> *Munna* >>> >> >> >> >> -- >> *Regards* >> >> *Munna* >> > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > -- *Regards* *Munna*
