Looks like still the same while the other MR application is working fine,
On Wed, Mar 30, 2016 at 3:15 AM, Alexander Pivovarov <apivova...@gmail.com> wrote: > for small cluster set the following settings > > yarn-site.xml > > <property> > <name>yarn.scheduler.minimum-allocation-mb</name> > <value>32</value> > </property> > > > capacity-scheduler.xml > > <property> > <name>yarn.scheduler.capacity.maximum-am-resource-percent</name> > <value>0.5</value> > <description> > Maximum percent of resources in the cluster which can be used to run > application masters i.e. controls number of concurrent running > applications. > </description> > </property> > > > Probably yarn can not allocate mem for AM container. dafault value is 0.1 > and Spark AM need 896 MB (and 0.1 gives just 393 MB which is not enough) > > On Tue, Mar 29, 2016 at 2:35 PM, Vineet Mishra <clearmido...@gmail.com> > wrote: > >> Yarn seems to be running fine, I have successful MR jobs completed on the >> same, >> >> *Cluster Metrics* >> *Apps Submitted Apps Pending Apps Running Apps Completed Containers >> Running Memory Used Memory Total Memory Reserved VCores Used VCores Total >> VCores Reserved Active Nodes Decommissioned Nodes Lost Nodes Unhealthy >> Nodes Rebooted Nodes* >> *1 0 0 1 0 0 B 8 GB 0 B 0 8 0 1 0 0 0 0* >> *User Metrics for dr.who* >> *Apps Submitted Apps Pending Apps Running Apps Completed Containers >> Running Containers Pending Containers Reserved Memory Used Memory Pending >> Memory Reserved VCores Used VCores Pending VCores Reserved* >> *0 0 0 1 0 0 0 0 B 0 B 0 B 0 0 0* >> *Show entriesSearch: * >> *ID* >> *User* >> *Name* >> *Application Type* >> *Queue* >> *StartTime* >> *FinishTime* >> *State* >> *FinalStatus* >> *Progress* >> *Tracking UI* >> *application_1459287061048_0001 myhost word count MAPREDUCE root.myhost >> Tue, 29 Mar 2016 21:31:39 GMT Tue, 29 Mar 2016 21:31:59 GMT FINISHED >> SUCCEEDED * >> *History* >> >> On Wed, Mar 30, 2016 at 2:52 AM, Alexander Pivovarov < >> apivova...@gmail.com> wrote: >> >>> check resource manager and node manager logs. >>> Maybe you find smth explaining why 1 app is pending >>> >>> do you have any app run successfully? *Apps Completed is 0 on the UI* >>> >>> >>> On Tue, Mar 29, 2016 at 2:13 PM, Vineet Mishra <clearmido...@gmail.com> >>> wrote: >>> >>>> Hi Alex/Surendra, >>>> >>>> Hadoop is up and running fine and I am able to run example on the same. >>>> >>>> *Cluster Metrics* >>>> *Apps Submitted Apps Pending Apps Running Apps Completed Containers >>>> Running Memory Used Memory Total Memory Reserved VCores Used VCores Total >>>> VCores Reserved Active Nodes Decommissioned Nodes Lost Nodes Unhealthy >>>> Nodes Rebooted Nodes* >>>> *1 1 0 0 0 0 B 3.93 GB 0 B 0 4 0 1 0 0 0 0* >>>> *User Metrics for dr.who* >>>> *Apps Submitted Apps Pending Apps Running Apps Completed Containers >>>> Running Containers Pending Containers Reserved Memory Used Memory Pending >>>> Memory Reserved VCores Used VCores Pending VCores Reserved* >>>> *0 1 0 0 0 0 0 0 B 0 B 0 B 0 0 0* >>>> >>>> Any Other trace? >>>> >>>> On Wed, Mar 30, 2016 at 2:31 AM, Alexander Pivovarov < >>>> apivova...@gmail.com> wrote: >>>> >>>>> check 8088 ui >>>>> - how many cores and memory available >>>>> - how many slaves are active >>>>> >>>>> run teragen or pi from hadoop examples to make sure that yarn works >>>>> >>>>> On Tue, Mar 29, 2016 at 1:25 PM, Surendra , Manchikanti < >>>>> surendra.manchika...@gmail.com> wrote: >>>>> >>>>>> Hi Vineeth, >>>>>> >>>>>> Can you please check resource(RAM,Cores) availability in your local >>>>>> cluster, And change accordingly. >>>>>> >>>>>> Regards, >>>>>> Surendra M >>>>>> >>>>>> -- Surendra Manchikanti >>>>>> >>>>>> On Tue, Mar 29, 2016 at 1:15 PM, Vineet Mishra < >>>>>> clearmido...@gmail.com> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> While starting Spark on Yarn on local cluster(Single Node Hadoop 2.6 >>>>>>> yarn) I am facing some issues. >>>>>>> >>>>>>> As I try to start the Spark Shell it keeps on iterating in a endless >>>>>>> loop while initiating, >>>>>>> >>>>>>> *6/03/30 01:32:38 DEBUG ipc.Client: IPC Client (1782965120) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost sending #11971* >>>>>>> *16/03/30 01:32:38 DEBUG ipc.Client: IPC Client (1782965120) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost got value #11971* >>>>>>> *16/03/30 01:32:38 DEBUG ipc.ProtobufRpcEngine: Call: >>>>>>> getApplicationReport took 1ms* >>>>>>> *16/03/30 01:32:38 INFO yarn.Client: Application report for >>>>>>> application_1459260674306_0003 (state: ACCEPTED)* >>>>>>> *16/03/30 01:32:38 DEBUG yarn.Client: * >>>>>>> * client token: N/A* >>>>>>> * diagnostics: N/A* >>>>>>> * ApplicationMaster host: N/A* >>>>>>> * ApplicationMaster RPC port: -1* >>>>>>> * queue: root.thequeue* >>>>>>> * start time: 1459269797431* >>>>>>> * final status: UNDEFINED* >>>>>>> * tracking URL: >>>>>>> http://myhost:8088/proxy/application_1459260674306_0003/ >>>>>>> <http://myhost:8088/proxy/application_1459260674306_0003/>* >>>>>>> * user: myhost* >>>>>>> >>>>>>> *16/03/30 01:45:07 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost sending #338* >>>>>>> *16/03/30 01:45:07 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost got value #338* >>>>>>> *16/03/30 01:45:07 DEBUG ipc.ProtobufRpcEngine: Call: >>>>>>> getApplicationReport took 2ms* >>>>>>> *16/03/30 01:45:08 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost sending #339* >>>>>>> *16/03/30 01:45:08 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost got value #339* >>>>>>> *16/03/30 01:45:08 DEBUG ipc.ProtobufRpcEngine: Call: >>>>>>> getApplicationReport took 2ms* >>>>>>> *16/03/30 01:45:09 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost sending #340* >>>>>>> *16/03/30 01:45:09 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost got value #340* >>>>>>> *16/03/30 01:45:09 DEBUG ipc.ProtobufRpcEngine: Call: >>>>>>> getApplicationReport took 2ms* >>>>>>> *16/03/30 01:45:10 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost sending #341* >>>>>>> *16/03/30 01:45:10 DEBUG ipc.Client: IPC Client (101088744) >>>>>>> connection to myhost/192.168.1.108:8032 <http://192.168.1.108:8032> from >>>>>>> myhost got value #341* >>>>>>> *16/03/30 01:45:10 DEBUG ipc.ProtobufRpcEngine: Call: >>>>>>> getApplicationReport took 1ms* >>>>>>> >>>>>>> Any leads would be appreciated. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
