And how many resources does spark need? On Fri, Dec 9, 2016 at 4:05 PM, Vijay Srinivasaraghavan < vijikar...@yahoo.com> wrote:
> Here is the slave state info. I see marathon is registered as > "slave_public" role and is configured with "default_accepted_resource_roles" > as "*" > > "slaves":[ > { > "id":"69356344-e2c4-453d-baaf-22df4a4cc430-S0", > "pid":"slave(1)@xxx.xxx.xxx.100:5051", > "hostname":"xxx.xxx.xxx.100", > "registered_time":1481267726.19244, > "resources":{ > "disk":12099.0, > "mem":14863.0, > "gpus":0.0, > "cpus":4.0, > "ports":"[1025-2180, 2182-3887, 3889-5049, > 5052-8079, 8082-8180, 8182-32000]" > }, > "used_resources":{ > "disk":0.0, > "mem":0.0, > "gpus":0.0, > "cpus":0.0 > }, > "offered_resources":{ > "disk":0.0, > "mem":0.0, > "gpus":0.0, > "cpus":0.0 > }, > "reserved_resources":{ > > }, > "unreserved_resources":{ > "disk":12099.0, > "mem":14863.0, > "gpus":0.0, > "cpus":4.0, > "ports":"[1025-2180, 2182-3887, 3889-5049, > 5052-8079, 8082-8180, 8182-32000]" > }, > "attributes":{ > > }, > "active":true, > "version":"1.0.1" > } > ], > > Regards > Vijay > On Friday, December 9, 2016 3:48 PM, Vinod Kone <vinodk...@apache.org> > wrote: > > > How many resources does the agent register with the master? How many > resources does spark task need? > > I'm guessing marathon is not registered with "test" role so it is only > getting un-reserved resources which are not enough for spark task? > > On Fri, Dec 9, 2016 at 2:54 PM, Vijay Srinivasaraghavan < > vijikar...@yahoo.com> wrote: > > I have a standalone DCOS setup (Single node Vagrant VM running DCOS > v.1.9-dev build + Mesos 1.0.1 + Marathon 1.3.0). Both master and agent are > running on same VM. > > Resource: 4 CPU, 16GB Memory, 20G Disk > > I have created a quota using new V1 API which creates a role "test" with > resource constraints of 0.5 CPU and 1G Memory. > > When I try to deploy Spark package, Marathon receives the request but the > task is in "waiting" state since it did not receive any offers from Master > though I don't see any resource constraints from the hardware perspective. > > However, when I deleted the quota, Marathon is able to move forward with > the deployment and Spark was deployed/up and running. I could see from the > Mesos master logs that it had sent an offer to the Marathon framework. > > To debug the issue, I was trying to create a quota but this time did not > provide any CPU and Memory (0 cpu and 0 mem). After this, when I try to > deploy Spark from DCOS UI, I could see Marathon getting offer from Master > and able to deploy Spark without the need to delete the quota this time. > > Did anyone notice similar behavior? > > Regards > Vijay > > > > >