Hi, That would usually mean that the application has not been allocated the executor resources from the resource manager yet.
On 2021/09/27 21:37:30, davvy benny <davv...@gmail.com> wrote: > Hi > I am trying to run spark programmatically from eclipse with these > configurations for hadoop cluster locally > SparkConf sparkConf = new > SparkConf().setAppName("simpleTest2").setMaster("yarn") > .set("spark.executor.memory", "1g") > .set("deploy.mode", "cluster") > .set("spark.yarn.stagingDir", > "hdfs://localhost:9000/user/hadoop/") > .set("spark.shuffle.service.enabled", "false") > .set("spark.dynamicAllocation.enabled", "false") > .set("spark.cores.max", "1") > .set("spark.executor.instances","2") > .set("spark.executor.memory","500m") // > .set("spark.executor.cores","1")// > > .set("spark.yarn.nodemanager.resource.cpu-vcores","4") > .set("spark.yarn.submit.file.replication", "1") > .set("spark.yarn.jars", > "hdfs://localhost:9000/user/hadoop/davben/jars/*.jar") > > When I check on the http://localhost:8088/cluster/apps/RUNNING I can see that > my job is submitted but y terminal loops saying > 21/09/27 23:36:33 WARN YarnScheduler: Initial job has not accepted any > resources; check your cluster UI to ensure that workers are registered and > have sufficient resources > > I ve noticed that this occurs after the application of a map on my Dataset. > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org