Is it receiving any data? If so, then it must be listening. Alternatively, to test these theories, you can locally running a spark standalone cluster (one node standalone cluster in local machine), and submit your app in client mode on that to see whether you are seeing the process listening on 9999 or not.
On Mon, Aug 10, 2015 at 12:43 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote: > I've verified all the executors and I don't see a process listening on the > port. However, the application seem to show as running in the yarn UI > > On Mon, Aug 10, 2015 at 11:56 AM, Tathagata Das <t...@databricks.com> > wrote: > >> In yarn-client mode, the driver is on the machine where you ran the >> spark-submit. The executors are running in the YARN cluster nodes, and the >> socket receiver listening on port 9999 is running in one of the executors. >> >> On Mon, Aug 10, 2015 at 11:43 AM, Mohit Anchlia <mohitanch...@gmail.com> >> wrote: >> >>> I am running as a yarn-client which probably means that the program that >>> submitted the job is where the listening is also occurring? I thought that >>> the yarn is only used to negotiate resources in yarn-client master mode. >>> >>> On Mon, Aug 10, 2015 at 11:34 AM, Tathagata Das <t...@databricks.com> >>> wrote: >>> >>>> If you are running on a cluster, the listening is occurring on one of >>>> the executors, not in the driver. >>>> >>>> On Mon, Aug 10, 2015 at 10:29 AM, Mohit Anchlia <mohitanch...@gmail.com >>>> > wrote: >>>> >>>>> I am trying to run this program as a yarn-client. The job seems to be >>>>> submitting successfully however I don't see any process listening on this >>>>> host on port 9999 >>>>> >>>>> >>>>> https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/streaming/JavaRecoverableNetworkWordCount.java >>>>> Active Jobs (2)Job IdDescriptionSubmittedDurationStages: >>>>> Succeeded/TotalTasks (for all stages): Succeeded/Total1foreachRDD at >>>>> JavaRecoverableNetworkWordCount.java:112 >>>>> <http://ec2-52-25-118-171.us-west-2.compute.amazonaws.com:8088/proxy/application_1438820875993_0007/jobs/job?id=1>2015/08/10 >>>>> 13:27:3651 s0/2 >>>>> 0/2 >>>>> 0start at JavaRecoverableNetworkWordCount.java:152 >>>>> <http://ec2-52-25-118-171.us-west-2.compute.amazonaws.com:8088/proxy/application_1438820875993_0007/jobs/job?id=0>2015/08/10 >>>>> 13:27:3551 s0/2 >>>>> 0/70 >>>>> >>>> >>>> >>> >> >
