Was providing IP address instead of FQDN. Providing FQDN helped.
Thanks,
From: Nasrulla Khan Haris
Sent: Wednesday, September 16, 2020 4:11 PM
To: dev@spark.apache.org
Subject: Spark-Locality: Hinting Spark location of the executor does not take
effect.
HI Spark developers,
If I want to hint s
Even though i provide hint of wn4's ip address
Spark schedules on wn1
2020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop]
scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0:
NODE_LOCAL, RACK_LOCAL, ANY
2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0]
cluster.YarnScheduler
Even though i provide hint of wn4's ip address
Spark schedules on wn1
2020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop]
scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0:
NODE_LOCAL, RACK_LOCAL, ANY
2020-09-17 01:21:50,039 DEBUG [dispatcher-event-loop-0]
cluster.YarnScheduler
Even though i provide hint of wn4's ip addressSpark schedules on
wn12020-09-17 01:21:50,038 DEBUG [dag-scheduler-event-loop]
scheduler.TaskSetManager: Valid locality levels for TaskSet 11.0:
NODE_LOCAL, RACK_LOCAL, ANY2020-09-17 01:21:50,039 DEBUG
[dispatcher-event-loop-0] cluster.YarnScheduler: pa
