Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/634#issuecomment-42188695
Note I had filed SPARK-1453 to make the wait configurable, see the
discussions there. I'm fine with removing this once the other one is
configurable. I would hesitate to do it before that.
Like was already mentioned there are 2 different timeouts, this is the
second and hardcoded to 3 seconds. I'm not really sure what the difference is
between this one and the ALLOCATOR_LOOP_WAIT_COUNT. I was originally thinking
of removing this one and just making the other configurable but I haven't
investigated in depth yet.
I don't know the details of spark.locality.wait to know if it works in all
cases. I assume the spark.locality.wait would work fine as long as you have an
input from like hadoop or somewhere that tells you the locations, but for RDD's
without the locations it wouldn't work to wait?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
