Process local implies the data is cached on the same jvm as the task, node local means its cached on the same system but not in the same jvm(on some other core perhaps). Wait modification is a tune process depends on your system configuration (memory vs disk vs network). I frankly never had to modify it..can you share your usecase that is requiring you to do that?
Mayur Rustagi Ph: +919632149971 h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com https://twitter.com/mayur_rustagi On Wed, Feb 19, 2014 at 1:59 AM, vinay Bajaj <vbajaj2...@gmail.com> wrote: > Hi > > It will be very helpful if anyone could elaborate your ideas on > spark.locality.wait and multiple locality levels (process-local, > node-local, rack-local and then any) and what is the best configuration i > can achieve by modifying this wait and what is the difference between > process local and node local. > > Thanks > Vinay Bajaj > > >
