I am porting a Hadoop job to Spark - One issue is that the workers need to
read files from hdfs reading a different file based on the key or in some
cases reading an object that is expensive to serialize.
This is easy if the worker has  access to the JavaSparkContext (I am
working in Java) but this cannot be serialized -
how can a worker read from a Path - assume hdfs

Reply via email to