Markus, I hear you. Sometimes things just behave the way we don't expect. http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4018748
-- Christopher T. Nguyen Co-founder & CEO, Adatao <http://adatao.com> linkedin.com/in/ctnguyen On Wed, Oct 16, 2013 at 12:30 PM, Markus Losoi <[email protected]>wrote: > > Markus, I'm guessing at your config which is probably one or more workers > running on the same node as > > your master, and the same node as your spark-shell. Which is why you're > expecting the workers to be > > able to read the same relative path? > > > If that's the case, the reason it doesn't work out as you expect is > because the workers have > > different working directories. > > Yes, you are in right. The problem was caused by the use of relative paths. > (I remember starting the worker and the driver program from the same path > though.) However, the system could have been a bit more elaborate that it > didn't find the files. Now I got the impression that everything went fine > until I got unexpected results. Perhaps there was something alarming in the > flood of log messages. > > Best regards, > Markus Losoi ([email protected]) > >
