Re: Output of Spark interpreter dramatically different from spark-shell

2017-05-08 Thread Alexander Bezzubov
Thanks for update on yarn-client. I get the difference between driver and executor, but why is there a difference between output in spark local mode running in spark-shell and Zeppelin on the same machine though? As a user, I would expect output to be the same, in case when executor runs on the s

Re: Output of Spark interpreter dramatically different from spark-shell

2017-05-08 Thread Jeff Zhang
This is expected. And I believe you are using local mode. You should be able to get the same output in yarn-client mode. println function is invoked on executor side, while IMain of spark repl only capture the output of driver. The reason you see the output of println in spark-shell is that the exe