Hi, I'd made some modifications to the spark source code in the master and reflected them to the slaves using rsync.
I followed this command: rsync -avL --progress path/to/spark-1.0.0 username@destinationhostname :path/to/destdirectory. This worked perfectly. But, I wanted to simultaneously rsync all the slaves. So, added the other slaves as following: rsync -avL --progress path/to/spark-1.0.0 username@destinationhostname :path/to/destdirectory username@slave2:path username@slave3:path and so on. But this didn't work. Anyway, for now, I did it individually for each node. Can someone give me the right syntax. Secondly, after this rsync, I find that my cluster has become tremendously slow!!! Sometimes the cluster is just shutting down. Job execution is not happening. Can someone throw some light on this aspect. thank you Karthik
