FileNotFoundException, while file is actually available

Hi, I see a lot of exceptions like the following during our machine learning pipeline calculation. Spark version 2.0.2. Sometimes it’s just few executors that fails with this message, but the job is successful. I’d appreciate any hint you might have. Thank you. 2017-02-05 07:56:47.022

newly added Executors couldn't fetch jar files from Master

Hi I’ve got working cluster for more, than couple of weeks with 20 workers. Everything was perfect. Today I added 4 more workers and all of them couldn’t fetch jar files from master. The following means to me that master is available to worker, it is registered there and it started

Re: SVM regression in Spark

I don’t think there is such an algo. Originally SVM is for classification, but there is some twicked version that do regression, but unfortunately this is not available in apache spark, AFAIK. > On 01 Dec 2016, at 02:53, roni wrote: > > Hi Spark expert, > Can anyone

Re: Spark shuffle: FileNotFound exception

Swapnil, What do you think might be the size of the file that’s not found? For spark version below 2.0.0 there might be issues with blocks of size 2g. Is the file actually on a file system? I’d try to increase default parallelism to make sure partitions got smaller. Hope, this helps. > On