回复：Re: Re: Re: Re: how to distributed run a bash shell in spark

Hi Akhil and all My previous code has some problems,all the executors are looping and running the same command. That's not what I am expecting.previous code: val shellcompare = List(run,sort.sh) val shellcompareRDD = sc.makeRDD(shellcompare) val result = List(aggregate,result)

回复：Re: Re: Re: Re: how to distributed run a bash shell in spark

Thanks Akhil， I checked the job UI again ,my app is running concurrently in all the executors. But some of the tasks got I/O exception. I will continue inspecting on this. java.io.IOException: Failed to create local dir in