Driver is the process that manages the execution across the cluster. So say your application is a "sql query" then the system spawns a shark-cli-driver that uses spark framework, hdfs etc to execute the query and deliver result. All this happens automatically so you dont need to worry about it as a user of spark/shark framework. Just go for a bigger machine with a master.
Mayur Rustagi Ph: +919632149971 h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com https://twitter.com/mayur_rustagi On Tue, Dec 31, 2013 at 7:01 PM, guxiaobo1982 <[email protected]> wrote: > Thanks for your reply, I am new hand at Spark, does driver mean the server > where user applications are commit? > > > > ------------------ Original ------------------ > *Sender:* "Mayur Rustagi"<[email protected]>; > *Send time:* Tuesday, Dec 31, 2013 9:55 PM > *To:* "user"<[email protected]>; > *Subject:* Re: Any best practice for hardware configuration for the > masterserver in standalone cluster mode? > > Master server needs to be a little beefy as the driver runs on it. We ran > into some issues around scaling due to master servers. You can offload the > drivers to workers or other machines then the master server can be smaller. > Regards > Mayur > > Mayur Rustagi > Ph: +919632149971 > h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com > https://twitter.com/mayur_rustagi > > > > On Tue, Dec 31, 2013 at 6:48 PM, guxiaobo1982 <[email protected]> wrote: > >> Him >> >> I read the following article regarding to hardware configurations for the >> worker servers in the standalone cluster mode, but what about the master >> server? >> >> http://spark.incubator.apache.org/docs/latest/hardware-provisioning.html >> >> >> Regards, >> >> Xiaobo Gu >> >> >
