If it is just for test purpose, why not use a smaller size of data and test it on your notebook. When you go for the cluster, you can go for 50GB (I am a newbie so my thought would be very naive)
Hey Namaskara~Nalama~Guten Tag~Bonjour -- Keigu Deepak 73500 12833 www.simtree.net, [email protected] [email protected] LinkedIn: www.linkedin.com/in/deicool Skype: thumsupdeicool Google talk: deicool Blog: http://loveandfearless.wordpress.com Facebook: http://www.facebook.com/deicool "Contribute to the world, environment and more : http://www.gridrepublic.org " On Wed, Jun 15, 2016 at 10:40 PM, Sergio Fernández <[email protected]> wrote: > In theory yes... the common sense say that: > > volume / resources = time > > So more volume on the same processing resources would just take more time. > On Jun 15, 2016 6:43 PM, "spR" <[email protected]> wrote: > >> I have 16 gb ram, i7 >> >> Will this config be able to handle the processing without my ipythin >> notebook dying? >> >> The local mode is for testing purpose. But, I do not have any cluster at >> my disposal. So can I make this work with the configuration that I have? >> Thank you. >> On Jun 15, 2016 9:40 AM, "Deepak Goel" <[email protected]> wrote: >> >>> What do you mean by "EFFECIENTLY"? >>> >>> Hey >>> >>> Namaskara~Nalama~Guten Tag~Bonjour >>> >>> >>> -- >>> Keigu >>> >>> Deepak >>> 73500 12833 >>> www.simtree.net, [email protected] >>> [email protected] >>> >>> LinkedIn: www.linkedin.com/in/deicool >>> Skype: thumsupdeicool >>> Google talk: deicool >>> Blog: http://loveandfearless.wordpress.com >>> Facebook: http://www.facebook.com/deicool >>> >>> "Contribute to the world, environment and more : >>> http://www.gridrepublic.org >>> " >>> >>> On Wed, Jun 15, 2016 at 9:33 PM, spR <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> can I use spark in local mode using 4 cores to process 50gb data >>>> effeciently? >>>> >>>> Thank you >>>> >>>> misha >>>> >>> >>>
