Not with hadoop but with Cassandra, i have seen 20x data locality
improvement on partitioned optimized spark jobs

Le sam. 14 avr. 2018 à 21:17, Mich Talebzadeh <mich.talebza...@gmail.com> a
écrit :

> Hi,
>
> This is a sort of your mileage varies type question.
>
> In a classic Hadoop cluster, one has data locality when each node includes
> the Spark libraries and HDFS data. this helps certain queries like
> interactive BI.
>
> However running Spark over remote storage say Isilon scaled out NAS
> instead of LOCAL HDFS becomes problematic. The full-scan Spark needs to
> do will take much longer when it is done over the network (access the
> remote Isilon storage) instead of local I/O request to HDFS.
>
> Has anyone done some comparative studies on this?
>
>
> Thanks
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>

Reply via email to