Hi, Thanks very much for the sharing. Really interesting data.
I just have a stupid question: Your workload is “wordcount”, right ? Why “rename” would happen ?? Thanks. -chen From: Johnson MDevadoss [mailto:[email protected]] Sent: Wednesday, March 25, 2015 6:04 AM To: Dariusz Chrząścik Cc: [email protected] Subject: Re: [Openstack] [Sahara] Swift performance We had identified a problem of running MapReduce on top of Swift mainly on the objects that cannot be renamed without a data copy. Also, there are some significant impact on job completion time for large input data and latency sensitive jobs. You can refer the following research paper which talks about performance study on swift & Hadoop. http://sc14.supercomputing.org/sites/all/themes/sc14/files/archive/tech_poster/poster_files/post192s2-file2.pdf On Mar 23, 2015, at 2:17 PM, Dariusz Chrząścik <[email protected]<mailto:[email protected]>> wrote: Hello, In my Sahara deployment, I am considering using a swift as an input/output data store. However, I am wondering if swift is eligible for big data processing. Does anyone have some experiences with such configuration? Is it efficient? Can you possibly point me to any articles, reports that compare hdfs performance with swift when running Hadoop Jobs over it? I have done some research in that matter but without success. I'd be grateful for any piece of advice. Regards, Dariusz Chrząścik _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected]<mailto:[email protected]> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : [email protected] Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
