Thanks for your reply! I am running 0.20.1 in pseudo-distributed mode in Ubuntu. I want to run interactive jobs with Hadoop and trying to see if Hadoop is suitable for that purpose or not. I wonder if there is anybody using Hadoop for interactive jobs where given a query, output is returned within an acceptable amount of time. Is Hadoop meant to be used only for batch processing?
> From: [email protected] > To: [email protected] > Date: Mon, 23 Nov 2009 22:30:43 -0800 > Subject: Re: Hadoop Performance > > Hi, > > Not sure about your hadoop version, and havent done much on single m/c setup > myself. However there is a IPC improvement bug filed @ > https://issues.apache.org/jira/browse/HADOOP-2864.Thanks! > > On 11/24/09 11:22 AM, "onur ascigil" <[email protected]> wrote: > > > > I am running Hadoop on a single machine and have some questions about its > performance. > I have a simple java program that runs breadth first search on a graph > with 5 nodes. It involves several map-reduce iterations. > > I observed that, Hadoop takes too long to produce > results on such a simple job. So I attached a java profiler to my mapreduce > job > (runJar) to see what is going on. The java profiler reported several IPC > connections to ports 54310 and 54311. Each of these IPCs to Jobtracker and > HDFS takes around 10 seconds! > > First of all why are these IPCs take this long? > And I am wondering if there is anyway to improve > the performance of these IPC calls. Does Hadoop > have such a large fixed-cost ? > > I would really appreciate any comments or suggestions. > Thanks in advance, > Onur > > _________________________________________________________________ > Windows Live Hotmail: Your friends can get your Facebook updates, right from > Hotmail®. > http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_4:092009 > _________________________________________________________________ Windows Live Hotmail: Your friends can get your Facebook updates, right from Hotmail®. http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_4:092009
