I am running Hadoop on a single machine and have some questions about its
performance.
I have a simple java program that runs breadth first search on a graph
with 5 nodes. It involves several map-reduce iterations.
I observed that, Hadoop takes too long to produce
results on such a simple job. So I attached a java profiler to my mapreduce job
(runJar) to see what is going on. The java profiler reported several IPC
connections to ports 54310 and 54311. Each of these IPCs to Jobtracker and
HDFS takes around 10 seconds!
First of all why are these IPCs take this long?
And I am wondering if there is anyway to improve
the performance of these IPC calls. Does Hadoop
have such a large fixed-cost ?
I would really appreciate any comments or suggestions.
Thanks in advance,
Onur
_________________________________________________________________
Windows Live Hotmail: Your friends can get your Facebook updates, right from
Hotmail®.
http://www.microsoft.com/middleeast/windows/windowslive/see-it-in-action/social-network-basics.aspx?ocid=PID23461::T:WLMTAGL:ON:WL:en-xm:SI_SB_4:092009
