Hi, I set up a 2-node Hadoop cluster, which seemed to be working fine. Then I tried to add a third node... To do this I: - shut down the cluster - cloned the existing slave - updated dfs.replication from 2 to 3 on all nodes - added the new slave to the slaves file on the master - cleaned out the dfs.data.dir on the new slave - restarted the cluster
It still works, but is now allocating all of the map tasks to a single node. Sometimes the controlling container is on that same node, sometimes on a different one. Which one or two of my 3 nodes are involved varies from run to run, so I know all three are functional. Running the balancer insists that no nodes are under- or over-utilised, and does nothing. The number of splits, on the same data source, also seems to have been reduced from 9 to 6 for some reason. I've tried multiple restarts, reformatting my data node, and even removing the new slave and undoing my config changes, but nothing seems to work. I can only guess that I've accidentally changed something else, but I have no idea what that might be. In case it's relevant, I'm using mongo-hadoop streaming - both my source and destination are mongo collections. All 3 nodes are able to connect to the mongo database. Any suggestions on where I could start looking would be appreciated. Catherine -- Illiud Latine dici non potest
