Hi Tareq, This is because libhdfs will keep a bit of state data (especially if the master was connected to HDFS).
Three suggestions: 1) [Likely to work] Fork the children first, then do any HDFS actions in the master. Alternately, don't have the master do any HDFS actions; have it fork a child which does them for it. 2) [May work] Try disconnecting the master from HDFS, then forking the children. 3) [If 2 doesn't work] Have the master disconnect, then fork the children, then have the children exec. Brian On Mar 22, 2012, at 11:38 PM, Tareq Aljabban wrote: > Hi, > I'm using libhdfs C interface to access HDFS. > The application accessing HDFS has a master process that forks many > processes, and these processes try to connect to HDFS. > Upon connection, one of the processes is exiting as soon as it reaches the > hdfsConnect(). > I tried simulating this behavior by creating a separate application where > the master process forks two processes that connect to HDFS and send > requests.. this worked without any problem.. so I got confused.. what's > really causing the connection problems then? > Any insights on this is much appreciated.
