Can anyone please help me here. On Tue, May 28, 2019 at 2:45 PM Behroz Sikander <behro...@gmail.com> wrote:
> Hello, > I have a client application which uses HDFS API to push data. My HDFS is > setup in HA mode. > My client application has certain timeout requirements e.g. an upload > shouldn't take longer than 10 seconds otherwise the operation is considered > a failure. > > Sometimes, I notice the following exceptions in my code > > 2019-05-23 14:13:31,356 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []: > Exception in createBlockOutputStream > java.io.IOException: Got error, status message , ack with firstBadLink as > xx.xx.xx.xx:50010 > at > org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtoUtil.checkBlockOpStatus(DataTransferProtoUtil.java:142) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1359) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1262) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)2019-05-23 > 14:13:31,357 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []: Abandoning > BP-1672040070-127.0.0.1-1527078068582:blk_1073742599_17752019-05-23 > 14:13:31,384 INFO Thread-11 org.apache.hadoop.hdfs.DFSClient []: Excluding > datanode > DatanodeInfoWithStorage[xx.xx.xx.xx:50010,DS-4be6740a-bad8-438b-99d6-fbc50d7760dd,DISK] > > > Even though the upload succeeds but takes around *20 seconds*. > > I am interested in the configuration property which controls this timeout, so > I can adjust it > > according to my client's requirements. Or is there any configuration which > says > > if the operation is not completed within X seconds, then HDFS API throws an > exception? > > Can it be ipc.client.connect.timeout? > > > Thanks. > > Behroz > >
