Kallam Reddy created FLUME-3147:
-----------------------------------
Summary: LeaseExpiredException while storing files into HDFS
Key: FLUME-3147
URL: https://issues.apache.org/jira/browse/FLUME-3147
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: 1.7.0
Reporter: Kallam Reddy
While storing events from Kafka into HDFS as avro containers we are getting
LeaseExpiredException exception, exception trace is given below.
It is because of multiple agents are trying to writer files into a particular
folder and the counter being generated to use in the file name is matched for
couple of agents.
Problematic code is at: org.apache.flume.sink.hdfs.BucketWriter.java
Issue is with getting the counter in
org.apache.flume.sink.hdfs.BucketWriter.java, which might be atomic for one
agent but could not be atomic with multiple agents in distributed environment.
{color:red}long counter = fileExtensionCounter.incrementAndGet();
String fullFileName = fileName + "." + counter; {color}
2017-08-01 19:06:00.634 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.hdfs.AbstractHDFS
Writer.isUnderReplicated:95) - Unexpected error while checking replication
factor
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flume.sink.hdfs.AbstractHDFSWriter.getNumCurrentReplicas(AbstractHDFSWriter.java:161)
at
org.apache.flume.sink.hdfs.AbstractHDFSWriter.isUnderReplicated(AbstractHDFSWriter.java:81)
at
com.omnitracs.otda.dte.flume.sink.hdfs.BucketWriter.shouldRotate(BucketWriter.java:631)
at
com.omnitracs.otda.dte.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:565)
at
com.omnitracs.otda.dte.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:430)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
No lease on /abc/xyz.1501545600001.avro.tmp (inode 9165696): File does not
exist.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
