[
https://issues.apache.org/jira/browse/FLUME-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14502221#comment-14502221
]
yinghua_zh edited comment on FLUME-2682 at 4/20/15 1:54 AM:
------------------------------------------------------------
The error log as follow:
org.apache.flume.ChannelException: Commit failed due to IO error
[channel=static_log_channel]
at
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:609)
at
org.apache.flume.channel.BasicTransactionSemantics.rollback(BasicTransactionSemantics.java:168)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:402)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Usable space exhaused, only 524005376 bytes
remaining, required 524288026 bytes
at org.apache.flume.channel.file.Log.rollback(Log.java:716)
at
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:607)
... 5 more
The data log file informaiton:
total 5806724
-rw-------. 1 root root 0 Apr 19 10:24 in_use.lock
-rw-------. 1 root root 0 Apr 19 10:39 log-1
-rw-------. 1 root root 47 Apr 19 10:39 log-1.meta
-rw-------. 1 root root 0 Apr 19 10:40 log-2
-rw-------. 1 root root 47 Apr 19 10:40 log-2.meta
-rw-------. 1 root root 0 Apr 19 10:41 log-3
-rw-------. 1 root root 47 Apr 19 10:41 log-3.meta
-rw-------. 1 root root 0 Apr 19 10:42 log-4
-rw-------. 1 root root 47 Apr 19 10:42 log-4.meta
-rw-------. 1 root root 1624109033 Apr 19 10:59 log-5
-rw-------. 1 root root 47 Apr 19 10:44 log-5.meta
-rw-------. 1 root root 1624132603 Apr 19 11:16 log-6
-rw-------. 1 root root 47 Apr 19 10:59 log-6.meta
-rw-------. 1 root root 1624127574 Apr 19 11:33 log-7
-rw-------. 1 root root 47 Apr 19 11:16 log-7.meta
-rw-------. 1 root root 1073663028 Apr 19 11:45 log-8
-rw-------. 1 root root 47 Apr 19 11:33 log-8.meta
But the data dir can not be written when the sink take event from the
filechannel normally.
was (Author: yinghua_zh):
The error log as follow:
| ERROR | [SinkRunner-PollingRunner-DefaultSinkProcessor] | Unable to deliver
event. Exception follows. |
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)
org.apache.flume.ChannelException: Commit failed due to IO error
[channel=static_log_channel]
at
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:609)
at
org.apache.flume.channel.BasicTransactionSemantics.rollback(BasicTransactionSemantics.java:168)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:402)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Usable space exhaused, only 524005376 bytes
remaining, required 524288026 bytes
at org.apache.flume.channel.file.Log.rollback(Log.java:716)
at
org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doRollback(FileChannel.java:607)
... 5 more
The data log file informaiton:
total 5806724
-rw-------. 1 root root 0 Apr 19 10:24 in_use.lock
-rw-------. 1 root root 0 Apr 19 10:39 log-1
-rw-------. 1 root root 47 Apr 19 10:39 log-1.meta
-rw-------. 1 root root 0 Apr 19 10:40 log-2
-rw-------. 1 root root 47 Apr 19 10:40 log-2.meta
-rw-------. 1 root root 0 Apr 19 10:41 log-3
-rw-------. 1 root root 47 Apr 19 10:41 log-3.meta
-rw-------. 1 root root 0 Apr 19 10:42 log-4
-rw-------. 1 root root 47 Apr 19 10:42 log-4.meta
-rw-------. 1 root root 1624109033 Apr 19 10:59 log-5
-rw-------. 1 root root 47 Apr 19 10:44 log-5.meta
-rw-------. 1 root root 1624132603 Apr 19 11:16 log-6
-rw-------. 1 root root 47 Apr 19 10:59 log-6.meta
-rw-------. 1 root root 1624127574 Apr 19 11:33 log-7
-rw-------. 1 root root 47 Apr 19 11:16 log-7.meta
-rw-------. 1 root root 1073663028 Apr 19 11:45 log-8
-rw-------. 1 root root 47 Apr 19 11:33 log-8.meta
But the data dir can not be written when the sink take event from the
filechannel normally.
> The file channel is not wroking when the sink is fault for a long time.
> -----------------------------------------------------------------------
>
> Key: FLUME-2682
> URL: https://issues.apache.org/jira/browse/FLUME-2682
> Project: Flume
> Issue Type: Bug
> Components: File Channel
> Affects Versions: v1.4.0
> Reporter: yinghua_zh
>
> The flume channel type is file and the sink is avro, the net work disconnect
> for two hours when I translate a 1G log file to the server.The avso sink take
> the event from file channel and sent it failed, the channel take every evnet
> will write WALS to the log file,but the event can not been sent
> successfully,so the filechannel disk will be full.When the network back to
> normal.and take the event ,but can not write WALS to the disk,how can I
> handle it?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
