[
https://issues.apache.org/jira/browse/FLUME-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995031#comment-13995031
]
Arun commented on FLUME-2376:
-----------------------------
Hi, for my case disk is not full. I am getting exception
Caused by: com.google.protobuf.InvalidProtocolBufferException: Message missing
required fields: type, transactionID, writeOrderID for log-183.
This file is of size around 50MB (maxFileSize) and there are data files even
after this file. mean i have files log-184, log-185 ... upto log-189
I ran integrity tool on the channel data but the file is not corrected. Tool
exits with exception because it throws IOException and not
CorruptEventException.
------------ stack trace for integrity tool --------------
Exception in thread "main" java.io.IOException: Unable to read next Transaction
from log file D:\projects\c3\flume-data\log-183 at offset 398048
at
org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:629)
at
org.apache.flume.tools.FileChannelIntegrityTool.run(FileChannelIntegrityTool.java:87)
at org.apache.flume.tools.FlumeToolsMain.run(FlumeToolsMain.java:64)
at org.apache.flume.tools.FlumeToolsMain.main(FlumeToolsMain.java:31)
Caused by: com.google.protobuf.InvalidProtocolBufferException: Message missing
required fields: type, transactionID, writeOrderID
at
com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
at
org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.buildParsed(ProtosFactory.java:2912)
at
org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.access$5200(ProtosFactory.java:2846)
at
org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader.parseDelimitedFrom(ProtosFactory.java:2804)
at
org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:196)
at
org.apache.flume.channel.file.LogFileV3$SequentialReader.doNext(LogFileV3.java:359)
at
org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:625)
... 3 more
> File Channel corrupted - java.io.IOException: Unable to read next Transaction
> from log file
> -------------------------------------------------------------------------------------------
>
> Key: FLUME-2376
> URL: https://issues.apache.org/jira/browse/FLUME-2376
> Project: Flume
> Issue Type: Bug
> Components: File Channel
> Affects Versions: v1.5.0
> Environment: windows
> Reporter: Arun
>
> I am using filechannel with dual checkpoint enabled.
> getting below error when I restart my agent.
> earlier process was abruptly killed. filechannel can not be recovered from
> state.
> Actually i am using version 1.4.0 with patch for
> https://issues.apache.org/jira/browse/FLUME-2307
> is there way to recover filechannel data.
> --------- stack trace -----------
> ERROR: Failed to initialize Log on [channel=c2]
> java.io.IOException: Unable to read next Transaction from log file
> /data1/collection/data/c2/flume-data/log-198 at offset 69616
> at
> org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:629)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.ReplayHandler.next(ReplayHandler.java:383)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:280)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at org.apache.flume.channel.file.Log.doReplay(Log.java:480)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at org.apache.flume.channel.file.Log.replay(Log.java:407)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:273)
> [my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> [my-agent-1.0-SNAPSHOT.jar:na]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> [na:1.7.0_45]
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> [na:1.7.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source) [na:1.7.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source) [na:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_45]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_45]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_45]
> Caused by: com.google.protobuf.InvalidProtocolBufferException: Message
> missing required fields: type, transactionID, writeOrderID
> at
> com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.buildParsed(ProtosFactory.java:2912)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.access$5200(ProtosFactory.java:2846)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader.parseDelimitedFrom(ProtosFactory.java:2804)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:196)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.LogFileV3$SequentialReader.doNext(LogFileV3.java:359)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> at
> org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:625)
> ~[my-agent-1.0-SNAPSHOT.jar:na]
> ... 13 common frames omitted
> Apr 28, 2014 11:43:02 PM org.apache.flume.channel.file.FileChannel
> start():284
> ERROR: Failed to start the file channel [channel=c2]
--
This message was sent by Atlassian JIRA
(v6.2#6252)
