Raymond, This was an issue which we fixed (FLUME-1380) and will be part of Flume-1.3.0. If you need it immediately, you could clone trunk/flume-1.3.0 branch and build it. You should not see this error once you do that. Or you can wait for the next release - Flume-1.3.0, which should be released in the next few weeks or so.
Thanks Hari -- Hari Shreedharan On Thursday, October 18, 2012 at 9:32 AM, Raymond Ng wrote: > Hi > > I'm getting the following exception when restarting flume after it was killed > manually, please advice > > > 2012-10-18 17:04:33,889 INFO [conf-file-poller-0] > DefaultLogicalNodeManager.java - Starting Channel fileChannel4 > 2012-10-18 17:04:33,914 INFO [lifecycleSupervisor-1-4] > DirectMemoryUtils.java - Unable to get maxDirectMemory from VM: > NoSuchMethodException: sun.misc.VM.maxDirectMemory(null) > 2012-10-18 17:04:33,916 INFO [lifecycleSupervisor-1-4] > DirectMemoryUtils.java - Direct Memory Allocation: Allocation = 1048576, > Allocated = 0, MaxDirectMemorySize = 1908932608, Remaining = 1908932608 > 2012-10-18 17:04:34,058 INFO [lifecycleSupervisor-1-0] > MonitoredCounterGroup.java - Component type: CHANNEL, name: fileChannel1 > started > 2012-10-18 17:04:34,058 INFO [lifecycleSupervisor-1-2] > MonitoredCounterGroup.java - Component type: CHANNEL, name: fileChannel2 > started > 2012-10-18 17:04:40,835 ERROR [lifecycleSupervisor-1-1] Log.java - Failed to > initialize Log > java.io.IOException: Header 80808080 not expected value: deadbeef > at > org.apache.flume.channel.file.TransactionEventRecord.fromDataInput(TransactionEventRecord.java:136) > at > org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:378) > at > org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:88) > at org.apache.flume.channel.file.Log.replay(Log.java:251) > at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:228) > at > org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > 2012-10-18 17:04:40,836 ERROR [lifecycleSupervisor-1-1] FileChannel.java - > Failed to start the file channel > java.io.IOException: Header 80808080 not expected value: deadbeef > at > org.apache.flume.channel.file.TransactionEventRecord.fromDataInput(TransactionEventRecord.java:136) > at > org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:378) > at > org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:88) > at org.apache.flume.channel.file.Log.replay(Log.java:251) > at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:228) > at > org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:237) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > 2012-10-18 17:04:53,740 INFO [lifecycleSupervisor-1-4] > MonitoredCounterGroup.java - Component type: CHANNEL, name: fileChannel4 > started > 2012-10-18 17:04:53,740 INFO [conf-file-poller-0] > DefaultLogicalNodeManager.java - Starting Sink hdfsSink4 > 2012-10-18 17:04:53,741 INFO [conf-file-poller-0] > DefaultLogicalNodeManager.java - Starting Sink hdfsSink2 > > ..... > ..... > 2012-10-18 17:10:36,789 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] > SinkRunner.java - Unable to deliver event. Exception follows. > java.lang.IllegalStateException: Channel closed [channel=fileChannel3] > at com.google.common.base.Preconditions.checkState(Preconditions.java:145) > at > org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:266) > at > org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:118) > at com.test.CustomHDFSSink.process(CustomHDFSSink.java:428) > at > org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) > at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) > at java.lang.Thread.run(Thread.java:662) > 2012-10-18 17:10:41,790 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] > SinkRunner.java - Unable to deliver event. Exception follows. > java.lang.IllegalStateException: Channel closed [channel=fileChannel3] > at com.google.common.base.Preconditions.checkState(Preconditions.java:145) > at > org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:266) > at > org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:118) > at com.test.CustomHDFSSink.process(CustomHDFSSink.java:428) > at > org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) > at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) > at java.lang.Thread.run(Thread.java:662) > > > > -- > Rgds > Ray
