GitHub user bwzhang2011 opened a pull request:
https://github.com/apache/flume/pull/27
SpillableMemoryChannel nullpointerException during recoverying for sink
configuration.
During some recovery testing and also testing for SpillableMemoryChannel. I
encountered some problem like this (which use 1.7 snapshot)
2015-11-12 10:58:00,663 (SinkRunner-PollingRunner-DefaultSinkProcessor)
[ERROR -
org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:221)]
Failed to commit transaction. Transaction rolled back.
java.lang.IllegalStateException: Channel closed [channel=ch_transaction].
Due to java.lang.NullPointerException: null
at
org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:345)
at
org.apache.flume.channel.SpillableMemoryChannel.getOverflowTx(SpillableMemoryChannel.java:796)
at
org.apache.flume.channel.SpillableMemoryChannel.access$600(SpillableMemoryChannel.java:51)
at
org.apache.flume.channel.SpillableMemoryChannel$SpillableMemoryTransaction.doTake(SpillableMemoryChannel.java:354)
at
org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
at
org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
at
org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:183)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:299)
at
org.apache.flume.channel.SpillableMemoryChannel.start(SpillableMemoryChannel.java:768)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
2015-11-12 10:58:00,664 (SinkRunner-PollingRunner-DefaultSinkProcessor)
[ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
java.lang.IllegalStateException: Channel closed [channel=ch_transaction].
Due to java.lang.NullPointerException: null
at
org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:345)
at
org.apache.flume.channel.SpillableMemoryChannel.getOverflowTx(SpillableMemoryChannel.java:796)
at
org.apache.flume.channel.SpillableMemoryChannel.access$600(SpillableMemoryChannel.java:51)
at
org.apache.flume.channel.SpillableMemoryChannel$SpillableMemoryTransaction.doTake(SpillableMemoryChannel.java:354)
at
org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
at
org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
at
org.apache.flume.sink.elasticsearch.ElasticSearchSink.process(ElasticSearchSink.java:183)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
we can simply reproduce such as follows:
1. start flume with channel and sink configuration
2. run some time to let such deliver data from source to sink destination
e.g elastic search
3. change the conf configuration and make the sink node to be some wrong
address and then
it would show connection refused for the sink
4. delivering data to be continue as usual like 2nd step for some time
5. recovery the sink address to the be right one and watch whether the
whole flow could be recovered as usual. and you will get the stack trace as
above.
By the way, I don't know why it continue to show the nullpointerException
by creating the createTransaction while recovering.
hope flume team could pay more attention to the stability of channel e.s.p
for the SpillableMemoryChannel one and better improve the exception control
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/flume flume-1.7
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/27.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #27
----
commit f1af0e96b525af5ce91bdcb8ad41da7da2ebba20
Author: Hari Shreedharan <[email protected]>
Date: 2013-08-22T06:52:15Z
FLUME-2174. Integration of morphline solr sink puts Lucene/Solr
dependencies on the default classpath
(Roman Shaposhnik via Hari Shreedharan)
commit 56275cb225e5ed20952e26087ea6a005cdcd9e77
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-12T00:18:22Z
FLUME-2184. flume-ng-morphline-solr-sink build failing due to incorrect
hadoop-common dependency declaration
(Jagat Singh via Hari Shreedharan)
commit dee057863602c28bf3a109139a1ce1bb8286af14
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-12T00:45:05Z
FLUME-2172. Update protocol buffer from 2.4.1 to 2.5.0
(Roshan Naik via Hari Shreedharan)
commit 9ef11916c8f3f7922b673f72c2ae6a410b26b48d
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-16T22:38:13Z
FLUME-2185. Upgrade morphlines to 0.7.0
(Wolfgang Hoschek via Hari Shreedharan)
commit 30d9d56c13f7a14606c9baa31dee15de16ebe9ba
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-20T04:53:55Z
FLUME-2189. Add support for IP filtering on AvroSource
(Ted Malaska via Hari Shreedharan)
commit 2cecd8f7479a4c510b2cae68738563ec982c92de
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-27T20:21:48Z
FLUME-2190. Add a source capable of feeding off of the Twitter Streaming API
(Roman Shaposhnik via Hari Shreedharan)
commit eb99bb41ea813ab9cdd3e9ec76d54bd6d3628a82
Author: Mike Percy <[email protected]>
Date: 2013-09-28T00:01:41Z
FLUME-2182. Spooling Directory Source will not ingest data completely when
a wide character appears at the edge of a buffer
(Sven Meys via Mike Percy)
commit e272d11295c55abab76412f28a80be00ef69c04e
Author: Mike Percy <[email protected]>
Date: 2013-09-28T04:48:05Z
FLUME-2198. Avro Source should disable itself if ipFilterRules contains
invalid rules
(Hari Shreedharan via Mike Percy)
commit 1e4a074311fd8c682196e0622978fdc66f921bab
Author: Hari Shreedharan <[email protected]>
Date: 2013-09-28T16:35:03Z
FLUME-2052. Spooling directory source should be able to replace or ignore
malformed characters
(Mike Percy via Hari Shreedharan)
commit 46e365dcb6504f7cbefa084c8a66a4c2d28a4854
Author: Mike Percy <[email protected]>
Date: 2013-10-04T00:25:57Z
FLUME-2202. AsyncHBaseSink should coalesce increments to reduce RPC
roundtrips
(Hari Shreedharan via Mike Percy)
commit 5bfed347df4ea63dac8bb90016044af806003819
Author: Arvind Prabhakar <[email protected]>
Date: 2013-10-04T01:25:02Z
FLUME-2191. HDFS Minicluster tests failing after protobuf upgrade.
(Hari Shreedharan via Arvind Prabhakar)
commit ea0a80bc27ee0c7b8630ea402a6e3fac7ff1b64c
Author: Mike Percy <[email protected]>
Date: 2013-10-08T21:00:31Z
FLUME-2200. HTTP Source should use "port" param for both SSL & cleartext
(Hari Shreedharan via Mike Percy)
commit e278a3f1f45c8dd38a00a17a1ffe0ab342a95520
Author: Mike Percy <[email protected]>
Date: 2013-10-09T01:12:10Z
FLUME-2208. Jetty's default SocketSelector leaks File descriptors
(Hari Shreedharan via Mike Percy)
commit 8742d591770e2bf1505c9228e36a26479350b8f8
Author: Mike Percy <[email protected]>
Date: 2013-10-10T01:49:31Z
FLUME-1666. Syslog source strips timestamp and hostname from log message
body
(Jeff Lord via Mike Percy)
commit 4416d5b69a5dca21564a07f37f559a3e435cce74
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-12T04:26:40Z
FLUME-2212. Upgrade to Morphlines-0.8.0
(Wolfgang Hoschek via Hari Shreedharan)
commit 8ea4789326f8aa56b1947bc9e826acc8ef49d6f5
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-14T22:56:54Z
FLUME-2159. Remove TestNettyAvroRpcClient.spinThreadsCrazily.
(Roshan Naik via Hari Shreedharan)
commit b1f122f23d05aa55492941b11a4a1f7b5a7b5dfe
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-16T21:24:13Z
FLUME-2213. MorphlineInterceptor should share metric registry across
threads for better (aggregate) reporting
(Wolfgang Hoschek via Hari Shreedharan)
commit 443ea3b8929ec34e5b2c8645aafe0cb397d273f6
Author: Roshan Naik <[email protected]>
Date: 2013-10-21T19:06:51Z
FLUME-2064: Typo/Grammar in flume main user doc under Scribe
(Ashish Paliwal via Roshan Naik)
commit f6bbc5c5af45cb12751a379c5e3060629365a6fd
Author: Mike Percy <[email protected]>
Date: 2013-10-21T19:04:22Z
FLUME-1666. Oops, forgot new test in previous commit
commit 933617039b81e1ade6593d96ef39508d78d8a518
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-25T00:47:53Z
FLUME-2210. UnresolvedAddressException when using multiple hostNames in
Elasticsearch sink configuration
(Dib Ghosh via Hari Shreedharan)
commit 9b9c0831ee15afe118e5ab53872ea52989fa4275
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-25T05:18:37Z
FLUME-2192. AbstractSinkProcessor stop incorrectly calls start
(Jeremy Karlson via Hari Shreedharan)
commit 93de6b83758a5b517d16cb4b727a6d09ea5c824d
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-31T01:30:49Z
FLUME-1851. Fix grammatical error in Flume User Guide.
(Ashish Paliwal via Hari Shreedharan)
commit e1dbe0eb1179af81c46a7774a3bfc0efd4705ce2
Author: Hari Shreedharan <[email protected]>
Date: 2013-10-31T06:13:09Z
FLUME-2206. ElasticSearchSink ttl field modification to mimic Elasticsearch
way of specifying TTL
(Dib Ghosh via Hari Shreedharan)
commit 7fc23d7d41757ce75058cead963b5bd54c395727
Author: Jarek Jarcec Cecho <[email protected]>
Date: 2013-10-31T19:54:10Z
FLUME-2229. Backoff period gets reset too often in OrderSelector
(Hari Shreedharan via Jarek Jarcec Cecho)
commit b7ef76b16bc4b585341a54876a9272dc5c6e758c
Author: Roshan Naik <[email protected]>
Date: 2013-11-07T03:30:42Z
FLUME-2065. Regex Extractor Interceptor config agent name inconsistent with
rest of docs
(Ashish Paliwal via Roshan Naik)
commit f53d62a359a3a74d0c0fe763ca2f547a188e612e
Author: Roshan Naik <[email protected]>
Date: 2013-11-07T19:42:05Z
FLUME-2233. MemoryChannel lock contention on every put due to
bytesRemaining Semaphore
(Hari Shreedharan via Roshan Naik)
commit b17626f72ec4e9afd99bf36a5898dfdb4ea13711
Author: Roshan Naik <[email protected]>
Date: 2013-11-07T21:07:24Z
FLUME-2231. Add details in Flume Ganglia config in User Guide
(Ashish Paliwal via Roshan Naik)
commit ad612c28bec3845775e12d6bc51ef724e6f78f06
Author: Jarek Jarcec Cecho <[email protected]>
Date: 2013-11-07T22:53:04Z
FLUME-2235. idleFuture should be cancelled at the start of append
(Hari Shreedharan via Jarek Jarcec Cecho)
commit d2d7d7f968c2eb78eda46822fd503d8772d69d7a
Author: Mike Percy <[email protected]>
Date: 2013-12-05T20:58:03Z
FLUME-2255. Correctly handle ChannelExceptions in SpoolingDirectorySource
(Hari Shreedharan via Mike Percy)
commit e371d565bd179ab000b5cfd9d771b5b38fc19cc3
Author: Hari Shreedharan <[email protected]>
Date: 2013-12-09T23:12:47Z
FLUME-2262. Log4j Appender should use timeStamp field not getTimestamp,
which was not available in older log4j versions.
(Brock Noland via Hari Shreedharan)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
