[
https://issues.apache.org/jira/browse/FLUME-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839041#comment-13839041
]
Ashish Paliwal commented on FLUME-2209:
---------------------------------------
[~hshreedharan] Got a fix for the issue. With the fix, the Sink would not get
in this hung state, and if table is created while its running, the Sink would
start like HBaseSink. Let me know and I shall upload the patch and open a
review request.
> AsyncHBaseSink will never recover if the column family does not exists for
> the first start
> -------------------------------------------------------------------------------------------
>
> Key: FLUME-2209
> URL: https://issues.apache.org/jira/browse/FLUME-2209
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Reporter: nijel
> Assignee: Ashish Paliwal
> Priority: Critical
>
> Hi,
> I am facing one issue. Initial analysis is as follows
> Using AsyncHBaseSink. During startup the configured column family does not
> exists and it is throwing exception as
> {noformat}
> 09 Oct 2013 14:45:56,691 ERROR [lifecycleSupervisor-1-2]
> (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) -
> Unable to start SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@5470be88 counterGroup:{
> name:null counters:{} } } - Exception follows.
> org.apache.flume.FlumeException: Could not start sink. Table or column family
> does not exist in Hbase.
> at
> org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:384)
> at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
> at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> After this, the sink is died and LifecycleSupervisor is trying to start again
> {noformat}
> Want to transition SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@2d7aece8 counterGroup:{
> name:null counters:{} } } from IDLE to START (failures:23)
> {noformat}
> Here it is getting exception as
> {noformat}
> 09 Oct 2013 14:49:26,043 ERROR [lifecycleSupervisor-1-2]
> (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:253) -
> Unable to start SinkRunner: {
> policy:org.apache.flume.sink.DefaultSinkProcessor@2d7aece8 counterGroup:{
> name:null counters:{} } } - Exception follows.
> java.lang.IllegalArgumentException: Please call stop before calling start on
> an old instance.
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
> at
> org.apache.flume.sink.hbase.AsyncHBaseSink.start(AsyncHBaseSink.java:344)
> at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
> at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
> at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {noformat}
> The issue i think is we are not closing the client when the exception comes.
> {noformat}
> if(fail.get()){
> sinkCounter.incrementConnectionFailedCount();
> throw new FlumeException(
> "Could not start sink. " +
> "Table or column family does not exist in Hbase.");
> } else {
> open = true;
> }
> {noformat}
> is this a issue ? or am i missing something ?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
