[
https://issues.apache.org/jira/browse/FLUME-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleksiy Sayankin updated FLUME-2553:
------------------------------------
Description:
1. Create hbase.conf file /root/hbase.conf:
hbase-agent.sources = tail
hbase-agent.sinks = sink1
hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exec
hbase-agent.sources.tail.command = tail -F /root/demo.txt
hbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = demo
hbase-agent.sinks.sink1.columnFamily = cf
hbase-agent.sinks.sink1.serializer =
org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp
hbase-agent.sinks.sink1.serializer.rowPrefix = 1
hbase-agent.sinks.sink1.serializer.suffix = timestamp
hbase-agent.channels.ch1.type=memory
2. Create demo file: /root/demo.txt:
value1
value2
value3
value4
value5
value6
3. Create table demo in hbase-0.98
hbase shell
create 'demo', 'cf'
4. Start flume
flume-ng agent --conf-file /root/hbase.conf --name hbase-agent
-Dflume.root.logger=INFO,console
Get error:
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=:/opt/mapr/hadoop/hadoop-2.4.1/lib/native:/opt/mapr/hadoop/hadoop-2.4.1/lib/native
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-504.el6.x86_64
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.name=root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:5181 sessionTimeout=90000
watcher=hconnection-0x52dc5605, quorum=localhost:5181, baseZNode=/hbase
14/11/14 11:08:34 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x52dc5605 connecting to ZooKeeper
ensemble=localhost:5181
14/11/14 11:08:34 INFO zookeeper.Login: successfully logged in.
14/11/14 11:08:34 INFO client.ZooKeeperSaslClient: Client will use
SIMPLE-SECURITY as SASL mechanism.
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/0:0:0:0:0:0:0:1:5181. Will attempt to SASL-authenticate using
Login Context section 'Client_simple'
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/0:0:0:0:0:0:0:1:5181, initiating session
14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/0:0:0:0:0:0:0:1:5181, sessionid = 0x149aeb28c60003b,
negotiated timeout = 40000
14/11/14 11:08:34 INFO Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Monitored counter
group for type: SINK, name: sink1: Successfully registered new MBean.
14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Component type:
SINK, name: sink1 started
14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
transaction.Transaction rolled back.
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
transaction.Transaction rolled back.
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor"
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
was:
1. Create hbase.conf file /root/hbase.conf:
hbase-agent.sources = tail
hbase-agent.sinks = sink1
hbase-agent.channels = ch1
hbase-agent.sources.tail.type = exec
hbase-agent.sources.tail.command = tail -F /root/demo.txt
hbase-agent.sources.tail.channels = ch1
hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
hbase-agent.sinks.sink1.channel = ch1
hbase-agent.sinks.sink1.table = demo
hbase-agent.sinks.sink1.columnFamily = cf
hbase-agent.sinks.sink1.serializer =
org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
hbase-agent.sinks.sink1.serializer.payloadColumn = col1
hbase-agent.sinks.sink1.serializer.keyType = timestamp
hbase-agent.sinks.sink1.serializer.rowPrefix = 1
hbase-agent.sinks.sink1.serializer.suffix = timestamp
hbase-agent.channels.ch1.type=memory
2. Create demo file: /root/demo.txt:
value1
value2
value3
value4
value5
value6
3. Create table demo in hbase-0.98
hbase shell
create 'demo', 'cf'
4. Start flume
flume-ng agent --conf-file /root/hbase.conf --name hbase-agent
-Dflume.root.logger=INFO,console
Get error:
14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
transaction.Transaction rolled back.
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
transaction.Transaction rolled back.
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor"
java.lang.NoSuchMethodError:
org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
at
org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
at
org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
> Error while saving data to hbase java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-2553
> URL: https://issues.apache.org/jira/browse/FLUME-2553
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.5.0.1
> Environment: Centos 6.6
> Reporter: Oleksiy Sayankin
> Fix For: v1.5.0.1
>
>
> 1. Create hbase.conf file /root/hbase.conf:
> hbase-agent.sources = tail
> hbase-agent.sinks = sink1
> hbase-agent.channels = ch1
> hbase-agent.sources.tail.type = exec
> hbase-agent.sources.tail.command = tail -F /root/demo.txt
> hbase-agent.sources.tail.channels = ch1
> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
> hbase-agent.sinks.sink1.channel = ch1
> hbase-agent.sinks.sink1.table = demo
> hbase-agent.sinks.sink1.columnFamily = cf
> hbase-agent.sinks.sink1.serializer =
> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
> hbase-agent.sinks.sink1.serializer.keyType = timestamp
> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
> hbase-agent.sinks.sink1.serializer.suffix = timestamp
> hbase-agent.channels.ch1.type=memory
> 2. Create demo file: /root/demo.txt:
> value1
> value2
> value3
> value4
> value5
> value6
> 3. Create table demo in hbase-0.98
> hbase shell
> create 'demo', 'cf'
> 4. Start flume
> flume-ng agent --conf-file /root/hbase.conf --name hbase-agent
> -Dflume.root.logger=INFO,console
> Get error:
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
> environment:java.library.path=:/opt/mapr/hadoop/hadoop-2.4.1/lib/native:/opt/mapr/hadoop/hadoop-2.4.1/lib/native
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
> environment:java.io.tmpdir=/tmp
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
> environment:java.compiler=<NA>
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client
> environment:os.version=2.6.32-504.el6.x86_64
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.name=root
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
> 14/11/14 11:08:34 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=localhost:5181 sessionTimeout=90000
> watcher=hconnection-0x52dc5605, quorum=localhost:5181, baseZNode=/hbase
> 14/11/14 11:08:34 INFO zookeeper.RecoverableZooKeeper: Process
> identifier=hconnection-0x52dc5605 connecting to ZooKeeper
> ensemble=localhost:5181
> 14/11/14 11:08:34 INFO zookeeper.Login: successfully logged in.
> 14/11/14 11:08:34 INFO client.ZooKeeperSaslClient: Client will use
> SIMPLE-SECURITY as SASL mechanism.
> 14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Opening socket connection to
> server localhost/0:0:0:0:0:0:0:1:5181. Will attempt to SASL-authenticate
> using Login Context section 'Client_simple'
> 14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Socket connection established to
> localhost/0:0:0:0:0:0:0:1:5181, initiating session
> 14/11/14 11:08:34 INFO zookeeper.ClientCnxn: Session establishment complete
> on server localhost/0:0:0:0:0:0:0:1:5181, sessionid = 0x149aeb28c60003b,
> negotiated timeout = 40000
> 14/11/14 11:08:34 INFO Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Monitored
> counter group for type: SINK, name: sink1: Successfully registered new MBean.
> 14/11/14 11:08:35 INFO instrumentation.MonitoredCounterGroup: Component type:
> SINK, name: sink1 started
> 14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
> transaction.Transaction rolled back.
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
> at
> org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
> at
> org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
> at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:745)
> 14/11/14 09:52:33 ERROR hbase.HBaseSink: Failed to commit
> transaction.Transaction rolled back.
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
> at
> org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
> at
> org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
> at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:745)
> Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor"
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.client.Increment.setWriteToWAL(Z)Lorg/apache/hadoop/hbase/client/Increment;
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:408)
> at org.apache.flume.sink.hbase.HBaseSink$4.run(HBaseSink.java:391)
> at
> org.apache.flume.sink.hbase.HBaseSink.runPrivileged(HBaseSink.java:427)
> at
> org.apache.flume.sink.hbase.HBaseSink.putEventsAndCommit(HBaseSink.java:391)
> at org.apache.flume.sink.hbase.HBaseSink.process(HBaseSink.java:344)
> at
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
> at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
