Thanks for your comments Hari,
I think the problem is at the time of accessing Hbase tables:
org.apache.flume.FlumeException: Could not load table, Models from Hbase
As it can't check tablename and ColumnFamilies configuration, agent can't
start. I've trying different configurations unsuccessfully:
hBaseConfig.set("hbase.master", hostname)
hBaseConfig.set("hbase.zookeeper.quorum", hostname)
hBaseConfig.set("hbase.zookeeper.property.clientPort", "2181");
Is there something wrong with this configuration?
Thanks
-----Original Message-----
From: Hari Shreedharan [mailto:[email protected]]
Sent: Wednesday, June 10, 2015 9:38 PM
To: [email protected]
Subject: Re: HBaseSink for Flume
Adding HBase and HDFS to embedded agent is non-trivial, especially with
changing versions and dependencies. It is unlikely your hack will actually work
without having everything hbase-client needs in its classpath (take a look at
how flume-ng script adds everything to its own classpath). I would suggest
running hbase classpath and adding eveyrhing from that to your app’s classpath
and trying.
Also, it looks like at some point the hbase sink is not started, not sure why.
Frankly, no one has really tested this, neither is it really supported.
Thanks,
Hari Shreedharan
> On Jun 10, 2015, at 3:13 AM, Tinte garcia, Miguel Angel
> <[email protected]> wrote:
>
> Hi,
> Yes, it looks it works including flume-ng-auth in the classpath. However,
> after configuring agent with properties it fails when tries to start:
> ...
> properties.put("hbaseSink.table", "Models");
> ...
> agent.configure(properties);
> agent.start();
> ...
> org.apache.flume.FlumeException: Could not load table, Models from
> Hbase
> SEVERE: ...Error while sending data to
> HBasejava.lang.IllegalStateException: Cannot put events unless started
> java.lang.IllegalStateException: Cannot put events unless started
> ...
>
> It looks it can't access to Hbase table defined in properties. Should I
> configure some property in the agent to access to HBase?
>
> Thanks again
>
>
> -----Original Message-----
> From: Hari Shreedharan [mailto:[email protected]]
> Sent: Saturday, June 06, 2015 12:08 AM
> To: [email protected]
> Subject: Re: HBaseSink for Flume
>
> No, you need to add flume-ng-auth module to your classpath.
>
> On Fri, Jun 5, 2015 at 2:38 AM, Tinte garcia, Miguel Angel <
> [email protected]> wrote:
>
>> Hi Hari,
>> Thanks for the answer. I've changed it to "Hbase" and this error has
>> disappeared but it seems that there is a problem with sink factory
>> when looking for HBase sink:
>>
>> Jun 05, 2015 11:25:47 AM
>> com.sun.jersey.spi.container.ContainerResponse
>> mapMappableContainerException
>> SEVERE: The exception contained within MappableContainerException
>> could not be mapped to a response, re-throwing to the HTTP container
>> java.lang.NoClassDefFoundError: org/apache/flume/auth/PrivilegedExecutor
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:191)
>> at
>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:67)
>> at
>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)
>> at
>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)
>> at
>> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(
>> A
>> bstractConfigurationProvider.java:103)
>>
>> Should I modify some class on org.apache.flume.sink package as well?
>>
>> Thanks
>>
>>
>> -----Original Message-----
>> From: Hari Shreedharan [mailto:[email protected]]
>> Sent: Thursday, June 04, 2015 12:45 AM
>> To: [email protected]
>> Subject: Re: HBaseSink for Flume
>>
>> Did you simply try “hbase” rather than HBaseSink?
>>
>> Thanks,
>> Hari Shreedharan
>>
>>
>>
>>
>>> On Jun 3, 2015, at 9:18 AM, Tinte garcia, Miguel Angel <
>> [email protected]> wrote:
>>>
>>> Hi devs,
>>> Sorry for asking in developer list but I’d like to ask you about a
>>> very
>> technical issue and probably you know better than anyone about flume code.
>> I want to include HBaseSink in EmbeddedAgent configuration but
>> currently seems not to be included, despite I’ve changed the code and
>> compiled it
>> again:
>>> org.apache.flume.FlumeException: Component type of HBaseSink is not
>>> in
>> allowed types of [AVRO, HBASE, HDFS]
>>>
>>> Could you please let me know if there is somehow to add this
>> functionality to EmbeddedAgent class?
>>>
>>> Thanks in advance
>>>
>>> This e-mail and the documents attached are confidential and intended
>> solely for the addressee; it may also be privileged. If you receive
>> this e-mail in error, please notify the sender immediately and destroy it.
>>> As its integrity cannot be secured on the Internet, the Atos group
>> liability cannot be triggered for the message content. Although the
>> sender endeavors to maintain a computer virus-free network, the
>> sender does not warrant that this transmission is virus-free and will
>> not be liable for any damages resulting from any virus transmitted.
>>>
>>> Este mensaje y los ficheros adjuntos pueden contener información
>> confidencial destinada solamente a la(s) persona(s) mencionadas
>> anteriormente y pueden estar protegidos por secreto profesional.
>>> Si usted recibe este correo electrónico por error, gracias por
>>> informar
>> inmediatamente al remitente y destruir el mensaje.
>>> Al no estar asegurada la integridad de este mensaje sobre la red,
>>> Atos
>> no se hace responsable por su contenido. Su contenido no constituye
>> ningún compromiso para el grupo Atos, salvo ratificación escrita por ambas
>> partes.
>>> Aunque se esfuerza al máximo por mantener su red libre de virus, el
>> emisor no puede garantizar nada al respecto y no será responsable de
>> cualesquiera daños que puedan resultar de una transmisión de virus.
>>
>> This e-mail and the documents attached are confidential and intended
>> solely for the addressee; it may also be privileged. If you receive
>> this e-mail in error, please notify the sender immediately and destroy it.
>> As its integrity cannot be secured on the Internet, the Atos group
>> liability cannot be triggered for the message content. Although the
>> sender endeavors to maintain a computer virus-free network, the
>> sender does not warrant that this transmission is virus-free and will
>> not be liable for any damages resulting from any virus transmitted.
>>
>> Este mensaje y los ficheros adjuntos pueden contener información
>> confidencial destinada solamente a la(s) persona(s) mencionadas
>> anteriormente y pueden estar protegidos por secreto profesional.
>> Si usted recibe este correo electrónico por error, gracias por
>> informar inmediatamente al remitente y destruir el mensaje.
>> Al no estar asegurada la integridad de este mensaje sobre la red,
>> Atos no se hace responsable por su contenido. Su contenido no
>> constituye ningún compromiso para el grupo Atos, salvo ratificación escrita
>> por ambas partes.
>> Aunque se esfuerza al máximo por mantener su red libre de virus, el
>> emisor no puede garantizar nada al respecto y no será responsable de
>> cualesquiera daños que puedan resultar de una transmisión de virus.
>>
> This e-mail and the documents attached are confidential and intended solely
> for the addressee; it may also be privileged. If you receive this e-mail in
> error, please notify the sender immediately and destroy it.
> As its integrity cannot be secured on the Internet, the Atos group liability
> cannot be triggered for the message content. Although the sender endeavors to
> maintain a computer virus-free network, the sender does not warrant that this
> transmission is virus-free and will not be liable for any damages resulting
> from any virus transmitted.
>
> Este mensaje y los ficheros adjuntos pueden contener información confidencial
> destinada solamente a la(s) persona(s) mencionadas anteriormente y pueden
> estar protegidos por secreto profesional.
> Si usted recibe este correo electrónico por error, gracias por informar
> inmediatamente al remitente y destruir el mensaje.
> Al no estar asegurada la integridad de este mensaje sobre la red, Atos no se
> hace responsable por su contenido. Su contenido no constituye ningún
> compromiso para el grupo Atos, salvo ratificación escrita por ambas partes.
> Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor no
> puede garantizar nada al respecto y no será responsable de cualesquiera daños
> que puedan resultar de una transmisión de virus.
This e-mail and the documents attached are confidential and intended solely for
the addressee; it may also be privileged. If you receive this e-mail in error,
please notify the sender immediately and destroy it.
As its integrity cannot be secured on the Internet, the Atos group liability
cannot be triggered for the message content. Although the sender endeavors to
maintain a computer virus-free network, the sender does not warrant that this
transmission is virus-free and will not be liable for any damages resulting
from any virus transmitted.
Este mensaje y los ficheros adjuntos pueden contener información confidencial
destinada solamente a la(s) persona(s) mencionadas anteriormente y pueden estar
protegidos por secreto profesional.
Si usted recibe este correo electrónico por error, gracias por informar
inmediatamente al remitente y destruir el mensaje.
Al no estar asegurada la integridad de este mensaje sobre la red, Atos no se
hace responsable por su contenido. Su contenido no constituye ningún compromiso
para el grupo Atos, salvo ratificación escrita por ambas partes.
Aunque se esfuerza al máximo por mantener su red libre de virus, el emisor no
puede garantizar nada al respecto y no será responsable de cualesquiera daños
que puedan resultar de una transmisión de virus.
