[jira] [Commented] (ATLAS-1805) Provide an Atlas hook to send Hbase Namespace/Table/column family metadata to Atlas

[Nixon Rodrigues (JIRA)]

    [ 
https://issues.apache.org/jira/browse/ATLAS-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145087#comment-16145087
 ] 

Nixon Rodrigues commented on ATLAS-1805:
----------------------------------------

[~cuijunheng]
Enable HBase Hook for Atlas

* cd  hbase/bin

ADD  HBase Atlas hook to the HBASE classpath.

* cat hbase
{code}
#!/bin/bash

BIGTOP_DEFAULTS_DIR=${BIGTOP_DEFAULTS_DIR-/etc/default}
[ -n "${BIGTOP_DEFAULTS_DIR}" -a -r ${BIGTOP_DEFAULTS_DIR}/hbase ] && . 
${BIGTOP_DEFAULTS_DIR}/hbase

if [ -d "atlas/hook/hbase" ]; then
  if [ -z "${HBASE_CLASSPATH}" ]; then
    export HBASE_CLASSPATH=atlas/hook/hbase/*
  else
    export HBASE_CLASSPATH=${HBASE_CLASSPATH}:atlas/hook/hbase/*
  fi
fi

export ATLAS_HOME=${ATLAS_HOME:-~/atlas}
export HBASE_HOME=${HBASE_HOME:-~/hbase}
export HADOOP_HOME=${HADOOP_HOME:-~/hadoop}
export HADOOP_CONF=${HADOOP_HOME}/conf
export ZOOKEEPER_HOME=${ZOOKEEPER_HOME:~/zookeeper}
export 
HBASE_CLASSPATH=$HADOOP_CONF:$HADOOP_HOME/*:$HADOOP_HOME/lib/*:$ZOOKEEPER_HOME/*:$HBASE_CLASSPATH
export HBASE_OPTS="-Dhdp.version=${HDP_VERSION} ${HBASE_OPTS}"
exec hbase/bin/hbase.distro "$@"

{code}


3)  create   atlas-application.properties in hbase/conf folder

* cat atlas-application.properties
{code}
atlas.authentication.method.kerberos=false
atlas.cluster.name=cl1
atlas.hook.hbase.keepAliveTime=10
atlas.hook.hbase.maxThreads=5
atlas.hook.hbase.minThreads=5
atlas.hook.hbase.numRetries=3
atlas.hook.hbase.queueSize=1000
atlas.hook.hbase.synchronous=false
atlas.jaas.KafkaClient.loginModuleControlFlag=required
atlas.jaas.KafkaClient.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.KafkaClient.option.keyTab=/etc/security/keytabs/hbase.service.keytab
atlas.jaas.KafkaClient.option.principal=hbase/_h...@example.com
atlas.jaas.KafkaClient.option.serviceName=cstm-kafka
atlas.jaas.KafkaClient.option.storeKey=True
atlas.jaas.KafkaClient.option.useKeyTab=True
atlas.kafka.bootstrap.servers=<kafka_host>:6667
atlas.kafka.hook.group.id=atlas
atlas.kafka.sasl.kerberos.service.name=cstm-kafka
atlas.kafka.security.protocol=PLAINTEXTSASL
atlas.kafka.zookeeper.connect=<zookeeper_host>:2181
atlas.kafka.zookeeper.connection.timeout.ms=30000
atlas.kafka.zookeeper.session.timeout.ms=60000
atlas.kafka.zookeeper.sync.time.ms=20
atlas.notification.create.topics=True
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.rest.address=http://<atlas_server>:21000
{code}


4) Adding the Atlas Hook to Hbase.
In Hbase Site.xml

{code} 
hbase.coprocessor.master.classes=org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
 {code}

 This is in addition to what is there in the property.

> Provide an Atlas hook to send Hbase Namespace/Table/column family metadata to 
> Atlas
> -----------------------------------------------------------------------------------
>
>                 Key: ATLAS-1805
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1805
>             Project: Atlas
>          Issue Type: Bug
>          Components: atlas-intg
>            Reporter: Ramesh Mani
>            Assignee: Ramesh Mani
>         Attachments: 
> 0001-ATLAS-1805-Provide-an-Atlas-hook-to-send-Hbase-Names.patch
>
>
> Provide an Atlas hook to send Hbase Namespace/Table/column family metadata to 
> Atlas



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)