-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71042/
-----------------------------------------------------------
(Updated July 11, 2019, 6:30 p.m.)
Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma,
Madhan Neethiraj, and Nixon Rodrigues.
Bugs: ATLAS-3321
https://issues.apache.org/jira/browse/ATLAS-3321
Repository: atlas
Description
-------
Atlas has the notion of clusterName which is used as a global namespace in
which you can define the context in which entity instances are defined and
grouped. A cluster name defines a scope of operation in atlas and is specified
in parent entity types - hive_db (Hive), hbase_namespace (HBase), fs_path (File
System), storm_topology (Storm), spark_db (Spark)
cluster name is also used to construct qualifiedName attribute in atlas -
hive_db - db_name@cluster_name
hive_table - db_name.table_name@cluster_name
hive_column - db_name.table_name.column_name@cluster_name
Cluster name works for simple environments wherein we have a single cluster and
single service instances running (single Hive or HBase service). In complex
deployments - a single cluster may host multiple instances of the same service
(two Hive services - Hive-1 and Hive-2), for such cases cluster name doesn't
help to provide context to individual service instances and metadata groupings.
So we introduce - "metadata namespace" where a single cluster can contain
multiple metadata namespaces. The qualified name for atlas will be generated
using metadata namespace name instead of cluster name.
For each hook enabled we specify the property - "atlas.metadata.namespace"
which defines the scope/context of the metadata.
Diffs (updated)
-----
addons/hbase-bridge/src/bin/import-hbase.sh b9c1a4bc0
addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
1825cd290
addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseBridge.java
1765c1848
addons/hive-bridge/src/bin/import-hive.sh 38ec3be24
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
5f8f846ae
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
76d6fe612
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
ffa56cea0
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
b3663da34
addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java
cbee7bf14
addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
d55aa5370
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/AtlasImpalaHookContext.java
1305f65a6
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java
b5fdb6dbd
addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java
4ea484f52
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java
f1d02375c
addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java
8ebb385cf
addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java
a7b9b0cd0
addons/kafka-bridge/src/bin/import-kafka.sh 4bfb8b96b
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/KafkaBridge.java
8755c9e60
addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
5397a4b0b
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
97668a3c6
common/src/main/java/org/apache/atlas/AtlasConstants.java 2b9f4118d
notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 0030276c6
Diff: https://reviews.apache.org/r/71042/diff/3/
Changes: https://reviews.apache.org/r/71042/diff/2-3/
Testing
-------
Testing in progress
Thanks,
Sarath Subramanian