Repository: atlas Updated Branches: refs/heads/master 620943ed8 -> 540129f5c
ATLAS-2603: [Docs] Add Atlas Kafka Bridge Documents Signed-off-by: Madhan Neethiraj <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/atlas/repo Commit: http://git-wip-us.apache.org/repos/asf/atlas/commit/c9924fdc Tree: http://git-wip-us.apache.org/repos/asf/atlas/tree/c9924fdc Diff: http://git-wip-us.apache.org/repos/asf/atlas/diff/c9924fdc Branch: refs/heads/master Commit: c9924fdc7bc9717455e8fb484332a24af062c302 Parents: 620943e Author: rmani <[email protected]> Authored: Wed Apr 25 10:28:12 2018 -0700 Committer: Madhan Neethiraj <[email protected]> Committed: Wed Apr 25 10:33:09 2018 -0700 ---------------------------------------------------------------------- docs/src/site/twiki/Bridge-Kafka.twiki | 34 +++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/atlas/blob/c9924fdc/docs/src/site/twiki/Bridge-Kafka.twiki ---------------------------------------------------------------------- diff --git a/docs/src/site/twiki/Bridge-Kafka.twiki b/docs/src/site/twiki/Bridge-Kafka.twiki new file mode 100644 index 0000000..7cdd548 --- /dev/null +++ b/docs/src/site/twiki/Bridge-Kafka.twiki @@ -0,0 +1,34 @@ +---+ Kafka Atlas Bridge + +---++ Kafka Model +The default Kafka model includes the following types: + * Entity types: + * kafka_topic + * super-types: !DataSet + * attributes: name, owner, description, type, classifications, term, clustername, topic , partitionCount, qualifiedName + +The entities are created and de-duped using unique qualified name. They provide namespace and can be used for querying as well: + * topic.qualifiedName - <topic>@<clusterName> + + +---++ Setup + binary files are present in apache-atlas-<release-vesion>-SNAPSHOT-kafka-hook.tar.gz + Copy apache-atlas-kafka-hook-<release-verion>-SNAPSHOT/hook/kafka folder to <atlas package>/hook/ directory + Copy apache-atlas-kafka-hook-<release-verion>-SNAPSHOT/hook-bin folder to <atlas package>/hook-bin/ directory + + * Copy <atlas-conf>/atlas-application.properties to the Kafka conf directory. +---++ Importing Kafka Metadata +org.apache.atlas.Kafka.bridge.KafkaBridge imports the Kafka metadata into Atlas using the model defined above. import-kafka.sh command can be used to facilitate this. + <verbatim> + Usage 1: <atlas package>/hook-bin/import-kafka.sh + Usage 2: <atlas package>/hook-bin/import-kafka.sh [-n <namespace regex> OR --namespace <namespace regex >] [-t <table regex > OR --table <table regex>] + Usage 3: <atlas package>/hook-bin/import-kafka.sh [-f <filename>] + File Format: + topic1 + topic2 + topic3 + </verbatim> + +The logs are in <atlas package>/logs/import-kafka.log + +Refer [[Configuration][Configuration]] for notification related configurations
