[
https://issues.apache.org/jira/browse/ATLAS-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626940#comment-16626940
]
t oo commented on ATLAS-2889:
-----------------------------
Plus ORC and Parquet. I assume this feature will run s3 get-object-tag under
the hood. btw, do you have one example of the atlasentity s3 json u pushed to
the topic with tags array? I got error about objectid can't be null
> S3 object tag import hook
> -------------------------
>
> Key: ATLAS-2889
> URL: https://issues.apache.org/jira/browse/ATLAS-2889
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core, atlas-intg
> Affects Versions: 1.1.0
> Reporter: t oo
> Priority: Major
>
> https://issues.apache.org/jira/browse/ATLAS-2708 introduced support for s3
> type.
>
> From the comments in the jira, there are some external steps needed to import
> object tags. This Jira is about creating the E2E functionality within Atlas.
> ie functionality in UI or an API that when supplied with argument of an s3
> bucket/slice then Atlas automatically imports all the object names/tags
> recursively from under that s3 bucket/slice.
>
> "It doesn't do it automatically through a listener like the hive hook. We do
> it via lambda functions, triggered, say, on the creation of S3 object or
> pseudodirectory or bucket. We package up the info into AtlasEntities and
> then publish to the ATLAS_HOOK kafka topic.
> You have to create your own Lambda code that creates AtlasEntities on the fly
> as trigerred by Lambda Function(on changes made to s3 object) and then push
> to Kafka Queue. This particular functionality is not part of Atlas tool as of
> now."
> cc: [~barbara] [~ayushmnnit]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)