[ 
https://issues.apache.org/jira/browse/ATLAS-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16626940#comment-16626940
 ] 

t oo commented on ATLAS-2889:
-----------------------------

Plus ORC and Parquet. I assume this feature will run s3 get-object-tag under 
the hood. btw, do you have one example of the atlasentity s3 json u pushed to 
the topic with tags array? I got error about objectid can't be null

> S3 object tag import hook
> -------------------------
>
>                 Key: ATLAS-2889
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2889
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core, atlas-intg
>    Affects Versions: 1.1.0
>            Reporter: t oo
>            Priority: Major
>
> https://issues.apache.org/jira/browse/ATLAS-2708 introduced support for s3 
> type.
>  
> From the comments in the jira, there are some external steps needed to import 
> object tags. This Jira is about creating the E2E functionality within Atlas. 
> ie functionality in UI or an API that when supplied with argument of an s3 
> bucket/slice then Atlas automatically imports all the object names/tags 
> recursively from under that s3 bucket/slice.
>  
> "It doesn't do it automatically through a listener like the hive hook.  We do 
> it via lambda functions, triggered, say, on the creation of S3 object or 
> pseudodirectory or bucket.  We package up the info into AtlasEntities and 
> then publish to the ATLAS_HOOK kafka topic.
> You have to create your own Lambda code that creates AtlasEntities on the fly 
> as trigerred by Lambda Function(on changes made to s3 object) and then push 
> to Kafka Queue. This particular functionality is not part of Atlas tool as of 
> now."
> cc: [~barbara] [~ayushmnnit]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to