[
https://issues.apache.org/jira/browse/ATLAS-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16933900#comment-16933900
]
ASF subversion and git services commented on ATLAS-3413:
--------------------------------------------------------
Commit 15e5bedbe470677ab8a8de4ecfa76c55f852c4e7 in atlas's branch
refs/heads/master from skoritala
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=15e5bed ]
ATLAS-3413: HMS should create a dummy process linking HDFS path to Hive table
for external table lineage
Signed-off-by: Sarath Subramanian <[email protected]>
> HMS should create a dummy process linking HDFS path to Hive table for
> external table lineage.
> ---------------------------------------------------------------------------------------------
>
> Key: ATLAS-3413
> URL: https://issues.apache.org/jira/browse/ATLAS-3413
> Project: Atlas
> Issue Type: Improvement
> Reporter: Sridhar
> Assignee: Sridhar
> Priority: Major
>
> HMS should create a dummy process linking HDFS path to Hive table for
> external table lineage.
> Currently, HS2 creates the linkage from file path to table. But, when an
> external table is created from impala or spark, there is no lineage is
> getting created from hdfs_path to table. As a fix, we would like HMS hook to
> create the lineage for external tables.
> Where ever possible, HMS hook should create the process with the same
> qualified name as that of what gets created as part of individual service
> specific hook. For now, HMS hook will resolve the HS2 service processes only.
> For impala and Spark, the process will be created a dummy hive process as a
> place holder to avoid missing lineages.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)