[
https://issues.apache.org/jira/browse/IMPALA-9053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954082#comment-16954082
]
ASF subversion and git services commented on IMPALA-9053:
---------------------------------------------------------
Commit b64efa76f7f0f46462400811ed3dcc878fac768d in impala's branch
refs/heads/master from Anurag Mantripragada
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b64efa7 ]
IMPALA-9053: DDLs should generate lineage graphs.
DDLs like 'create table' should generate minimal lineage graphs so
that consumers like Atlas can use information like 'queryText' to
establish lineages.
This change adds a call to the computeLineageGraph() method during
analysis phase of createTable which populates the graph with basic
information like queryText. If it is a CTAS, this graph is enhanced
in the "insert" phase with dependencies.
Testing:
Add an EE test to verify lineage information and also to check it
is flushed to disk properly.
Change-Id: Ia6c7ed9fe3265fd777fe93590cf4eb2d9ba0dd1e
Reviewed-on: http://gerrit.cloudera.org:8080/14458
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> DDLs should generate lineage graphs.
> ------------------------------------
>
> Key: IMPALA-9053
> URL: https://issues.apache.org/jira/browse/IMPALA-9053
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Anurag Mantripragada
> Assignee: Anurag Mantripragada
> Priority: Critical
>
> {{DDLs like create table should generate minimal lineage for consumers like
> Atlas to establish their relationship with the underlying table locations.}}
> {{For example, in a create external table DDL, Atlas has no way to establish
> lineage between the query and the table created.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
