Anurag Mantripragada created IMPALA-9096:
--------------------------------------------
Summary: Create external table ddls should send column lineages.
Key: IMPALA-9096
URL: https://issues.apache.org/jira/browse/IMPALA-9096
Project: IMPALA
Issue Type: Improvement
Reporter: Anurag Mantripragada
Create external table with specified columns should create column lineages for
tools like Altas to consume.
For example:
create EXTERNAL TABLE IF NOT EXISTS friday_ext6
(STUD_ID int,
DEPT_ID int,
NAME string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE;
Currently we send a lineage like:
{code:java}
{
"queryText":"create EXTERNAL TABLE IF NOT EXISTS friday_ext5 (STUD_ID int,
DEPT_ID int, NAME string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE LOCATION
‘/warehouse/tablespace/external/hive/testdb.db/friday_ext5’",
"queryId":"4b471ac0ca2b0f93:029db79c00000000",
"hash":"867fae20bc6c8254c05774cc923a99fa",
"user":"admin",
"timestamp":1572028716,
"endTime":1572028716,
"edges":[],
"vertices":[],
"tableLocation":"hdfs://sid-cdp-2-1.gce.cloudera.com:8020/warehouse/tablespace/external/hive/testdb.db/friday_ext"
}
{code}
Atlas needs fully qualified table name to create lineage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
