[
https://issues.apache.org/jira/browse/IMPALA-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anurag Mantripragada updated IMPALA-9096:
-----------------------------------------
Description:
Create external table with specified columns should create column lineages for
tools like Altas to consume.
For example:
create EXTERNAL TABLE IF NOT EXISTS friday_ext6
(STUD_ID int,
DEPT_ID int,
NAME string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE;
Currently we send a lineage like:
{code:java}
{
"queryText":"create EXTERNAL TABLE IF NOT EXISTS friday_ext5 (STUD_ID int,
DEPT_ID int, NAME string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE LOCATION
‘/warehouse/tablespace/external/hive/testdb.db/friday_ext5’",
"queryId":"4b471ac0ca2b0f93:029db79c00000000",
"hash":"867fae20bc6c8254c05774cc923a99fa",
"user":"admin",
"timestamp":1572028716,
"endTime":1572028716,
"edges":[],
"vertices":[],
"tableLocation":"hdfs://sid-cdp-2-1.gce.cloudera.com:8020/warehouse/tablespace/external/hive/testdb.db/friday_ext"
}
{code}
Atlas needs fully qualified table name to create lineage.
was:
Create external table with specified columns should create column lineages for
tools like Altas to consume.
For example:
create EXTERNAL TABLE IF NOT EXISTS friday_ext6
(STUD_ID int,
DEPT_ID int,
NAME string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE;
Currently we send a lineage like:
{code:java}
{
"queryText":"create EXTERNAL TABLE IF NOT EXISTS friday_ext5 (STUD_ID int,
DEPT_ID int, NAME string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’
STORED AS TEXTFILE LOCATION
‘/warehouse/tablespace/external/hive/testdb.db/friday_ext5’",
"queryId":"4b471ac0ca2b0f93:029db79c00000000",
"hash":"867fae20bc6c8254c05774cc923a99fa",
"user":"admin",
"timestamp":1572028716,
"endTime":1572028716,
"edges":[],
"vertices":[],
"tableLocation":"hdfs://sid-cdp-2-1.gce.cloudera.com:8020/warehouse/tablespace/external/hive/testdb.db/friday_ext"
}
{code}
Atlas needs fully qualified table name to create lineage.
> Create external table ddls should send column lineages.
> -------------------------------------------------------
>
> Key: IMPALA-9096
> URL: https://issues.apache.org/jira/browse/IMPALA-9096
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Anurag Mantripragada
> Priority: Minor
>
> Create external table with specified columns should create column lineages
> for tools like Altas to consume.
>
> For example:
> create EXTERNAL TABLE IF NOT EXISTS friday_ext6
> (STUD_ID int,
> DEPT_ID int,
> NAME string
> )
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ‘,’
> STORED AS TEXTFILE;
> Currently we send a lineage like:
> {code:java}
> {
> "queryText":"create EXTERNAL TABLE IF NOT EXISTS friday_ext5 (STUD_ID int,
> DEPT_ID int, NAME string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,’
> STORED AS TEXTFILE LOCATION
> ‘/warehouse/tablespace/external/hive/testdb.db/friday_ext5’",
> "queryId":"4b471ac0ca2b0f93:029db79c00000000",
> "hash":"867fae20bc6c8254c05774cc923a99fa",
> "user":"admin",
> "timestamp":1572028716,
> "endTime":1572028716,
> "edges":[],
> "vertices":[],
>
> "tableLocation":"hdfs://sid-cdp-2-1.gce.cloudera.com:8020/warehouse/tablespace/external/hive/testdb.db/friday_ext"
> }
> {code}
> Atlas needs fully qualified table name to create lineage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
