Stamatis Zampetakis created HIVE-27726:
------------------------------------------
Summary: No lineage for constant literals
Key: HIVE-27726
URL: https://issues.apache.org/jira/browse/HIVE-27726
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Affects Versions: 4.0.0-beta-1
Reporter: Stamatis Zampetakis
Consider the following statement where we create a table based on another table
and some constant expressions/literals.
{code:java}
CREATE TABLE tbl1 AS (SELECT 'Bob', 'Alice', key, value || 'some' FROM src)
{code}
Observe that column 0, and column 1 of tbl1 originate from constant literals.
Currently (commit e5a7ce2f091da1f8a324da6e489cda59b9e4bfc6), there is no
lineage information for columns originating from constants.
The {{org.apache.hadoop.hive.ql.hooks.LineageLogger}} will display the
following the aforementioned DDL statement.
{noformat}
POSTHOOK: Lineage: tbl1._c0 SIMPLE []
POSTHOOK: Lineage: tbl1._c1 SIMPLE []
POSTHOOK: Lineage: tbl1._c3 EXPRESSION [(src)src.FieldSchema(name:value,
type:string, comment:default), ]
POSTHOOK: Lineage: tbl1.key SIMPLE [(src)src.FieldSchema(name:key, type:string,
comment:default), ]
{noformat}
This is not really a bug since we cannot really say that something is broken
but maybe there is a way to reflect that {{c0}} and {{c1}} originate from
constants to avoid the misconception that lineage is missing.
Note, that constant folding may also lead to the same behavior since
expressions may be simplified to constants and the latter do not appear in the
lineage output.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
