[
https://issues.apache.org/jira/browse/HIVE-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745937#comment-14745937
]
Mark Grover commented on HIVE-11139:
------------------------------------
I have a dynamic partitioning query but at the end of the query it shows me an
error message like:
{quote}
ERROR : Result schema has 2 fields, but we don't get as many dependencies
{quote}
Going through the source code, led me to this commit. Was this tested to make
sure it works fine with dynamic partitioning. Here's my query btw?
{code}
SET hive.exec.dynamic.partition.mode=nonstrict;
DROP TABLE IF EXISTS default.src_mark;
CREATE TABLE default.src_mark (first string, word string)
PARTITIONED BY (length int)
STORED AS PARQUET;
INSERT INTO TABLE default.src_mark PARTITION(length) SELECT first, word, length
FROM spark_hive.src_flat;
{code}
And, I verified that all the values in src_flat conform to the schema. Also, at
the very least it would be helpful to know what the number of dependencies and
what their names were in the error message:
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/hooks/LineageLogger.java#L251
Your thoughts would be much appreciated!
> Emit more lineage information
> -----------------------------
>
> Key: HIVE-11139
> URL: https://issues.apache.org/jira/browse/HIVE-11139
> Project: Hive
> Issue Type: Improvement
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 1.3.0, 2.0.0
>
> Attachments: HIVE-11139.1.patch, HIVE-11139.2.patch,
> HIVE-11139.3.patch
>
>
> HIVE-1131 emits some column lineage info. But it doesn't support INSERT
> statements, or CTAS statements. It doesn't emit the predicate information
> either.
> We can enhance and use the dependency information created in HIVE-1131,
> generate more complete lineage info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
