[
https://issues.apache.org/jira/browse/HIVE-22301?focusedWorklogId=454826&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-454826
]
ASF GitHub Bot logged work on HIVE-22301:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jul/20 13:24
Start Date: 06/Jul/20 13:24
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on pull request #1210:
URL: https://github.com/apache/hive/pull/1210#issuecomment-654235092
I feel that there is a strong correlation between something being
`!complete` and `dummy_partition`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 454826)
Time Spent: 20m (was: 10m)
> Hive lineage is not generated for insert overwrite queries on partitioned
> tables
> --------------------------------------------------------------------------------
>
> Key: HIVE-22301
> URL: https://issues.apache.org/jira/browse/HIVE-22301
> Project: Hive
> Issue Type: Bug
> Components: lineage
> Affects Versions: 3.1.2
> Reporter: Sidharth Kumar Mishra
> Assignee: Zoltan Haindrich
> Priority: Major
> Labels: pull-request-available
> Attachments: ScreenShot HookContext.png, ScreenShot
> RunPostExecHook.png, ScreenShot runBeforeExecution.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Problem: When I run the below mentioned queries, the last query should have
> given the proper hive lineage info (through HookContext) from table_b to
> table_t.
> * Create table table_t (id int) partitioned by (dob date);
> * Create table table_b (id int) partitioned by (dob date);
> * from table_b a insert overwrite table table_t select a.id,a.dob;
> Note : for CTAS query from a partitioned table , this issue is not seen. Only
> for insert queries like insert into <table> select * from <table> and query
> like above, issue is seen.
>
> Technical Observations:
> At HookContext (passed from hive.ql.Driver to Hive Hook of Atlas through
> hookRunner.runPostExecHooks call) contains no outputs. Check below screenshot
> from IntelliJ.
> !ScreenShot RunPostExecHook.png|width=728,height=427!
>
> I found that the PrivateHookContext is getting created with proper outputs
> value as shown below initially:
> !ScreenShot HookContext.png|width=714,height=541!
> The same is passed properly to runBeforeExecutionHook as shown below:
> !ScreenShot runBeforeExecution.png|width=719,height=620!
>
> Later when we pass HookContext to runPostExecHooks, there is no output
> populated. Kindly check the reason and let me know if you need any further
> information from my end.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
