[
https://issues.apache.org/jira/browse/HIVE-24009?focusedWorklogId=487618&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-487618
]
ASF GitHub Bot logged work on HIVE-24009:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Sep/20 03:21
Start Date: 22/Sep/20 03:21
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on a change in pull request
#1472:
URL: https://github.com/apache/hive/pull/1472#discussion_r492224561
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java
##########
@@ -63,19 +63,19 @@
private VectorizationContext taskVectorizationContext;
- protected transient JobConf jc;
- private transient boolean inputFileChanged = false;
+ protected JobConf jc;
Review comment:
I actually tried not keeping these fields but I was running into all
sorts of issues like unable to serialize/de-serialize or plan generating
without metadata etc.
I am not sure if we need to keep all of these fields or we can selectively
choose, I went by almost all in interest of time. If Gopal or Rajesh thinks
that this may cause performance issue I can open a follow-up to investigate and
choose fields selectively.
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
##########
@@ -387,6 +387,12 @@
protected volatile boolean disableJoinMerge = false;
protected final boolean defaultJoinMerge;
+ /*
+ * This is used by prepare/execute statement
+ * Prepare/Execute requires operators to be copied and cached
+ */
+ protected Map<String, TableScanOperator> topOpsCopy = null;
Review comment:
Original operator tree shape is changed when going through physical
transformations and task generation (don't know why though), as a result this
operator tree can not be used later to regenerate tasks or re-running physical
transformations. Therefore we make a copy and cache it after operator tree is
generated.
I will leave a comment.
##########
File path: ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
##########
@@ -84,12 +84,13 @@ Stage-0
Select Operator [SEL_40] (rows=1 width=4)
Output:["_col0"]
TableScan [TS_24] (rows=1 width=4)
- Output:["id"]
+ default@tb2,tb2,Tbl:COMPLETE,Col:NONE,Output:["id"]
Review comment:
Yeah I think this is likely side effect of some changes in w.r.t
serialization/de-serialization. Although this is positive side effect now that
we have more information in explain plan.
##########
File path: ql/src/test/results/clientpositive/llap/constprog_dpp.q.out
##########
@@ -84,12 +84,13 @@ Stage-0
Select Operator [SEL_40] (rows=1 width=4)
Output:["_col0"]
TableScan [TS_24] (rows=1 width=4)
- Output:["id"]
+ default@tb2,tb2,Tbl:COMPLETE,Col:NONE,Output:["id"]
<-Map 6 [CONTAINS] vectorized, llap
Reduce Output Operator [RS_45]
Limit [LIM_44] (rows=1 width=2)
Number of rows:1
Select Operator [SEL_43] (rows=1 width=0)
Output:["_col0"]
TableScan [TS_29] (rows=1 width=0)
+ default@tb2,tb2,Tbl:PARTIAL,Col:COMPLETE
Review comment:
I confirmed that this is expected. I compared this plan against master
(with explain.user set to false) and there is no difference in the plan.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 487618)
Time Spent: 1h 20m (was: 1h 10m)
> Support partition pruning and other physical transformations for EXECUTE
> statement
> -----------------------------------------------------------------------------------
>
> Key: HIVE-24009
> URL: https://issues.apache.org/jira/browse/HIVE-24009
> Project: Hive
> Issue Type: Sub-task
> Reporter: Vineet Garg
> Assignee: Vineet Garg
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Current partition pruning (compile time) isn't kicked in for EXECUTE
> statements.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
