Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Ashutosh Chauhan 


> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Line 1488 (original), 1488 (patched)
> > 
> >
> > This table object is never read from. Any reason for its existence.
> 
> Jesús Camacho Rodríguez wrote:
> It is passed to StatsWork in L1533. I think we do not need the full 
> table, as StatsWork will only use some of the information. I can tackle in a 
> follow-up?

Ok, lets take this as a follow-up.


> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
> > Lines 341 (patched)
> > 
> >
> > We don't store comments for regular tables in property? Is this special 
> > for (materialized) views?
> 
> Jesús Camacho Rodríguez wrote:
> In fact, apparently we do: L779 in CreateTableDesc.java. The code was 
> originally based on that one, thus they should be quite similar.

Right. I got confused with column comments which are stored in a seperate field 
in metastore table.


- Ashutosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
---


On Nov. 18, 2017, 12:33 a.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> ---
> 
> (Updated Nov. 18, 2017, 12:33 a.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14487
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> e7af5e004fb560b574b82f6d1b60517511802f37 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> e9ae590f4c291ad00e385899822ef944d50cdcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
> 55b1da90300bdc864e9258d1c49c3c21791f3b64 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> f3f1a7e8da52332d93aa57865617befd6c07a2fc 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
> d139155a1d1433ca9186994cef2616f007a66fe9 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
> 0621bd413ec578f30a236de0be5774fc83457aa5 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
> 9425f6e89113afd82d1cdca30c75da6101b63b4a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
> 1770046869170c30ebb27a84db7566d477bd9877 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
> c09589c5f23825ea39b85737e04cc0e365e61fea 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
> 3c1e92af12185bb95e27fbf0af3038ca27566197 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
>  PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
> 1749cb023ae2e4476ac386b667a708f50f8db462 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
> PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
>  81a7950773983e28a0f36ce806b6cda394993ac0 
>   ql/src/test/results/clientpositive/create_view.q.out 
> 8763fdb75fc910f75c53346c9541703e800c128c 
>   ql/src/test/results/clientpositive/create_view_translate.q.out 
> e5748976bdb271bef386e6820be5236a36d4c941 
>   ql/src/test/results/clientpositive/explain_ddl.q.out 
> f0e54c515377e7e6e002b7cc9ae986340a93acce 
>   ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
> PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
>  PRE-CREATION 
>   
>

Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
---

(Updated Nov. 18, 2017, 12:33 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487


Repository: hive-git


Description (updated)
---

HIVE-14487


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
e7af5e004fb560b574b82f6d1b60517511802f37 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
e9ae590f4c291ad00e385899822ef944d50cdcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
55b1da90300bdc864e9258d1c49c3c21791f3b64 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
f3f1a7e8da52332d93aa57865617befd6c07a2fc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
d139155a1d1433ca9186994cef2616f007a66fe9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
0621bd413ec578f30a236de0be5774fc83457aa5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
9425f6e89113afd82d1cdca30c75da6101b63b4a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
1770046869170c30ebb27a84db7566d477bd9877 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
c09589c5f23825ea39b85737e04cc0e365e61fea 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
3c1e92af12185bb95e27fbf0af3038ca27566197 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
 PRE-CREATION 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
 PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
1749cb023ae2e4476ac386b667a708f50f8db462 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
 PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
 81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/create_view.q.out 
8763fdb75fc910f75c53346c9541703e800c128c 
  ql/src/test/results/clientpositive/create_view_translate.q.out 
e5748976bdb271bef386e6820be5236a36d4c941 
  ql/src/test/results/clientpositive/explain_ddl.q.out 
f0e54c515377e7e6e002b7cc9ae986340a93acce 
  ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
 PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out 
befb6e79929ca7fffc9ee56544c25a27c2a84cbd 
  ql/src/test/results/clientpositive/llap/union_top_level.q.out 
a0102321711de82100a5b117b853a364109a2622 
  ql/src/test/results/clientpositive/llap/vector_windowing.q.out 
776ae3e085840dcaa298c143966ff1f12e508c3e 
  ql/src/test/results/clientpositive/materialized_view_create.q.out 
0fca69f4fa94a24482d09a1b30a72611eec6c643 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 
81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out
 f715803438d100599fc669cb9b1086523cd44099 
  ql/src/test/results/clientpositive/materialized_view_describe.q.out 
8b7691279c0b24c599656c372de22ef9cf83f8f7 


Diff: https://reviews.apache.org/r/63893/diff/4/

Changes: https://reviews.apache.org/r/63893/diff/3-4/


Testing
---


Thanks,

Jesús Camacho Rodríguez

Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
---

(Updated Nov. 18, 2017, 12:31 a.m.)


Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487


Repository: hive-git


Description (updated)
---

wip


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
e7af5e004fb560b574b82f6d1b60517511802f37 
  
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveMaterializedViewsRegistry.java
 51b6ef58fc196be716c4b07287fbc83503d1df50 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
e9ae590f4c291ad00e385899822ef944d50cdcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
55b1da90300bdc864e9258d1c49c3c21791f3b64 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
f3f1a7e8da52332d93aa57865617befd6c07a2fc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
d139155a1d1433ca9186994cef2616f007a66fe9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
0621bd413ec578f30a236de0be5774fc83457aa5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
9425f6e89113afd82d1cdca30c75da6101b63b4a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
1770046869170c30ebb27a84db7566d477bd9877 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
c09589c5f23825ea39b85737e04cc0e365e61fea 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
3c1e92af12185bb95e27fbf0af3038ca27566197 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
 PRE-CREATION 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
 PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
1749cb023ae2e4476ac386b667a708f50f8db462 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
 PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
 81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/create_view.q.out 
8763fdb75fc910f75c53346c9541703e800c128c 
  ql/src/test/results/clientpositive/create_view_translate.q.out 
e5748976bdb271bef386e6820be5236a36d4c941 
  ql/src/test/results/clientpositive/explain_ddl.q.out 
f0e54c515377e7e6e002b7cc9ae986340a93acce 
  ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
 PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out 
befb6e79929ca7fffc9ee56544c25a27c2a84cbd 
  ql/src/test/results/clientpositive/llap/union_top_level.q.out 
a0102321711de82100a5b117b853a364109a2622 
  ql/src/test/results/clientpositive/llap/vector_windowing.q.out 
776ae3e085840dcaa298c143966ff1f12e508c3e 
  ql/src/test/results/clientpositive/materialized_view_create.q.out 
0fca69f4fa94a24482d09a1b30a72611eec6c643 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 
81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out
 f715803438d100599fc669cb9b1086523cd44099 
  ql/src/test/results/clientpositive/materialized_view_describe.q.out 
8b7691279c0b24c599656c372de22ef9cf83f8f7 


Diff: https://reviews.apache.org/r/63893/diff/3/

Changes: https://reviews.apache.org/r/63893/diff/2-3/


Testing
---


Thanks,

Jesús Camacho Rodríguez

Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Jesús Camacho Rodríguez 


> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Line 1488 (original), 1488 (patched)
> > 
> >
> > This table object is never read from. Any reason for its existence.

It is passed to StatsWork in L1533. I think we do not need the full table, as 
StatsWork will only use some of the information. I can tackle in a follow-up?


> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
> > Lines 1515 (patched)
> > 
> >
> > Is it necessary to make metastore call to get Table? Can this be 
> > avoided.

Observe this is the same as L1495. As mentioned above, I think we do not need 
the complete table for StatsWork, just a few properties. However, to make sure 
we have everything needed, I took it from metastore as it is already an 
existing one. I can take care of this in the follow-up to check whether we need 
to pass the complete table.


> On Nov. 17, 2017, 11:22 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
> > Lines 341 (patched)
> > 
> >
> > We don't store comments for regular tables in property? Is this special 
> > for (materialized) views?

In fact, apparently we do: L779 in CreateTableDesc.java. The code was 
originally based on that one, thus they should be quite similar.


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
---


On Nov. 17, 2017, 7:26 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> ---
> 
> (Updated Nov. 17, 2017, 7:26 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14487
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> e7af5e004fb560b574b82f6d1b60517511802f37 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> e9ae590f4c291ad00e385899822ef944d50cdcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
> 55b1da90300bdc864e9258d1c49c3c21791f3b64 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> f3f1a7e8da52332d93aa57865617befd6c07a2fc 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
> d139155a1d1433ca9186994cef2616f007a66fe9 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
> 0621bd413ec578f30a236de0be5774fc83457aa5 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
> 9425f6e89113afd82d1cdca30c75da6101b63b4a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
> 1770046869170c30ebb27a84db7566d477bd9877 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
> c09589c5f23825ea39b85737e04cc0e365e61fea 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
> 3c1e92af12185bb95e27fbf0af3038ca27566197 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
>  PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
> 1749cb023ae2e4476ac386b667a708f50f8db462 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
> PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
>  81a7950773983e28a0f36ce806b6cda394993ac0 
>   ql/src/test/results/clientpositive/create_view.q.out 
> 8763fdb75fc910f75c53346c9541703e800c128c 
>   ql/src/test/results/clientpositive/create_view_translate.q.out 
> e5748976bdb271bef386e6820be5236a36d4c941 
>   ql/src/test/results/clientpositive/explain_ddl.q.out 
>

Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Ashutosh Chauhan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/#review191406
---




ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
Line 1488 (original), 1488 (patched)


This table object is never read from. Any reason for its existence.



ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
Lines 1515 (patched)


Is it necessary to make metastore call to get Table? Can this be avoided.



ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Lines 12708 (patched)


May use getTableObjectByName() to cache table object.



ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java
Lines 341 (patched)


We don't store comments for regular tables in property? Is this special for 
(materialized) views?


- Ashutosh Chauhan


On Nov. 17, 2017, 7:26 p.m., Jesús Camacho Rodríguez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63893/
> ---
> 
> (Updated Nov. 17, 2017, 7:26 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE-14487
> https://issues.apache.org/jira/browse/HIVE-14487
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14487
> 
> 
> Diffs
> -
> 
>   itests/src/test/resources/testconfiguration.properties 
> 3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
> e7af5e004fb560b574b82f6d1b60517511802f37 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
> 67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
> e9ae590f4c291ad00e385899822ef944d50cdcde 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
> 55b1da90300bdc864e9258d1c49c3c21791f3b64 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
> f3f1a7e8da52332d93aa57865617befd6c07a2fc 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
> d139155a1d1433ca9186994cef2616f007a66fe9 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
> 0621bd413ec578f30a236de0be5774fc83457aa5 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
> 9425f6e89113afd82d1cdca30c75da6101b63b4a 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
> 1770046869170c30ebb27a84db7566d477bd9877 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
> c09589c5f23825ea39b85737e04cc0e365e61fea 
>   ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
> 3c1e92af12185bb95e27fbf0af3038ca27566197 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
>  PRE-CREATION 
>   
> ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
>  PRE-CREATION 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
> 1749cb023ae2e4476ac386b667a708f50f8db462 
>   ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
> PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
>  81a7950773983e28a0f36ce806b6cda394993ac0 
>   ql/src/test/results/clientpositive/create_view.q.out 
> 8763fdb75fc910f75c53346c9541703e800c128c 
>   ql/src/test/results/clientpositive/create_view_translate.q.out 
> e5748976bdb271bef386e6820be5236a36d4c941 
>   ql/src/test/results/clientpositive/explain_ddl.q.out 
> f0e54c515377e7e6e002b7cc9ae986340a93acce 
>   ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
> PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
>  PRE-CREATION 
>   
> ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
>  PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out 
> PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out 
> PRE-CREATION 
>

Re: Review Request 63893: HIVE-14487

2017-11-17 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
---

(Updated Nov. 17, 2017, 7:26 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Updating q files


Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487


Repository: hive-git


Description
---

HIVE-14487


Diffs (updated)
-

  itests/src/test/resources/testconfiguration.properties 
3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
e7af5e004fb560b574b82f6d1b60517511802f37 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
e9ae590f4c291ad00e385899822ef944d50cdcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
55b1da90300bdc864e9258d1c49c3c21791f3b64 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
f3f1a7e8da52332d93aa57865617befd6c07a2fc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
d139155a1d1433ca9186994cef2616f007a66fe9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
0621bd413ec578f30a236de0be5774fc83457aa5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
9425f6e89113afd82d1cdca30c75da6101b63b4a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
1770046869170c30ebb27a84db7566d477bd9877 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
c09589c5f23825ea39b85737e04cc0e365e61fea 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
3c1e92af12185bb95e27fbf0af3038ca27566197 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
 PRE-CREATION 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
 PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q 
1749cb023ae2e4476ac386b667a708f50f8db462 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
 PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/beeline/materialized_view_create_rewrite.q.out
 81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/create_view.q.out 
8763fdb75fc910f75c53346c9541703e800c128c 
  ql/src/test/results/clientpositive/create_view_translate.q.out 
e5748976bdb271bef386e6820be5236a36d4c941 
  ql/src/test/results/clientpositive/explain_ddl.q.out 
f0e54c515377e7e6e002b7cc9ae986340a93acce 
  ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
 PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/selectDistinctStar.q.out 
befb6e79929ca7fffc9ee56544c25a27c2a84cbd 
  ql/src/test/results/clientpositive/llap/union_top_level.q.out 
a0102321711de82100a5b117b853a364109a2622 
  ql/src/test/results/clientpositive/llap/vector_windowing.q.out 
776ae3e085840dcaa298c143966ff1f12e508c3e 
  ql/src/test/results/clientpositive/materialized_view_create.q.out 
0fca69f4fa94a24482d09a1b30a72611eec6c643 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 
81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out
 f715803438d100599fc669cb9b1086523cd44099 
  ql/src/test/results/clientpositive/materialized_view_describe.q.out 
8b7691279c0b24c599656c372de22ef9cf83f8f7 


Diff: https://reviews.apache.org/r/63893/diff/2/

Changes: https://reviews.apache.org/r/63893/diff/1-2/


Testing
---


Thanks,

Jesús Camacho Rodríguez

Review Request 63893: HIVE-14487

2017-11-16 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63893/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-14487
https://issues.apache.org/jira/browse/HIVE-14487


Repository: hive-git


Description
---

HIVE-14487


Diffs
-

  itests/src/test/resources/testconfiguration.properties 
3b1005f85c419b1fa5646bc5a4b96788250e7c4b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
e7af5e004fb560b574b82f6d1b60517511802f37 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 
67739a1db9fc52a67f4f5ea7dba80fe0e95750c8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 
e9ae590f4c291ad00e385899822ef944d50cdcde 
  ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 
55b1da90300bdc864e9258d1c49c3c21791f3b64 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 
f3f1a7e8da52332d93aa57865617befd6c07a2fc 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
1de3dd7230c5c3708fd7dd6baf9c6913e91cab3c 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 
d139155a1d1433ca9186994cef2616f007a66fe9 
  ql/src/java/org/apache/hadoop/hive/ql/plan/BasicStatsWork.java 
0621bd413ec578f30a236de0be5774fc83457aa5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/CreateViewDesc.java 
9425f6e89113afd82d1cdca30c75da6101b63b4a 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ImportTableDesc.java 
1770046869170c30ebb27a84db7566d477bd9877 
  ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 
c09589c5f23825ea39b85737e04cc0e365e61fea 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 
3c1e92af12185bb95e27fbf0af3038ca27566197 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_no_grant.q
 PRE-CREATION 
  
ql/src/test/queries/clientnegative/materialized_view_authorization_rebuild_other.q
 PRE-CREATION 
  ql/src/test/queries/clientpositive/materialized_view_create_rewrite_3.q 
PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_no_grant.q.out
 PRE-CREATION 
  
ql/src/test/results/clientnegative/materialized_view_authorization_rebuild_other.q.out
 PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_create.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_2.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out
 PRE-CREATION 
  
ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_multi_db.q.out
 PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_describe.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_drop.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/llap/materialized_view_rewrite_ssb_2.q.out 
PRE-CREATION 
  ql/src/test/results/clientpositive/materialized_view_create.q.out 
0fca69f4fa94a24482d09a1b30a72611eec6c643 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite.q.out 
81a7950773983e28a0f36ce806b6cda394993ac0 
  ql/src/test/results/clientpositive/materialized_view_create_rewrite_3.q.out 
PRE-CREATION 
  
ql/src/test/results/clientpositive/materialized_view_create_rewrite_multi_db.q.out
 f715803438d100599fc669cb9b1086523cd44099 
  ql/src/test/results/clientpositive/materialized_view_describe.q.out 
8b7691279c0b24c599656c372de22ef9cf83f8f7 


Diff: https://reviews.apache.org/r/63893/diff/1/


Testing
---


Thanks,

Jesús Camacho Rodríguez