> On March 24, 2020, 6:29 a.m., Jesús Camacho Rodríguez wrote:
> > ql/src/test/results/clientpositive/ctas_char.q.out
> > Lines 65 (patched)
> > <https://reviews.apache.org/r/72234/diff/4/?file=2214846#file2214846line65>
> >
> >     Is this correct? Did insert (sort) run with multiple reducers? I would 
> > expect we are running with a single reducer. There are also similar changes 
> > in files above. Can we verify references/sort spec/plan/exec is correct?

I added disabling CBO if `SORT BY` has a `LIMIT` and fall back to non-CBO path.


> On March 24, 2020, 6:29 a.m., Jesús Camacho Rodríguez wrote:
> > ql/src/test/results/clientpositive/input4_limit.q.out
> > Line 50 (original)
> > <https://reviews.apache.org/r/72234/diff/4/?file=2214850#file2214850line50>
> >
> >     Reducer is gone. Is this expected?

I added disabling CBO if `SORT BY` has a `LIMIT` and fall back to non-CBO path.


- Krisztian


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72234/#review220061
-----------------------------------------------------------


On March 26, 2020, 4:49 p.m., Krisztian Kasa wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72234/
> -----------------------------------------------------------
> 
> (Updated March 26, 2020, 4:49 p.m.)
> 
> 
> Review request for hive and Jesús Camacho Rodríguez.
> 
> 
> Bugs: HIVE-22785
>     https://issues.apache.org/jira/browse/HIVE-22785
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Update/delete/merge statements not optimized through CBO
> 
> 
> Diffs
> -----
> 
>   
> itests/hive-blobstore/src/test/results/clientpositive/map_join_on_filter.q.out
>  653faab00a 
>   itests/src/test/resources/testconfiguration.properties 3510016c07 
>   ql/src/java/org/apache/hadoop/hive/ql/QueryProperties.java 9c61b316e2 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelDistribution.java
>  e5f4c8492e 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelFactories.java 
> 04b3888a25 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJson.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelJsonImpl.java 
> 0d45eb0c61 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOptUtil.java 
> e647b88961 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveSortExchange.java
>  880cae70f9 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortExchangeTransposeRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveProjectSortTransposeRule.java
>  871c411e70 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelFieldTrimmer.java
>  53d68e872a 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortLimitPullUpConstantsRule.java
>  e51b2b6ebc 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortPullUpConstantsRule.java
>  PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTConverter.java
>  e03e96ff12 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/PlanModifierForASTConv.java
>  31619c0314 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/HiveSortExchangeVisitor.java
>  68227db1ee 
>   
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/opconventer/JoinVisitor.java
>  0286d54ea0 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 6589eeb39b 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/RewriteSemanticAnalyzer.java 
> 31068cb8c3 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 
> 679ae2e1e6 
>   ql/src/test/queries/clientpositive/authorization_view_disable_cbo_1.q 
> be50b69830 
>   ql/src/test/queries/clientpositive/sort.q cab2712810 
>   ql/src/test/queries/clientpositive/sort_acid.q PRE-CREATION 
>   ql/src/test/results/clientnegative/materialized_view_no_cbo_rewrite.q.out 
> 2b7ff65c7a 
>   ql/src/test/results/clientnegative/materialized_view_no_cbo_rewrite_2.q.out 
> 6850290412 
>   ql/src/test/results/clientnegative/update_notnull_constraint.q.out 
> 86bfc67480 
>   ql/src/test/results/clientpositive/acid_view_delete.q.out 3771c3ba63 
>   ql/src/test/results/clientpositive/authorization_view_disable_cbo_1.q.out 
> b609982bb9 
>   ql/src/test/results/clientpositive/auto_join0.q.out 665cf28dea 
>   ql/src/test/results/clientpositive/auto_join15.q.out 6ea9db28f0 
>   ql/src/test/results/clientpositive/auto_join20.q.out 6bbcb47084 
>   ql/src/test/results/clientpositive/auto_join21.q.out b0af07c93e 
>   ql/src/test/results/clientpositive/auto_join23.q.out 9dcfc1aa6f 
>   ql/src/test/results/clientpositive/auto_join28.q.out 6b27398f1e 
>   ql/src/test/results/clientpositive/auto_join29.q.out ade39bdc1a 
>   ql/src/test/results/clientpositive/auto_join31.q.out 73651d36f9 
>   ql/src/test/results/clientpositive/cbo_rp_auto_join0.q.out 72e041e767 
>   ql/src/test/results/clientpositive/correlationoptimizer14.q.out b8d764841b 
>   ql/src/test/results/clientpositive/identity_project_remove_skip.q.out 
> 1176c7c0d4 
>   ql/src/test/results/clientpositive/input_part7.q.out 15a388e88a 
>   ql/src/test/results/clientpositive/join0.q.out cda0f3f0a5 
>   ql/src/test/results/clientpositive/join15.q.out 664f2e0614 
>   ql/src/test/results/clientpositive/join20.q.out 9327cf453a 
>   ql/src/test/results/clientpositive/join21.q.out d25eb5cb09 
>   ql/src/test/results/clientpositive/join23.q.out 4f8f0d8c14 
>   ql/src/test/results/clientpositive/join40.q.out b0942ea3e7 
>   ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out 699398bc9d 
>   ql/src/test/results/clientpositive/llap/acid_vectorization_original.q.out 
> 2e43110b35 
>   ql/src/test/results/clientpositive/llap/auto_join0.q.out 31b776fd5d 
>   ql/src/test/results/clientpositive/llap/auto_join21.q.out df866d8ce6 
>   ql/src/test/results/clientpositive/llap/auto_join29.q.out 9c3a0b1388 
>   ql/src/test/results/clientpositive/llap/auto_join30.q.out d81dadb799 
>   ql/src/test/results/clientpositive/llap/check_constraint.q.out b4acc55f15 
>   ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_3.q.out 
> 2522677d95 
>   
> ql/src/test/results/clientpositive/llap/dynpart_sort_optimization_acid.q.out 
> 57dbbe34c3 
>   ql/src/test/results/clientpositive/llap/enforce_constraint_notnull.q.out 
> 74d0190dfe 
>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out b82a055079 
>   ql/src/test/results/clientpositive/llap/identity_project_remove_skip.q.out 
> a8a5e38602 
>   ql/src/test/results/clientpositive/llap/insert_into_default_keyword.q.out 
> bdf1a65cb1 
>   ql/src/test/results/clientpositive/llap/join0.q.out 0a81c5bca3 
>   ql/src/test/results/clientpositive/llap/llap_acid.q.out b8a4058f6c 
>   ql/src/test/results/clientpositive/llap/llap_acid_fast.q.out b91c3fffa7 
>   ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out 
> 29deff9eca 
>   ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out 
> 3a287bcda8 
>   ql/src/test/results/clientpositive/llap/runtime_stats_merge.q.out 
> edcdfb4175 
>   ql/src/test/results/clientpositive/llap/semijoin.q.out 46ff455ecc 
>   ql/src/test/results/clientpositive/llap/sort_acid.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/llap/sqlmerge.q.out ad21ef626d 
>   ql/src/test/results/clientpositive/llap/sqlmerge_stats.q.out 9e31c64f83 
>   ql/src/test/results/clientpositive/llap/vector_join30.q.out 2fa252d942 
>   ql/src/test/results/clientpositive/llap/vector_leftsemi_mapjoin.q.out 
> 951f824c30 
>   ql/src/test/results/clientpositive/llap/vector_mapjoin_complex_values.q.out 
> ec8b1d9bc8 
>   
> ql/src/test/results/clientpositive/llap/vector_outer_reference_windowed.q.out 
> 94fa3cfb3a 
>   ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out 
> 8ce9b76619 
>   ql/src/test/results/clientpositive/mapjoin_filter_on_outerjoin.q.out 
> ef7ea4ecf9 
>   ql/src/test/results/clientpositive/mapjoin_test_outer.q.out dbc87d5f8d 
>   ql/src/test/results/clientpositive/no_hooks.q.out ffff0752fc 
>   ql/src/test/results/clientpositive/parallel_join0.q.out cda0f3f0a5 
>   ql/src/test/results/clientpositive/pcs.q.out 15c6c212fd 
>   ql/src/test/results/clientpositive/perf/tez/cbo_query51.q.out a072421bad 
>   ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query51.q.out 
> 3d81b6bcb5 
>   ql/src/test/results/clientpositive/perf/tez/constraints/query51.q.out 
> 2ceca679e5 
>   ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out 
> b2bde8d48c 
>   ql/src/test/results/clientpositive/perf/tez/query51.q.out 5dc11619a4 
>   ql/src/test/results/clientpositive/perf/tez/query70.q.out 6107ec0367 
>   ql/src/test/results/clientpositive/ppd_join4.q.out 67d8e21133 
>   ql/src/test/results/clientpositive/sort.q.out 42cbd83fdc 
>   ql/src/test/results/clientpositive/spark/auto_join0.q.out 300c087686 
>   ql/src/test/results/clientpositive/spark/auto_join15.q.out 096d9857a0 
>   ql/src/test/results/clientpositive/spark/auto_join20.q.out 1b5150d419 
>   ql/src/test/results/clientpositive/spark/auto_join21.q.out 6cd15fd348 
>   ql/src/test/results/clientpositive/spark/auto_join23.q.out 268eb76741 
>   ql/src/test/results/clientpositive/spark/auto_join28.q.out fc55988949 
>   ql/src/test/results/clientpositive/spark/auto_join29.q.out 4bb41d7f10 
>   ql/src/test/results/clientpositive/spark/auto_join30.q.out 0d3c6df069 
>   ql/src/test/results/clientpositive/spark/auto_join31.q.out a20b6abb12 
>   ql/src/test/results/clientpositive/spark/dynamic_rdd_cache.q.out bef3125869 
>   ql/src/test/results/clientpositive/spark/identity_project_remove_skip.q.out 
> 11dcc6d860 
>   ql/src/test/results/clientpositive/spark/join0.q.out ed11931ba1 
>   ql/src/test/results/clientpositive/spark/join15.q.out e586f33a74 
>   ql/src/test/results/clientpositive/spark/join20.q.out 481eef2d40 
>   ql/src/test/results/clientpositive/spark/join21.q.out 4c117cd4b5 
>   ql/src/test/results/clientpositive/spark/join23.q.out 6823bdb935 
>   ql/src/test/results/clientpositive/spark/join40.q.out 3efa147330 
>   ql/src/test/results/clientpositive/spark/mapjoin_filter_on_outerjoin.q.out 
> 54f08850de 
>   ql/src/test/results/clientpositive/spark/mapjoin_test_outer.q.out 
> 7850255f2c 
>   ql/src/test/results/clientpositive/spark/parallel_join0.q.out 3acb2ec941 
>   ql/src/test/results/clientpositive/spark/ppd_join4.q.out bde0232ac6 
>   ql/src/test/results/clientpositive/spark/semijoin.q.out 691d0933ab 
>   ql/src/test/results/clientpositive/spark/sort.q.out f0c5bd2983 
>   ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out 
> 563a6a684f 
>   ql/src/test/results/clientpositive/spark/union_ppr.q.out 14a63b7e95 
>   
> ql/src/test/results/clientpositive/tez/acid_vectorization_original_tez.q.out 
> 02b4f6bf32 
>   ql/src/test/results/clientpositive/union_ppr.q.out 89985d0b1a 
> 
> 
> Diff: https://reviews.apache.org/r/72234/diff/9/
> 
> 
> Testing
> -------
> 
> mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestCliDriver 
> -Dqfile=sort_acid.q -pl itests/qtest -Pitests
> 
> 
> Thanks,
> 
> Krisztian Kasa
> 
>

Reply via email to