Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review79228 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java https://reviews.apache.org/r/32370/#comment128449 As we discussed: 1. Move the supported JoinAlgorithm to Sub Class (i.e target exec engine) 2. Move Cost Computation to sub class/target exec engine 3. This logic here should consult target exec engine for supported algorithms, iterate through them and find the cheapest one with out actually knowing anything about algorithm itself. - John Pullokkaran On April 6, 2015, 9:30 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 6, 2015, 9:30 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java fcf09a5de0e318c6fb69664a8dd618f2d9ae84e5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 7, 2015, 7:12 p.m.) Review request for hive and John Pullokkaran. Changes --- Address John's comments. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c8e9b52258eb209535a2bbfe512cf2d04178cb4b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdMemory.java 207f402013b5c5b2d4ada5493122427ddce9270d ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdParallelism.java 95c2be50c07f0ed6da373425e60f1185cb2cfe2b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review79235 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java https://reviews.apache.org/r/32370/#comment128458 This could be in subclass; this will make HiveCostModel opaque to a specific exec engine's algorithms. - John Pullokkaran On April 7, 2015, 7:12 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 7, 2015, 7:12 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c8e9b52258eb209535a2bbfe512cf2d04178cb4b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdMemory.java 207f402013b5c5b2d4ada5493122427ddce9270d ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdParallelism.java 95c2be50c07f0ed6da373425e60f1185cb2cfe2b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review79253 --- Ship it! Ship It! - John Pullokkaran On April 7, 2015, 7:18 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 7, 2015, 7:18 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c8e9b52258eb209535a2bbfe512cf2d04178cb4b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdMemory.java 207f402013b5c5b2d4ada5493122427ddce9270d ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdParallelism.java 95c2be50c07f0ed6da373425e60f1185cb2cfe2b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
On April 7, 2015, 8:38 p.m., John Pullokkaran wrote: Ship It! Remaining review comment will be addressed in HIVE-10243. - John --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review79253 --- On April 7, 2015, 7:18 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 7, 2015, 7:18 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c8e9b52258eb209535a2bbfe512cf2d04178cb4b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdMemory.java 207f402013b5c5b2d4ada5493122427ddce9270d ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdParallelism.java 95c2be50c07f0ed6da373425e60f1185cb2cfe2b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 7, 2015, 7:18 p.m.) Review request for hive and John Pullokkaran. Changes --- Added override annotation that was missing. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c8e9b52258eb209535a2bbfe512cf2d04178cb4b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdMemory.java 207f402013b5c5b2d4ada5493122427ddce9270d ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdParallelism.java 95c2be50c07f0ed6da373425e60f1185cb2cfe2b ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
On March 31, 2015, 3:20 p.m., Jesús Camacho Rodríguez wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java, line 54 https://reviews.apache.org/r/32370/diff/3/?file=907858#file907858line54 We could do that, but I don't feel strongly about architecting it that way, as we would be mixing two different cost models into the same method: one based on cardinality, and one based on CPU+IO+cardinality. Thus, the cost coming from that method for NONE and any other algorithm e.g. COMMON should not be even comparable, right? Only one model is allowed per query; so it shouldn't cause comparisons of cost from tow different models right? - John --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review78367 --- On March 27, 2015, 4:15 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:15 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated April 6, 2015, 9:30 p.m.) Review request for hive and John Pullokkaran. Changes --- Addressing John's comments. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 7adb38342bfaf72f152a16006bc0bfecbb28f5ed ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java f411d9029cf244b66ef1d1591ea55f11f7cb9d27 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java fcf09a5de0e318c6fb69664a8dd618f2d9ae84e5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
On March 30, 2015, 11:48 p.m., John Pullokkaran wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java, line 54 https://reviews.apache.org/r/32370/diff/3/?file=907858#file907858line54 Can't we walk through the algorithms and compare cost here as opposed to in the sub classes. Fall back to using cardinality if only possible algorithm is NONE We could do that, but I don't feel strongly about architecting it that way, as we would be mixing two different cost models into the same method: one based on cardinality, and one based on CPU+IO+cardinality. Thus, the cost coming from that method for NONE and any other algorithm e.g. COMMON should not be even comparable, right? - Jesús --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review78284 --- On March 27, 2015, 4:15 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:15 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review78367 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java https://reviews.apache.org/r/32370/#comment126929 We could do that, but I don't feel strongly about architecting it that way, as we would be mixing two different cost models into the same method: one based on cardinality, and one based on CPU+IO+cardinality. Thus, the cost coming from that method for NONE and any other algorithm e.g. COMMON should not be even comparable, right? - Jesús Camacho Rodríguez On March 27, 2015, 4:15 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:15 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review78284 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java https://reviews.apache.org/r/32370/#comment126808 Can't we walk through the algorithms and compare cost here as opposed to in the sub classes. Fall back to using cardinality if only possible algorithm is NONE - John Pullokkaran On March 27, 2015, 4:15 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:15 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:15 p.m.) Review request for hive and John Pullokkaran. Changes --- Override annotation. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 27, 2015, 4:12 p.m.) Review request for hive and John Pullokkaran. Changes --- New patch with different hierarchy among classes/methods. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java f2c5408d913bfe2648c4e1e1e43b1bbc5f43a549 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
On March 25, 2015, 8:51 p.m., John Pullokkaran wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java, line 42 https://reviews.apache.org/r/32370/diff/1/?file=902177#file902177line42 How about modifying this t0: 1. Get Join Algorithms 2. Walk through Join Algorithm a. Get Cost for Join b. Find cheapest This way algorithm walking and finding cost would be generic. John, I have been refactoring the code to do it how you proposed. Actually, I am not convinced about the result; at the end it seems more difficult to extend the hierarchy in case a new cost model is added, plus null values need to be given to some parameters needed to decide which algorithms are available (e.g. maxMemory). Another idea would be that HiveOnTezCostModel extends HiveDefaultCostModel instead of implementing HiveCostModel directly; I have done this and it looks clean. - Jesús --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review77792 --- On March 21, 2015, 5:49 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 21, 2015, 5:49 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c53d6ae80fe9c8111f609bfebd8530eca67d27b7 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
Re: Review Request 32370: HIVE-10040
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/#review77792 --- ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java https://reviews.apache.org/r/32370/#comment126066 How about modifying this t0: 1. Get Join Algorithms 2. Walk through Join Algorithm a. Get Cost for Join b. Find cheapest This way algorithm walking and finding cost would be generic. - John Pullokkaran On March 21, 2015, 5:49 p.m., Jesús Camacho Rodríguez wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32370/ --- (Updated March 21, 2015, 5:49 p.m.) Review request for hive and John Pullokkaran. Bugs: HIVE-10040 https://issues.apache.org/jira/browse/HIVE-10040 Repository: hive-git Description --- CBO (Calcite Return Path): Pluggable cost modules [CBO branch] Diffs - ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveDefaultRelMetadataProvider.java 977313a5a632329fc963daf7ff276ccdd59ce7c5 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCost.java 41604cd0af68e7f90296fa271c42debc5aaf743a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveDefaultCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveOnTezCostModel.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/cost/HiveRelMdCost.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveAggregate.java 9a8a5da81b92c7c1f33d1af8072b1fb94e237290 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveFilter.java 3e45a3fbed3265b126a3ff9b6ffe44bee24453ef ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveJoin.java e2b010b641d48ea1bf04750ddf5eb24fb3a7fcbe ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveLimit.java 5fc64f3e8c97fc8988bc35be39dbabf78dd7de24 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveProject.java 6c215c96190f0fcebe063b15c2763c49ebf1faaf ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/reloperators/HiveTableScan.java c53d6ae80fe9c8111f609bfebd8530eca67d27b7 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdCollation.java 4984683c3c8c6c0378a22e21fd6d961f3901f25c ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdDistribution.java f846dd19899af51194f3407ef913fcb9bcc24977 ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java dabbe280278dc80f00f0240a0c615fe6c7b8533a ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdUniqueKeys.java 95515b23e409d73d5c61e107931727add3f992a6 Diff: https://reviews.apache.org/r/32370/diff/ Testing --- Thanks, Jesús Camacho Rodríguez
