[jira] [Created] (IMPALA-7766) Perform constant folding within an expression

2018-10-25 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7766: --- Summary: Perform constant folding within an expression Key: IMPALA-7766 URL: https://issues.apache.org/jira/browse/IMPALA-7766 Project: IMPALA Issue Type:

[jira] [Updated] (IMPALA-7769) Constant folding sometimes introduces an unnecessary cast

2018-10-25 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7769: Description: Consider the following query: {code:sql} SELECT IFNULL(NULL + 1, id) FROM

[jira] [Created] (IMPALA-7769) Constant folding sometimes introduces an unnecessary cast

2018-10-25 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7769: --- Summary: Constant folding sometimes introduces an unnecessary cast Key: IMPALA-7769 URL: https://issues.apache.org/jira/browse/IMPALA-7769 Project: IMPALA

[jira] [Updated] (IMPALA-7769) Constant folding sometimes introduces an unnecessary cast

2018-10-25 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7769: Description: Consider the following query: {code:sql} SELECT IFNULL(NULL + 1, id) FROM

[jira] [Commented] (IMPALA-7766) Perform constant folding within an expression

2018-10-25 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664552#comment-16664552 ] Paul Rogers commented on IMPALA-7766: - [~tarmstrong], makes sense, that seems how a parser would

[jira] [Updated] (IMPALA-7750) Additional FE optimizations for CASE expressions

2018-10-23 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7750: Description: The current FE {{CASE}} rewrite code in

[jira] [Created] (IMPALA-7750) Additional FE optimizations for CASE expressions

2018-10-23 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7750: --- Summary: Additional FE optimizations for CASE expressions Key: IMPALA-7750 URL: https://issues.apache.org/jira/browse/IMPALA-7750 Project: IMPALA Issue Type:

[jira] [Updated] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7754: Description: The analyzer has a chain of rules which fire in order without (as noted above)

[jira] [Assigned] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers reassigned IMPALA-7754: --- Assignee: (was: Paul Rogers) > Expressions sometimes not re-analyzed after rewrite >

[jira] [Comment Edited] (IMPALA-7655) Codegen output for conditional functions (if,isnull, coalesce) is very suboptimal

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657532#comment-16657532 ] Paul Rogers edited comment on IMPALA-7655 at 10/29/18 5:34 PM: --- Work on

[jira] [Comment Edited] (IMPALA-7655) Codegen output for conditional functions (if,isnull, coalesce) is very suboptimal

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659482#comment-16659482 ] Paul Rogers edited comment on IMPALA-7655 at 10/29/18 5:35 PM: --- To

[jira] [Commented] (IMPALA-7747) Review and modernize conditional function rewrites

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667528#comment-16667528 ] Paul Rogers commented on IMPALA-7747: - Other issues found in IMPALA-7655 and deferred: h4. Special

[jira] [Assigned] (IMPALA-7747) Clean up the Expression Rewriter

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers reassigned IMPALA-7747: --- Assignee: (was: Paul Rogers) Priority: Minor (was: Major) Description:

[jira] [Assigned] (IMPALA-7769) Handle CAST(NULL AS type) in rewrites

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers reassigned IMPALA-7769: --- Assignee: (was: Paul Rogers) > Handle CAST(NULL AS type) in rewrites >

[jira] [Updated] (IMPALA-7769) Handle CAST(NULL AS type) in rewrites

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7769: Description: Consider the following query: {code:sql} SELECT IFNULL(NULL + 1, id) FROM

[jira] [Updated] (IMPALA-7769) Handle CAST(NULL AS type) in rewrites

2018-10-29 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7769: Summary: Handle CAST(NULL AS type) in rewrites (was: Constant folding sometimes introduces an

[jira] [Updated] (IMPALA-7793) CASE must not rewrite CAST(literal AS type) exprs

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7793: Summary: CASE must not rewrite CAST(literal AS type) exprs (was: CASE statement does not handle

[jira] [Updated] (IMPALA-7793) Do not rewrite CAST(non-null-literal AS type) exprs

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7793: Summary: Do not rewrite CAST(non-null-literal AS type) exprs (was: CASE must not rewrite

[jira] [Commented] (IMPALA-7793) CASE must not rewrite CAST(literal AS type) exprs

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675587#comment-16675587 ] Paul Rogers commented on IMPALA-7793: - This issue identified two problems. First, V2 of Decimal

[jira] [Created] (IMPALA-7805) NumericLiteral toSql() should render zero as 0, not 0-E38, 0.000, etc.

2018-11-05 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7805: --- Summary: NumericLiteral toSql() should render zero as 0, not 0-E38, 0.000, etc. Key: IMPALA-7805 URL: https://issues.apache.org/jira/browse/IMPALA-7805 Project: IMPALA

[jira] [Created] (IMPALA-7818) Standardize use of Expr predicates

2018-11-06 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7818: --- Summary: Standardize use of Expr predicates Key: IMPALA-7818 URL: https://issues.apache.org/jira/browse/IMPALA-7818 Project: IMPALA Issue Type: Improvement

[jira] [Created] (IMPALA-7826) Potential NPE in CatalogOpExecutor

2018-11-07 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7826: --- Summary: Potential NPE in CatalogOpExecutor Key: IMPALA-7826 URL: https://issues.apache.org/jira/browse/IMPALA-7826 Project: IMPALA Issue Type: Bug

[jira] [Created] (IMPALA-7831) Revisit expression rewriting integration with planner

2018-11-07 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7831: --- Summary: Revisit expression rewriting integration with planner Key: IMPALA-7831 URL: https://issues.apache.org/jira/browse/IMPALA-7831 Project: IMPALA Issue

[jira] [Updated] (IMPALA-7808) Refactor Analyzer for easier debugging

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7808: Description: The analysis steps in {{SelectStmt}} and {{AnalysisContext}} are large and

[jira] [Created] (IMPALA-7808) Refactor Analyzer for easier debugging

2018-11-05 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7808: --- Summary: Refactor Analyzer for easier debugging Key: IMPALA-7808 URL: https://issues.apache.org/jira/browse/IMPALA-7808 Project: IMPALA Issue Type:

[jira] [Updated] (IMPALA-7808) Refactor Analyzer for easier debugging

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7808: Description: The analysis steps in {{SelectStmt}} and {{AnalysisContext}} are large and

[jira] [Updated] (IMPALA-7805) NumericLiteral toSql() should render zero as 0, not 0-E38, 0.000, etc.

2018-11-05 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7805: Description: Testing of other issues revealed a somewhat bizarre aspect of how the planner

[jira] [Created] (IMPALA-7807) Analysis test fixture to enable deeper testing

2018-11-05 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7807: --- Summary: Analysis test fixture to enable deeper testing Key: IMPALA-7807 URL: https://issues.apache.org/jira/browse/IMPALA-7807 Project: IMPALA Issue Type:

[jira] [Created] (IMPALA-7821) Split up tests functions in ExprRewriteRulesTest for easier debugging

2018-11-06 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7821: --- Summary: Split up tests functions in ExprRewriteRulesTest for easier debugging Key: IMPALA-7821 URL: https://issues.apache.org/jira/browse/IMPALA-7821 Project: IMPALA

[jira] [Updated] (IMPALA-7750) Prune trivial ELSE clause in CASE simplification

2018-11-06 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7750: Description: A trivial optimization is to omit ELSE if it adds no value: {code:sql} CASE WHEN id

[jira] [Commented] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-11-07 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679132#comment-16679132 ] Paul Rogers commented on IMPALA-7754: - There be dragons here. It seems that, without rewrites, we

[jira] [Created] (IMPALA-7838) Planner's handling of parenthesized expressions is awkward

2018-11-08 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7838: --- Summary: Planner's handling of parenthesized expressions is awkward Key: IMPALA-7838 URL: https://issues.apache.org/jira/browse/IMPALA-7838 Project: IMPALA

[jira] [Commented] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-11-08 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680342#comment-16680342 ] Paul Rogers commented on IMPALA-7754: - Another misfire, in {{PlannerTest}},

[jira] [Updated] (IMPALA-7838) Planner's handling of parenthesized expressions is awkward

2018-11-08 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7838: Description: Consider two simple queries: {code:sql} SELECT 2 + 3 * 4 FROM ... SELECT (2 + 3) *

[jira] [Commented] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-11-08 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680326#comment-16680326 ] Paul Rogers commented on IMPALA-7754: - Another misfire, in {{PlannerTest}}, {{hdfs.test}}:

[jira] [Created] (IMPALA-7841) Refactor QueryStmt, other analysis code for easier debugging

2018-11-08 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7841: --- Summary: Refactor QueryStmt, other analysis code for easier debugging Key: IMPALA-7841 URL: https://issues.apache.org/jira/browse/IMPALA-7841 Project: IMPALA

[jira] [Created] (IMPALA-7842) Refactor FrontEnd to make plan fragments available for testing

2018-11-10 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7842: --- Summary: Refactor FrontEnd to make plan fragments available for testing Key: IMPALA-7842 URL: https://issues.apache.org/jira/browse/IMPALA-7842 Project: IMPALA

[jira] [Work started] (IMPALA-7823) Clean up Java warnings, fix minor issues

2018-11-13 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7823 started by Paul Rogers. --- > Clean up Java warnings, fix minor issues >

[jira] [Work started] (IMPALA-7842) Refactor FrontEnd to make plan fragments available for testing

2018-11-13 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7842 started by Paul Rogers. --- > Refactor FrontEnd to make plan fragments available for testing >

[jira] [Created] (IMPALA-7855) Excessive type widening leads to unnecessary casts

2018-11-15 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7855: --- Summary: Excessive type widening leads to unnecessary casts Key: IMPALA-7855 URL: https://issues.apache.org/jira/browse/IMPALA-7855 Project: IMPALA Issue

[jira] [Created] (IMPALA-7823) Clean up Java warnings, fix minor issues

2018-11-06 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7823: --- Summary: Clean up Java warnings, fix minor issues Key: IMPALA-7823 URL: https://issues.apache.org/jira/browse/IMPALA-7823 Project: IMPALA Issue Type:

[jira] [Updated] (IMPALA-7821) Split up test functions in ExprRewriteRulesTest for easier debugging

2018-11-06 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7821: Summary: Split up test functions in ExprRewriteRulesTest for easier debugging (was: Split up

[jira] [Commented] (IMPALA-7769) Handle CAST(NULL AS type) in rewrites

2018-11-08 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679202#comment-16679202 ] Paul Rogers commented on IMPALA-7769: - An example failure in {{PlannerTest.testEmpt()}}, for the

[jira] [Comment Edited] (IMPALA-7754) Expressions sometimes not re-analyzed after rewrite

2018-11-08 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679132#comment-16679132 ] Paul Rogers edited comment on IMPALA-7754 at 11/8/18 1:54 AM: -- There be

[jira] [Created] (IMPALA-7847) Standardize expression error message in analyzer to ease testing

2018-11-13 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7847: --- Summary: Standardize expression error message in analyzer to ease testing Key: IMPALA-7847 URL: https://issues.apache.org/jira/browse/IMPALA-7847 Project: IMPALA

[jira] [Updated] (IMPALA-7846) Analyzer expression error checks done in wrong order

2018-11-13 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7846: Priority: Trivial (was: Minor) Issue Type: Improvement (was: Bug) > Analyzer expression

[jira] [Created] (IMPALA-7844) Analysis code incorrectly attempts to support ordinals in HAVING clause

2018-11-13 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7844: --- Summary: Analysis code incorrectly attempts to support ordinals in HAVING clause Key: IMPALA-7844 URL: https://issues.apache.org/jira/browse/IMPALA-7844 Project:

[jira] [Created] (IMPALA-7845) Clarify Impala's policy for use of column aliases

2018-11-13 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7845: --- Summary: Clarify Impala's policy for use of column aliases Key: IMPALA-7845 URL: https://issues.apache.org/jira/browse/IMPALA-7845 Project: IMPALA Issue Type:

[jira] [Created] (IMPALA-7846) Analyzer expression error checks done in wrong order

2018-11-13 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7846: --- Summary: Analyzer expression error checks done in wrong order Key: IMPALA-7846 URL: https://issues.apache.org/jira/browse/IMPALA-7846 Project: IMPALA Issue

[jira] [Comment Edited] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-10-02 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635969#comment-16635969 ] Paul Rogers edited comment on IMPALA-7564 at 10/2/18 6:59 PM: -- In an RDBMS,

[jira] [Commented] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-10-02 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635969#comment-16635969 ] Paul Rogers commented on IMPALA-7564: - In an RDBMS, the PK is hinted by a unique index. But, Hive

[jira] [Created] (IMPALA-7692) Profile averages not computed for Exchange InactiveTotalTime TIME_NS counter

2018-10-10 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7692: --- Summary: Profile averages not computed for Exchange InactiveTotalTime TIME_NS counter Key: IMPALA-7692 URL: https://issues.apache.org/jira/browse/IMPALA-7692 Project:

[jira] [Commented] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-10-01 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634796#comment-16634796 ] Paul Rogers commented on IMPALA-7564: - The particular issue here appears to be the {{substr()}}

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621096#comment-16621096 ] Paul Rogers edited comment on IMPALA-7310 at 9/20/18 1:50 AM: -- Simplest

[jira] [Commented] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621222#comment-16621222 ] Paul Rogers commented on IMPALA-7310: - [~jeszyb], agree completely. Here I'm digging down to do a

[jira] [Commented] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621196#comment-16621196 ] Paul Rogers commented on IMPALA-7310: - Here, it is worth pointing out the risk of any change. The

[jira] [Commented] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621126#comment-16621126 ] Paul Rogers commented on IMPALA-7310: - As noted above, the code uses -1 as an "undefined"

[jira] [Commented] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621171#comment-16621171 ] Paul Rogers commented on IMPALA-7310: - The original description pointed out the method that computes

[jira] [Created] (IMPALA-7602) Definition of NDV differs between planner and stats mechanism

2018-09-20 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7602: --- Summary: Definition of NDV differs between planner and stats mechanism Key: IMPALA-7602 URL: https://issues.apache.org/jira/browse/IMPALA-7602 Project: IMPALA

[jira] [Created] (IMPALA-7601) Define a-priori selectivity and NDV values

2018-09-20 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7601: --- Summary: Define a-priori selectivity and NDV values Key: IMPALA-7601 URL: https://issues.apache.org/jira/browse/IMPALA-7601 Project: IMPALA Issue Type:

[jira] [Commented] (IMPALA-7601) Define a-priori selectivity and NDV values

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622709#comment-16622709 ] Paul Rogers commented on IMPALA-7601: - Please see [~tarmstr...@cloudera.com]'s comment in

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622712#comment-16622712 ] Paul Rogers edited comment on IMPALA-7310 at 9/20/18 9:08 PM: -- Per the

[jira] [Commented] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622712#comment-16622712 ] Paul Rogers commented on IMPALA-7310: - Per the suggestion of [~jeszyb], created IMPALA-7601 to

[jira] [Commented] (IMPALA-7604) In AggregationNode.computeStats, handle cardinality overflow better

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623051#comment-16623051 ] Paul Rogers commented on IMPALA-7604: - [~tarmstrong], in my experience, using planner memory

[jira] [Commented] (IMPALA-7603) Incorrect NDV expression for col1 op col2

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622779#comment-16622779 ] Paul Rogers commented on IMPALA-7603: - Turns out that a similar limitation exists for functions.

[jira] [Updated] (IMPALA-7603) Incorrect NDV expression for col1 op col2

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7603: Description: Consider the 

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622856#comment-16622856 ] Paul Rogers edited comment on IMPALA-7310 at 9/21/18 12:04 AM: --- The

[jira] [Created] (IMPALA-7603) Incorrect NDV expression for col1 op col2

2018-09-20 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7603: --- Summary: Incorrect NDV expression for col1 op col2 Key: IMPALA-7603 URL: https://issues.apache.org/jira/browse/IMPALA-7603 Project: IMPALA Issue Type: Bug

[jira] [Created] (IMPALA-7604) In AggregationNode.computeStats, handle cardinality overflow better

2018-09-20 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7604: --- Summary: In AggregationNode.computeStats, handle cardinality overflow better Key: IMPALA-7604 URL: https://issues.apache.org/jira/browse/IMPALA-7604 Project: IMPALA

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-20 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622856#comment-16622856 ] Paul Rogers edited comment on IMPALA-7310 at 9/20/18 11:36 PM: --- The

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621434#comment-16621434 ] Paul Rogers edited comment on IMPALA-7310 at 9/20/18 3:02 AM: -- Odd. Looked

[jira] [Commented] (IMPALA-7560) Better selectivity estimate for != (not equals) binary predicate

2018-09-21 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624019#comment-16624019 ] Paul Rogers commented on IMPALA-7560: - Created a unit test for this. {noformat} runTest("SELECT

[jira] [Created] (IMPALA-7608) Estimate row count from file size when no stats available

2018-09-21 Thread Paul Rogers (JIRA)
Paul Rogers created IMPALA-7608: --- Summary: Estimate row count from file size when no stats available Key: IMPALA-7608 URL: https://issues.apache.org/jira/browse/IMPALA-7608 Project: IMPALA

[jira] [Updated] (IMPALA-7601) Define a-priori selectivity and NDV values

2018-09-21 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Description: Impala makes extensive use of table stats during query planning. For example, the

[jira] [Updated] (IMPALA-7608) Estimate row count from file size when no stats available

2018-09-21 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7608: Description: Impala makes heavy use of stats, which is a good thing. Stats feed into query

[jira] [Updated] (IMPALA-7603) Incorrect NDV expression for col1 mathop col2

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7603: Summary: Incorrect NDV expression for col1 mathop col2 (was: Incorrect NDV expression for col1

[jira] [Updated] (IMPALA-7601) Improve default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Summary: Improve default selectivity values (was: Define better default selectivity values) >

[jira] [Updated] (IMPALA-7601) Define better default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Summary: Define better default selectivity values (was: Define a-priori selectivity and NDV

[jira] [Updated] (IMPALA-7601) Improve default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Description: Impala makes extensive use of table stats during query planning. For example, the

[jira] [Updated] (IMPALA-7601) Improve default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Description: Impala makes extensive use of table stats during query planning. For example, the

[jira] [Commented] (IMPALA-7601) Improve default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626203#comment-16626203 ] Paul Rogers commented on IMPALA-7601: - Based on the above reasoning, here is a recommended revised

[jira] [Updated] (IMPALA-7601) Improve default selectivity values

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Description: Impala makes extensive use of table stats during query planning. For example, the

[jira] [Updated] (IMPALA-7601) Improve cardinality and selectivity estimates

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rogers updated IMPALA-7601: Description: Impala makes extensive use of table stats during query planning. For example, the

[jira] [Comment Edited] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626577#comment-16626577 ] Paul Rogers edited comment on IMPALA-7564 at 9/25/18 12:24 AM: --- Great

[jira] [Commented] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626577#comment-16626577 ] Paul Rogers commented on IMPALA-7564: - Great description. I think we can tease apart some of the

[jira] [Commented] (IMPALA-7604) In AggregationNode.computeStats, handle cardinality overflow better

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626508#comment-16626508 ] Paul Rogers commented on IMPALA-7604: - Thanks, [~tarmstrong], for the very clear explanation. bq. I

[jira] [Comment Edited] (IMPALA-7564) Conservative FK/PK join type detection with complex equi-join conjuncts

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626577#comment-16626577 ] Paul Rogers edited comment on IMPALA-7564 at 9/25/18 12:20 AM: --- Great

[jira] [Commented] (IMPALA-7601) Improve cardinality and selectivity estimates

2018-09-24 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626643#comment-16626643 ] Paul Rogers commented on IMPALA-7601: - [~tarmstrong], thanks for the note. My thought is to make

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626483#comment-16626483 ] Paul Rogers edited comment on IMPALA-7310 at 9/26/18 7:44 PM: -- Final

[jira] [Comment Edited] (IMPALA-7310) Compute Stats not computing NULLs as a distinct value causing wrong estimates

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622856#comment-16622856 ] Paul Rogers edited comment on IMPALA-7310 at 9/26/18 7:45 PM: -- The planner

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers edited comment on IMPALA-7501 at 9/26/18 11:35 PM: --- A quick

[jira] [Commented] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers commented on IMPALA-7501: - Analysis: * Impala's {{LocalCatalog}} contains a list of

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers edited comment on IMPALA-7501 at 9/26/18 11:07 PM: --- Analysis:

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers edited comment on IMPALA-7501 at 9/26/18 11:07 PM: --- Analysis:

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers edited comment on IMPALA-7501 at 9/26/18 11:21 PM: --- Analysis:

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629410#comment-16629410 ] Paul Rogers edited comment on IMPALA-7501 at 9/26/18 11:21 PM: --- Analysis:

[jira] [Commented] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629621#comment-16629621 ] Paul Rogers commented on IMPALA-7501: - So the above was probably looking in the wrong haystack.

[jira] [Commented] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629554#comment-16629554 ] Paul Rogers commented on IMPALA-7501: - Analysis: * Impala's {{LocalCatalog}} contains a list of

[jira] [Comment Edited] (IMPALA-7501) Slim down metastore Partition objects in LocalCatalog cache

2018-09-26 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629554#comment-16629554 ] Paul Rogers edited comment on IMPALA-7501 at 9/27/18 12:09 AM: --- Analysis:

[jira] [Commented] (IMPALA-7560) Better selectivity estimate for != (not equals) binary predicate

2018-09-19 Thread Paul Rogers (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620946#comment-16620946 ] Paul Rogers commented on IMPALA-7560: - The table in DRILL-5254 suggests how to use the NDV value to

<    1   2   3   4   5   6   7   8   9   >