[jira] [Resolved] (CALCITE-1654) Serialized messages are generated even though they are not logged
[ https://issues.apache.org/jira/browse/CALCITE-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved CALCITE-1654. - Resolution: Fixed Fixed in https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=commit;h=1b11385687171946d961fc4c836ee348f682b00a > Serialized messages are generated even though they are not logged > - > > Key: CALCITE-1654 > URL: https://issues.apache.org/jira/browse/CALCITE-1654 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Sergey Soldatov >Assignee: Josh Elser >Priority: Critical > Fix For: avatica-1.10.0 > > > [~sergey.soldatov] made a good find through some careful log reading. > In ProtobufTranslationImpl, some TRACE logging that I had added in > CALCITE-1159 is causing a bunch of garbage to be created on the heap. > The trace logging messages should be wrapped in a conditionals to avoid the > generation of the message itself. I was a little too overzealous thinking > that the slf4j api was saving me. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CALCITE-1654) Serialized messages are generated even though they are not logged
[ https://issues.apache.org/jira/browse/CALCITE-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879235#comment-15879235 ] Josh Elser commented on CALCITE-1654: - Put up a pull-request with the trivial change.. https://github.com/apache/calcite/pull/380 > Serialized messages are generated even though they are not logged > - > > Key: CALCITE-1654 > URL: https://issues.apache.org/jira/browse/CALCITE-1654 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Sergey Soldatov >Assignee: Josh Elser >Priority: Critical > Fix For: avatica-1.10.0 > > > [~sergey.soldatov] made a good find through some careful log reading. > In ProtobufTranslationImpl, some TRACE logging that I had added in > CALCITE-1159 is causing a bunch of garbage to be created on the heap. > The trace logging messages should be wrapped in a conditionals to avoid the > generation of the message itself. I was a little too overzealous thinking > that the slf4j api was saving me. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CALCITE-1069) In Aggregate, combine GROUPING columns into one GROUP_ID column
[
https://issues.apache.org/jira/browse/CALCITE-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879141#comment-15879141
]
Julian Hyde commented on CALCITE-1069:
--
I'm thinking of an alternative solution. Currently, as you know, an
{{Aggregate}} with more than one grouping set returns more columns than one
with only one grouping set. We have been arguing in HIVE-12923 about whether
there should be 1 extra column (Hive's preference) or N extra columns
(Calcite's preference).
My new proposal is that there should be no extra columns. We make {{GROUPING}}
into an aggregate function, and if you want those extra columns you can add
calls to {{GROUPING}}.
If the row type of {{Aggregate}} is same regardless of the number of grouping
sets, it will simplify a bunch of things. For example, it would be easier to
write a rule that pushes down the Filter "group_id = 2", because we wouldn't
have to worry about disappearing columns, and whether they are used.
> In Aggregate, combine GROUPING columns into one GROUP_ID column
> ---
>
> Key: CALCITE-1069
> URL: https://issues.apache.org/jira/browse/CALCITE-1069
> Project: Calcite
> Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Julian Hyde
>
> Grouping sets are currently implemented in Calcite using a bit to indicate
> each
> of the grouping columns. For instance, consider the following group by clause:
> GROUP BY CUBE (a, b)
> The generated Aggregate operator in Calcite will have a row schema consisting
> of [a, b, GROUPING(a), GROUPING(b)], where GROUPING( x ) is a boolean field
> indicator which represents whether x is participating in the group by clause.
> In contrast, Hive's implementation stores a single number corresponding to
> the GROUPING bit vector associated with a row (this is the result of the
> GROUPING_ID function in RDBMS such as MSSQLServer, Oracle, etc). Thus, the
> row schema of the Aggregate operator is [a, b, GROUPING_ID(a,b)].
> This difference is creating a mismatch between Calcite and Hive. As of now,
> we work around this mismatch in the Hive side: we create our own GROUPING_ID
> function applied over those columns. However, we have some issues related to
> predicates pushdown, constant propagation, join project transpose rule
> (HIVE-12923)
> etc., that we need to continue solving as new rules are added to Hive
> optimizer. In short, this is making the code on the Hive side harder and
> harder to maintain.
> This jira is intended to modify the implementation on the Calcite side to
> that we need not make workarounds/hacks in Hive to support Grouping IDs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1515) Support TableFunctionScan in RelBuilder
[
https://issues.apache.org/jira/browse/CALCITE-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879107#comment-15879107
]
Julian Hyde commented on CALCITE-1515:
--
Regarding 1. Note that {{RelBuilder.scan}} calls {{RelOptTable
RelOptSchema.getTableForMember(List names)}}. We'd need something
analogous for table-functions.
Also, {{RelBuilder}} has a field {{private final RelFactories.TableScanFactory
scanFactory}} and it will need an analogous field {{private final
RelFactories.TableFunctionScanFactory tableFunctionScanFactory}}. {{interface
TableFunctionScanFactory}} does not exist yet.
> Support TableFunctionScan in RelBuilder
> ---
>
> Key: CALCITE-1515
> URL: https://issues.apache.org/jira/browse/CALCITE-1515
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Anton Mushin
>Assignee: Julian Hyde
>Priority: Minor
>
> Calcite has a TableFunctionScan, which is a RelNode, takes a function, 0 or
> more RelNode inputs, and 0 or more other arguments.
> RelBuilder does not support TableFunctionScan yet.
> add In RelBuilder TableFunctionScanFactory which will create
> LogicalTableFunctionScan like as for LogicalTableScan and factory for others
> RelNodes [
> [example|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L125]
> ]
> also adding a functionScan method, analogous to the scan method but for table
> functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (CALCITE-1652) Allow GROUPING to have multiple arguments, like GROUPING_ID
[
https://issues.apache.org/jira/browse/CALCITE-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde resolved CALCITE-1652.
--
Resolution: Fixed
Fix Version/s: 1.12.0
Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/abfcc79b.
> Allow GROUPING to have multiple arguments, like GROUPING_ID
> ---
>
> Key: CALCITE-1652
> URL: https://issues.apache.org/jira/browse/CALCITE-1652
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
> Fix For: 1.12.0
>
>
> When initially added to the SQL standard {{GROUPING}} took one argument, a
> column reference. Now it can have multiple arguments, like the
> {{GROUPING_ID}} function.
> In SQL standard 2014, see section 6.9 set function specification, and
> feature T433, "Multiargument GROUPING function".
> This change would extend {{GROUPING}} to allow one or more arguments (i.e.
> what {{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
> {{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
> arguments.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1515) Support TableFunctionScan in RelBuilder
[
https://issues.apache.org/jira/browse/CALCITE-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879056#comment-15879056
]
Julian Hyde commented on CALCITE-1515:
--
Regarding 2: use {{ImmutableList.copyOf(inputs)}}.
> Support TableFunctionScan in RelBuilder
> ---
>
> Key: CALCITE-1515
> URL: https://issues.apache.org/jira/browse/CALCITE-1515
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Anton Mushin
>Assignee: Julian Hyde
>Priority: Minor
>
> Calcite has a TableFunctionScan, which is a RelNode, takes a function, 0 or
> more RelNode inputs, and 0 or more other arguments.
> RelBuilder does not support TableFunctionScan yet.
> add In RelBuilder TableFunctionScanFactory which will create
> LogicalTableFunctionScan like as for LogicalTableScan and factory for others
> RelNodes [
> [example|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/tools/RelBuilder.java#L125]
> ]
> also adding a functionScan method, analogous to the scan method but for table
> functions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1641) Base functions support for MATCH_RECOGNIZE
[
https://issues.apache.org/jira/browse/CALCITE-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879048#comment-15879048
]
Julian Hyde commented on CALCITE-1641:
--
The Oracle documentation is copyright. I don't think you can copy it. You need
to describe the API in your own words.
> Base functions support for MATCH_RECOGNIZE
> --
>
> Key: CALCITE-1641
> URL: https://issues.apache.org/jira/browse/CALCITE-1641
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Affects Versions: 1.11.0
>Reporter: Zhiqiang He
>Assignee: Zhiqiang He
> Labels: features
>
> MATCH_RECOGNIZE syntax like
> this:https://docs.oracle.com/database/121/DWHSG/pattern.htm#DWHSG8980
> Only pattern and define is supported in first step.
> h1. PATTERN: Defining the Row Pattern to Be Matched
> The PATTERN keyword specifies the pattern to be recognized in the ordered
> sequence of rows in a partition. Each variable name in a pattern corresponds
> to a Boolean condition, which is specified later using the DEFINE component
> of the syntax.
> The PATTERN clause is used to specify a regular expression. It is outside the
> scope of this material to explain regular expression concepts and details. If
> you are not familiar with regular expressions, you are encouraged to
> familiarize yourself with the topic using other sources.
> The regular expression in a PATTERN clause is enclosed in parentheses.
> PATTERN may use the following operators:
> * Concatenation
> Concatenation is used to list two or more items in a pattern to be matched in
> that order. Items are concatenated when there is no operator sign between two
> successive items. For example: PATTERN (A B C).
> * Quantifiers
> Quantifiers define the number of iterations accepted for a match. Quantifiers
> are postfix operators with the following choices:
> ** \* — 0 or more iterations
> ** + — 1 or more iterations
> ** ? — 0 or 1 iterations
> ** {n} — n iterations (n > 0)
> ** {n,} — n or more iterations (n >= 0)
> ** {n,m} — between n and m (inclusive) iterations (0 <= n <= m, 0 < m)
> ** {,m} — between 0 and m (inclusive) iterations (m > 0)
> ** reluctant quantifiers — indicated by an additional question mark following
> a quantifier (*?, +?, ??, {n,}?, { n, m }?, {,m}?). See "Reluctant Versus
> Greedy Quantifier" for the difference between reluctant and non-reluctant
> quantifiers.
> The following are examples of using quantifier operators:
> ** A* matches 0 or more iterations of A
> ** A{3,6} matches 3 to 6 iterations of A
> ** A{,4} matches 0 to 4 iterations of A
> * Alternation
> Alternation matches a single regular expression from a list of several
> possible regular expressions. The alternation list is created by placing a
> vertical bar (|) between each regular expression. Alternatives are preferred
> in the order they are specified. As an example, PATTERN (A | B | C) attempts
> to match A first. If A is not matched, it attempts to match B. If B is not
> matched, it attempts to match C.
> * Grouping
> Grouping treats a portion of the regular expression as a single unit,
> enabling you to apply regular expression operators such as quantifiers to
> that group. Grouping is created with parentheses. As an example, PATTERN ((A
> B){3} C) attempts to match the group (A B) three times and then seeks one
> occurrence of C.
> * PERMUTE
> See "How to Express All Permutations" for more information.
> * Exclusion
> Parts of the pattern to be excluded from the output of ALL ROWS PER MATCH are
> enclosed between {- and -}. See "How to Exclude Portions of the Pattern from
> the Output".
> * Anchors
> Anchors work in terms of positions rather than rows. They match a position
> either at the start or end of a partition.
> ** ^ matches the position before the first row in the partition.
> ** $ matches the position after the last row in the partition.
> As an example, PATTERN (^A+$) will match only if all rows in a partition
> satisfy the condition for A. The resulting match spans the entire partition.
> * Empty pattern (), matches an empty set of rows
> This section contains the following topics:
> Reluctant Versus Greedy Quantifier
> Operator Precedence
> h2. Reluctant Versus Greedy Quantifier
> Pattern quantifiers are referred to as greedy; they will attempt to match as
> many instances of the regular expression on which they are applied as
> possible. The exception is pattern quantifiers that have a question mark ? as
> a suffix, and those are referred to as reluctant. They will attempt to match
> as few instances as possible of the regular expression on which they are
> applied.
> The difference between greedy and reluctant quantifiers appended to a single
> pattern variable is illustrated as follows: A* tries to map as many rows
[jira] [Commented] (CALCITE-1652) Allow GROUPING to have multiple arguments, like GROUPING_ID
[
https://issues.apache.org/jira/browse/CALCITE-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879012#comment-15879012
]
Julian Hyde commented on CALCITE-1652:
--
You're right. I was mistaken. GROUPING (and GROUPING_ID) accepts 1 or more
arguments.
> Allow GROUPING to have multiple arguments, like GROUPING_ID
> ---
>
> Key: CALCITE-1652
> URL: https://issues.apache.org/jira/browse/CALCITE-1652
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> When initially added to the SQL standard {{GROUPING}} took one argument, a
> column reference. Now it can have multiple arguments, like the
> {{GROUPING_ID}} function.
> In SQL standard 2014, see section 6.9 set function specification, and
> feature T433, "Multiargument GROUPING function".
> This change would extend {{GROUPING}} to allow one or more arguments (i.e.
> what {{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
> {{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
> arguments.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1439) Handling errors during constant reduction
[
https://issues.apache.org/jira/browse/CALCITE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879010#comment-15879010
]
Julian Hyde commented on CALCITE-1439:
--
Yes, let's do that. ReduceExpressionsRule compares the string representations
of the expressions before and after, so it would work neatly with your proposal.
> Handling errors during constant reduction
> -
>
> Key: CALCITE-1439
> URL: https://issues.apache.org/jira/browse/CALCITE-1439
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> A literal is a constant, and so a cast of a literal is also a constant, but
> when we do constant reduction sometimes there are errors if the cast is not
> valid. For example, the query
> {code}
> values cast('' as integer)
> {code}
> gives {{ExceptionInInitializerError}} and, to make matters worse, the Avatica
> JDBC driver does not catch the exception (because it is a
> {{java.lang.Error}}) and wrap it as a {{java.sql.SQLException}} as it ought
> to.
> Note that {{cast('1.2' as integer)}} is also invalid but {{cast(' -1 ' as
> integer)}} is valid.
> This issue probably also applies to divide-by-zero and other exceptions
> evaluating scalar expressions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878997#comment-15878997
]
Julian Hyde commented on CALCITE-1653:
--
[~rusanu], One more thing. The rex package should not depend on the plan
package (except for RelOptUtils). So, can you rename {{interface
RelOptPlanner.Executor}} to {{org.apache.calcite.rex.RexExecutor}}. If you
think it will help compatibility, deprecate {{RelOptPlanner.Executor}} and have
it as a base class for the new {{RexExecutor}}.
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Attachments: CALCITE-1653.00.patch
>
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878987#comment-15878987
]
Julian Hyde commented on CALCITE-1653:
--
[~jcamachorodriguez], See my comments on CALCITE-1417 about why we can't use
the planner's executor. Basically, there are life-cycle issues; there might not
be a planner available yet. CALCITE-1536 will improve life cycle issues but we
still might decide to not use the (expensive) full executor at every point in
the query preparation process. For instance, at sql-to-rel time, we might use a
simple executor that refuses to simplify {{CAST(string AS DATE)}}.
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Attachments: CALCITE-1653.00.patch
>
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1654) Serialized messages are generated even though they are not logged
Josh Elser created CALCITE-1654: --- Summary: Serialized messages are generated even though they are not logged Key: CALCITE-1654 URL: https://issues.apache.org/jira/browse/CALCITE-1654 Project: Calcite Issue Type: Bug Components: avatica Reporter: Sergey Soldatov Assignee: Josh Elser Priority: Critical Fix For: avatica-1.10.0 [~sergey.soldatov] made a good find through some careful log reading. In ProtobufTranslationImpl, some TRACE logging that I had added in CALCITE-1159 is causing a bunch of garbage to be created on the heap. The trace logging messages should be wrapped in a conditionals to avoid the generation of the message itself. I was a little too overzealous thinking that the slf4j api was saving me. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Hyde reassigned CALCITE-1653:
Assignee: Remus Rusanu (was: Julian Hyde)
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Attachments: CALCITE-1653.00.patch
>
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878939#comment-15878939
]
Julian Hyde commented on CALCITE-1653:
--
The change looks basically good. Can you please make the executor mandatory?
(The method overload without the executor would supply the default executor.)
Also can you please rebase onto latest master branch, squash, and remove merge
commits.
We may have to revisit this (and break things) when we fix CALCITE-1536. There
might no longer be a link from RelNode to RelOptPlanner; executor could become
a property of the cluster.
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
> Attachments: CALCITE-1653.00.patch
>
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1620) CAST('' AS BINARY) is supported by Hive, but not by Calcite
[
https://issues.apache.org/jira/browse/CALCITE-1620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878907#comment-15878907
]
Remus Rusanu commented on CALCITE-1620:
---
CALCITE-1653 makes this obsolete. Using the Hive executor during constant
simplification yields the same result as in Hive.
> CAST('' AS BINARY) is supported by Hive, but not by Calcite
> ---
>
> Key: CALCITE-1620
> URL: https://issues.apache.org/jira/browse/CALCITE-1620
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> Hive supports {{CAST('1' as BINARY)}}. Result is 0x31 (can be tested with
> {noformat}SELECT HEX(CAST('1' as BINARY));{noformat}. If I'm not mistaken,
> the CAST result is the UTF-8 encoding of the input string.
> CALCITE 1.11 throws AssertionException during compilation for such an
> expression:
> {noformat}
> Thread [db11b6e6-8fa7-48b3-82ef-c5b4550327ec main] (Suspended (exception
> AssertionError))
> ConstantExpression.(Type, Object) line: 50
> Expressions.constant(Object, Type) line: 586
> OptimizeShuttle.visit(UnaryExpression, Expression) line: 279
> UnaryExpression.accept(Shuttle) line: 37
> Expressions.acceptExpressions(List, Shuttle)
> line: 3184
> MethodCallExpression.accept(Shuttle) line: 60
> TernaryExpression.accept(Shuttle) line: 45
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878893#comment-15878893
]
Remus Rusanu commented on CALCITE-1653:
---
https://github.com/apache/calcite/pull/379
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
> Attachments: CALCITE-1653.00.patch
>
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878479#comment-15878479
]
Remus Rusanu edited comment on CALCITE-1653 at 2/22/17 5:31 PM:
In some cases the executor has to percolate through quite a lengthy stack to
reach the place is needed:
{noformat}
Thread [00a86df3-c98c-4766-b3ee-3ab82a63219e main] (Suspended (exception
NumberFormatException))
BigDecimal.(double, MathContext) line: 895
RexBuilder.clean(Object, RelDataType) line: 1390
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1237
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1234
RexExecutable.reduce(RexBuilder, List, List) line: 84
RexExecutorImpl.reduce(RexBuilder, List, List) line:
129
RexUtil.simplifyCast(RexBuilder, RexCall, RelOptPlanner$Executor) line:
2387 <-- we need the executor here
RexUtil.simplify(RexBuilder, RexNode, boolean, RelOptPlanner$Executor)
line: 1626
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2984
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2953
RexCall.accept(RexVisitor) line: 107
RexUtil$ExprSimplifier(RexShuttle).apply(RexNode) line: 275
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList, boolean) line: 468
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList) line: 445
ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(RelOptRuleCall)
line: 268
HepPlanner(AbstractRelOptPlanner).fireRule(RelOptRuleCall) line: 316
HepPlanner.applyRule(RelOptRule, HepRelVertex, boolean) line: 506
HepPlanner.applyRules(Collection, boolean) line: 385
HepPlanner.executeInstruction(HepInstruction$RuleCollection) line: 279
HepInstruction$RuleCollection.execute(HepPlanner) line: 72
HepPlanner.executeProgram(HepProgram) line: 210
HepPlanner.findBestExp() line: 197
CalcitePlanner$CalcitePlannerAction.hepPlan(RelNode, boolean,
RelMetadataProvider, RelOptPlanner$Executor, HepMatchOrder, RelOptRule...)
line: 1749 <-- we have the executor here
{noformat}
was (Author: rusanu):
In some cases the executor has to percolate through quite a lengthy stack to
reach the place is needed:
{{noformat}}
Thread [00a86df3-c98c-4766-b3ee-3ab82a63219e main] (Suspended (exception
NumberFormatException))
BigDecimal.(double, MathContext) line: 895
RexBuilder.clean(Object, RelDataType) line: 1390
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1237
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1234
RexExecutable.reduce(RexBuilder, List, List) line: 84
RexExecutorImpl.reduce(RexBuilder, List, List) line:
129
RexUtil.simplifyCast(RexBuilder, RexCall, RelOptPlanner$Executor) line:
2387 <-- we need the executor here
RexUtil.simplify(RexBuilder, RexNode, boolean, RelOptPlanner$Executor)
line: 1626
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2984
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2953
RexCall.accept(RexVisitor) line: 107
RexUtil$ExprSimplifier(RexShuttle).apply(RexNode) line: 275
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList, boolean) line: 468
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList) line: 445
ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(RelOptRuleCall)
line: 268
HepPlanner(AbstractRelOptPlanner).fireRule(RelOptRuleCall) line: 316
HepPlanner.applyRule(RelOptRule, HepRelVertex, boolean) line: 506
HepPlanner.applyRules(Collection, boolean) line: 385
HepPlanner.executeInstruction(HepInstruction$RuleCollection) line: 279
HepInstruction$RuleCollection.execute(HepPlanner) line: 72
HepPlanner.executeProgram(HepProgram) line: 210
HepPlanner.findBestExp() line: 197
CalcitePlanner$CalcitePlannerAction.hepPlan(RelNode, boolean,
RelMetadataProvider, RelOptPlanner$Executor, HepMatchOrder, RelOptRule...)
line: 1749 <-- we have the executor here
{{noformat}}
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878802#comment-15878802
]
Remus Rusanu commented on CALCITE-1653:
---
Turns out traversing that stack is not hard at all.
{{ReduceExpressionsRule.reduceExpressions}} can get the executor from the
{{RelNode}} and then extending {{RexUtil$ExprSimplifier}} to take an executor
is trivial. I'm testing it now and it looks promising.
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1651) Difference in DATE cast between Hive and Calcite
[
https://issues.apache.org/jira/browse/CALCITE-1651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878799#comment-15878799
]
Remus Rusanu commented on CALCITE-1651:
---
CALCITE-1653 makes this problem irrelevant as using the Hive executor for
constant simplification yields the expected result (expected by Hive, that is).
> Difference in DATE cast between Hive and Calcite
>
>
> Key: CALCITE-1651
> URL: https://issues.apache.org/jira/browse/CALCITE-1651
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> Hive {{CAST('' as DATE)}} is handled by Java's {{Date.valueOf}} and
> thus expects {{-mm-dd}}. In Hive {{SELECT CAST('1' as DATE);}} yields
> {{NULL}} but in Calcite is {{0001-12-31}}.
> I'm opening this as Calcite issue to track HIVE-15708 (migrate to Calcite
> 1.12) blockers, not sure if solution will require Calcite changes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1650) Difference in BOOLEAN cast between Hive and Calcite
[
https://issues.apache.org/jira/browse/CALCITE-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878789#comment-15878789
]
Remus Rusanu commented on CALCITE-1650:
---
In any case, CALCITE-1653 makes this a moot point because suing the Hive
executor for constant simplification circumvents the problem entirely and
yields results that are consistent with Hive.
> Difference in BOOLEAN cast between Hive and Calcite
> ---
>
> Key: CALCITE-1650
> URL: https://issues.apache.org/jira/browse/CALCITE-1650
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> Hive {{CAST( ... AS BOOLEAN)}} gives true for anything 'not zero' (eg.
> {{CAST(1 as BOOLEAN)}} is {{true}}) while Calcite uses strict Java
> {{Boolean.valueOf(...)}} which yields true only for the string 'true'.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1650) Difference in BOOLEAN cast between Hive and Calcite
[
https://issues.apache.org/jira/browse/CALCITE-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878781#comment-15878781
]
Remus Rusanu commented on CALCITE-1650:
---
I think the relevant snippet is 6.13 20) (SQL2011) which describes only the
string and boolean SD cases, so numeric source is not covered.
> Difference in BOOLEAN cast between Hive and Calcite
> ---
>
> Key: CALCITE-1650
> URL: https://issues.apache.org/jira/browse/CALCITE-1650
> Project: Calcite
> Issue Type: Bug
> Components: core
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> Hive {{CAST( ... AS BOOLEAN)}} gives true for anything 'not zero' (eg.
> {{CAST(1 as BOOLEAN)}} is {{true}}) while Calcite uses strict Java
> {{Boolean.valueOf(...)}} which yields true only for the string 'true'.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878569#comment-15878569
]
Jesus Camacho Rodriguez commented on CALCITE-1653:
--
Simplifying cast on constant was a good idea, but to remain as extensible as we
were before, I think we should rely on the planner associated Executor if it is
available, instead of using default Calcite executor. [~julianhyde], what do
you think? This should be an easy fix or am I missing something?
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
[
https://issues.apache.org/jira/browse/CALCITE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878479#comment-15878479
]
Remus Rusanu commented on CALCITE-1653:
---
In some cases the executor has to percolate through quite a lengthy stack to
reach the place is needed:
{{noformat}}
Thread [00a86df3-c98c-4766-b3ee-3ab82a63219e main] (Suspended (exception
NumberFormatException))
BigDecimal.(double, MathContext) line: 895
RexBuilder.clean(Object, RelDataType) line: 1390
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1237
RexBuilder.makeLiteral(Object, RelDataType, boolean) line: 1234
RexExecutable.reduce(RexBuilder, List, List) line: 84
RexExecutorImpl.reduce(RexBuilder, List, List) line:
129
RexUtil.simplifyCast(RexBuilder, RexCall, RelOptPlanner$Executor) line:
2387 <-- we need the executor here
RexUtil.simplify(RexBuilder, RexNode, boolean, RelOptPlanner$Executor)
line: 1626
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2984
RexUtil$ExprSimplifier.visitCall(RexCall) line: 2953
RexCall.accept(RexVisitor) line: 107
RexUtil$ExprSimplifier(RexShuttle).apply(RexNode) line: 275
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList, boolean) line: 468
ReduceExpressionsRule.reduceExpressions(RelNode, List,
RelOptPredicateList) line: 445
ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(RelOptRuleCall)
line: 268
HepPlanner(AbstractRelOptPlanner).fireRule(RelOptRuleCall) line: 316
HepPlanner.applyRule(RelOptRule, HepRelVertex, boolean) line: 506
HepPlanner.applyRules(Collection, boolean) line: 385
HepPlanner.executeInstruction(HepInstruction$RuleCollection) line: 279
HepInstruction$RuleCollection.execute(HepPlanner) line: 72
HepPlanner.executeProgram(HepProgram) line: 210
HepPlanner.findBestExp() line: 197
CalcitePlanner$CalcitePlannerAction.hepPlan(RelNode, boolean,
RelMetadataProvider, RelOptPlanner$Executor, HepMatchOrder, RelOptRule...)
line: 1749 <-- we have the executor here
{{noformat}}
> Allow for custom RelOptPlanner.Executor in RexUtil.simplify
> ---
>
> Key: CALCITE-1653
> URL: https://issues.apache.org/jira/browse/CALCITE-1653
> Project: Calcite
> Issue Type: Bug
>Reporter: Remus Rusanu
>Assignee: Julian Hyde
>
> {{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
> evaluating literals and expression in {{simplifyCast}}. This causes value
> differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (CALCITE-1653) Allow for custom RelOptPlanner.Executor in RexUtil.simplify
Remus Rusanu created CALCITE-1653:
-
Summary: Allow for custom RelOptPlanner.Executor in
RexUtil.simplify
Key: CALCITE-1653
URL: https://issues.apache.org/jira/browse/CALCITE-1653
Project: Calcite
Issue Type: Bug
Reporter: Remus Rusanu
Assignee: Julian Hyde
{{RexUtil.simplify}} (and derivatives) use a default {{EXECUTOR}} for
evaluating literals and expression in {{simplifyCast}}. This causes value
differences from Hive, triggering issues like CALCITE-1650, CALCITE-1651.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1439) Handling errors during constant reduction
[
https://issues.apache.org/jira/browse/CALCITE-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877968#comment-15877968
]
Jesus Camacho Rodriguez commented on CALCITE-1439:
--
{quote}
The only error that should be masked is if the constant reducer encounters a
SQL runtime error while evaluating the user's expression.
{quote}
+1 on that. [~julianhyde], what about returning the original expression in the
corresponding _reducedValues_ slot in that case? I think that would be a better
way to handle this case.
> Handling errors during constant reduction
> -
>
> Key: CALCITE-1439
> URL: https://issues.apache.org/jira/browse/CALCITE-1439
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> A literal is a constant, and so a cast of a literal is also a constant, but
> when we do constant reduction sometimes there are errors if the cast is not
> valid. For example, the query
> {code}
> values cast('' as integer)
> {code}
> gives {{ExceptionInInitializerError}} and, to make matters worse, the Avatica
> JDBC driver does not catch the exception (because it is a
> {{java.lang.Error}}) and wrap it as a {{java.sql.SQLException}} as it ought
> to.
> Note that {{cast('1.2' as integer)}} is also invalid but {{cast(' -1 ' as
> integer)}} is valid.
> This issue probably also applies to divide-by-zero and other exceptions
> evaluating scalar expressions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CALCITE-1652) Allow GROUPING to have multiple arguments, like GROUPING_ID
[
https://issues.apache.org/jira/browse/CALCITE-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated CALCITE-1652:
-
Description:
When initially added to the SQL standard {{GROUPING}} took one argument, a
column reference. Now it can have multiple arguments, like the {{GROUPING_ID}}
function.
In SQL standard 2014, see section 6.9 set function specification, and
feature T433, "Multiargument GROUPING function".
This change would extend {{GROUPING}} to allow one or more arguments (i.e. what
{{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
{{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
arguments.
was:
When initially added to the SQL standard {{GROUPING}} took one argument, a
column reference. Now it can have multiple arguments, like the {{GROUPING_ID}}
function.
In SQL standard 2014, see section 6.9 set function specification, and
feature T433, "Multiargument GROUPING function".
This change would extend {{GROUPING}} to allow zero or more arguments (i.e.
what {{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
{{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
arguments.
> Allow GROUPING to have multiple arguments, like GROUPING_ID
> ---
>
> Key: CALCITE-1652
> URL: https://issues.apache.org/jira/browse/CALCITE-1652
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> When initially added to the SQL standard {{GROUPING}} took one argument, a
> column reference. Now it can have multiple arguments, like the
> {{GROUPING_ID}} function.
> In SQL standard 2014, see section 6.9 set function specification, and
> feature T433, "Multiargument GROUPING function".
> This change would extend {{GROUPING}} to allow one or more arguments (i.e.
> what {{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
> {{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
> arguments.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CALCITE-1652) Allow GROUPING to have multiple arguments, like GROUPING_ID
[
https://issues.apache.org/jira/browse/CALCITE-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877960#comment-15877960
]
Jesus Camacho Rodriguez commented on CALCITE-1652:
--
Changes LGTM [~julianhyde], +1.
I have updated the description of the issue: GROUPING function accepts 'one or
more' arguments instead of 'zero or more'.
> Allow GROUPING to have multiple arguments, like GROUPING_ID
> ---
>
> Key: CALCITE-1652
> URL: https://issues.apache.org/jira/browse/CALCITE-1652
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Julian Hyde
>
> When initially added to the SQL standard {{GROUPING}} took one argument, a
> column reference. Now it can have multiple arguments, like the
> {{GROUPING_ID}} function.
> In SQL standard 2014, see section 6.9 set function specification, and
> feature T433, "Multiargument GROUPING function".
> This change would extend {{GROUPING}} to allow one or more arguments (i.e.
> what {{GROUPING_ID}} currently does), and make {{GROUPING_ID}} a synonym for
> {{GROUPING}}. {{GROUP_ID}} is not affected, and continues to take zero
> arguments.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
