[jira] [Commented] (CALCITE-4014) calcite convert(column using utf8mb4) throws exception
[ https://issues.apache.org/jira/browse/CALCITE-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113884#comment-17113884 ] groobyming commented on CALCITE-4014: - Thanks [~danny0405] :) > calcite convert(column using utf8mb4) throws exception > -- > > Key: CALCITE-4014 > URL: https://issues.apache.org/jira/browse/CALCITE-4014 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.21.0 >Reporter: groobyming >Priority: Major > Attachments: image-2020-05-20-17-00-28-611.png > > > Hello everyone, I found that when performing the following sql > verification, I will be prompted that the UTF8MB4 field cannot be found in > the table, but after reading the source code calcite is built-in to support > this convert function, is it not fully implemented yet? > > SQL is as follows: > select convert(name using utf8mb4) from test b where name = '4' > > Error Message: > !image-2020-05-20-17-00-28-611.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (CALCITE-3221) Add a sort-merge union algorithm
[
https://issues.apache.org/jira/browse/CALCITE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis reassigned CALCITE-3221:
Assignee: (was: Stamatis Zampetakis)
> Add a sort-merge union algorithm
>
>
> Key: CALCITE-3221
> URL: https://issues.apache.org/jira/browse/CALCITE-3221
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Stamatis Zampetakis
>Priority: Minor
>
> Currently, the union operation offered by Calcite is based on a {{HashSet}}
> (see
> [EnumerableDefaults.union|https://github.com/apache/calcite/blob/d98856bf1a5f5c151d004b769e14bdd368a67234/linq4j/src/main/java/org/apache/calcite/linq4j/EnumerableDefaults.java#L2747])
> and necessitates reading in memory all rows before returning a single
> result.
> Apart from increased memory consumption the operator is blocking and also
> destroys the order of its inputs.
> The goal of this issue is to add a new union algorithm (EnumerableMergeUnion
> ?) exploiting the fact that the inputs are sorted which consumes less memory
> and retains the order of its inputs.
> Most likely the implementation of the merge join can be useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3541) Avoid transformations to Enumerable nodes for custom SqlOperators
[
https://issues.apache.org/jira/browse/CALCITE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis reassigned CALCITE-3541:
Assignee: (was: Stamatis Zampetakis)
> Avoid transformations to Enumerable nodes for custom SqlOperators
> -
>
> Key: CALCITE-3541
> URL: https://issues.apache.org/jira/browse/CALCITE-3541
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> Most Enumerable converter rules apply a transformation as soon as the
> {{RelNode}} class and its convention match those defined by the rule.
> However, there are use-cases that we would like to restrict the matches even
> further to avoid generating unimplementable plans that will fail at runtime.
> The most prominent example comes from extending the standard operator set
> with new {{SqlOperator}} s that appear in filters and projections as part of
> a row expression ({{RexNode}}). If we use the default instance of the
> {{EnumerableCalcRule}} we might end-up with a plan that will fail at runtime
> since the new operator is not handled by the Enumerable convention. Most
> likely there is a {{RelNode}} in another convention that can handle this new
> operator.
> We could avoid such undesirable transformations by allowing instantiations of
> the Enumerable converter rules with user-defined predicates. This also means
> adding public constructors to the rules that currently they do not have one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-4019) Visit SqlInsert with SqlShuttle cause NullPointerException
[
https://issues.apache.org/jira/browse/CALCITE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113957#comment-17113957
]
xzh_dz commented on CALCITE-4019:
-
Hi [~yanlin-Lynn]~
I have much interest on this JIRA and want to work on it ~
Can you assign this one to me ?
> Visit SqlInsert with SqlShuttle cause NullPointerException
> --
>
> Key: CALCITE-4019
> URL: https://issues.apache.org/jira/browse/CALCITE-4019
> Project: Calcite
> Issue Type: Bug
>Reporter: Wang Yanlin
>Priority: Major
>
> When visit a SqlInsert with SqlShuttle, got NPE, the stack trace is
> {noformat}
> java.lang.NullPointerException
> at com.google.common.collect.Iterators$8.transform(Iterators.java:817)
> at
> com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at org.apache.calcite.sql.parser.SqlParserPos.sum(SqlParserPos.java:274)
> at
> org.apache.calcite.sql.parser.SqlParserPos.plusAll(SqlParserPos.java:183)
> at org.apache.calcite.sql.SqlOperator.createCall(SqlOperator.java:251)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.result(SqlShuttle.java:117)
> at
> org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.result(SqlShuttle.java:101)
> at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:67)
> at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:39)
> at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
> at
> org.apache.calcite.sql.parser.SqlParserTest.testSqlParserPosPlus(SqlParserTest.java:8791)
> {noformat}
> Add this case in SqlParserTest to reproduce.
> return a new same SqlIdentifier for each SqlIdentifier, make sure
> [update|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql/util/SqlShuttle.java#L117]
>
> happends
> {noformat}
> @Test void testSqlParserPosPlus() throws Exception {
> final String sql = "insert into emps select * from emps";
> SqlNode sqlNode = getSqlParser(sql).parseStmt();
> sqlNode.accept(new SqlShuttle(){
> @Override
> public SqlNode visit(SqlIdentifier identifier) {
> return new SqlIdentifier(identifier.names,
> identifier.getParserPosition());
> }
> });
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3986) Provide config option to omit assigning an alias that is identical to its identifier.
[ https://issues.apache.org/jira/browse/CALCITE-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114299#comment-17114299 ] Julian Hyde commented on CALCITE-3986: -- I agree. We should eliminate "AS abc" if column "abc" would have had that alias already. The rules for deriving aliases of expressions are DBMS-specific, so we will have to be careful. Also we'll have to be careful about case sensitivity. "SELECT abc AS "abc" FROM t" would be redundant on PostgreSQL but not on Oracle. BigQuery has particularly has particularly arcane rules about case-sensitivity - tables and columns have different case sensitivity rules, and I don't even know about table aliases and column aliases - so special care required there. > Provide config option to omit assigning an alias that is identical to its > identifier. > - > > Key: CALCITE-3986 > URL: https://issues.apache.org/jira/browse/CALCITE-3986 > Project: Calcite > Issue Type: Improvement >Reporter: Drew Schmitt >Priority: Minor > > The current alias assignment logic with the SqlValidator config option > identifierExpansion() set to true will eagerly try to choose an alias, even > if the alias name ends up being the same as the identifier itself. For > example, it will assign the alias "foo" for the table "foo". > It's nice to have alias assignment, but making it identical to the identifier > doesn't always provide a whole lot of value and can make queries more verbose > than they need to be. > Suggest we add another SqlValidator config option (defaulted to the current > behavior) to allow a caller to disable alias assignment in the cases that it > would be identical to the identifier. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3221) Add a sort-merge union algorithm
[
https://issues.apache.org/jira/browse/CALCITE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114302#comment-17114302
]
Haisheng Yuan commented on CALCITE-3221:
Looks like CALCITE-4017 has a dependency on this JIRA.
I would not suggest adding a new physical operator {{EnumerableMergeUnion}}. We
can extends {{EnumerableUnion}} by adding a field {{collation}}, to indicate
whether to preserve ordering and what is the order. Default is EMPTY, which
means no ordering needs to be preserved. Because if the parent operator doesn't
require any collation, it is always good to not preserve order. If it is
required by parent operator, we can pass down the required collation to UNION's
children on demand, and returns its new collation, by overriding
{{EnumerableUnion#passThroughTraits()}}.
> Add a sort-merge union algorithm
>
>
> Key: CALCITE-3221
> URL: https://issues.apache.org/jira/browse/CALCITE-3221
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: Stamatis Zampetakis
>Priority: Minor
>
> Currently, the union operation offered by Calcite is based on a {{HashSet}}
> (see
> [EnumerableDefaults.union|https://github.com/apache/calcite/blob/d98856bf1a5f5c151d004b769e14bdd368a67234/linq4j/src/main/java/org/apache/calcite/linq4j/EnumerableDefaults.java#L2747])
> and necessitates reading in memory all rows before returning a single
> result.
> Apart from increased memory consumption the operator is blocking and also
> destroys the order of its inputs.
> The goal of this issue is to add a new union algorithm (EnumerableMergeUnion
> ?) exploiting the fact that the inputs are sorted which consumes less memory
> and retains the order of its inputs.
> Most likely the implementation of the merge join can be useful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (CALCITE-784) LogicalAggregate's create method discards any collation traits from input
[ https://issues.apache.org/jira/browse/CALCITE-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haisheng Yuan resolved CALCITE-784. --- Resolution: Invalid Closing it, since it should be done on physical operators, not logical operators. > LogicalAggregate's create method discards any collation traits from input > - > > Key: CALCITE-784 > URL: https://issues.apache.org/jira/browse/CALCITE-784 > Project: Calcite > Issue Type: Bug > Components: core, stream >Reporter: Milinda Pathirage >Priority: Major > Attachments: CALCITE-784-0.patch > > > LogicalAggregate's create method gets the trait set of Convention.NONE from > input's cluster, but doesn't use any trait information from the input. But to > have proper collation trait set we need to consider input's collation trait > set when inferring LogicalAggregate's traits. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3541) Avoid transformations to Enumerable nodes for custom SqlOperators
[
https://issues.apache.org/jira/browse/CALCITE-3541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114290#comment-17114290
]
Xiening Dai commented on CALCITE-3541:
--
hi [~zabetak], not sure what you mean by adding a predicate. I think we already
have a user predicate for ConverterRule constructor today, no?
{code:java}
public ConverterRule(Class clazz,
Predicate predicate, RelTrait in, RelTrait out,
RelBuilderFactory relBuilderFactory, String descriptionPrefix) {
}
{code}
> Avoid transformations to Enumerable nodes for custom SqlOperators
> -
>
> Key: CALCITE-3541
> URL: https://issues.apache.org/jira/browse/CALCITE-3541
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.21.0
>Reporter: Stamatis Zampetakis
>Priority: Major
>
> Most Enumerable converter rules apply a transformation as soon as the
> {{RelNode}} class and its convention match those defined by the rule.
> However, there are use-cases that we would like to restrict the matches even
> further to avoid generating unimplementable plans that will fail at runtime.
> The most prominent example comes from extending the standard operator set
> with new {{SqlOperator}} s that appear in filters and projections as part of
> a row expression ({{RexNode}}). If we use the default instance of the
> {{EnumerableCalcRule}} we might end-up with a plan that will fail at runtime
> since the new operator is not handled by the Enumerable convention. Most
> likely there is a {{RelNode}} in another convention that can handle this new
> operator.
> We could avoid such undesirable transformations by allowing instantiations of
> the Enumerable converter rules with user-defined predicates. This also means
> adding public constructors to the rules that currently they do not have one.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3972) Allow RelBuilder to create RelNode with convention and use it for trait convert
[ https://issues.apache.org/jira/browse/CALCITE-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114280#comment-17114280 ] Xiening Dai commented on CALCITE-3972: -- We discover that with this change there are a few rules (SortProjectTranspose, SortJoinTranspose, SortJoinCopy, etc) which choose to match LogicalSort won't fire on the EnumerableSort enforcer. Although all the UTs are passed, potentially there are missing opportunities for better plans. And it cannot be fixed by simply adding an enumerable instance of the rule (to match Enumerable join and sort instead) because the rule implementation today doesn't consider join input trait requirement, and would generate a wrong alternative. So it feels to me that to solve this problem completely, we should use the trait propagation framework instead (CALCITE-3896) and deprecate these rules in long run. So part of this change - generating EnumerableSort during trait convert, would have to be removed. But I still see other usage scenarios of RelBuilder with conventions. I would like to still commit this change, so I can work on subsequent issues (such as CALCITE-3455). > Allow RelBuilder to create RelNode with convention and use it for trait > convert > --- > > Key: CALCITE-3972 > URL: https://issues.apache.org/jira/browse/CALCITE-3972 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > 1. Provide Convention.transformRelBuilder() to transform an existing > RelBuilder into one with specific convention. > 2. RelBuilder provides withRelFactories() method to allow caller swap the > underlying RelFactories and create a new builder. > 3. Use the new interface in RelCollationTraitDef for converting into > RelCollation traits > We can avoid ~1/3 of total rule firings in a N way join case with this change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3455) Redundant rule firing for both logical and physical nodes
[ https://issues.apache.org/jira/browse/CALCITE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114283#comment-17114283 ] Xiening Dai commented on CALCITE-3455: -- After adding transformation rule, this is no longer an issue for Enumerable. But for any other conventions that don't inherit PhysicalNode, there are still redundant rule firings. Once CALCITE-3972 is resolved, we can create new physical nodes with the convention specified to avoid additional rule firing. > Redundant rule firing for both logical and physical nodes > - > > Key: CALCITE-3455 > URL: https://issues.apache.org/jira/browse/CALCITE-3455 > Project: Calcite > Issue Type: Bug >Reporter: Xiening Dai >Assignee: Xiening Dai >Priority: Major > Labels: pull-request-available > Time Spent: 4h 40m > Remaining Estimate: 0h > > While I was looking at CALCITE-2970, I noticed that some of the rules are > fired for both logical and physical nodes. For example, ProjectMergeRule > matches Project.class, so it’s fired for LogicalProject. But then after > LogicalProject is converted into EnummerableProject, the same rule is fired > again for the physical rels. Same for EnumerableLimitRule, > SortRemoveConstantKeysRule, etc. > This seems to be unnecessary. When ProjectMerge is applied to LogicalProject > nodes, we already generate all possible alternatives with merged projects. We > just need to convert the LogicalProject into EnumerableProject. There’s no > need to merge EnumerableProject again. > If I update those rules to only match logical nodes, the planning time of the > case in CALCITE-2970 is reduced ~30%. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-4017) Implement trait propagation for Enumerable Setop
[ https://issues.apache.org/jira/browse/CALCITE-4017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17114314#comment-17114314 ] Haisheng Yuan commented on CALCITE-4017: [~amaliujia] I think we can still move forward. We just want the plan, not the execution. > Implement trait propagation for Enumerable Setop > - > > Key: CALCITE-4017 > URL: https://issues.apache.org/jira/browse/CALCITE-4017 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: Haisheng Yuan >Priority: Major > > Mainly for Union operator, not sure about Minus and Intersect. I haven't > check how is Enumerable Minus, Intersect's executor implemented. -- This message was sent by Atlassian Jira (v8.3.4#803005)
