[jira] [Commented] (CALCITE-6270) Support FORMAT in CAST from Numeric and BYTES to String (Enabled in BigQuery)
[ https://issues.apache.org/jira/browse/CALCITE-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827761#comment-17827761 ] Julian Hyde commented on CALCITE-6270: -- [~jerin_john], Please use Calcite's type names in jira summaries. > Support FORMAT in CAST from Numeric and BYTES to String (Enabled in BigQuery) > - > > Key: CALCITE-6270 > URL: https://issues.apache.org/jira/browse/CALCITE-6270 > Project: Calcite > Issue Type: Bug >Reporter: Jerin John >Assignee: Jerin John >Priority: Minor > > Calcite is missing the below formats supported in BQ CAST with FORMAT: > * [Numeric to > STRING|https://cloud.google.com/bigquery/docs/reference/standard-sql/format-elements#format_numeric_type_as_string] > * [BYTES <-> > STRING|https://cloud.google.com/bigquery/docs/reference/standard-sql/format-elements#format_bytes_as_string] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6327) getValidatedNodeTypeIfKnown should never throw
[ https://issues.apache.org/jira/browse/CALCITE-6327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827760#comment-17827760 ] Julian Hyde commented on CALCITE-6327: -- A test case would be useful. > getValidatedNodeTypeIfKnown should never throw > -- > > Key: CALCITE-6327 > URL: https://issues.apache.org/jira/browse/CALCITE-6327 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.36.0 >Reporter: Claude Brisson >Priority: Major > > During validation, when a SqlNode has been rewritten (for instance when a > COALESCE call has been rewritten as a CASE call) but does not yet have a > RelDataType, the method SqlValidatorImpl.getValidatedNodeTypeIfKnown() throws > an exception because it relies on > SqlValidatorImpl.getValidatedNodeType(originalExpr), not on > SqlValidatorImpl.getValidatedNodeTypeIfKnown(originalExpr). > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6327) getValidatedNodeTypeIfKnown should never throw
[ https://issues.apache.org/jira/browse/CALCITE-6327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827759#comment-17827759 ] Julian Hyde commented on CALCITE-6327: -- That sounds right. However, mutations of the AST make me nervous. I know we mutate the AST for a few reasons (e.g. expanding '*') but if you are doing an unsanctioned mutation then the blame is on you, not Calcite. (Since Calcite was originally written, I have become more of a pure functional programmer and I wish we should remove all mutations.) > getValidatedNodeTypeIfKnown should never throw > -- > > Key: CALCITE-6327 > URL: https://issues.apache.org/jira/browse/CALCITE-6327 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.36.0 >Reporter: Claude Brisson >Priority: Major > > During validation, when a SqlNode has been rewritten (for instance when a > COALESCE call has been rewritten as a CASE call) but does not yet have a > RelDataType, the method SqlValidatorImpl.getValidatedNodeTypeIfKnown() throws > an exception because it relies on > SqlValidatorImpl.getValidatedNodeType(originalExpr), not on > SqlValidatorImpl.getValidatedNodeTypeIfKnown(originalExpr). > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6317) Incorrect constant replacement when group keys are NULL
[
https://issues.apache.org/jira/browse/CALCITE-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827758#comment-17827758
]
Julian Hyde commented on CALCITE-6317:
--
Since the root cause is {{RelMdPredicates}}, can you add a test to
{{RelMetadataTest}}. It should be similar to {{testGetPredicatesForLiteralAgg}}.
> Incorrect constant replacement when group keys are NULL
> ---
>
> Key: CALCITE-6317
> URL: https://issues.apache.org/jira/browse/CALCITE-6317
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Mihai Budiu
>Assignee: Mihai Budiu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.37.0
>
>
> Here is a query taken from agg.iq:
> {code:sql}
> select deptno, gender, grouping_id(deptno, gender, deptno), count(*) as c
> from emp
> where deptno = 10
> group by rollup(gender, deptno)
> {code}
> The query plan initially is
> {code}
> LogicalProject(DEPTNO=[$1], GENDER=[$0], EXPR$2=[$2], C=[$3]), id = 72
> LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
> EXPR$2=[GROUPING_ID($1, $0, $1)], C=[COUNT()]), id = 71
> LogicalProject(GENDER=[$2], DEPTNO=[$1]), id = 70
> LogicalFilter(condition=[=($1, 10)]), id = 66
> LogicalTableScan(table=[[schema, EMP]]), id = 65
> {code}
> After applying PROJECT_REDUCE_EXPRESSIONS the plan looks like:
> {code}
> LogicalProject(DEPTNO=[CAST(10):INTEGER], GENDER=[$0], EXPR$2=[$2],
> C=[$3]), id = 82
> LogicalAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]],
> EXPR$2=[GROUPING_ID($1, $0, $1)], C=[COUNT()]), id = 78
> LogicalProject(GENDER=[$2], DEPTNO=[CAST(10):INTEGER]), id = 84
> LogicalFilter(condition=[=($1, 10)]), id = 74
> LogicalTableScan(table=[[schema, EMP]]), id = 65
> {code}
> The problem is in the outer LogicalProject, where the value 10 has replaced
> DEPTNO.
> However, DEPTNO can also be NULL, because of the groups in the
> LogicalAggregate.
> The constant should not be pushed past the aggregation.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6314) Add RANDOM function (enabled in Postgres library)
[ https://issues.apache.org/jira/browse/CALCITE-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827757#comment-17827757 ] James Duong commented on CALCITE-6314: -- Yes, my Jira issues were because I was a committer on the old Arrow board but was not yet a contributor on Calcite. > Add RANDOM function (enabled in Postgres library) > - > > Key: CALCITE-6314 > URL: https://issues.apache.org/jira/browse/CALCITE-6314 > Project: Calcite > Issue Type: Sub-task >Reporter: James Duong >Assignee: James Duong >Priority: Major > Labels: pull-request-available > Fix For: 1.37.0 > > > This is an alias for RAND(), except it should not support passing in a seed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827756#comment-17827756
]
Julian Hyde commented on CALCITE-6209:
--
Thank you for adding the link. And, thanks for the fix!
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Labels: pull-request-available
> Fix For: avatica-1.25.0
>
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
>
[jira] [Commented] (CALCITE-6314) Add RANDOM function (enabled in Postgres library)
[ https://issues.apache.org/jira/browse/CALCITE-6314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827755#comment-17827755 ] Julian Hyde commented on CALCITE-6314: -- Looks like you figured it out. By the way, when we're figuring out who did what work, we generally use the author of the git commit. > Add RANDOM function (enabled in Postgres library) > - > > Key: CALCITE-6314 > URL: https://issues.apache.org/jira/browse/CALCITE-6314 > Project: Calcite > Issue Type: Sub-task >Reporter: James Duong >Assignee: James Duong >Priority: Major > Labels: pull-request-available > Fix For: 1.37.0 > > > This is an alias for RAND(), except it should not support passing in a seed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-2980) Implement the FORMAT clause of the CAST operator
[
https://issues.apache.org/jira/browse/CALCITE-2980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827753#comment-17827753
]
Julian Hyde commented on CALCITE-2980:
--
A good test is really valuable. I wouldn't disable the whole test, because
you'll never find time to re-enable it. So, I'd take the time to selectively
disable tests. Use 'if' rather than comments, to minimize changes to the .iq
file.
It's possible that it's less effort to implement a format string element than
to disable a test. If so, do that.
> Implement the FORMAT clause of the CAST operator
>
>
> Key: CALCITE-2980
> URL: https://issues.apache.org/jira/browse/CALCITE-2980
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Jerin John
>Priority: Major
> Labels: pull-request-available
>
> SQL:2016 adds an optional {{FORMAT format}} clause to the {{CAST}} operator.
> It is a standard way to do what functions like {{TO_DATE}}, {{TO_NUMBER}},
> {{TO_CHAR}}, {{TO_TIMESTAMP}} have done in an ad hoc way (and with differing
> specifications among databases).
> Here is an example:
> {code:java}
> cast('01-05-2017' as date format 'DD-MM-')
> {code}
> The following paragraphs are copied from IMPALA-4018, which describes
> implementing this in Impala. (That case also describes cases where the
> implementations of {{TO_TIMESTAMP}} etc. in Hive, Impala, Oracle and
> PostgreSQL are not consistent with each other. We should take note as we
> implement these functions in Calcite.)
> SQL:2016 defines the following datetime templates
> {noformat}
> ::=
> { }...
> ::=
>
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> ::=
>
> |
> |
> |
> |
> |
> |
> |
> ::=
> | YYY | YY | Y
> ::=
> | RR
> ::=
> MM
> ::=
> DD
> ::=
> DDD
> ::=
> HH | HH12
> ::=
> HH24
> ::=
> MI
> ::=
> SS
> ::=
> S
> ::=
> FF1 | FF2 | FF3 | FF4 | FF5 | FF6 | FF7 | FF8 | FF9
> ::=
> A.M. | P.M.
> ::=
> TZH
> ::=
> TZM
> {noformat}
> SQL:2016 also introduced the {{FORMAT}} clause for {{CAST}} which is the
> standard way to do string <> datetime conversions
> {noformat}
> ::=
> CAST
>AS
> [ FORMAT ]
>
> ::=
>
> |
> ::=
>
> |
> ::=
>
> {noformat}
> For example:
> {noformat}
> CAST( AS [FORMAT ])
> CAST( AS [FORMAT ])
> cast(dt as string format 'DD-MM-')
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6332) Optimization CoreRules.AGGREGATE_EXPAND_DISTINCT_AGGREGATES_TO_JOIN produces incorrect results for aggregates with groupSets
[
https://issues.apache.org/jira/browse/CALCITE-6332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827752#comment-17827752
]
Julian Hyde commented on CALCITE-6332:
--
There are two algorithms to rewrite distinct aggregates. The older one uses
join. The newer one, introduced in CALCITE-732, uses grouping sets.
The difference is whether you call {{Config.withJoin(true)}}. As far as I know,
the newer one is superior in all cases. The older one may have bugs, or
limitations such as this one - not being able to handle an {{Aggregate}} with
grouping sets.
One solution is to make the join-based rule refuse to fire if it sees grouping
sets. The other is to discourage people from using that rule.
> Optimization CoreRules.AGGREGATE_EXPAND_DISTINCT_AGGREGATES_TO_JOIN produces
> incorrect results for aggregates with groupSets
>
>
> Key: CALCITE-6332
> URL: https://issues.apache.org/jira/browse/CALCITE-6332
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.36.0
>Reporter: Mihai Budiu
>Priority: Minor
>
> The optimization rule does not seem to consider the groupSets at all.
> The following two queries produce the same resulting plan:
> {code:sql}
> select count(distinct deptno) as cd, count(*) as c
> from emp
> group by cube(deptno)
> {code}
> {code:sql}
> select count(distinct deptno) as cd, count(*) as c
> from emp
> group by deptno
> {code}
> (Notice that one query has a cube, while the other one doesn't)
> The produced plan is:
> {code}
> LogicalProject(CD=[$1], C=[$2]), id = 196
> LogicalAggregate(group=[{0}], CD=[COUNT($0)], C=[$SUM0($1)]), id = 201
> LogicalAggregate(group=[{0}], C=[COUNT()]), id = 198
> LogicalProject(DEPTNO=[$8]), id = 192
> LogicalTableScan(table=[[schema, EMP]]), id = 163
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6330) Print the average row size when explaining an operator at the ALL_ATTRIBUTES level
[
https://issues.apache.org/jira/browse/CALCITE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mengdou updated CALCITE-6330:
-
Summary: Print the average row size when explaining an operator at the
ALL_ATTRIBUTES level (was: Print the average row size when explaining an
operator)
> Print the average row size when explaining an operator at the ALL_ATTRIBUTES
> level
> --
>
> Key: CALCITE-6330
> URL: https://issues.apache.org/jira/browse/CALCITE-6330
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: mengdou
>Assignee: mengdou
>Priority: Minor
>
> In this time, when we dump the plan of a RelNode Tree, there is no average
> row size exported in the output string, even if
> SqlExplainLevel.ALL_ATTRIBUTES is specified.
> Because the implementation in explain_() in class RelWriterImpl doesn't
> include the metric average_row_size:
>
> {code:java}
> switch (detailLevel) {
> case ALL_ATTRIBUTES:
> s.append(": rowcount = ")
> .append(mq.getRowCount(rel))
> .append(", cumulative cost = ")
> .append(mq.getCumulativeCost(rel));
> }
> switch (detailLevel) {
> case NON_COST_ATTRIBUTES:
> case ALL_ATTRIBUTES:
> if (!withIdPrefix) {
> // If we didn't print the rel id at the start of the line, print
> // it at the end.
> s.append(", id = ").append(rel.getId());
> }
> break;
> } {code}
>
> So I'd like to add this metric by calling md.getAverageRowSize(rel)
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6330) Print the average row size when explaining an operator
[
https://issues.apache.org/jira/browse/CALCITE-6330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mengdou updated CALCITE-6330:
-
Summary: Print the average row size when explaining an operator (was:
Print the average row size when when explaining an operator)
> Print the average row size when explaining an operator
> --
>
> Key: CALCITE-6330
> URL: https://issues.apache.org/jira/browse/CALCITE-6330
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: mengdou
>Assignee: mengdou
>Priority: Minor
>
> In this time, when we dump the plan of a RelNode Tree, there is no average
> row size exported in the output string, even if
> SqlExplainLevel.ALL_ATTRIBUTES is specified.
> Because the implementation in explain_() in class RelWriterImpl doesn't
> include the metric average_row_size:
>
> {code:java}
> switch (detailLevel) {
> case ALL_ATTRIBUTES:
> s.append(": rowcount = ")
> .append(mq.getRowCount(rel))
> .append(", cumulative cost = ")
> .append(mq.getCumulativeCost(rel));
> }
> switch (detailLevel) {
> case NON_COST_ATTRIBUTES:
> case ALL_ATTRIBUTES:
> if (!withIdPrefix) {
> // If we didn't print the rel id at the start of the line, print
> // it at the end.
> s.append(", id = ").append(rel.getId());
> }
> break;
> } {code}
>
> So I'd like to add this metric by calling md.getAverageRowSize(rel)
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
