aminghadersohi commented on code in PR #44712:
URL: https://github.com/apache/superset/pull/44712#discussion_r4113366839
##########
superset/db_engine_specs/kylin.py:
##########
@@ -46,18 +46,52 @@ class KylinEngineSpec(BaseEngineSpec): # pylint:
disable=abstract-method
"default_port": 7070,
}
+ # Kylin validates every query with Calcite; queries no cube can answer are
+ # pushed down as SQL text to Spark SQL. ``FLOOR(<ts> TO <unit>)`` cannot be
+ # parsed by Spark, so the grains only use CAST, TIMESTAMPADD and field
+ # functions, which both paths accept. Weeks start on Sunday (DAYOFWEEK is 1
+ # for Sunday on both paths).
_time_grain_expressions = {
None: "{col}",
- TimeGrain.SECOND: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO SECOND) AS
TIMESTAMP)", # noqa: E501
- TimeGrain.MINUTE: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO MINUTE) AS
TIMESTAMP)", # noqa: E501
- TimeGrain.HOUR: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO HOUR) AS
TIMESTAMP)",
- TimeGrain.DAY: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO DAY) AS DATE)",
- TimeGrain.WEEK: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO WEEK) AS
DATE)",
- TimeGrain.MONTH: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO MONTH) AS
DATE)",
- TimeGrain.QUARTER: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO QUARTER) AS
DATE)",
- TimeGrain.YEAR: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO YEAR) AS
DATE)",
+ TimeGrain.SECOND: (
+ "TIMESTAMPADD(SECOND, HOUR({col}) * 3600 + MINUTE({col}) * 60"
+ " + SECOND({col}), CAST(CAST({col} AS DATE) AS TIMESTAMP))"
+ ),
+ TimeGrain.MINUTE: (
+ "TIMESTAMPADD(MINUTE, HOUR({col}) * 60 + MINUTE({col}),"
+ " CAST(CAST({col} AS DATE) AS TIMESTAMP))"
+ ),
+ TimeGrain.HOUR: (
+ "TIMESTAMPADD(HOUR, HOUR({col}), CAST(CAST({col} AS DATE) AS
TIMESTAMP))"
+ ),
+ TimeGrain.DAY: "CAST({col} AS DATE)",
+ TimeGrain.WEEK: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFWEEK({col}), CAST({col} AS DATE))"
+ ),
+ TimeGrain.MONTH: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFMONTH({col}), CAST({col} AS DATE))"
+ ),
+ TimeGrain.QUARTER: (
+ "TIMESTAMPADD(MONTH, 3 * QUARTER({col}) - 3,"
+ " TIMESTAMPADD(DAY, 1 - DAYOFYEAR({col}), CAST({col} AS DATE)))"
+ ),
+ TimeGrain.YEAR: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFYEAR({col}), CAST({col} AS DATE))"
Review Comment:
No change. In Calcite, `TIMESTAMPADD` with a day-or-coarser unit returns the
type of its datetime operand (`SqlTimestampAddFunction.deduceType` only
promotes to TIMESTAMP for HOUR and finer), so WEEK/MONTH/QUARTER/YEAR on
`CAST(... AS DATE)` already come back as DATE, same as DAY. Those grains were
also checked live on Kylin 4.0.0 on both the cube and pushdown paths.
##########
superset/db_engine_specs/kylin.py:
##########
@@ -46,18 +46,52 @@ class KylinEngineSpec(BaseEngineSpec): # pylint:
disable=abstract-method
"default_port": 7070,
}
+ # Kylin validates every query with Calcite; queries no cube can answer are
+ # pushed down as SQL text to Spark SQL. ``FLOOR(<ts> TO <unit>)`` cannot be
+ # parsed by Spark, so the grains only use CAST, TIMESTAMPADD and field
+ # functions, which both paths accept. Weeks start on Sunday (DAYOFWEEK is 1
+ # for Sunday on both paths).
_time_grain_expressions = {
None: "{col}",
- TimeGrain.SECOND: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO SECOND) AS
TIMESTAMP)", # noqa: E501
- TimeGrain.MINUTE: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO MINUTE) AS
TIMESTAMP)", # noqa: E501
- TimeGrain.HOUR: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO HOUR) AS
TIMESTAMP)",
- TimeGrain.DAY: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO DAY) AS DATE)",
- TimeGrain.WEEK: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO WEEK) AS
DATE)",
- TimeGrain.MONTH: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO MONTH) AS
DATE)",
- TimeGrain.QUARTER: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO QUARTER) AS
DATE)",
- TimeGrain.YEAR: "CAST(FLOOR(CAST({col} AS TIMESTAMP) TO YEAR) AS
DATE)",
+ TimeGrain.SECOND: (
+ "TIMESTAMPADD(SECOND, HOUR({col}) * 3600 + MINUTE({col}) * 60"
+ " + SECOND({col}), CAST(CAST({col} AS DATE) AS TIMESTAMP))"
+ ),
+ TimeGrain.MINUTE: (
+ "TIMESTAMPADD(MINUTE, HOUR({col}) * 60 + MINUTE({col}),"
+ " CAST(CAST({col} AS DATE) AS TIMESTAMP))"
+ ),
+ TimeGrain.HOUR: (
+ "TIMESTAMPADD(HOUR, HOUR({col}), CAST(CAST({col} AS DATE) AS
TIMESTAMP))"
+ ),
+ TimeGrain.DAY: "CAST({col} AS DATE)",
+ TimeGrain.WEEK: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFWEEK({col}), CAST({col} AS DATE))"
+ ),
+ TimeGrain.MONTH: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFMONTH({col}), CAST({col} AS DATE))"
+ ),
+ TimeGrain.QUARTER: (
+ "TIMESTAMPADD(MONTH, 3 * QUARTER({col}) - 3,"
+ " TIMESTAMPADD(DAY, 1 - DAYOFYEAR({col}), CAST({col} AS DATE)))"
+ ),
+ TimeGrain.YEAR: (
+ "TIMESTAMPADD(DAY, 1 - DAYOFYEAR({col}), CAST({col} AS DATE))"
+ ),
Review Comment:
No change. A bare integer year column was never a valid input here: the
previous grains did `CAST({col} AS TIMESTAMP)`, which Calcite rejects for an
INTEGER operand, so this is not a regression. These grains target
DATE/TIMESTAMP columns (the only kind Kylin exposes as temporal).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
