aminghadersohi opened a new pull request, #44712:
URL: https://github.com/apache/superset/pull/44712
### SUMMARY
Every time-grain chart on Apache Kylin failed.
Kylin validates each query with Calcite. A query that a cube can answer runs
there; any other query is pushed down as SQL text to Spark SQL. The current
grains break both paths:
1. **Spark pushdown.** `FLOOR(<ts> TO <unit>)` is not Spark SQL, so every
time-grain query on a table without a matching cube fails with `no viable
alternative at input 'CAST(FLOOR(...) TO'`. On the Calcite path, `FLOOR(... TO
WEEK)` also floors to a Thursday-based week, and `FLOOR(... TO QUARTER)`
returned its input unchanged on Kylin 4.0.0.
2. **Alias shadowing.** Kylin's Calcite resolves identifiers in `GROUP
BY`/`ORDER BY` against SELECT aliases first. Superset labels a time grain with
the source column's name (`<grain of "TS"> AS "TS" ... GROUP BY <grain of
"TS">`), so the `"TS"` inside `GROUP BY` resolves to the alias, and Calcite
rejects the query with `Expression 'TS' is not being grouped`. This happens
even with a valid grain expression.
Changes:
- The grains use only `CAST`, `TIMESTAMPADD` and field functions (`HOUR`,
`DAYOFWEEK`, `DAYOFYEAR`, `QUARTER`, …), which both Calcite and Kylin's Spark
pushdown accept. Result types are the same as before: `TIMESTAMP` for
second/minute/hour, and `DATE` for day and coarser grains. Weeks start on
Sunday.
- `_mutate_label` adds a `__` suffix to each label, so no label shadows its
source column. Superset renames the result columns back to the expected labels
(`labels_expected`), as it does for other engines that mutate labels.
### TESTING INSTRUCTIONS
Unit tests: `pytest tests/unit_tests/db_engine_specs/test_kylin.py`. 9 of
the new cases fail on master and all 12 pass here.
Live, on Apache Kylin 4.0.0 (the `apachekylin/apache-kylin-standalone:4.0.0`
image, Spark 2.4.7), with a Hive table `(id INT, amount DECIMAL(12,2), ts
TIMESTAMP, ...)` loaded into a project:
- SECOND, MINUTE, HOUR, DAY, WEEK, MONTH, QUARTER and YEAR each return
correctly bucketed rows on the pushdown path (`pushDown=true`), and again after
a cube was built on the table (`pushDown=false`);
- the old `FLOOR(... TO DAY)` fails on both, and `CAST("TS" AS DATE) AS "TS"
... GROUP BY CAST("TS" AS DATE)` still fails with "not being grouped".
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
