bmuraykhi opened a new issue, #45738:
URL: https://github.com/apache/superset/issues/45738
### Bug description
On ClickHouse, a chart with a **series limit** (top-N series) returns wrong
numbers when one of its series dimensions is a calculated column or a SQL
expression whose label is a dataset column name or appears in its own
expression. Every value comes out multiplied by the series limit, and the limit
stops filtering any series. No error is raised, so the wrong numbers look
plausible.
This started with #44715, *fix(clickhouse): rename SELECT aliases that
shadow a source column*, and is present in `7.0.0rc2` and on `master`.
**Cause.** `rename_shadowing_aliases` renames the outer alias `<label>` to
`<label>__`. The series-limit subquery already uses `<label>__` for the same
dimension, and the outer query joins on that unqualified name:
```sql
SELECT toStartOfDay(toDateTime(`ts`)) AS `ts__`, (upper(label)) AS `app__`,
sum(amount) AS `total`
FROM `default`.`t`
INNER JOIN (
SELECT (upper(label)) AS `app__`, sum(amount) AS `mme_inner__`
FROM `default`.`t` WHERE ... GROUP BY `app__` ORDER BY sum(amount) DESC
LIMIT 3
) AS `series_limit` ON (upper(label)) = `app__`
...
```
ClickHouse resolves an identifier to a SELECT alias of the current query
before anything else. So `app__` in the `ON` clause is the outer alias, and the
condition becomes `upper(label) = upper(label)`, which is always true. Each row
joins every one of the N subquery rows.
**How to reproduce**, in plain ClickHouse 25.3:
```sql
CREATE TABLE t (ts DateTime, label String, amount Float64) ENGINE =
MergeTree ORDER BY ts;
INSERT INTO t SELECT toDateTime('2024-01-01 00:00:00') + number * 60,
concat('l', toString(number % 10)), (number % 10) + 1 FROM numbers(1000);
SELECT count() AS series, sum(total) AS total FROM (
SELECT upper(label) AS app__, sum(amount) AS total FROM t
INNER JOIN (SELECT upper(label) AS app__, sum(amount) AS mme_inner__ FROM
t GROUP BY app__ ORDER BY mme_inner__ DESC LIMIT 3) AS series_limit
ON upper(label) = app__
GROUP BY upper(label)
);
-- returns 10 series, 16500 (all series, each counted 3 times)
-- expected 3 series, 2700
```
In Superset:
1. Create a ClickHouse dataset on `t` with a calculated column `app` =
`upper(label)`.
2. Build a line or bar chart: x-axis `ts`, dimension `app`, metric
`SUM(amount)`, series limit 3.
3. The chart shows all 10 series, each with 3× its real value.
On 6.1.0 the same chart shows the top 3 series with correct values.
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.11
### Node version
Not applicable
### Browser
Not applicable
### Additional context
- The problem is in the generated SQL, not the frontend. It affects every
engine that sets `select_alias_shadows_source_column = True`, which is
ClickHouse (`clickhouse` and `clickhousedb`).
- Reproduced on `7.0.0rc2` (`e4aee71`) and `master` (`9c31d12`).
- No stack trace; the query succeeds on ClickHouse 25.x and later.
Behaviour of the generated SQL by ClickHouse version (same table and query
as above):
| ClickHouse | `<label>__` collision (current) | unique alias (fixed) |
|---|---|---|
| 23.8, 24.3, 24.8 | fails: `Code: 403 ... Cannot get JOIN keys from JOIN ON
section` | 3 series, 2700 |
| 25.3, 25.8, 26.3, 26.10 | 10 series, 16500 (silently wrong) | 3 series,
2700 |
Older versions reject `upper(label) = upper(label)` as a join without keys;
from 25.x it is accepted and every row joins every subquery row.
`prefer_column_name_to_alias = 1` avoids it, but that flips the resolution
order #44715 relies on.
### Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to
my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug
report.
- [x] I have checked Superset's logs for errors and if I found a relevant
Python stacktrace, I included it here as text in the "additional context"
section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
