Akshat-Jain commented on code in PR #16804:
URL: https://github.com/apache/druid/pull/16804#discussion_r1696474274
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/scan/ScanQueryKit.java:
##########
@@ -178,6 +183,11 @@ public QueryDefinition makeQueryDefinition(
}
}
+ // If window has an empty over, we want a single worker to process entire
data for window function evaluation.
+ if (windowHasEmptyOver) {
+ scanShuffleSpec = MixShuffleSpec.instance();
Review Comment:
As mentioned on the [other
comment](https://github.com/apache/druid/pull/16804#discussion_r1696410451),
MSQ ingestion with window functions is broken right now. Adding more details
here though.
I tried this query:
```sql
REPLACE INTO "test_1" OVERWRITE ALL
WITH "ext" AS (
select __time, countryName, cityName, channel
from "drill_wikipedia"
where countryName in ('Guatemala')
group by __time, countryName, cityName, channel
order by countryName, cityName, channel
)
SELECT
"__time",
"countryName",
"channel",
row_number() over (partition by cityName order by countryName, cityName,
channel) as c1,
count(channel) over (partition by cityName order by countryName, cityName,
channel) as c2
FROM "ext"
PARTITIONED BY DAY
```
with the following query context:
```json
{
"maxNumTasks": 2,
"enableWindowing": true,
"rowsPerSegment": 1
}
```
It fails because of NPE, but it doesn't even go to the ScanQueryKit. The
following stages are created for the above query:
<img width="1227" alt="image"
src="https://github.com/user-attachments/assets/dfccd261-9e49-4706-87ba-076c943a7ef6";>
I also tried changing `maxNumTasks`, but it runs into the same issue.
So, I feel that my changes in ScanQueryKit should be unrelated? This NPE is
certainly something we need to fix, but it's also not because of this PR's
changes.
Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
