SperedArGoadeg opened a new issue, #19003:
URL: https://github.com/apache/druid/issues/19003
### Affected Version
35.0.1
### Description
When running an MSQ ingestion with the `waitUntilSegmentsLoad` option
enabled in the query context, the task may return a `SUCCESS` status even
though the generated segments are not yet loaded.
This behavior is **intermittent** and does not occur systematically for
every ingestion.
When the issue occurs, any query against the target datasource fails with
the following error:
```
{
"error": "druidException",
"errorCode": "invalidInput",
"persona": "USER",
"category": "INVALID_INPUT",
"errorMessage": "Object 'my-datasource' not found (line [1], column
[753])",
"context": {
"sourceType": "sql",
"line": "1",
"column": "753",
"endLine": "1",
"endColumn": "790"
}
}
```
Example query (columns removed for confidentiality):
```
SELECT __time,other_column FROM "my-datasource" LIMIT 1
```
Anonymized SQL query:
```
INSERT INTO "my-datasource"
WITH ext AS (
SELECT *
FROM TABLE(
EXTERN(
'{"type":"hdfs","paths":"hdfs://hdfs-namenodes:8020/data.parquet"}',
'{"type":"parquet"}'
)
)
EXTEND ( start_date BIGINT, other_column VARCHAR )
)
SELECT MILLIS_TO_TIMESTAMP(start_date) AS __time, other_column
FROM ext
PARTITIONED BY ALL
```
Query context (from native query):
```
A similar issue can also occur where the task returns a `SUCCESS` status
while not all columns of the datasource are available yet. In this case, the
datasource exists and is queryable, but some columns are temporarily missing.
More informations:
- Cluster setup:
- 2 brokers
- 1 coordinator
- 2 historicals
- Deployment:
- Middle manager less mode
- Kubernetes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
