shuturmurgh opened a new issue, #19715:
URL: https://github.com/apache/pinot/issues/19715
`RefreshSegmentTask` regenerates segments using the current table schema.
During this, stored values would be converted to the schema data type. This
creates a problem for our older offline tables where segment metadata and the
current table schema do not match e.g.,:
1. Segment stores STRING, schema declares BOOLEAN
2. Segment stores INT, schema declares LONG
Ongoing table segment refresh fails queries with merge exceptions
Suggestions
1. Add an opt-in mode to RefreshSegmentTask
Example configuration:
```
{
"taskTypeConfigsMap": {
"RefreshSegmentTask": {
"indexOnly": "true"
}
}
}
```
2. Introduce a separate task similar to
https://docs.startree.ai/corecapabilities/manage-data/alter-table-task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
