shuturmurgh commented on issue #19715: URL: https://github.com/apache/pinot/issues/19715#issuecomment-6095693530
> Changing an existing column’s data type is backward incompatible. If users need to make that change, they need to refresh all segments in the table so the stored types consistently match the new schema. The challenge here is that table and segment schema have always diverged. Our offline ingestion pipelines used to allow near-compatible column data types e.g., int (segment schema) vs long (table schema). Pinot does not fail queries in these scenarios if all segments have the same column data type. But it returns failures due to merge exceptions if same column has different data types in two segments. This'd cause query failures in the middle of table refresh. ~100s of our offline ingestion pipelines face this problem our intention is to unblock segment refresh for them before they're updated to be schema-compatible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
