adriangb commented on issue #12010: URL: https://github.com/apache/datafusion/issues/12010#issuecomment-2291247886
To give a concrete examples, we might have a schema evolution that looks like: ``` day=2020-01-01/1.parquet - (name Utf8, department Utf8) -- department column made a partition day=2020-01-02/department=abc/1.parquet - (name Utf8,) -- added salary column day=2020-01-03/department=abc/1.parquet - (name Utf8, salary Float32) -- department moved back to a regular column day=2020-01-04/1.parquet - (name Utf8, salary Float32, department Utf8) -- name made LargeUtf8 (or Utf8View, etc.) day=2020-01-05/1.parquet - (name LargeUtf8, salary Float32, department Utf8) ``` Where our "final"/"current" table schema is `(day Date32, name LargeUtf8, salary Float32, department Utf8)`. (fwiw in reality some of these columns, e.g. `department`, would be dict-encoded) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
