It sounds like a feature that we could add. I think there are two concerns. First, we don't want to process the files for multiple specs at the same time because we probably want a consistent schema. Second, there's probably some confusion over how to select a spec since we don't like users to need to work with IDs in the format directly.
On Wed, Dec 6, 2023 at 10:28 PM Pucheng Yang <py...@pinterest.com.invalid> wrote: > Based on what I understand, it seems there is no particular reason, and > seems like a feature to be added on. > > On Wed, Dec 6, 2023 at 8:06 PM Pucheng Yang <py...@pinterest.com> wrote: > >> Hi community, >> >> May I know why manifest rewrite will only touch files that have the >> latest spec id? What will be the suggestion if we want to rewrite manifest >> files that belong to non current spec id? >> >> Manifest selection logic: >> https://github.com/apache/iceberg/blob/6a9d3c77977baff4295ee2dde0150d73c8c46af1/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java#L295 >> >> Best, >> Pucheng >> > -- Ryan Blue Tabular
