still need some help on reviewing the PR that reverted/removed the JSON spec for content file and file scan task.
https://github.com/apache/iceberg/pull/9771/files On Wed, Feb 21, 2024 at 4:01 PM Jack Ye <yezhao...@gmail.com> wrote: > I see. I was asking for the devlist discussion history, because this is > related to our proposal discussion. I think we should establish some rules > like "no change should be added to any spec without devlist discussions", > and then we can use this rule to justify the removal of this spec change > that was not discussed on devlist. > > -Jack > > On Wed, Feb 21, 2024 at 3:47 PM Steven Wu <stevenz...@gmail.com> wrote: > >> here is the PR for spec update: >> https://github.com/apache/iceberg/pull/9771 >> >> > Was there any prior discussions on devlist for adding it to the spec? >> >> Jack, there is no separate discussion on adding it to the spec. It was a >> mistake on my part. it was added in the PR from 8 months ago as linked. >> >> [2] https://github.com/apache/iceberg/pull/6934/files >> >> On Wed, Feb 21, 2024 at 2:21 PM Ryan Blue <b...@tabular.io> wrote: >> >>> I think I would probably remove it from the spec with a note and a >>> pointer to the class that implements it. >>> >>> Right now we don't have anyone that I'm aware of relying on this >>> serialization format across engines so it isn't a format-level contract. >>> Though we should note that Flink relies on the format to evolve in >>> compatible ways across versions. I think that means that we don't make any >>> guarantees about how it evolves and it can be safely removed since it is >>> not a contract that we are committed to maintaining. >>> >>> Ryan >>> >>> On Wed, Feb 21, 2024 at 1:30 PM Jack Ye <yezhao...@gmail.com> wrote: >>> >>>> Was there any prior discussions on devlist for adding it to the spec? >>>> Could you help link those conversations? >>>> >>>> Thanks, >>>> Jack Ye >>>> >>>> On Wed, Feb 21, 2024 at 1:05 PM Steven Wu <stevenz...@gmail.com> wrote: >>>> >>>>> >>>>> In the recent PR review [1], Ryan and emkornfield has raised a >>>>> question why file scan task JSON serialization was added to the table spec >>>>> [2]. We seems to have a consensus that it *shouldn't* have been added >>>>> to the table spec. >>>>> >>>>> Now the question is what's the process of removing an invalid section >>>>> from table spec? Can we simply remove it? >>>>> >>>>> Thanks, >>>>> Steven >>>>> >>>>> [1] https://github.com/apache/iceberg/pull/9728#discussion_r1492846630 >>>>> [2] https://github.com/apache/iceberg/pull/6934/files >>>>> >>>> >>> >>> -- >>> Ryan Blue >>> Tabular >>> >>
