Hello Matthias,
Thanks for your response. As described in FLIP-347, I have thoroughly reviewed all instances where the IOReadableWritable serialization of the Path class is used, and I have identified three classes that still utilize it for de/serializing the Path. 1. FileSourceSplitSerializer: This class de/serializes the Path during the process of de/serializing the FileSourceSplit. 2. Two tests: TestManagedSinkCommittableSerializer, TestManagedFileSourceSplitSerializer. For 1, it can be modified to not serialize the Path by IOReadableWritable. For 2, these two tests will be deprecated before Flink 2.0. BTW, in my POC of this FLIP, I removed the IOReadableWritable usage in 1/2 and it has passed the CI on Azure. I seems that the IOReadableWritable of Path is not used by DataSet API or other components. I have documented these findings in FLINK-5336 [1]. Best regards, Wencong Liu [1] https://issues.apache.org/jira/browse/FLINK-5336 At 2023-07-21 16:22:34, "Matthias Pohl" <matthias.p...@aiven.io.INVALID> wrote: >There's a kind-of-related issue FLINK-4758 [1] that proposes removing the >IOReadableWritable interface from more classes. It was briefly mentioned in >the must-have work items discussion [2]. > >I'm not too sure about the usage of IOReadableWritable: ...whether it would >go away with the removal of the DataSet API in general (the Jira issue has >DataSet as a component), anyway. > >Otherwise, might it make sense to extend the scope of this FLIP? > >[1] https://issues.apache.org/jira/browse/FLINK-4758 >[2] https://lists.apache.org/thread/gf0h4gh3xfsj78cpdsxsnj70nhzcmv9r > >On Fri, Jul 21, 2023 at 6:04 AM Xintong Song <tonysong...@gmail.com> wrote: > >> +1 >> >> Best, >> >> Xintong >> >> >> >> On Fri, Jul 21, 2023 at 10:54 AM Wencong Liu <liuwencle...@163.com> wrote: >> >> > Hi devs, >> > >> > I would like to start a discussion on FLIP-347: Remove IOReadableWritable >> > serialization in Path [1]. >> > >> > >> > The Path class is currently mutable to support IOReadableWritable >> > serialization. However, many parts >> > of the code assume that the Path is immutable. By making the Path class >> > immutable, we can ensure >> > that paths are stored correctly without the possibility of mutation and >> > eliminate the occurrence of subtle errors. >> > As such I propose to modify the Path class to no longer implement the >> > IOReadableWritable interface. >> > Looking forward to your feedback. >> > [1] >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-347%3A+Remove+IOReadableWritable+serialization+in+Path >> > Best regards, >> > >> > >> > Wencong Liu >>
