Hi Team, Any further comments or suggestions on the possible approaches discussed above.
On Tue, Mar 19, 2024, 09:55 Arpit Goyal <goyal.arpit...@gmail.com> wrote: > @Luke Chen @Kamal Chandraprakash <kamal.chandraprak...@gmail.com> @Greg > Harris Any suggestion on the above two possible approaches. > On Sun, Mar 17, 2024, 13:36 Arpit Goyal <goyal.arpit...@gmail.com> wrote: > >> >>>> In summary , There are two possible solution to handle the above >> scenario when producer snapshot file found to be null >> >> 1. *Generate empty producer snapshot file at run time when copying >> LogSegment* >> >> >> - This will not require any backward compatibility dependencies with >> the plugin. >> - It preserves the contract i.e producerSnapshot files should be >> mandatory. >> - We could have a metric which helps us to assess how many times >> empty snapshot files have been created. >> >> 2. * Make producerSnapshot files optional * >> >> - This would break the contract with the plugin and would require >> defining a set of approaches to handle it which is mentioned earlier in >> the >> thread. >> - If we make producer Snapshot optional , We would not be handling >> the error which @LukeChen mentioned when producerSnapshot >> accidentally deleted a given segment. But this holds true for >> TransactionalIndex. >> - The other question is do we really need to make the field optional. >> The only case where this problem can occur is only when the topic migrated >> from < 2.8 version. >> >>
