wangyong9999 opened a new pull request, #10315: URL: https://github.com/apache/paimon/pull/10315
### Purpose Close #10314. A `FileIndexWriter` receives every row of a data file in file order, but `FileIndexer#createWriter()` gives it no information about which data file it indexes. Plugins whose output references the data file, such as an external `key -> (data file, row position)` mapping, have to rely on fragile workarounds. This change passes a `FileIndexWriterContext` with the data file path and schema id to file index writers: - Add `FileIndexWriterContext` and a default method `FileIndexer#createWriter(FileIndexWriterContext)` that falls back to `createWriter()`, so built-in indexes and existing plugins are unaffected. - `DataFileIndexWriter` takes the context and passes it to top-level and nested map index writers. - Every call site provides the context of the data file being indexed: `RowDataFileWriter`, `KeyValueDataFileWriter`, `KeyValueClusteringFileWriter`, and `FileIndexProcessor`, which passes the existing file and its original schema id when rewriting indexes. No storage format change. ### Tests - `DataFileIndexWriterTest`: an append table rolled into several files across a schema change, and a primary-key table; every data file has exactly one index writer, which receives the file path, the file's schema id and the indexed values in file order. - `FileIndexProcessorTest`: rewriting the index of an existing file passes that file and its original schema id. - `mvn -pl paimon-core -Pfast-build -DwildcardSuites=none -Dtest='*FileIndex*,*FileWriter*Test,AppendOnlyWriterTest,KeyValueFileReadWriteTest,AppendOnlySimpleTableTest' test`: 300 tests passed. The three new tests fail without this change. - Java Spotless, Checkstyle and `git diff --check` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
