sundapeng opened a new pull request, #9161: URL: https://github.com/apache/paimon/pull/9161
## Summary This PR turns the existing `FileIOBehaviorTestBase` into a provider-neutral FileIO contract and documents the guarantees that Paimon callers may rely on. The contract is shared by LocalFileIO, Hadoop local, HDFS, and S3/MinIO without requiring POSIX directory markers or provider-specific metadata probes. ## Changes - Expand the shared contract to 47 tests covering input/output streams, status and listing, delete and mkdir, non-conflicting rename, copy helpers, and two-phase publication and cleanup. - Define the same guarantees in the public FileIO, FileStatus, SeekableInputStream, and TwoPhaseOutputStream documentation. - Keep provider-dependent cells explicit and unspecified, including missing/file listing inputs, the missing-delete return value, file/directory path collisions, rename conflicts and atomicity, invalid seek offsets, two-phase existing-target behavior, directory-marker layout, and request counts. - Align LocalFileIO with the contract by returning immutable status snapshots and making `copyFile(..., false)` reject an existing destination without changing its content. - Make renaming-based two-phase discard and cleanup remove only the current writer's staged file. - Construct the S3 multipart helper through S3A so Hadoop installs its required upload callbacks. No new runtime capability or provider profile is introduced. Request-count assertions remain provider-specific work; this contract only defines observable preconditions and postconditions. ## Testing - [x] `mvn -pl paimon-common -am -Pfast-build -DwildcardSuites=none -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -Dtest='LocalFileIOBehaviorTest,HadoopLocalFileIOBehaviorTest,RenamingTwoPhaseOutputStreamTest,LocalFileIOTest' test` (106 tests) - [x] `mvn -pl paimon-common -am -Pfast-build -DwildcardSuites=none -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -Dtest='HdfsBehaviorTest' test` (51 tests) - [x] `DOCKER_HOST=unix:///run/podman/podman.sock TESTCONTAINERS_RYUK_DISABLED=true mvn -pl paimon-filesystems/paimon-s3-impl -am -Pfast-build -DwildcardSuites=none -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -Dtest='S3FileIOTest,S3MultiPartUploadTest' test` (53 tests) - [x] `mvn -pl paimon-common,paimon-filesystems/paimon-s3-impl -am -Pfast-build -DskipTests compile` - [x] `mvn -pl paimon-common,paimon-filesystems/paimon-s3-impl -am -Pfast-build -DskipTests spotless:check` The new tests were written before the implementation fixes. On the previous implementation they reproduced the LocalFileIO copy-conflict bug, mutable FileStatus observations, cross-target two-phase discard, and the S3 multipart null-callback failure; all pass after the fixes above. ## Compatibility The FileIO method signatures and serialization shape are unchanged. The PR narrows previously ambiguous documentation to the portable behavior already required by Paimon and deliberately leaves storage-specific edge cases outside the common contract. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
