LuciferYang opened a new pull request, #8922: URL: https://github.com/apache/hadoop/pull/8922
### Description of PR JIRA: https://issues.apache.org/jira/browse/HADOOP-20059 `AbstractFileSystem#create` rejects a block size that is not a multiple of the checksum size by checking `blockSize % bytesPerChecksum`. `bytesPerChecksum` is a local that stays at -1 unless the caller passes `CreateOpts.bytesPerChecksum`, and `x % -1` is always 0, so the check never fires in the usual case. On HDFS, `FileContext#create` with such a block size reaches the NameNode, which creates the file (or deletes the old one with `OVERWRITE`), and only then does `DFSOutputStream` throw. This patch checks against `checksumOpt.getBytesPerChecksum()`, the value the stream is created with. On file systems that ignore the block size, `FileContext#create` with a block size that is not a multiple of the checksum size (512 by default) now fails, as it did before HADOOP-8240. `DistributedFileSystem#create` has the same create-then-check order inside `DFSClient`, which needs a separate HDFS change. ### How was this patch tested? Added `FileContextCreateMkdirBaseTest#testCreateBlockSizeNotMultipleOfChecksumSize`: creating with a block size of 1000 throws `HadoopIllegalArgumentException` and leaves no file behind. Without the fix the create succeeds on the local and viewfs subclasses. The case is in the shared base class, so it also runs in the S3A, OSS and ADL FileContext tests, which I did not run against a store. ### For code changes: - [x] Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: Have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? ### AI Tooling - [x] The PR includes the phrase "Contains content generated by Claude Opus". - [x] My use of AI contributions follows the ASF legal policy https://www.apache.org/legal/generative-tooling.html Contains content generated by Claude Opus. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
