rivernate opened a new issue #8439: URL: https://github.com/apache/pulsar/issues/8439
**Is your enhancement request related to a problem? Please describe.** It doesn't appear that we currently include the `Content-MD5` when we upload the ledgers using JCloud. While we haven't seen any corruption from offloading so far, it would be nice to send the `Content-MD5` so that services that support validation could check the integrity of the upload **Describe the solution you'd like** Looking at the code here: https://github.com/apache/pulsar/blob/ec2de258ce39b38311e92bab6882c198070d8975/tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/BlobStoreManagedLedgerOffloader.java#L176 It should be straight forward to add the `Content-MD5` to the ContentMetaData. I have checked that JCould supports this here: https://github.com/apache/jclouds/blob/94f09325ba1442ed64fc0a0eba7fe8b336f621ce/core/src/main/java/org/jclouds/io/MutableContentMetadata.java#L35 And AWS supports the validation of the individual parts of the Multipart upload, see: https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html The biggest downside I can see is that we would need to buffer the InputStream in the broker to calculate the MD5Hash ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
