[
https://issues.apache.org/jira/browse/HDDS-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chung En Lee reassigned HDDS-12542:
-----------------------------------
Assignee: Chung En Lee
> Support S3 signed multi chunks payload verification
> ---------------------------------------------------
>
> Key: HDDS-12542
> URL: https://issues.apache.org/jira/browse/HDDS-12542
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Chung En Lee
> Priority: Major
> Attachments: screenshot-1.png
>
>
> Currently we use SignedChunkInputStream for parsing chunk payload with chunk
> signatures. However, no chunk signatures verification is done.
> We can support this feature. We might need to dynamically change the
> WriteChunk size based on the S3 chunk size specified (i.e. 1 S3 chunk = 1
> WriteChunk) to fit more to S3 SDK behavior and prevent multiple round-trips
> per S3 chunk payload (Edit: might not be good since a single Ozone chunk is
> significantly larger than S3 chunk). Additionally, we also need to handle the
> trailer (if any).
> I expect that the solution is not straightforward since there are differences
> in architecture between Ozone and AWS S3, so some design document might be
> needed. Few things to note:
> * We might need to keep track of the previous chunk signature since the
> subsequent chunk signature is derived from the previous chunk signature
> * Support trailer SignedChunksInputStream: Perhaps adding a simple boolean
> flag and check for the trailer afterwards
> * Checksum verification location: S3G or DN?
> Resources
> [https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html]
> [https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming-trailers.html]
> !screenshot-1.png|width=690,height=350!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
