Chung-En Lee created HDDS-15652:
-----------------------------------
Summary: Support S3 signed multi chunks payload verification
Key: HDDS-15652
URL: https://issues.apache.org/jira/browse/HDDS-15652
Project: Apache Ozone
Issue Type: Sub-task
Reporter: Ivan Andika
Assignee: Chung-En Lee
Currently we use SignedChunkInputStream for parsing chunk payload with chunk
signatures. However, no chunk signatures verification is done.
We can support this feature. We might need to dynamically change the WriteChunk
size based on the S3 chunk size specified (i.e. 1 S3 chunk = 1 WriteChunk) to
fit more to S3 SDK behavior and prevent multiple round-trips per S3 chunk
payload (Edit: might not be good since a single Ozone chunk is significantly
larger than S3 chunk). Additionally, we also need to handle the trailer (if
any).
I expect that the solution is not straightforward since there are differences
in architecture between Ozone and AWS S3, so some design document might be
needed. Few things to note:
* We might need to keep track of the previous chunk signature since the
subsequent chunk signature is derived from the previous chunk signature
* Support trailer SignedChunksInputStream: Perhaps adding a simple boolean
flag and check for the trailer afterwards
* Checksum verification location: S3G or DN?
Resources
[https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html]
[https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming-trailers.html]
!screenshot-1.png|width=690,height=350!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
