jojochuang commented on PR #12045:
URL: https://github.com/apache/ozone/pull/12045#issuecomment-6866268759

   ## MD5MD5CRC: stripe-level cap vs composite byte-level cap
   
   The fix correctly bounds both paths when a replica lists **whole extra 
stripes** beyond OM `blockLength` (the scenario described in HDDS-16974). For 
default **COMPOSITE_CRC**, `blockSize > 0` on the inner loops matches the 
committed length.
   
   **Medium concern — `computeMd5Crc` only stops at stripe granularity**
   
   In `computeMd5Crc`, `remaining` is decremented by `chunkSize * numData` per 
`ChunkInfo`, and the full data portion of `stripeChecksum` (parity stripped) is 
fed to the MD5 digester whenever `remaining > 0` at the start of the stripe — 
there is no byte-level cap on how much of that stripe blob is hashed.
   
   ```java
   if (remaining <= 0) {
     break;
   }
   // ...
   digester.update(byteWrap);
   remaining -= chunkSize * numData;
   ```
   
   If the **last committed** block ends mid-stripe (`0 < remaining < chunkSize 
* numData`) but the replica still reports a **full** stripe checksum blob (same 
class of “replica has more than OM committed” as the extra-stripe case, but 
within the last stripe), composite mode can stop correctly via 
`Math.min(blockSize, chunkSize)` / `remainingChunkSize`, while MD5 can 
**over-include** CRC bytes for that stripe.
   
   That asymmetry only affects **MD5MD5CRC** users (not the default combine 
mode), but it is worth either:
   
   1. Aligning MD5 with composite by only updating the digest for the portion 
of the stripe that fits in `remaining` (bytes of data checksum corresponding to 
remaining logical bytes), or  
   2. Documenting that MD5MD5CRC is stripe-aligned only, plus a unit test with 
`blockLength = stripeLen + partial` and an oversized/trailing stripe in the 
chunk list.
   
   `testStripeBeyondBlockLengthIsIgnored` uses stripe-aligned `blockLength = 2 
* stripeLen`; a partial-last-stripe case would lock in behavior for MD5 vs 
COMPOSITE_CRC.
   
   Not blocking for the reported hang/wrong-checksum-on-extra-stripe bug; 
suggesting as follow-up or small extension in this PR if MD5MD5CRC for EC 
remains supported.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to