arunsarin85 opened a new pull request, #11337:
URL: https://github.com/apache/ozone/pull/11337
## What changes were proposed in this pull request?
This PR reduces runtime of the integration test TestStreamBlockInputStream
by cutting redundant mini-cluster I/O and aligning cluster setup with other
stream-read tests, without changing what the test validates (multi-block stream
read, pre-read on/off, seek, positioned read, checksum-off, empty block).
Please describe your PR in detail:
- Cluster: RF=1 and one datanode (same pattern as TestChunkInputStream /
TestStreamRead); all writes use getRepConfig() so replication matches the
cluster.
- testReadKey: Fewer but representative read buffer sizes (divisors
{1,2,5,10} and fixed {4KB, 256KB, 16MB} instead of 10 + 7 full-key reads); one
random-length key instead of two.
- testReadKeyFully: Single-byte reads limited to BYTES_PER_CHECKSUM (256 KB)
instead of the full ~17 MB key; bulk checks use assertArrayEquals.
- testSeek: 20 random seeks instead of 100; forward/backward scans use fixed
seekSize steps (avoids a zero-step loop and keeps block/chunk-aligned coverage).
- Positioned read: 2 random rounds instead of 5 (edge positions unchanged).
- testAll: @ParameterizedTest for pre-read on/off (same two scenarios as
before).
Speed (local)
After mvn clean install -DskipTests -DskipShade -DskipRecon -DskipDocs
-Dmdep.analyze.skip=true:
Total | testReadKey | testAll (preRead) | testAll (no preRead)
-- | -- | -- | --
Before | 77.34 s | 43.37 s | 11.83 s | 6.77 s
After | 28.29 s | 0.83 s | 14.66 s | 0.87 s
Roughly 63% faster overall (~77s → ~29s)
**Current & future improvements :**
In this PR (current):
- Less repeated full-key reading and byte-at-a-time I/O over 17 MB.
- Smaller, appropriate mini-cluster for stream-read integration.
- Same scenarios: multi-block key size, streamReadBlock, pre-read 0 vs
default, checksum NONE, seek error paths, empty block, positioned read.
Possible follow-ups (not in this PR):
- Run testReadEmptyBlock only once (e.g. when preRead == true) for ~1–2s on
the second parameterized run.
- Add middle fixed buffer sizes (e.g. 64 KB, 1 MB) if reviewers want closer
parity with the old power-of-4 matrix at small extra cost.
- Optionally merge testReadKey into a single @Test with testAll to match the
class comment (minor JUnit overhead; weaker per-method CI timing).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16099
## How was this patch tested?
mvn -pl :ozone-integration-test test -Dtest=TestStreamBlockInputStream
-DskipShade -DskipRecon -DskipDocs (before/after locally)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
