discivigour opened a new pull request, #8663: URL: https://github.com/apache/paimon/pull/8663
## What changed - Materialize known-length `BlobRef` values into an exactly sized byte array. - Keep the existing streaming fallback for descriptors without a known length. - Reject values larger than the maximum Java byte-array size. - Ensure the input stream is closed after materialization. - Add coverage for direct known-length reads and stream closure. ## Why `BlobRef.toData()` previously used a dynamically growing buffer even though `BlobDescriptor` already exposes the remaining data length. That caused avoidable intermediate allocations, copies, and chunked reads when materializing large blobs. ## Impact Known-length blob references now materialize with a single exact allocation and direct read. Unknown-length descriptors retain the existing behavior. ## Performance Test dataset: a 6 GB BlobView table with 1,000 rows, each containing a 6,400 KiB blob. | Metric | Before | After | Improvement | | --- | ---: | ---: | ---: | | Elapsed time | 174.918 s | 75.282 s | 56.96% lower | - Speedup: **2.32×** - Throughput improvement: **approximately 132.35%** - Time saved per round: **99.636 seconds** ## Validation - `mvn -pl paimon-common -Dtest=BlobTest test` - 6 tests passed, 0 failures -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
