realAaronWu commented on code in PR #584:
URL: https://github.com/apache/paimon-cpp/pull/584#discussion_r4656788109
##########
include/paimon/read_context.h:
##########
@@ -208,6 +209,8 @@ class PAIMON_EXPORT ReadContext {
void SetReadSchema(std::unique_ptr<ArrowSchema> schema);
private:
+ friend class FileStoreTable;
Review Comment:
Across 204 checked-in table schema files, the median JSON was 689 bytes, p95
was 2,238, and the largest was 7,691. I don’t have production schema-size or
reader-concurrency telemetry.
I measured Linux ARM64 with 1, 8 and 32 independently constructed managed
readers. Each held its first projected Parquet batch. At 32 readers, the
median-sized fixture retained 21.75 KiB of allocator space for JSON, 1.21% of
the additional live heap for those readers and batches. The largest fixture
retained 240.75 KiB, or 2.43%. Those were 0.59% and 2.01% of this benchmark’s
total allocator-reported heap, respectively. Each independent context held one
copy. The primary and fallback readers share that copy. The patch releases it
after setup; the typed schema and parsing peak are unchanged.
These savings alone don’t justify giving all of `FileStoreTable` private
access. My preference is to defer the optimization pending a workload that
shows meaningful retention. If we pursue it, I’d scope access to a single
internal release helper after fallback initialization and preserve the outer
authorization context for lazy reads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]