bvolpato commented on PR #39619: URL: https://github.com/apache/beam/pull/39619#issuecomment-5531939021
Thanks for pushing on this, @reuvenlax. I've added benchmarks with 1,024 top-level fields and nested rows, alongside the small schemas and both encoding modes. Each nested field contains a two-field row, so the largest case has 2,048 scalar leaves. The full results and commands are in the PR description. My earlier claim of equivalent steady-state performance was too strong. Nested dynamic decoding is about 1.5% slower, and a repeat with the implementation order reversed confirms that cost. The repeat also measures a 4.6% slowdown for the small nested static case. Large dynamic encoding is about 1% faster in the full matrix. Uncached coder creation is 1.9x to 36.7x faster across these shapes, but that does not establish a meaningful improvement in overall worker startup or pipeline runtime. Beyond creation latency, the change removes the per-schema generated classes and the machinery needed to create them. I have not measured the effect on a full worker. The JVM can optimize generated classes differently even when they call the same Java delegates, so the matching source-level loops alone do not settle the performance question. I've kept this as a draft and updated the description to make the measured decoding cost explicit. The new benchmark coverage is in 3326a88b66; the 47 targeted RowCoder and SchemaCoder tests pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
