chaokunyang opened a new pull request, #3910: URL: https://github.com/apache/fory/pull/3910
## Why? Count-driven collection, map, and compatible-skip reads can legally consume fewer bytes than items when element codecs have empty bodies. Without independent work accounting, a small input can force disproportionate CPU work or repeated skip iterations, and count-derived capacity can be allocated without proportional backing bytes. ## What does this PR do? - Adds a root-scoped `maxUnbackedContainerItems` allowance, defaulting to 8192, across Java, C++, Rust, Swift, Python/Cython, Go, JavaScript, C#, and Dart. - Settles collection shortfall every 1024 items and map shortfall at existing chunk boundaries, with final-tail settlement and root-only failure cleanup. - Gates count-derived allocation using readable bytes plus the remaining allowance, while preserving existing graph-memory accounting. - Uses compile-time, generated, or resolved serializer progress facts so proven-positive collection/map paths keep the prior loop shape and do not access the new budget or cursor state. - Keeps custom, dynamic, unknown, and compatible-skip bodies conservative; rejects zero map chunks only where the owning loop would otherwise make no progress. - Adds focused root-reset, allocation, stream-compaction, generated-code, static-fast-path, map-chunk, and compatible-skip tests plus public/security documentation. - Does not change the wire encoding or RefReader/RefWriter architecture. ## Related issues None. ## AI Contribution Checklist - [x] Substantial AI assistance was used in this PR: `yes` - [x] I included the completed checklist and AI Usage Disclosure below. - [x] I can explain and defend all important changes without AI help. - [x] I reviewed the AI-assisted code changes line by line before submission. - [x] I completed self-review and fixed issues before requesting AI review. - [x] Two fresh AI reviewers reviewed final HEAD `e42ab7a3624e8e1bf16c9c6ce379713fe61e1465`: one Fory-guided full-diff reviewer and one independent general reviewer. - [x] All actionable comments were addressed and both final reviewers reported no actionable findings. - [x] Final persisted review-artifact links will be added immediately after PR creation. - [x] Relevant local verification was run and reviewed. - [x] Tests and documentation were updated. - [x] Apache-2.0-compatible provenance was confirmed; no incompatible third-party code was introduced. ### AI Usage Disclosure - substantial_ai_assistance: yes - scope: design refinement, cross-runtime implementation, tests, documentation, self-review, and fix iteration - affected_files_or_subsystems: collection/map/compatible-skip readers, read contexts, generated/static serializer metadata, configuration, tests, and deserialization documentation across all supported runtimes - ai_review: line-by-line self-review completed; the Fory-guided review found two hot-path issues, both accepted portions were fixed and inner-reviewed; a fresh full-diff Fory reviewer and a separate independent general reviewer then both returned `No actionable findings.` - ai_review_artifacts: persisted PR-comment links will be inserted immediately after PR creation - human_verification: focused and full runtime suites were reviewed, including Java (287 focused), Kotlin (88 runtime + 24 KSP), Scala (108 with 2 ignored), C++ (five focused targets, 123 tests), Rust workspace checks/tests, Swift full package (267), Python/Cython, Go, JavaScript, C#, Dart, formatting/static checks, and relevant xlang suites - performance_verification: no benchmark was run per explicit maintainer instruction; positive generated/static paths are structurally free of budget/cursor accounting, with generated-code and focused fast-path assertions - provenance_license_confirmation: Apache-2.0-compatible provenance confirmed; no incompatible third-party code introduced ## Does this PR introduce any user-facing change? - [x] Does this PR introduce any public API change? - [ ] Does this PR introduce any binary protocol compatibility change? The public configuration gains the bounded unbacked-container-item limit. The wire encoding is unchanged. ## Benchmark Not run, per explicit maintainer instruction for this robustness task. The implementation preserves positive static/generated loop shape and adds no per-element allocation; focused source/codegen assertions cover removal of budget/cursor work from proven-positive paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
