realAaronWu opened a new pull request, #462: URL: https://github.com/apache/paimon-cpp/pull/462
### Purpose **Validation draft:** local correctness, fault-injection, sanitizer and independent review gates are complete. Upstream compiler/architecture CI is pending; this is not marked ready for upstream review yet. Related integration work: https://github.com/sirius-db/sirius/issues/2211 and https://github.com/polardb/duckdb-paimon/pull/145. This is an independent native implementation; it does not change the bridge or depend on the cancellation PRs. Add an opt-in `TableScan::CreateWithPlanningBudget` entry point. The query-owned budget admits requested storage **before allocation**, including metadata input, JSON DOM/parser stacks, Avro schema/records, Zstandard workspace and decoded output, temporary containers, native file metadata, splits and ownership blocks. Exhaustion returns `CapacityError`; allocation failure returns `OutOfMemory`. Neither failure returns a partial plan or falls back to unrestricted planning. The limit is **cumulative admitted allocation bytes**: frees do not replenish it. This conservative contract bounds temporary coexistence as well as final storage; it is not an RSS or peak-live measurement. Fixed diagnostics, the budget control object, process-owned immutable constants and existing shared services are outside the variable payload. Retained caller schema/path/options are charged before accepting the scan. Escaped nonempty splits keep the budget/allocator alive. The initial profile is explicit: Linux/libstdc++ 9+ C++11 ABI, retained schema, positive numeric snapshot ID, local unpartitioned append-only bucket-unaware tables, matching file schema, version-2 ADD-only Avro manifests, null/Zstandard compression and Parquet/ORC files. Unknown decoded Zstandard lengths are supported with admitted streaming workspace/output growth. Unsupported profiles, DELETE entries, duplicates, wrong writer types and malformed inputs fail explicitly. Other platforms/old ABI/GCC8 return `NotImplemented` through a compiled factory stub. Ordinary `TableScan::Create` keeps its existing path. Generic Avro/Arrow decoding does not route all variable allocations through an admission allocator. The bounded metadata decoder therefore uses budget-backed parser/PMR storage and static Zstandard workspace. Existing native publication constructors receive explicit pre-admission based on the qualified GNU string/vector/shared-pointer implementation; no per-row estimate substitutes for actual allocation control. The path is synchronous and does not use shared metadata caches or pruning statistics. ### Tests Qualified locally on Linux x86_64, GNU 15.3, pinned bundled dependencies: - PASS: full native library build; registered `paimon-bounded-table-scan-test` through CTest, 12 cases. - PASS: real Parquet full-field and ORC `id`-projection file inventory/data parity, independent ID values, and reading an escaped split after scan/plan destruction. Existing ORC TIME decoding is outside this change. - PASS: exact limit / one byte short / zero limit, concurrent shared admission, failure at every arena/publication allocation, malformed/ambiguous metadata and unsupported profile rejection. - PASS: actual pool and `operator new` requests observed, with long-string observation/failure calibration. The static test runtime includes out-of-line libstdc++ string allocations; retained-input credits cannot mask missing publication admission. - PASS: four disposable mutations are rejected by assertion-specific failures: removing string admission, quota checking, JSON failure draining, or writer-schema validation. - PASS: the same 12 cases with ASAN + UBSAN, leak detection enabled and no suppression files; independent reviewer replay also passed. - PASS: existing core suite, 2,478/2,478; repository-wide pre-commit hooks and affected-file hooks after final changes. - PASS: unsupported C++11 ABI syntax checks for implementation and test translation units. Actual GCC8/AArch64/compiler matrix execution is pending CI. A serial warm Debug comparison (100 samples, tiny checked-in fixtures, same source/dependencies) measured ordinary → bounded median planning time of 6.10 → 2.68 ms for Parquet and 2.74 → 1.49 ms for ORC. This is narrow fixture evidence, not general performance qualification; cold-cache, Release and concurrent workloads were not measured. Independent isolated source/delivery review covered final `8f03510b`, reconciled 5,473 source file hashes and saved binaries, replayed normal/sanitizer cases and audited the four mutation closures. No blocking internal findings remain for the documented qualified profile. Upstream CI remains pending. ### API and Format Add exported `PlanningBudget` and an additive static scan factory. Existing scan/read interfaces and persisted formats are unchanged. Native `Plan`/`DataSplitImpl` outputs are consumed by the existing readers. This API bounds metadata planning; later data-reader buffers and caller serialization/copies require their own policy. ### Documentation Add the bounded-planning user guide with the admission contract, supported/rejected profile, ownership and allocation qualification, limitations, and usage example. ### Generative AI tooling Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
