Doris-Breakwater commented on issue #68538: URL: https://github.com/apache/doris/issues/68538#issuecomment-5862952907
Breakwater-GitHub-Analysis-Slot: slot_a8ce927cd201 Maintainer triage (read-only code review; no reproduction yet): **Confirmed in the `3.1.4-rc02` source snapshot:** In cloud schema-change COMMIT, `process_schema_change_job` writes a recycle record for each existing shadow-tablet rowset in `[2, alter_version]`, then promotes every listed temporary output rowset. `finish_tablet_job` commits those writes, the tablet metadata/statistics updates, and job cleanup in **one transaction per tablet**. There is no batching or `approximate_bytes()` guard in this path. The FDB error 2101 is mapped to `TXN_BYTES_TOO_LARGE`, whose message is "Transaction exceeds byte limit". The reported error is therefore consistent with this commit exceeding FDB's transaction limit. This does **not** mean one transaction spans all 5,754 tablets, or that every rowset in the table is copied into it. See [`meta_service_job.cpp`](https://github.com/apache/doris/blob/3.1.4-rc02/cloud/src/meta-service/meta_service_job.cpp#L1273-L1490) and [`txn_kv.cpp`](https://github.com/apache/doris/blob/3.1.4-rc02/cloud/src/meta-store/txn_kv .cpp#L163-L189). The load lazy-commit path does split conversions into batches based on rowset metadata size and `txn_lazy_max_rowsets_per_batch`; it is a useful precedent, but schema-change COMMIT also changes tablet visibility, statistics, recycle state, and job state. A fix needs durable progress/idempotent retry and must keep final activation consistent with those updates; simply splitting the existing loop into independent commits would need careful correctness review. The FE's ordinary schema-change task retry setting covers delete-bitmap-lock and network errors, not this reported `INVALID_ARGUMENT` failure ([`AlterJobV2.java`](https://github.com/apache/doris/blob/3.1.4-rc02/fe/fe-core/src/main/java/org/apache/doris/alter/AlterJobV2.java#L267-L275)). **Still needed to confirm this instance and size driver:** exact FE/BE/MetaService build hashes (the review used `3.1.4-rc02`, which may differ from the deployed 3.1.4 build); failing base/shadow tablet IDs and job ID; MetaService log line with the underlying FDB error code for that commit; counts of the shadow rowsets recycled and temporary output rowsets promoted for that tablet; and the approximate transaction bytes or serialized rowset-metadata sizes. A small-load/low-rowset reproduction and a high-rowset reproduction on the same build would establish the threshold. Please redact deployment identifiers as needed. **Next step:** Treat this as a likely schema-change transaction-size limitation, measure the failing tablet's metadata footprint, then add a regression case above the FDB transaction limit while designing a byte-bounded, resumable commit protocol. The reported pre-compaction workaround is plausible if it actually reduces rowsets for the affected tablet; check compaction completion and rowset counts before rerunning the ALTER. A retry alone is unlikely to help while the transaction contents remain the same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
