JingsongLi commented on code in PR #9243:
URL: https://github.com/apache/paimon/pull/9243#discussion_r3790974418
##########
paimon-core/src/main/java/org/apache/paimon/table/source/InnerTableRead.java:
##########
@@ -63,6 +64,11 @@ default InnerTableRead forceKeepDelete() {
return this;
}
+ @Override
+ default InnerTableRead withReadBatchSizeController(ReadBatchSizeController
controller) {
+ return this;
Review Comment:
[P2] Avoid silently ignoring the controller on physical wrapper reads.
`AuditLogRead` wraps a real `dataRead` but inherits this no-op, and
`BinlogRead` inherits that wrapper, so `withReadBatchSizeController` appears
successful while the underlying ORC/Parquet reader never receives the
controller. Runtime batch reductions and memory/backpressure control therefore
do nothing on those paths. Please override and delegate in `AuditLogRead`,
audit similar wrappers such as `LookupCompactDiffRead`, and add wrapper-path
tests.
##########
paimon-core/src/main/java/org/apache/paimon/table/format/FormatTableRead.java:
##########
@@ -73,6 +74,12 @@ public TableRead withIOManager(IOManager ioManager) {
return this;
}
+ @Override
+ public TableRead withReadBatchSizeController(ReadBatchSizeController
controller) {
+ read.withReadBatchSizeController(controller);
Review Comment:
[P1] Keep the controller scoped to this `TableRead`. `read` is the shared
`FormatReadBuilder` reused by every `FormatTableRead` returned from
`newRead()`, and its per-file readers are created lazily. Configuring a second
read can therefore overwrite the first read's controller, so later suppliers
from the first read observe the second controller even without concurrency.
This also stores a non-serializable `ReadBatchSizeController` in a serializable
`ReadBuilder`; serializing the builder after this call throws
`NotSerializableException`. Please keep/capture the controller on
`FormatTableRead`, pass that snapshot into reader creation, and add multi-read
isolation plus serialization regression tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
