fengruotj opened a new pull request, #11069:
URL: https://github.com/apache/paimon/pull/11069

   ### Purpose 
    
     Closes #11054. 
    
     Introduce `sink.committer-managed-memory` and 
`sink.committer-offheap-memory` for regular global committers and combined CDC 
multi-table committers, including propagation from database synchronization 
table options. Both options have no default value, must be positive, and 
require `sink.committer-memory`. The existing CPU default of one core still 
applies unless `sink.committer-cpu` is set. 
    
     For example, the following configuration requests 256 MiB of task heap, 64 
MiB of task off-heap, and 128 MiB of managed memory for each shared slot in the 
committer group: 
    
     ```properties 
     sink.committer-memory = 256 mb 
     sink.committer-offheap-memory = 64 mb 
     sink.committer-managed-memory = 128 mb 
     ``` 
    
     Committer budgets are independent of explicit writer budgets and do not 
change the writer's managed-memory allocator or allocation weight. These 
settings do not apply to the JobManager-side coordinator commit path. 
    
     ### Tests 
    
     - `CommitterMemoryTest`: StreamGraph and JobGraph resource propagation, 
byte precision, writer/committer resource independence with the managed 
allocator enabled and disabled, missing heap and invalid values, and unchanged 
defaults. 
     - `FlinkCdcMultiTableSinkTest`: database-builder propagation of committer 
budgets, writer/committer resource separation, configuration validation, and 
legacy-constructor compatibility. 
     - `WriterOffHeapMemoryTest` and `WriterManagedMemoryTest`: regression 
coverage for existing writer resource requests and managed-memory allocation 
behavior. 
     - `FlinkSinkTest`: regression coverage for coordinator commit 
prerequisites and checkpoint compatibility. 
    
     Local validation passed with zero failures, errors, or skipped tests: 
    
     - Flink 1 profile: 68 tests across the five classes above (common: Flink 
1.20.1; CDC: Flink 1.20.4; JDK 11). 
     - Flink 2 profile: 54 tests across the four common-module classes (Flink 
2.2.0; JDK 17).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to