mao-liu commented on issue #7030:
URL: https://github.com/apache/paimon/issues/7030#issuecomment-3743832794
I have also investigated using the WriterCoordinator, including extending
the StoreCompactOperator to support coordinated writes.
The performance however has not been promising, and memory usage on the
Flink Job Manager is extremely high.
- For a manifest directory of <1GB, the job manager needed >10G allocated to
writer coordinator cache, and still observed cache misses >60% of the time, as
well as occasional OOMs leading to job crashes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
