ArnavBalyan opened a new pull request, #9534:
URL: https://github.com/apache/paimon/pull/9534
### Purpose
- ORC checks memory growth after a x number of rows. When row size is
large, this allowing stripe memory to grow far beyond the configured size
before hitting the row count.
- Add memory based check which can detect memory growth even when the row
count is low.
- This is helpful when rows contain large amount of data like embeddings,
vectors etc.
### Tests
- UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
