Myasuka commented on pull request #14744: URL: https://github.com/apache/flink/pull/14744#issuecomment-777255056
@StephanEwen , yes I agree with your concern. Current memory control mechanism of RocksDB cannot handle any malloced memory not inserted into block cache or write buffer, those just uncompressed data block could have OOM risk. Actually we are trying several optimizations to improve customer experience: 1. Partitioned index option https://github.com/apache/flink/pull/14341 to help avoid performance problem during memory control. 1. Try to bump RocksDB after 6.10+ to have more restrict block cache size limit. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
