itsvikramagr commented on issue #24922: [SPARK-28120][SS]  Rocksdb state 
storage implementation
   > cool. is the issue here [#24922 
   Yes this is now resolved. I have made the following changes to resolve it 
   - There was a typo in setting dataBlockSize. Instead of KBs, I was setting 
it to Bytes. 
   - For range scan, I was creating a RocksIterator which was not closed 
   - I fixed some of the rocksDB configs. Taken help from 
In particular, set Max open files, strictly limited caches (No pinning of 
metadata in cache), Disable FillCache for range scan operations.
   Overall the memory usage is now contained and it's around 1-3 GB depending 
upon the data.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to