Noted, thanks! On Sat, Dec 12, 2020 at 2:28 AM David Anderson <dander...@apache.org> wrote:
> RocksDB can not be configured to spill to another filesystem or object > store. It is designed as an embedded database, and each task manager needs > to have sufficient disk space for its state on the host disk. You might be > tempted to use network attached storage for the working state, but that's > usually a bad idea (for performance reasons). > > Regards, > David > > On Fri, Dec 11, 2020 at 10:59 PM Rex Fenley <r...@remind101.com> wrote: > >> Hi, >> >> I'm wondering if the RocksDB state storage has any way of automatically >> spilling into S3 (assuming that's what's set for checkpoints) when the host >> disk starts filling up too much? If not, is the expectation that the host >> disk must always carry all the relevant data in RocksDB, meaning we're >> bound by the size of host storage? >> >> Thanks! >> >> -- >> >> Rex Fenley | Software Engineer - Mobile and Backend >> >> >> Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> >> | FOLLOW US <https://twitter.com/remindhq> | LIKE US >> <https://www.facebook.com/remindhq> >> > -- Rex Fenley | Software Engineer - Mobile and Backend Remind.com <https://www.remind.com/> | BLOG <http://blog.remind.com/> | FOLLOW US <https://twitter.com/remindhq> | LIKE US <https://www.facebook.com/remindhq>
