Thank you, Robert, for your answer. Could you kindly further elaborate on number 1 as I am not familiar with Plasma codebase yet? Are you saying persistence is available out of the box? else what specific things need to be added to Plasma codebase to make this happen?
Thank you, Mike On Thu, Jan 18, 2018 at 11:43 PM, Robert Nishihara < [email protected]> wrote: > Hi Mike, > > 1. I think yes, though we'd need to turn off the automatic LRU eviction > that happens when the store fills up. > > 3. I think there are some edge cases and it depends what is in your > DataFrame, but at least if it consists of numerical data then the two > representations should use the same underlying data in shared memory. > > On Thu, Jan 18, 2018 at 11:37 PM Mike Sam <[email protected]> wrote: > > > I am interested to implement an arrow based persisted cache store and I > > have a few related questions: > > > > 1. > > > > Is it possible just to use Plasma for this goal? > > (My understanding is that it is not persistable) > > Else, what is the recommended way to do so? > > 2. > > > > Is feather the better file format for persistence to avoid > > re-transcoding hot chunks? > > 3. > > > > When Pandas load data from plasma/arrow, is it doubling the memory > > usage? (One for the arrow representation, one for pandas > representation) > > > > -- > > Thanks, > > Mike > > > -- Thanks, Mike
