That looks like it would solve it, thanks! On Tue, Mar 31, 2015 at 4:24 PM, Jie Yu <[email protected]> wrote:
> Hi, in Mesos 0.23.0 (the next release), you'll be able to use the > persistence primitives provided by Mesos. > https://issues.apache.org/jira/browse/MESOS-1554 > > Basically, your framework can create a persistent volume (has a unique > handle) while launching a task and re-use that handle when re-launching the > task. The same persistent volume will be mounted into the sandbox. Any data > stored in the volume will be persisted. > > - Jie > > On Tue, Mar 31, 2015 at 4:00 PM, Charles Allen < > [email protected]> wrote: > >> I am working on potentially porting druid.io to a mesos framework. One >> limitation for production use is that there is a lot of data cached locally >> on disk that does not need to be re-fetched during a rolling restart. >> >> If I were to take the simplest mesos route, each instance of the >> disk-cache-heavy task would have its own executor and would have to refresh >> the disk cache from deep storage each time it starts. >> >> A more complex route would be to have a standalone executor which handles >> the forking and restarts of tasks in order to maintain the working >> directory of the task. >> >> A slightly more hacky way of doing it would be to allow the >> disk-cache-heavy task to each have their own executor but use a common >> SharedFilesystem. But I'm not clear if SharedFilesystem would persist >> beyond an executor's lifespan. >> >> In such a case (where on-disk data would need to be "immediately" >> available after a rolling restart) is there a recommended approach to >> making sure the data persists properly? >> >> Thanks, >> Charles Allen >> >> >> > -- <http://www.metamarkets.com/> Charles Allen PhDSr. Software Engineer| METAMARKETS <http://www.metamarkets.com/>m 765.490.0454|t @drcrallen <https://twitter.com/drcrallen>[email protected]
