Benjamin Mahler created MESOS-740:
-------------------------------------
Summary: Create an Archive abstraction.
Key: MESOS-740
URL: https://issues.apache.org/jira/browse/MESOS-740
Project: Mesos
Issue Type: Improvement
Reporter: Benjamin Mahler
I was thinking about this lately given the resource monitor, the slave, and the
master all keep archived information around at the cost of increased memory
consumption.
An archive abstraction would simplify the code in these components, as well as
help ensure the master / slave remain below a certain amount of memory
consumption. This proves useful if were were to start placing the slave inside
a cgroup with a memory limit.
The archive could periodically monitor memory consumption (or use memory
threshold notifications) to decide when to GC some of the archived information.
This would be slightly analogous to how we GC the disk based on disk usage.
Thinking about what this may look like:
1. Archive is map-like and holds the data using <Key, Value>. This would need
to be templated and as such may require that separate components create
separate Archives.
2. Archive distributes Futures that become ready when memory consumption needs
to be lowered. Components can set callbacks on these futures to routines that
remove data from their historical state. At this point Archive is really just a
MemoryMonitor abstraction.
[~benjaminhindman] I heard you may have been thinking about this as well at one
point, let me know what ideas you have!
--
This message was sent by Atlassian JIRA
(v6.1#6144)
