+jpeach The polling mechanism is used by the "disk/du" isolator to handle the case where we don't have filesystem support for enforcing a quota on a per-directory basis. I believe the "disk/xfs" isolator will stop writes with EDQUOT without killing the task:
http://mesos.apache.org/documentation/latest/isolators/disk-xfs/ On Tue, Nov 28, 2017 at 1:19 PM, Gabriel Hartmann <[email protected]> wrote: > I agree with pretty much everything Hendrik just said with the exception > of the use of disk quota. The polling mechanism employed for enforcing > disk usage implies that any breach of the disk usage limit by a Task also > implies loss of access to that data forever. This is true for ROOT volumes > at least. MOUNT volumes can be configured to map to "real" devices which > can provide normal write failures when exceeding disk limits instead of > essentially revoking all access to data forever. > > On Mon, Nov 27, 2017 at 11:34 PM Hendrik Haddorp <[email protected]> > wrote: > >> As said, I only use persistent volumes with my only scheduler straight >> on Mesos so do not exactly know how this works in Marathon... >> >> The persistent volume is created on a Mesos agent and basically ends up >> being a folder on that hosts disk. So yes, you can not use the volume on >> a different agent/slave. For marathon you would need to set a hostname >> constraint that makes sure the same host is used when restarting the >> task. You won't be able to use fail over to different agents just have >> Marathon restart your task once it fails. Also only one task at a time >> can have the volume bound. >> >> Yes, you can achieve persistence in pretty much the same way by using a >> hostpath but then you are using implicit knowledge about your >> environment, which is not very clean in my opinion, and thus have a >> tighter coupling. The nice thing about persistent volumes is that they >> are managed by Mesos. I do not need to tell the Mesos admin that I need >> space at some location. I do not need to do something special if I have >> multiple instances running as they get all their own directory. And I >> can programatically destroy the volume and then the directory on the >> host gets deleted again (at least since Mesos 1.0). So in my opinion the >> usage of persistent volumes is much cleaner. But there are certainly use >> cases that do not really work with them, like being able to fail over to >> different host. For that you would wither need a shared network mount or >> storage like HDFS. Btw, the Mesos containerizer should also enforce disk >> quotas so your task would not be able to fill the filesystem. >> >> On 27.11.2017 16:11, Dino Lokmic wrote: >> > yes I did. So I don't have to prepare it before task? I can't use >> > volume created on slave A, from slave B >> > >> > Once task fails where will it be restarted? Do I have to specify host? >> > >> > If I do, it means I can achieve "persistence" same way I deploy now, >> > by specifying hostpath for volume and hostname >> > >> > .... >> > "constraints": [ >> > [ >> > "hostname", >> > "CLUSTER", >> > "MYHOSTNAME" >> > ] >> > ], >> > "container": { >> > "type": "DOCKER", >> > "volumes": [ >> > { >> > "containerPath": "/opt/storm/storm-local", >> > "hostPath": "/opt/docker_data/storm/storm-local", >> > "mode": "RW" >> > }, >> > { >> > "containerPath": "/opt/storm/logs", >> > "hostPath": "/opt/docker_logs/storm/logs", >> > "mode": "RW" >> > }, >> > { >> > "containerPath": "/home/xx/runtime/storm", >> > "hostPath": "/home/xx/runtime/storm", >> > "mode": "RO" >> > } >> > ], >> > "docker": { >> > "image": "xxx/storm-1.1.0", >> > "network": "HOST", >> > "portMappings": [], >> > "privileged": false, >> > "parameters": [], >> > "forcePullImage": true >> > } >> > }, >> > >> > .... >> > >> > >> > >> > On Mon, Nov 27, 2017 at 3:05 PM, Hendrik Haddorp >> > <[email protected] <mailto:[email protected]>> wrote: >> > >> > I have my own scheduler that is performing a create operation. As >> > you are using Marathon this call would have to be done by Marathon. >> > Did you read >> > https://mesosphere.github.io/marathon/docs/persistent-volumes.html >> > <https://mesosphere.github.io/marathon/docs/persistent-volumes.html> >> ? >> > >> > On 27.11.2017 14:59, Dino Lokmic wrote: >> > >> > @hendrik >> > >> > How did you create this >> > "my-volume-227927c2-3266-412b-8572-92c5c93c051a" volume? >> > >> > On Mon, Nov 27, 2017 at 7:59 AM, Hendrik Haddorp >> > <[email protected] <mailto:[email protected]> >> > <mailto:[email protected] >> > <mailto:[email protected]>>> wrote: >> > >> > Hi, >> > >> > I'm using persistent volumes directly on Mesos, without >> > Marathon. >> > For that the scheduler (like Marathon) has to first >> > reserve disk >> > space and then create a persistent volume with that. The >> next >> > resource offer message then contain the volume in "disk" >> > resource >> > part of the offer. Now you can start your task. In the >> > request you >> > would need to include the resources and for the >> > "container" part >> > of the request you would have: >> > volumes { >> > container_path: "/mount/point/in/container" >> > host_path: >> > "my-volume-227927c2-3266-412b-8572-92c5c93c051a" >> > mode: RW >> > } >> > >> > The container path is the mount point in your container >> > and the >> > host path is the id of your persistent volume. >> > >> > In case you use marathon the documentation should be this: >> > https://mesosphere.github.io/marathon/docs/persistent- >> volumes.html >> > <https://mesosphere.github.io/marathon/docs/persistent- >> volumes.html> >> > >> > <https://mesosphere.github.io/marathon/docs/persistent- >> volumes.html >> > <https://mesosphere.github.io/marathon/docs/persistent- >> volumes.html>> >> > >> > regards, >> > Hendrik >> > >> > >> > On 23.11.2017 10:00, Dino Lokmic wrote: >> > >> > I have few machines on Linode and I run Mesos there. Can >> > someone explain to me, how to set volumes right. >> > >> > Now I run taks via marathon like this >> > >> > ... >> > >> > "constraints": [ >> > [ >> > "hostname", >> > "CLUSTER", >> > "HOSTNAME" >> > ] >> > ], >> > "container": { >> > "type": "DOCKER", >> > "volumes": [ >> > { >> > "containerPath": "/opt/storm/storm-local", >> > "hostPath": "/opt/docker_data/storm/storm- >> local", >> > "mode": "RW" >> > } >> > ], >> > "docker": { >> > "image": "xxxx", >> > "network": "HOST", >> > "portMappings": [], >> > "privileged": false, >> > "parameters": [], >> > "forcePullImage": true >> > } >> > }, >> > ... >> > >> > So if task is restarted I can be sure it has access to >> > previously used data. >> > You can see I have scaling problem and my task is >> > depending on >> > this node. >> > >> > I would like for my apps to be node independent and >> > also that >> > they have redundant data. >> > >> > What is best practice for this? >> > >> > I want to scale aplication to 2 instances, I1 and I2 >> > >> > Instance I1 runs on agent A1 and uses volume V1 >> > Instance I2 runs on agent A2 and uses volume V2 >> > >> > If agent A1 stops, I1 is restared to A3 and uses V1 >> > If V1 failes I1 uses copy of data from V3... >> > >> > >> > Can someone point to article describing this, or at >> > least give >> > me few "keywords" >> > >> > >> > Thanks >> > >> > >> > >> > >> > >> > >> >>
