On 14 June 2011 12:02, Javier Lopez Cacheiro <[email protected]> wrote: > Hi William, > >>> Yes, I am aware of this limitation. In fact I would be very interested >>> in an alternative solution to control disk usage per directory. Do you >>> know if this is currently possible in GE? >>> >>> I have been thinking several times in implementing something for this >>> purpose, using dynamic disk quotas or running periodically du. >>> >> Assuming this is for local scratch space on Linux one thing I've been >> considering is using almost the whole disk as swap and then mounting a >> suitably sized tmpfs on $TMPDIR in the prolog >> followed by a umount in the epilog. Have to be careful to ensure it >> didn't cause real memory to be devoted to the VFS rather than >> processes though. > Taking your approach has made me think of an alternative where LVM could > be used in an easy way, creating a VG for scratch and configuring the > LVs on demand for each job using the size requested. At the end the LV > would be easily removed. > > To reduce the delay during filesystem creation, probably xfs could be > used that is much faster to create. > > It is interesting to explore these options using prolog/epilog, even if > the final solution for all purposes would probably be implemented inside > shepperd. Inside shepherd? uggh. One of the things I like about GE is that it follows the software tools philosophy pretty closely. It does relatively little but has hooks in almost all the right places to let you add needed functionality. I suppose one advantage of shepherd or starter_method is that you could combine it with Linux namespaces to make a very private directory. When combined with tmpfs it might even auto free the space without an explicit umount. > > Cheers, > Javier >> William >>> >
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
