Re: Suggestion for hammer cleanup
Matthew Dillon wrote: > I think a hammerd is a good idea or more likely a 'hammer cleanup' > with a deamon-mode option that leaves it running in the background, > but operating on the disk very lightly (as in very, VERY lightly) > until the time schedule tells it that it really has to run harder. > > Implementing this would not be entirely trivial. The only way > it can be done properly is for the kernel HAMMER ioctls to throw in > the load checks in the B-Tree scanning loop and to release the b-tree > cursor locks while it is sleeping. The sleeps can't really be done > in the hammer user utility. > > -Matt this is really interesting that I stumble across this tonight.. as I am taking a break from working on libhammer.. hmm RG
Re: Suggestion for hammer cleanup
I think a hammerd is a good idea or more likely a 'hammer cleanup' with a deamon-mode option that leaves it running in the background, but operating on the disk very lightly (as in very, VERY lightly) until the time schedule tells it that it really has to run harder. Implementing this would not be entirely trivial. The only way it can be done properly is for the kernel HAMMER ioctls to throw in the load checks in the B-Tree scanning loop and to release the b-tree cursor locks while it is sleeping. The sleeps can't really be done in the hammer user utility. -Matt
Re: Suggestion for hammer cleanup
On Mon, Oct 18, 2010 at 9:31 PM, wrote: > What about hammerd? > > * Starts when system starts > > * Wakes up every 10 seconds or so to check current system load/memory > usage to estimate if its appropriate to run a cleanup operation at this > time. > > * Can have a text file configuration where you can specify maximum > size(GB) of history per PFS or % of disk space > > * Remembers last time it was run (stores information on disk in some text > file) > > * Maybe it can detect if it's a laptop and running on battery to defer > cleanup till power is plugged in? > > The daemon itself would be fairly inexpensive to run. > > Petr > > I rather like the idea of hammer(e)d; I've been joking for a while about a HammerTimeDaemon, which would watch for specific udev events from a certain USB key/disk and hammer mirror-stream onto it... hammerd would be a nice place to put hammer things to run on some conditions rather than a schedule. -- vs
Re: Suggestion for hammer cleanup
What about hammerd? * Starts when system starts * Wakes up every 10 seconds or so to check current system load/memory usage to estimate if its appropriate to run a cleanup operation at this time. * Can have a text file configuration where you can specify maximum size(GB) of history per PFS or % of disk space * Remembers last time it was run (stores information on disk in some text file) * Maybe it can detect if it's a laptop and running on battery to defer cleanup till power is plugged in? The daemon itself would be fairly inexpensive to run. Petr
Re: Suggestion for hammer cleanup
On Mon, Oct 18, 2010 at 4:39 PM, Chris Turner wrote: > Samuel J. Greear wrote: >> >> That said, I think it would be fine to commit one or more optional >> stopgap measures/scripts to the RC system, for mobile users and etc., >> as long as it is well documented that they may go away if a better >> solution is developed or derived. > > not to flamebait or something - but: > > wouldn't the system crontab be a better place? > or perhaps create an /etc/periodic/hourly ? > > the 'newsyslog' is in /etc/crontab which is the only hourly > job I can think of off hand.. > > the script can still of course source rc.conf or something > to configure the job if the job is enabled by default > Sure, my only real point was that it should be a knob that users have to turn on (not on by default). I do not know what the precedent is for periodic/rc cross-pollination. Sam
Re: Suggestion for hammer cleanup
Samuel J. Greear wrote: That said, I think it would be fine to commit one or more optional stopgap measures/scripts to the RC system, for mobile users and etc., as long as it is well documented that they may go away if a better solution is developed or derived. not to flamebait or something - but: wouldn't the system crontab be a better place? or perhaps create an /etc/periodic/hourly ? the 'newsyslog' is in /etc/crontab which is the only hourly job I can think of off hand.. the script can still of course source rc.conf or something to configure the job if the job is enabled by default
Re: Suggestion for hammer cleanup
On Mon, Oct 18, 2010 at 4:06 PM, Chris Turner wrote: > elekktrett...@exemail.com.au wrote: >> >> Suggestions? > > quick-fix / hack wise - > > probably setup some job to run way more often > that checks the status & makes a determination - > or move the job to something like anacron, etc > > although, in a laptop situation - you might want > to manage this manually - because the heavy work > from reblocking, etc could easily suck the juice > out of your battery when you dont want it.. > Until / unless we have the disk scheduler tuned to a point that we can just run a "prunerandreblockerd" all the time, I think it would be sufficient (and in my opinion, preferred) to simply warn the administrator based on some metric, time since last reblock, egregious amount of storage used by history, low amount of free disk space, etc., or some combination, if it is to be on by default. ... That is, if we do anything at all. I think making it clear in the HAMMER documentation that the periodic maintenance must be performed, and why, should be sufficient. I do not feel that it is terribly obvious right now. That said, I think it would be fine to commit one or more optional stopgap measures/scripts to the RC system, for mobile users and etc., as long as it is well documented that they may go away if a better solution is developed or derived. If there were a consensus, it is possible we could see about getting a script or two written for RC and/or documentation updates made as a part of Google Code-In, if DragonFly is accepted. Assuming there are no volunteers participating in this thread. Sam
Re: Suggestion for hammer cleanup
elekktrett...@exemail.com.au wrote: Suggestions? quick-fix / hack wise - probably setup some job to run way more often that checks the status & makes a determination - or move the job to something like anacron, etc although, in a laptop situation - you might want to manage this manually - because the heavy work from reblocking, etc could easily suck the juice out of your battery when you dont want it..
Re: Suggestion for hammer cleanup
Probably it will be nicer for desktop users to have a "run when history is using more than X space" than by periodic. On Mon, Oct 18, 2010 at 02:49, wrote: > I think we definitely must have hammer cleanup not depending on periodic. > Especially laptop/workstation users, since they may be running only couple > of hours a day. > > The system should be able to automatically initiate hammer cleanup > whenever it's been more than 1 day/x hours since last time it was run. It > doesn't matter if its day or night. > > Suggestions? > > Petr > > -- http://dfbsd.trackbsd.org.ar
Suggestion for hammer cleanup
I think we definitely must have hammer cleanup not depending on periodic. Especially laptop/workstation users, since they may be running only couple of hours a day. The system should be able to automatically initiate hammer cleanup whenever it's been more than 1 day/x hours since last time it was run. It doesn't matter if its day or night. Suggestions? Petr
