I work for a hosting company and for some time we have had issues with quotas not being properly enforceable.
After being somewhat sick of it I decided to patch our bacula to support quotas (3.0.2). Now, I am not a C coder by any means and dont expect what i've written to be good enough for general release. But I am testing this patch on a trial backup server and will be monitoring it for the next week. I am aware this is not the most up to date version of bacula and as such I dont expect this to be accepted. However I do intend on forward porting it but first I am needing some advise regarding the patch, how I am doing this and where and what problems I might come up against. Some of the things I would like pointers on - The placement of some of the functions and the files they are in. - The most appropriate spot to place functions and if I have put them in the right place. - Cosmetic and coding designs being incorrect and where to correct them. - How to use bregex to parse the jobbytes value I am getting back. - Any other criticisms. Again I do not consider myself a C programmer so am open to much criticism in this regard. I dont think I've made this production quality at all. How this works. The Client Resource can be updated with four configuration options: Hard Quota = <Y bytes> Soft Quota = <X bytes> Soft Quota Grace Period = <X period> Strict Quota = <yes|no> Hard quota: The limit that a job must be canceled at after over-running this value. Soft Quota: The limit for when bacula starts warning about quotas and initiating a grace period. Soft Quota Grace Period: The amount of time bacula will permit going over the soft quota. Strict Quota: This changes the behaviour of the soft quota enforcement. With this set to 'yes' after gracetime expires a client must be below his soft quota to continue to have backups. With this set to no the client is enforced up to the maximum quota they had just prior to the softquota gracetime expiring. How it works. Aside from the various function calls performed in two places and the DB functions the basic idea is this: A job runs. Hard / Soft quotas are checked. If over quota at this point the job cancels. This prevents a job running immediately and providing pointless timeouts. When the job starts, when the scheduling watchdog kicks in and the job is running a "status client" command is issued to the client. The jobbytes value parsed and the quota value determined by the sum of all previous jobs within the job retention period of that client plus the current jobs jobbytes value. If it goes over then the job is canceled. I altered the backup report slightly to report quotas, current quota amount and the quota limits per job run to make it easier to determine the current quota the user had assigned. Finally I added a new option for bconsole purge->quotas. This zeroes the values in the quota table for that row effectively resetting quotas. Database changes. I added a new table called Quota which contains the clients id, a softquota and gracetime value. These are used to enforce soft quotas when strict quota checking is off and to store the gracetime period. Its early days running this patch on our own systems at the moment but we expect this to reduce the number of failed jobs waiting to time out on our system by about 90%. Supplied is the patch and the sql dump. Things I would be keen on adding is a gracetime value to the job report (how much time left) but my crappy coding in time manipulation in C is holding me back. Also I want to change the jobbytes parser (kept in fs_cmds.c) to use bregex as I expect it will be massively less uglier and more successful at parsing the string I want. There is also some portions of the code I simply dont properly understand that well which I have more or less fudged from other functions. In particular is the jobbytes parser (again) regarding copying the jcr->client record in correctly. I am also interested in your feedback in regards to the idea that Im affectively running status client every minute for all running jobs that support quotas. Does this pose any long term problems? It was unavoidable to be honest but I think it probably does scale into the thousands still. Also I added then didnt use some variables. Bad practice to leave them in I know! I'll be removing these for proper production use. Again, I know this is bacula 3 and I know that wont impress you lot - it just makes things harder for you to review - but I will be happy to forward port this myself however rather to do this properly and to do it properly requires your input. NOTE: This version contains a openssl patch made to run in fedora. It doesnt affect compilation of redhat boxes but fedora wouldnt let me compile this without it!
bacula-quotas.patch.gz
Description: GNU Zip compressed data
-- MySQL dump 10.11 -- -- Host: localhost Database: bacula -- ------------------------------------------------------ -- Server version 5.0.77 /*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */; /*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */; /*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */; /*!40101 SET NAMES utf8 */; /*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */; /*!40103 SET TIME_ZONE='+00:00' */; /*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */; /*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */; /*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */; /*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */; -- -- Table structure for table `Quota` -- DROP TABLE IF EXISTS `Quota`; SET @saved_cs_client = @@character_set_client; SET character_set_client = utf8; CREATE TABLE `Quota` ( `ClientId` int(10) unsigned default NULL, `GraceTime` bigint(20) default '0', `QuotaLimit` bigint(20) unsigned default '0', `id` int(11) NOT NULL auto_increment, PRIMARY KEY (`id`) ) ENGINE=MyISAM AUTO_INCREMENT=23 DEFAULT CHARSET=latin1; SET character_set_client = @saved_cs_client; /*!40103 SET time_zo...@old_time_zone */; /*!40101 SET sql_mo...@old_sql_mode */; /*!40014 SET foreign_key_chec...@old_foreign_key_checks */; /*!40014 SET unique_chec...@old_unique_checks */; /*!40101 SET character_set_clie...@old_character_set_client */; /*!40101 SET character_set_resul...@old_character_set_results */; /*!40101 SET collation_connecti...@old_collation_connection */; /*!40111 SET sql_not...@old_sql_notes */; -- Dump completed on 2010-07-05 21:25:10
------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
