Re: [Bacula-users] multiple spool files per job
Hi, this feature would be really great. We have three Backup serves which all have single Jobs to backup (each job app. 13TB). The Time to despool is absolutely waste of time and could be used more efficient. This feature request is more than 4 Years old and still not implemented. Maybe I am wrong and I am sorry to say this, but as the data volume is growing all the time and this feature is still missing, I've got a notion that Bacula is not anymore up to date or state of the art. Regards, Stefan -Ursprüngliche Nachricht- Von: Alan Brown [mailto:a...@mssl.ucl.ac.uk] Gesendet: Montag, 10. Oktober 2011 13:00 An: James Harper Cc: bacula-users@lists.sourceforge.net Betreff: Re: [Bacula-users] multiple spool files per job James Harper wrote: Is there a way to make bacula write multiple spool files per job? Two would do. What I'm seeing is that 4 jobs start, all hit their spool limit around the same time, then all wait in a queue until the file is despooled. The despool happens fairly quickly (much quicker than the spooling due to network and server fd throughput) so it isn't a huge problem, but it would be better if the sd could just switch over to another spool file when despooling starts so that the backup can continue uninterrupted. I believe this is in the todo list, but it will happen faster if someone submits already-written code. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
On 13.10.2011 11:51, Stefan Lubitz wrote: Hi, this feature would be really great. We have three Backup serves which all have single Jobs to backup (each job app. 13TB). The Time to despool is absolutely waste of time and could be used more efficient. This feature request is more than 4 Years old and still not implemented. Maybe I am wrong and I am sorry to say this, but as the data volume is growing all the time and this feature is still missing, I've got a notion that Bacula is not anymore up to date or state of the art. as with any opensource tool you are free to send in code which enables the feature. or hire somebody to write it for you and the community. or buy the bacula enterprise version and tell them you really need the concurrent spooling/despooling feature. - Thomas James Harper wrote: Is there a way to make bacula write multiple spool files per job? Two would do. What I'm seeing is that 4 jobs start, all hit their spool limit around the same time, then all wait in a queue until the file is despooled. The despool happens fairly quickly (much quicker than the spooling due to network and server fd throughput) so it isn't a huge problem, but it would be better if the sd could just switch over to another spool file when despooling starts so that the backup can continue uninterrupted. I believe this is in the todo list, but it will happen faster if someone submits already-written code. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
Il 13/10/2011 11:51, Stefan Lubitz ha scritto: Hi, this feature would be really great. We have three Backup serves which all have single Jobs to backup (each job app. 13TB). The Time to despool is absolutely waste of time and could be used more efficient. This feature request is more than 4 Years old and still not implemented. Maybe I am wrong and I am sorry to say this, but as the data volume is growing all the time and this feature is still missing, I've got a notion that Bacula is not anymore up to date or state of the art. Regards, Stefan Hmmm... sounds like the perfect candidate for a paid enterprise feature request. I belive the creators of Bacula would be happy to hear from you. Just a hint... -- Marcello Romani -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
Marcello Romani wrote: Hmmm... sounds like the perfect candidate for a paid enterprise feature request. I belive the creators of Bacula would be happy to hear from you. I looked into this earlier this year. Bacula Enterprise subscriptions are several _thousand_ euro per year, which is simply too much for us (and I daresay most academic sites). I'm not sure why Kern has priced this out of the reach of most organisations who'd be willing to pay 500-900 euro/year. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
Il 13/10/2011 15:16, Alan Brown ha scritto: Marcello Romani wrote: Hmmm... sounds like the perfect candidate for a paid enterprise feature request. I belive the creators of Bacula would be happy to hear from you. I looked into this earlier this year. Bacula Enterprise subscriptions are several _thousand_ euro per year, which is simply too much for us (and I daresay most academic sites). I'm not sure why Kern has priced this out of the reach of most organisations who'd be willing to pay 500-900 euro/year. I've never looked closely at their pricing, since we're a small shop and are doing reasonably well on our own so far, but if the prices are all-or-nothing in the range of thousands of dollars per year, as you write, then yes, I agree with you there's a gap in their support offering. Or maybe there are other companies offering bacula support / consulting which are able to fill this gap (as I'd expect with an opensource enterprise-grade software)... (ok I'll stop here since I've got nothing more to add than my own speculations :-) -- Marcello Romani -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2d-oct ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
James Harper wrote: Is there a way to make bacula write multiple spool files per job? Two would do. What I'm seeing is that 4 jobs start, all hit their spool limit around the same time, then all wait in a queue until the file is despooled. The despool happens fairly quickly (much quicker than the spooling due to network and server fd throughput) so it isn't a huge problem, but it would be better if the sd could just switch over to another spool file when despooling starts so that the backup can continue uninterrupted. I believe this is in the todo list, but it will happen faster if someone submits already-written code. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] multiple spool files per job
Is there a way to make bacula write multiple spool files per job? Two would do. What I'm seeing is that 4 jobs start, all hit their spool limit around the same time, then all wait in a queue until the file is despooled. The despool happens fairly quickly (much quicker than the spooling due to network and server fd throughput) so it isn't a huge problem, but it would be better if the sd could just switch over to another spool file when despooling starts so that the backup can continue uninterrupted. I'm spooling to internal RAID, then despooling to external USB. While spooling isn't really advised when the backup target is a disk, doing it this way means I can run multiple jobs at once without causing interleaving in the backup file (single sd volume) or severe filesystem fragmentation (if one sd volume per job). Internal RAID writes at ~100MB/second while the USB disk writes at ~30MB/second so it turns out to be a pretty effective way to do what I want except that despooling is causing a bottleneck. Any suggestions? Thanks James -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2dcopy2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple spool files per job
James Harper schrieb: Is there a way to make bacula write multiple spool files per job? Two would do. What I'm seeing is that 4 jobs start, all hit their spool limit around the same time, then all wait in a queue until the file is despooled. The despool happens fairly quickly (much quicker than the spooling due to network and server fd throughput) so it isn't a huge problem, but it would be better if the sd could just switch over to another spool file when despooling starts so that the backup can continue uninterrupted. I'm spooling to internal RAID, then despooling to external USB. While spooling isn't really advised when the backup target is a disk, doing it this way means I can run multiple jobs at once without causing interleaving in the backup file (single sd volume) or severe filesystem fragmentation (if one sd volume per job). Internal RAID writes at ~100MB/second while the USB disk writes at ~30MB/second so it turns out to be a pretty effective way to do what I want except that despooling is causing a bottleneck. Any suggestions? No, this has been on the feature requests list for a while now. Spooling nearly doubles my time for large backups. http://thread.gmane.org/gmane.comp.sysutils.backup.bacula.devel/15351 Item 10: Concurrent spooling and despooling within a single job. http://www.bacula.org/git/cgit.cgi/bacula/plain/bacula/projects?h=Branch-5.1 Ralf -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2dcopy2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
