Re: [Bacula-users] Windows File Daemon maximum speed at 36 MByte/s
Zitat von Josh Fisher jfis...@pvct.com: On 11/29/2014 5:43 AM, Patrick wrote: Hi Josh, Thanks for your reply. You can try to isolate the problem. Try running a Windows backup with VSS, compression, and encryption all turned off. Compare a Windows VM against a Linux VM on the same host if possible. I don’t use compression or encryption, but VSS. I disabled VSS and get about 40 MByte/s. If I also set acl support = no, I get about 43 MByte/s. It’s better, but far away from the 90 MByte/s from a linux box with similar data. Last night I have took a look at the bacula console during the full backups and could see one Windows clients with 65 MByte/s. This server hosts a SQL Server with big database dump files. The Bacula File Daemon is faster with big files. How can I tune the Bacula File Daemon to increase the transfer rate for a lot of small files? Can I set the maximum memory usage? Is a ramdisk which collects the files from the hard drive and send it as bunches to bacula possible? You can change the MaximumNetworkBufferSize to 32k in both the client and SD configurations. Slowdowns of Windows clients have been noted before for 64k+ buffer sizes. You might also consider tweaking the NTFS filesystem on the Windows client. The filesystem parameters are in HKLM/system/CurrentControlSet/Control/FileSystem. Setting the DWORD value NtfsDisableLastAccessUpdate to nonzero will prevent a write to the disk each time a file is accessed. It is equivalent to the noatime option of ext4, so can definitely impact jobs like bacula-fd that are reading lots of files. Technical details in case of NTFS can be found here for example: http://technet.microsoft.com/de-de/library/bb457112%28en-us%29.aspx Regards Andreas -- Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=157005751iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Optimize backup/database
Zitat von bdelagree bacula-fo...@backupcentral.com: This post follows one of my old posts (see : http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/bacula-25/slow-backup-how-to-optimize-122683/index-15.html) My backups are still long . After listening to the network and servers I think my problem comes from the database. After much research some say migrate from MySQL to PostgreSQL . Before doing so I need your opinion , is there a real added value? You should first check how long it actually takes to insert the attributes in the database. With this information you can decide if you are optimizing the right point. For example our biggest job has around 1.1TB with 4 mio files which grow to around 1.2G attributes per job according to the logs. This job finish within around 32 hours but inserting the attributes only takes ~3 minutes. We have additional overhead per file and overall throughput because using encryption and Windows isn't that fast either. So what is the time it takes to insert the attributes? What is the transfer speed you get? Do you use data/attribute spooling as you should? What is the transfer speed to tape (LTO-5)? Without carefully measuring this values no one could say if you got a real value from switching to PostgreSQL. Regards Andreas -- Time is money. Stop wasting it! Get your web API in 5 minutes. www.restlet.com/download http://p.sf.net/sfu/restlet ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Is Windows still supported.
Zitat von Christian-Josef Schrattenthaler cjs1...@me.com: Hi! I am planning to install a backupserver and Bacula seems to be a cool solution. But is there still support for Windows? I read an article which says that Windows support will be stopped, but I can't find any information about this or about Windows on the Bacula website... Thanks, Christian. Windows (as client) is still supported, but you have to buy newer Windows Binaries from Bacula Systems : http://www.baculasystems.com/windows-binaries-for-bacula-community-users Regards Andreas -- ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] UPDATE - ERROR Spooling/Backups with large amounts of data from windows server 2012
Zitat von Hans Thueminger bac...@ipf.tuwien.ac.at: Dear Bacula Users! After another two months of testing I'm able to make an update: Hans Thueminger wrote, On 20.11.2013 16:58: Dear Bacula Users! I'm a newbie and have been trying to configure a Bacula backup solution for our Institute for two months. I'm excited of this Software (thanks to the developers for that great work) and I absolutely want to use it :-) At the moment, unfortunately I can't use it as I would like. At the moment it looks very good :-) We have to backup a Fileserver (Windows-Server-2012, Bacula client 5.2.10) with 400TB of Data with an Autochanger (IBM-TS3500) with two Drives (TS1140). We operate a dedicated Backupserver (CentOS 6.4) with Bacula (5.2.13) which is connectetd with a dedicated 1 GBit Interface to the Fileserver. The two drives of the autochanger are directly connected to a QLogic QLE2562 Dual Port Fibre Channel HBA (on the backupserver). To be ensured, the drives are operated in streaming mode, the backupserver has a 35TB spool area. There are several filesystems connected to the fileserver: 3x120TB 4x40TB As Andreas (lst_ho...@kwsoft.de) suggested, we made some hardware changes of the Backupserver. Now for each Drive is a dedicated 1 TB Spooldrive (two SSDs RAID 0) available . The first thing we have to do is to split the 3x120TB in 6x60TB since Microsoft does not support VSS for filesystems (64TB-8GB) not even with the latest release (Server-2012-R2). This is a not documented design and we got this information from the Microsoft support team. To re-emphasize this, in Windows-Server-2012 it's not a problem to create and use a filesystem 64TB but it's not possible to make a backup of any file of any size in such a filesystem; not even with the in the operating system included Windows-Server-Backup program :-( With the trick to use a mount point, instead of a drive letter, it's no problem to start a backup of the 3x120TB Filesystems (the mount point is located on a smaller disk drive and with this disk drive the VSS is not a problem). Of course there is no snapshot created from the open files of the mounted filesystem. But better a backup with some missing files instead of no backup! Now I'm glad to make backups and restores of files of our 4x40TB Filesystems with Bacula if they are not too big ( 5TB). That works fine. If they are too big ( 15TB) I always get an error after creating the second or third or sometimes subsequent spoolfile (Error: lib/bsock.c...). Never for the first spoolfile! I've tried several spool sizes (from 500GB to 16TB) and different network settings. As attachment (bacula_mailing_list_some_error_logs.txt) you can find some logs, when the error occurred. What I have also tried: - using different Networkinterfaces (at the moment an ethernet cable is directly connected (no switch) between the fileserver and the backupserver and this connection is used for the backups (checked with netstat)) - heartbeat: enabling on the SD (60 seconds) and net.ipv4.tcp_keepalive_time also set to 60 - AllowCompression Yes/No - many, many hours for trials As Tom (thom...@mtl.mit.edu) considered me, I corrected the configuration of the heartbeat and checked it with wireshark and now I'm sure it works. What else did I do? After running a backup for 6 days I realized that there is watchdog counter which stopps the backup after this time :-( After that reconfiguration and a new compilation I started a Backup of a 86 TByte filesystem and now I'm quite proud to tell you, that it worked :-) After 21 days running I have this data on 14 tapes. And I also restored some of the directories, everything worked fine. It would be interesting to hear what speed you get for despooling to tape with the TS1140 Drives? Maybe you should get a 10GB path between your server and the backup server, but biggest problem is that Windows support for concurrent jobs isn't that great, so your spooling stops as long as your despooling to tape :-( But fine that you finally got it running, the long lasting, data heavy TCP connections used by Bacula reveal problems in corners where you don't expect it anymore (NIC/Switches/Packet-Filters). Regards Andreas -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Supports Windos Server and SAP Application backup or not...!!!
Zitat von sh...@palincorporates.com: Hi Team, I have configured Bacula 5.2.12 on RHEL 6.4 some time back and it is working fine. Now I want to configured this Solution to take backup of some clients which are already in production. Please confirm me this is possible to take backup of these clients or not using Bacula 5.2.12/13 server installed on RHEL6.4 Clients Applications Windows 2003 Server Enterprise 32 bit Windows 2003 Server Enterprise 64 bit SQL server 2008 (ERP process) Windows 2003 Server Standard Edition 32 bit Windows 2003 Server Enterprise Edition 64 bit SQL server 2010 SUSE Linux 10 DB6/SAP Application Windows XP Windows 2003 Server Enterprise 32 bit Windows 2003 Server Enterprise 64 bit Oracle DB On Windows OS Bacula uses VSS (Volume Shadow Copy) to backup open files in a save way. If your Application support VSS (MS SQL Server does for sure) your are fine for full backups, incremental backups of for example a database are not possible this way. Please also tell me open source bacula client is available for all above systems or not. There are Windows Client binaries to download for a fee: http://www.baculasystems.com/windows-binaries-for-bacula-community-users Regards Andreas -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Supports Windos Server and SAP Application backup or not...!!!
Zitat von compdoc comp...@hotrodpc.com: Please also tell me open source bacula client is available for all above systems or not. Having to pay and then getting only 3 months of updates seems a bad thing to do to people. These clients are free: http://sourceforge.net/projects/bacula/files/Win32_64/ They once where free, now they are outdated and will be more as time goes by... Regards Andreas -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] unsubscribe
Zitat von r.schuitema...@kpn.com: UNSUBSCRIBE List-Unsubscribe: https://lists.sourceforge.net/lists/listinfo/bacula-users, mailto:bacula-users-requ...@lists.sourceforge.net?subject=unsubscribe -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] ERROR Spooling/Backups with large amounts of data from windows server 2012
Zitat von Hans Thueminger bac...@ipf.tuwien.ac.at: Dear Bacula Users! I'm a newbie and have been trying to configure a Bacula backup solution for our Institute for two months. I'm excited of this Software (thanks to the developers for that great work) and I absolutely want to use it :-) At the moment, unfortunately I can't use it as I would like. We have to backup a Fileserver (Windows-Server-2012, Bacula client 5.2.10) with 400TB of Data with an Autochanger (IBM-TS3500) with two Drives (TS1140). We operate a dedicated Backupserver (CentOS 6.4) with Bacula (5.2.13) which is connectetd with a dedicated 1 GBit Interface to the Fileserver. The two drives of the autochanger are directly connected to a QLogic QLE2562 Dual Port Fibre Channel HBA (on the backupserver). To be ensured, the drives are operated in streaming mode, the backupserver has a 35TB spool area. There are several filesystems connected to the fileserver: 3x120TB 4x40TB Looks like a decent invest in hardware ;-) The first thing we have to do is to split the 3x120TB in 6x60TB since Microsoft does not support VSS for filesystems (64TB-8GB) not even with the latest release (Server-2012-R2). This is a not documented design and we got this information from the Microsoft support team. To re-emphasize this, in Windows-Server-2012 it's not a problem to create and use a filesystem 64TB but it's not possible to make a backup of any file of any size in such a filesystem; not even with the in the operating system included Windows-Server-Backup program :-( With up to Windows 2008 R2 the supported volume size was 16TB with Windows 2012 it is 64TB. Note that their are other constraints with VSS when used with volumes containing many files or having heavy load while doing snapshots. That said you should always be able to backup without VSS, but open files get you in trouble in this case. Now I'm glad to make backups and restores of files of our 4x40TB Filesystems with Bacula if they are not too big ( 5TB). That works fine. If they are too big ( 15TB) I always get an error after creating the second or third or sometimes subsequent spoolfile (Error: lib/bsock.c...). Never for the first spoolfile! I've tried several spool sizes (from 500GB to 16TB) and different network settings. As attachment (bacula_mailing_list_some_error_logs.txt) you can find some logs, when the error occurred. What I have also tried: - using different Networkinterfaces (at the moment an ethernet cable is directly connected (no switch) between the fileserver and the backupserver and this connection is used for the backups (checked with netstat)) - heartbeat: enabling on the SD (60 seconds) and net.ipv4.tcp_keepalive_time also set to 60 - AllowCompression Yes/No - many, many hours for trials So you really have files with 15TB in size? What would be worth a try is the following: - Increase the maximum file size for the tape drive. The default is 1G and it limits how big the blocks on tape are between EOF markers. Maybe the counter per file is an int and you therefore have trouble with 15TB files? - You should increase the default block size written to tape with maximum block size set to for example 2M. Warning: You could not read already written tapes with non matching block sizes. - The spool area doesn't need to be that big, but really fast to saturate the tape drive and keep them streaming. Recommended is something like fast SSD or similar. Is there anybody out there who has a similar environment which is working or can point me in the right direction? I'd behappy for anyfurther questions or suggestions since at the moment I'm at a loss... We are way smaller in file size, so use the suggested as theoretical advice... Regards Andreas -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] ERROR Spooling/Backups with large amounts of data from windows server 2012
Zitat von Hans Thueminger bac...@ipf.tuwien.ac.at: lst_ho...@kwsoft.de wrote, On 21.11.2013 10:07: [...] With up to Windows 2008 R2 the supported volume size was 16TB with Windows 2012 it is 64TB. Note that their are other constraints with VSS when used with volumes containing many files or having heavy load while doing snapshots. That said you should always be able to backup without VSS, but open files get you in trouble in this case. I didn't find a way to make a backup without VSS, neither with the in the operating system included Windows-Server-Backup program nor with Bacula. But while writing this sentence I had an idea: What is happening, if backing up a mount point instead of a drive letter? Now I've just mounted the 120TB Filesystems as a mount point in C: (which is a 300GB filesystem) and look at here: 21-Nov 12:49 bacula-sd JobId 292: Spooling data ... 21-Nov 12:49 fs2-fd JobId 292: Generate VSS snapshots. Driver=Win64 VSS, Drive(s)=C and the status says: JobId 292 Job fs2-PHOTO.2013-11-21_12.46.00_11 is running. VSS Full Backup Job started: 21-Nov-13 12:47 Files=31,223 Bytes=308,963,855,104 Bytes/sec=89,192,798 Errors=0 Files Examined=31,223 Processing file: C:/PHOTO/Projects/PHOTO-Projects/09_ALS_Kaernten/GailtalLatschur/Dif-Gailtal-Latschur/s577_s592_p02.tif Projects is the mountpoint for the 120TB filesystem! By now trying to backup G:/PHOTO-Projects/ (which is the same 120TB filesyste as above), I always received the following error: 17-Sep 15:54 fs2-fd JobId 36: Generate VSS snapshots. Driver=Win64 VSS, Drive(s)=G 17-Sep 15:55 fs2-fd JobId 36: Fatal error: CreateSGenerate VSS snapshots failed. ERR=The operation completed successfully. It seems, that this is a way to trick bacula and windows :-) Of course with this workaround we still have the problems with open files, but the actual problem which I want to discuss with you is the error I receive after creating the second or third or sometimes subsequent spoolfile: Uhm, no. The idea was to set enable vss = no, but as said this only works some sort of if the volume in question does not have open files. Now I'm glad to make backups and restores of files of our 4x40TB Filesystems with Bacula if they are not too big ( 5TB). That works fine. If they are too big ( 15TB) I always get an error after creating the second or third or sometimes subsequent spoolfile (Error: lib/bsock.c...). Never for the first spoolfile! I've tried several spool sizes (from 500GB to 16TB) and different network settings. As attachment (bacula_mailing_list_some_error_logs.txt) you can find some logs, when the error occurred. What I have also tried: - using different Networkinterfaces (at the moment an ethernet cable is directly connected (no switch) between the fileserver and the backupserver and this connection is used for the backups (checked with netstat)) - heartbeat: enabling on the SD (60 seconds) and net.ipv4.tcp_keepalive_time also set to 60 - AllowCompression Yes/No - many, many hours for trials So you really have files with 15TB in size? What would be worth a try is the following: Sorry, that was badly written of me. It's not the size of one file, it's the size of all files. So what I wanted to write was: if the amount of files to be backuped is not too large ( 5TB) it works. If the amount of the files to be backuped ist larger than 15TB it fails always! This might point to another problem case. We had a similar problem on our main filer with backup until around 2TB succeeded, anything above had a ~50% to fail with network errors. We switched the NIC and use some Intel PlugIn card and the problem went away. - Increase the maximum file size for the tape drive. The default is 1G and it limits how big the blocks on tape are between EOF markers. Maybe the counter per file is an int and you therefore have trouble with 15TB files? I guess the spool file is written as one file to the tape which would mean, that for every spool file only one EOF marker would be written? Can you confirm that, or are I'm wrong? What would you suggest to set for maximum file size? No, the spool file is only to decouple the streaming to tape from network and client speed and delays. There is not much buffering when writing to tape so the spool area need to be able to constantly deliever data faster tahn the tape can consume. It is still written with the maximum block size per transfer and a EOF marker every maximum file size to tape. - You should increase the default block size written to tape with maximum block size set to for example 2M. Warning: You could not read already written tapes with non matching block sizes. Ok and thank you for the warning, that's rather good to know! - The spool area doesn't need to be that big, but really fast to saturate the tape drive and keep them streaming. Recommended is something like
Re: [Bacula-users] Fwd: Spooling attrs takes forever
Zitat von Charles Douglass chas.dougl...@gmail.com: On 11/13/2013 09:01 AM, Phil Stracchino wrote: On 11/13/13 10:59, Charles Douglass wrote: Thanks for the reply. I know nothing about MySQL optimization. I downloaded the mysqltuner.pl script and I will run it after the next full backup to see if it has any recommendations. The mysqltuner script is a pretty decent basic tool. The thing you need to keep in mind regarding the mysql-large, mysql-huge sample configurations that are STILL widely distributed in MySQL packages (and still widely used by the unsuspecting) is that most of those sample configurations were originally written back when a large server was one that might have as much as a whole 32MB of RAM. These days, that is less than the compiled-in default size of some individual MySQL *buffers*. So the unwary install the suggested configurations, and can't understand why MySQL is performing like a geriatric tortoise. I ran mysqltuner before the backup and increased a couple of buffer sizes as well as taking this suggestion: innodb_flush_log_at_trx_commit=0 Then I ran a full backup and see this: 18-Nov 12:45 maple-sd JobId 46: Sending spooled attrs to the Director. Despooling 153,470,245 bytes ... 18-Nov 21:41 maple-dir JobId 46: Bacula maple-dir 5.2.6 (21Feb12): Build OS: x86_64-pc-linux-gnu ubuntu 13.04 So spooling/despooling took just about 9 hours. Even for MySQL on a not too fast machine this is ridiculus slow. We despool around 1.2GB attributes within around 2 minutes to our Postgres server. Something is still wrong with your setup. Have you checked with tools like iotop,top,vmstat what the server is actually doing during the 9 hours. Regards Andreas -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow backup, how to optimize ?
Zitat von bdelagree bacula-fo...@backupcentral.com:
Hello,
Thank you for the quick response.
My library is connected to a dedicated server only to services (PDC,
DHCP, DNS, LDAP, and Bacula)
This server is not designed to host files, so he has little space.
In addition, the MySql database is already 35Gb...
I can dedicate reasonably 50Gb on this one.
The size of the database mostly depend on the number of
files/directories and your retention policy.
Is there one DataSpooling for bacula, or one by drive ?
You should set it by drive and calculate for the number of parallel
jobs you are using.
Knowing I have a lot of servers to backup (8 servers for about 2.8
Tb of data), is that enough or i need to find another system for
Data Spooling?
df -h from my srv-infra-sm
Filesystem Size Used
Avail Use% Mounted on
rootfs 20G 836M
20G 5% /
udev 10M 0
10M 0% /dev
tmpfs 1.6G 1.7M
1.6G 1% /run
/dev/disk/by-uuid/465b04fb-de46-409b-928a-ec01ba98373e 20G 836M
20G 5% /
tmpfs 5.0M 4.0K
5.0M 1% /run/lock
tmpfs 1.6G 8.0K
1.6G 1% /run/shm
/dev/sda4 109G 40G
69G 37% /var
tmpfs 7.9G 0
7.9G 0% /tmp
New options in bacula-sd.conf (thinking that we need a spool by drive)
Device {
Name = Drive-0
.
.
.
Maximum Spool Size = 24gb
Maximum Job Spool Size = 12gb
Spool Directory = /var/lib/bacula/spool/drive0
}
Device {
Name = Drive-1
.
.
.
Maximum Spool Size = 24gb
Maximum Job Spool Size = 12gb
Spool Directory = /var/lib/bacula/spool/drive0
}
If you backup all 8 machines concurrently you should set your Job
Spool Size to something around available diskspace / 8. Also be
aware that you need additional spool space for spooling attributes
which is enabled by default when using data spooling. Think of data
spooling as some form of cache to pack things together before
committing to database and tape.
Regards
Andreas
--
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow backup, how to optimize ?
Zitat von bdelagree bacula-fo...@backupcentral.com: Hello, This summer we invested in a PowerVault TL2000 library with two LTO5 drives to safeguard our various servers. Today two of my servers take to save a lot because they contain many small files for low volume (see the bottom of post) All my other servers backups quickly (20,000 KB/s to 30,000 KB/s) As explained in the documentation for Bacula I added the following option in the StorageDaemon and FileDaemon of these servers: Maximum Network Buffer Size = 65536 But that did not change anything ... Did I forget something? Something wrong? There's an other options that I have not seen? Be sure to use attribute spooling and if you have some fast local storage at the backup server data spooling. http://www.bacula.org/5.2.x-manuals/en/main/main/Data_Spooling.html Regards Andreas -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] choosing database.
Zitat von Mauro mrsan...@gmail.com: On 19 September 2013 21:53, lst_ho...@kwsoft.de wrote: Zitat von Konstantin Khomoutov flatw...@users.sourceforge.net: Our biggest single job has about 1TB data with some 3.3 million files. This lead to around 1GB spooled attributes which will be absorbed in less then 2 minutes. Database is Postgres with some minimal tuning: shared_buffers = 128MB temp_buffers = 32MB work_mem = 32MB maintenance_work_mem = 64MB checkpoint_segments = 32 checkpoint_completion_target = 0.9 That's all Database hovers around 21GB in size with with some 57 million rows in the file table. how much ram you have in your server? The whole server has 8GB, but that only needed for Bacula director in case of restoring some million files ;-) Regards Andreas -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] choosing database.
Zitat von Konstantin Khomoutov flatw...@users.sourceforge.net: On Thu, 19 Sep 2013 17:06:48 +0200 Mauro mrsan...@gmail.com wrote: Hello. I'm using bacula in a linux debian system. I've to backup about 30 hosts. I've choose postresql as database. What do you think about? Better mysql or postgres? Unless you will have directories with unusually high quantities of files on those hosts (for instance, backup directories maintained by rsnapshot, huge mail spools etc) both should perform well. The basic idea is that each (full) filename backed up makes up one record in one of the tables of the Bacula catalog, plus each unique pathname makes up one record in its another table. Consequently, when you have *huge* number of distinct files in a backup job, this might kneel your database. I've personally seen despooling of attributes for some 800k files taking more than 24 hours with a reasonably tuned (for high-memory) MySQL instance using the MyISAM storage backend. I'm not sure would Postrges beat MySQL in this situation, and if yes then by which margin. In this particular case I've just switched to using the bpipe plugin for backing up that particular fileset -- simply because restoring individual files from it makes little sense anyway. Our biggest single job has about 1TB data with some 3.3 million files. This lead to around 1GB spooled attributes which will be absorbed in less then 2 minutes. Database is Postgres with some minimal tuning: shared_buffers = 128MB temp_buffers = 32MB work_mem = 32MB maintenance_work_mem = 64MB checkpoint_segments = 32 checkpoint_completion_target = 0.9 That's all Database hovers around 21GB in size with with some 57 million rows in the file table. If you are familiar with Postgres there is no reason to not use it. Regards Andreas -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Building directory tree for JobId(s) very slow
Zitat von Dimitri Maziuk dmaz...@bmrb.wisc.edu: On 08/14/2013 01:13 PM, azurIt wrote: Where's my mistake? Could you suggest me some optimization. Hi, only one little suggestion from me - upgrade to MySQL 5.5. I didn't have much luck with postgres on a single-cpu (I think 6-core or maybe 4) and 8 or 12GB RAM (it's off at the moment can't find the printed specs). Our current server for big backups is dual quad-core xeon with 48GB RAM and hardware raid, running postgres 8.4 -- *that* doesn't take hours to run the queries. Our test machine with a single SATA need around 5 minutes for building the tree for a 4.2 million file backup. The main machine with SSD is somewhat faster. Both run Bacula with PostgreSQL 9.1 and only tuned five parameters in postgresql.conf. Not sure how you get hours for the file tree in bconsole. If you are using Bat and bvfs that would be another story. Regards Andreas -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Unneeded database index
Zitat von azurIt azu...@pobox.sk: Hi, i'm having some MySQL performance difficulties so i started to search what can i do better. My table 'File' had these indexes created: CREATE INDEX file_jobid_idx on File (JobId); CREATE INDEX file_jpf_idx on File (JobId, PathId, FilenameId); Which looks correct according to documentation: http://www.bacula.org/en/dev-manual/main/main/Catalog_Maintenance.html BUT! The first index apperas to be unneeded as it's part of the second index. According to MySQL documentation, 'any leftmost prefix of the index can be used by the optimizer to find row': http://dev.mysql.com/doc/refman/5.5/en/multiple-column-indexes.html I suggest to remove it. azur Would also apply to PostgreSQL i guess: http://www.postgresql.org/docs/9.2/interactive/indexes-multicolumn.html Any comment why the additional single column index could be useful? Regards Andreas -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-dir run failure on Ubuntu 12.04.02
Zitat von uhog-v...@spamex.com: Hi folks, For what it's worth, I tried compiling Bacula 5.0.3 on Ubuntu 12.0.4 today. I currently have Bacula 5.0.3 running on Ubuntu 10.0.4 on another server. The compilation failed on 12.0.4 for the director though the SD and FD both compiled. I played around a bit but then decided just for kicks to reinstall this new machine back to 10.0.4 and try that. After that, I did an update, and then loaded the build-essential, libncurses5-dev, and libmysqlclient-dev packages. I then built 5.2.13 and it builds and runs with no problem. So whatever is going on must be related to something happening with Ubuntu packaging at 12.0.4 that causes Bacula to not link in everything it needs. Since my primary goal is to get moved to different hardware, I'm just going to continue running on 10.0.4 on the new hardware with the 5.2.13 Bacula since 10.0.4 is supported until midway through 2015. After I get everything migrated and stabilized, maybe I'll play with debugging this 12.0.4 problem again in a virtual machine or something in my spare time. Thanks again for the ideas, I learned a bit more about digging around in the symbols on linux so it was worth that anyway. If I do get back to it and find a solid answer, I'll update the list. Mike Some time ago i build 5.2.13 the Debian/Ubuntu way as .deb packages on Ubuntu 12.04. I can provide you a link for download without any warranty of course. Maybe i can even dig out the thing which have to be done to get the deb compiled. Regards Andreas -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula security
Zitat von Grant emailgr...@gmail.com: I'm currently pushing backups from each system to a central backup server via rdiff-backup. However, I realized that push backups are not safe because if one of the systems is compromised, the infiltrator could delete all of that system's backups with a command like this: rdiff-backup --remove-older-than 1s backup@12.34.56.78::/path/to/backup The Bacula client can't delete its own backups, so it is safe against that problem. Pull backups don't seem secure because if the central server is compromised, the infiltrator would have root read privileges on every backed-up system and would thereby be able to gain root access to those systems. Bacula does have root read (and write) privileges on every backed-up system, but you can encrypt the backups before sending them to the central server. Bacula can also sign the backups, so the client can verify that a restore doesn't contain modified data. You still have to keep the encryption/signing keys secure of course. Thanks for your help. I don't think I have the b*lls to give root read/write on every system to the backup server. :) - Grant You are free to operate the FD (Client) with any permission you like, but you have to take care that the FD is able to read anything you like to backup and i case of restore it should be able to write and maybe to chown the files in question. Regards Andreas -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Too much files to save in catalog - MysQL performance tips.
Zitat von Luis H. Forchesatto luisforchesa...@gmail.com: Greetings. I'd like to discuss my situation here where I have a job who backups only 200GB in files, but it has 2 million more files to save on catalog (MySQL). When the file copy is complete the storage server, who also runs the director, spend many hours to save the files in the catalog. This also cause low mysqld performance at all. Is there any tip to optimize the catalog operation or MySQL performance to make the job less resource hungry? The job runs once a week with no concurrent jobs. Director and storage runs at the same computer but the client is another server, at the same network who can transport up to 1Gb of data between the servers. Any tips will be apretiated. Maybe the wrong database? Our main filer has ~2.7 million files per full backup which pile up to ~1GB attribute data to go in the backup database (PostgreSQL). It finish to despool the attributes within 23 minutes on a commodity hardware. If you have to stick with MySQL use Bacula attribute spooling feature and tune MySQL for fastest insert with something along these general tips: - Raise the size of the redologs - Enlarge the buffers - Use a really fast dedicated i/o channel for the db Some further reading is here: http://stackoverflow.com/questions/7585301/how-tos-for-mysql-innodb-insert-performance-optimization Regards Andreas -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Recommended hardware
Zitat von Francisco Garcia Perez fga...@gmail.com: Hello, I have a PowerVault 124T, but I want buy a new backup system with support for my old LTO-2 and LTO-3 tapes with at least 24 slots, 2 drives, a barcode scanner. What do you recommend me? If you really need support for reading your old LTO-2 tapes you are limited to LTO-4 because the read compatibility is maintained two steps down as far as i know. Instead of this you should rather use at least LTO-5 today and copy your data still needed from the old LTO tapes. As of brands recommended for tape libraries: Most of the mid-sized libraries (HP, IBM, Dell) are rebranded BDTs which work well with Bacula. For the tape drives some say that full-height are more solid than the half-height, but i don't have first hand experience for this. Regards Andreas -- Get 100% visibility into Java/.NET code with AppDynamics Lite It's a free troubleshooting tool designed for production Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap2 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Migrating to Newer Version of PGSQL
Zitat von Doug Sampson do...@dawnsign.com: Questions: 1) Do I actually need to remove the bacula packages? From Bacula point of view, no. But your package system might force you because of dependencies. If you compile from source it will be better to link Bacula against the actual used PostgreSQL libs. 2) Are there any special steps that I'm missing when configuring the bacula databases? Do I need to run 'update-bacula-tables' somewhere in there? The update-bacula-tables is only needed for Bacula updates. Database upgrades has no need to later the data/schema used by Bacula. 3) Is PGSQL 9.2 considered stable? 9.1 had some serious bugs but these appeared to have been squashed. There should be no big difference because 9.2 is a minor release. In general PostgreSQL is one of the more reliable databases. 4) Are there any special parameters I need to consider when going from PGSQL 8.3 to 9.2 especially in the postgresql.conf? Tune for inserts: http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server 5) Any other caveats? You might run Baculas dbcheck to prune unused entries in the database before migration. 6) pg_upgrade: I came across this new function. Apparently you can upgrade databases from 8.3.x to 9.2.x with some caveats as described in the Limitations in Upgrading from PostgreSQL 8.3 section of this link http://www.postgresql.org/docs/9.2/static/pgupgrade.html. If you've used this, would you care to share your experiences? No, never used, but in general the PostgreSQL database and tools simply does what is expected, so it should just work. Regards Andreas -- Own the Future-Intelreg; Level Up Game Demo Contest 2013 Rise to greatness in Intel's independent game demo contest. Compete for recognition, cash, and the chance to get your game on Steam. $5K grand prize plus 10 genre and skill prizes. Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with concurrent mixed priority jobs
Zitat von stefano scotti scottistefan...@gmail.com: Hi, My director allows 5 concurrent jobs, and each job has *Allow Mixed Priority * set to yes. My aim is that even if a slow job takes a lot of time to finish, there will be other 4 job slots to use so that the backup system can still work despite of a really really slow job. So i can have 4 slow jobs and my system will keep scheduling fast jobs anyway. Now, the problem is that only jobs with an higher priority will be scheduled, lower priority jobs still have to wait even if there are 4 free job slots! Is it possible to instruct bacula to let lower priority job be scheduled not worrying about the priority of current scheduled jobs but only on the number of free job slots? Hello, Maybe a misunderstanding on my side but you could simply set all jobs to the same priority if you only care about free slots... Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula version 5.2.13 released
Zitat von kern bacula-fo...@backupcentral.com: Andreas, Although Qt is the best GUI from a programmer's point of view that I have ever used, it is terribly non-compatible between versions, and it is sufficiently complicated that it still has a significant number of stability (bug) issues. Thus we try to keep bat on the most stable and most recent release possible. If you go to the Qt site (http://qt-project.org/downloads) you will see that Qt 4.8.4 is the most recent stable version within the 4.x series. We are not yet ready to try 5.0. Previously we did not force Bacula to compile with the version we are developing/testing/using and we had nothing but support problems. So now, the Bacula bat source code requires the version we are using. You can modify it if you want, but you will probably end up with a less stable bat. I am using Ubuntu 12.04 as my development system, but like you I would not consider replacing the Qt on the system. This is why we produce a depkgs-qt package (see source forge download), which has Qt 4.8.4. You simply build it, and setup the appropriate environment variables (in a script) and then build bat with it. The day that Qt becomes stable and all the distros have some stable compatible version will be a big welcome for us, but we are not yet there. I believe that all this is documented in our manual. Regards, Kern Hm, ok but in my case trying to build Bacula deb-packages with the Ubuntu build environment (debuild) does not work out of the box because it fails including binary/pre-compiled qt-libs :-( For now i have resorted to lower the required version to 4.8.1 as delivered by Ubuntu and hope they will backport important fixes. This works fine beside some obscure document kernstodos which is listed somewhere as needed documentation but is not found :-) With kernstodos deleted in the bacula-common.docs list all went fine. BTW: I see no mention of Bug#1936 so i guess a fix is not yet included?? Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula version 5.2.13 released
Zitat von Kern Sibbald k...@sibbald.com: Hello, Bacula Community Release: This is to let you know that we have released Bacula version 5.2.13 to Source Forge. It is also in the git repository. You will find a summary of the release, which has 20 bug fixes at the bottom of this email. Important changes since last release: [ReleaseNotes] IQ 19Feb13 - Fix build/configure problems with bpluginfo.c - Refactor lock_volumes so most lock a vol rather than globally - Add virtualfull-extreme test - Apply patch for chio-changer-openbsd from bug #1984 -- Implements li - Add bat Mac patch from bug #1953 - Fix bug #1812 cannot run Copy/Migrate jobs from bat - Fix loading of bat translations. Patch from bug #1890 - Fix text input in bat. Fixe bug #1965 - Remove Device from show command as it is not used or updated - Fix #1982 update enabled keyword in help command - Fix update_postgresql_tables 10 to 11 - Clean after building doc binaries requested by Willem vd Akker for D build - Require Qt version 4.8.4 Hm, Ubuntu 12.04 ships with Qt version 4.8.1 and i'm for sure not fond of replacing it by source compiled Qt version. What is the reason for 4.8.4 instead of 4.8.0?? Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula version 5.2.13 released
Zitat von Simone Caronni negativ...@gmail.com: On 21 February 2013 10:32, lst_ho...@kwsoft.de wrote: Hm, Ubuntu 12.04 ships with Qt version 4.8.1 and i'm for sure not fond of replacing it by source compiled Qt version. What is the reason for 4.8.4 instead of 4.8.0?? Bug fixing on the qt side, probably. The minimum version required by the code to compile succesfully is qt 4.6.2. Bacula for Fedora/RHEL is patched this way to accept a lower version that is shipped in RHEL 6 as system libraries and works fine. We're lowering the requirement since June 2012 and so far we did not face any issue. http://pkgs.fedoraproject.org/cgit/bacula.git/plain/bacula-5.2.13-bat-build.patch Regards, --Simone Thanks, will try that road. Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] backup concurrency
Zitat von Antony Mayi antonym...@yahoo.com: From: Jérôme Blion jerome.bl...@free.fr To: bacula-users@lists.sourceforge.net Sent: Wednesday, 20 February 2013, 9:41 Subject: Re: [Bacula-users] backup concurrency Le 2013-02-20 10:19, Antony Mayi a écrit : Hi community, I am running backups of multiple database servers and each backup job is defined with RunScript command that dumps the databases on each particular server and these dumps are then taken for backups. The dumps take different times on each server - minutes to couple of hours. Since the dumping doesn't involve bacula storage daemon I want another jobs to be running in parallel that can be sending data to storage while the db servers are dumping the databases. Also the database dumps can be running in parallel instead of sequentially as that's purely local matter. I've increased the Maximum Concurrent Jobs in Director resource but I can still see only one job running at a time. I don't think I want to increase this as well in Storage resource and definitely not in the Job resource. What am I missing here? ...running bacula 5.2 thx, Antony. Hello, What is the state of the other jobs while one is running? How did you set up your Maximum Concurrent jobs? good point. Running Jobs: Console connected at 20-Feb-13 11:02 JobId Level Name Status == 45 Full DB1Backup.2013-02-19_23.05.00_21 is running 46 Full CatalogBackup.2013-02-19_23.10.00_22 is waiting for higher priority jobs to finish 47 Increme Srv1Backup.2013-02-20_11.02.40_26 is waiting on max Storage jobs 48 Increme Srv2Backup.2013-02-20_11.02.45_27 is waiting on max Storage jobs so it seems the DB1Backup is blocking the storage although not using it really since it is busy with running the RunScript db dump which takes in this case about 4 hours. so that suggests I need to increase the concurrency for accessing the Storage which I wanted to avoid to not to interleave writes from multiple Jobs. I find it quite inefficient blocking the storage resource for several hours although not really using it. Is there a way around it or is the only approach enabling concurrent access to the Storage (which is discouraged from restoration performance point of view)? thx, Antony. If you want to avoid concurrent (interleaved) write to tape by multiple jobs have a look at the data spooling feature. With this you split up your jobs in configurable chunks which will be written to tape non-interleaved. Your second possibility is to limit the actual tape device to 1 job, but not the storage daemon (not sure if this works though). Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] FW: difficulty reading tapes after upgrade 3.0.2 - 5.2.12
Zitat von GILES Stephen stephen.gi...@ed.ac.uk:
Hello,
Is anyone able to suggest anything, or point me in a direction?
Have I hit an unknown bug as I can't think what else to try.
My bacula-sd.conf looks like this.
Please see additional information below.
--
Storage { # definition of myself
Name = iota-sd
SDPort = 9103 # Director's port
WorkingDirectory = /var/spool/bacula
Pid Directory = /var/run
Maximum Concurrent Jobs = 20
}
Director {
Name = iota-dir
Password = ###
}
# NEO2000 tapedrive1
Device {
Name = NeoLTO3-0
Drive Index = 0
Media Type = LTO3
Archive Device = /dev/nst0
Device Type = Tape
AutoChanger = yes;
AutomaticMount = yes;
AutoSelect = yes;
AlwaysOpen = no;
RemovableMedia = yes;
RandomAccess = no;
Maximum Network Buffer Size = 262144
Maximum File Size = 10gb
Minimum blocksize = 262144
Maximum blocksize = 262144
Spool Directory = /bacula-spool/1/
}
# NEO2000 jukebox
Autochanger {
Name = NEO2000-1
Device = NeoLTO3-0
Changer Command = /usr/libexec/bacula/mtx-changer %c %o %S %a %d
Changer Device = /dev/sg13
}
Messages {
Name = Standard
director = iota-dir = all
}
Hello
The documentation for the on-tape-format is here:
http://www.bacula.org/2.4.x-manuals/en/developers/Storage_Media_Output_Format.html
and at first glance old tape from version 3.x should be readable and
http://www.bacula.org/5.2.x-manuals/en/main/main/Bacula_Security_Issues.html#SECTION00431
also suggest
so.
One thing to try is to only set the Maximum blocksize = 262144 and
leave the Minimum blocksize at the default (0) to prevent Bacula from
using *fixed* blocksize.
Regards
Andreas
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PKI encryption for one job
Zitat von Tomasz Rozycki rozycki.tom...@gmail.com: Hello, Is it possibility to set PKI encryption only for one job on FD ? Best regards, Tomasz To my knowledge, no. PKI data encryption is used to prevent every one including even the Bacula director or storage daemon not owning the keys from having access to unencrypted data. With this it should not be possible to switch off encryption outside of the FD configuration and the FD has no knowledge about jobs. Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] dell PV-124T works with btape, but not with jobs
Zitat von Jonathan Horne jho...@skopos.us: as title says, i have a PV-124T that works with btape like a champ, but not within bacula for running jobs. when i submit a job to run to tape, the jobs just sit there. i have identical job configuration on another server with a larger power vault tape drive, and its runing like a top. when you submit a tape job, it quickly changes to running state. if i can work with btape but not with jobs, here can i start the troubleshooting? any advice greatly appreciated, jonathan Hello check the permissions for the changer scripts and the changer/tape devices. Bacula normaly run as unpriviled user and maybe the script permissions don't match. Regards Andreas -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] multiple tape drives
Zitat von Jonathan Horne jho...@skopos.us: My boss needs me to take the 2 tape drives inside the PV4000 and form them into 1 logical drive and stripe the backups across them simultaneously. Please help me figure this out, is this possible? Hello while it is possible to use more the one drive concurrently with low maximum concurrent jobs for the drives and prefer mounted volumes = no in the pool/job definition, you should have a closer look in the documentation about the downsides. It is also not possible to use more than one drive per job and you cannot switch drives within a job. We have tested this once and it worked reasonable well with data spooling and two drives, but it is not really a supported use-case. Regards Andreas -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] HP ultrium problem
Zitat von Tomasz Rozycki rozycki.tom...@gmail.com: Hi all, I've problem with my tapes. I've used HP Ultrium 200GB LTO-1 tape and bacula after 100Gb date marked volume as Full. Why do that? Should I disable software compression for the tape job? Bacula mark the tape as full becuase it is full. LTO-1 has 100GB native capacity, all other number are marketing driven and can only be reached with bloated data. If you disable the software compression in Bacula you might get more speed because the tape compression is faster, but regarding data on tape nothing will change. Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula - backup.c:892 Network send error
Zitat von Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] uthra.r@nasa.gov: I am on bacula 5.2.12 and I backup to LTO5 tapes. I started running a big FULL backup job. The job failed with the following error after backing up 2.2TB of data. I had the same problem with a full backup of another client. Start time: 23-Jan-2013 22:24:37 End time: 24-Jan-2013 20:23:57 Elapsed time: 21 hours 59 mins 20 secs Priority: 10 FD Files Written: 934,995 SD Files Written: 0 FD Bytes Written: 2,272,911,402,468 (2.272 TB) SD Bytes Written: 0 (0 B) Rate: 28712.9 KB/s ERROR: JobId 4183: Fatal error: backup.c:892 Network send error to SD. ERR=Broken pipe 24-Jan 20:23 lindy-dir JobId 4183: Here is a portion of the email report I got from bacula: JobId 4183: 3305 Autochanger load slot 34, drive 0, status is OK. JobId 4183: Wrote label to prelabeled Volume B01127L5 on device Drive-1 (/dev/nst0) 24-Jan 13:27 lindy-sd JobId 4183: New volume B01127L5 mounted on device Drive-1 (/dev/nst0) at 24-Jan-2013 13:27. JobId 4183: Despooling elapsed time = 01:35:51, Transfer rate = 67.92 M Bytes/second 24-Jan 14:54 lindy-sd JobId 4183: Spooling data again ... JobId 4183: User specified spool size reached. JobId 4183: Writing spooled data to Volume. Despooling 966,367,832,315 bytes ... JobId 4183: Fatal error: backup.c:892 Network send error to SD. ERR=Broken pipe 24-Jan 20:23 lindy-dir JobId 4183: Error: Bacula lindy-dir 5.2.12 (12Sep12): Could someone please guide me on this? I would really appreciate it. Thank you Uthra Hello try replacing the NIC and/or driver software. We had similar trouble until we replaced onboard NIC with a server grade PCIe card. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula - backup.c:892 Network send error
Zitat von Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] uthra.r@nasa.gov: Thank you for your reply. Backup of these clients have been running fine all these days. Only when I started a FULL back-up this week of the two clients they failed: Client1: JobId 4183: Fatal error: backup.c:892 Network send error to SD. ERR=Broken pipe 24-Jan 20:23 Client2: JobId 4180: Fatal error: backup.c:892 Network send error to SD. ERR=Broken pipe 24-Jan 13:16 ldy-sd JobId 4180: Fatal error: spool.c:301 Fatal append error on device Drive-1 (/dev/nst0): ERR=block.c:1045 Read error on fd=5 at file:blk 0:0 on device Drive-1 (/dev/nst0). ERR=Input/output error. Uthra As said we also had this kind of problems. On Machine with ~20G failed only from time to time but always at Full. The other machine failed all Full to tape after some random amount of data ranging from 300G to 1.6T from around 1.8T. After replacing the NIC at the bacula server the problem went away. Before this we have tested all sort of other network related things to no avail, so it might also be the problem you are facing. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Django Admin?
Zitat von Uwe Schuerkamp uwe.schuerk...@nionex.net: Hi folks, before I duplicate anyone's previous efforts I'd like to find out if anyone has already created a Django definition for Bacula's DB tables? I think it'd be quite cool to have a web-based reporting monitoring interface, and Django seems like the perfect solution for implementing such a thing given that Bacula comes with a python interface. All the best looking forward to hearing from you, There is also Bacula-Web which should do something similar, no? http://www.bacula-web.org/ Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fwd: Is my tape jukebox borked?
Zitat von John Drescher dresche...@gmail.com: -- Forwarded message -- From: Alan McKay alan.mckay+bac...@gmail.com Date: Tue, Jan 22, 2013 at 2:51 PM Subject: Re: [Bacula-users] Is my tape jukebox borked? To: John Drescher dresche...@gmail.com On Tue, Jan 22, 2013 at 2:44 PM, John Drescher dresche...@gmail.com wrote: Are you sure /dev/sg0 is the changer? Evidently not! OK, this gets me one step closer. Honestly I have not found a good simple here is where you start when you know nothing tutorial on bacula. The online docs don't seem to have it. Thanks ! We'll see where I get now. At least btape now seems to start without barfing so now I will go follow that btape example. At least on Linux you could use lsscsi -g to find the devices. The medimux is the changer. Next is to use /dev/tape/by-id/ instead /dev/sgX in the Bacula config with the matching device ID, so no trouble with udev. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Should I upgrade my win FDs from 3.0.3 to 5.x?
Zitat von Tamas Csillag tamas.csil...@gcd.ie: Hi All, Sorry if this was already asked, tried to do my homework by google and changelog but no joy. We're happily running Bacula on our Windows boxes, both 3.0.3 and 5.x versions. My question is, is there any reason to upgrade my 3.0.3 clients to 5.x? Any serious security bugfix, increased performance, possibility of dropping support for 3.x FDs in the future? To my knowledge there are no security fixes which are client related. The support on the public mailing list usually involves get the latest version first, but if you have paid support this doesn't matter. You should decide if you need the new features and what the costs/risks of updates are, to get the picture if the update is needed. It might also be of help to browse the changelog for windows key-word. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122412 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] parallelizing ClientRunBeforeJobs
Zitat von Tilman Schmidt t.schm...@phoenixsoftware.de: One of my Bacula servers backs up 18 clients. Many of these have a lengthy ClientRunBeforeJob. (SVN hot backups, database dumps, Windows system state backups - you get the picture.) As a consequence, more than half of the elapsed time of my nightly backup runs is actually spent waiting for one of the clients to complete its Before job while everything else sits idle. How are others handling this situation? Any smart ideas how to get the before jobs to run in parallel, preferably keeping the actual writing to tape sequential? Hello we use the spooling feature and run all *jobs* concurrently. With this you get adjustable sized blocks written to tape sequential to maximize throughput and concurrent spooling for all other jobs. Works trouble free but you need a really fast spool device for anything faster than LTO-3. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] save DB's online
Zitat von Sven Gehr mailingli...@dreampixel.de: Hi@all, is it possible to backu databases e.g. mysql, pgsql (on other hosts) with bacula online? There are many different possibilities: - Use the dump utility of the DB to get a consistent dump to backup maybe with a ClientRunBeforeJob - Use the DB provided possibillities to set the DB in Backup-Mode and let Bacula simply save the files - Use the Enterprise Plugins (http://www.baculasystems.com/products/bacula-enterprise-plugins/postgresql-plugin) which provide one or both of the above in an easy way Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Data encryption fails if backed up from fifo
Zitat von Axel Rau axel@chaos1.de: Am 15.01.2013 um 23:05 schrieb Axel Rau: Files backed up with readfifo = yes seem to be backed up fine: --- Software Compression: None VSS:no Encryption: yes Accurate: no Volume name(s): DB1-DB-DAILY-0120 Volume Session Id: 15 Volume Session Time:1358198139 Last Volume Bytes: 15,341,071,607 (15.34 GB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination:Backup OK --- Trying to restore them fails with following err: --- 15-Jan 22:21 db1-fd JobId 2492: Error: restore.c:772 \ Missing encryption session data stream for /usr/local/pgsql/bacula_backup/fifo/development.data.dump --- No problems to backup and restore flat files with data encryption. This is bacula 5.2.12 (12Sep12). Tested with clients on FreeBSD 8.2 and 9.1. If nobody has any comments on this, I will file a bug report. Axel Yup, would be best. We use Data encryption but no FIFO so no comment from our side. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO-5 Tapes full after between 30-130gb Dell PV-124T
Zitat von bdoran bill.do...@concordia.ca: Hi, I'm using a Bacula setup in conjunction with a Dell PV-124T which has 16 LTO-5 tapes. The system works correctly except that the tapes become extremely quickly. There is no consistency between the amount written to each tape though it is generally in the range of 30-130gb - far less then the expected volume. Check if you are really using LTO-5 media. Test with a known working tape loaded. Bacula simply writes to tape until the tape-drive reports a write error, this is used as tape-full marker. Regards Andreas -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] windows enterprise is not working, x32/x64.
Zitat von Jean-Louis Dupond jean-lo...@dupond.be: Could you check what http://artemis.dupie.be/bacula-win64-5.2.12.exe does? :) Is there any chance to get a detailed description how you setup the build environment? Thanks Andreas -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula 5.2.12 randomly fails to recycle disk based volumes
Zitat von Uwe Schuerkamp uwe.schuerk...@nionex.net: On Fri, Jan 11, 2013 at 02:52:05PM +, Martin Simmons wrote: Do you know which volume is used? There seems to be some confusion between incremental-0058 and incremental-0508. Is that a common feature of other failures like this? The problem is that this occurs only every once in a while, for the most part volume recycling works as advertised. Do you suggest I should delete the volume in question and have bacula create a new one? Can you log all queries in the mysql server to see if that records anthing interesting? I guess such a log would grow to be quite large, right? Esp. as I have no way of knowing when the error occurs, and I expect a *lot* of queries to be fired up on a 200GB+ File table ;) Fact is that Bacula has to wait for the query doing the recycling/pruning task and MySQL does not finish in a reasonable time. With this the following errors are possible: - Wrong usage of SQL query by Bacula, but this should get problems everytime not at random occasions - Bad planer decisions inside MySQL - Scalability problems with MySQL While it might be possible to tune-up the MySQL part of Bacula, your short term solution is more at the MySQL side. Do you have the latest release and checked/repaired all index and optimizer values? If yes, you might try export/import or replicate/switch-over or maybe even switch to PostgreSQL. Regards Andreas -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula on Ubuntu 12.04.01
Zitat von Sven Gehr mailingli...@dreampixel.de:
Hi@all,
I want to install bacula on a ubuntu-server (12.04.01-LTS / 64bit). I
install with the following command:
aptitude install postgresql
aptitude install bacula-director-pgsql bacula-common-pgsql
bacula-sd-pgsql bacula-fd bacula-sd-tools bacula-console
and answer the questions:
Konfigurieren der Datenbank für bacula-director-pgsql mit
dbconfig-common: Ja
PostgreSQL-Anwendungspasswort für bacula-director-pgsql:
secret
Now I want test my director with:
bacula-dir -t
and get the error:
bacula-dir: dird.c:954 Konnte Catalog MyCatalog, database bacula
nicht öffnen.
bacula-dir: dird.c:959 postgresql.c:241 Unable to connect to PostgreSQL
server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect;
max_connections exceeded.
15-Jan 10:04 bacula-dir ERROR TERMINATION
Bitte die Konfigurationsdatei korrigieren: bacula-dir.conf
The role 'bacula' is available:
postgres=# \dg[+]
List of roles
Role name | Attributes | Member of
| Description
---++---+-
bacula|| {}
|
postgres | Superuser, Create role, Create DB, Replication | {}
|
but I miss the script's (create_bacula_database, make_bacula_tables and
grant_bacula_privileges). Where can I find the scripts on my
ubuntu-system?
Hello
you should check pg_hba.conf. It is set restrictive on a stock Ubuntu
12.04 install and be aware that per default unix sockets are used if
you dont at least specify localhost as target.
Regards
Andreas
--
Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS
and more. Get SQL Server skills now (including 2012) with LearnDevNow -
200+ hours of step-by-step video tutorials by Microsoft MVPs and experts.
SALE $99.99 this month only - learn more at:
http://p.sf.net/sfu/learnmore_122512
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula on Ubuntu 12.04.01
Zitat von Sven Gehr mailingli...@dreampixel.de: I add the line: hostlocalhost all 127.0.0.1/32trust an restart the server. Same problem again. Check if DB Adress = localhost is set in your Bacula-Directory config file, if not the unix sockets are used, which work by trusting the matching OS user. It does not work if you test as root or other non-bacula user. You should also reset the above line to md5 instead of trust Regards Andreas -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] UbuntuRepositorie?
Zitat von Sven Gehr mailingli...@dreampixel.de: Hi@all, I use ubuntu server 12.04 LTS because of the long suport an here are the version 5.2.5 available. For other linux-distros I found repos with newer version: http://download.opensuse.org/repositories/home:/dassit:/bacula/ This is a common problem of using distribution packages. The release version is merely frozen and only security fixes will be backported. If there is a busy maintainer you sometimes get personal repositories but for Ubuntu none exists to my knowledge... but not for ubuntu. Does anyone have a repo with newer versions? We have build Ubuntu 12.04 x64 style deb-packages for our own server with the 5.2.12, but there is no public repository. I can sent you a URL in personal mail to the as-is tar.gz archiv. Regards Andreas -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] getting my tape drive online
Zitat von Jonathan Horne jho...@skopos.us: My drive appears to be working correctly, and the devices I poll here correspond to configurations I use in my config file. Tests all seem fine (to my layman's observation), but in practice actual commands are failing. Have you also checked as user Bacula is running as? Maybe some missing permission on a script/directory? Regards Andreas -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restore symantec backup with Bacula???
Zitat von dummycerberus bacula-fo...@backupcentral.com: Hello, I've several media with a full backup previously made with Symantec Netbackup Enterprise Server v7.1.0.4... is it possible to import, catalague and restores that bakcup with Bacula? There is (beside some historical) no standard for on-Tape format, so every backup solution uses it's own. That said your only solution is to restore the files with Netbackup and backup again with Bacula. Regards Andreas -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] getting my tape drive online
Zitat von Jonathan Horne jho...@skopos.us: i have a 44 slot PV-TL4000 that i have just brought online with my bacula. my first attempt at labeling the media based on the barcodes failed. below is the output, can anyone give me some ideas on what to start checking? Have you tested your autochanger according to http://www.bacula.org/5.2.x-manuals/en/main/main/Autochanger_Resource.html chapter Testing Autochanger and Adapting mtx-changer script? What OS and what version of Bacula? Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] getting my tape drive online
Zitat von lst_ho...@kwsoft.de: Zitat von Jonathan Horne jho...@skopos.us: i have a 44 slot PV-TL4000 that i have just brought online with my bacula. my first attempt at labeling the media based on the barcodes failed. below is the output, can anyone give me some ideas on what to start checking? Have you tested your autochanger according to http://www.bacula.org/5.2.x-manuals/en/main/main/Autochanger_Resource.html chapter Testing Autochanger and Adapting mtx-changer script? What OS and what version of Bacula? And BTW the PV-TL4000 looks like yet another rebranded BDT device (http://www.bdt.de/en/products/storage-automation/tape-storage/) which are also sold by HP, so it should work with the standard script. What connection is used FC or SAS? Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PGSQL: make_catalog_backup.pl dumps everything?
Zitat von Wolfgang Denk w...@denx.de: Hello, I noticed that running make_catalog_backup.pl with a PostgreSQL database runs pg_dump -c (i. e. without specifying a database), which results in a full dump of the whole DB server, including far more than the Bacula catalog. Is this a bug, or intentional? If the latter, what is the rationale? I would say that this is intentional for people just want to use Bacula and therfore need a database. If you use the Bacula database in a non standard way eg. for more that just Bacula, you should also be able to roll out your own backup strategie for the DB. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Windows Support?
Zitat von pedro moreno syxt...@gmail.com: I had seen the news about the enterprise binaries for windows, need them, is not to expensive no issue, my question is simple: Once I start using the windows binaries, I need to contact them for support or this list is still open for help for windows clients? Thanks!!! With the binary fee you only get the binaries. If you need a support contract you have to buy one. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Cannot start bacula-fd on Windows XP Professional
Zitat von Luca Bertoncello bertonce...@frischergehts.net: Hi, list! I need to backup a machine with Windows XP Professional. I installed Bacula and I configured it, such as a Linux machine. Unfortunately, I cannot start the FD... If I start it from console I don't get any message, but the service does not work. Any idea, what I can check? Hello in my case it were errors in the config file most of the time. On one occasion if have screwed the permissions so loading dlls was not possible... Regards Andreas -- Master SQL Server Development, Administration, T-SQL, SSAS, SSIS, SSRS and more. Get SQL Server skills now (including 2012) with LearnDevNow - 200+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only - learn more at: http://p.sf.net/sfu/learnmore_122512 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] switching to data encryption
Zitat von Axel Rau axel@chaos1.de: Hello, I just turned on data encryption of my existing bacula infrastructure. Question: Can I mix unencrypted and encrypted data in a volume, or should I start encryption on new volumes? You can mix encrypted and unencrypted data without problem. For the SD it doesn't matter at all. Be sure to have a disaster plan for your keys and well documented (FD) settings, though. Test backup: --- 05-Jan 13:45 chaos1-sd JobId 2280: Ready to append to end of Volume ZEUS-DAYLY-0004 size=5833955 05-Jan 13:45 chaos1-fd JobId 2280: Warning: xattr.c:1150 llistxattr error on file /private/var/spool/postfix/public/pickup: ERR=Operation not permitted 05-Jan 13:45 chaos1-fd JobId 2280: Warning: xattr.c:1150 llistxattr error on file /private/var/spool/postfix/public/qmgr: ERR=Operation not permitted 05-Jan 13:46 chaos1-fd JobId 2280: Warning: Encountered 2 xattr errors while doing backup These are (Postfix) unix sockets, no idea why xattr spits warnings, but has nothing todo with encryption. Maybe it's related to the OS used... 05-Jan 13:46 chaos1-sd JobId 2280: Elapsed time=00:01:36, Transfer rate=346 Bytes/second 05-Jan 13:46 chaos1-dir JobId 2280: Bacula chaos1-dir 5.2.12 (12Sep12): Build OS: i386-apple-darwin10.8.0 osx 10.8.0 JobId: 2280 Job:Zeus_Backup.2013-01-05_13.45.13_03 Backup Level: Incremental, since=2013-01-05 12:50:51 Client: zeus 5.2.12 (12Sep12) i386-apple-darwin10.8.0,osx,10.8.0 FileSet:Zeus Full Set 2012-03-28 23:56:33 Pool: Zeus Inc Backup Pool (From Job IncPool override) Catalog:MyCatalog (From Client resource) Storage:File (From Job resource) Scheduled time: 05-Jan-2013 13:44:57 Start time: 05-Jan-2013 13:45:15 End time: 05-Jan-2013 13:46:52 Elapsed time: 1 min 37 secs Priority: 10 FD Files Written: 33 SD Files Written: 33 FD Bytes Written: 22,079 (22.07 KB) SD Bytes Written: 33,231 (33.23 KB) Rate: 0.2 KB/s Software Compression: 42.6 % VSS:no Encryption: yes Accurate: no Volume name(s): ZEUS-DAYLY-0004 Volume Session Id: 1 Volume Session Time:1357389875 Last Volume Bytes: 5,869,434 (5.869 MB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination:Backup OK This is the important part: Backup OK --- Restore: --- 05-Jan 13:48 chaos1-dir JobId 2281: Start Restore Job Zeus_Restore_Files.2013-01-05_13.48.23_04 05-Jan 13:48 chaos1-dir JobId 2281: Using Device FileStorage 05-Jan 13:48 chaos1-sd JobId 2281: Ready to read from volume ZEUS-DAYLY-0004 on device FileStorage (/UV0/bacula). 05-Jan 13:48 chaos1-sd JobId 2281: Forward spacing Volume ZEUS-DAYLY-0004 to file:block 0:5833955. 05-Jan 13:48 chaos1-fd JobId 2281: -rw-r- 1 root wheel 16384 2013-01-05 13:37:55 /usr/local/bacula/restore/private/etc/aliases.db 05-Jan 13:48 chaos1-sd JobId 2281: End of Volume at file 0 on device FileStorage (/UV0/bacula), Volume ZEUS-DAYLY-0004 05-Jan 13:48 chaos1-sd JobId 2281: End of all volumes. 05-Jan 13:48 chaos1-fd JobId 2281: Error: openssl.c:86 OpenSSL digest Verify final failed: ERR=error:04077068:rsa routines:RSA_verify:bad signature 05-Jan 13:48 chaos1-fd JobId 2281: Error: restore.c:1246 Signature validation failed for file /usr/local/bacula/restore/private/etc/aliases.db: ERR=Signature is invalid Maybe mixed encrypted and unencrypted data within on Job/Pool. If you have switched on encryption/signatures at FD you will not be able to restore unsigned data, not sure about unencrypted but i guess it also does not work. So you should first do a full, than try to restore. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122912 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] LTO5 speed
Zitat von f.staed...@dafuer.de: Hi Jesper, Well, if you are sending uncompressible data, then above picture looks like a fully saturated LTO5-drive operating at optimal speed since the 140MB/s are for compressible data to the drive. thanks for your quick answer. So the sound of the drive like starting and stopping in about 10 second intervals are normal? While writing this - is maybe Maximum File Size of SD the reason? Since transferred data is about 1GB and it is writing an EOF mark to tape? If so, a full tape will have about 1500 EOF marks - can I safely put it to a higher value? As far as I understand the only drawback are maybe slower restores of files. But I don't think that would be a problem, since new data arrive in 5-10GB chunks. Thanks, Frank. Hello from what i remember the manual recommends something like 2GB Maximum File Size for LTO2-3, so i guess LTO5 would be served well with 4GB. Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Plans for support block-based dedupe?
Zitat von Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2013/1/3 Silver Salonen sil...@serverock.ee Excellent news! Any ETA when the plugin will be available? This year... :) Q3/Q4. It is not ready for production yet, but we use it for daily backups and it works very good. best regards -- Radosław Korzeniewski rados...@korzeniewski.net Hello do you have any numbers you could share how good real world dedupe will be? Regards Andreas -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] PostgreSQL - how to exclude bacula database?
Zitat von Dan Langille d...@langille.org: On Dec 27, 2012, at 8:40 PM, Jérôme Blion wrote: You can use pg_dump to backup databases separately. (as far as I know, the pg_dump creates consistent backup by defaults, whereas mysqldump does not by default) Say what? mysqldump doesn't produce a valid backup? Back in ancient times there where problems with some backends doesn't produce valid *online* backups with mysqldump, but that's a long gone story as far as i know. Regards Andreas -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Microsoft Windows Binaries
Zitat von Geert Stappers geert.stapp...@vanadgroup.com: Op 20121217 om 22:39 schreef lst_ho...@kwsoft.de: Zitat von Kern Sibbald k...@sibbald.com: } [Quarterly Bacula Status Report, december 2012] 5. Getting Bacula Windows Binaries As you are probably aware, because of the work involved, the community Windows File daemon binaries are no longer produced by the Bacula project. As an an alternative, I proposed to Bacula Systems to offer their Windows Enterprise binaries as a service to the Community. They agreed, and have chosen a price that is designed to simply cover the costs of producing and distributing those binaries. You can now obtain these Bacula Enterprise Windows binaries from Bacula Systems SA by clicking on the following link http://www.baculasystems.com/windows-binaries-for-bacula-community-users. These binaries allow you to continue using the Bacula community version, while keeping Bacula well integrated into your Windows environment. These binaries contain the Windows File daemons only. As was previously the case for the Community binaries, the Windows Storage daemon and Director are not provided. Installers for both 32 and 64 bit binaries is included in the offer. The extra Enterprise Windows plugins are in separate packages and are not part of this offering. The support for any problems will be handled through the Community Bugs database. Hello two questions about the Windows binaries: - Is it correct that the fee is intended for the right of 90 days downloads/updates? - Is it possible to get something other than PayPal for payment and list the payment options at start? FWIW: PayPal solves some of the artificial problems of international money transfer. There was a time of troublesome international money transfer, but those where the days before the Internet. Cheers Geert Stappers While it might be that PayPal solves some trouble it also create some additional grief. We have no PayPal company account and probably never will, so the question is how to pay the binary fee without PayPal. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Microsoft Windows Binaries
Zitat von Gary R. Schmidt g...@mcleod-schmidt.id.au: On 19/12/2012 8:08 PM, lst_ho...@kwsoft.de wrote: Zitat von Geert Stappers geert.stapp...@vanadgroup.com: Op 20121217 om 22:39 schreef lst_ho...@kwsoft.de: Zitat von Kern Sibbald k...@sibbald.com: } [Quarterly Bacula Status Report, december 2012] 5. Getting Bacula Windows Binaries As you are probably aware, because of the work involved, the community Windows File daemon binaries are no longer produced by the Bacula project. As an an alternative, I proposed to Bacula Systems to offer their Windows Enterprise binaries as a service to the Community. They agreed, and have chosen a price that is designed to simply cover the costs of producing and distributing those binaries. You can now obtain these Bacula Enterprise Windows binaries from Bacula Systems SA by clicking on the following link http://www.baculasystems.com/windows-binaries-for-bacula-community-users. These binaries allow you to continue using the Bacula community version, while keeping Bacula well integrated into your Windows environment. These binaries contain the Windows File daemons only. As was previously the case for the Community binaries, the Windows Storage daemon and Director are not provided. Installers for both 32 and 64 bit binaries is included in the offer. The extra Enterprise Windows plugins are in separate packages and are not part of this offering. The support for any problems will be handled through the Community Bugs database. Hello two questions about the Windows binaries: - Is it correct that the fee is intended for the right of 90 days downloads/updates? - Is it possible to get something other than PayPal for payment and list the payment options at start? FWIW: PayPal solves some of the artificial problems of international money transfer. There was a time of troublesome international money transfer, but those where the days before the Internet. Cheers Geert Stappers While it might be that PayPal solves some trouble it also create some additional grief. We have no PayPal company account and probably never will, so the question is how to pay the binary fee without PayPal. Use your own PayPal account, and claim the cost back from the company. Just the same as when you buy some tapes or computer bits or whatever in a hurry. I don't have a personal PayPal account either and buy smaller computer parts in a hurry in a shop. What's the big problem with doing that? It's strange that one need a bank account at a special bank (=PayPal) to buy something. It was the idea of bank accounts that you can use *any* bank account to pay without hard cash. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Microsoft Windows Binaries
Zitat von James Harper james.har...@bendigoit.com.au: While it might be that PayPal solves some trouble it also create some additional grief. We have no PayPal company account and probably never will, so the question is how to pay the binary fee without PayPal. Last time I made a payment somewhere (ebay I think) I was able to use my credit card via paypal without actually having a paypal account. I'm not sure if that's a universal thing or if it depends on the recipient of the payment. James You now have a PayPal account as far as know... Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula-Web 5.2.11 is available from now
Zitat von Uwe Schuerkamp uwe.schuerk...@nionex.net: On Mon, Dec 17, 2012 at 07:28:38AM +, bacula-...@dflc.ch wrote: Dear all, I'm really happy to announce that a new version of Bacula-Web (5.2.11) is available. As this version contain major bug fixes and some new features too, i'd suggest everybody to upgrade to this version. All details and the release note is available in the download section (http://bacula-web.org/download.html) The documentation will be updated soon. Thanks for the updated version. The download link points to 5.2.2 however, and there seems to be no version named .11 on the webserver, either. All the best, Uwe Huh? From my point of internet the download links to http://bacula-web.org/tl_files/downloads/bacula-web-5.2.11.tar.gz Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Quarterly Bacula Status Report
Zitat von Kern Sibbald k...@sibbald.com: 5. Getting Bacula Windows Binaries As you are probably aware, because of the work involved, the community Windows File daemon binaries are no longer produced by the Bacula project. As an an alternative, I proposed to Bacula Systems to offer their Windows Enterprise binaries as a service to the Community. They agreed, and have chosen a price that is designed to simply cover the costs of producing and distributing those binaries. You can now obtain these Bacula Enterprise Windows binaries from Bacula Systems SA by clicking on the following link http://www.baculasystems.com/windows-binaries-for-bacula-community-users. These binaries allow you to continue using the Bacula community version, while keeping Bacula well integrated into your Windows environment. These binaries contain the Windows File daemons only. As was previously the case for the Community binaries, the Windows Storage daemon and Director are not provided. Installers for both 32 and 64 bit binaries is included in the offer. The extra Enterprise Windows plugins are in separate packages and are not part of this offering. The support for any problems will be handled through the Community Bugs database. Hello two questions about the Windows binaries: - Is it correct that the fee is intended for the right of 90 days downloads/updates? - Is it possible to get something other than PayPal for payment and list the payment options at start? Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula fatal error all the time
Zitat von b_...@mail.ru: On Nov 29, 2012, at 11:45 PM, lst_ho...@kwsoft.de wrote: Zitat von b_...@mail.ru: On Nov 29, 2012, at 11:05 PM, Dan Langille d...@langille.org wrote: On Nov 29, 2012, at 2:26 PM, b_...@mail.ru wrote: Hello, I have a couple of hosts which backup through bacula. I have confronted with next situation, one host can't perform full backup, process of backup always finished with this error: Error: bsock.c:389 Write error sending 262144 bytes to Storage daemon:IP ADDRESS:9103: ERR=Broken pipe Fatal error: backup.c:1190 Network send error to SD. ERR=Broken pipe Error: Director's comm line to SD dropped Error: Bacula dir.bacula.HOSTNAME. 5.2.6 Time of occurrence of this error is not always the same, here are the logs: Elapsed time: 11 mins 56 secs Elapsed time: 1 hour 10 mins 18 secs Elapsed time: 47 mins 36 secs Elapsed time: 1 hour 14 mins 40 secs Elapsed time: 1 hour 40 mins 1 sec I don't think that this related to network issue. But client side have a lot of I/O operation, disk system is busy almost all the time. Maybe this is a cause. How can I solve this? I don't see any timeout directives which can help me. Client side is FreeBSD 7.2 amd64 bacula 5.2.6 HEADS UP. FreeBSD 7.2 was end-of-life'd in 2010 which means no security patches will be issued for it. Upgrading is recommended. see http://www.freebsd.org/security/#unsup That that matter, that's an old version of FreeBSD too. :) I know, but upgrade in this case is impossible and this is not related to our problem with bacula I think Server side (DIR and SD on the same host) is FreeBSD 9.0 amd64 bacula 5.2.6 (also tried 5.2.12 with the same result) Have you looked at trying the 'Heartbeat Interval' settings? You should try setting on the SD. There is also a 'Heartbeat Interval' on the FD, but that doesn't seem to be the error you're getting. yes, I have tried to play with Heartbeat Interval. Unfortunately doesn't help. this is not related to NIC in my case, the same issue with various cards, OS. The same behaviour with FreeBSD, CentOS. Broadcom and Intel cards. I have no idea what could be a cause, anybody have have any thoughts? Try with another NIC. We first had problems with our Bacula Server failing two clients out of ~20 with connection failures randomly. After ditching the Onboard GE (Marvell PHY) and using a PCIe NIC on the Server the problem went away. It is a problem outside Bacula which simply uses a standard long-lived TCP connection. The amount of data transfered over a single connection for a long time does reveal subtile bugs in hardware and OS/drivers from time to time. So your only chance to solve is too swap related parts (NIcs, Switches, Router, cable ...) until the culprit is found. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] init.d files missing after upgrading on openSUSE
Zitat von Silver Salonen sil...@serverock.ee: Hi. Some time ago Bacula 5.2.12 came into Archiving:/Backup repository of openSUSE 12.2, but now there are no Bacula files in /etc/init.d. There is also nothing about Bacula in service --status-all. How can I start my FD now? This is a question to direct to the openSUSE folks. Every distribution has to provide a start-stop script matching their startup system of choice. Otherwise you have to read the man page of bacula-fd to find out what parameters to supply. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] [OT] Nasty defekt in Windows VSS!
Hello FYI: Today Microsoft announced a patch solving a problem which could lead to silent data corruption when using VSS for backup (http://support.microsoft.com/kb/2748349). Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres database
Zitat von Joseph L. Casale jcas...@activenetwerx.com: What process keeps the bacula database size under control? None, configuration does. Check your various retention levels for sane values. If there are a lot of clients/files/jobs a monthly dbcheck could also help to purge out no longer needed entries. Also if PostgreSQl is a really old version ( 8.4) the OP should check if autovacuum is running. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula stops writing to disk volume after 2GB
Zitat von m.list mail.list...@googlemail.com: Bacula Version: 5.2.5 I have configured bacula to write volumes to disk, however bacula stops writing to the volume as soon as it reaches 2gb. The file system is not an issue as I have stored files larger than 2gb. 06-Dec 17:22 backup-sd JobId 8421: End of Volume Full-Monthly-0005 at 0:2147475577 on device FileStorage (/nfs/backup-pool). Write of 64512 bytes got 8069. If this is really NFS double-check that both sides agree to use at least NFSv3 as older NFS versions have a 2GB limit and Bacula clearly is not able to write more data to the file in question. On the other side you can easily work around this with splitting many volumes of 2GB if your expected data size isn't that big. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Trouble getting jobs to run simultaneously
Zitat von Troy Kocher tkoc...@mtadistributors.com: All, Recently I've made changes to my bacula-dir and bacula-sd to with the intent of running simultaneous jobs. After the changes my testing seems to indicate I still don't have it right. I launched my first job 'Image_Backup' manually with it's default priority, then launched the retail job modifying the priority to 1, with the hope to force a simultaneous backup. The retail job sat waiting for the image job to complete. I've attached the relevant sections from my bacula-dir.conf. If anyone has any helpful thoughts I'd really appreciate it. Troy Hello Only Jobs with the *same* priority run concurrently if all resources used support concurrent use. The value of priority is only used to decide which Job to start first if more than one is scheduled at the same time. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Error could not open WriteBootstrap file
Zitat von Pinky Vivona ptviv...@gmail.com: Hello, Can someone tell me about this error message? Error: Could not open WriteBootstrap file: /etc/bacula/SERVER1_File.bsr: ERR=Permission denied\ --- Rather clear, no? You have configured to use /etc/bacula path to store the bootstrap, but the Bacula DIR does not run as root in most cases, so no write access to /etc. You should use something like /var/lib/bacula as path for writing files. Regards Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Waiting on what, now.....?
Zitat von Phil Stracchino ala...@metrocast.net: I just tried to restore two files to my workstation. I have two SDs, one running an LTO4 drive attached to the workstation (babylon5), one controlling a 12TB ZFS array on my NAS server (babylon4). Here's the BSR: Storage=babylon4-file Volume=INCR-20121129-04:30 MediaType=File Device=FileStorage VolSessionId=296 VolSessionTime=1349979764 VolAddr=165656-114423 FileIndex=3108-3109 Count=2 But director status says waiting on Storage babylon5-sd, and the console says: $ $ $ Warning default storage overridden by babylon5-sd on command line. Warning default storage overridden by babylon5-sd on command line. Bootstrap records written to /var/lib/bacula/epsilon3-dir.restore.1.bsr Looks like some setting has forced Bacula to use the (wrong) SD. Check your restore Job for all settings which might pin to a SD like Pool statements and the like. Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about Filesize from tape files
Zitat von Dimitrij Hilt dimitrij.h...@fhe3.com: Am 03.12.2012 02:21, schrieb Dan Langille: On Dec 2, 2012, at 6:40 PM, Dimitrij Hilt wrote: Hi there, i have bacula system on debian squeeze with packages from back ports. What version of Bacula are you running? Version: 5.2.6 (both DIR and SD) comming from Debian backports package 5.2.6+dfsg-1~bpo60+1. I use file based backups and i'm wondering why tape files are created as sparse file: du --block-size 1 file-0399 134742016 file-0399 du --bytes file-0399 70112117file-0399 What does the above represent? What command did you issue? Any idea how i can avoid it? It waste to much disc space. Not yet. I don't know what I'm looking at. :) Thanks. Once more. If i copy this file localy and back, it shrinks to normal size and both 'du' are same. Restore from shrinked file is still possible. Not sure but from my knowledge Bacula does not delete/truncate files by default when they are reused but simply overwrite them. You could try Action On Purge = Truncate to shrink the files before reuse. Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about Filesize from tape files
Zitat von Dimitrij Hilt dimitrij.h...@fhe3.com: Am 03.12.2012 10:23, schrieb lst_ho...@kwsoft.de: Zitat von Dimitrij Hilt dimitrij.h...@fhe3.com: Am 03.12.2012 02:21, schrieb Dan Langille: On Dec 2, 2012, at 6:40 PM, Dimitrij Hilt wrote: Hi there, i have bacula system on debian squeeze with packages from back ports. What version of Bacula are you running? Version: 5.2.6 (both DIR and SD) comming from Debian backports package 5.2.6+dfsg-1~bpo60+1. I use file based backups and i'm wondering why tape files are created as sparse file: du --block-size 1 file-0399 134742016 file-0399 du --bytes file-0399 70112117 file-0399 What does the above represent? What command did you issue? Any idea how i can avoid it? It waste to much disc space. Not yet. I don't know what I'm looking at. :) Thanks. Once more. If i copy this file localy and back, it shrinks to normal size and both 'du' are same. Restore from shrinked file is still possible. Not sure but from my knowledge Bacula does not delete/truncate files by default when they are reused but simply overwrite them. You could try Action On Purge = Truncate to shrink the files before reuse. These files was not reused. It is new system and create new (possible sparse) files. Action On Purge = Truncate is allready inplace. No idea then. In our case newly labeled files are 196 bytes in size. On the other hand you need to provide the space anyway at backup runtime, if not, you could simply label less files so less space is occupied... Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] BAT problems - Restore
Zitat von Pedro Bordin Hoffmann pe...@bc-rs.com: The strange thing is that sometimes it work. But I need to try a few times before it work. I already try to wait for 2 hours (thinking it could be the pgsql) but no success. Can be because of folders with names in ISO-8859? Like á é ç ? This problem occurs since version 3 of bacula, in almost every server I have bacula installed. Hello we first tried Bat on Windows and gave up on it because of this issues. As far as i remember it was due to a bug in the qt libs used/bundled with Bat-Windows. We now use Bat only on Linux with much better results. As it is not a simple client GUI but a full Bacula (Directory) Manager, you should not run it on every client anyway IMMHO. Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] concurrent jobs with multiple directors
Zitat von Silver Salonen sil...@serverock.ee: On Thursday 29 November 2012 16:28:11 lst_ho...@kwsoft.de wrote: Zitat von Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2012/11/29 Silver Salonen sil...@serverock.ee ** On Thursday 29 November 2012 15:43:32 Radosław Korzeniewski wrote: Hello, 2012/11/29 Silver Salonen sil...@serverock.ee sd1--dir1 --- client1 --- dir2--sd2 Is your client1 a Windows machine? If so, do you use VSS Enable = Yes in your FileSet resource? Yes. So, the current limitation of bacula-fd client on windows does not permit concurrent jobs. Sorry. So if I want them to run concurrently, I have to disable VSS on at least one of these jobs? You have to try. You have to disable VSS on all jobs for this Windows machine. Furthermore don't try this with data encryption or your FD will crash. If you configure both jobs without VSS and the FD without encryption it might work, but i still wonder if a multi director configuration is supported anyway. Regards Andreas I now got a natural test-case :) 1) I disabled VSS only on the job started 2nd and everything is still working OK (so I don't have to disable VSS on ALL jobs) 2) I also use encryption and I did not disable it, everything is still working OK (so I don't have to disable encryption) 3) The 2nd job ran parallel to the 1st one (so disabling VSS only on the job(s) started later is enough) Do the two jobs really run in parallel eg. is there any progress on both of them? We have also tested this and with VSS enabled on the first job all subsequent jobs get started but simply wait for the first to finish. With no VSS at all both jobs started working and after some GB of the data the FD crash within openssl (http://bugs.bacula.org/view.php?id=1940). We only have one director but it is a client side thing as far as i know. Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restarting lots of Jobs after a problem. How?
Zitat von Oliver Hoffmann o...@dom.de: Hi all, recently I had the following problem. After a reboot of the bacula server due to malfunction of the attached RAID all jobs very not running again. OK, that is what I expected but not what I wanted as starting 50 or more jobs by hand with bconsole is a bit tiresome. Thus I simply waited for the next backup run but that means no backups for that night. Is there a way to accomplish a mass restart of jobs? My first thought was scripting the bconsole but maybe there is a build-in solution or something easier for that? Thx for ideas, Oliver You can alter the schedule time used and restart the director, but most of the time starting backup jobs out of the backup window is a bad idea anyway. If the jobs failed instead of get skipped you can have a look at the Reschedule On Error jobs setting. Regards Andreas -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula fatal error all the time
Zitat von Phil Stracchino ala...@metrocast.net: On 11/29/12 16:45, lst_ho...@kwsoft.de wrote: Try with another NIC. We first had problems with our Bacula Server failing two clients out of ~20 with connection failures randomly. After ditching the Onboard GE (Marvell PHY) and using a PCIe NIC on the Server the problem went away. I had similar problems at one point with an nVidia nForce chipset NIC, until I discovered the magic configuration trick to fix the problem. Once I figured that out, the problem went away for good. So what was the trick?? Regards Andreas -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Quick fix for Dir Inserting Attributes
Zitat von Julien Cochennec julien.cochen...@mnhn.fr: Hi Everybody, I have a Dir Inserting Attribute (it's been 2 days now) so all my backups are stuck because of this. Is there a quick fix, or even a dirty fix to get rid of this? I saw solutions about compiling again without a batch-insert option, but I really don't want to do that. I can configure anything on mysql server to make things go faster. Thanks. You really should be more specific about your problem: - Which Bacula version? - Installed from scratch or has it worked before? - What kind of Jobs are failing? - Any errror in the Bacula/Db/Systemlog? and so on... Normally inserting attributes, even many of them is not a problem so there is no dirty fix to apply. Regards Andreas -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Quick fix for Dir Inserting Attributes
Zitat von lst_ho...@kwsoft.de: Zitat von Julien Cochennec julien.cochen...@mnhn.fr: Hi Everybody, I have a Dir Inserting Attribute (it's been 2 days now) so all my backups are stuck because of this. Is there a quick fix, or even a dirty fix to get rid of this? I saw solutions about compiling again without a batch-insert option, but I really don't want to do that. I can configure anything on mysql server to make things go faster. Thanks. You really should be more specific about your problem: - Which Bacula version? - Installed from scratch or has it worked before? - What kind of Jobs are failing? - Any errror in the Bacula/Db/Systemlog? and so on... Normally inserting attributes, even many of them is not a problem so there is no dirty fix to apply. For example one of our bigger jobs insert ~800MB attributes from around 2,2 Mio. files in less than 90 seconds. So it is clearly something wrong with your setup and you need to solve the problem and no workaround. Regards Andreas -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Quick fix for Dir Inserting Attributes
Zitat von Julien Cochennec julien.cochen...@mnhn.fr: On 30/11/2012 10:53, lst_ho...@kwsoft.de wrote: Zitat von Julien Cochennec julien.cochen...@mnhn.fr: Hi Everybody, I have a Dir Inserting Attribute (it's been 2 days now) so all my backups are stuck because of this. Is there a quick fix, or even a dirty fix to get rid of this? I saw solutions about compiling again without a batch-insert option, but I really don't want to do that. I can configure anything on mysql server to make things go faster. Thanks. You really should be more specific about your problem: - Which Bacula version? - Installed from scratch or has it worked before? - What kind of Jobs are failing? - Any errror in the Bacula/Db/Systemlog? and so on... Bacula version is 5.0.1 It worked since many month, almost one year, first time I get this problem. The First Job that is hanging is many To (almost 4 To) and the others seem to be waiting for this one (?) Mysqladmin processlist shows this : | 1772 | bacula | localhost:58412 | bacula | Query | 105617 | Sending data | INSERT INTO Path (Path) SELECT a.Path FROM (SELECT DISTINCT Path FROM batch) AS a WHERE NOT EXISTS ( | | 1950 | bacula | localhost:58825 | bacula | Query | 100362 | Locked | LOCK TABLES Path write, batch write, Path as p write Might be that MySQL is deadlocked, but i don't know MySQL enough to confirm... Where is the /bacula/Db/Systemlog? I found no file like this on my server. This means search the Bacula log, the Database (MySQL) log and the OS (Linux) system log for trouble. Your syslog configuration should tell you where to find these log files. Regards Andreas -- Keep yourself connected to Go Parallel: TUNE You got it built. Now make it sing. Tune shows you how. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
Zitat von Olivier Delestre olivier.deles...@univ-rouen.fr: Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you*must* be using a PostgreSQL that was built with the*|--|enable-thread-safety* option, otherwise you will get *data corruption*. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) Debian/Ubuntu does a little different because they split Bacula in dynamic loadable parts to use the same main binaries with .so modules for different databases. In /usr/lib/bacula/ you can find the modules used and they include libpthread.so, so you are done. 2) else since the update, i have a client with a state of dir inserting attribute for 3 hours :( This Client have 1,8 To and 16 Millions of files. An idea if something change since ?? If you have done an inplace upgrade from 10.04 carefully check your database schema and index. As far as i remember there was a (Ubuntu) bug regarding Bacula database when doing OS upgrade. Also have a look here: http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server (checkpoint_segments) Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] concurrent jobs with multiple directors
Zitat von Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2012/11/29 Silver Salonen sil...@serverock.ee ** On Thursday 29 November 2012 15:43:32 Radosław Korzeniewski wrote: Hello, 2012/11/29 Silver Salonen sil...@serverock.ee sd1--dir1 --- client1 --- dir2--sd2 Is your client1 a Windows machine? If so, do you use VSS Enable = Yes in your FileSet resource? Yes. So, the current limitation of bacula-fd client on windows does not permit concurrent jobs. Sorry. So if I want them to run concurrently, I have to disable VSS on at least one of these jobs? You have to try. You have to disable VSS on all jobs for this Windows machine. Furthermore don't try this with data encryption or your FD will crash. If you configure both jobs without VSS and the FD without encryption it might work, but i still wonder if a multi director configuration is supported anyway. Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] (no subject)
Zitat von Luis H. Forchesatto luisforchesa...@gmail.com: Good morning Andreas. In my case both machines (client and storage/director) are linux OS. They are at the same network and the backup are transported via network. I think that way is better to keep one job per server, am I right? Thanks for answering me. :) If you have more than one I/O channel (multiple disk(s)) and your network is not saturated from one job you can try if your Bacula server (SD) is able to handle another one concurrently with raising throughput. No problem with Unix (Linux) but not usefull in all cases. Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] disaster recovery of the actual bacula server
Zitat von Jonathan Horne jho...@skopos.us: I am getting ready to deploy my first bacula server to our production environment. I would like to practice 2 scenarios of recovery. First, I will recover the configuration files and mysql database (from dump files), and hopefully continue on. The second, I will recover only the configuration files. Both scenarios, will tear down the server and rebuild new OS and reattach the LUN where the volume pools are stored. The first seems pretty straight forward, especially is the mysqldump is current after the most recent backups that ran. The 2nd, poses the question of how to import existing volume pool data into a new bacula director installation? You need the bscan utility to rebuild the catalog from your backup volumes: http://www.bacula.org/5.2.x-manuals/en/utility/utility/Volume_Utility_Tools.html#SECTION0027 Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula fatal error all the time
Zitat von b_...@mail.ru: On Nov 29, 2012, at 11:05 PM, Dan Langille d...@langille.org wrote: On Nov 29, 2012, at 2:26 PM, b_...@mail.ru wrote: Hello, I have a couple of hosts which backup through bacula. I have confronted with next situation, one host can't perform full backup, process of backup always finished with this error: Error: bsock.c:389 Write error sending 262144 bytes to Storage daemon:IP ADDRESS:9103: ERR=Broken pipe Fatal error: backup.c:1190 Network send error to SD. ERR=Broken pipe Error: Director's comm line to SD dropped Error: Bacula dir.bacula.HOSTNAME. 5.2.6 Time of occurrence of this error is not always the same, here are the logs: Elapsed time: 11 mins 56 secs Elapsed time: 1 hour 10 mins 18 secs Elapsed time: 47 mins 36 secs Elapsed time: 1 hour 14 mins 40 secs Elapsed time: 1 hour 40 mins 1 sec I don't think that this related to network issue. But client side have a lot of I/O operation, disk system is busy almost all the time. Maybe this is a cause. How can I solve this? I don't see any timeout directives which can help me. Client side is FreeBSD 7.2 amd64 bacula 5.2.6 HEADS UP. FreeBSD 7.2 was end-of-life'd in 2010 which means no security patches will be issued for it. Upgrading is recommended. see http://www.freebsd.org/security/#unsup That that matter, that's an old version of FreeBSD too. :) I know, but upgrade in this case is impossible and this is not related to our problem with bacula I think Server side (DIR and SD on the same host) is FreeBSD 9.0 amd64 bacula 5.2.6 (also tried 5.2.12 with the same result) Have you looked at trying the 'Heartbeat Interval' settings? You should try setting on the SD. There is also a 'Heartbeat Interval' on the FD, but that doesn't seem to be the error you're getting. yes, I have tried to play with Heartbeat Interval. Unfortunately doesn't help. Try with another NIC. We first had problems with our Bacula Server failing two clients out of ~20 with connection failures randomly. After ditching the Onboard GE (Marvell PHY) and using a PCIe NIC on the Server the problem went away. Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Catastrophic error. Cannot write overflow block to device
Zitat von Luca Bertoncello bertonce...@frischergehts.net: Hello, list! Since 3 days I cannot backup my server... I always get this error: 28-Nov 00:40 skynet-sd JobId 36: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device sata-changer-drive-0 (/var/lib/bacula/changer1/0/drive0). ERR=No space left on device28-Nov 00:40 skynet-sd JobId 36: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device sata-changer-drive-0 (/var/lib/bacula/changer1/0/drive0). ERR=No space left on device28-Nov 00:40 skynet-sd JobId 36: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device sata-changer-drive-0 (/var/lib/bacula/changer1/0/drive0). ERR=No space left on device28-Nov 00:40 skynet-sd JobId 36: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device sata-changer-drive-0 (/var/lib/bacula/changer1/0/drive0). ERR=No space left on device28-Nov 00:40 skynet-sd JobId 36: Fatal error: device.c:192 Catastrophic error. Cannot write overflow block to device sata-changer-drive-0 (/var/lib/bacula/changer1/0/drive0). ERR=N space left on device28-Nov 00:40 skynet-sd JobId 36: Job write elapsed time = 01:39:58, Transfer rate = 15.66 M Bytes/second 28-Nov 00:40 skynet-fd JobId 36: Fatal error: backup.c:1019 Network send error to SD. ERR=Connection reset by peer Hello could you explain what kind of device this is and how it is configured in bacula? Regards Andreas -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] (no subject)
Zitat von Luis H. Forchesatto luisforchesa...@gmail.com: Greetings. I'm running bacula and I got 3 jobs for a single server that backup the entire filesystem. Searching how to enable multiple jobs at the same time i found the Maximum Concurrent Jobs directive but also found this line: We recommend that you carefully test your multiple concurrent backup including doing thorough restore testing before you put it into production. Font: http://www.bacula.org/fr/dev-manual/Tips_and_Suggestions.html#SECTION0081818000 Which problems may occur when running multiple jobs from the same server? You need careful testing and understanding to not actually slow down your backups because of concurrency. If you do not use data encryption most of the time the I/O will be your bottleneck and it doesn't help to load the same I/O channel with more than one backup at once for example. If you want to do this with a Windows clients be aware that it doesn't work with neither VSS nor Data Encryption. If it is Unix with more than one I/O channel and a fast connection to your backup server you might get more speed with concurrent backups. Regards Andreas -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Catastrophic error. Cannot write overflow block to device
Zitat von Luca Bertoncello bertonce...@frischergehts.net: Am Wed, 28 Nov 2012 15:11:29 +0100 schrieb lst_ho...@kwsoft.de: could you explain what kind of device this is and how it is configured in bacula? I use 6 hard-disk with vchanger to simulate a tape-library. What do you need from the bacula configuration? Clearly Bacula has problem to write to a volume urgently needed, so you should first trouble-shoot your storage device(s) and have a look for vchanger errors. Regards Andreas -- Keep yourself connected to Go Parallel: INSIGHTS What's next for parallel hardware, programming and related areas? Interviews and blogs by thought leaders keep you ahead of the curve. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula tapes marked FULL too early - VolBytes too low
Zitat von Dan Langille d...@langille.org:
On Nov 22, 2012, at 9:40 AM, Jacky Carimalo wrote:
Le 20/11/2012 16:39, Tilman Schmidt a écrit :
Am 20.11.2012 17:14, schrieb Jacky Carimalo:
Le 20/11/2012 15:32, John Drescher a écrit :
Also you should avoid /dev/st0 and use /dev/nst0 since /dev/st0 can
cause corruption in your tapes.
But I have no choice :
bacula2-64:~/bacula/bin# lsscsi -g
[0:0:0:0]diskIFT A16F-R2221 347C /dev/sdb /dev/sg2
[0:0:1:0]tapeHP Ultrium 3-SCSI G24H /dev/st0 /dev/sg3
[0:0:1:1]tapeHP Ultrium 3-SCSI G24H /dev/st1 /dev/sg4
[0:0:1:2]mediumx OVERLAND NEO Series 0604 /dev/sch0 /dev/sg5
[0:0:1:3]process Over NEO VIA FCO 2.22 - /dev/sg6
...
I think you have. Just because /dev/nst0 isn't listed by lsscsi
doesn't mean it isn't there. Look:
[ts@backup ~]$ lsscsi -g
[0:0:0:0]cd/dvd HL-DT-ST DVDRAM GSA-H12N UL01 /dev/sr0 /dev/sg0
[2:0:0:0]diskATA ST3250620NS 3.AE /dev/sda /dev/sg1
[3:0:0:0]diskATA ST3250620NS 3.AE /dev/sdb /dev/sg2
[5:0:5:0]tapeHP Ultrium 3-SCSI G54W /dev/st0 /dev/sg3
[5:0:5:1]mediumx HP 1x8 autoloader 1.50 /dev/sch0 /dev/sg4
[ts@backup ~]$ ls -l /dev/*st0*
crw-rw 1 root tape 9, 128 19. Okt 16:59 /dev/nst0
crw-rw 1 root tape 9, 224 19. Okt 16:59 /dev/nst0a
crw-rw 1 root tape 9, 160 19. Okt 16:59 /dev/nst0l
crw-rw 1 root tape 9, 192 19. Okt 16:59 /dev/nst0m
crw-rw 1 root tape 9, 0 19. Okt 16:59 /dev/st0
crw-rw 1 root tape 9, 96 19. Okt 16:59 /dev/st0a
crw-rw 1 root tape 9, 32 19. Okt 16:59 /dev/st0l
crw-rw 1 root tape 9, 64 19. Okt 16:59 /dev/st0m
So, I used as you well indicates : /dev/nst0 and /dev/nst1.
Autochanger {
Name = OVL_LTO-3
Device = OVL_LTO-3_Drive-1, OVL_LTO-3_Drive-2
Changer Device = /dev/changerovllto3
Changer Command = /root/bacula/bin/mtx-changer %c %o %S %a %d
}
Device {
Name = OVL_LTO-3_Drive-1
Drive Index = 0
Media Type = LTO-3
AutoChanger = yes
Archive Device = /dev/nst1
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
LabelMedia = yes;
RandomAccess = yes;
Alert Command = sh -c 'tapeinfo -f %c |grep TapeAlert|cat'
}
Device {
Name = OVL_LTO-3_Drive-2
Drive Index = 1
Media Type = LTO-3
AutoChanger = yes
Archive Device = /dev/nst0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = no;
LabelMedia = yes;
RandomAccess = yes;
Alert Command = sh -c 'tapeinfo -f %c |grep TapeAlert|cat'
}
---
But there are still errors :
dmesg :
Nov 22 12:33:22 singleton.u06.univ-nantes.prive kernel:
[72917.185264] lpfc :08:00.0: 0:(0):0713 SCSI layer issued
Device Reset (1, 1) return x2002
Nov 22 12:33:22 singleton.u06.univ-nantes.prive kernel:
[72917.190481] st1: Sense Key : Unit Attention [current]
Nov 22 12:33:22 singleton.u06.univ-nantes.prive kernel:
[72917.190487] st1: Add. Sense: Bus device reset function occurred
...
---
with errors in bacula job :
2012-11-22 12:33:22 bacula.u11.univ-nantes.prive-
Error: block.c:590 Write error at 44:14976 on device
OVL_LTO-3_Drive-1 (/dev/nst1). ERR=Input/output error.
Error: Error writing final EOF to tape. This Volume may not be readable.
dev.c:1566 ioctl MTWEOF error on OVL_LTO-3_Drive-1 (/dev/nst1).
ERR=Input/output error.
...
---
I will try cleaning procedures as last solution for me ?
Have we already determined that the SCSI chain is properly terminated?
[0:0:1:3]process Over NEO VIA FCO 2.22 - /dev/sg6
Looks like FibreChannel to me...
But for sure the OP should check cabeling and to use the recommended
Firmware on both ends.
Regards
Andreas
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula tapes marked FULL too early - VolBytes too low
Zitat von Jacky Carimalo jacky.carim...@univ-nantes.fr: Le 20/11/2012 15:32, John Drescher a écrit : On Tue, Nov 20, 2012 at 9:30 AM, John Drescher dresche...@gmail.com wrote: On Tue, Nov 20, 2012 at 10:15 AM, Jacky Carimalo jacky.carim...@univ-nantes.fr wrote: I put Volume Use Duration = 36 months, but always having sometimes : Error: block.c:590 Write error at 88:2499 on device OVL_LTO-3_Drive-2 (/dev/st0). ERR=Erreur d'entrée/sortie. Error: Error writing final EOF to tape. This Volume may not be readable. dev.c:1566 ioctl MTWEOF error on OVL_LTO-3_Drive-2 (/dev/st0). ERR=Erreur d'entrée/sortie. Look at your kernel logs / dmesg for scsi or tape errors. Also you should avoid /dev/st0 and use /dev/nst0 since /dev/st0 can cause corruption in your tapes. John But I have no choice : bacula2-64:~/bacula/bin# lsscsi -g [0:0:0:0]diskIFT A16F-R2221 347C /dev/sdb /dev/sg2 [0:0:1:0]tapeHP Ultrium 3-SCSI G24H /dev/st0 /dev/sg3 [0:0:1:1]tapeHP Ultrium 3-SCSI G24H /dev/st1 /dev/sg4 [0:0:1:2]mediumx OVERLAND NEO Series 0604 /dev/sch0 /dev/sg5 [0:0:1:3]process Over NEO VIA FCO 2.22 - /dev/sg6 ... If it is Linux have a look in /dev/tape/by-id/ For example in our case: lrwxrwxrwx 1 root root 10 Nov 19 18:23 scsi-350014380032b4c68 - ../../sg10 lrwxrwxrwx 1 root root 9 Nov 19 18:23 scsi-350014380032b4c69 - ../../st0 lrwxrwxrwx 1 root root 10 Nov 19 18:23 scsi-350014380032b4c69-nst - ../../nst0 so we choose /dev/tape/by-id/scsi-350014380032b4c69-nst which has also the advantage of not changing if devices are added/removed as it is the case with /dev/sgX. Regarding your problem with the device/bus reset you should carefully check cabeling and adapter firmware/driver. Regards Andreas -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File jobs purged when they should not
Zitat von Felip Moll lip...@gmail.com:
Hi people,
Recently I found a problem when trying to recover my files from a Job. The
Files of a Job are purged when they shouldn't be.
###
### RELEVANT CONFIGURATIONS
###
Job {
Name = Backup Dades : Dominator 6m
Client = dominator-fd
JobDefs = JobTemplate
Fileset = home
Schedule = CicleMensual-6mesos-22h
Storage = Autochanger
Pool = Cintes-Data-6mesos
}
Schedule {
Name = CicleMensual-6mesos-22h
Run = Full january 1 at 22:00
Run = Full july 1 at 22:00
Run = Incremental feb-jun 1 at 22:00
Run = Incremental aug-dec 1 at 22:00
}
Client {
Name = dominator-fd
Address = dominator.domain.com
FDPort = 9102
Catalog = MyCatalog
Password = # password for FileDaemon
File Retention = 10 years # 10 years
Job Retention = 10 years # six months
AutoPrune = yes # Prune expired Jobs/Files
}
Pool {
Name = Cintes-Data-6mesos
Pool Type = Backup
Recycle = yes
AutoPrune = yes
File Retention = 1 month
Job Retention = 6 months
Volume Retention = 12 months
}
###
###
###
###
### What happens when I try to restore
###
Enter JobId(s), comma separated, to restore: 7444
You have selected the following JobId: 7444
Building directory tree for JobId(s) 7444 ...
For one or more of the JobIds selected, no files were found,
so file selection is not possible.
Most likely your retention policy pruned the files.
Do you want to restore all the files? (yes|no): no
You should be aware of the priority for purging. From my knowledge
the pool purging has priority against the setting for the client.
Furthermore the purge is inclusive as purging jobs do purge all
related file records and so on. For your example you have a File
Retention = 1 month and a Job Retention = 6 months which means
after 1 month you can only restore the full job without file selection
(that's what bacula suggested).
Regards
Andreas
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use modify time, not access time
Zitat von Wouter van Marle wou...@squirrel-systems.com: On Fri, 2012-11-16 at 15:46 +1100, Gary R, Schmidt wrote: On 16.11.2012 13:52, Wouter van Marle wrote: [SNIP] Result: yesterday I moved a big mail folder with subfolders, causing the You *moved* a directory, and you are surprised that the files in it are being backed up? Sorry I was incomplete. Files were accidentally moved, and subsequently moved back to the exact original location. Wouter. From the manual: The File daemon (Client) decides which files to backup for an Incremental backup by comparing start time of the prior Job (Full, Differential, or Incremental) against the time each file was last modified (st_mtime) and the time its attributes were last changed(st_ctime). If the file was modified or its attributes changed on or after this start time, it will then be backed up. And as far as i understand http://www.kernel.org/doc/man-pages/online/pages/man2/stat.2.html st_ctime is changed by changing inode information, which is used to set the location in the directory tree. So i guess moving around a directory indeed will include it in the next backup. Regards Andreas -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use modify time, not access time
Zitat von Dan Langille d...@langille.org: On 2012-11-15 23:46, Gary R, Schmidt wrote: On 16.11.2012 13:52, Wouter van Marle wrote: [SNIP] Result: yesterday I moved a big mail folder with subfolders, causing the You *moved* a directory, and you are surprised that the files in it are being backed up? If I recall correctly, this has long been described as a 'bug' by some, but not as you describe. From http://www.bacula.org/5.0.x-manuals/en/main/main/Configuring_Director.html In addition, if you move a directory rather than copy it, the files in it do not have their modification time (st_mtime) or their attribute change time (st_ctime) changed. As a consequence, those files will probably not be backed up by an Incremental or Differential backup which depend solely on these time stamps. If you move a directory, and wish it to be properly backed up, it is generally preferable to copy it, then delete the original. What we are seeing here is the opposite of that. Bacula has never seen these files in that location before, if it *didn't* back them up that would be a total failure. What is described above is not the expected behavior. In fact, it's the opposite of the expected behavior, given that Bacula, by default, use mtime to decide what to backup with an incremental/differential. It does not use 'have I seen this file here before'. I believe that's an 'accurate backup' option. You are right. On second thought it is more obvious that st_ctime is *not* changed because the inode pointing to the directory is changed on move, not the directory inode itself i guess. So maybe the OP has done a mv across mount points? On what filesystem was this behaviour observed by the way? Regards Andreas -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with backup: always full backup started
Zitat von Luca Bertoncello bertonce...@frischergehts.net:
Hello,
I configured Bacula (version 5.0.1 from Ubuntu-repositories) on an
Ubuntu-Lucid server at office.
The director is configured to start a full backup between Monday and
Tuesday, and an incremental backup between Tuesday and Friday.
Here die configuration:
Job {
Name = Skynet
JobDefs = JobSkynet
Enabled = yes
RunBeforeJob = /etc/bacula/updateSlots.sh
RunAfterJob = /usr/local/sbin/bacula2nagios \%n\ 0 \%e %l %v\
RunAfterFailedJob = /usr/local/sbin/bacula2nagios \%n\ 1 \%e %l
%v\
}
JobDefs {
Name = JobSkynet
Type = Backup
Level = Incremental
Client = skynet-fd
FileSet = Skynet-Set
Schedule = WeeklyCycle
Storage = changer1
Messages = Standard
Pool = File
Priority = 10
Write Bootstrap = /var/lib/bacula/%c.bsr
}
Schedule {
Name = WeeklyCycle
Run = Full mon at 22:00
Run = Incremental tue-fri at 23:00
}
Now, this Monday was started a full backup, and it runs without any
problem.
Yesterday should an INCREMENTAL backup started, but it runs a full one.
In the report E-Mail I can see:
13-Nov 23:00 skynet-dir JobId 16: No prior Full backup Job record found.
13-Nov 23:00 skynet-dir JobId 16: No prior or suitable Full backup
found in catalog. Doing FULL backup.
Now, why cannot bacula find the previous full backup?
Hello
have you checked if the fileset has not been altered after starting
the Full backup? Try to start a incremental by hand after a full
backup to see what's happen.
Regards
Andreas
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] OpenSSL error building Bacula 5.2.12
Zitat von Romo, Abel abel.r...@bnc.ca: Thanks for your suggestion Jummo. I tried --with-openssl=/usr/include/openssl as you recommended Same thing. Tried --with-openssl with no parameter... exact same error. :O( Abel Romo Hello have you checked if the header files are installed at all? On most distributions today you have to install something like packagename-dev to get the development headers. Regards Andreas -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Limiting only despooling|writing concurrent jobs.
Zitat von ALyarskiy bacula-fo...@backupcentral.com: Zitat von ALyarskiy bacula-forum at backupcentral.com: Zitat von ALyarskiy bacula-forum at backupcentral.com: How can i limit writing jobs to 1 without limiting spooling jobs? I want to have only one job writing on the tape but spool other jobs in parallel. This is the default as far as i understand. The tape is locked exclusive from one writer while other jobs which run concurrently will spool data as long as the maximum spool size is not reached. Works fine here but you need a really fast spool area if you want to feed something like LTO-4/5/6 at full speed. Regards Andreas It is true for jobs with SpoolData=yes, but i have few jobs that have SpoolData=no, so they write data right on the tape. I want to limit jobs that have SpoolData=no by 1 concurrent, so only 1 job could write data on the tape, and permit the rest jobs with SpoolData=yes to spool data on local drive concurrently (5-7 streams). You might try the Maximum Concurrent Jobs directive for the tape device but i guess with this you will block the despooling until all non spooling jobs have passed. Why do you want to have spooling and non-spooling jobs anyway for the same device? Regards Andreas Yes, now i use Maximum Concurrent Jobs limitation on device level. I want to write large jobs right on tape ( usually 50 Gb) - there are no network bottlenecks for this clients, and the rest (there are may be networks bottlenecks sometimes) spool to the local drive. When local spool used backup process have to read from client, write on local drive, then read from it and write the backup to the tape. For large backups it is HUGE overhead. Also there is problem when you have 4-5 large backups in same time - local drive becomes bottleneck (i have 2 Gb network). So it is reasonable to have a possibility to limit concurrent jobs at different levels. For example - 6 Jobs running in same time, 2 of them are large and writing directly to the tape one by one, rest 4 are spooling to local drive and waiting those 2 to complete. It all depends. Spooling data is nearly no overhead if you use some cheap local storage as you should. Plugin some SATA or if you can afford SSD, do a RAID0 and spooling will give you the following benefits: - Data on tape is written with the spool size per job so your restore will be a lot faster - You will be able to drive your tape at maximum speed regardles of network congestion or clients searching files and therefore prevent shoe-shining Of course it would be nice to have some additional controls in some situations, but it looks like no one have written code for this until now. Regards Andreas -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Volume has been set to FULL and not USED
Zitat von Sven Tegethoff tegeth...@udobaer.de: * This means that I've 4 volumes damaged? I tried this night with a new volume and the backup finished correctly. All the logs tell us is that the tape drive could not write to the tape - without further investigation, it's impossible to say whether the tapes are at fault, the tape drive is becoming unreliable, or if there's something wonky with your controller or cabling. * why bacula doesn't sent me a mail with that error? From a logical point of view: Because the director only sends mails at the END of a job, or when a job needs intervention. The write error did not terminate the job, nor did it require intervention - failing tapes are not uncommon, so the strategy is to just continue on the next tape. Ideally, you always have an extra volume in the scratch pool as a spare, and the situation gets resolved automatically, even when it happens at 3am without any human presense. So the behaviour makes sense in a way. That being said, I agree it would be very nice in some situations if there was a feature to receive extra email notifications for I/O-Errors instead of relying on the user to scan their daily logs for it. You might have a look at the Alert Command in the SD configuration: http://www.bacula.org/5.2.x-manuals/en/main/main/Storage_Daemon_Configuratio.html#SECTION00243 With this you should get tape errors reported after the job if your drive is willing to tell. Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula query
Zitat von krishna pawankar krishnapawan...@yahoo.co.in: Hi All, I want to install bacula file daemon client on IBM AIX system with version Bacula 5.2.6. But i am not able to install it. Can anybody guide me in this case since i have used the same file daemon client binary on Centos wihout any problems. Linux and AIX are *not* binary compatible even if the CPU would be the same (Power).You have to compile a working binary on AIX but you should use the latest 5.2.12 of course. Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Priority Problem / Question
Zitat von Uwe Schuerkamp uwe.schuerk...@nionex.net: Hi folks, I'm having a problem understanding job priorities. We don't allow mixed jobs to run concurrently, so based on that I've set up my catalog backup at the lowest priority (25) while regular backups run at 10, some at prio 15 to assure the catalog backup won't start before all regular backups are finished. This works ok on the end of the backup run, but as our database is quite large (170GB), the backup job itself can run well into the incrementals start time (usually 17.00), and here's the problem: Instead of waiting for the backupcatalog job to finish, bacula (5.2.12 compiled from source on centos 6) happily fires off the prio 10 level incrementals. As we shut down the mysql database for the duration of the backup (via runbeforejob), this of course leads to havoc with bacula being unable to select / insert stuff from and into the database resulting in all backup jobs to fail ungracefully. Question: Shouldn't bacula wait for the prio25 job to finish before firing off the prio10 incrementals when mixed prio runs are not allowed? Hello from my knowledge the jobs are created but take on hold as long as other jobs with different priority are running. So it looks like the catalog connections are established as part of the job creation. That said you should not need to stop the catalog database for the dump, in fact i wonder that it works at all to backup the catalog this way. If you are in doubt about taking online dumps you can also use PostgreSQL which might be a good choice at that scale anyway ;-) Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula director consumes all CPU after full backup
Zitat von Felix Schwarz felix.schw...@oss.schwarz.eu: Hi Luis, Am 03.11.2012 12:57, schrieb Luis H. Forchesatto: Quantos arquivos são backupeados? How many files are backed up? (I used Google translate, let's hope I understood you correctly) I checked the base backup job and that one had ~700k files with a total of ~65 GB (uncompressed). That shouldn't be too hard for bacula, isn't it? No, we have around 3.5 billion files with some jobs with not problem, but we use PostgreSQL as backend db and don't do base jobs. You should have a look at your sqlite settings and first test without base jobs. If your backups work add base jobs to the setup and see what happens. Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula director consumes all CPU after full backup
Zitat von Radosław Korzeniewski rados...@korzeniewski.net: Hello, 2012/11/5 lst_ho...@kwsoft.de No, we have around 3.5 billion files with some jobs with not problem, Wow, 3.5 _billion_ files on a single job. Amazing. Sorry, confused by the million/billion (short/long scale). Should have been more precise with 3.5 x 10^6 (german million not milliarde) but we use PostgreSQL as backend db and don't do base jobs. Could you share some information about your setup. It is very interesting how Bacula and PostgreSQL can handle this kind of _huge_ backup job. What hardware? What archive device (disk? tape? both?). I can imagine that you should have no less then 10 _billion_ files in your catalog. The largest Bacula deployment I ever see was a few hundred million files in a whole catalog, not a single job. So we are, depending on the short/long scale you have in mind by factor 1000 or 100 smaller than expected ;-) The real numbers are only 1.45 TB and 3.500.000 files for one machine. The others are somewhat smaller but there is still another with 1.200.000 files. The whole catalog is around 7GB as of now, so no problem. So let's repeat: I will never use ambiguous billion again... Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Splitting a job into two parallel jobs
Zitat von Chris Adams cmad...@hiwaay.net: Once upon a time, lst_ho...@kwsoft.de lst_ho...@kwsoft.de said: What about using more than one snapshot? If done right snapshots are cheap, so you should have no problem to take a snapshot/mount per job to get two jobs running. But i doubt you will gain much with parallel backups on one filesystem/io-channel anyway. I'll have to check the space (not sure it is available right now). I do know that my old backup software did run the same backup (with parallel trees) somewhat faster (and that was to tape rather than disk). If your client filesystem is slow it does not matter if you backup to disk or tape. Parallel backups on such filesystems only help if you don't saturate the io-channels of this filesystem because of CPU or other bottleneck. Currently with VSS you have a limit of one job per windows machine running, as we already find out :-( I thought that might be the issue. I may be able to work around this by not using VSS for these particular trees, at least for testing. But as said if you don't have independent io-channels or you are CPU bound (data encryption) you won't gain very much by running more than one job concurrently. I used multiple parallel trees with my previous backup software, in part because it seemed to have a high overhead with lots of files (one of these servers has a Maildir store with a couple of million files). Those backups did finish faster (as part of my overall network backup to tape) than my current test backups (just running a single job to disk, with an upgraded backup server and upgraded switches in-between the servers). That's what got me looking at trying to run parallel jobs with Bacula (at least to test out the speed to see if it helped). Backup software need to maintain metadata per file which create some overhead per file obviously. In case of Bacula this metadata are stored in the director database, so maybe you should have a look if this database is configured for high insert rate. If the database in question is at its max you will gain nothing by inserting in parallel with more than one job. Regards Andreas -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
