Re: [Bacula-users] Runafterjob issue
On Thu, Oct 04, 2012 at 12:38:44PM +0800, Wouter van Marle wrote: [...] RunAfterJob = echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup %v' | batch Running this command (replacing the %v with an actual file name of course) from the command line works fine. The process is scheduled by batch to run as soon as system load allows (usually instantly) and it's running in the background, sending output to user bacula which ultimately ends up in my mail box. But running from Bacula fails. In the result e-mailed to me when the job is finished I see this: 04-Oct 05:05 acorn.squirrel-dir JobId 34: AfterJob: /usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup Squirrel-BackupCatalog.2012-10-04_05.05.00_04 | batch The `echo` part is gone, and it didn't run. An obvious hack would be to create a shell script that is called by Bacula and that in turn starts the upload, but that shouldn't be necessary. This is a typical problem: the string to execute is not passed through any shell but rather executed directly, so in your case the OS probably finds /bin/echo and passes it three arguments, /usr/local/bin/glacier-cmd ... 00-04, | and batch, which echo happily prints to its stdout and exits successfully. To not mess with writing the real shell script, consider embedding a call to the shell to your script string, like this: RunAfterJob = /bin/sh -c \/that/cmd ... '%v' | batch\ -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Network error with FD during Backup: ERR=Connection reset by peer
Hi Tom, Thank you for your answer. The heartbeats are only setup when a job with a client is initiated. So, there should be no activity when no job is running. When you initiate a job with the client, the director sets up a connection with the client telling the client what storage daemon to use. The client then initiates a connection back to that storage daemon. If you have the heartbeat settings in place as you do then you should see heartbeat packets sent from the client back to the director in order to keep that connection alive while the data is being sent back to the storage daemon. In addition, you may see heartbeat packets send from the storage daemon to the client. I'd have to re-look at the code but I believe this is used in the scenario where the storage daemon is waiting for a volume to write the data to (i.e. operator intervention). If the heartbeat setting is on then the storage daemon will send heartbeats back to the client in order to keep the connection alive while it waits. Yesterday I waited for the job to finish the first tape and then wait for me to insert the next one. I opened wireshark to see if there is a heartbeat during waiting - and there was none. During the job the heartbeat was active. From what you wrote the heartbeat should be active when waiting for a tape. Could you try to confirm that (have a look at the code)? As one side of the backup is a VMware server I had a closer look to the configuration of this environment. As far as I know Michael's environment (the starter of this thread) is also including VMware. So this might be interesting for him. My job cancels exactly 15 min after entering the wait mode for a new tape. In the VMware settings there is an idle timeout set to 900 sec (i.e. 15 min). The timeout doesn't exactly fit to that kind of connection, but you never know. I disabled this timeout now and restarted my backup. In 7 hours I will see the result. But even if this setting caused the trouble, I would have thought the heartbeat should solve this (idle connection timeout). Again, it would be good to know if the heartbeat should be active during waiting for a tape. Thank you again. Markus -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Runafterjob issue
That works, thanks for the quick reply! Would be nice to have this mentioned in the docs, if it's there I totally missed it. Writing a real script is of course easy, it's just messy and harder to see what's going on if I revisit this which may be years from now. Wouter. To not mess with writing the real shell script, consider embedding a call to the shell to your script string, like this: RunAfterJob = /bin/sh -c \/that/cmd ... '%v' | batch\ -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Runafterjob issue
Related issues I just ran into improving my script. The RunAfterJob now reads (one line): RunAfterJob = /bin/sh -c \echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup %v \\\%j (%l, since %s)\\\ ' | batch\ Which results in these messages (two lines): 04-Oct 21:43 acorn.squirrel-dir JobId 44: shell command: run AfterJob /bin/sh -c echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup \Users.2012-10-04_21.43.17_03 (Incremental, since 2012-10-04 21:38:58)\ ' | batch 04-Oct 21:43 acorn.squirrel-dir JobId 44: AfterJob: Users.2012-10-04_21.43.17_03: -c: line 0: unexpected EOF while looking for matching `'' But when run from the command prompt, the command works just fine (one line): $ /bin/sh -c echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup \Users.2012-10-04_21.43.17_03 (Incremental, since 2012-10-04 21:38:58)\ ' | batch And the %v translation does not work properly (the whole %-translation thing seems to be hit and miss, I had several more issues when trying to get some sample outputs and so). When I put '/backup/bacula/%v' in the script, it does NOT replace the %v with the actual file name. Without the path, it does the replacement. But I need the path for the command to actually work! It seems that going for a second script is the only way out here :-( And I'll think of some improvement of the docs. Adding an example will probably do the job, both by clarifying the issue and by giving it more focus. Wouter. On Thu, 2012-10-04 at 17:40 +0400, Konstantin Khomoutov wrote: On Thu, 04 Oct 2012 20:44:33 +0800 Wouter van Marle wou...@squirrel-systems.com wrote: To not mess with writing the real shell script, consider embedding a call to the shell to your script string, like this: RunAfterJob = /bin/sh -c \/that/cmd ... '%v' | batch\ That works, thanks for the quick reply! Would be nice to have this mentioned in the docs, if it's there I totally missed it. Writing a real script is of course easy, it's just messy and harder to see what's going on if I revisit this which may be years from now. http://bit.ly/O7z6SU reads this: In addition, the command string is parsed then fed to the OS, which means that the path will be searched to execute your specified command, but there is no shell interpretation, as a consequence, if you invoke complicated commands or want any shell features such as redirection or piping, you must call a shell script and do it inside that script. I would say the wording is imperfect. If you can propose a better one, feel free to file a feature request with http://bugs.bacula.org/ -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] wilddir not working as expected
I have following fileset:
FileSet {
Name = Web Crawler
Include {
Options {
signature = MD5
compression = GZIP9
exclude = yes
wilddir = target
}
File = /home/crawler
}
Exclude {
File = /home/crawler/nutch-1.3/runtime/local/segments
File = /home/crawler/solr/example/solr/nutch/data
File = /home/crawler/.m2/repository
}
}
idea is not to backup directories which have target in name like:
./conf/target
./packages/tools/target
./packages/info/target
./packages/plugins/target
./info/target
./src/target
./src/plugin/index-basic/target
./src/plugin/parse-html/target
./src/plugin/urlnormalizer-pass/target
./src/plugin/parse-js/target
./src/plugin/microformats-reltag/target
./src/plugin/protocol-ftp/target
./src/plugin/urlnormalizer-basic/target
but it didn't work, all target directories are backed up. Can anybody
spot a mistake?
--
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problems with make-mysql-tables script
Here is the error message: ERROR 1064 (42000) at line 415: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '@)' at line 1 Creation of MySQL tables succeeded. Line 415 from make_mysql_tables script : JobId integer NOT NULL, The script was taken from latest Bacula release. I'm using this command, when I running the script - ./make_mysql_tables -p 2012/10/3, Dan Langille d...@langille.org: On 2012-10-01 13:35, George Sokolov wrote: Hello, I got problems with make_mysql_tables script. When I running the script, I got the message about SQL syntax error, under the line 414 or 415. My Bacula version is 5.2.6 from Debian back ports repository. MySQL version is 5.1 Any ideas ? Thanks in advance. 5.2.12 is the latest, first, I'd go to that version. Yes, I know how some operating systems lag behind. I'm sorry about that. Can you paste the error message? Lines 414 or 415, whatever it was... What command did you issue when using make_mysql_tables? -- Dan Langille - http://langille.org/ -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Network error with FD during Backup: ERR=Connection reset by peer
Yesterday I waited for the job to finish the first tape and then wait for me to insert the next one. I opened wireshark to see if there is a heartbeat during waiting - and there was none. During the job the heartbeat was active. From what you wrote the heartbeat should be active when waiting for a tape. Could you try to confirm that (have a look at the code)? Marcus, I think that you should be seeing heartbeats in this case. What version of the Storage Daemon server are you running? I am looking at 5.2.10 and up as far as the code. Can you run it in debug mode? If so, set the debug level to 400 and you should get some messages in the output if the heartbeat logic is working. The heartbeat is sent from inside this method: /* * Wait for SysOp to mount a tape on a specific device. Returns: W_ERROR, W_TIMEOUT, W_POLL, W_MOUNT, or W_WAKE */ int wait_for_sysop(DCR *dcr) Inside that method, there is a particular debug line: Dmsg0(dbglvl, Send heartbeat to FD.\n); Anyhow, if you're not seeing this debug output then it is not sending a heartbeat for whatever reason. If you see it then it is sending it so the problem lies elsewhere if you're still not seeing it arriving at it's destination. hope this helps, --tom -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Simultaneousky backup to local HDD and Amazon S3
Hello everyone, I'm a newbie to Bacula and want to verify whether my requirement can get done using bacula. I want to know whether it's possible to simultaneously make backups to local HDD and Amazon S3 from the same source in Bacula. Can someone please help me with this? Thanks in advance. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Simultaneousky backup to local HDD and Amazon S3
On Thu, Oct 4, 2012 at 8:59 PM, Pubudu Perera suharsha...@gmail.com wrote: Hello everyone, I'm a newbie to Bacula and want to verify whether my requirement can get done using bacula. I want to know whether it's possible to simultaneously make backups to local HDD and Amazon S3 from the same source in Bacula. Can someone please help me with this? Thanks in advance. I would backup to the local drive then mirror that with rsync to S3 via s3fs fuse. John -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Simultaneousky backup to local HDD and Amazon S3
Thank John! So, that means there's no way to does the job simultaneously ? On Fri, Oct 5, 2012 at 7:05 AM, John Drescher dresche...@gmail.com wrote: On Thu, Oct 4, 2012 at 8:59 PM, Pubudu Perera suharsha...@gmail.com wrote: Hello everyone, I'm a newbie to Bacula and want to verify whether my requirement can get done using bacula. I want to know whether it's possible to simultaneously make backups to local HDD and Amazon S3 from the same source in Bacula. Can someone please help me with this? Thanks in advance. I would backup to the local drive then mirror that with rsync to S3 via s3fs fuse. John -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
