Hello,
On Wed, 2009-05-27 at 14:28 +0200, Kern Sibbald wrote:
> It is quite possible that mysqldump creates a much larger dump file
> particularly if you have compression turned on.
>
> Perhaps you filled up your output filesystem.
definitely NO
>
> We have tested this many times and never had such problems. You might try
> testing with a much smaller database to ensure that you are using the proper
> syntax, ... before working with such a large database.
>
> One possibility is that some value in the plugin is limited to 32 bits (i.e.
> a
> bit less than 4GB).
>
I tried to dump single bacula catalog table,
it worked without problems, FD Bytes Written are than 4 GB,
Plugin = "bpipe:/MYSQL/bacula.sql:mysqldump -f --opt bacula File:gzip >
/var/tmp/bacula-File.sql.gz"
I feel helpless, dumping large File table (more than 10GB uncompressed)
works in bpipe mode. But dumping the whole bacula database using bpipe
results in incomplete dump.
Here is the job log using the above Plugin line.
troll-dir JobId 25217: Start Backup JobId 25217,
Job=Backup-CatalogPipe.2009-05-27_15.08.03_11
troll-dir JobId 25217: Using Device "FileStorage"
troll-sd JobId 25217: Volume "Backup-0615" previously written, moving to end of
data.
troll-sd JobId 25217: Ready to append to end of Volume "Backup-0615"
size=30604524317
troll-sd JobId 25217: Job write elapsed time = 00:25:26, Transfer rate = 2.696
M bytes/second
troll-dir JobId 25217: Bacula troll-dir 3.0.1 (30Apr09): 27-May-2009 15:33:31
Build OS: i686-pc-linux-gnu redhat Enterprise release
JobId: 25217
Job: Backup-CatalogPipe.2009-05-27_15.08.03_11
Backup Level: Full
Client: "troll-fd" 3.0.1 (30Apr09)
i686-pc-linux-gnu,redhat,Enterprise release
FileSet: "CatalogPipe" 2009-05-27 11:30:04
Pool: "DiskBackup" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "File" (From Job resource)
Scheduled time: 27-May-2009 15:07:59
Start time: 27-May-2009 15:08:05
End time: 27-May-2009 15:33:31
Elapsed time: 25 mins 26 secs
Priority: 11
FD Files Written: 1
SD Files Written: 1
FD Bytes Written: 4,114,114,420 (4.114 GB)
SD Bytes Written: 4,114,114,606 (4.114 GB)
Rate: 2696.0 KB/s
Software Compression: 70.1 %
VSS: no
Encryption: no
Accurate: no
Volume name(s): Backup-0615
Volume Session Id: 924
Volume Session Time: 1242371530
Last Volume Bytes: 34,721,566,627 (34.72 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
troll-dir JobId 25217: Begin pruning Jobs.
troll-dir JobId 25217: No Jobs found to prune.
troll-dir JobId 25217: Begin pruning Files.
troll-dir JobId 25217: No Files found to prune.
troll-dir JobId 25217: End auto prune.
Regards
Ulrich
> Regards,
>
> Kern
>
> On Wednesday 27 May 2009 13:35:53 Ulrich Leodolter wrote:
> > Hello,
> >
> > I tried to setup bpipe plugin for Catalog backup.
> > Below you can see FileSet/Job config, Backup/Restore logs.
> > Everything looks ok, but mysqldump is incomplete!
> >
> > Running "mysqldump -f --opt bacula > bacula.sql" from shell
> > results in 14G bacula.sql file.
> >
> > /MYSQL/bacula.sql is only 3,325,644,283 bytes as u can see in
> > restore log.
> >
> > Here are the last lines of restored file /var/tmp/bacula.sql.gz,
> > last lines look ok, but dump file is definitely incomplete.
> >
> > A A ou BAA I BKEH+e BEE6La BJWk5Q A A E','dBR5u4Hk9YoQc9qFPSQEfvDv4Rg');
> > /*!40103 SET time_zo...@old_time_zone */;
> >
> > /*!40101 SET sql_mo...@old_sql_mode */;
> > /*!40014 SET foreign_key_chec...@old_foreign_key_checks */;
> > /*!40014 SET unique_chec...@old_unique_checks */;
> > /*!40101 SET character_set_clie...@old_character_set_client */;
> > /*!40101 SET character_set_resul...@old_character_set_results */;
> > /*!40101 SET collation_connecti...@old_collation_connection */;
> > /*!40111 SET sql_not...@old_sql_notes */;
> >
> > -- Dump completed on 2009-05-27 10:56:00
> >
> >
> >
> > It looks like mysqldump called from bpipe terminates/stops
> > while dumping File.
> >
> > [r...@troll tmp]# zgrep "CREATE TABLE" /var/tmp/bacula.sql.gz
> > CREATE TABLE `BaseFiles` (
> > CREATE TABLE `CDImages` (
> > CREATE TABLE `Client` (
> > CREATE TABLE `Counters` (
> > CREATE TABLE `Device` (
> > CREATE TABLE `File` (
> > [r...@troll tmp]#
> >
> >
> > Regards
> > Ulrich
> >
> > -------
> >
> >
> > FileSet {
> > Name = CatalogPipe
> > Ignore FileSet Changes = yes
> > Include {
> > Options {
> > signature = MD5
> > compression = GZIP
> > }
> > Plugin = "bpipe:/MYSQL/bacula.sql:mysqldump -f --opt bacula:gzip >
> > /var/tmp/bacula.sql.gz" }
> > }
> >
> > Job {
> > Name = Backup-CatalogPipe
> > Type = Backup
> > Level = Full
> > Client = troll-fd
> > FileSet = "CatalogPipe"
> > Storage = File
> > Messages = StandardMailOnError
> > Pool = DiskBackup
> > Write Bootstrap = "/opt/bacula/working/%n.bsr"
> > Priority = 11
> > SpoolData = no
> > Allow Duplicate Jobs = no
> > }
> >
> >
> >
> >
> > troll-dir JobId 25211: Start Backup JobId 25211,
> > Job=Backup-CatalogPipe.2009-05-27_12.48.41_59
> > troll-dir JobId 25211: Using Device "FileStorage"
> > troll-sd JobId 25211: Volume "Backup-0615" previously written, moving to
> > end of data. troll-sd JobId 25211: Ready to append to end of Volume
> > "Backup-0615" size=29564841571 troll-sd JobId 25211: Job write elapsed time
> > = 00:07:17, Transfer rate = 2.290 M bytes/second troll-dir JobId 25211:
> > Bacula troll-dir 3.0.1 (30Apr09): 27-May-2009 12:56:00 Build OS:
> > i686-pc-linux-gnu redhat Enterprise release JobId:
> > 25211
> > Job: Backup-CatalogPipe.2009-05-27_12.48.41_59
> > Backup Level: Full
> > Client: "troll-fd" 3.0.1 (30Apr09)
> > i686-pc-linux-gnu,redhat,Enterprise release FileSet:
> > "CatalogPipe" 2009-05-27 11:30:04
> > Pool: "DiskBackup" (From Job resource)
> > Catalog: "MyCatalog" (From Client resource)
> > Storage: "File" (From Job resource)
> > Scheduled time: 27-May-2009 12:48:34
> > Start time: 27-May-2009 12:48:43
> > End time: 27-May-2009 12:56:00
> > Elapsed time: 7 mins 17 secs
> > Priority: 11
> > FD Files Written: 1
> > SD Files Written: 1
> > FD Bytes Written: 1,001,064,037 (1.001 GB)
> > SD Bytes Written: 1,001,064,218 (1.001 GB)
> > Rate: 2290.8 KB/s
> > Software Compression: 69.9 %
> > VSS: no
> > Encryption: no
> > Accurate: no
> > Volume name(s): Backup-0615
> > Volume Session Id: 918
> > Volume Session Time: 1242371530
> > Last Volume Bytes: 30,566,617,501 (30.56 GB)
> > Non-fatal FD errors: 0
> > SD Errors: 0
> > FD termination status: OK
> > SD termination status: OK
> > Termination: Backup OK
> >
> > troll-dir JobId 25211: Begin pruning Jobs.
> > troll-dir JobId 25211: No Jobs found to prune.
> > troll-dir JobId 25211: Begin pruning Files.
> > troll-dir JobId 25211: No Files found to prune.
> > troll-dir JobId 25211: End auto prune.
> >
> >
> >
> > troll-dir JobId 25212: Start Restore Job
> > RestoreFiles.2009-05-27_12.57.43_02
> > troll-dir JobId 25212: Using Device "FileStorage"
> > troll-sd JobId 25212: Ready to read from volume "Backup-0615" on device
> > "FileStorage" (/disk0/bacula/files). troll-sd JobId 25212: Forward spacing
> > Volume "Backup-0615" to file:block 6:3795037795. troll-sd JobId 25212: End
> > of Volume at file 7 on device "FileStorage" (/disk0/bacula/files), Volume
> > "Backup-0615" troll-sd JobId 25212: End of all volumes.
> > troll-dir JobId 25212: Bacula troll-dir 3.0.1 (30Apr09): 27-May-2009
> > 13:00:19 Build OS: i686-pc-linux-gnu redhat Enterprise
> > release JobId: 25212
> > Job: RestoreFiles.2009-05-27_12.57.43_02
> > Restore Client: troll-fd
> > Start time: 27-May-2009 12:57:45
> > End time: 27-May-2009 13:00:19
> > Files Expected: 1
> > Files Restored: 1
> > Bytes Restored: 3,325,644,283
> > Rate: 21595.1 KB/s
> > FD Errors: 0
> > FD termination status: OK
> > SD termination status: OK
> > Termination: Restore OK
> >
> > troll-dir JobId 25212: Begin pruning Jobs.
> > troll-dir JobId 25212: No Jobs found to prune.
> > troll-dir JobId 25212: Begin pruning Files.
> > troll-dir JobId 25212: No Files found to prune.
> > troll-dir JobId 25212: End auto prune.
>
>
>
--
Ulrich Leodolter <[email protected]>
Oesterreichische Bibliothekenverbund und Service GmbH
Bruennlbadgasse 17/2A, A-1090 Wien
Fax +43 1 4035158-30
Tel +43 1 4035158-21
Web http://www.obvsg.at
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
_______________________________________________
Bacula-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-devel
