Hello list!

Sorry to trouble you with what's probably a simple problem, but I'm now
looking at the very real possibility of wiping all our backups clean and
starting from scratch if I can't fix it... :(

I'm having problems with some Full backups, which run for between 1 and
2 hours, appearing to "time out" after the data transfer from the FD to
the SD. The error message (shown below) shows that the data transfer
completes, often in about 1hr30min, and then Bacula does nothing until
the job has been running for 2 hours at which point it gives an FD

Other Full backups (which don't take as long) run correctly, and for
most of the time Inc and Diff backups also run correctly. However, a
small % of backups will fail at random, also with FD errors but at
random times-elapsed during the job... this I have been ascribing to
network fluctuations! The difference is that re-running these random
failures will succeed, whilst this particular Full failure doesn't! ;)

I've already tried setting a heartbeat interval of 20 minutes in the
FD/SD and DIR conf files (thinking that the FD -> Dir connection was
timing out) but this doesn't change anything.

In the time between the data transfer finishing and the timeout,
Postgres has an open connection with a "COPY batch FROM STDIN"
transaction in progress, which at the timeout produces errors in the
Postgres log that I have also shown below.

I'm happy to post portions of the conf files if needed, but they're huge
and might well lead to tl;dr!

Any suggestions as to how I can troubleshoot this further would be most

Nick Lock.

12-Aug 14:18 exa-bacula-dir JobId 5514: Start Backup JobId 5514,
12-Aug 14:18 exa-bacula-dir JobId 5514: There are no more Jobs
associated with Volume "scavenger-full-1250". Marking it purged.
12-Aug 14:18 exa-bacula-dir JobId 5514: All records pruned from Volume
"scavenger-full-1250"; marking it "Purged"
12-Aug 14:18 exa-bacula-dir JobId 5514: Recycled volume
12-Aug 14:18 exa-bacula-dir JobId 5514: Using Device
12-Aug 14:18 exa-bacula-sd JobId 5514: Recycled volume
"scavenger-full-1250" on device
"FileStorageScavenger" (/srv/bacula/volume/web-scavenger), all previous
data lost.
12-Aug 14:18 exa-bacula-dir JobId 5514: Max Volume jobs exceeded.
Marking Volume "scavenger-full-1250" as Used.
12-Aug 15:49 exa-bacula-sd JobId 5514: Job write elapsed time =
01:31:41, Transfer rate = 401.4 K bytes/second
12-Aug 16:18 exa-bacula-dir JobId 5514: Fatal error: Network error with
FD during Backup: ERR=Connection reset by peer
12-Aug 16:18 exa-bacula-dir JobId 5514: Fatal error: No Job status
returned from FD.
12-Aug 16:18 exa-bacula-dir JobId 5514: Error: Bacula exa-bacula-dir
2.4.4 (28Dec08): 12-Aug-2009 16:18:09
  Build OS:               x86_64-pc-linux-gnu debian lenny/sid
  JobId:                  5514
  Job:                    backup_scavenger.2009-08-12_14.18.06.04
  Backup Level:           Full
  Client:                 "scavenger" 2.4.4 (28Dec08)
  FileSet:                "full-scavenger" 2009-04-16 15:58:05
  Pool:                   "scavenger-full" (From Job FullPool override)
  Storage:                "FileScavenger" (From Job resource)
  Scheduled time:         12-Aug-2009 14:18:03
  Start time:             12-Aug-2009 14:18:09
  End time:               12-Aug-2009 16:18:09
  Elapsed time:           2 hours 
  Priority:               10
  FD Files Written:       0
  SD Files Written:       81,883
  FD Bytes Written:       0 (0 B)
  SD Bytes Written:       2,208,578,175 (2.208 GB)
  Rate:                   0.0 KB/s
  Software Compression:   None
  VSS:                    no
  Storage Encryption:     no
  Volume name(s):         scavenger-full-1250
  Volume Session Id:      5
  Volume Session Time:    1250080970
  Last Volume Bytes:      2,212,857,316 (2.212 GB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  Error
  SD termination status:  OK
  Termination:            *** Backup Error ***

Postgres Log:

2009-08-12 16:18:09 BST ERROR:  unexpected message type 0x58 during COPY
from stdin
2009-08-12 16:18:09 BST CONTEXT:  COPY batch, line 81884: ""
2009-08-12 16:18:09 BST STATEMENT:  COPY batch FROM STDIN
2009-08-12 16:18:09 BST LOG:  could not send data to client: Broken pipe
2009-08-12 16:18:09 BST LOG:  could not receive data from client:
Connection reset by peer
2009-08-12 16:18:09 BST LOG:  unexpected EOF on client connection

Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Bacula-users mailing list

Reply via email to