Re: [Bacula-users] client initiated backups - bconsole vs tray?

2018-07-06 Thread Stephen Thompson



Well this led to unexpected results.  Still 9.0.6, but running both FD 
and DIR in foreground with d900 both show startup messages, show console 
connecting to FD, show FD connecting to DIR when "proxy" is sent, but 
then when any command is sent and hangs, NEITHER FD NOR DIRECTOR output 
anything at all!


2000 OK Hello 214
Enter a period to cancel a command.
*proxy
2000 proxy OK.
*status


FD output...
fd: hello.c:262-0 Connecting to Director DIRECTOR:9101
fd: watchdog.c:197-0 Registered watchdog 7fc73401fa68, interval 15 one shot
fd: btimers.c:145-0 Start thread timer 7fc73401d498 tid 7fc73bf4c700 for 
15 secs.

fd: bsock.c:237-0 Current A.B.C.D:9101 All W.X.Y.Z:9101
fd: bsock.c:166-0 who=Director daemon host=DIRECTOR port=9101
fd: bsock.c:349-0 OK connected to server  Director daemon DIRECTOR:9101.
fd: btimers.c:203-0 Stop thread timer 7fc73401d498 tid=7fc73bf4c700.
fd: watchdog.c:217-0 Unregistered watchdog 7fc73401fa68
fd: watchdog.c:197-0 Registered watchdog 7fc73401d498, interval 15 one shot
fd: btimers.c:177-0 Start bsock timer 7fc734005d18 tid=7fc73bf4c700 for 
15 secs at 1530890871
fd: cram-md5.c:133-0 cram-get received: auth cram-md5 
<195401314.1530890871@dir> ssl=2

fd: cram-md5.c:157-0 sending resp to challenge: jlJ1z7+S47xwcCkb2S+GGD
fd: cram-md5.c:76-0 send: auth cram-md5 challenge 
<88308421.1530890871@fd> ssl=2

fd: cram-md5.c:95-0 Authenticate OK GD/TjH/8Dwc+4C0mJ8+2oD
fd: tls.c:392-0 Check subject name name
fd: bnet.c:280-0 TLS client negotiation established.
fd: hello.c:335-0 >dird: 1000 OK auth
fd: hello.c:342-0 November 2017)

fd: hello.c:345-0 1000 OK: 103 DIRECTOR Version: 9.0.6 (20 November 2017)



DIR output
dir: bnet.c:569-0 socket=6 who=client host=A.B.C.D port=9101
dir: jcr.c:931-0 set_jcr_job_status(0, C)
dir: jcr.c:940-0 OnEntry JobStatus=0 newJobstatus=C
dir: jcr.c:951-0 Set new stat. old: 0,0 new: C,0
dir: jcr.c:956-0 leave setJobStatus old=0 new=C
dir: job.c:1760-0 wstorage=STORAGE
dir: job.c:1769-0 wstore=STORAGE where=Job resource
dir: job.c:1429-0 JobId=0 created Job=-Console-.2018-07-06_08.27.51_05
dir: jcr.c:931-0 set_jcr_job_status(0, R)
dir: jcr.c:940-0 OnEntry JobStatus=C newJobstatus=R
dir: jcr.c:951-0 Set new stat. old: C,0 new: R,0
dir: jcr.c:956-0 leave setJobStatus old=C new=R
dir: cram-md5.c:69-0 send: auth cram-md5 challenge 
<195401314.1530890871@dir> ssl=2
dir: cram-md5.c:133-0 cram-get received: auth cram-md5 
<88308421.1530890871@fd> ssl=2

dir: cram-md5.c:157-0 sending resp to challenge: GD/TjH/8Dwc+4C0mJ8+2oD
lawson-dir: bnet.c:230-0 TLS server negotiation established.


I'm going to build 9.0.8 and see if I get different results.
I believe I skipped TLS with the same results.
Stephen


On 7/6/18 7:23 AM, Stephen Thompson wrote:



Yes, it does print 2000 proxy OK, but then in my case, the 'run' below 
would hang.  And as I said, running the bacula-fd in the foreground 
shows a successful connection to Director when successful, but then 
nothing more.  Also an unsuccessful connection (on purpose) is output 
form both the FD and the DIR, so they are definitely talking.  Hmmm... I 
will try your foregrounded director suggestion.


BTW, I'm also using TLS, which I'm hoping is not muddying the waters.

Oh, and technically I'm running 9.0.6, so perhaps I should upgrade as well.


Stephen



On 7/6/18 3:50 AM, Martin Simmons wrote:

It works for me in 9.0.8:

Connecting to Director localhost:9102
2000 OK Hello 214
Enter a period to cancel a command.
*proxy
2000 proxy OK.
*run
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
  1: Client1
  ...

Does it print "2000 proxy OK." and the "*" prompt after the proxy 
command?

You could try running the Director in the foreground with -d900.

__Martin




On Thu, 5 Jul 2018 17:30:31 -0700, Stephen Thompson said:


Thanks Martin.

That got me a step closer, but still not working.

If I run bacula-fd in foreground, I can see that when I execute proxy
command the FD outputs a successful connected to Director message.  But
running any other command under proxy in bconsole just hangs with no
output from FD or from Director.

Hmmm...
Stephen


On 7/5/18 8:21 AM, Martin Simmons wrote:

On Tue, 3 Jul 2018 16:04:56 -0700, Stephen Thompson said:


All,

I've been trying to setup client initiated backups via FD 
remote=yes and

bconsole with no success.  Regardless of the ACLs defined on Director,
the only command available on client's bconsole is "status" and even
that is the status of the local FD, not the DIR status.  Every other
command yields...

2999 Invalid command


You are not connected directly to the Director command loop after 
connecting

bconsole to the local FD.  According to the test
(regress/tests/remote-console-test), you need to use the proxy command
(without any arguments) to connect to the Director.

__Martin



--
Stephen Thompson   Berkeley Seismo Lab
step...@seismo.berkeley.edu    215 McCone Hall

Re: [Bacula-users] Incrementals use 170GB on-disk volume storage, but only 60GB when restored

2018-07-06 Thread Martin Simmons
Maybe your fileset causes files to be copied more than once (e.g. by listing
the same directory via different File= lines)?

Also, what kind of filesystem are you using for the restored directory?  Maybe
it is compressing the data on restore (since du reports the disk used, not the
size of the files)?

You could try running bls on the volume and use your favourite scripting
language to add up the file sizes that it prints to see the expected total
size.

Also, use bls -j to check that the volume really does contain just one job.

__Martin


> On Thu, 5 Jul 2018 17:28:39 +0200, Uwe Schuerkamp said:
> 
> Dear bacula list,
> 
> I'm having a weird problem with incremental backups of one of our machines.
> 
> We do an online (to disk backup) compressed incremental backup into
> single volumes (one volume per incremental job, 10 volumes recycled
> automatically) and everything appears to work fine.
> 
> However, the on-disk usage of the volume is a whopping 170GB, but when
> I do a complete restore of this incremental job the restored directory
> occupies only around 60GB of diskspace.
> 
> Here's a bacula web statistic from one of the incremental jobs:
> 
> 460031 Incr 1920 170.45 GB 2018-06-14 16:52:11 2018-06-14 18:52:32
> 02:00:21 24.17 MB/s 0.05
> 
> And here's the "du -sh" report on the fully restored incemental ("mark
> *"  in "/") job listed above:
> 
> bacula_restore]# du -sh .
> 96G .
> 
> We don't get any error messages during restore except for two tomcat
> logfiles which changed in size during the backup, but those are only a
> few hundred MB and don't account for the huge difference in disk space
> usage we're experiencing here.
> 
> Also, the restore logs shows some messages in the form of
> 
> "file  has been deleted",
> 
> but those are also mostly files that are nowhere near large enough to
> explain the discrepancy in size. We've also tried removing the
> "accurate" flag from the job in question and reloading the config, but
> that hasn't solved the problem, either.
> 
> 
> We've been running bacula for over a decade now on six backup server
> instances with hundreds of clients and a few dozen TB of archive
> space, so I'd not consider myself a bacula newbie in the sense of the
> word, but this one has me puzzled... any advice would be greatly
> appreciated.
> 
> Server version: 9.0.6 compiled from source on centos6
> Catalog DB: MariaDB 10.x
> Client fd version:   bacula-client-5.0.0-13.el6.x86_64
> bacula-common-5.0.0-13.el6.x86_64
> 
> Thanks very much in advance for any ideas on the matter & all the best,
> 
> Uwe
> 
> -- 
> Uwe Schürkamp | email: 
> Geschäftsführer: Ralf Schürmann | Dr. Manfred Heinen
> 
> 
> 
> 
> 
> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] client initiated backups - bconsole vs tray?

2018-07-06 Thread Martin Simmons
I suggest trying it without TLS in case that is the cause.

__Martin


> On Fri, 6 Jul 2018 07:23:55 -0700, Stephen Thompson said:
> 
> Yes, it does print 2000 proxy OK, but then in my case, the 'run' below 
> would hang.  And as I said, running the bacula-fd in the foreground 
> shows a successful connection to Director when successful, but then 
> nothing more.  Also an unsuccessful connection (on purpose) is output 
> form both the FD and the DIR, so they are definitely talking.  Hmmm... I 
> will try your foregrounded director suggestion.
> 
> BTW, I'm also using TLS, which I'm hoping is not muddying the waters.
> 
> Oh, and technically I'm running 9.0.6, so perhaps I should upgrade as well.
> 
> 
> Stephen
> 
> 
> 
> On 7/6/18 3:50 AM, Martin Simmons wrote:
> > It works for me in 9.0.8:
> > 
> > Connecting to Director localhost:9102
> > 2000 OK Hello 214
> > Enter a period to cancel a command.
> > *proxy
> > 2000 proxy OK.
> > *run
> > Automatically selected Catalog: MyCatalog
> > Using Catalog "MyCatalog"
> > A job name must be specified.
> > The defined Job resources are:
> >   1: Client1
> >   ...
> > 
> > Does it print "2000 proxy OK." and the "*" prompt after the proxy command?
> > You could try running the Director in the foreground with -d900.
> > 
> > __Martin
> > 
> > 
> > 
> >> On Thu, 5 Jul 2018 17:30:31 -0700, Stephen Thompson said:
> >>
> >> Thanks Martin.
> >>
> >> That got me a step closer, but still not working.
> >>
> >> If I run bacula-fd in foreground, I can see that when I execute proxy
> >> command the FD outputs a successful connected to Director message.  But
> >> running any other command under proxy in bconsole just hangs with no
> >> output from FD or from Director.
> >>
> >> Hmmm...
> >> Stephen
> >>
> >>
> >> On 7/5/18 8:21 AM, Martin Simmons wrote:
>  On Tue, 3 Jul 2018 16:04:56 -0700, Stephen Thompson said:
> 
>  All,
> 
>  I've been trying to setup client initiated backups via FD remote=yes and
>  bconsole with no success.  Regardless of the ACLs defined on Director,
>  the only command available on client's bconsole is "status" and even
>  that is the status of the local FD, not the DIR status.  Every other
>  command yields...
> 
>  2999 Invalid command
> >>>
> >>> You are not connected directly to the Director command loop after 
> >>> connecting
> >>> bconsole to the local FD.  According to the test
> >>> (regress/tests/remote-console-test), you need to use the proxy command
> >>> (without any arguments) to connect to the Director.
> >>>
> >>> __Martin
> >>>
> >>
> >> -- 
> >> Stephen Thompson   Berkeley Seismo Lab
> >> step...@seismo.berkeley.edu215 McCone Hall
> >> Office: 510.664.9177   University of California
> >> Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760
> >>
> 
> -- 
> Stephen Thompson   Berkeley Seismo Lab
> step...@seismo.berkeley.edu215 McCone Hall
> Office: 510.664.9177   University of California
> Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760
> 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Volume weirdness

2018-07-06 Thread Donna Hofmeister
This is for version 7.4.7, btw.

In my bacula-dir-conf file, I have the following pool stanza:

# File Pool definition
Pool
{
  Name = File
  Pool Type= Backup
  Recycle  = yes  # Bacula can automatically recycle
Volumes
  AutoPrune= yes  # Prune expired volumes
  Volume Retention = 30 days  # 30 days
  Maximum Volume Bytes = 1G   # Limit Volume size to something
reasonable
  Maximum Volumes  = 350  # Limit number of Volumes in Pool
  Maximum Volume Jobs  = 1# Force a Volume switch after 1 job
  Volume Use Duration  = 14h  # Force volume switch
  Label Format =
"Vol-${Client}-${Pool}-${Year}${Month:p/2/0/r}${Day:p/2/0/r}-${Hour:p/2/0/r}${Minute:p/2/0/r}-${JobId}-${NumVols}"
  Purge oldest Volume  = yes
}

For *some* of my clients, the file name is exactly per the 'label format'.
But for others, the ${Client} name is the name of my Bacula server. For one
client in particular, the volume name is bizarre:

Volume name(s): Vol-bacula.allegro.com-fd-
File-20180604-2312-66-96|Vol-bacula.allegro.com-fd-File-
20180705-2305-241-248

fwiw, here's the directory listing:

-rw-r- 1 root root   258 Jul  5 23:05
Vol-bacula.allegro.com-fd-File-20180604-2312-66-96
-rw-r- 1 root root 819894860 Jul  5 23:08
Vol-bacula.allegro.com-fd-File-20180705-2305-241-248
-rw-r- 1 root root  34647949 Jul  5 23:10
Vol-bacula.allegro.com-fd-File-20180705-2310-246-249

Any idea why this is happening? All of my client configuration files (both
on the client and the server) are 'fleshed out' from a template and so are
virtually identical (less the 'personalization' bits).   - donna

-- 
Donna Hofmeister
Allegro Consultants, Inc.
408-252-2330
Visit us on Linkedin

Like us on Facebook 
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula 9.0.8 with Aligned-sd

2018-07-06 Thread zaiyers
You don't need to update the clients as long as they are still able to 
connect to the SR (which 7.4.7 is).
I did not experience any problems connecting from a 7.4.7 FD to a 9.0.8 
SR with the aligned plugin enabled.


Afaik the client isn't aware of what type of device the SR is writing to.


On 07/06/2018 02:39 PM, Olivier Delestre wrote:

Hi,

For my test i installed a bacula 9.0.8 / Postgresql / Zfs on Ubuntu 18 Lts

I configured the File for the Storage with Plugin Directory = 
"/opt/bacula/plugins" on the server


And the value for the Device Type = Aligned, ...

I just a question, my client are on version 7.4.7 for the production 
server (still on 7.4.7) , The dedup works with this version ?  or do I 
have to upgrade version the client too ?


Server Production 7.4.7 <--> Client 7.4.7
Server Test 9.0.8 <--> Client 7.4.7, the same as above

That's run for the moment ( 9 To ) but with the mechanism of dedup (??? ).

Thanks

--


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] possibly new mtx timing bug in 9x?

2018-07-06 Thread Stephen Thompson



Not sure if anyone else is seeing this, but sporadically, perhaps 2-3 
times a month, after running various version of bacula on the same 
server/tape library for 10 years now and now running 9.0.6, we are 
seeing cases where bacula want's user intervention to mount a tape in a 
drive, but the tape is already in the drive AND bacula put it there. 
The only thing I can think is that the tape load step is somehow timing 
out and then not making the check to see whether the tape made it to the 
drive or not.


thanks,
Stephen
--
Stephen Thompson   Berkeley Seismo Lab
step...@seismo.berkeley.edu215 McCone Hall
Office: 510.664.9177   University of California
Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] client initiated backups - bconsole vs tray?

2018-07-06 Thread Stephen Thompson




Yes, it does print 2000 proxy OK, but then in my case, the 'run' below 
would hang.  And as I said, running the bacula-fd in the foreground 
shows a successful connection to Director when successful, but then 
nothing more.  Also an unsuccessful connection (on purpose) is output 
form both the FD and the DIR, so they are definitely talking.  Hmmm... I 
will try your foregrounded director suggestion.


BTW, I'm also using TLS, which I'm hoping is not muddying the waters.

Oh, and technically I'm running 9.0.6, so perhaps I should upgrade as well.


Stephen



On 7/6/18 3:50 AM, Martin Simmons wrote:

It works for me in 9.0.8:

Connecting to Director localhost:9102
2000 OK Hello 214
Enter a period to cancel a command.
*proxy
2000 proxy OK.
*run
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
  1: Client1
  ...

Does it print "2000 proxy OK." and the "*" prompt after the proxy command?
You could try running the Director in the foreground with -d900.

__Martin




On Thu, 5 Jul 2018 17:30:31 -0700, Stephen Thompson said:


Thanks Martin.

That got me a step closer, but still not working.

If I run bacula-fd in foreground, I can see that when I execute proxy
command the FD outputs a successful connected to Director message.  But
running any other command under proxy in bconsole just hangs with no
output from FD or from Director.

Hmmm...
Stephen


On 7/5/18 8:21 AM, Martin Simmons wrote:

On Tue, 3 Jul 2018 16:04:56 -0700, Stephen Thompson said:


All,

I've been trying to setup client initiated backups via FD remote=yes and
bconsole with no success.  Regardless of the ACLs defined on Director,
the only command available on client's bconsole is "status" and even
that is the status of the local FD, not the DIR status.  Every other
command yields...

2999 Invalid command


You are not connected directly to the Director command loop after connecting
bconsole to the local FD.  According to the test
(regress/tests/remote-console-test), you need to use the proxy command
(without any arguments) to connect to the Director.

__Martin



--
Stephen Thompson   Berkeley Seismo Lab
step...@seismo.berkeley.edu215 McCone Hall
Office: 510.664.9177   University of California
Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760



--
Stephen Thompson   Berkeley Seismo Lab
step...@seismo.berkeley.edu215 McCone Hall
Office: 510.664.9177   University of California
Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula 9.0.8 with Aligned-sd

2018-07-06 Thread Olivier Delestre

Hi,

For my test i installed a bacula 9.0.8 / Postgresql / Zfs on Ubuntu 18 Lts

I configured the File for the Storage with Plugin Directory = 
"/opt/bacula/plugins" on the server


And the value for the Device Type = Aligned, ...

I just a question, my client are on version 7.4.7 for the production 
server (still on 7.4.7) , The dedup works with this version ?  or do I 
have to upgrade version the client too ?


Server Production 7.4.7 <--> Client 7.4.7
Server Test 9.0.8 <--> Client 7.4.7, the same as above

That's run for the moment ( 9 To ) but with the mechanism of dedup (??? ).

Thanks

--

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] client initiated backups - bconsole vs tray?

2018-07-06 Thread Martin Simmons
It works for me in 9.0.8:

Connecting to Director localhost:9102
2000 OK Hello 214
Enter a period to cancel a command.
*proxy
2000 proxy OK.
*run
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
 1: Client1
 ...

Does it print "2000 proxy OK." and the "*" prompt after the proxy command?
You could try running the Director in the foreground with -d900.

__Martin



> On Thu, 5 Jul 2018 17:30:31 -0700, Stephen Thompson said:
> 
> Thanks Martin.
> 
> That got me a step closer, but still not working.
> 
> If I run bacula-fd in foreground, I can see that when I execute proxy 
> command the FD outputs a successful connected to Director message.  But 
> running any other command under proxy in bconsole just hangs with no 
> output from FD or from Director.
> 
> Hmmm...
> Stephen
> 
> 
> On 7/5/18 8:21 AM, Martin Simmons wrote:
> >> On Tue, 3 Jul 2018 16:04:56 -0700, Stephen Thompson said:
> >>
> >> All,
> >>
> >> I've been trying to setup client initiated backups via FD remote=yes and
> >> bconsole with no success.  Regardless of the ACLs defined on Director,
> >> the only command available on client's bconsole is "status" and even
> >> that is the status of the local FD, not the DIR status.  Every other
> >> command yields...
> >>
> >> 2999 Invalid command
> > 
> > You are not connected directly to the Director command loop after connecting
> > bconsole to the local FD.  According to the test
> > (regress/tests/remote-console-test), you need to use the proxy command
> > (without any arguments) to connect to the Director.
> > 
> > __Martin
> > 
> 
> -- 
> Stephen Thompson   Berkeley Seismo Lab
> step...@seismo.berkeley.edu215 McCone Hall
> Office: 510.664.9177   University of California
> Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760
> 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Incrementals use 170GB on-disk volume storage, but only 60GB when restored

2018-07-06 Thread Uwe Schuerkamp
Dear Olivier,

thanks for your reply.

On Fri, Jul 06, 2018 at 08:35:45AM +0200, Olivier Delestre wrote:

> Do you use the Aligned plugin for the SD ?
>  I do not know about your mystery
> keep us in touch

We're not using any plugins, just a plain bacula install compiled from
source. Our online volume storage is an HP MSA dumb disk array
(classical "spinning rust" hds, no SDDs involved at all not even for
catalog db backend storage).

Would it help to include the bextract log? We've tried both methods (running a 
restore job from "bconsole" and using "bextract" to extract the entire 
incremental volume content).

All the best,

Uwe

-- 
Uwe Schürkamp | email: 
Geschäftsführer: Ralf Schürmann | Dr. Manfred Heinen







--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Incrementals use 170GB on-disk volume storage, but only 60GB when restored

2018-07-06 Thread Olivier Delestre

Thanks for your experience.

I install a 3th server yesterday with repo of bacula 9.0.8 + Postgresql 
on Ubuntu 18 Lts ( Not CentOs have pb of rpm at start. Yet ?? ). With a 
Pool Zfs ( comp =lz4 , dedup on )


I will test during the holidays with client ( 7.4.7 ). The other server 
is production on Bacula 7.4.7.


Do you use the Aligned plugin for the SD ?
 I do not know about your mystery
keep us in touch





Le 05/07/2018 à 17:28, Uwe Schuerkamp a écrit :

Dear bacula list,

I'm having a weird problem with incremental backups of one of our machines.

We do an online (to disk backup) compressed incremental backup into
single volumes (one volume per incremental job, 10 volumes recycled
automatically) and everything appears to work fine.

However, the on-disk usage of the volume is a whopping 170GB, but when
I do a complete restore of this incremental job the restored directory
occupies only around 60GB of diskspace.

Here's a bacula web statistic from one of the incremental jobs:

460031 Incr 1920 170.45 GB 2018-06-14 16:52:11 2018-06-14 18:52:32
02:00:21 24.17 MB/s 0.05

And here's the "du -sh" report on the fully restored incemental ("mark
*"  in "/") job listed above:

bacula_restore]# du -sh .
96G .

We don't get any error messages during restore except for two tomcat
logfiles which changed in size during the backup, but those are only a
few hundred MB and don't account for the huge difference in disk space
usage we're experiencing here.

Also, the restore logs shows some messages in the form of

"file  has been deleted",

but those are also mostly files that are nowhere near large enough to
explain the discrepancy in size. We've also tried removing the
"accurate" flag from the job in question and reloading the config, but
that hasn't solved the problem, either.


We've been running bacula for over a decade now on six backup server
instances with hundreds of clients and a few dozen TB of archive
space, so I'd not consider myself a bacula newbie in the sense of the
word, but this one has me puzzled... any advice would be greatly
appreciated.

Server version: 9.0.6 compiled from source on centos6
Catalog DB: MariaDB 10.x
Client fd version:   bacula-client-5.0.0-13.el6.x86_64
bacula-common-5.0.0-13.el6.x86_64

Thanks very much in advance for any ideas on the matter & all the best,

Uwe



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users