Re: [Bacula-users] An special day for honoring someone who has done a extremely nice job in the open source world

2022-03-24 Thread Kern Sibbald

Hello Egoitz,

Thanks for your very kind email with birthday wishes.  I am really 
honored that you call me a mentor, thanks.  I have always done my best 
to produce a high end backup product (though it is a bit complicated) 
that is robust and reliable with all the features needed by the 
community as well as small enterprises.  Though it is sometimes hard, I 
feel that we succeeded in creating a friendly and helpful email list.


Thanks for using Bacula and for your email.

Concerning your plugin:  in general we accept all contributions that are 
useful, follow our current design conditions (in the Developers 
document) and for which we have a signed CAA (copyright assignment 
agreement).  This agreement allows Bacula Systems to use the code, but 
even more important for me, it protect all Bacula users from someone 
introducing code then claiming we are using his proprietary code (this 
is what Bareos attempted to do to Bacula). I can remember only on case 
where we we not able to use the code, and that was code developed by 
UKFast (ISP) and released by the developer without permission.  We we 
never able to get permission to use the code from UKFast.  This code 
implemented quotas.  Instead of integrating unauthorized code, we wrote 
our own simpler and more efficient quota code.  Note, Bareos integrated 
the unauthorized code, so perhaps one day their users may be exposed to 
license problems.


If you finish your code, and you are willing to submit a CAA, then I 
recommend that you submit it to Bacula (Now that I am retired, Eric 
decides exactly what is integrated and what is not).   In my opinion, it 
would be a very nice addition.


I am very pleased that Bacula Systems has agreed to look after 
maintaining the community version and continue adding the Bacula Systems 
new features.  This means that Bacula will be getting better and better 
and continue to adapt to the ever changing IT backup/restore needs.


Thank you very much for recognizing my contributions, and above all 
thank you for using Bacula and developing code for it.  I wish you all 
the best in the future.


Kind regards,
Kern
(Currently in Puerto Rico until May then back to Barcelona).



On 3/21/2022 4:40 AM, ego...@ramattack.net wrote:


Good morning people :)


Today is a special day. Is the birthday of someone now deservedly 
retired. This person is Kern Sibbald. I had the opportunity of knowing 
about his birthday, through one nice person in this mailing lists. 
Kern is much more than a nice codder, who has write a extremely 
important tool which is the base of most of our backups. At least for 
me is one of my mentors, one of the persons from which I would like to 
learn because it has done a nice job with Bacula. For all these 
reasons, I wanted to say "thank you so much Kern and have an extremely 
happy birthday!!".


I wanted too, for making the most of this lines, to ask a little gift 
to Kern :) :). I'm working as some of you know, in building a Bacula 
pluggin for creating an open source delta encoding plugin for the fd. 
It would be hugely nice :) :) (as someone told me too in this list) if 
official Delta plugin of Bacula, could be distributed with the 
Community source of Bacula. I'll go on writting my own plugin anyway, 
because I wanted to learn how it works and for being able to customize 
some of the backups I do here, but it would be really nice to have the 
Delta plugin as part of the Community edition of this nice piece of 
software. I had to try :) :) .


Anyway and independently of what Kern decides about the gift I have 
asked :) :) (I had to try it... mainly after someone encouraged asking 
it :) :) ), I wanted to emphasize my recognition about Kern's person, 
due to all his contributions to the open source world. These ideas, 
anyway, would never change in my mind about you.


So for ending this email, I think there are not more appropiate words 
for being remarked as the following ones : "Kern, Thank you so much".



Cheers :)





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [External] Free Bacula Health Check

2021-08-26 Thread Kern Sibbald

Hello Marc,

Good point ...

I will forward your suggestion to the persons responsible for handling 
this in Bacula Systems.


Best regards,
Kern

On 8/24/21 6:20 PM, mark.berg...@pennmedicine.upenn.edu wrote:

In the message dated: Tue, 24 Aug 2021 16:37:33 +0200,
The pithy ruminations from Kern Sibbald on
[[External] [Bacula-users] Free Bacula Health Check] were:
=> Hello,

Kern,

=>
=> Thank you for using Bacula.  Although I am now mostly retired (but busier 
than ever), as Chairman of the Board of Bacula Systems,
=> I do keep up on new things, especially the multiple things that Bacula 
Systems is doing to improve the Bacula Community user
=> experience.
=>
=> One such step is that Bacula Systems is offering the Bacula Community users access to 
a free "Health Check" service. This is a
=> nice way to get your Bacula Community implementation 'health checked' by 
another advanced Bacula expert (besides yourself, of
=> course!). In my experience, running the check usually results in the user 
learning how he or she may be able to further improve -
=> or get more out of - Bacula Community. The offer from Bacula Systems is open 
to anyone with a deployment of 50 agents or more. I
=> hope you find it helpful.

That's a great offer.

While "50 agents" is one measure of the scale of Bacula installation,
perhaps it shouldn't be the only qualification for the health check. For
example, we backup less than 25 machines, but have about 450TB of data
being backed up to tape.

Thanks,

Mark

=>
=> Best regards,
=> Kern
=>
=> PS: if the above link does not work for you, or like me, you don't like following 
email links, please go to: "https://
=> www.bacula.org/free-backup-system-review" to sign up for the offer.
=>
=> ___
=> Bacula-users mailing list
=> Bacula-users@lists.sourceforge.net
=> https://lists.sourceforge.net/lists/listinfo/bacula-users
=>





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Free Bacula Health Check

2021-08-24 Thread Kern Sibbald

  
  
Hello,

Thank you for using Bacula.  Although I am now mostly retired (but
busier than ever), as Chairman of the Board of Bacula Systems, I do
keep up on new things, especially the multiple things that Bacula
Systems is doing to improve the Bacula Community user experience.  

One such step is that Bacula Systems is offering the Bacula
Community users access to a free "Health
  Check" service. This is a nice way to get your Bacula
Community implementation 'health checked' by another advanced Bacula
expert (besides yourself, of course!). In my experience, running the
check usually results in the user learning how he or she may be able
to further improve - or get more out of - Bacula Community. The
offer from Bacula Systems is open to anyone with a deployment of 50
agents or more. I hope you find it helpful.  

Best regards,
Kern

PS: if the above link does not work for you, or like me, you don't
like following email links, please go to:
"https://www.bacula.org/free-backup-system-review" to sign up for
the offer.
  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Which OS is best for bacula ?

2021-07-12 Thread Kern Sibbald

Hello again,

I can see bacula installation placement is just as hot a topic as it has 
always been.  However, I sense a trend toward accepting the /opt/bacula 
concept.  I do wish we could convince a distro such as Debian, but they 
have their point of view, and I for one do not plan to argue with them.


I started Linux on rpm systems and used them for many years.  In 
switching to Ubuntu (which most of you know is derived from Debian) I 
had to learn apt and the other Debian packing tools.  They are a bit 
complicated, but over the years I have found them to be superior to rpm 
tools.  With the uncertainty about CentOS, you might consider switching 
to a debs based system.  It will come with a learning curve, but be 
worth it.


What ever your OS or your installation directory, thanks for using 
Bacula -- enjoy it :-)


Best regards,
Kern

PS: I was also reminded that Bacula runs quite well on OpenBSD, which I 
find fantastic!


On 7/12/21 2:57 PM, Josh Fisher wrote:

On 7/11/21 5:10 PM, Josip Deanovic wrote:

On Sunday 2021-07-11 18:31:24 Kern Sibbald wrote:

Hello Sven,

Yes, I am aware of the FHS policy, which is perfectly fine for 99.9% of
all installed programs, but not ideal for Bacula.


Hello,

There is nothing special about Bacula in that regard.



I disagree. Yes, in its day-to-day operation it's requirements are the 
same as many other apps, storage for binaries and configuration and 
also uses temp/spool storage. However, because of its role in disaster 
recovery, simplicity and speed of deployment is much more critical for 
Bacula than for most apps. It indeed is special in that regard, and a 
/opt install is simpler and faster to deploy when restoring Bacula 
services to bare metal in a disaster recovery scenario.



Many (if not most) programs require modifications to conform to a 
specific

standard unless the program is following particular rules from the very
beginning.

Many programs are modified by package maintainers who sometimes have to
come up with patches for programs to work as expected because authors
sometimes chose to hardcode paths and other options which should be
configurable.


We are dealing here with two opposite opinions and two different points
of view because two types of people are involved here: developers and
sysadmins.

For a developer the easiest solution is to keep everything under a 
single
directory and to ship a bunch of libraries with the program 
distribution.

This keeps program more predictive and the deployment less complicated.

Developers usually don't have to bother with the software deployment and
the service maintenance.
Every bit of work they manage to spare during the application 
development

by keeping their environment simple will usually get payed by a sysadmin
during the deployment and service maintenance.


On Unix (and Linux) /opt directory used to be a directory which usually
contained optional software distributions provided by a third party
companies or vendors.
It is rarely used by OpenSource tools or applications.

Unless there is no other way (e.g. there is a commercial software
distribution with a specific requirements), /opt shouldn't be used
by a software distribution. Using /opt is usually a sign of a bad
support (e.g. packaging and testing) for a specific system.


Note that I am talking in general, not about Bacula specifically.

Software that tend to use /opt for its installation usually brings
several commonly available libraries locked to a specific version.
Providing commonly available libraries already available on the system
with the particular software distribution and linking programs with
such libraries makes it good thing for the application stability and
simplicity of its deployment and ultimately vendor support.

However, from a non-developer view there are no visible benefits.
 From sysadmin view, it looks like yet another nonstandard software
installation with an unusual update procedure which usually turns out
to be more of a migration and less of a simple minor version update.

Another problem is that it is most likely that libraries shipped with
a third-party software will receive their security and bugfix updates
far too slow or more commonly, will not receive any updates at all
which adds to an overall potential system vulnerability footprint.
Also, shipping such common libraries will duplicate the code used on
the system.
Putting software under /opt could also complicate a diskless setup a 
bit.


Now imagine the horror on the system if most of the services on the
system would follow such practice. Such system would be very hard
to maintain and the code sharing in memory would be pretty low.


Directory structure on Linux systems is the only thing that was
relatively well standardized throughout the Linux distributions since
the earliest days of Linux and it is interesting to see that what is
considered to be a properly packaged software for one person could
at the same time be described as "spre

Re: [Bacula-users] Which OS is best for bacula ?

2021-07-11 Thread Kern Sibbald

Hello Sven,

Yes, I am aware of the FHS policy, which is perfectly fine for 99.9% of 
all installed programs, but not ideal for Bacula.


Best regards,
Kern

On 7/9/21 11:07 PM, Sven Hartge wrote:

On 09.07.21 14:22, Kern Sibbald wrote:

I would be a lot happier if they would learn to install it in 
/opt/bacula instead of spreading it all over the filesystem though.


Sorry, no can do, the Debian Policy clearly forbids this, the FHS has 
to be adhered to.


Grüße,
Sven.


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Which OS is best for bacula ?

2021-07-09 Thread Kern Sibbald

  
  
Hello,

As originator of Bacula (and still the highest in the gitstats list
of contributors), my personal vote is for Ubuntu.  That said,
I would be very happy to run Bacula on any version of Debian -- the
Bacula packagers for Debian work closely with the Bacula project --
I would be a lot happier if they would learn to install it in
/opt/bacula instead of spreading it all over the filesystem though. 
Note: Ubuntu also spreads the installation all over the place, so I
do my own packaging as recommended in the manual.  In light of the
wide spread attacks of ransomware, it is even more important to
ensure that restoration of Bacula is simple and easy.  There is a
lot of info on ransomeware on bacula.org.

There is no need you should have to pay for a Bacula Windows client
either today or any time in the future.  Bacula Systems will ensure
that the community Bacula runs on the most popular operating systems
(Windows client, MacOS, FreeBSD, Solaris, even Oracle, ...)

Best regards,
Kern

On 7/8/21 9:47 PM, Thing wrote:


  
  Hi,


Which OS you choose depends on which one is best suited to
  your application(s), I just work on that then try and make the
  ancillary apps fit. If Bacula does not, use a different backup
  product.
  
  
  Centos7 is effectively approaching  end of life as Redhat
kills it off to make you pay, so I wouldnt suggest that OS
or say Stream for this reason.    Windows 2012 R2 is also
end of life so best to migrate that to 2019? and you might
need to pay for a Bacula Windows client? (you did in the
past)   You need to look at this as as Bacula or indeed any
application advances 2012 will be left behind. 
  
  
  Your problem will be what version of Bacula will work
across all of these.   In my case I ended up with Bacula on
Debian as that was the bulk of my OSes and then had to port
Bacula client Centos7 to a compatible version, it was a
total nightmare I never got a client for Mac OS to work.  
  
  
  regards
  
  
  Steven
  
  
  
  

  
  
  
On Wed, 7 Jul 2021 at 17:20,
  Marc Ferrand <marcferr...@gmail.com> wrote:


  If you had a choice, on which system would you
install bacula and why ?
These are OSes I'm familiar with in order or preference
  :
lint Mint (20, fork of Ubuntu/Debian), CentOS 7 (free
  version of RHEL), Windows 10 (Desktop), DragonFly BSD
  (fork of freeBSD), Windows Server 2012 R2.
Any advice or suggestion is welcome, thanks in advance,
  Marc.
  
  ___
  Bacula-users mailing list
  Bacula-users@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/bacula-users

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] .config/bacula.org/bat.conf ???

2021-06-12 Thread Kern Sibbald

Hello,

I suspect that this file is created by Qt to remember (as you suggest) 
the last position and size of Bat's main window.


Best regards,
Kern

On 6/9/21 12:06 PM, Erik P. Olsen wrote:

I've just noticed that the subject file exists and is being updated when I exit 
BAT. It
contains the following four lines:

[MainWin]
state=@ByteArray(\0\0\0\xff\0\0\0\0\xfd\0\0\0\x1\0\0\0\0\0\0\0\xb8\0\0\x3\xa4\xfc\x2\0\0\0\x1\xfb\0\0\0\x14\0\x64\0o\0\x63\0k\0W\0i\0\x64\0g\0\x65\0t\x1\0\0\0?\0\0\x3\xa4\0\0\0\xb3\0\a\xff\xff\0\0\x6\xc2\0\0\x3\xa4\0\0\0\x4\0\0\0\x4\0\0\0\b\0\0\0\b\xfc\0\0\0\x1\0\0\0\x2\0\0\0\x1\0\0\0\xe\0t\0o\0o\0l\0\x42\0\x61\0r\x1\0\0\0\0\xff\xff\xff\xff\0\0\0\0\0\0\0\0)
winPos=@Point(-10 37)
winSize=@Size(1920 1019)

What's the purpose of this file and is it perhaps something that may be used to 
control
position and size of BAT's main window (just guessing?





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] S3 "Driver" Doesn't Work 11.0.2

2021-05-07 Thread Kern Sibbald

Hello,

Yes, I guess I should stay retired :-)

Best regards,
Kern

On 5/7/21 12:39 PM, Martin Simmons wrote:

On Thu, 6 May 2021 18:01:41 +0200, Kern Sibbald said:

Your fundamental problem in making the S3 plugin work is that plugin
has the wrong name -- this can be seen from the -D200 debug output
you supplied.  The Bacula SD is looking for plugins named:
xxx-sd.so, and your plugin is named xxx.so, so Bacula is not finding
it.

I think that is wrong.  The files named xxx-sd.so are SD plugins, not drivers.

__Martin





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] LTO-7 btape speed test

2021-05-06 Thread Kern Sibbald

  
  
Hello,

Nice.  That is very good throughput.  Thanks for reporting your
results ...

Best regards,
Kern

On 5/5/21 7:05 AM, fk+bacula--- via
  Bacula-users wrote:


  
  I
have found  the solution:
  There
was a cooling issue inside the server which mounts the LTO
drive. A fan wasn't working. The higher temperature slows
down the drive drive speed.
Now I have backed up:
  Despooling elapsed time = 00:39:31, Transfer rate = 285.3 M
Bytes/second
  Using 20 GB file size and block size with 512k, wave Audio
Files, no hardware compression, 1%  lz4 software compression
  
  Thanks, Frank
  
  Am 26.04.2021 um 16:49 schrieb Udo Kaune:
  
  

Hi,

The speed test is not to the tape but to and from the spool
drive only. The transfer rates are from real data (audio files).

HTH

Am 26.04.21 um 16:17 schrieb Kern
  Sibbald:


  
  Hello,
  
  If you are sending blocks of zeros to a tape drive, it is
  useless to compare speeds to anything that sends real data. 
  Zero data input to most tape drives is almost completely
  optimized (compressed) out, so one gets very large,
  unrealistic speeds.  Try sending random data then you will get
  a better idea of how fast the drive actually can write.
  
  Best regards,
  Kern
  
  On 4/25/21 12:36 PM, Udo Kaune
wrote:
  
  

In regard to possible compression 360MB/sec seems slow for a
spool drive serving an LTO-7. This is data from an 8 drive
RAID 6 DELL box (PERC730P):

root@pangaea:~# dd if=/dev/zero of=/dev/null bs=1M
count=32000
33554432000 bytes (34 GB, 31 GiB) copied, 1.89847 s, 17.7
GB/s

root@pangaea:~# dd if=/dev/zero of=test.zero bs=1M
count=32000 conv=fsync
33554432000 bytes (34 GB, 31 GiB) copied, 29.3056 s, 1.1
GB/s

root@pangaea:~# dd if=test.zero of=/dev/null bs=1M
33554432000 bytes (34 GB, 31 GiB) copied, 27.9066 s, 1.2
GB/s

root@pangaea:~# tapeinfo -f /dev/nst0
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULTRIUM-HH7 '
Revision: 'K4K1'
Attached Changer API: No
SerialNumber: '10WT141102'
MinBlock: 1
MaxBlock: 8388608
SCSI ID: 0
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: 0x78
Density Code: 0x5c
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0xff
DeCompType: 0xff
Block Position: 5264131
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3

Device {
  Name = IBM-LTO7
  Media Type = LTO
  Archive Device = /dev/nst0
  Automatic Mount = yes
  Always Open = yes
  Removable Media = yes
  Random Access = no
  Maximum Spool Size = 256GB
  Spool Directory = /home/bacula/spool
  Maximum File Size = 32GB
  # Maximum Block Size = 524288
  Maximum Block Size = 1048576
  Backward Space File = yes
  Fast Forward Space File = yes
}

19-Apr 16:46 pangaea-dir JobId 1466: Start Backup JobId
1466, Job=srv.2021-04-19_16.00.00_05
19-Apr 16:46 pangaea-dir JobId 1466: Using Device "IBM-LTO7"
to write.
19-Apr 16:46 pangaea-sd JobId 1466: Spooling data ...
19-Apr 17:10 pangaea-sd JobId 1466: User specified Device
spool size reached: DevSpoolSize=256,000,176,844
MaxDevSpoolSize=256,000,000,000
19-Apr 17:10 pangaea-sd JobId 1466: Writing spooled data to
Volume. Despooling 256,000,176,844 bytes ...
19-Apr 17:26 pangaea-sd JobId 1466: Despooling elapsed time
= 00:15:54, Transfer rate = 268.3 M Bytes/second
19-Apr 17:26 pangaea-sd JobId 1466: Spooling data again ...
19-Apr 17:52 pangaea-sd JobId 1466: User specified Device
spool size reached: DevSpoolSize=256,000,176,881
MaxDevSpoolSize=256,000,000,000
19-Apr 17:52 pangaea-sd JobId 1466: Writing sp

Re: [Bacula-users] Tape question: fast forward and rewind

2021-05-06 Thread Kern Sibbald

  
  
Hello,

You probably forgot to use the non-rewinding drive name when
assigning the SD to a drive. 

e.g /dev/st0   is a rewinding drive
 /dev/nst0  is a non-rewinding drive

You want the non-rewinding drive when using Bacula.  See the
documentation.

Best regards,
Kern

On 5/5/21 6:53 AM, fk+bacula--- via
  Bacula-users wrote:


  
  Hi
there,
I'm using a Quantum LTO-7 tape drive. When I start a job,
the drive does with the tape:
  bacula-sd JobId 269: Volume "CRInc-001" previously written,
  moving to end of data.
bacula-sd JobId 269: Ready to append to end of Volume
  "CRInc-001" at file=5.
bacula-sd JobId 269: Spooling data ...
  so far so good. After 30 min, the tape drive rewinds the tape
to BOT (spooling is still in progress) and after spooling is
completed, the tape is moved again to the end of data and then
it starts despooling.
  Question 1): is the rewinding of the tape after 30 min a
feature of bacula or of the tape drive? Is it to protect the out
taken tape? Can I change the 30 min time and how long will be
good?
  
  Question 2): Can I avoid moving to the end before starting the
data spooling? Why does bacula do this? If is there during
despooling no enough space left, it will ask for a new volume.
Or other spoken: can it start spooling without checking the end
of the tape? How to configure it?
  
  Question 3): Would it be better to do "backup to disk to tape"
instead of using spooling to prevent unnecessary forward and
back winding of the tapes?
  Thanks, Frank
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] S3 "Driver" Doesn't Work 11.0.2

2021-05-06 Thread Kern Sibbald

  
  
Hello,

Though I am "mostly" retired, I thought I might respond to some of
your points.

First, I am sorry you are having problems building the S3 plugin.

The Bacula project does test a lot, and puts out source code prior
to release in many cases, so that users can test it and report any
problems.
The one thing that Bacula does not do is test every build option
possible -- there are probably thousands of different OS, library,
and build (deb, rpm, ...) permutations, any one of which could have
difficulties.

You asked in a previous email why we don't test on Ubuntu.  Well all
the initial development and testing of the S3 plugin was done on
Ubuntu.  I can testify to that because I am the Bacula developer who
did it -- this message is being written on a Ubuntu 20.-4 system :-)

In your case, I see that you are doing a number of build operations
that can lead to minor problems because of OS specific scripting for
debs and such.  Normally, Bacula developers build and install the
libs3 directly.  You are creating (apparently) a libs3 debs package
-- that is something that Bacula developers never do and have never
tested.

Your fundamental problem in making the S3 plugin work is that plugin
has the wrong name -- this can be seen from the -D200 debug output
you supplied.  The Bacula SD is looking for plugins named:
xxx-sd.so, and your plugin is named xxx.so, so Bacula is not finding
it.

There is a reasonable possibility that if you simply rename your
plugin from xxx.so to xxx-sd.so (I am assuming you know how to
replace xxx with the correct name), the plugin *might* work.  

If you would like to get reasonable answers from this list, it might
help if you would remember that this is free software that relies on
community participation.

One last thing: I take issue with your statements about Bacula
Systems.  Bacula Systems binaries are heavily tested, and they do
work.  In the off chance they don't the company will quickly
(usually within a day or two) build new binaries that do work.  I do
not mean to imply that you or anyone else must buy a Bacula Systems
subscription to get working software.  However, I do think it likely
that at some point a Bacula Systems developer will probably see this
problem and fix it or explain the build problem.

Good luck,
Kern

On 5/5/21 3:55 AM, Jim Joyce wrote:


  
  
  
As you can see, nobody except imaginary super admins have been
able to load the S3 plug-in. IT DOES NOT WORK. I implore the
community to demand the code & package be rescinded from the
repo until somebody with magical coding prowess can get it to
work. Community users are wasting their time and energy on
something that will not work unless you fork the darn thing and
start all over again!
  

  
  
bacula.org package versions you are trying to install.
  
 9.6.7 debs
  
libs3 version and installation commands
  
    cd libs3-20200523
  
  
    make deb
  
    cd build/pkg
  
    dpkg -i libs3_4.1.bac_amd64.deb
  
    root@backup:~/libs3-20200523/build/pkg# apt show libs3
   Package: libs3
   Version: 4.1.bac
   Status: install ok installed
operating system exact version
   cat /etc/debian_version 
     10.9
bacula-sd -d 200 loading  debug output
root@backup:/opt/bacula/bin# ./bacula-sd -d 200
  root@backup:/opt/bacula/bin# bacula-sd:
address_conf.c:274-0 Initaddr 0.0.0.0:9103
  
  backup-sd: jcr.c:131-0 read_last_jobs seek to 192
  backup-sd: jcr.c:138-0 Read num_items=2
  backup-sd: plugins.c:97-0 load_plugins
  backup-sd: plugins.c:133-0 Rejected plugin: want=-sd.so
name=bpipe-fd.so len=11
  backup-sd: plugins.c:133-0 Rejected plugin: want=-sd.so
name=bacula-sd-cloud-driver-9.6.7.so len=31
  backup-sd: plugins.c:121-0 Failed to find any plugins in
/opt/bacula/plugins
...

root@backup:/opt/bacula/plugins# ls
  bacula-sd-cloud-driver-9.6.7.so  bpipe-fd.so

  
steps to reproduce the error.
Install 9.6.7, install S3 lib hack, define a storage
device using the S3 driver in bacula-sd.conf. Volia! Get the
dreaded "driver not loaded" error.


  
  

  
  
  From: Heitor
  Faria 
  Sent: Monday, May 3, 2021 7:09 PM
  To: sruckh 
  Cc: bacula-users
  
  Subject: Re: [Bacula-users] S3 "Driver&q

Re: [Bacula-users] How to convert the catalogue to Bacula 11.0.2 mode.

2021-04-29 Thread Kern Sibbald

Hello,

Normally, it is automatic if you upgrade from a package.  If you do the 
upgrade manually, you must run the "update_bacula_tables" script that is 
in the new version.  It is normally found in the /src/cats 
directory.


Best regards,
Kern

On 4/28/21 3:17 PM, Erik P. Olsen wrote:

Hello,

I am planning to upgrade to Bacula 11.0.2 but reading about how to convert the 
catalogue
makes me a bit uneasy. The statement is: "Normally, it is automatic". What does 
it mean
that it is automatic? Does it actually run the script by itself first time 
Bacula 11.02
is started?





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] LTO-7 btape speed test

2021-04-26 Thread Kern Sibbald

  
  
Hello,

If you are sending blocks of zeros to a tape drive, it is useless to
compare speeds to anything that sends real data.  Zero data input to
most tape drives is almost completely optimized (compressed) out, so
one gets very large, unrealistic speeds.  Try sending random data
then you will get a better idea of how fast the drive actually can
write.

Best regards,
Kern

On 4/25/21 12:36 PM, Udo Kaune wrote:


  
  In regard to possible compression 360MB/sec seems slow for a spool
  drive serving an LTO-7. This is data from an 8 drive RAID 6 DELL
  box (PERC730P):
  
  root@pangaea:~# dd if=/dev/zero of=/dev/null bs=1M count=32000
  33554432000 bytes (34 GB, 31 GiB) copied, 1.89847 s, 17.7 GB/s
  
  root@pangaea:~# dd if=/dev/zero of=test.zero bs=1M count=32000
  conv=fsync
  33554432000 bytes (34 GB, 31 GiB) copied, 29.3056 s, 1.1 GB/s
  
  root@pangaea:~# dd if=test.zero of=/dev/null bs=1M
  33554432000 bytes (34 GB, 31 GiB) copied, 27.9066 s, 1.2 GB/s
  
  root@pangaea:~# tapeinfo -f /dev/nst0
  Product Type: Tape Drive
  Vendor ID: 'IBM '
  Product ID: 'ULTRIUM-HH7 '
  Revision: 'K4K1'
  Attached Changer API: No
  SerialNumber: '10WT141102'
  MinBlock: 1
  MaxBlock: 8388608
  SCSI ID: 0
  SCSI LUN: 0
  Ready: yes
  BufferedMode: yes
  Medium Type: 0x78
  Density Code: 0x5c
  BlockSize: 0
  DataCompEnabled: yes
  DataCompCapable: yes
  DataDeCompEnabled: yes
  CompType: 0xff
  DeCompType: 0xff
  Block Position: 5264131
  Partition 0 Remaining Kbytes: -1
  Partition 0 Size in Kbytes: -1
  ActivePartition: 0
  EarlyWarningSize: 0
  NumPartitions: 0
  MaxPartitions: 3
  
  Device {
    Name = IBM-LTO7
    Media Type = LTO
    Archive Device = /dev/nst0
    Automatic Mount = yes
    Always Open = yes
    Removable Media = yes
    Random Access = no
    Maximum Spool Size = 256GB
    Spool Directory = /home/bacula/spool
    Maximum File Size = 32GB
    # Maximum Block Size = 524288
    Maximum Block Size = 1048576
    Backward Space File = yes
    Fast Forward Space File = yes
  }
  
  19-Apr 16:46 pangaea-dir JobId 1466: Start Backup JobId 1466,
  Job=srv.2021-04-19_16.00.00_05
  19-Apr 16:46 pangaea-dir JobId 1466: Using Device "IBM-LTO7" to
  write.
  19-Apr 16:46 pangaea-sd JobId 1466: Spooling data ...
  19-Apr 17:10 pangaea-sd JobId 1466: User specified Device spool
  size reached: DevSpoolSize=256,000,176,844
  MaxDevSpoolSize=256,000,000,000
  19-Apr 17:10 pangaea-sd JobId 1466: Writing spooled data to
  Volume. Despooling 256,000,176,844 bytes ...
  19-Apr 17:26 pangaea-sd JobId 1466: Despooling elapsed time =
  00:15:54, Transfer rate = 268.3 M Bytes/second
  19-Apr 17:26 pangaea-sd JobId 1466: Spooling data again ...
  19-Apr 17:52 pangaea-sd JobId 1466: User specified Device spool
  size reached: DevSpoolSize=256,000,176,881
  MaxDevSpoolSize=256,000,000,000
  19-Apr 17:52 pangaea-sd JobId 1466: Writing spooled data to
  Volume. Despooling 256,000,176,881 bytes ...
  19-Apr 18:08 pangaea-sd JobId 1466: Despooling elapsed time =
  00:16:13, Transfer rate = 263.1 M Bytes/second
  19-Apr 18:08 pangaea-sd JobId 1466: Spooling data again ...
  19-Apr 18:16 pangaea-sd JobId 1466: Committing spooled data to
  Volume "G-0620-6". Despooling 56,071,784,596 bytes ...
  19-Apr 18:19 pangaea-sd JobId 1466: Despooling elapsed time =
  00:03:04, Transfer rate = 304.7 M Bytes/second
  19-Apr 18:19 pangaea-sd JobId 1466: Elapsed time=01:32:46,
  Transfer rate=102.0 M Bytes/second
  19-Apr 18:19 pangaea-sd JobId 1466: Sending spooled attrs to the
  Director. Despooling 6,326 bytes ...
  19-Apr 18:19 pangaea-dir JobId 1466: Bacula pangaea-dir 9.6.7
  (10Dec20):
  
  HTH, Udo

Am 21.04.21 um 09:14 schrieb fk+bacula--- via Bacula-users:
  
  

Hello
  there,
I'm using a Quantum LTO-7 HH drive with Ultrium LTO-7 tapes.
  The streamer is connected on a SAS HAB microsemi 1100-4i.
I set the Maximum block size = 524288
and run: speed file_size=3 skip_raw
with zero data and bacula block structure Total Write rate =
  311.7 MB/s
  with random data: Total Write rate = 94.74 MB/s
For backup I use the spooling function. Two SATA hdd's are
  build a RAID-0 (stripe) array.
  This array is used for holding the spooling data (spool
  directory)

testing:
   write test  
  dd if=/dev/zero of=/backup_spool/testfile bs=1M count=204800
  conv=fdatasync,notrunc
  204

Re: [Bacula-users] LTO-7 btape speed test

2021-04-21 Thread Kern Sibbald

  
  
Hello,

From my experience (up to LTO-5), using 524288 as a block size is a
reasonable size.  However LTO-7 drives are even faster, so you might
try some tests with Maximum block size = 1048576.  Some people
recommend much larger sizes, but in my tests (LTO-5) there is little
speed gain with even bigger block sizes, and according to a Quantum
technical tape specialist, you might experience more errors.

Also you should set Maximum File Size = 5G or bigger.  This
determines how often Bacula writes EOF marks, which really slow down
writing.  The bigger Maximum File Size is the faster (in general)
Bacula can stream to tape.  However, if you set it too big it will
take longer to get a single file back.  Bacula can position quickly
to a block boundary but must search within a block for any given
file.  If you are restoring everything, then Bacula will only have
to search on the first file to be restored -- all the rest of the
files will be found sequentially.

Good luck.

Kern

On 4/21/21 9:14 AM, fk+bacula--- via
  Bacula-users wrote:


  
  Hello
there,
  I'm using a Quantum LTO-7 HH drive with Ultrium LTO-7 tapes.
The streamer is connected on a SAS HAB microsemi 1100-4i.
  I set the Maximum block size = 524288
  and run: speed file_size=3 skip_raw
  with zero data and bacula block structure Total Write rate = 311.7
  MB/s
with random data: Total Write rate = 94.74 MB/s
  For backup I use the spooling function. Two SATA hdd's are
build a RAID-0 (stripe) array.
This array is used for holding the spooling data (spool
directory)
  
  testing:
 write test  
dd if=/dev/zero of=/backup_spool/testfile bs=1M count=204800
conv=fdatasync,notrunc
204800+0 Datensätze ein
204800+0 Datensätze aus
214748364800 bytes (215 GB, 200 GiB) copied, 596,839 s, 360
  MB/s

 read test 
dd if=/backup_spool/testfile of=/dev/null bs=1M count=204800
204800+0 Datensätze ein
204800+0 Datensätze aus
214748364800 bytes (215 GB, 200 GiB) copied, 625,029 s, 344
  MB/s
  Have made test backup (1,1 TByte) and have monitored a constant
write rate of 200 MByte/s during despooling.
  
  Are the write values of the tape drive OK for a LTO-7
environment?
  Thanks for any hints, Frank
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] windows file - fileset path

2020-12-12 Thread kern
Hello,Try putting the path in double quotes.Regards,KernSent from my Galaxy
 Original message From: "mac-eduki2.co.uk"  
Date: 12/12/20  15:43  (GMT+01:00) To: Bacula-users@lists.sourceforge.net 
Subject: [Bacula-users] windows file - fileset path Hi thereHope all is wellMay 
I ask if anyone can offer advice on setting the file path  in my file set for 
Windows 2012 r2 to target program filesI have other File paths to direct 
folders working correctly, for exampleFile = c:/MyStuffThis is where I want to 
goFile = C:/ProgramFiles(x86)/SmarterTools/SmarterMail/ServiceLeaving spaces 
between Program and Files does not seem to work eitherI am on Bacula 7.6Error 
message from bconsole  Could not stat 
"C:/ProgramFiles(x86)/SmarterTools/SmarterMail/Service": ERR=The system cannot 
find the path specified.Any help would be appreciatedKind 
RegardsBrad___Bacula-users mailing 
listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Signing keys

2020-12-11 Thread Kern Sibbald

Hello,

There was a small error in my turnover to Eric.  I accidentally gave him 
an old signing key for the release packages.  This means that those of 
you who check the signing key on the source downloads got errors.  This 
should now be corrected -- I have replaced all the .sig files that go 
with the source release.  And I have now given Eric the correct private 
key so he can also sign packages :-)


Please note: the only files that have changed are the ones named .sig.  
All the source files are fine and have not changed, so no need to 
re-download the source files.


Sorry for the inconvenience.

If you find any additional problems, please let me know.

Best regards,
Kern


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Status Report 9 December 2020

2020-12-09 Thread Kern Sibbald

Hello,

I would like to discuss the following subjects:

1. Bacula Release Status

2. Next Major Bacula Version

3. New Bacula Project Manager

===

1. Bacula Release Status
The latest Bacula version 9.6.7 will be released shortly.  This
is a bug fix release that fixes a number of bugs especially the
MySQL/MariaDB schema problems.  It also has a MySQL/MariaDB retry
on a write failure.

2. Next Major Bacula Version
The next Major Bacula Version (release 11.0.0) will be made within
a week or so. Although this release has been extensively tested, at
this time until we get feedback from community testers, we consider
it a Beta version.

Due to increasing divergences of the Bacula Systems Enterprise and the
Community versions, Eric Bollengier undertook a rather big project to make a
"universal Bacula core code" version that includes both the Enterprise and
Community code in a way that we can easily switch between the two 
versions, as

well as much more easily backport new features to and from the Community
version.  The first version that contains this "universal Bacula code" is
version 11.0.0.

Some of the new features in this code are:
- New catalog format giving improved performance
- Automatic TLS PSK encrypted communications
- Support for Client behind NAT
- Continuous Data Protection (CDP) Plugin
- Built-in Client scheduler
- Global Director Autoprune directive
- Events/Audit features
- New Baculum features
- Ask to mount/create volume when disk space is low
- Simplification of the Windows FileSet with File=/
- Security enhancements for Restricted Consoles

The following version to be released around June 2021 will contain:
- Kubernetes Plugin
- Generic Cloud Plugin

So as you see many new things are coming, and they will keep coming because
Bacula Systems is creating lots of new code and backporting it.

3. New Bacula Project Manager
In January 2021, it will be 21 years that I (Kern) have worked on the Bacula
project. Now it is time to leave the work to younger and enthusiastic
new people. So effective with Bacula release 11.0.0, Eric Bollengier
will replace me as the Bacula Project Manager. Eric has been working
on Bacula since 2005 and with Bacula Systems since its creation in
2008.  He is currently the most experienced and knowledgeable Bacula
architect and programmer.

Even though I am giving the project management
responsibility to Eric, I will be around for at least a year to help
out where and when I can.

I must say that I am very proud to have worked with so many Bacula
contributors and users, who are very friendly, kind, and knowledgeable.
Thank you all for helping by using Bacula or contributing to it.

Thanks for using Bacula -- be happy and stay safe.

Kern



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-devel] Release 11.0.0 not signed by correct key

2020-12-09 Thread Kern Sibbald

Hello Everyone,

Bacula version 11.0.0 is indeed a major new release, and in a few hours, 
I will send an email announcing it.  However, it is not quite ready -- 
there are a number of minor cleanups we need to do before it is ready 
for Beta testing -- namely, fix some incorrect copyrights, make sure the 
packages are properly signed, ...


Unfortunately, it was released before being entirely ready, but we have 
removed the release and will make it probably next week -- possibly 
later this week.
In the mean time, we hope to release 9.6.7 tomorrow, which has a few 
minor fixes -- including MariaDB compatible schemas and a retry when a 
MySQL/MariaDB failure occurs.


Best regards,
Kern

On 12/7/20 4:32 PM, Sven Hartge wrote:

On 07.12.20 16:23, Phil Stracchino wrote:

On 12/7/20 10:15 AM, Phil Stracchino wrote:

So I presume it safe to infer that 11.0 is a MAJOR update?



I'll pull it and do a test build on Solaris 11.


...except I need to pull from a different target, right? Someone wnat
to give me a pointer here?


Well, 11.0.0 *was* available from 
https://sourceforge.net/projects/bacula/files/ but it seems to have 
been pulled, I guess because of the problems with the GPG signature I 
mentioned.


Grüße,
Sven.


___
Bacula-devel mailing list
bacula-de...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] ERR=Function not implemented

2020-10-05 Thread Kern Sibbald

  
  
Hello,

Sven gave a very nice comment.  However, please note that the error
message was generated by the
operating system, and though it might be a bit confusing in this
context, it is probably reasonable, because
Bacula was apparently attempting something not permitted on that
"virtual" file.

Best regards,
Kern

On 10/4/20 6:46 PM, Marc Chamberlin via
  Bacula-users wrote:


  
  Hello -  I am running Bacula Version: 9.6.6 on OpenSuSE Leap
15.2 and I just set it up to backup my laptop. It worked except
that I got tons of warning messages such as the following -
  04-Oct 03:05 bigbang-fd JobId 1782: Error: Read error on file /sys/kernel/slab/:d-0001024/alloc_calls. ERR=Function not implemented
04-Oct 03:05 bigbang-fd JobId 1782: Error: Read error on file /sys/kernel/slab/:d-0001024/free_calls. ERR=Function not implemented
04-Oct 03:05 bigbang-fd JobId 1782: Error: Read error on file /sys/kernel/slab/TCPv6/alloc_calls. ERR=Function not implemented
04-Oct 03:05 bigbang-fd JobId 1782: Error: Read error on file /sys/kernel/slab/TCPv6/free_calls. ERR=Function not implemented

I took a look at these files and the only thing of interest is that these are all empty files that were installed by the distro. But these files do exist and do have the same ownership/permissions that other files have that are successfully backed up.  Any ideas on how to clear up this sea of warning messages I am getting?

FWIW - Seems to me that the developers could have come up with a better ERR message. Like saying what function was not implemented, or explaining why this failure is occurring. Something to give the users a better idea on how to handle/fix the problem. Or at a minimum telling the user that a serious problem occurred and to contact the developers if no solution is available for users. IMHO of course...

Thanks in advance for helping me with this,  Marc...

  -- 
_   _   .   .   .       .   .   .   _   _       .   _  
_   _   _   .       .   .   .           _   .   .       .  
        .   _   _       .   _       _   _   .   .   .      
.   _   _   .       _   .   .   _       .   _   _          
_   _       .   _       .   _   .       _   .   _   . 
Computers: the final frontier. These are the voyages of the
  user Marc. 
  His mission: to explore strange new hardware. To seek out new
  software and new applications. To boldly go where no Marc has
  gone before!
(Attached is my public key to be used for encryption and
  sending encrypted email to m...@marcchamberlin.com.)
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 5th and 6th week of the month

2020-09-28 Thread Kern Sibbald

Hello,

You have to look at the problem from a slightly different point of view.

The filters or specifiers 1st, ... 6th simply say that if the
current day of the year falls on the Nth week of a particular
month then the Job will run, otherwise, that particular run
directive will be disregarded.

A week is defined as Monday through Sunday.  This is an arbitrary
definition that is fixed in Bacula.  That is 1st applies to any day
that falls in the 1st week of a month.  Note: not all weeks have
a Monday or a Tuesday or any other day (see below for a
month that has only Sunday in the first week).  The 1st week may
only be a Sunday if the first day of a given month is on Sunday.
In that case the following day (Monday) will be in the 2nd week of the
month.

With that logic, it turns out that it is possible to have a month
that has at least one day in each of 6 separate weeks.

Take as an example a 31 day month where the first day of the
month falls on a Sunday.  In that case, the 30th and 31st days
of the month will fall on Monday and Tuesday in the 6th week
of that month thus the need for 6th.

Best regards,
Kern

On 9/28/20 2:31 PM, djosip+n...@linuxpages.net wrote:

Hello,


Somewhere with Bacula 7, a "6th" week was introduced for Job Scheduler.

I couldn't find the definition of the 6th week in the documentation
and I can only assume that it is meant to be used to target the first
few days of the month which can occur before the "1st" week of the month.

Could someone clarify the "6th" week keyword?


I would also like to know the exact definition of the 5th week of the
month as the Bacula documentation defines week like this:
"A week starts with Monday and ends with Sunday.".

If the 5th week of the month covers the rest of the days after the "4th"
week of the month, the documentation should be fixed and the definition
of the week should be replaced with:
"The first week starts with Monday and ends with Sunday."



Regards!





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.6.6

2020-09-24 Thread Kern Sibbald

Hello,

We are pleased to announce the release of Bacula version 9.6.6. This is 
a minor bug fix release.


Unless there is some new major bug found, this will be the last of the 
9.6.x releases. The next release major release (a really big one) is 
currently scheduled for December.


More about this in a subsequent status report ...Please see the 
ReleaseNotes for how to build Bacula 9.6.5 with the correct libs3 for 
use with the Amazon Cloud.


The binaries (rpm, deb) should be ready in a week or two.

Thank you for using Bacula,
Kern

== ReleaseNotes =
18Sep20
 - baculum: Fix displaying date and time on volume view page
 - baculum: Fix #2564 changing volume status to Read-Only
 - baculum: Fix saving multicombobox control values
 - baculum: Fix multicombobox and multitextbox controls to work with 
PHP 7.4

 - baculum: Fix #2562 displaying date and time in tables on Safari
 - Update copyright year
 - Clarify SD vbackup device error message
 - Fix MT2554 :update upload_cache call in dircmd.c
 - zero malloced memory when SMARTALLOC not enabled
 - baculum: Fix #2558 saving day or day range in schedule resource - 
reported by

   Jose Alberto
 - Fix MT2554 :upgrade upload_cache interface.
 - Eliminate compiler errors when smartalloc is turned off
 - baculum: Fix date formatter to work with PHP 7.4
 - baculum: Fix configure Bacula hosts page to work with PHP 7.4
 - baculum: Update Portuguese translations
 - baculum: Change colours in table headers and borders
 - baculum: Add remove runscript button
 - Fix #6366 About an issue with verify job level=DiskToCatalog
 - baculum: Close modal windows on pressing ESC key
 - baculum: Fix logout button on authorization failed page
 - baculum: Add local user authentication method support
 - baculum: Add date and time formatters - idea proposed by Wanderlei 
Huttel

 - baculum: Enable re-running jobs in async way and visual improvements
 - baculum: Change cursor over selectable table
 - Fix build script copyright detection
 - release: add code to detect Bacula Systems copyrights and fail release
 - Fix compilation of bsnapshot on Fedora

Bugs fixed/closed since last release:
2558 6366



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Error to compile bacula 9.6.5 in CentOS 7.8

2020-09-12 Thread Kern Sibbald

  
  
Hello Wanderlei,

using:  

make -j 8

is improper syntax and will probably cause a failure.  If you want
to run 8 simultaneous compiles, you must use:

make -j8

Note: without the space.

Best regards,
Kern

On 9/8/20 1:59 PM, Wanderlei Huttel
  wrote:


  
  

  

  
  
  Hello Radoslaw! 

I've tried to compile without the flag "-O2" but
unfortunately it doesn't work! 

I usually compile Bacula using "make -j 8".
Then I've done a test and run only "make" command
  and now it  worked!
  
  Even with CFLAGS="-g -O2 -Wall" 

  
  
  

  

  

  

  

  Best regards
  
  
  Wanderlei
  Hüttel
  
  

  

  

  

  

  
  

  

  
  
  
Em seg., 7 de set. de 2020 às
  03:37, Radosław Korzeniewski <rados...@korzeniewski.net>
  escreveu:


  
Hello,
  
  


  czw., 3 wrz 2020 o
16:52 Wanderlei Huttel <wanderleihut...@gmail.com>
napisał(a):
  
  

  

  

  

  

  

  

  

  

  

  

  
  
  I'm
  trying to
  compile bacula
  in CentOS 7.8
  and I'm
  getting amd
  error!
  
  It looks the
  error is in
  the dird and
  is a c++ bug.
  
  How to avoid

Re: [Bacula-users] Question about the aligned plug

2020-08-24 Thread Kern Sibbald

Hello,

Please go to www.bacula.org and download the binaries that correspond to 
your system.


There is no key for the aligned volume plugin.

If you have any questions or problems, you must subscribe to 
bacula-users email list on source forge, then you can send emails to 
that list and hopefully get answers (no guarantee).  The web site 
documents how to join this list.


Best regards,

Kern

On 8/23/20 9:10 PM, inż. Adam Aniołek wrote:

Good morning,
I use bacula system but no deduplication in the community version. Due 
to the constantly growing amount of data, I wanted to use the zfs + 
aligned plugin. Therefore, I am asking you to provide a key for this 
plugin. If it is possible?


best regards
Adam Aniołek




--
This e-mail and any attached files are confidential and may be legally 
privileged. If you are not the addressee, any disclosure, 
reproduction, copying, distribution, or other dissemination or use of 
this communication is strictly prohibited. If you have received this 
transmission in error please notify the sender immediately and then 
delete this mail.
E-mail transmission cannot be guaranteed to be secure or error free as 
information could be intercepted, corrupted, lost, destroyed, arrive 
late or incomplete, or contain viruses. The sender therefore does not 
accept liability for any errors or omissions in the contents of this 
message which arise as a result of e-mail transmission or changes to 
transmitted date not specifically approved by the sender.
If this e-mail or attached files contain information which do not 
relate to our professional activity we do not accept liability for 
such information.





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Issue with config setting "Max Virtual Full Interval"

2020-08-16 Thread Kern Sibbald

  
  
Hello,
This means that you do not have both a read device and a write
  device defined for the job that is running.  Unfortunately the
  error message does not tell you which device (read or write) is
  not defined in the job definition.
Perhaps if you turn on a debug level of about 150 or possibly
  200, you might see more clearly what is wrong.  In the next
  version, I will improve the error message to be specific about
  which of the two devices (read and write) is not defined.

Best regards,
Kern

On 8/15/20 12:40 PM, Shaligram Bhagat,
  Yateen (Nokia - IN/Bangalore) wrote:


  
  
  
  
  
  Hi Martin & Radoslaw,
   
  This is an old issue raised by me in Jan 2020; which both
of you had responded to earlier.
  I am seeing it again as the particular scenario causing
the issue did not appear till now since my last reporting.
   
  I am using bacula v 9.4.4 on Centos 6.4. for disk file
based backup.
   
  The backup scheme is weekdays incremental and virtual
full back-up on Saturday.
   
  Things were running fine till the interval between the
last virtual  full back up job and the current incremental
job was less than Max Virtual Full Interval = 7 days
   
   When this interval exceeded Max Virtual Full Interval =
7 days ;  for an incremental backup, bacula tried to create
a new virtual full back-up by consolidating the latest
virtual full backup and all subsequent incremental backups
(...fair enough this
is as expected) But while consolidating the jobs ; it gave
error :
   
   
  


  
Job Log
  bacula-server-dir 
   No prior or suitable Full backup found in
catalog. Doing Virtual FULL backup.
  bacula-server-dir shell command: run BeforeJob
"/opt/bacula/srpg/scripts/validate_testbed.sh
blhwsync11"
  bacula-server-dir Consolidating
JobIds=49733,49985,50189,50390,50575,50769,50964
   Start Virtual Backup JobId 51162,
Job=blhwsync11.2020-08-15_15.40.22_18
   BeforeJob: blhwsync11 is available in
regress.params and reachable
  bacula-server-dir shell command: run AfterJob
"/opt/bacula/srpg/scripts/send_fail_mail.sh 51162
blhwsync11.2020-08-15_15.40.22_18
sas-backup-ad...@list.nokia.com"
  bacula-server-sd Elapsed time=443746:10:30,
Transfer rate=0  Bytes/second
  bacula-server-dir 
  Error: Bacula bacula-server-dir 9.4.4 (28May19):
    Build OS:  
x86_64-redhat-linux-gnu-bacula redhat 
    JobId:  51162
    Job:   
blhwsync11.2020-08-15_15.40.22_18
    Backup Level:   Virtual Full
    Client: "blhwsync11-fd" 9.4.4
(28May19) x86_64-redhat-linux-gnu-bacula,redhat,
    FileSet:    "blhwsync11" 2020-07-29
13:30:02
    Pool:   "TestbedFull-F" (From
Job VFullPool override)
    Catalog:    "MyCatalog" (From
Client resource)
    Storage:    "StorageF" (From Pool
resource)
    Scheduled time: 15-Aug-2020 15:40:22
    Start time: 13-Aug-2020 22:01:41
    End time:   13-Aug-2020 22:11:43
    Elapsed time:   10 mins 2 secs
    Priority:   10
    SD Files Written:   0
    SD Bytes Written:   0 (0 B)
    Rate:   0.0 KB/s
    Volume name(s): 
    Volume Session Id:  28
    Volume Session Time:    1597408469
    Last Volume Bytes:  1 (1 B)
    SD Errors:  1
    SD termination status:  Error
    Termination:    *** Backup Error ***
  bacula-server-sd Fatal
error: Read and write devices not properly
initialized.
  bacula-server-dir Using Device "DeviceF1&quo

Re: [Bacula-users] Solution for a strange network error affecting long running backups

2020-08-08 Thread kern

Hello Josh,You did a really nice job tracking this down.Bottom line for me: 
these kinds of network software errors are really scarry :-(Best 
regards,KernPS: the new code that allows Bacula to reconnect is now complete.  
However is will be sometime before it will be released, because we do lots of 
testing -- especially stress testing.Sent from my Samsung Galaxy smartphone.
 Original message From: Josh Fisher  Date: 
8/5/20  18:58  (GMT+01:00) To: bacula-users@lists.sourceforge.net Subject: 
[Bacula-users] Solution for a strange network error affecting long
  running backups Thought I would share my experience debugging a strange 
problem affecting long running backups. A particular job was failing with a 
"network send error to SD" at between 90-120 minutes after 100s of GB were 
written. Enabling heartbeat on Dir, SD, and client had no effect and the 
problem persisted.Some background. The client was a KVM VM running Centos 7 and 
Bacula 9.6.5. Bacula SD and Dir run together on one node of a 
Pacemaker-Corosync cluster, also Centos 7 and Bacula 9.6.5. Bacula server 
daemons can failover successfully for incremental backups, but not for full (no 
dual-port backup devices). Cluster uses a mix of DRBD volumes and iSCSI LUNs. 
There are three networks involved; one dedicated to DRBD, one dedicated to 
iSCSI, and a LAN connecting everything else. There were no obvious problems 
with any other VMs or cluster nodes. There didn't appear to be any networking 
issues. In both VM and cluster nodes, OS is Centos 7.8 with stock Centos kernel 
3.10.0-1127.13.1 and qemu-kvm 1.5.3-173I have had issues with Bacula jobs 
failing due to intermittent network issues in the past and they turned out to 
be either hardware errors or buggy NIC drivers. Therefore, the first thing I 
tried was moving the client VM to run on the same cluster node that the Bacula 
daemons were running on. This way the VM's virtual NIC and the cluster node's 
physical NIC are attached to the same Linux bridge interface, so traffic 
between the two should never go on the wire, eliminating the possibility of 
switch, wiring, and other external hardware problems. No luck. Exactly the same 
problem.Next I turned on debugging for the SD. This produced a tremendous 
amount of logging with no errors or warnings until after several hundred GB of 
data was received from the client and suddenly there was a bad packet received, 
causing the connection to be dropped. The Bacula log didn't lie. There was 
indeed a network send error. But why?Not having any knowledge of the internals 
of the Linux bridge device code, I thought that perhaps the host's physical NIC 
also attached to the bridge that bacula-sd is listening on, might somehow cause 
a problem. To eliminate that, I swapped the NIC in that host. I didn't have a 
different type of NIC to try, so it was replaced with another Intel i350 and of 
course used the same igb driver. Didn't work, but shows that it's not likely a 
NIC hardware error. Could a bug in the igb driver cause this? Maybe, but the 
NIC appeared to work flawlessly for everything else on the cluster node, 
including a web server VM connected to it through the same bridge. Or could it 
be the virtio_net driver? Again, it appears to work fine for the web server VM, 
but let's start with the virtio_net driver, since virtio_net (the client VM) is 
the sender and igb (bacula-sd listening on the cluster node's physical NIC) is 
the receiver.So I searched for virtio-net and/or qemu-kvm network problems. I 
didn't find anything like this, exactly, but I did find that people reported VM 
network performance problems and latency issues and that, several qemu-kvm 
versions ago, the solution was to disable some TCP offloading features. I 
didn't have high expectations, but I disabled segmentation offload (TCP and 
UDP), as well as generic receive offload, on all involved NICs, started the job 
again, and SURPRISE, it worked! Ran for almost 3 hours, backing up 700 GB 
compressed and had no errors.Conclusion: There is a bug somewhere! I think 
maybe the checksum calculation is failing when segmentation offload is enabled. 
It seems that checksum offload works so long as segmentation offload is 
disabled. I didn't try disabling checksum offload and re-enabling segmentation 
offload, nor did I try re-enabling generic receive offload.To disable 
segmentation offload I used:/sbin/ethtool -K ethX tso off gso off gro offI 
disabled those on all interfaces involved. It may only be necessary to do this 
on one of the involved interfaces. I don't know. I just don't have time to try 
all permutations, and this seems to work with little or no performance 
degradation, at least in my case.Once again, Bacula shows itself to be the most 
demanding network app that I know of, and so able to trigger all of the most 
obscure and intermittent networking 
problems.___Bacula-users mailing 

Re: [Bacula-users] bacula - optimize storage for cloud sync

2020-07-08 Thread Kern Sibbald

  
  
Hello Ziga,
Yes, you might be able to do what you want using a "debug"
  feature of the Bacula Cloud driver.  It is not very well
  documented, but there is one section "3 File Driver for the Cloud"
  in the "Bacula Cloud Backup" that mentions it.
Basically instead of using the "S3" driver in the Cloud resource
  of your Storage Daemon, you use "File" and the HostName becomes
  the path where the Cloud volumes (directories + parts) will be
  written.  For example, I use the following for writing to disk
  instead of an S3 cloud.
Cloud {
    Name = DummyCloud
    Driver = "File"
    HostName = "/home/kern/bacula/k/regress/tmp/cloud"
    BucketName = "DummyBucket"
    AccessKey = "DummyAccessKey"
    SecretKey = "DummySecretKey"
    Protocol = HTTPS
    UriStyle = VirtualHost
  }

The Device resource looks like:
    Device {
    Name = FileStorage1
    Media Type = File1
    Archive Device = /home/kern/bacula/k/regress/tmp
    LabelMedia = yes;   # lets Bacula label
  unlabelled media
    Random Access = Yes;
    AutomaticMount = yes;   # when device opened, read
  it
    RemovableMedia = no;
    AlwaysOpen = no;
  
    Device Type = Cloud
  
    Cloud = DummyCloud
  }

I know the code runs and produces correct output, but I am not
  sure how it will work in your environment.  If it works, great. 
  If it doesn't work, for the near future, unfortunately I cannot
  provide support, but at some point (probably 3-6 months) the
  project may support this feature.
Note: the next version of Bacula coming in a few months will have
  a good number of new features and improvements to the Cloud driver
  (more bconsole commands for example).

Good luck, and best regards,
Kern
PS: If it does work for you, I would appreciate it if you would
  document it and publish it on the bacula-users list so others can
  use the Oracle cloud.




On 7/8/20 3:41 PM, Žiga Žvan wrote:


  
  Hi Mr. Kern,
  
  My question was a bit different. I have noticed that Oracle S3
is not compatible, therefore I have implemented Oracle Storage
gateway (a docker image that uses local filesystem as a cache
and moves the data automatically to oracle cloud). I have this
filesystem mounted (nfsv4) on bacula server and I am able to
backup data to this storage (and hence in cloud).
  I have around 1 TB data daily and I'm a bit concerned about the
bandwidth. It will take app. 4 hours to sync to the cloud and I
need to count in the future growth. As long as bacula writes
data to one file/volume, where it stores full and incremental
backups, this is not optimal for the cloud (the file will change
and all the data will upload each day). I have noticed that
bacula stores data differently in the cloud configuration.
Volume is not a file, but a folder with fileparts. This would be
better for me, because only some fileparts would change and move
to the cloud via Storage gateway. So the question is:
Can I configure bacula-sd to store data in fileparts, without
actual cloud sync? Is this possible? I have tried several
configurations of a bacula-sd device with no luck. Should  I
configure some dummy cloud resource?
  Kind regards,
   Ziga Zvan
  
  
  On 07/07/2020 14:40, Kern Sibbald
wrote:
  
  

Hello,
Oracle S3 is not compatible with Amazon S3 or at least with
  the libs3 that we use to interface to AWS and other compatible
  S3 cloud offerings.  

Yes, Bacula Enterprise has a separate Oracle cloud driver
  that they wrote.  There are no plans at the moment to backport
  it to the community version.
Best regards,
Kern

On 7/7/20 8:43 AM, Žiga Žvan wrote:


  
  

  
  Dear all,
  I'm testing communty version of bacula in order to
change backup sw for app. 100 virtual and physical
hosts. I would like to move all the data to local
storage and then move them to public cloud (Oracle
Object storage).

I believe that community version of the software suites
our needs. I have installed:
-version 9.6.5 of bacula on centos 7 computer
-oracle storage gateway (similar to aws SG - it moves
data to object storage and exposes it localy as nfsv4;
f

Re: [Bacula-users] bacula - optimize storage for cloud sync

2020-07-07 Thread Kern Sibbald

  
  
Hello,
Oracle S3 is not compatible with Amazon S3 or at least with the
  libs3 that we use to interface to AWS and other compatible S3
  cloud offerings.  

Yes, Bacula Enterprise has a separate Oracle cloud driver that
  they wrote.  There are no plans at the moment to backport it to
  the community version.
Best regards,
Kern

On 7/7/20 8:43 AM, Žiga Žvan wrote:


  
  

  
  Dear all,
  I'm testing communty version of bacula in order to change
backup sw for app. 100 virtual and physical hosts. I would
like to move all the data to local storage and then move
them to public cloud (Oracle Object storage).

I believe that community version of the software suites our
needs. I have installed:
-version 9.6.5 of bacula on centos 7 computer
-oracle storage gateway (similar to aws SG - it moves data
to object storage and exposes it localy as nfsv4; for bacula
this is backup destination).
  I have read this two documents regarding bacula and cloud
https://blog.bacula.org/whitepapers/CloudBackup.pdf
https://blog.bacula.org/whitepapers/ObjectStorage.pdf
  It is mentioned it the document above, that Oracle Object
storage is not supported at the moment. 
Is it possible to configure bacula Storage device in
a way that uses Cloud media format (directory with
file parts as a volume, instead of a single file as a
volume)  without actual cloud sync (Storage Gateway
does this in my case)? I am experimenting with variations of
the definition bellow, but I am unable to solve this issue
for now (it tries to initialize cloud plugin or it writes to
a file, instead of a directory). 
  
  Device {
  Name = FSOciCloudStandard
#  Device type = Cloud
  Device type = File
#  Cloud = OracleViaStorageGateway
  Maximum Part Size = 100 MB
#  Media Type = File
  Media Type = CloudType
  Archive Device = /mnt/baculatest_standard/backup
  LabelMedia = yes;   # lets Bacula label
unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;   # when device opened,
read it
  RemovableMedia = no;
  AlwaysOpen = no;
}
  

Is there any plan to support oracle object storage in near
future? It has S3 compatible API and bacula enterprise supports
it...
  Kind regards,
Ziga Zvan
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] file: bacula-sd-cloud-driver-9.6.5.so ?

2020-06-26 Thread Kern Sibbald

  
  
Hello,
It seems to me that you must explicitly do the following to have
  the cloud driver installed
cd 
./configure 
make   # or what I do is "make -j9"
make install
cd src/stored
make src/stored install-cloud
I suspect that you did not do the last two commands, and that was
  probably because I forgot to document it as such.  I developed the
  driver about 3 years ago, and now it is automatically built for me
  by scripts when I want to test it.  Sorry.
Note, the driver is a shared object, but it is not a plugin, so
  it normally resides in the libs directory rather than the plugins
  directory.   In the next major version of Bacula (version 11.0.0),
  it will be a loadable driver and thus will not be required as it
  is in this version.

Best regards,
Kern

On 6/25/20 11:21 PM, r0...@nxlplyx.com
  wrote:


  
  Hi,
  
  I
tried to backup to an S3 provider with a shiny new 9.6.5
compilation on Debian 10, but I got this message:
  ===
  
  25-Jun
16:36 debian-sd JobId 2: Fatal error: init_dev.c:462
[SF0020] dlopen of SD driver=cloud at
/opt/bacula/lib/bacula-sd-cloud-driver-9.6.5.so failed:
ERR=/opt/bacula/lib/bacula-sd-cloud-driver-9.6.5.so: cannot
open shared object file: No such file or directory.
  ===
  
  I
compiled the S3 library distribution, I did not get any
errors.  
  
  I
searched the distribution and build directories, but nothing
resembling this was there.
  Where
did I go wrong?
  - Al
  
  
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Cloud driver for OCI

2020-06-23 Thread Kern Sibbald

  
  
Hello Sven,
Good point

libs3.git is currently available on www.bacula.org in exactly the
  same way that bacula.git is available.  However, I have never
  tried to clone it as someone other than myself, so I am not sure
  if it will actually work for you.  Please let me know if it does
  not -- probably some permissions.  My intention is to have the
  libs3 git repository publicly available on bacula.org.

The simplest and main way I have currently released libs3 is in a
  tar file that was pulled directly from the above mentioned repo. 
  With that, at least users can build Bacula even if they do not
  know how to access git.

Best regards,
Kern

On 6/23/20 6:28 PM, Sven Hartge wrote:


  On 23.06.20 17:26, kern wrote:


  
What I would appreciate the most would be for a community to submit a
patch to our current libs3 code base.

  
  
You having the bacula-libs3 code available via a public git repository
might be a first step in that direction.

Grüße,
Sven.


  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula 9.6.3 - Could not open Cloud driver

2020-06-23 Thread Kern Sibbald

  
  
Oops.  It should have read:
"Unfortunately it seems that the S3 driver NEVER
  worked on 9.6.3.   You will need 9.6.4 or 9.6.5."


Sorry for the confusion.

    
Kern



On 6/23/20 5:08 PM, kern wrote:


  
  Hello,
  
  
  Unfortunately it seems that the S3 driver worked
on 9.6.3.   You will need 9.6.4 or 9.6.5.
  
  
  Best regards,
  Kern
  
  
  
  
  
  
  
Sent from my
  Samsung Galaxy smartphone.
  
  
  
  
  
  
 Original message 
From: Mario Pranjic  
Date: 6/22/20 13:23 (GMT+01:00) 
To: bacula-users@lists.sourceforge.net 
Subject: [Bacula-users] Bacula 9.6.3 - Could not open Cloud
  driver 


  
  Hi,
  I am trying to set-up Bacula with Cloud driver to write to S3
bucket.
  OS: Ubuntu 18.04 LTS (fully patched)

Bacula packages (installed from Bacula repository on
bacula.org):
ii  bacula-client 
9.6.3-1 amd64   
Network backup, recovery and verification - file daemon
ii  bacula-cloud-storage  
9.6.3-1 amd64   
Network backup, recovery and verification - Cloud Storage Plugin
ii  bacula-common 
9.6.3-1 amd64   
Network backup, recovery and verification - common support files
ii  bacula-console
9.6.3-1 amd64   
Network backup, recovery and verification - text console
ii  bacula-postgresql 
9.6.3-1 amd64   
Network backup, recovery and verification - PostgreSQL storage
for Director
  
  Default installation works like a charm (as usual), but then I
tried to set-up cloud storage in bacula-sd (following https://blog.bacula.org/whitepapers/CloudBackup.pdf).
  I followed example on pages 8 and 9 and came up with (keys and
realnames are stripped):
  Device {
  Name = CloudStorage
  Device Type = Cloud
  Cloud = TEST1
  Archive Device = /data/bacula/backup-cloud
  Maximum Part Size = 10 MB
  Media Type = CloudType
  LabelMedia = yes
  Random Access = yes
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no
}

Cloud {
 Name = TEST1
  Driver = "S3"
  HostName = "s3.amazonaws.com"
  BucketName = "mybycket"
  AccessKey = "access_key_here"
  SecretKey = "secret_access_key_here"
  Protocol = HTTPS
  UriStyle = VirtualHost
  Truncate Cache = No
  Upload = EachPart
  Region = "us-east-1"
  MaximumUploadBandwidth = 5MB/s
}

  
  When I try to reference it in bacula-dir.conf:
  JobDefs {
  Name = "DefaultJob"
  Type = Backup
  Level = Incremental
  Client = bacula-fd
  FileSet = "Full Set"
  Schedule = "WeeklyCycle"
  Storage = CloudType    #referencing cloud storage
  Messages = Standard
  Pool = File
  SpoolAttributes = yes
  Priority = 10
  Write Bootstrap = "/opt/bacula/working/%c.bsr"
}
  
  Bacula director fails to start with bacula-sd error:
Jun 22 12:38:42 bacula bacula-sd: cloud_dev.c:690 Could not open
Cloud driver type=1 for Device=CloudStorage.
  
  Am I looking at some potential bug here or I got something
wrong?
  
  All bacula packages are taken from bacula repository,
bacula-cloud-storage.
  Plugin directory in bacula-sd is defined:
  Plugin Directory = "/opt/bacula/plugins"

And it contains:
-rw-r--r-- 1 root root 509216 Apr  7 10:01
bacula-sd-cloud-driver-9.6.3.so

Any help is more than welcomed!
  Thanks!
  Best regards
  
  -- 
Mario

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Compile error - s3 drivers for Bacula 9.4.2 (Ubuntu 20.04 LTS)

2020-06-23 Thread kern
Hello,There are compiler compatibility problems with 20.04 and Bacula 9.6.3.  
Please try Bacula 9.6.5, but be sure to use the libs3 as documented in the 
release notes.Best regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Mario Pranjic  Date: 
6/23/20  12:45  (GMT+01:00) To: bacula-users@lists.sourceforge.net Subject: 
[Bacula-users] Compile error - s3 drivers for Bacula 9.4.2 (Ubuntu 20.04 LTS) 
Hi,
I am trying to use Bacula which comes with Ubuntu 20.04 LTS (OS
  is fully patched):
ii  bacula-bscan
  9.4.2-2ubuntu5    amd64    network backup
  service - bscan tool
  ii  bacula-client   
  9.4.2-2ubuntu5    all  network backup
  service - client metapackage
  ii  bacula-common   
  9.4.2-2ubuntu5    amd64    network backup
  service - common support files
  ii  bacula-common-pgsql 
  9.4.2-2ubuntu5    amd64    network backup
  service - PostgreSQL common files
  ii  bacula-console  
  9.4.2-2ubuntu5    amd64    network backup
  service - text console
  ii  bacula-director 
  9.4.2-2ubuntu5    amd64    network backup
  service - Director daemon
  ii  bacula-director-pgsql   
  9.4.2-2ubuntu5    all  network backup
  service - PostgreSQL storage for Director
  ii  bacula-fd   
  9.4.2-2ubuntu5    amd64    network backup
  service - file daemon
  ii  bacula-sd   
  9.4.2-2ubuntu5    amd64    network backup
  service - storage daemon
  ii  bacula-server   
  9.4.2-2ubuntu5    all  network backup
  service - server metapackage
  

In regards to s3 drivers, I follow: 
https://www.bacula.org/bacula-release-9-4-2/
Compile fails with error:
root@bacula:~# tar xzf libs3-20181010.tar.gz 
  root@bacula:~# cd libs3-20181010/
root@bacula:~/libs3-20181010# make deb
  make: xml2-config: Command not found
  make: xml2-config: Command not found
  build/obj/acl.do: Compiling dynamic object
  build/obj/bucket.do: Compiling dynamic object
  build/obj/error_parser.do: Compiling dynamic object
  build/obj/general.do: Compiling dynamic object
  build/obj/object.do: Compiling dynamic object
  build/obj/request.do: Compiling dynamic object
  src/request.c: In function ‘request_curl_code_to_status’:
  src/request.c:1709:5: error: duplicate case value
   1709 | case CURLE_SSL_CACERT:
    | ^~~~
  src/request.c:1705:5: note: previously used here
   1705 | case CURLE_PEER_FAILED_VERIFICATION:
    | ^~~~
  make: *** [GNUmakefile:223: build/obj/request.do] Error 1



Any advice what to look is more than welcomed. Am I looking at
  some bug or am I missing something?
Thanks in advance!
Best regards,

-- 
  Mario

  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Cloud driver for OCI

2020-06-23 Thread kern
Hello,Yes, Oracle S3 is not compatible with AWS.  Bacula Systems has a separate 
driver for Oracle but at the current time I cannot say when we will get their 
code.  The first step is getting our current S3 driver to work correctly.What I 
would appreciate the most would be for a community to submit a patch to our 
current libs3 code base.Best regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Mario Pranjic  Date: 
6/23/20  07:32  (GMT+01:00) To: bacula-users@lists.sourceforge.net Subject: 
[Bacula-users] Cloud driver for OCI 
Hi,
I need to point bacula-sd to write directly to object storage
  bucket in Oracle Cloud (OCI) in order to use OCI archive storage
  bucket as target where Bacula will write backup data.

Although there is cloud driver available in Bacula, according to
  documentation, it seems that community edition supports only AWS
  for such configuration.


https://www.baculasystems.com/corporate-data-backup-software-solutions/professional-backup-software/enterprise-community-comparison/
Is that so? Can someone confirm this?
If true, is there any roadmap to show when/if OCI object storage
  will be supported in Bacula community edition?
Thanks in advance!
Best regards,

-- 
  Mario

  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula 9.6.3 - Could not open Cloud driver

2020-06-23 Thread kern
Hello,Unfortunately it seems that the S3 driver worked on 9.6.3.   You will 
need 9.6.4 or 9.6.5.Best regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Mario Pranjic  Date: 
6/22/20  13:23  (GMT+01:00) To: bacula-users@lists.sourceforge.net Subject: 
[Bacula-users] Bacula 9.6.3 - Could not open Cloud driver 
Hi,
I am trying to set-up Bacula with Cloud driver to write to S3
  bucket.
OS: Ubuntu 18.04 LTS (fully patched)
  
  Bacula packages (installed from Bacula repository on bacula.org):
  ii  bacula-client 
  9.6.3-1 amd64   
  Network backup, recovery and verification - file daemon
  ii  bacula-cloud-storage  
  9.6.3-1 amd64   
  Network backup, recovery and verification - Cloud Storage Plugin
  ii  bacula-common 
  9.6.3-1 amd64   
  Network backup, recovery and verification - common support files
  ii  bacula-console
  9.6.3-1 amd64   
  Network backup, recovery and verification - text console
  ii  bacula-postgresql 
  9.6.3-1 amd64   
  Network backup, recovery and verification - PostgreSQL storage for
  Director

Default installation works like a charm (as usual), but then I
  tried to set-up cloud storage in bacula-sd (following 
https://blog.bacula.org/whitepapers/CloudBackup.pdf).
I followed example on pages 8 and 9 and came up with (keys and
  realnames are stripped):
Device {
    Name = CloudStorage
    Device Type = Cloud
    Cloud = TEST1
    Archive Device = /data/bacula/backup-cloud
    Maximum Part Size = 10 MB
    Media Type = CloudType
    LabelMedia = yes
    Random Access = yes
    AutomaticMount = yes
    RemovableMedia = no
    AlwaysOpen = no
  }
  
  Cloud {
   Name = TEST1
    Driver = "S3"
    HostName = "s3.amazonaws.com"
    BucketName = "mybycket"
    AccessKey = "access_key_here"
    SecretKey = "secret_access_key_here"
    Protocol = HTTPS
    UriStyle = VirtualHost
    Truncate Cache = No
    Upload = EachPart
    Region = "us-east-1"
    MaximumUploadBandwidth = 5MB/s
  }
  

When I try to reference it in bacula-dir.conf:
JobDefs {
    Name = "DefaultJob"
    Type = Backup
    Level = Incremental
    Client = bacula-fd
    FileSet = "Full Set"
    Schedule = "WeeklyCycle"
    Storage = CloudType    #referencing cloud storage
    Messages = Standard
    Pool = File
    SpoolAttributes = yes
    Priority = 10
    Write Bootstrap = "/opt/bacula/working/%c.bsr"
  }

Bacula director fails to start with bacula-sd error:
  Jun 22 12:38:42 bacula bacula-sd: cloud_dev.c:690 Could not open
  Cloud driver type=1 for Device=CloudStorage.

Am I looking at some potential bug here or I got something wrong?

All bacula packages are taken from bacula repository,
  bacula-cloud-storage.
Plugin directory in bacula-sd is defined:
    Plugin Directory = "/opt/bacula/plugins"
  
  And it contains:
  -rw-r--r-- 1 root root 509216 Apr  7 10:01
  bacula-sd-cloud-driver-9.6.3.so
  
  Any help is more than welcomed!
Thanks!
Best regards

-- 
  Mario
  

  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] areas for improvement?

2020-06-13 Thread Kern Sibbald

  
  
Hello David,
Thanks for your confirmation of the problems.  I have a few
  suggestions for you:
1. Talk to Bacula Systems.  They give Universities very nice
  discounts, and they do have client initiated backup.  Bacula
  Systems has by default a subscription model, but certain customers
  prefer a one time purchase, which is possible.

2. If that is not an option, please let me know, and I will check
  with Bacula Systems for some pointers.  I have not personally used
  this feature but helped design it.  I believe it is already in
  Bacula community, but if it is not, then it surely will be
  available in the next version (probably around the end of the
  year).  

Best regards,
Kern

On 6/12/20 7:39 PM, David Brodbeck
  wrote:


  
  




  On Wed, Jun 10, 2020 at 8:41
AM Josh Fisher <jfis...@jaybus.com> wrote:
  
I still feel that Bacula's design is correct. Yes, 802.3az
changes the 
always-on nature of a connection, allowing either side to
temporarily 
power down its transmitter to save energy, but the standard
itself 
doesn't change the original goal of a persistent connection.
It is the 
switch firmware and/or NIC device drivers that claim to
support it, but 
do not. It makes sense for Bacula to be as robust as
possible, but this 
is not a Bacula design flaw. It is a work-around for buggy
hardware.
  
  
  
  I've also run into this when trying to back up over a
VPN. The backup time can easily exceed the VPN's maximum
session time.
  
  
  It's fair to argue that both NAT routers and VPNs are a
corruption of TCP/IP's design intent, but it doesn't seem
likely we'll be rid of them any time soon. Bacula doesn't
work very well with either. Besides the connection drop
issues, I haven't yet gotten client-initiated backups to
work from behind a NAT, and I haven't found anyone who's
confirmed they have it working, either.
  
  
  None of this, of course, is an issue when backing up
always-on servers with static IPs -- which is Bacula's
focus. The problems come in when it's used to back up
endpoints. Unfortunately I haven't yet found anything else
to use for that that lets me control my own data and isn't a
subscription model. (In academic departments, it's much
easier to find money for one-time expenses than it is to
find a consistent source of it.)
  
  

-- 

  David Brodbeck
System Administrator, Department of Mathematics
  University of California, Santa Barbara
  
  

  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.6.5

2020-06-12 Thread Kern Sibbald

Hello,

Unfortunately due to a Enterprise backport of code to the Community
version, three files were released to version 9.6.4 with Enterprise
copyrights.  In addition there were apparently several missing image
files in the documentation.  Thus we have withdrawn version 9.6.4
and replaced it with version 9.6.5.  If either of the just mentioned
problems worry you, please update to version 9.6.5.  Otherwise versions
9.6.4 and 9.6.5 are identical. I regret any inconvenience.

This is to announce that we have released Bacula version 9.6.5 to
Source Forge and to www.bacula.org.  This is a major security and
bug fix release. We suggest everyone to upgrade as soon as possible,
especially if you are using or want to use the S3 cloud driver. Note: I have
done some significant testing on building and running the S3 cloud drive
with Amazon cloud.  In doing so, I found and fixed a significant bug that
caused the cloud driver (and possibly other drivers) that caused memory
loss (orphaned buffers).

One significant improvement is for the AWS S3 cloud driver. First the
code base has been brought much closer to the Enterprise version (still
a long ways to go). Second is that the community code now uses the latest
version of libs3 as maintained by Bacula Systems.  The libs3 code is
available as a tar file for Bacula version 9.6.5 at:

   www.bacula.org/downloads/libs3-20200523.tar.gz

Version 9.6.5 and later must be compiled with the above libs3 version or 
later.

To build libs3:
- Remove any libs3 package loaded by your OS
- Download above link
- tar xvfz libs3-20200523.tar.gz
- cd libs3-20200523
- make  # should have no errors
- sudo make install

Then when you do your Bacula ./configure  it should automatically
detect and use the libs3. The output from the ./configure will show
whether or not libs3 was found during the configuration. E.g.
   S3 support:    yes
in the output from ./configure.

The binary files should be released within the next week or two.

For your convenience, I have included some more of the details from the 
ReleaseNotes file below.


Thanks for using Bacula.

Be happy,

Kern

=== Additional details from the Release Notes.

11Jun20
 - Add configure variables to baculabackupreport. Patch from bug #2538
 - Fix orphaned buffers in cloud by adding truncate argument to 
end_of_job()
 - Improve clarity of Cloud part mismatches and make it an INFO message 
rather

   than a WARNING since it corrects the catalog.
 - Small trivial change to check_parts
 - Backport more Enterprise cloud parts changes
 - Backport cloud upload code from Enterprise
 - Update s3_driver.c to new libs3 API calling sequence
 - Fix tray-monitor installation
 - Recompile ./configure
 - Add ./configure code to check for and enable/disable S3 support
 - win32: Fix org#2547 About possible NULL pointer dereference in 
get_memory_in

 - Ensure cloud driver loaded when listing cloud volumes
 - baculum: Request #2546 support for full restore when file records 
for backup

   job are pruned
 - baculum: Fix problem with authorization error after upgrade
 - baculum: Add UPGRADE file
 - baculum: Fix returning value in TStyleDiff - generated notice with PHP
   7.4
 - baculum: Remove execute bit for framework scripts
 - baculum: Fix displaying empty column button in table column 
visibility menu

  reported by Wanderlei Huttel
 - baculum: Update Polish translations
 - baculum: Update Portuguese translations
 - baculum: New user management. LDAP support. Role-based access control.
 - Fix new compiler warnings + always use bstrncpy not strncpy to 
ensure EOS at

   end of string
 - Return smartalloc buffers zeroed -- future performance improvement
 - Improve scanning data/time, fixes bug #2472
 - Make ABORT mention segfault to clarify non-bug #2528
 - Make reading a short block a warning rather than error
 - baculum: Fix validators in run job modal window
 - Remove unused -t option in dbcheck.c -- fixes bug #2453
 - Fix bug 2523 -- spurious extra linking
 - Fix bug #2534 possible double free in error case
 - Fix possible sscanf overflows
 - Fix overflow from malicious FD reported by Pasi Saarinen
 - baculum: Add option to show time in job log - idea proposed by Wanderlei
   Huttel
 - baculum: Add show log time parameter to job log endpoint
 - baculum: Add tip about using table row selection
 - Fix bug #2525 seg fault when doing estimate with accurate and MD5
 - baculum: Fix issues with SELinux support reported by Neil MacGregor
 - Correct some copyrights
 - Add Docker plugin rpm spec files

Bugs fixed/closed since last release:
2453 2472 2525 2528 2534 2538 2546 2547





___
Bacula-devel mailing list
bacula-de...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Cannot make bacula-sd-cloud driver to worker

2020-06-11 Thread Kern Sibbald

  
  
Hello,
As I have mentioned to others, but do so again in case you have
  not seen it, you will probably have much better luck with Bacula
  9.6.4.  However, you will need to remove the libs3-dev and libs3-2
  (or what ever they are) from your Ubuntu system, load the new
  libs3 source code (document in ReleaseNotes for Bacula 9.6.4) and
  build it from source.
I have successfully tested the building and running of the 9.6.3
  code on Ubuntu 18.04 and Ubuntu 20.04.

Best regards,
Kern

On 6/1/20 5:48 PM, esca wrote:


  Hello,


Im trying to setup bacula with S3 driver(using s3 from
  Scaleway ) on a brand new ubuntu 18.04.04 LTS vm but the
  sd-daemon always crash


I have installed the packages directly from bacula.org 


# apt-cache policy bacula-cloud-storage
  bacula-cloud-storage:
    Installed: 9.6.3-1
    Candidate: 9.6.3-1
    Version table:
   *** 9.6.3-1 500



my director conf is:


Storage {
    Name = SCW_S3_PAR
    Address = dev-vm05.mydomain                # N.B. Use a
  fully qualified name here
    SDPort = 9103
    Password = "some strings"
    Device = SCW_S3_PAR
    Media Type = CloudType
    Maximum Concurrent Jobs = 6
  }



and my bacula-sd.conf


evice {
    Name = SCW_S3_PAR
    Device Type = Cloud
    Cloud = SCW_S3_PAR
    Archive Device = /opt/bacula/backups
    Maximum Part Size = 10 GB
    Media Type = CloudType
    LabelMedia = yes
    Random Access = yes
    AutomaticMount = yes
    RemovableMedia = no
    AlwaysOpen = no
  }
  
  Cloud {
    Name = SCW_S3_PAR
    Driver = "S3"
    HostName = "s3.fr-par.scw.cloud"
    BucketName = "bacula"
    AccessKey = "myaccesskey"
    SecretKey = "mysecretkey"
    Protocol = HTTPS
    UriStyle = Path
    Truncate Cache = AfterUpload
    Upload = EachPart
    Region = "fr-par"
   MaximumUploadBandwidth = 500MB/s
  }



All services start just fine but when I start a job the
  Storage Daemon crash


dev-vm05:/opt/bacula# bin/bacula-sd -d100 -dt -v -f
  01-Jun-2020 17:26:11 bacula-sd: address_conf.c:274-0 Initaddr
  0.0.0.0:9103
  01-Jun-2020 17:26:11 dev-vm05: jcr.c:131-0 read_last_jobs seek
  to 192
  01-Jun-2020 17:26:11 dev-vm05: jcr.c:138-0 Read num_items=1
  01-Jun-2020 17:26:11 dev-vm05: plugins.c:97-0 load_plugins
  01-Jun-2020 17:26:11 dev-vm05: plugins.c:133-0 Rejected
  plugin: want=-sd.so name=bacula-sd-cloud-driver-9.6.3.so
  len=31
  01-Jun-2020 17:26:11 dev-vm05: plugins.c:133-0 Rejected
  plugin: want=-sd.so name=bpipe-fd.so len=11
  01-Jun-2020 17:26:11 dev-vm05: plugins.c:121-0 Failed to find
  any plugins in /opt/bacula/plugins
  01-Jun-2020 17:26:11 dev-vm05: stored.c:613-0 calling init_dev
  SCW_S3_PAR
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:152-0 Num drivers=15
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:165-0 loadable=1
  type=14 loaded=0 name=cloud handle=0
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:432-0 loadable=1
  type=14 loaded=0 name=cloud handle=0
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:437-0 Open SD driver
  at /opt/bacula/plugins/bacula-sd-cloud-driver-9.6.3.so
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:440-0 Driver=cloud
  handle=7f04440014b0
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:442-0 Lookup
  "BaculaSDdriver" in driver=cloud
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:444-0 Driver=cloud
  entry point=7f044a4e4f30
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:212-0 init_dev
  allocated: 7f0444001af8
  01-Jun-2020 17:26:11 dev-vm05: init_dev.c:393-0 init_dev:
  tape=0 dev_name=/opt/bacula/backups
  01-Jun-2020 17:26:11 dev-vm05: dev.c:1101-0
  DEVICE::register_metrics called. 0x7f0444001af8
  collector=0x560259842328
  01-Jun-2020 17:26:11 dev-vm05: stored.c:615-0 SD init done
  SCW_S3_PAR (0x7f0444001af8)
  01-Jun-2020 17:26:11 dev-vm05: acquire.c:671-0 Attach
  0x44003b48 to dev "SCW_S3_PAR" (/opt/bacula/backups)
  01-Jun-2020 17:26:11 dev-vm05: bnet_server.c:86-0 Addresses 0.0.0.0:9103
  01-Jun-2020 17:26:21 dev-vm05: bsoc

Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 S3 module on Debian10.

2020-06-11 Thread Kern Sibbald

Hello,

I recommend that you try with Bacula version 9.6.4, because I have 
recently built and tested the s3 driver.  It does require an updated 
libs3 (see release notes) and to build the libs3 I seem to recall that 
you need to install curl-dev and some other dev library.  It should tell 
you during the build.


Best regards,

Kern

On 5/30/20 7:31 PM, r0...@nxlplyx.com wrote:

Hi,

I have given up trying to get the S3 module to compile  on Ubuntu
20.04.   No suggestions have worked thus far.

I switched operating systems and am trying to compile the S3 module on
Debian 10.

The missing curl.h error disappeared I was getting in Ubuntu.

Unfortunately the following now occurs.  Does anyone have any suggestions?

---

$ make deb
build/obj/request.do: Compiling dynamic object
src/request.c: In function ‘request_curl_code_to_status’:
src/request.c:1709:5: error: duplicate case value
  case CURLE_SSL_CACERT:
  ^~~~
src/request.c:1705:5: note: previously used here
  case CURLE_PEER_FAILED_VERIFICATION:
  ^~~~
src/request.c: In function ‘sort_and_urlencode_query_string’:
src/request.c:865:9: error: ‘strncat’ specified bound 1 equals source
length [-Werror=stringop-overflow=]
  strncat(result, "&", 1);
  ^~~
cc1: all warnings being treated as errors
make: *** [GNUmakefile:223: build/obj/request.do] Error 1

---

Al.







___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Fwd: Compilation Errors of Bacula 9.6.3 on Debian10.

2020-06-11 Thread Kern Sibbald

  
  
Hello,
This problem is due to a newer compiler which falsely detects
  what it thinks are errors.
This should be fixed in version 9.6.4 ...  I did not actually
  test it on Debian 10 but I did test it on Ubuntu 20.04, which I
  think uses the same newer version of C++.

Best regards,
Kern

On 6/4/20 9:22 PM, r0...@nxlplyx.com
  wrote:


  
  
  
  

  I tried to compile the main program 9.6.3. That is not
working so well.  I am getting a number of errors like the following. 
Can someone tell me if I am missing something?

Are there patches involved?

Can anyone help?

===

htable.c: In member function 'void htable::grow_table()':
htable.c:230:36: warning: 'void* memcpy(void*, const void*, size_t)'
writing to an object of non-trivially copyable type 'class htable'; use
copy-assignment or copy-initialization instead [-Wclass-memaccess]
    memcpy(big, this, sizeof(htable));  /* start with original class data */
    ^
In file included from ../lib/tree.h:26,
 from ../lib/lib.h:59,
 from ../bacula.h:169,
 from htable.c:42:
../lib/htable.h:77:7: note: 'class htable' declared here
 class htable : public SMARTALLOC {
   ^~

==

Al


  
Please do a 

 make VERBOSE=1 clean

and then retry with 

 make VERBOSE=1 deb

Grüße,
Sven.


  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Fwd: Compilation Errors of Bacula 9.6.3 on Debian10.

2020-06-11 Thread Kern Sibbald

  
  
Hello,
This is very likely corrected with Bacula version 9.6.4
Best regards,
Kern

On 6/7/20 4:07 PM, Radosław
  Korzeniewski wrote:


  
  
Hello,


  czw., 4 cze 2020 o 21:23 r0...@nxlplyx.com
<r0...@nxlplyx.com>
napisał(a):
  
  

  I
  tried to compile the main program 9.6.3. That is not
  
  

  working so well.  I am getting a number of errors like the following. 

  

  
  The error like this below is a warning only and compile
does not stop. The correction is not trivial as it requires
some deep code refactoring.
  So, if your compilation stops, please show the real
error.
  
  
  best regards

-- 
Radosław Korzeniewski
  rados...@korzeniewski.net
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.6.4

2020-06-11 Thread Kern Sibbald

Hello,

This is to announce that we have released Bacula version 9.6.4 to
Source Forge and to www.bacula.org.  This is a major security and
bug fix release. We suggest everyone to upgrade as soon as possible,
especially if you are using or want to use the S3 cloud driver. Note: I have
done some significant testing on building and running the S3 cloud drive
with Amazon cloud.  In doing so, I found and fixed a significant bug that
caused the cloud driver (and possibly other drivers) that caused memory
loss (orphaned buffers).

One significant improvement is for the AWS S3 cloud driver. First the
code base has been brought much closer to the Enterprise version (still
a long ways to go). Second is that the community code now uses the latest
version of libs3 as maintained by Bacula Systems.  The libs3 code is
available as a tar file for Bacula version 9.6.4 at:

   www.bacula.org/downloads/libs3-20200523.tar.gz

Note: Version 9.6.4 must be compiled with the above libs3 version or later.
To build libs3:
- Remove any libs3 package loaded by your OS
- Download above link
- tar xvfz libs3-20200523.tar.gz
- cd libs3-20200523
- make  # should have no errors
- sudo make install

Then when you do your Bacula ./configure  it should automatically
detect and use the libs3. The output from the ./configure will show
whether or not libs3 was found during the configuration. E.g.
   S3 support:    yes
in the output from ./configure.

The binary files should be released within the next week or two.

For your convenience, I have included some more of the details from the 
ReleaseNotes file below.


Thanks for using Bacula.

Be happy,

Kern

=== Additional details from the Release Notes.

08Jun20
 - Add configure variables to baculabackupreport. Patch from bug #2538
 - Fix orphaned buffers in cloud by adding truncate argument to 
end_of_job()
 - Improve clarity of Cloud part mismatches and make it an INFO message 
rather

   than a WARNING since it corrects the catalog.
 - Small trivial change to check_parts
 - Backport more Enterprise cloud parts changes
 - Backport cloud upload code from Enterprise
 - Update s3_driver.c to new libs3 API calling sequence
 - Fix tray-monitor installation
 - Recompile ./configure
 - Add ./configure code to check for and enable/disable S3 support
 - win32: Fix org#2547 About possible NULL pointer dereference in 
get_memory_in

 - Ensure cloud driver loaded when listing cloud volumes
 - baculum: Request #2546 support for full restore when file records 
for backup

   job are pruned
 - baculum: Fix problem with authorization error after upgrade
 - baculum: Add UPGRADE file
 - baculum: Fix returning value in TStyleDiff - generated notice with PHP
   7.4
 - baculum: Remove execute bit for framework scripts
 - baculum: Fix displaying empty column button in table column 
visibility menu

  reported by Wanderlei Huttel
 - baculum: Update Polish translations
 - baculum: Update Portuguese translations
 - baculum: New user management. LDAP support. Role-based access control.
 - Fix new compiler warnings + always use bstrncpy not strncpy to 
ensure EOS at

   end of string
 - Return smartalloc buffers zeroed -- future performance improvement
 - Improve scanning data/time, fixes bug #2472
 - Make ABORT mention segfault to clarify non-bug #2528
 - Make reading a short block a warning rather than error
 - baculum: Fix validators in run job modal window
 - Remove unused -t option in dbcheck.c -- fixes bug #2453
 - Fix bug 2523 -- spurious extra linking
 - Fix bug #2534 possible double free in error case
 - Fix possible sscanf overflows
 - Fix overflow from malicious FD reported by Pasi Saarinen
 - baculum: Add option to show time in job log - idea proposed by 
Wanderlei

   Huttel
 - baculum: Add show log time parameter to job log endpoint
 - baculum: Add tip about using table row selection
 - Fix bug #2525 seg fault when doing estimate with accurate and MD5
 - baculum: Fix issues with SELinux support reported by Neil MacGregor
 - Correct some copyrights
 - Add Docker plugin rpm spec files

Bugs fixed/closed since last release:
2453 2472 2525 2528 2534 2538 2546 2547





___
Bacula-devel mailing list
bacula-de...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] areas for improvement?

2020-06-11 Thread Kern Sibbald

Hello Josh,

Yes, you are correct, it was simply a design decision and reasonable at 
the time.  However, with the knowledge that I have today of how networks 
evolved that I did not have then, were I to do it over, I would add a 
Bacula native persistent connection (using reconnection when the line 
drops).  In fact, the original design contains a sequential block number 
that is not used, but was to be the basis for subsequent code that would 
reconnect.


I appreciate your comments. Thanks.

Kern

On 6/10/20 5:41 PM, Josh Fisher wrote:


On 6/10/2020 8:04 AM, Kern Sibbald wrote:

Hello,

...

Now on the fact that line drops cancel jobs: First Bacula was 
designed with the concept that it would have a stable communications 
line as is supposed to be provided by TCP/IP, which Bacula uses.  
This was a correct design based on networks at the time, but on 
retrospect, I should have included comm line restarts in the original 
design.  In my opinion, the real problem is that modern switches for 
all sorts of good reasons do not really support the original design 
goals of TCP/IP.



I still feel that Bacula's design is correct. Yes, 802.3az changes the 
always-on nature of a connection, allowing either side to temporarily 
power down its transmitter to save energy, but the standard itself 
doesn't change the original goal of a persistent connection. It is the 
switch firmware and/or NIC device drivers that claim to support it, 
but do not. It makes sense for Bacula to be as robust as possible, but 
this is not a Bacula design flaw. It is a work-around for buggy hardware.




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems trying out trial

2020-06-11 Thread Kern Sibbald

  
  
Hello,
You are using the Enterprise version of Bacula from the Bacula
  Systems website because the community version does not have a
  trial version (only the real thing) and also does not have BWeb
  (but Baculum instead).  I suggest that you contact Bacula Systems
  (see their website). They will be happy to help you.
Best regards,
Kern

On 6/10/20 1:52 PM, Liam Kruize wrote:


  
  Hello,
  
  
  The past couple of weeks I have been trying out the Bacula
trial. I ran into some problems however whilst trying it out.
  
  
  Right now I was trying out to make a back-up of a Windows
client using BWeb. However I can't select the job I created.
When I go to the job selection menu, it is not possible for me
to choose a job to run. It also does not show any of the jobs
that were already present in the trial. I am also unable to log
out from the BWeb interface, I can't go back to the page with
the reset button on it.
  
  
  I hope anyone could help me with this! Thanks in advance.
  
  
  
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] areas for improvement?

2020-06-10 Thread Kern Sibbald

Hello,

For some reason I never received the original email (very odd). I think 
that Gary has done a very good job at responding.  I'll give you my take 
on this, but please excuse me if I duplicate what has already been said.


First on the SQL database, which is as has been pointed out not 
stateless: I have never seen database communications drops reported as a 
problem.  For me it has never been a problem because I run the Director 
and the database backend (Postgres for me) on the same machine, so it is 
as far as I understand not using the communications lines.


Now on the fact that line drops cancel jobs: First Bacula was designed 
with the concept that it would have a stable communications line as is 
supposed to be provided by TCP/IP, which Bacula uses.  This was a 
correct design based on networks at the time, but on retrospect, I 
should have included comm line restarts in the original design.  In my 
opinion, the real problem is that modern switches for all sorts of good 
reasons do not really support the original design goals of TCP/IP.


That said, I have been aware of the problem that Alan brings up, and 
Bacula does have the ability to restart jobs at the point where the job 
failed under certain conditions such as a comm line drop.  This feature 
seems to be rarely used, but is quite effective in the case where one 
has lots of comm line failures.


For some time, I have had in mind a project to make Bacula restart a 
comm line connection after a drop, however, as Gary points out, this is 
far from being a trivial project.  Bacula Systems currently has a 
project well along the way to implement this feature, and from what I 
have heard, it is now in the testing phase and will probably be in the 
next Bacula Enterprise release. When it will appear in the community 
version is not clear.


Concerning priorities of projects: to the best of my knowledge no one 
has submitted a bug report or a request for this feature other than Alan 
who submitted a request for this feature some time ago in the Enterprise 
version. For Bacula Systems, a lot of time and consideration is devoted 
twice a year to examine new feature requests and decide which to 
implement.  Every six months key managers and an outside consultant are 
requested to submit the most important feature requests.  They are then 
sorted by a number of conditions such as: difficulty of the project, 
number of users impacted, overall need for Bacula, ...  All that then 
works down to a Roadmap for the next release (in roughly 6 months) and 
the following release (in roughly 1 year).  The six month roadmap will 
then be approved by the company managers and reviewed at the bi-annual 
company meeting.  Generally the six month roadmaps do not change much 
(sometimes a feature is dropped or added).  The 1 year roadmap can 
change as you might imagine.


Bottom line: this is a very complex "feature" request, but it is now 
well along in development, and so will be available at some time in the 
not so distant future.


Gary: thanks for your insights :-)

Alan: I am not responding to all your comments, but will say that I 
believe that you have misunderstood certain things about Bacula Systems, 
how they decided what is important, etc.  One of the nice things about 
open source, is that if you are unhappy with what it does, you have all 
the source code, and you can either implement what you want, or hire 
someone to do it.  Having an Enterprises agreement does not necessarily 
meant that any feature request will be immediately implemented -- have 
you ever tried to get Microsoft to fix a bug or implement a new feature?


Best regards,

Kern

On 5/27/20 4:13 PM, Gary R. Schmidt wrote:

On 27/05/2020 23:17, Alan Brown wrote:


I've been running Bacula for ~15 years (community/enterprise) and have
identified a few areas which are in desperate of improvement:

For an "enterprise" grade backup system, it's amazingly fragile in a few
areas (particularly in actual Enterprise networks!)


Bacula DOES NOT LIKE and does not handle network interruptions _at all_
if backups are in progress. This _will_ cause backups to abort - and
these aborted backups are _not_ resumable

Similarly, if there's any kind of disruption between the director and
database, the only fix is to restart the director


What that means is that Bacula _cannot_ be used with a High Availability
database because network interruptions (when switching servers) are part
of the HA paradigm.

It also means that operators have to be _extremely_ careful about
allowing automated or other system upgrades


In days of multi-TB backup sets, this is turning into a showstopping
problem.

As we are an Enteprise customer this has been raised with Baculasystems
but been given _very_ low priority.   I'd like to hear opinions from the
wider community on this


Opinion: I know bugs aren't sexy to work on but these need fixing, not
being brushed off. This is the difference between LA

Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04

2020-05-17 Thread kern
Hello Josh,Yes, it can be implemented the way you describe, and in fact Bacula 
has all the necessary code but some needs to write the driver (a bunch of 
scripts).I am sure it will happen in the not too distant future.Best regards, 
KernSent from my Samsung Galaxy smartphone.
 Original message From: Josh Fisher  Date: 
5/11/20  16:52  (GMT+01:00) To: Kern Sibbald , Sven Hartge 
, bacula-users@lists.sourceforge.net Subject: Re: 
[Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04 


On 5/11/2020 7:38 AM, Kern Sibbald
  wrote:


  
  Hello Sven,
  I share your concerns, but here are a few mitigating factors:
  1. I am not aware of any other C/C++ S3 library that will do
the same job and is well maintained.  I have to admit I have not
looked recently, so any suggestions would be welcome.



I'm not aware of other C libraries either, but why not use AWS
  CLI high-level commands in much the same way that tape autochanger
  script commands are implemented? AWS CLI is actively maintained by
  Amazon and so shouldn't  have the maintenance issues. These are
  simple mv, cp, ls, etc. commands somewhat equivalent to their Unix
  command namesakes. They shouldn't change even when/if Amazon
  changes the underlying S3 protocols.




  2. Bacula Systems actively uses libs3 with its customers, and
any corrections that they make will also be in the Bacula
community libs3 git repo (under the same GNU Lesser V3 license)
  3. Amazon (or other s3 supplier's) tools can be used to restore
Bacula S3 Volumes to disk, and once that is done, Bacula can
read those volumes regardless of what S3 library it is linked
with.  This is because when the Volumes are on disk, Bacula
reads them as standard OS files.  libs3 is only used to move
files from a local disk to and from the S3 cloud.
  
  Best regards,
  Kern
  
  On 5/10/20 3:24 PM, Sven Hartge
wrote:
  
  
On 10.05.20 15:01, Kern Sibbald wrote:



  I agree with Sven, libs3 is a big disaster.  It works well but the
author abandoned it, and many things have changed since then.  For the
moment, we have a version that works with AWS (don't expect it to work
with a number of other S3 implementations, which are not compatible with
AWS).


Adding to that: Other than the horrendous possible security flaws
present in libs3 (try to compile with a recent GCC and see for yourself)
the nature of anything cloud-bases is inherently volatile.

The AWS-API may change at any given moment (and it has in the past),
making libs3 incompatible without updates.

And without an upstream author implementing those changes, your backups
are more or less gone.

My very pessimistic view on the situation is: Don't use any backup
solution using libs3 if you value your data.

But: YMMV.

Grüße,
Sven.






___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04

2020-05-11 Thread Kern Sibbald

  
  
Hello Sven,
I share your concerns, but here are a few mitigating factors:
1. I am not aware of any other C/C++ S3 library that will do the
  same job and is well maintained.  I have to admit I have not
  looked recently, so any suggestions would be welcome.
2. Bacula Systems actively uses libs3 with its customers, and any
  corrections that they make will also be in the Bacula community
  libs3 git repo (under the same GNU Lesser V3 license)
3. Amazon (or other s3 supplier's) tools can be used to restore
  Bacula S3 Volumes to disk, and once that is done, Bacula can read
  those volumes regardless of what S3 library it is linked with. 
  This is because when the Volumes are on disk, Bacula reads them as
  standard OS files.  libs3 is only used to move files from a local
  disk to and from the S3 cloud.

Best regards,
Kern

On 5/10/20 3:24 PM, Sven Hartge wrote:


  On 10.05.20 15:01, Kern Sibbald wrote:


  
I agree with Sven, libs3 is a big disaster.  It works well but the
author abandoned it, and many things have changed since then.  For the
moment, we have a version that works with AWS (don't expect it to work
with a number of other S3 implementations, which are not compatible with
AWS).

  
  
Adding to that: Other than the horrendous possible security flaws
present in libs3 (try to compile with a recent GCC and see for yourself)
the nature of anything cloud-bases is inherently volatile.

The AWS-API may change at any given moment (and it has in the past),
making libs3 incompatible without updates.

And without an upstream author implementing those changes, your backups
are more or less gone.

My very pessimistic view on the situation is: Don't use any backup
solution using libs3 if you value your data.

But: YMMV.

Grüße,
Sven.


  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04

2020-05-10 Thread Kern Sibbald

  
  
Hello,
Please permit me to make a small correction to what Sven said. 
  The libs3 that is needed is indeed updated from the version
  released on Debian and Ubuntu, but it supplied by the Bacula
  Community (i.e. Kern) rather than Bacula Systems.  A working
  version can be found on the bacula web site at:
https://www.bacula.org/downloads/libs3-20181010.tar.gz
I have also pushed what is probably a more recent version of this
  library to the public git repo named libs3.git on www.bacula.org. 
  You can access it similarly to the way you access the current
  bacula.git repository.  Note: I have not personally tested this
  version of libs3.
I agree with Sven, libs3 is a big disaster.  It works well but
  the author abandoned it, and many things have changed since then. 
  For the moment, we have a version that works with AWS (don't
  expect it to work with a number of other S3 implementations, which
  are not compatible with AWS).
Best regards,
Kern

On 5/10/20 11:04 AM, Sven Hartge wrote:


  On 09.05.20 22:48, r0...@nxlplyx.com wrote:


  
It is the S3 modules that I need.  This may help explain why I have not
been able to get the S3 module of Bacula 9.6.3 to work in for Ubuntu 18.04.

  
  
To get the S3 driver working, you need the special libs3 provided by
Bacula systems. The libs3 included in Ubuntu (and Debian) will not work.

(This is also why the Debian and Ubuntu packages don't include the S3
driver in the first place, because of the whole disaster that is libs3
to begin with.)

Grüße,
Sven.


  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] backup problem to new storage server

2020-04-19 Thread Kern Sibbald

  
  
Hello,
Among the many possibilities for your current problems, I suspect
  that your Director and Storage Daemon may not be the same version.
Best regards,
Kern

On 4/17/20 8:26 AM, Thing wrote:


  
  After rebooting my systems I see,


" Last Volume Bytes:      0 (0 B)
  Non-fatal FD errors:    1
  SD Errors:              0
  FD termination status:
  SD termination status:
  Termination:            *** Backup Error ***

17-Apr 18:24 bacula-dir JobId 0: Fatal error: bsock.c:569 Packet
size=1073741835 too big from "Storage daemon:192.168.1.21:9103.
T                            erminating connection.
*
?
  
  
  
On Fri, 17 Apr 2020 at 18:01,
  Thing <thing.th...@gmail.com> wrote:


    Hi,


I have just replaced my storage server with a 4tb
  external disk (Debian9) with a new server (debian10.3) and
  plugged the 4tb disk into it and I cannot get backups to
  work.


In messages on the director I see,


"Incremental    Backup    10  17-Apr-20 19:05  
   Backupdell6430-003-fd Remote-0022
Incremental    Backup    10  17-Apr-20 19:05  
 Backupemail-001-fd Remote-0022
Incremental    Backup    10  17-Apr-20 19:05  
 Backuprmtp-001-fd  Remote-0022
Full           Backup    11  17-Apr-20 23:10  
 BackupCatalog      Vol-0002


Running Jobs:
Console connected at 17-Apr-20 17:32
 JobId  Type Level     Files     Bytes  Name            
 Status
==
 12752  Back Full          0         0  

  
  
  Terminated Jobs:
   JobId  Level    Files      Bytes   Status   Finished    
     Name

   8><---
   12747  Full          1    638.9 M  OK       16-Apr-20
  23:10 BackupCatalog
   12748  Full          0         0   Error    17-Apr-20
  17:17 BackupCatalog
   12749  Full          0         0   Error    17-Apr-20
  17:20 BackupCatalog
   12750  Full          0         0   Error    17-Apr-20
  17:26 BackupCatalog
   12751  Full          0         0   Error    17-Apr-20
  17:36 BackupCatalog
  
  
  You have messages.
  *messages
  17-Apr 17:37 bacula-dir JobId 12752: Start Backup JobId
  12752, Job=BackupCatalog.2020-04-17_17.37.21_04
  17-Apr 17:37 bacula-dir JobId 12752: Error: getmsg.c:185
  Malformed message: CatReq JobId=12752 FindMedia=1
  pool_name=File media_type=File vol_type=1
  
  
  I copied the bacula.sd and
bacula.fd files off the old server to the new server and
changed the IP and server name and restarted.  I
ahve edited the director and restarted it also.
  
  
  the director is,
  root@bacula:/etc/bacula# bconsole
Connecting to Director 192.168.1.104:9101
1000 OK: 102 bacula-dir Version: 7.4.4 (20 September
2016)
Enter a period to cancel a command.
  
  
  
  the sd version is,
  
  
  *status
Status available for:
     1: Director
     2: Storage
     3: Client
     4: Scheduled
     5: All
Select daemon type for status (1-5): 2
Automatically selected Storage: File
Connecting to Storage daemon File at 192.168.1.21:9103

dell6430-003-sd Version: 9.4.2 (04 February 2019)
x86_64-pc-linux-gnu debian buster/sid
Daemon started 17-Apr-20 17:36. Jobs: run=0, running=0.
You have messages.
  
  
  
  Not sure what's wrong here, incompatibility?
  
  
  
  
  
  
  thanks,
  
  
  Steven
  
  

  

  
  
  
  
  
  ___
Bacula-user

Re: [Bacula-users] Bacula 9.6.x: "volume use duration" not working as expected

2020-03-22 Thread kern
Hello,That volume is set to  a 1 year volume use duration not one day.  
Possibly you updated the pool but forgot to update existing volumes with the 
new resource.Best regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Uwe Schuerkamp 
 Date: 3/20/20  09:55  (GMT+01:00) To: Bacula Users 
Mailing List  Subject: [Bacula-users] 
Bacula 9.6.x: "volume use duration" not working as
  expected Hi folks,I have a set of tapes that I'd like to use in a daily 
rotation to backup online disk volumes to tape.I defined a pool for this like 
so:Pool {  Name = offline_weekly  Pool Type = Backup  Recycle = yes  AutoPrune 
= yes  Volume Retention = 1 year  Recycle Oldest Volume = yes  Purge Oldest 
Volume = yes  Maximum Volumes = 7  Volume Use Duration = 1 day}Also, I've 
labelled 7 LTO6 tapes and assigned them to the pool.The backup works fine, but 
all the jobs end up on the first tape(probably until it fills up). I would 
expect that the "volume useduration" would cause bacula (9.6.2 compiled from 
source on UbuntuServer LTS 18.04) to load the next tape in the pool after 24 
hours (sowhen the next offline backup job runs), then the next one and so 
onuntil it reaches tape 1 again, then writing the next offline job onthat 
volume and so on, until all tapes are full and then it shouldstart to recycle 
the oldest volume.Here's the pool how bacula sees it at the moment (I've also 
tried"update pool from resource" without any 
effect):+++-+-+-+--+-+--+-+|
 PoolId | Name   | NumVols | MaxVols | MaxVolBytes | VolRetention | 
Enabled | PoolType | LabelFormat 
|+++-+-+-+--+-+--+-+|
 11 | offline_weekly |   7 |   7 |   0 |   31,536,000 | 
  1 | Backup   | *   
|+++-+-+-+--+-+--+-+And
 here's what "update pool from resource" 
shows:+-+---+---+++---+---+|
 PoolId | Name   | NumVols | MaxVols | UseOnce | UseCatalog | 
AcceptAnyVolume | CacheRetention | VolRetention | VolUseDuration | MaxVolJobs | 
MaxVolFiles | MaxVolBytes | AutoPrune | Recycle | ActionOnPurge | PoolType | 
LabelType | LabelFormat | Enabled | ScratchPoolId | RecyclePoolId | NextPoolId 
| MigrationHighBytes | MigrationLowBytes | MigrationTime 
|+++-+-+-++-++--+++-+-+---+-+---+--+---+-+-+---+---+++---+---+|
 11 | offline_weekly |   7 |   7 |   0 |  1 |   
    0 |  0 |   31,536,000 | 86,400 |  0 |   
0 |   0 | 1 |   1 | 0 | Backup   | 0 | 
*   |   1 | 0 | 0 |  0 |
  0 | 0 | 0 
|+++-+-+-++-++--+++-+-+---+-+---+--+---+-+-+---+---+++---+---+(sorry
 for the long lines)and here's an example output of a "list jobs on volume" 
query for the one tape that is currently in active use by bacula in the pool 
above:+---+---+-+--+---+---+++|
 JobId | Name  | StartTime   | Type | Level | Files | Bytes 
 | Status 
|+---+---+-+--+---+---+++|
   417 | offline_daily | 2020-03-17 14:08:21 | B    | F | 4,167 | 
33,078,435,599 | T  ||   422 | offline_daily | 2020-03-18 06:21:34 | B    | 
F | 4,167 | 33,155,008,843 | T  ||   430 | offline_daily | 2020-03-19 
06:21:33 | B    | F | 4,324 | 38,389,879,305 | T  ||   437 | 
offline_daily | 2020-03-20 04:01:32 | B    | F | 4,328 | 41,592,356,545 | T 
 
|+---+---+-+--+---+---+++Am
 I misunderstanding something about "volume use duration" here?Thanks in 
advance for your help & all the best,Uwe-- Uwe Schürkamp | email: 
___Bacula-users
 mailing 
listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users___

Re: [Bacula-users] New tape drive

2020-03-14 Thread kern
Hello,My experience is that the larger the block size one uses the more prone 
the drive is to errors especially when the drive is dirty.  A number of years 
ago I had the chance to speak to a Quantum tape expert and he recommended a max 
of 500K -- at that time we were talking about an LTO-5.  Since then tape 
technology has improved significantly, but just the same at some point when 
increasing the block size more errors will occur.  I recommend testing various 
block sizes to see what the performance gains really are as I suspect from my 
own testing that performance increases quickly become minimal.  Unless there 
are some spectacular performance increases, I would not recommend block sizes 
of more than 1-2 megabytes on LTO-6 and above and 1 megabyte max for LTO-5.Best 
regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Steven Hammond  
Date: 3/13/20  15:55  (GMT+01:00) To: bacula-users@lists.sourceforge.net 
Subject: Re: [Bacula-users] New tape drive Does anyone have recommendations for 
MAXIMUM BLOCK SIZE for a LTO-7 tape drive (HP).  We are currently using 2M on 
the LTO-5 with success (and it seemed to improve the speed).  I didn't know if 
there was a better setting for the LTO-7 (e.g., 4M) that should be used.  Any 
suggestions?Steven HammondTechnical Chemical CompanyCleburne, TXOn 3/10/2020 
12:38 PM, Gary R. Schmidt wrote:> On 11/03/2020 02:26, Steven Hammond wrote:>> 
We are currently using a LTO-5 drive.  We are upgrading to a LTO-7 >> drive.  I 
noticed the directive in our pools STORAGE=LTO-5. I'd like >> the existing 
pools to use LTO-7 (since it can still read LTO-5 >> tapes).  Can I just change 
the directive in the pool to use our new >> LTO-7 tape drive OR will this mess 
up the existing LTO-5 tapes in the >> pool.  I was wanting to use the existing 
pools, but I could create >> new ones if necessary.  Thanks.>>> It's just a 
tag, it has no meaning, you could use "Bilbo-Baggins" and > as long as you are 
consistent it doesn't matter what the contents of > the "Storage" directive 
is.>> Cheers,>     Gary    B-)>> P.S.  It's considered impolite to 
change the Subject of an existing > thread on a mailing list to start a new 
discussion, it buggers up MUAs > that use the "Reference:" header to group 
things.>>> ___> Bacula-users 
mailing list> Bacula-users@lists.sourceforge.net> 
https://lists.sourceforge.net/lists/listinfo/bacula-users___Bacula-users
 mailing 
listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tape issue: less size than before for LTO-4 tapes

2020-03-14 Thread kern
Hello,From the error message printed by Bacula (in German) I would strongly 
suspect that something caused a reset on the drive or the auto changer.  This 
can happen if two processes are using the changer at the same time and someone 
is issuing a smartctl or another such program.I would recommend that you enable 
Bacula's TapeAlert code, which is not on by default.  It will keep a record of 
any tape alerts, which could help you track down the problem.I would also 
recommend that you do two cleanings of each drive.  One cleaning is not always 
sufficient (or more correctly said, one cleaning did not resolve a tape proble 
I had on an LTO-4 drve).Best regards,KernSent from my Samsung Galaxy smartphone.
 Original message From: Pierre Bernhardt 
 Date: 3/14/20  04:54  (GMT+01:00) To: 
bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Tape issue: less 
size than before for LTO-4 tapes Am 13.03.20 um 08:28 schrieb Pierre 
Bernhardt:> Am 10.03.20 um 20:27 schrieb Pierre Bernhardt:>> Write failed at 
block 8758373. stat=-1 ERR=Auf dem Gerät ist kein Speicherplatz mehr 
verfügbar>> btape: btape.c:411-0 Volume bytes=565.0 GB. Write rate = 25.08 
MB/s>> btape: btape.c:612-0 Wrote 1 EOF to "HP Ultrium 4-1" 
(/dev/tape/by-id/scsi-HU1914570A-nst)> > Thats very interesting. On the other 
drive which can store ~ 750-800 GiByte (drive 1)> pcp shows me for the btape 
test ~ 80 MByte/s:> > # Device r/s w/s  kb_r/s  kb_w/s   r_pct   
w_pct   o_pct    Rs/s   o_cnt> st0 0.00 1259.59   0   81258    
0.00   98.23   98.23    0.00    0.00> > So it looks really there is really an 
issue on the drive 0 or the connection path.> I will wait for the end of the 
both tests> (btape on drive 1 and backup test on drive 0)It's really dependend 
on the tape drive. With same tape on the other drive thebtape test will write 
many more data on this tape:Write failed at block 12611722. stat=-1 ERR=Auf dem 
Gerät ist kein Speicherplatz mehr verfügbarbtape: btape.c:411-0 Volume 
bytes=813.6 GB. Write rate = 79.21 MB/sbtape: btape.c:612-0 Wrote 1 EOF to "HP 
Ultrium 4-2" (/dev/tape/by-id/scsi-HU19145705-nst)The backup test shows me not 
the best:# Device r/s w/s  kb_r/s  kb_w/s   r_pct   w_pct   o_pct   
 Rs/s   o_cntst1 0.00  259.29   0   16727    0.00   79.75   
79.75    0.00    0.00st1 0.00  259.19   0   16721    0.00   
79.75   79.75    0.00    0.00…16 MByte/s is to less.And the backuptest shows me 
on drive 0 only a stored size of 475 MByte.I would like to check FC connection 
for lip resets, error stats and so on.I checked the "files" in 
/sys/class/fc_host/host?/statistics but all which couldhave a problem value 
shows me only 0x0 values.So my first guess is there is no really problem on the 
hba-drive-connections.I will now try to clean the drive with a fresh cleaning 
tape and use btapeagain for a test. I think I will not get big different 
result, but it'sa 
try.Cheers,Pierre___Bacula-users 
mailing 
listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.6.3

2020-03-10 Thread Kern Sibbald

Hello,

This is to announce that we have released Bacula version 9.6.3 to Source 
Forge and to www.bacula.org.  This version primarily has fixed a number 
of incorrect copyrights.  If you are using Bacula 9.6.0, 9.6.1, or 
9.6.2, we recommend that you update, but it i not required.  The 
following fixes have been made since version 9.6.2:


 - Eliminate false error when droping postgres table MAC
 - Apply Carsten's character set fix for the docs. Many thanks!
 - Fix logic error in clearing bit on Windows
 - baculum: Update Portuguese translations
 - baculum: Update Polish translations
 - baculum: Add patch to PRADO framework 4.0.1 for supporting 
PostgreSQL 12

  catalog database
 - baculum: Add support for PostgreSQL 12 catalog database
 - Enhance failed bpipe to changer error message
 - Clean up some incorrect copyrights
 - Correct spelling errors in messages
 - Add to plugins links
 - baculum: Add bulk actions for job history and volume tables
 - baculum: Update DataTables and its plugins
 - docker: Update copyright headers.
 - Update BSD copyright on *.conf.in files
 - docker: Remove unneeded tar binary.
 - Fix workaround for Sun C++ recommended by Phil Stracchino
 - baculum: Update Polish translations
 - baculum: Update Portuguese translations

For more details see the ReleaseNotes or ChangeLog files.  The binary 
files should be released within the next week.


Thanks for using Bacula.

Be happy,

Kerm



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula Release 9.6.2 /packages for yum & apt?

2020-03-07 Thread kern
Yes they are coming.  It usually takes a week or two to build them.Sent 
from my Samsung Galaxy smartphone.
 Original message From: Jari Fredriksson  Date: 
3/6/20  02:11  (GMT+01:00) To: bacula-users@lists.sourceforge.net Subject: 
[Bacula-users] Bacula Release 9.6.2 /packages for yum & apt? Hello!Will there 
be 9.6.2 on https://www.bacula.org/packages/x/ for rpm/deb to be used 
in package managers? I see only 9.0.0 - 9.4.4.Thanks, 
jarif___Bacula-users mailing 
listBacula-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/bacula-users___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula Crashed / Segementation Violation

2020-03-03 Thread Kern Sibbald

Hello,

There is no reason for Bacula to crash at any point.  The watchdog may 
cancel jobs after a period of time, but that is not a Bacula crash.


Best regards,

Kern

On 3/2/20 2:02 PM, Yale Kozinski wrote:

Does that include a job that requests regular intervention? If so, I guess I 
need to recompile Bacula before attempting to backup 50 terabytes of data onto 
LTO6 media. I’m fairly new to Bacula and I was planning to just change tapes 
every day while in the office, but if it’s just going to crash for no good 
reason after a week then it hardly seems worth bothering with. There should 
really be something in the documentation about this, since it’s kind of an ugly 
surprise to come across just when you think you’ve got everything working.

Any way to disable this misfeature entirely? I am quite certain that I’ll know 
when my backups aren’t working anymore (because I’ll stop getting emails 
telling me to change tapes). At a minimum, this really seems like the sort of 
“gotcha” that should be mentioned in the FAQ…

Yale


On Mar 2, 2020, at 03:20, Josh Fisher  wrote:

Bacula has a built-in watchdog that kills a job that runs for more than 6 
days. That period can be extended at compile time, so you have to compile your 
own binaries after a change to the source. I don't remember where in the 
source, but this has come up before and should be searchable.

If you have already extended the watchdog timout, a signal 11 is almost always 
a software bug and the devs should be able to tell where in the code this 
happened from the traceback. That said, Bacula running for a long time using 
lots of pointers is also a decent test of memory, as well as i/o. Hardware 
errors, anything that causes a bit flip in RAM, usually results in a signal 11. 
But it is far more likely to be a software issue and you should file a bug 
report..



On 3/1/2020 6:22 PM, Chaz Vidal wrote:
Greetings all,
Our Bacula system crashed on Friday with a segmentation violation.

The system has been attempting to do a full backup of over 130TB of data over 
the past few weeks which we've appeared to have lost because of the crash.

Feb 28 09:56:31 <> bacula-dir[4211]: Bacula interrupted by signal 
11: Segmentation violation
Feb 28 09:56:31 <> bacula-dir[4211]: Kaboom! bacula-dir, bacula-dir 
got signal 11 - Segmentation violation at 28-Feb-2020 09:56:31. Attempting traceback.
Feb 28 09:56:31 <> bacula-dir[4211]: Kaboom! exepath=/usr/sbin/
Feb 28 09:56:31 <> bacula-dir: Bacula interrupted by signal 11: 
Segmentation violation
Feb 28 09:56:31 <> bacula-dir[4211]: Calling: /usr/sbin/btraceback 
/usr/sbin/bacula-dir 4211 /var/lib/bacula
Feb 28 09:56:31 <> postfix/smtpd[59719]: connect from 
localhost[127.0.0.1]
Feb 28 09:56:31 <> postfix/smtpd[59719]: 71CC36008A: 
client=localhost[127.0.0.1]
Feb 28 09:56:31 <> postfix/cleanup[59722]: 71CC36008A: 
message-id=<20200227232631.71CC36008A@<>.company.com>
Feb 28 09:56:31 <> postfix/qmgr[14399]: 71CC36008A: 
from=>.company.com>, size=593, nrcpt=1 (queue active)
Feb 28 09:56:31 <> postfix/smtpd[59719]: disconnect from 
localhost[127.0.0.1] helo=1 mail=1 rcpt=1 data=1 quit=1 commands=5
Feb 28 09:56:31 <> bacula-dir[4211]: It looks like the traceback 
worked...
Feb 28 09:56:31 <> bacula-dir[4211]: LockDump: 
/var/lib/bacula/bacula.4211.traceback
Feb 28 09:56:31 <> bacula-dir[4211]: bacula-dir: lockmgr.c:1221-0 
lockmgr disabled

I do not know how to read a traceback file to understand what may have been 
going on.  We are attempting to restart the backup again but unless we 
understand what happened the crash may appear again.

We are running Bacula Version: 9.4.2

Appreciate if anyone can share any insight?

Attempt to dump current JCRs. njcrs=7
threadid=0x7fb497491f40 JobId=0 JobStatus=R jcr=0x55980a04a4f8 
name=*JobMonitor*.2020-02-11_15.29.48_01
 use_count=1 killable=0
 JobType=I JobLevel=
 sched_time=11-Feb-2020 15:29 start_time=11-Feb-2020 15:29
 end_time=01-Jan-1970 09:30 wait_time=01-Jan-1970 09:30
 db=(nil) db_batch=(nil) batch_started=0
 wstore=0x55980a01ff28 rstore=0x55980a01ff28 wjcr=(nil) 
client=0x55980a026128 reschedule_count=0 SD_msg_chan_started=0
threadid=0x7fb495897700 JobId=104686 JobStatus=R jcr=0x7fb48806aea8 
name=job1.2020-02-11_17.38.56_13
 use_count=2 killable=1
 JobType=B JobLevel=F
 sched_time=11-Feb-2020 17:38 start_time=11-Feb-2020 17:38
 end_time=01-Jan-1970 09:30 wait_time=21-Feb-2020 16:50
 db=0x7fb4880059a8 db_batch=(nil) batch_started=0
 wstore=0x7fb48803fc18 rstore=(nil) wjcr=(nil) client=0x7fb4880481a8 
reschedule_count=0 SD_msg_chan_started=1
BDB=0x7fb4880059a8 db_name=bacula db_user=bacula connected=true
 cmd="UPDATE Media SET InChanger=0, Slot=0 WHERE Slot=25 AND StorageId IN 
(10) AND MediaId!=794" changes=1814
 RWLOCK=0x7fb4880059c0 

Re: [Bacula-users] Bacula Crashed / Segementation Violation

2020-03-03 Thread Kern Sibbald

Hello,

To respond to a seg fault, we need a number of things:

1. The Bacula version of all your components (Dir, SD, FD)

2. Where did you get your binairies?  Download from where, 
build-yourself.  If you built them, what commands did you use?


3. A traceback.  I don't see a valid traceback in your message. Your 
message does include a jcr dump, but that is not sufficient.


Best regards,

Kern

On 3/2/20 11:15 PM, Chaz Vidal wrote:

Thanks Josh for the response.

My team have recently just taken over this Bacula system and have it on 
previous reports that we have had backup jobs running for more than 6 days so 
potentially this Watchdog is not being triggered.

In archive examples I found that when the watchdog is triggered it would almost 
always write that information to the log file.  In this case there was no 
mention of Watchdog activity so I can only assume it is a software issue as you 
have mentioned?

I'm happy to file a bug report if there isn't anything that jumps out in the 
traceback file for anyone?

Thanks
Chaz


Chaz Vidal | ICT Infrastructure | Tel: +61-8-812804397 | Mob: +61-492-874-982 | 
chaz.vi...@sahmri.com

-Original Message-
From: Josh Fisher 
Sent: Monday, 2 March 2020 9:45 PM
To: Chaz Vidal ; Bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] Bacula Crashed / Segementation Violation

Caution

This email originated from outside the organisation. Do not click links or open 
attachments unless you recognise the sender and know the content is safe to do 
so.

Bacula has a built-in watchdog that kills a job that runs for more than
6 days. That period can be extended at compile time, so you have to compile 
your own binaries after a change to the source. I don't remember where in the 
source, but this has come up before and should be searchable.

If you have already extended the watchdog timout, a signal 11 is almost always 
a software bug and the devs should be able to tell where in the code this 
happened from the traceback. That said, Bacula running for a long time using 
lots of pointers is also a decent test of memory, as well as i/o. Hardware 
errors, anything that causes a bit flip in RAM, usually results in a signal 11. 
But it is far more likely to be a software issue and you should file a bug 
report..


On 3/1/2020 6:22 PM, Chaz Vidal wrote:

Greetings all,
Our Bacula system crashed on Friday with a segmentation violation.

The system has been attempting to do a full backup of over 130TB of data over 
the past few weeks which we've appeared to have lost because of the crash.

Feb 28 09:56:31 <> bacula-dir[4211]: Bacula interrupted by
signal 11: Segmentation violation Feb 28 09:56:31 <> 
bacula-dir[4211]: Kaboom! bacula-dir, bacula-dir got signal 11 - Segmentation violation at 
28-Feb-2020 09:56:31. Attempting traceback.
Feb 28 09:56:31 <> bacula-dir[4211]: Kaboom!
exepath=/usr/sbin/ Feb 28 09:56:31 <> bacula-dir: Bacula
interrupted by signal 11: Segmentation violation Feb 28 09:56:31
<> bacula-dir[4211]: Calling: /usr/sbin/btraceback
/usr/sbin/bacula-dir 4211 /var/lib/bacula Feb 28 09:56:31
<> postfix/smtpd[59719]: connect from localhost[127.0.0.1]
Feb 28 09:56:31 <> postfix/smtpd[59719]: 71CC36008A:
client=localhost[127.0.0.1] Feb 28 09:56:31 <>
postfix/cleanup[59722]: 71CC36008A:
message-id=<20200227232631.71CC36008A@<>.company.com>
Feb 28 09:56:31 <> postfix/qmgr[14399]: 71CC36008A:
from=>.company.com>, size=593, nrcpt=1 (queue
active) Feb 28 09:56:31 <> postfix/smtpd[59719]: disconnect from 
localhost[127.0.0.1] helo=1 mail=1 rcpt=1 data=1 quit=1 commands=5 Feb 28 09:56:31 
<> bacula-dir[4211]: It looks like the traceback worked...
Feb 28 09:56:31 <> bacula-dir[4211]: LockDump:
/var/lib/bacula/bacula.4211.traceback
Feb 28 09:56:31 <> bacula-dir[4211]: bacula-dir:
lockmgr.c:1221-0 lockmgr disabled

I do not know how to read a traceback file to understand what may have been 
going on.  We are attempting to restart the backup again but unless we 
understand what happened the crash may appear again.

We are running Bacula Version: 9.4.2

Appreciate if anyone can share any insight?

Attempt to dump current JCRs. njcrs=7
threadid=0x7fb497491f40 JobId=0 JobStatus=R jcr=0x55980a04a4f8 
name=*JobMonitor*.2020-02-11_15.29.48_01
  use_count=1 killable=0
  JobType=I JobLevel=
  sched_time=11-Feb-2020 15:29 start_time=11-Feb-2020 15:29
  end_time=01-Jan-1970 09:30 wait_time=01-Jan-1970 09:30
  db=(nil) db_batch=(nil) batch_started=0
  wstore=0x55980a01ff28 rstore=0x55980a01ff28 wjcr=(nil)
client=0x55980a026128 reschedule_count=0 SD_msg_chan_started=0
threadid=0x7fb495897700 JobId=104686 JobStatus=R jcr=0x7fb48806aea8 
name=job1.2020-02-11_17.38.56_13
  use_count=2 killable=1
  JobType=B JobLevel=F
  sched_time=11-Feb-2020 17:38 start_time=11-Feb-2020 17:38
  end_time=01-Jan-1

Re: [Bacula-users] Failed job notification to job specific email ids

2020-02-28 Thread Kern Sibbald

  
  
Hello Yateen,

Each Job can have a different message resource and thus messages
  can be sent where you want.  If you backup multiple users in a
  single Job then I do not see any simple way of doing what you
  want.
Best regards,
Kern

On 2/28/20 10:07 AM, Shaligram Bhagat,
  Yateen (Nokia - IN/Bangalore) wrote:


  
  
  
  
Hello
 
I am using Bacula 9.4.4.
community version on Centos 6.4
 
I need to send email
notification on backup job failure to specific users
associated with that job.
I do not think there is
a Job/Jobdef  directive to associate the user email id for
job fail notification?
 
If not how this can be
achieved? I know I can run RunAfterFailedJob script, but how
to associate a job to a specific email id?
 
Thanks
Yateen
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] New to Bacula: Restore Job Question

2020-02-27 Thread Kern Sibbald

  
  
Hello,
Sorry, I completely misunderstood your problem.  Perhaps someone
  on the list can give you some help.
Best regards,
Kern

On 2/26/20 11:00 PM, Chaz Vidal wrote:


  
  
  
  
Thanks for the response Kern and appreciate
  it.
 
I have read through the restore chapter of
  the manual and used the restore command from bconsole, sorry I
  should have clarified.
 
So I’m just wondering if there was a free
  drive for the restore shouldn’t the job start using it? Or am
  I missing something?  The submitted restore job was waiting on
  a high priority process to finish but because a restore job is
  priority 1 I expected it to utilise the free drive.
 
Thanks so much for the help.
 
Chaz
 
 
 

  
From: Kern
Sibbald 

Sent: Wednesday, 26 February 2020 8:23 PM
To: Chaz Vidal ;
Bacula-users@lists.sourceforge.net
Subject: Re: [Bacula-users] New to Bacula:
Restore Job Question
  

 

  Hello,
  Restore is a very interactive process, so rather than
running a restore job, one normally uses the restore command
in bconsole.  It is also possible to do graphical restores
with bat or with Baculum (web GUI).
  The restore command in bconsole will prompt you for
everything you need, but choosing files to restore is a bit
more complicated, so I recommend reading the Restore chapter
of the manual.
  Best regards,
  Kern
  
 
  

  

  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.6.2 + Status Report

2020-02-26 Thread Kern Sibbald

Hello,

I would like to discuss the following subjects:

1. Bacula Release Status

2. Bacula Binaries

3. Windows Binaries

4. Next Bacula Version

5. Bacula's 20th Anniversary

===

1. Bacula Release Status
The latest Bacula version 9.6.2 was released today to Source Forge and to
www.bacula.org.  This is a new version release since the last one was
version 9.4.4.  This version includes many enhancements to Baculum (the
Web GUI interface to Bacula), new Statistic for the Dir, SD and FD that
can be shown in Baculum or sent to Graphite, and a Docker plugin. We are
a bit behind in the Docker documentation, but it will come in the next
couple of weeks.

Note: versions 9.6.0 and 9.6.1 were withdrawn due to a bug in building
the readline version of Bacula.

2. Bacula Binaries
Thanks to the build support from Bacula Systems, we now have binaries for a
number of popular distributions.  If you need additional distributions,
please notify us on the bacula-users email list.

The binary release format is with standard packaging using debs rpms, … 
that are

specific to each system.  However, unlike most Linux distributions, we
install the Community Binaries into the directory /opt/bacula as
recommended in the Bacula manual and as used by the Bacula Enterprise
Edition.  This Bacula file release standard is not uncommon and vastly
simplifies installation, support, backup and recovery of Bacula itself.
Basically, most of the release will be installed into /opt/bacula except
for the few system files that must be placed in certain locations such as
the start/stop scripts and the man text.

To get access to Bacula pre-built binaries please go to
www.bacula.org and on the main menu bar select Downloads -> Bacula Binary
Package Download, then fill out the registration.  Note, ultimately you
will receive a special download link that will give you access to the
binaries.  Please keep that link for future access to the binary repository
for updates and new releases.

3. Windows Binaries
Since version 9.0.0, we have re-integrated the Windows File daemon source
code, and built Windows Binaries packages.  They can be found in
www.bacula.org -> Downloads -> Binary Download Center. Version 9.6.2 are
currently available.

4. Next Bacula Version
We expected to include many more features in the next version thanks to 
Bacula

Systems work, but unfortunately these new features were more complicated to
backport than I had imagined, and I had to settle for a more modest release
(9.6.2). Based on the backporting difficulties due to the divergence of the
Community and Enterprise versions, Bacula Systems has been working hard to
make a "universal Bacula core code" version that includes both the 
Enterprise
and Community code in a way that we can easily switch between the two 
versions,

as well as much more easily backport new features to and from the Community
version.

Hopefully the next Bacula version will include a large number of new 
features written by Bacula
Systems.  If all goes well this version will be released in about 9 
months (roughly the end of 2020).


5. Bacula's 20th Anniversary
For those of you familiar with Bacula, you may know that Bacula had
its 20th birthday on January 1, 2020.  Twenty years of Bacula

Thanks for using Bacula — be hap because there is much more to
come.

Kern

==


   Release Notes for Bacula 9.6.2

This is a new release with several new features and a number of bug
fixes.  The catalog database format remains unchanged from the 9.4.4 release

Note: Release-9.6.0 had a build error when using readline, and Release-9.6.1
had an inappropriate file size for the readline history file, so both 
releases

have been withdrawn.

Major Baculum New Features:
 - SELinux support
 - New graph types
 - Graphical client status
 - Graphical running job status
 - Capability to start, stop and restart components
 - Support for commands that can take a long time (label, estimate...)
 - List job files tab on the job history page
 - Bandwidth limit setting for client and for job
 - New statistics configuration page
 - Improvements to responsive interface
 - Option to show size unit values as decimal or binary bytes
 - Support for new directives
 - New Web controls (password, speed, multiple textbox)
 - New API functions
 - Job history list on job page

Bacula New features:
- Docker plugin.  Documentation for this plugin is not yet ready,
    but will be forthcoming within a few weeks.
- Statistics Collector for Dir, FD, and SD (interface to Graphite)
  - New Statistics resource
    Documentation for this feature is in the New Features section of
    the main manual
- Support for MacOS suspend in File daemon
- SD SyncOnClose directive in Device resource

26Feb20
 - Apply fix for history size from Martin Simmons
 - Fix missing part of patch 8135b9d21d -- readline truncate fix
 - baculum: Fix using bconsole with sudo o

Re: [Bacula-users] New to Bacula: Restore Job Question

2020-02-26 Thread Kern Sibbald

  
  
Hello,
Restore is a very interactive process, so rather than running a
  restore job, one normally uses the restore command in bconsole. 
  It is also possible to do graphical restores with bat or with
  Baculum (web GUI).
The restore command in bconsole will prompt you for everything
  you need, but choosing files to restore is a bit more complicated,
  so I recommend reading the Restore chapter of the manual.

Best regards,
Kern

On 2/23/20 11:19 PM, Chaz Vidal wrote:


  
  
  
  
Hi All,
I’m new to Bacula and we’ve recently took
  over a system in this new organization. I have been reading
  the manuals and started looking after the system we have
  inherited.
 
I’ve tried searching the archives but I
  have a few questions that hoping to get clarity on.
 
I attempted to create a new restore job but
  once submitted it sits waiting on a high priority job to
  finish.
 
There are a couple of priority 10 backups
  still running but there was free drive that could have been
  used by the restore job.
 
According to the manuals, there is a new
  directive that could be used to ensure mixed priority jobs can
  run together (Allow Mixed Priority) which the restore job did
  not have by default.
 
I would have thought that the free drive
  could be used and therefore doesn’t have to wait and because
  the restore job is a Priority 1 it should have just started
  it?
 
Hope someone can help and please excuse the
  newbie question.
 
Thanks
Chaz
 
 
 
 
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] bacula archive/off-site-backup to S3 storage, what's the best way?

2020-02-20 Thread Kern Sibbald

  
  
Hello,

Concerning lack of documentation as you noted below:  yes, we can
always improve documentation.  In this case, the need for special
libraries is documented.  For example in the ReleaseNotes file, it
is documented under 3 releases.  The most recent one states:

=
Release 9.4.3

This is a bug fix release for version 9.4.2.  It includes a number
of bug
fixes and patches. 

[snip] ...

S3 driver: If you are trying to build the S3 drivers, please
remember to use the
community supplied (from Bacula Enterprise) version of libs3.so
found at:

https://www.bacula.org/downloads/libs3-20181010.tar.gz
==

Not



On 2/17/20 3:27 PM, Erik Geiger wrote:


  
  

  
  


  On Mon, Feb 17, 2020 at 1:41
PM Radosław Korzeniewski 
wrote:
  
  

  Hello,
  
  
wt., 11 lut 2020 o
  18:41 Erik Geiger 
  napisał(a):


  

  You are referring
to this documentation? https://www.bacula.org/9.4.x-manuals/en/main/New_Features_in_9_4_0.html#SECTION00300100

  



Yes.
 

  

  I wasn't able to
build bacula with cloud support and I can't use
the rpm packages from bacula as they aren't
supported by the puppet module I'm using. So I
was looking for something like an "after" job
syncing to S3 with aws cli or the like.

  



Sorry for that. You should create a ticket at bugs.bacula.org to show
  that something is wrong about it.
  

  
  
  
  Hi
Radoslaw,
  
  
  Turned
out that I was able to build when using the libs3 provided
by bacula [https://www.bacula.org/downloads/libs3-20181010.tar.gz]
  Sadly
that wasn't really documented.
  
  
  

  
 

  

  But if the Cloud
Storage functionality is the way to go I'll
figure out how to compile with S3 support.
  
  So if I get the
documentation right the backup is first stored
to the local disk and afterwards moved to the
cloud while I could still do a restore from
local disk as long as I configure the configure
the "cache Retention", right?

  



The default local disk backup for cloud storage
  works as a cache only with fully configurable behavior
  and retention. You can use it as the single archive
  storage, so during backup all your data will be saved
  on local disks and then synced into S3 as configured.
  You can use it as a DR storage using Copy Jobs where
  your local disks will be your main storage which will
  be copied into a storage cloud after a local backup.
  All possible configurations depends on your
  requirements.
  
  I hope it helps.

  
  
  
  I do
have the cloud backup running, now. All works even better
than expected regarding the S3 upload. I also realised that
I can use "Cache Retention" so the local disk won't run out
of disk pace while still allowing fast restores within the
"Cache Retention" period.
  
  
  Tanks
again,
  
  
  Erik
  

  best regards
  -- 
  Radosław Korzeniewski
rados...@korzeniewski.net

  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  



Re: [Bacula-users] [EXTERNAL] Re: Bacula - waiting to reserve a device

2020-02-12 Thread Kern Sibbald

  
  
Hello,

One thing to keep in mind is that Bacula is designed to have
multiple simultaneous jobs writing to multiple devices all at the
same time.  Jobs will wait without notification their turn.  

However Bacula is not designed to deal with over committing your
resources -- i.e. have multiple incompatible jobs waiting for a
single tape drive or whatever. 

What Bacula does is when a job starts, it attempts to reserve all
the devices it will need (read/write) -- normally for Backup it is
just one write drive.  If all the devices are not available, it will
back out the attempted reserve and wait.  This allows other jobs to
possibly obtain reservations on those devices, and thus prevents
deadlocks.  

If you are in a condition of over commitment (i.e. Bacula cannot
reserve the needed devices), you are operating outside the design
limits of Bacula.  At that point, Bacula does its best to warn you
even to the extent of being annoying that something is wrong, namely
it cannot reserve the needed devices.  

In general, this particular problem is resolved by doing proper
scheduling so that you do not over commit your resources.  I.e.
schedule your jobs so that you do not have conflicting device
demands.

Best regards,
Kern

On 2/11/20 6:07 PM, Radosław
  Korzeniewski wrote:


  
  
Hello,


  pon., 10 lut 2020 o
14:50 Rao, Uthra R. (GSFC-672.0)[ADNET SYSTEMS INC] <uthra.r@nasa.gov>
napisał(a):
  
  

  
I have three Tape drives in the
  Tape Library. Say for example if two of the tape
  drives are busy running two different jobs from Pool A
  & B.  If two more jobs are waiting to start from
  Pool C & D
  

  
  
  
  First of all when designing a tape backup in Bacula you
should consider lowering the number of defined Pools to the
lowest acceptable level. This helps avoid resource shortage
during the backup window. This is my recommendation based on
multiple deployment experience.
   
  

  
 since only one Tape Drive is
  available the job from Pool C runs and the job from
  Pool D is still waiting for the Tape Drive and
  complains “ waiting to reserve a device” through email
  until a Tape drive become available to run the job.
  

  
  
  
  I can fully understand that it complains about it as it
cannot reserve the drive and has to wait. Could you explain
what is your expected behavior in this case?
  
  
  Thanks

-- 
Radosław Korzeniewski
  rados...@korzeniewski.net
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [EXTERNAL] Re: bacula cloud backup

2020-02-07 Thread Kern Sibbald

Hello,

If I am not mistaken, the RAIT code in Amanda was taken from code 
written long ago by CERN.   That code was mainly for speed rather than 
redundancy.


Writing to multiple simultaneous devices has always been on the projects 
list for Bacula, however, until recently it has never been a priority 
since there are alternatives such as Copy jobs. Admittedly Copying is 
not as good as writing two simultaneous streams for certain applications.


As the multiple simultaneous writing code is now more important, I hope 
to see it implemented in the not too distant future.


Best regards,
Kern

On 1/31/20 4:25 PM, dmaziuk via Bacula-users wrote:

On 1/31/2020 3:33 AM, Radosław Korzeniewski wrote:



Amanda had "RAIT" since forever.



If you would be so kind and have an idea how it could be implemented in
bacula and so kind to describe it on this list it would be 
implemented by

developers.


I don't have any idea how it could be implemented *in bacula* since I 
never looked at any bacula code.


In general sending the same stream of bytes to two different places 
instead of one is not exactly rocket science. I guess it and can be 
implemented in several places, e.g. special storage inside SD, or two 
different SDs.


I would probably send to two SDs at once because it allows for more 
flexible setups: the 2nd SD can write to "the cloud", it can be on the 
off-site box, etc. Whereas amanda-style RAIT device only protects 
against single disk/tape failure.


Dima



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula with two SDs

2020-01-15 Thread Kern Sibbald

Hello,

You need to use two (or more) different MediaType = xxx on the two SDs.  
No given MediaType should be on both SDs.  If you do that, Bacula will 
know which SD to access.


Best regards,

Kern

On 1/15/20 8:08 PM, Kelly Price wrote:

Hi
 I have run low on storage on my main server, so I added a second server 
with more disk space as a second SD server.

It mostly works, but bacula gets confused about what backup file is on which 
server. For instance, I have file Diff-A on server two, but jobs on server one 
want to write to it. How do I let bacula know that the file is on server two 
and should not be used for backups on server one? It keeps trying to mount 
Diff-A on server one.

Thanks,
Kelly

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Question about bacula

2019-12-13 Thread Kern Sibbald

Hello,

Some time ago, the project provided bare metal recovery, but the boot 
process has evolved significantly since then with many differences 
between vendor, which made maintenance of the BMR too onerous.  However, 
Bacula Systems as a part of their commercial offering does offer 
selected Linux distro as well as Window complete BMR capabilities.


One reasonable alternative is to ensure you have complete backups of the 
whole system, then in a disaster situation, reinstall the original OS 
followed by doing a full Bacula restore.  For those of you who use the 
Bacula recommended install options (most files going in /opt/bacula) 
restoring a fully functional Bacula is easy, and then once done, 
restoring the whole OS as of the last backup is not hard.  This 
procedure works fine (I have done it on several of my computers) however 
it may be slightly more time consuming than using the Bacula Systems BMR.


Best regards,
Kern

On 12/13/19 9:46 AM, Pierre Bernhardt wrote:

Hello,

I think what you mean is a bare metal recovery procedure.
This is generally possible but needs some special preparations and instructions.
It is not a full out of the box recovery procedure.

It depends on how secure you backup you servers.

I've already written a complex base article but I think
this is not the time to publish them here. I don't make
people to read boring and unfinished alpharelease stuff :-)

Cheers,
Pierre

Am 12.12.19 um 16:49 schrieb Gregor Burck:

Hi,

I've already a running system with bacula 9.4.4 and baculum.
My main question is, could I make a desaster recovery of my Windows and Linux 
Server?

It seem to me, sat bacula only made File Backup?

I suggest more than a veeam thing, but then I've to use Bacula Enterprise, that 
support things like hyper visor, SQL and Exchange and other features?

In the moment I've a proxmox cluster and use a mix from Backupassist and the 
proxmox own backup, I want to replace this solution thru an centrelized 
Backupserver.

Bye

Gregor





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [EXTERNAL] SAS HBA support on Linux

2019-12-04 Thread Kern Sibbald

  
  
Hello Radek,
Yes, I would like to see lin_tape support, and though I vaguely
  remember something about a lin_tape patch, there are a number of
  things necessary to get a patch accepted:
1. I must be able to get a signed CAA (copyright assignment
  agreement).  A number of people who send patches never follow
  through with the CAA.  (for those who have, many thanks!)
2. The patch must put the code into a new driver that can be
  loaded in place of the current tape driver.
3. The patch must follow our programming guidelines or not be too
  far from it (I can always fix a few minor things).
Best regards,
Kern

On 10/29/19 1:57 PM, Radosław
  Korzeniewski wrote:


  
  
Hello,


  wt., 22 paź 2019 o
10:46 Kern Sibbald <k...@sibbald.com>
napisał(a):
  
  (...) so this would leave
either a community submission, or Bacula Systems to
implement it.
  



I think the Community can prepare and test the required code as
Enterprise clients do not want to participate in such
development. I have a two BEE customers which just simply drop
lin_tape drive when they got info it is not supported under
Bacula and switch to plain st.
So, unless Bacula Systems decide to make an investment
  which could never payoff the Community should handle this.
  AFAIR on this list should be a patch prepared by some
  community developer which add support for lin_tape driver to
  Bacula. Unfortunate the patch was rejected by Kern at that
  time. I think we can use this patch and port it to the current
  Bacula version source code and allow a broad Community users
  to test this. We can add a compile time parameter which will
  simply enable this support and mark it extremely experimental.
  
  
  What do you think about it?
  -- 
  Radosław Korzeniewski
rados...@korzeniewski.net

  

  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [EXTERNAL] SAS HBA support on Linux

2019-10-22 Thread Kern Sibbald

On 10/21/19 6:00 PM, Alan Brown wrote:
> On 21/10/2019 16:13, Dimitri Maziuk via Bacula-users wrote:
>> On 10/21/2019 9:14 AM, Clark, Patti via Bacula-users wrote:
>> ...
>>> My question (rhetorical), why do you want to use an OS that is 
>>> recently released and still bleeding for a backup server?  I 
>>> understand upgrade headaches, but backup servers need to be rock solid.
>> Why use any RHEL > 6 for any kind of server?
>>
> Because RHEL6 is EOL and won't even install on newer hardware?
>
> (EOS is some time off, but it's on life support mode now - bug/security 
> fixes only and one of the more annoying "features" is that RHEL6 doesn't 
> support ECN properly - it has ECN but not ECN fallback, so it can hammer 
> the living daylights out of routers. In addition it's quite difficult to 
> fully secure a php-using webserver on RHEL6 using stock RPMs)
>
>
> As for "Why use RHEL?" - Corporate policy. The scientific Linux 
> community standardised on redhat 20 years ago and they're unlikely to 
> change anytime soon despite arguments about other flavours being 
> "better" (Most of them are still using Fortran code rooted in f77 which 
> breaks if you attempt to update it, or IDL despite it being "simply awful")
>
>
> I spend an awful lot of time making things work in a RHEL environment 
> when it "just works" in Debianish ones, but the choice of environment 
> isn't my call.
>
> Everything has its positives and negatives - as Kern knows, I _really_ 
> want Bacula to support the IBM lin_tape driver because it works far 
> better with LTOs and changers - particularly in FC/SAS multipath 
> environments but Baculasystems won't do that because the st driver works 
> just fine(*) and the RHEL vs Debian arguments work in much the same way.

My comments for the above paragraph:

Yes, a lot of decisions in Bacula and elsewhere are tradeoffs.  In the
case of the IBM lin_tape driver, I am sure that what you write about it
being better is correct.  However there needs to be a technical person
familiar with Bacula to implement it, and that person needs access to
such a tape setup (probably onsite).  I am past the time when I am
available to develop major new "tricky" technology, so this would leave
either a community submission, or Bacula Systems to implement it.  It is
quite possible that Bacula Systems would implement this feature if it
turns out to be something that a number (3 or more) of important
customers want -- i.e. it is mainly a cost/benefit question for them.

Best regards,

Kern

>
>
> (*) For some values of "fine" - st and sg don't handle multipathing _at 
> all_ and you end up with door locking contention issues that have caused 
> us issues for 15 years
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-27 Thread Kern Sibbald

  
  
Hello,

See below ...

On 9/26/19 6:22 PM, David Brodbeck
  wrote:


  
  
Part of the problem is it takes upwards of ten
  minutes for a job to fail when a workstation isn't available
  -- which is entirely correct, since the network connection has
  to time out. 
  

Well actually the wait time has a default (currently 3 minutes), but
it can be defined by the administrator using the FDConnectTimeout. 
At the current time, the value is "global" for all clients because
it is set in the Director resource.  At some future time, it might
be worth while to allow a client by client connect timeout to be
set.


  
However, the SD reservation is made *before* it
  tries to contact the FD, so I end up with resource starvation
  where jobs that are waiting to time out tie up resources that
  could be used by other jobs. I'm guessing the assumption is
  that clients will always be available, but the SD might be
  maxed out, so the code assumes it's more efficient not to
  contact a client until the director knows it has the resources
  to actually run the job.
  

For your particular case, though it is logical to attempt to acquire
all the resources prior to connecting to the client, it might make a
certain sense to connect to the FD before trying to reserve the
devices.  


  

  
  
  One option would be to stagger the start times of my jobs
so only the maximum the SD can handle get launched in any
given 10 minute window, but that adds a lot of complexity to
my configuration, since I currently can just allow JobDefs
to pull in the schedule for all clients. I'd have to define
start times individually, and maintain those in order to
keep them balanced as I add/remove clients. Adding enough
disks for the worst case isn't going to be possible. (I'm
assuming one client per spindle is optimal for disk arrays
-- maybe that's too conservative?)
  
  
  I've just been putting up with the error messages rather
than deal with the added maintenance of that approach. The
extra alert emails can be dealt with by filtering my
incoming email.

  


Best regards,
Kern


  




  On Thu, Sep 26, 2019 at 1:28
    AM Kern Sibbald <k...@sibbald.com> wrote:
  
  Hello,

Bacula does already attempt to acquire the needed devices in
the SD and
then backs them out if all the needed resources cannot be
obtained. 
This works quite nicely.   Consequently, while the job is
waiting the
resources are released in the SD. 

The problem occurs because the SD realizes that the
resources are not
available, so it will wait a short period of time trying
again to
acquire the resources, which is what one wants for virtually
all jobs. 
When it cannot acquire the resources the SD will fail the
job.  The
problem occurs because the user is over committing the SD
resources. 
The solution is to get more drives or modify how you run
jobs.

From what I understand in this case is that the user has a
large number
of jobs that regularly fail and thus the user explicitly
over commits
the resources.  The consequent is that Bacula works as it
should but the
user gets lots of messages about the SD not being able to
get resources.

Bacula was designed in a way were it expects to have the
needed
resources available (i.e. the configuration should be
optimized for the
available resources).  It also handles the case where you
over load the
SD (too many jobs for available resources), but in that case
it will
warn you, which is exactly what 99% of all users want.

One possible solution would be to add a new directive that
suppresses
the reservation failure message.  However there is very
likely a better
solution with the existing Bacula, I just do not know what
it is at this
time.  This is the first time in 19 years that this problem
has come up,
so before changing anything in the code, it has to be very
clearly
understood, which

Re: [Bacula-users] False "Intervention needed" flood

2019-09-27 Thread Kern Sibbald
Hello,

Of course anything that would improve Bacula is possible.  Concerning
possible source code changes that do what you call exponential back-off,
it is not something that I have on my personal agenda. 

Unfortunately normally what happens is that someone develops new code
and then submits it. This is not the best approach because if the
feature or design is not appropriate for Bacula, the code may not be
accepted for inclusion in Bacula.

A far better procedure would be for some person to design the feature,
submit the design to the project, get the design approved, then
implement and submit the code. 

Best regards,
Kern



On 9/26/19 2:41 PM, Gary R. Schmidt wrote:
> On 26/09/2019 18:37, Kern Sibbald wrote:
>> See below ...
>>
>> On 9/25/19 7:38 PM, Andrea Venturoli wrote:
> [SNIP]
>>>
>>> Yes: as I said it's not a big problem, just an annoyance.
>>> *Not* solving the resource contention problem, but just avoiding to
>>> nag about it would probably be enough (at least to me).
>>
>> Bacula is not capable of solving resource contentions itself other than
>> trying again.
>
> Would it be very difficult to change Bacula so that rather than trying
> every five (or whatever) minutes it did an exponential back-off?
>
> Such as, after the first failure try again in five minutes, then again
> in fifteen, then in sixty minutes, then two hours, four, eight,
> sixteen, too many retries I am giving up?
>
> I don't think this change would be simple, but I think it should be
> possible.
>
> Cheers,
>     Gary    b-)
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-26 Thread Kern Sibbald
See below ...

On 9/25/19 7:38 PM, Andrea Venturoli wrote:
> On 2019-09-25 18:42, David Brodbeck wrote:
>> I'm not sure how Max Full Interval solves the problem. I have used it
>> (before we switched to virtual full) but it doesn't really pertain to
>> resource contention. Maybe I didn't explain the problem very well.
>
> I agree.
> I'm *also* using Max Full Interval & co, but I fail to see how this
> relates to the problem we are discussing.
>
> Could you elaborate on this?

Aside from waiting for the next nightly backup due to an off-line
machine a major problem with having unavailable clients is that a Full
could be skipped by having the job failed.  Max Full Interval resolves
this issue.  It does not however resolve the issue that if the machine
is always off during the backup window, the machine might never be
backed up, which is where rescheduling can help.
>
>
>
>
>> It's a minor issue, so I'm not too concerned about it.
>
> Yes: as I said it's not a big problem, just an annoyance.
> *Not* solving the resource contention problem, but just avoiding to
> nag about it would probably be enough (at least to me).

Bacula is not capable of solving resource contentions itself other than
trying again.
>
>
>
>  bye & Thanks
> av.
>



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-26 Thread Kern Sibbald
Hello,

Bacula does already attempt to acquire the needed devices in the SD and
then backs them out if all the needed resources cannot be obtained. 
This works quite nicely.   Consequently, while the job is waiting the
resources are released in the SD. 

The problem occurs because the SD realizes that the resources are not
available, so it will wait a short period of time trying again to
acquire the resources, which is what one wants for virtually all jobs. 
When it cannot acquire the resources the SD will fail the job.  The
problem occurs because the user is over committing the SD resources. 
The solution is to get more drives or modify how you run jobs.

From what I understand in this case is that the user has a large number
of jobs that regularly fail and thus the user explicitly over commits
the resources.  The consequent is that Bacula works as it should but the
user gets lots of messages about the SD not being able to get resources.

Bacula was designed in a way were it expects to have the needed
resources available (i.e. the configuration should be optimized for the
available resources).  It also handles the case where you over load the
SD (too many jobs for available resources), but in that case it will
warn you, which is exactly what 99% of all users want.

One possible solution would be to add a new directive that suppresses
the reservation failure message.  However there is very likely a better
solution with the existing Bacula, I just do not know what it is at this
time.  This is the first time in 19 years that this problem has come up,
so before changing anything in the code, it has to be very clearly
understood, which is not the case (at least for me).

Another solution is for the user to modify the source code and remove
the warning message.

Best regards,
Kern

On 9/25/19 10:50 AM, Andrea Venturoli wrote:
> On 2019-09-25 10:19, Radosław Korzeniewski wrote:
>> Hello,
>>
>> sob., 21 wrz 2019 o 00:52 David Brodbeck > <mailto:brodb...@math.ucsb.edu>> napisał(a):
>>
>>     I think this is a somewhat unfortunate design decision, to be
>>     honest. (...)
>>
>>
>> So what should be the best design in this case which should solve the
>> problem?
>
> I'm not so into the code to tell for sure.
> Maybe rescheduling should release the SD once the job first fails and
> reserve again when it starts the next time?
>
>  bye & Thanks
> av.
>



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-25 Thread Kern Sibbald

  
  
Hello,

Bacula was not designed to handle the case you are describing where
you have multiple clients that are offline.  This was not really a
design decision, but rather a design limitation because when the
code was implemented there was no restart functionality. 

A possible way to fix the problem would be to use Max Full Interval
and Max Diff Interval ...

Best regards,
Kern

On 9/21/19 12:19 AM, David Brodbeck
  wrote:


  
  




  On Thu, Sep 19, 2019 at 6:33
AM Kern Sibbald <k...@sibbald.com> wrote:
  
  

  Hello,
  
  
  I
concur with David.  When these jobs are scheduled,
Bacula will attempt to acauire the needed Storage
resources.  When the resources are busy the job waits,
and after a certain time, Bacula will inform you that
the resources are not available.
  
  
  These
messages generally occur when you over commit the SD
resources.  If you are using disk, increasing the
maximum simultaneous jobs in the Device resource and
restarting the SD will generally solve the problem, but
you might also have to assign more Storage devices
depending on what you are doing.

  
  
  
  I think this is a somewhat unfortunate design decision,
to be honest. I back up a fairly large number of
workstations, and on any given night a certain number of
them will be off or otherwise unavailable. I have these jobs
set to reschedule on failure so that they run when the
workstation eventually gets switched on.
  
  
  The problem is, with the behavior you mention, I can't
accurately control how many simultaneous *running* jobs are
using storage. If I set the max to the number of jobs that I
actually want to be able to use storage simultaneously, I
end up with some jobs that could otherwise run waiting for
resources because those resources are committed to a job
that's retrying a workstation that may or may not appear. If
I compensate by setting the max higher, I risk
overcommitting my storage bandwidth on nights when all the
workstations *are* available.
  
  

-- 

  David Brodbeck
System Administrator, Department of Mathematics
  University of California, Santa Barbara
  
  

  

  


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Storage Group with multiple disks

2019-09-25 Thread Kern Sibbald

  
  
Hello,

It sounds like you need to learn how Bacula uses the Media Type
directive.  By defining different Media Types for different devices,
you ensure that Bacula can find the right device.  In general every
device that has a different device location (different i-scsi or a
different path to the device location) should have a different Media
Type.  That way, the Director will know what device to use for
Volumes it needs to read.

Best regards,
Kern

On 9/24/19 8:12 AM, Shaligram Bhagat,
  Yateen (Nokia - IN/Bangalore) wrote:


  
  
  
  
Hi All,
 
I am a newbie to Bacula.
 
We are piloting Bacula v
9.4.4 for my project and intend to use the VirtualFull
backup scheme.
 
We have 20 number of
 i-scsi disks  (disk1 through disk20) on Bacula SD server.

We have defined a
storage group StorageGroup consolidating the abovementioned
disks defined as 20 Devices, each Device configured to refer
to each of the scsi disk.
e.g. Device1 à disk1, Device2
  à disk2 ..and so on.
 
Things went fine with
the initial full backups of various clients simultaneously.
Bacula  picked up the next available device and created
volumes under the selected Device’s disk.
Then we took incremental
backups. Again Bacula wrote the incremental volumes on the
selected Device’s disk.
 
However when we tried to
do a VirtualFull job, Bacula again tried to use the next
available drive, lets say Device1(disk1) and tried to read
the constituent volumes (previous full + incremental )
However the constituent
 volumes had been created through a different Device, lets
say disk7. So Bacula reported that the candidate volume is
not found under the current job’s Device (disk1).
 
Question is : In case of
VirtualFull job;  how to tell Bacula to locate/read the
constituent volumes through the Device (disk) where they
were originally created.
 
Thanks in advance,
 
Yateen Bhagat
 

 
 
 
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-19 Thread Kern Sibbald

On 9/19/19 4:18 PM, Andrea Venturoli wrote:

On 2019-09-19 15:33, Kern Sibbald wrote:

Hello,


Hello.



I concur with David.  When these jobs are scheduled, Bacula will 
attempt to acauire the needed Storage resources.  When the resources 
are busy the job waits, and after a certain time, Bacula will inform 
you that the resources are not available.


Fine.
However, I believe the subject is misleading then: no "intervention" 
is actually needed.

The load is high, but the problem will go away by itself.


Well it is possible it is a over commit causing a "deadlock". Bacula is 
not smart enough to figure out what is not right.  If you get your use 
of SD resources set correctly you should not get the messages.




Is there a switch to turn these messages off?



There is no simple switch to turn the messages off, but perhaps you 
could filter them ether in the Message resource or by where you send them.


Kern





If you are using disk, increasing the maximum simultaneous jobs in 
the Device resource and restarting the SD will generally solve the 
problem, 


Ok, I had increased maximum jobs, but didn't restart the SD.
I'll try again.



 bye & Thanks
av.





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] False "Intervention needed" flood

2019-09-19 Thread Kern Sibbald

  
  
Hello,


I concur with David.  When these jobs
  are scheduled, Bacula will attempt to acauire the needed Storage
  resources.  When the resources are busy the job waits, and after a
  certain time, Bacula will inform you that the resources are not
  available.


These messages generally occur when you
  over commit the SD resources.  If you are using disk, increasing
  the maximum simultaneous jobs in the Device resource and
  restarting the SD will generally solve the problem, but you might
  also have to assign more Storage devices depending on what you are
  doing.


Best regards,
Kern



On 9/12/19 8:17 PM, David Brodbeck
  wrote:


  
  I think what's going on here is even though these
jobs aren't running they've reserved a device for when (if) they
do. This is probably exceeding the maximum job limit for your
SD.
  
  
On Wed, Sep 11, 2019 at 5:14
  AM Andrea Venturoli <m...@netfence.it> wrote:

Hello.
  
  I've got an installation of 9.4.3 on FreeBSD/amd64, where
  several 
  clients are doing backups on a few SDs.
  Since some of these clients are laptops, I've activated
  "Reschedule On 
  Error" to let Bacula try again in an hour if a laptop is not
  there.
  
  
  
  Since then, I'm getting a lot of messages like the following:
  
  Subject: Bacula: Intervention needed for
  XXX.2019-09-11_10.00.00_56
  Body:
  11-Sep 10:25 XXXr-sd JobId 15335: JobId=15335, Job 
  XXX.2019-09-11_10.00.00_56 waiting to reserve a device.
  
  This happens with several different clients.
  
  
  
  The point here is that no manual intervention is really
  needed: the 
  backup will eventually succeed automatically if the laptop
  arrives in 
  the LAN or fail if it doesn't.
  I think there's nothing I can do or should do.
  
  So, how can I get rid of these messages?
  Is this a bug or a feature I don't understand?
  
    bye & Thanks
          av.
  
  
  ___
  Bacula-users mailing list
  Bacula-users@lists.sourceforge.net
  https://lists.sourceforge.net/lists/listinfo/bacula-users

  
  
  
  
  -- 
  
David Brodbeck
  System Administrator, Department of Mathematics
University of California, Santa Barbara


  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Out of range value for column 'FileIndex' at row 1

2019-09-08 Thread Kern Sibbald

  
  
Hello,


It looks to me like you forgot to run
  the update_bacula_tables script prior to running jobs.


Best regards,
Kern



On 9/8/19 11:07 AM, Eyermann, Frank
  wrote:


  
  
  
  
Dear all,
 
since upgrading to 9.4.4
(from 9.2.1) most of my backups fail.
 
Error message is:
Fatal error: sql_create.c:841 Fill
File table Query failed: INSERT INTO File (FileIndex, JobId,
PathId, FilenameId, LStat, MD5, DeltaSeq) SELECT
batch.FileIndex, batch.JobId, Path.PathId,
Filename.FilenameId,batch.LStat, batch.MD5, batch.DeltaSeq
FROM batch JOIN Path ON (batch.Path = Path.Path) JOIN
Filename ON (batch.Name = Filename.Name): ERR=Out of range
value for column 'FileIndex' at row 1
 
I’m using a MySQL
database.
 
Digging around in
database and increasing MySQL logging, I’ve found that
bacula is inserting records with negative FileIndex into
batch, but in Table File column FileIndex
is unsigned bigint.
 
Best regards,
 
Frank
 
 
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Enrypting all data in transit and at rest

2019-08-14 Thread Kern Sibbald

  
  
Hello Heitor,

I recommend that you check the details of the BEE automatic
encryption feature.  If I am not mistaken you are referring to the
new PSK (private shared key) that BEE uses for authentication.  This
means that Bacula will be much more secure against man-in-the-middle
attacks and such.  However, I don't think it helps with data
encryption.

I am currently trying to get it into the next community version.  I
believe that the person who implement the BEE PSK is or will be
working on creating a community patch for it ...

Best regards,
Kern

On 8/14/19 3:50 PM, Heitor Faria wrote:


  
  

  
  Hello Lauri (forgot to copy the group),
  
>Also, this information needs to then
  travel the network connections in

>the picture where it says File
  Attributes? I suppose I can then use

>Bacula TLS

>(https://www.bacula.org/5.2.x-manuals/en/main/main/Bacula_TLS_Communications.html)

>to protect all that unencyrpted data
  between the File Daemon, Storage

>Daemon and Director. Securing Director -
  Catalog DBMS connection is

>then out of scope of Bacula. Encrypting
  those will ensure my data is

>protected while in transit?

>

>In order to encrypt all data in transit
  and at rest I need to

>- Enable Data Encryption for the Volumes

>- Configure Bacula TLS
  
  I guess if you use data encryption you dont
need to use TLS, because data is already encrypted.
  
  If you just to encrypt transferred data you
only need TLS.
  
  Anyway, BEE has a very exciting new feature
which is automatic TLS, much easier to set up. Not sure if
it will be available in Community Bacula next V. 11 release.



Regards,

-- 


  
  
  

  

  

  
MSc
Heitor Faria
   CEO
Bacula LATAM
  
  
mobile1:
+ 1 909 655-8971
   mobile2:
+ 55 61 98268-4220
  
  


  

  


  

  


  
  
América
Latina
  
  
bacula.lat
  | bacula.com.br

  

  

  

  
  
  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Restore fails with "Restore OK -- warning file count mismatch"

2019-07-25 Thread Kern Sibbald


On 7/25/19 7:45 PM, Kai Gallasch wrote:


Am 25.07.19 um 11:45 schrieb Martin Simmons:

On Wed, 24 Jul 2019 22:55:10 +0200, Kai Gallasch said:

Am 23.07.19 um 19:57 schrieb Martin Simmons:

bls -v -V T00109L4 /dev/nsa0

Hi Martin.

To debug this I recreated the bacula postgres database, labeled the
volume T00111L4 and did a full backup of the client ns.free.de which was
successful. After this I tried a restore of /etc/resolv.conf which
failed again with message "Restore OK -- warning file count mismatch"

Then I did "bls -v -V T00111L4 /dev/nsa0"

I attached the output below (bls output shortened to only resolv.conf)
All the files of the full backup seem to be on this tape.

That is very strange.

Can you try it with a fileset containing just a single top level file (again
with an empty database & tape)?

e.g.

File = /COPYRIGHT

Hi.

Backing up and restoring the single file /COPYRIGHT works!
The same result shows if I extend it to /COPYRIGHT and the /etc directory.

As soon as I have File = / in the config, the restore fails with the
"file count mismatch"

I also tried setting onefs=no but the error persists.

And I also did a backup of a linux VM (hosted on FreeBSD) with bacula
and both backup and restore work without problems. So maybe this is
FreeBSD related.

Would it be beneficial for understandind the problem if I set up a dummy
FreeBSD Backupclient (minimal install, bacula backup-client) do a full
backup on a empty tape and failing restore ("file count mismatch) and
send the full output of the bls command over?

BTW: Kern: Most of my backup clients are FreeBSD Jails and inside them
resolv.conf is not a Symlink. I am backing up a zfs Filesystem mounted
on "/" and in the Backup job am ignoring other filesystems (onefs=yes)
as nullfs, devfs, tempfs, etc. This had worked for me for years.


To the best of my knowledge, nothing has changed concerning exactly 
which files are chosen and how they are backed in *many* years.  What 
has changed is that on FreeBSD one now compiles with Clang rather than 
GNU C++.


Best regards,

Kern




Kai.


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] preserving ACL permissions

2019-07-25 Thread Kern Sibbald

Hello,

I saw that someone mentioned using Samba.  Bacula does not save and 
restore acl across NFS or Samba.  To do so you must have a filesystem 
that is mounted locally  on the File Daemon.


Best regards,

Kern

On 7/25/19 12:08 PM, Adam Weremczuk wrote:

On 25/07/19 10:55, Martin Simmons wrote:

Yes, the FileSet may be the problem -- try reordering the Options 
clauses so
that the "default" Options clause containing xattrsupport etc is the 
last one.


The manual says: "However, one additional point is that in the case 
that no

match was found, Bacula will use the options found in the last Options
resource. As a consequence, if you want a particular set of "default" 
options,

you should put them in an Options resource after any other Options."

__Martin

Thanks Martin,
Would you be kind enough to re-craft my config file based on the 
original?
It's an important server with about 800GB of data being backed up over 
several hours.

Therefore I would much prefer to avoid any horrible mistakes.
I'm assuming any filest change forces the next backup to be full 
regardless of the schedule?



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Restore fails with "Restore OK -- warning file count mismatch"

2019-07-25 Thread Kern Sibbald

Hello,

Your problem is probably due to the fact that resolv.conf is a link 
rather than a file.  If this is the case (as it is on my machine) trying 
to restore only the link without restoring the actual file, may confuse 
Bacula.  I would recommend you restore a few other files from the /etc 
directory just to see if they are found or not.  Also after you have 
marked resolv.conf you might try lsmarked as well as a dir while the 
Bacula restore program is cd'ed to /etc.  This might show you more 
information about what Bacula has actually saved for /etc/resolv.conf


Best regards,

Kern

On 7/23/19 7:57 PM, Martin Simmons wrote:

I would start by using "bls -v -V T00109L4 /dev/nsa0" (the command line
utility) to see what is on the T00109L4 tape.

Can you repeat it with a small backup to a tape that contains nothing else?

__Martin



On Tue, 23 Jul 2019 14:37:46 +0200, Kai Gallasch said:

Hi.

For long years I have been running Bacula Server on FreeBSD connected to
a 60 Slot tape library. Some weeks ago I noticed that restoring was not
possible any more. Backup jobs run successfully, but the restore fails
with "Restore OK", but no files restored.

This is really dangerous for users that blindly expect that a restore
will work, if the backup run was successful! In situations when you need
your backup the most, you find out too late that you cannot restore.

FreeBSD 11.2-RELEASE-p11
bacula9-server-9.4.3
postgresql95-client-9.5.18
postgresql95-server-9.5.18

What I already tried:
- dbcheck (All)
- reindexing postgres db
- relabel tapes, restart with a fresh postgresdb
- raise debug level
- start director in debug mode (-d 200) to find the problem
- try to restore single files locally on the backup server
- try to restore single files remotely on the client
- try to do full restores of last successful backup of a client
- disable all backup clients but one.

How can I debug this? The config did not change for years.
Please find below some output of the a failed restore run.

Any help appreciated.
Kai.



Select the Client (1-65): ns.free.de
Expected a positive integer, got: ns.free.de
Select the Client (1-65): 37
Automatically selected FileSet: ns.free.de-fileset
+---+---+--+-+-++
| jobid | level | jobfiles | jobbytes| starttime   |
volumename |
+---+---+--+-+-++
|   643 | F |   29,551 | 944,425,507 | 2019-07-21 02:00:03 |
T00109L4   |
|   707 | I |   86 | 208,334,401 | 2019-07-22 01:23:04 |
T00109L4   |
|   772 | I |   97 | 209,640,913 | 2019-07-23 01:23:04 |
T00109L4   |
+---+---+--+-+-++
You have selected the following JobIds: 643,707,772

Building directory tree for JobId(s) 643,707,772 ...
++
27,528 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /
$ cd etc
cwd is: /etc/

$ add resolv.conf
1 file marked.
$ done
Bootstrap records written to /bacula/workdir/singsing-dir.restore.5.bsr

The Job will require the following (*=>InChanger):
Volume(s) Storage(s)SD Device(s)
===

*T00109L4  LTO-3 Drive-1


Volumes marked with "*" are in the Autochanger.


1 file selected to be restored.

Using Catalog "MyCatalog"
Run Restore job
JobName: RestoreFiles
Bootstrap:   /bacula/workdir/singsing-dir.restore.5.bsr
Where:   /var/spool/bacula/restore
Replace: Always
FileSet: singsing.mgmt.free.de-fileset
Backup Client:   ns.free.de
Restore Client:  ns.free.de
Storage: LTO-3
When:2019-07-23 12:29:07
Catalog: MyCatalog
Priority:10
Plugin Options:  *None*
OK to run? (yes/mod/no): mod
Parameters to modify:
  1: Level
  2: Storage
  3: Job
  4: FileSet
  5: Restore Client
  6: When
  7: Priority
  8: Bootstrap
  9: Where
 10: File Relocation
 11: Replace
 12: JobId
 13: Plugin Options
Select parameter to modify (1-13): 9
Please enter the full path prefix for restore (/ for none): /tmp
Run Restore job
JobName: RestoreFiles
Bootstrap:   /bacula/workdir/singsing-dir.restore.5.bsr
Where:   /tmp
Replace: Always
FileSet: singsing.mgmt.free.de-fileset
Backup Client:   ns.free.de
Restore Client:  ns.free.de
Storage: LTO-3
When:2019-07-23 12:29:07
Catalog: MyCatalog
Priority:10
Plugin Options:  *None*
OK to run? (yes/mod/no): yes
Job queued. JobId=793
*
You have mes

Re: [Bacula-users] AWS S3 Backup

2019-07-19 Thread Kern Sibbald

  
  


On 7/18/19 10:14 PM, Bryan Sockel
  wrote:


  
  Looking over my setup again.  I am not sure if the S3
Cloud driver is being compiled correctly.  How can i tell to see
if the if it was compiled correctly.
   
  I am following the process here: https://stackoverflow.com/questions/12172504/installing-the-libs3-on-linux but
with the drivers from the bacula.org website.  I am never able
to find the dependencies in step 5.  Tried installing the 2
rpm's created with the the nodep switch. 
  

   

  
   
   
  

  Thank You
   
  From: "Bryan Sockel
  (bryan.soc...@mdaemon.com)"
  
  To: "Bryan
Sockel" , "Bryan Sockel via
Bacula-users (bacula-users@lists.sourceforge.net)"
, "Kern Sibbald
(k...@sibbald.com)" 
Date: Thu, 18 Jul 2019 10:11:07 -0500
Subject: Re: [Bacula-users] AWS S3 Backup
 
Should i be concerned that
  the plugins are being Rejected?
 
plugins.c:133-0 Rejected
  plugin: want=-sd.so name=s3 len=2
  plugins.c:133-0 Rejected plugin: want=-sd.so
  name=libnss3.so len=10
  plugins.c:133-0 Rejected plugin: want=-sd.so
  name=libs3.so.3 len=10

  

  



These objects (files) are not Bacula plugins.  I suspect that
  your definition of the PluginDirectory is not pointing to the
  Bacula plugins directory.  

You are probably using some old distro supplied configuration,
  which installs Bacula in may directories on your filesystem --
  this is *very* bad particularly for disaster recovery.  I
  recommend that you remove all old binaries, rebuild with the
  recommended Bacula ./configure set (installs almost everything in
  /opt/bacula/...).  Then try again.
Best regards,
Kern


  

  

   
 

  
Thank You,

  Bryan
  Sockel
  Systems Administrator
  o:
  817-601-3222 ext: 233    e:
  bryan.soc...@mdaemon.com
  
  MDaemon
Technologies
  Simple
Secure Email
  
  Visit us on Facebook
  | LinkedIn
  | YouTube
  
  Sent using the MDaemon
Messaging Server
  

 
  From: "Bryan Sockel via Bacula-users
(bacula-users@lists.sourceforge.net)"

To: "Kern Sibbald (k...@sibbald.com)"
,
bacula-users@lists.sourceforge.net
Date: Thu, 18 Jul 2019 08:04:17 -0500
Subject: Re: [Bacula-users] AWS S3 Backup
 
Hello,
 
Here is the
  requested Information:
 
Plugin Directory = "/usr/lib64"
ls  /usr/lib64/bacula-sd-cloud-driver-9.4.4.so
  /usr/lib64/bacula-sd-cloud-driver-9.4.4.so
 

  
 
bacula-sd -d20
  [root@bacula bacula]# bacula-sd:
  address_conf.c:274-0 Initaddr 0.0.0.0:9103
  bacula-sd: init_dev.c:435-0 Open SD driver at
  /usr/lib64/bacula-sd-cloud-driver-9.4.4.so
  bacula-sd: bnet_server.c:86-0 Addresses
  0.0.0.0:9103
  bacula-sd: init_dev.c:440-0 Lookup
  "BaculaSDdriver" in driver=cloud
  bacula-sd: init_dev.c:442-0 Driver=cloud entry
  point=7f295e762e90
  bacula-sd: stored.c:610-0 SD init done
  CloudStorage-1
  List plugins. Hook count=0
  List plugins. Hook count=0
 
 

  status storage=Cloud
Connecting to Storage daemon Cloud at
bacula:9103
  bacula-sd Version: 9.4.4 (28 May 2019)
x86_64-pc-linux-gnu redhat (Core)
Daemon started 18-Jul-19 07:48. Jobs: run=0,
running=0.
 Heap: heap=135,168 smbytes=31,242
max_bytes=150,467 bu

Re: [Bacula-users] Bacula 9.44 Aligned -Disk Driver Not Working

2019-07-19 Thread Kern Sibbald

Hello Martin,

Yes, correct Bacula (and Bacula Systems) does not yet release source 
code because we have a patent pending on the technique. Working with the 
USPTO on this particular patent is very frustrating.  We are also 
working with the EU patent reviewers, and though it takes time, the 
process is moving forward nicely (I hope).


I will talk to the package builder to see if he (or possibly the QA 
department) can test and possibly fix the binaries.


Best regards,

Kern

On 7/18/19 12:27 PM, Martin Simmons wrote:

I can see why Bacula Systems don't make the SOURCE available yet, but shipping
a useless binary package is probably not what they intended so you should ask
them about it.

__Martin




On Thu, 18 Jul 2019 11:30:32 +0200, Olivier Delestre said:

Hi,

All my servers are install with the Bacula Community packages from repo :
baseurl=http://bacula.org/packages/x/rpms/9.4.4/el7/x86_64

bacula-aligned.x86_64    9.4.4-1.el7 @Bacula-Community
bacula-libs.x86_64   9.4.4-1.el7 @Bacula-Community
bacula-postgresql.x86_64 9.4.4-1.el7 @Bacula-Community
--
/opt/bacula/plugins/
-rwxr-x---. 1 root bacula  7776 Jun 25 10:41
bacula-sd-aligned-driver-9.4.4.so
--
And in my conf sd :
Storage { # definition of myself
    Name = -sd
    SDPort = 9103  # Director's port
    WorkingDirectory = "/opt/bacula/working"
    Pid Directory = "/opt/bacula/working"
### NEW 9.0.8
    Plugin Directory = "/opt/bacula/plugins"
###

I just notice that the community version still does not have this
feature ( Aligned Volume ).

https://www.bacula.org/bacula-binary-package-download/
"and Bacula Systems has been advised that the source should not be made
available until the patent is granted."

Thanks. and let us know when it is available.





Le 17/07/2019 à 14:34, Martin Simmons a écrit :

The open source code contains a null aligned driver, but if your
/opt/bacula/plugins/bacula-sd-aligned-driver-9.4.4.so was installed from the
Bacula Community packages then you will have to ask Bacula Systems about it
(https://www.bacula.org/bacula-binary-package-download/).

__Martin



On Wed, 17 Jul 2019 11:53:45 +0200, Olivier Delestre said:

Hi,

I'm sure the aligned volume works in July 2018 for my test ( with older
version than 9.4.4 ).

I remenber i have 2 files generate for the "tape" ( Test001 and
Test001.add  ).

But not today.

The Plugin is still avalaible with the Bacula 9.4.4 ??


Le 11/07/2019 à 18:49, J W a écrit :

I just installed Bacula 9.44 Community and am backing up to a ZFS
drive. It works fine with a regular setup.

If I switch to using the Aligned device type then I get the error
below. I switched back to a regular setup but would like to use
deduplication since it would save me a lot of space because my full
backups are 5 TB but most of the files are not changing between ful
backups.
-
       Device "Aligned-Disk" requested by DIR could not be opened or
does not exist.
11-Jul 09:29 jw-sd JobId 13: Error: [SE0003] Lookup of symbol
"BaculaSDdriver" in driver Aligned-Disk for device
/opt/bacula/plugins/bacula-sd-aligned-driver-9.4.4.so
<http://bacula-sd-aligned-driver-9.4.4.so> failed:
ERR=/opt/bacula/plugins/bacula-sd-aligned-driver-9.4.4.so
<http://bacula-sd-aligned-driver-9.4.4.so>: undefined symbol:
BaculaSDdriver
---
My OS is latest Centos 7. I installed Bacula from the Bacula 9.4.4
repo. It appeared to install the plugin successfuly. Yum shows it as
installed. It seems like Bacula is finding it since it lists the
correct path to the plugin.

I set up Bacula for deduplication using the directions at
http://www.bacula.lat/block-level-deduplication-with-aligned-volumes-tutorial-bacula-7-9-9-0-and-above/?lang=en.
See my device entry quoted further down below.

I ran a few programs on the plugin to show the symbols but they do not
show BaculaSDdriver in the plugin. I do not really know what I am
doing with that. I also looked at the plugin itself for text with this
value but did not see it.

I saw a post (wsee below) where someone has the same problem with
9.4.3. I could not figure out how to reply to that post since I just
joined the bacula-users list today.

Any idea how to fix this problem or if it is a bug?

Thanks,
Joe

*[Bacula-users] Aligned volumes support - Bacula Community 9.4.3
<https://sourceforge.net/p/bacula/mailman/message/36712185/>*
From: Pedro Freire Popiolek  - 2019-07-08 23:26:41

--
Bacula-Community]
name=CentOS - Bacula - Community
baseurl

Re: [Bacula-users] AWS S3 Backup

2019-07-18 Thread Kern Sibbald

  
  
Hello,
Please insure that the PluginDirectory is defined in the
  bacula-sd.conf file.  Then check and ensure that the plugin is in
  that directory.
Finally, the following commands could potentially help us:
stop the storage daemon
start it by hand in a shell window with -d20 as one of the
  options.

bconsole
status storage=
where  is the name of any storage device.
The status of the plugins should be sent to the stdout of the
  shell window where you restarted the storage daemon.
The output of the compile and link of the storage daemon would
  possibly be helpful.



Best regards,
Kern

On 7/17/19 5:40 PM, Bryan Sockel via
  Bacula-users wrote:


  
  Hey Guys,
   
  Trying to setup a new backup server to run backups to
an AWS S3 Bucket.  I was able build both required version of the
libs3 driver and bacula community install (9.4.4) and all
appears to be correct.  I need a little help setting up my
bacula-dir and bacula-sd files in order to use the backup.  I
think i have the Bacula-SD file configs correct, but beyond that
i am unsure of how to proceed.
   
   
  bacula-sd.conf 
   
  
Device {
    Name = CloudStorage
    Device Type = Cloud
    Cloud = S3Cloud
    Archive Device = /backups/CloudStorage
    Maximum Part Size = 1000
    Media Type = CloudType
    LabelMedia = yes
    Random Access = Yes;
    AutomaticMount = yes
    RemovableMedia = no
    AlwaysOpen = no
  }
Cloud {
    Name = S3Cloud
    Driver = "S3"
    HostName = "s3.amazonaws.com"
    BucketName = "s3-test"
    AccessKey = "AccessKey"
    SecretKey = "SecretKey"
    Protocol = HTTPS
    UriStyle = VirtualHost
    Truncate Cache = AfterUpload
    Upload = EachPart
    Region = "us-east-1"
    MaximumUploadBandwidth = 5MB/s
  }
   
  
   
  

  
Thank You
  Bryan
  
 

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Safely reusing tapes that are still labelled as Full

2019-07-15 Thread Kern Sibbald

Hello,

I set my volume retention time to one year, but currently I do not have 
enough tapes to keep them a year, which means that ultimately Bacula 
asks me to label a new tape, which I don't have.  So what I do from time 
to time is select oldest Volume that is marked Full (or sometimes Error) 
and I purge it.  For me (may not be your case) the risk that I lose some 
valid data is essentially zero because the other volumes have many Full 
backups on them.  In addition when I have needed to restore data it has 
always been to restore the most recent backups.


Obviously the best situation is to have sufficient backup space (either 
on disk or tape) to be able to keep backups for the period you specify 
as your Volume Retention time.


Best regards,

Kern

On 7/10/19 2:07 AM, Phil Stracchino wrote:

On 7/9/19 2:30 PM, William Muriithi wrote:

Hello

I stated using bacula about 6 months ago.  When I originally set it up, I just 
specified Volume retention period.  The File Retention period, and the Job 
Retention period periods are longer and hence most of the tapes aren't still 
labelled as ready for recycle.

Since I have almost used up all the tapes now, I want to re-use the older once 
but I am wondering what would be the safest way to ensure I can choose tapes 
that will never be needed should I require getting some data from the backups.?

Regards,
William


Honestly, if you have inconsistent retention times, you're largely just
going to have to figure out for yourself which volumes are safe for you
to purge and reuse.





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage

2019-07-05 Thread Kern Sibbald

Hello Martin,

I didn't know it exists, so I will take a look at it.  I imagine it will 
probably be a rather large rewrite of the existing Bacula code for AWS 
S3, but if it can fix all the problems that we have with libs3, it would 
probably be worth the effort.


Best regards,

Kern


On 7/1/19 4:29 PM, Martin Simmons wrote:

On Mon, 1 Jul 2019 13:11:07 +0200, Kern Sibbald said:

Do you see any other alternative for a C/C++ than to use libs3?

Have you already ruled out using the official SDK
(https://aws.amazon.com/sdk-for-cpp/)?

__Martin




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage

2019-07-01 Thread Kern Sibbald

  
  
Hello Sven,
Yes, this is a problem.  I remember we discussed libs3 some time
  ago, but I did not realize that it has a number of security
  issues.  I do know that it is no longer maintained by the original
  maintainer.  Bacula Systems has been updating the source code as
  changes are needed to make our cloud driver work.
Do you see any other alternative for a C/C++ than to use libs3?
For other S3 vendors, Bacula Systems has resorted in using the
  vendor supplied command line tools to access the their clouds. 
  This gets around the incompatibility of S3 implementations
  problems, but it opens new problems in that the vendors typically
  supply binaries, and if they have problems or bugs there is no way
  to fix them.  So far, I have no good solution to the problem.   I
  would be interested in any of your suggestions.  

In fact, if I thought there were enough Bacula users using AWS
  S3, I would even consider fixing and maintaining the libs3 package
  myself.  Of course that would be workable only if Debian and
  perhaps other vendors would adopt such a project.
Best regards,
Kern

On 6/30/19 5:17 PM, Sven Hartge wrote:


  On 30.06.19 16:57, Sven Hartge wrote:


  
We would need to ship the special libs3 either as embedded code inside
the Bacula source package or package it as a separate libs3-bacula
package. Both will get an instant veto from the Security Team and the
Release Managers. (Debian Policy §4.13)

  
  
The main problem here is that libs3 is already in Debian, but in the
ancient version 2.0 which doesn't even work with today's S3 anymore. (It
really should be removed from Debian, to be honest.)

After version 2.0, there have no new official releases been made by the
upstream developer and looking at the commit history on Github it seems
largely abandoned, aside from some merges PRs here and there.

When Bacula Systems implemented the Cloud driver in 2016.12.09 it used
the then git-HEAD of the library but it was still in flux back then, so
any version a bit older or a bit newer will not work with Bacula, so
even if Debian shipped a newer version, it would be incompatible with
Bacula.

Now, that special version from 2016.12.09 still contains some very
jarring problems including buffer overflows or memory leaks.

No distribution would want to support that code.

But we already went over this back when 9.4.0 was released and Carsten
and I tried to package the Cloud driver for Debian but gave up when it
dawned on us what an ordeal it would be to achieve this in the
constraints of Debian.

Grüße,
Sve.


  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] cloud_dev could not open Cloud driver type=1 for Device=CloudStorage

2019-06-30 Thread Kern Sibbald

  
  
Hello Sven,
The Bacula project provides the correct libs3 package at 

https://www.bacula.org/downloads/libs3-20181010.tar.gz

Normally when building, you download the .tar.gz file, detar it,
  build the package then make sure your packaging uses the special
  version rather than the one supplied on the system.
Best regards,
Kern



On 6/28/19 6:24 PM, Sven Hartge wrote:


  On 28.06.19 17:32, Antonino Balsamo wrote:


  
I'm running bacula 9.4.2 on stretch with debian packages from
stretch-backports. I've added the cloud plugin from the Bacula package
and I got the following error:

JobId 2: Fatal error: cloud_dev.c:690 Could not open Cloud driver type=1
for Device=CloudStorage

Then I removed all packages and installed 9.4.4 from Bacula archive
including cloud plugin, the error remains the same.

Going to the source code, it seems that HAVE_LIBS3 always evaluate to
false.

I do not think I have the talent to recompile the packages, but maybe
I'm doing a mistake somewhere else?

  
  
The Cloud driver is not compiled and supported in the Debian packages
because it needs a very special, very old version of libs3, not present
in Debian and unfortunately it is also not possible to get that version
into Debian at the current moment.

So I am afraid Debian will not be able to provide Cloud-support in
Bacula for the foreseeable future.

Grüße,
Sven
(Debian Bacula Co-Maintainer)


  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tape SL150 (Oracle). Drive Read-Only. Licence?

2019-06-26 Thread Kern Sibbald

  
  
Perhaps the write tab on the cassette
  is in the read-only position.  



On 6/26/19 2:11 AM, Jose Alberto wrote:


  
  Hi.


The 2 drive is LTO5.   The tape is LTO5. 


S/O     Debian 9 .


mtx -f  /dev/nst0  weof    (OK on Drive 1).
unload,   and load drive1

  mtx -f  /dev/nst1  weof    (Result=  read-only).
  Same tape 



btape and tapeinfo show  same configuration. 



 And  view "interesting"  config  on  WEB Tape.   is: 
    Drive0 is Bridge. and  Drive1 is not Bridge. 


from the rest I see no difference between the drives.
  unless at the hardware level it is already defective.













  
  
  
On Mon, Jun 24, 2019 at 10:53
      AM Kern Sibbald <k...@sibbald.com> wrote:


  
Hello,


You
  did not mention whether or not you are using Bacula.  For
  Bacula there is no license blockage.  Often writing of
  older LTO formats (e.g. writing LTO-3 on an LTO-5) drive
  is not supported by the drive manufacturer.  In your case,
  I suspect that the problem is in your OS where the drive
  device filename is not defined to have write permission.


chmod
  +w   might solve the problem.a


Best
  regards,
Kern



On
  6/20/19 8:02 PM, Jose Alberto wrote:


  Hi. I have an Oracle SL150 library of 2
Drive LTO5 and 96 Slot.

One of the 2 Slots does not write. weof show me result:
read-only

The other drive if you write without problems.

It is possible that at a physical level there is some
blockage by license to be able to use the 2 drives
completely.


Regars. 




-- 
#
  #   Sistema Operativo: Debian      #
  #        Caracas, Venezuela          #
  #
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  

  
  
  
  
  -- 
  #
#   Sistema Operativo: Debian      #
#        Caracas, Venezuela          #
#



  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Tape SL150 (Oracle). Drive Read-Only. Licence?

2019-06-24 Thread Kern Sibbald

  
  
Hello,


You did not mention whether or not you
  are using Bacula.  For Bacula there is no license blockage.  Often
  writing of older LTO formats (e.g. writing LTO-3 on an LTO-5)
  drive is not supported by the drive manufacturer.  In your case, I
  suspect that the problem is in your OS where the drive device
  filename is not defined to have write permission.


chmod +w   might solve the
  problem.a


Best regards,
Kern



On 6/20/19 8:02 PM, Jose Alberto wrote:


  
  Hi. I have an Oracle SL150 library of 2 Drive LTO5
and 96 Slot.

One of the 2 Slots does not write. weof show me result:
read-only

The other drive if you write without problems.

It is possible that at a physical level there is some blockage
by license to be able to use the 2 drives completely.


Regars. 




-- 
#
  #   Sistema Operativo: Debian      #
  #        Caracas, Venezuela          #
  #
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [ISSUE ] Not able to find Bacula rpm for 7.4.4-2

2019-06-24 Thread Kern Sibbald

  
  
Hello,


Bacula version 7.4.x is very old by our
  standards, and it is no longer supported (i.e. no bug fixes) by
  the project, though we do answer questions about it if we can.


Sorry but you must either use distro
  supplied binaries (maybe even older) or upgrade to the latest
  Bacula version if you want community binaries.



On 6/20/19 8:07 AM, Roshni Chatterjee
  wrote:


  
  
  
  
 
Hi Team ,
 
I require the baseurl/repo path ( required
  in /etc/yum.redpos.d/Bacula.repo) for Bacula version 7.4.4 .
The following site  
https://bacula.org/packages/rpms 
  contains only v9.0.0 to v9.4.3 . Previous versions are not
  available .
 
 
Any help would be  highly appreciated .
 
 
Regards,
Roshni
  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users




  


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 9.4.3/9.4.4 problem with full backups

2019-06-17 Thread Kern Sibbald

Hello Phil,

I vaguely remember seeing some code a programmer changed that eliminated 
the "old" kludge to allow Solaris to run multiple threads.  The comment 
I saw was that this was no longer needed on newer Solaris machines.  
Maybe the programmer didn't realize that there are still lots of old 
systems supported by the community.


I suggest that you check if anything has changed with Solaris thread 
concurrency.  See src/baconfig.h (notes about solaris).



Best regards,
Kern

On 6/3/19 4:40 PM, Phil Stracchino wrote:

I'm not sure exactly what is causing it, but ...

My schedule does nightly incrementals, weekly differentials, and Fulls
on the first Monday morning of every month.  For the second month in a
row - last month with 9.4.3, this month with 9.4.4 - at the first tape
volume switch in the monthly full backups, the SD has gone catatonic
trying to mount the second tape, and the Director has become
unresponsive and then crashed.  After restarting both and re-queueing
all of the monthly backups, they run successfully but slowly.

I don't know what's going on here.  A memory leak?  Nothing useful is
logged and I've been unable to come up with a way to usefully
troubleshoot the problem.  The only thing I can think of right now is to
restart the SD and Director via cron an hour or so before the monthly
full backup, but...   that idea grates on my nerves.  And I don't know
whether it would even help.

This did not, to my recollection, happen with 9.4.2.

Any suggestions?  Short of rolling back to 9.4.2?







___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Duplicate Backups with OneFS=no and the directory winding up in the list multiple times?

2019-06-17 Thread Kern Sibbald

Hello,

One other point about this. Semi-automatic detection of backing up files 
multiple times could be done with the estimate bconsole command turning 
on listing, then processing the output through sort and looking for 
duplicates.


Perhaps an interesting "feature" might be to add another option to 
estimate that would do exactly what I mentioned above then display all 
the duplicates.  Once a proper FileSet is made, there should really be 
no need to go to the extra expense of looking for duplicates.


Best regards,
Kern

On 6/5/19 12:46 AM, Larry Rosenman wrote:

On 06/04/2019 5:25 pm, Chandler wrote:

Larry Rosenman wrote on 6/4/19 14:56:
 Is there a way to have bacula see if it's backed up a path 
already in this run and not do that?


What you could try is to do a restore job, then pick "list jobs where
a given file is saved" and you could get that listing.

Then in the director backup settings, you can update the Fileset
resource so it doesn't have duplicates and add "Ignore Fileset Changes
= yes" to prevent another Full backup from running, but ideally you
would run a new Full backup after updating the Fileset resource.

HTH,


I've already made that change, and the new Full's are running/have run.

I was just wondering if it would make sense for Bacula to become 
smarter about this,

and be helpful and not duplicate the files.







___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Duplicate Backups with OneFS=no and the directory winding up in the list multiple times?

2019-06-17 Thread Kern Sibbald

Hello,

Yes, Bacula could become much smarter about this, but it requires two 
things:
1. Bacula will require *significantly* more resources (CPU and memory) 
to do such a backup.
2. You would need to find a programmer interested in writing the 
necessary code -- it is not a monster project, but it is not trivial 
either.  Much of the code for such a project already exists in the 
Accurate code of the FD.


Best regards,
Kern

On 6/5/19 12:46 AM, Larry Rosenman wrote:

On 06/04/2019 5:25 pm, Chandler wrote:

Larry Rosenman wrote on 6/4/19 14:56:
 Is there a way to have bacula see if it's backed up a path 
already in this run and not do that?


What you could try is to do a restore job, then pick "list jobs where
a given file is saved" and you could get that listing.

Then in the director backup settings, you can update the Fileset
resource so it doesn't have duplicates and add "Ignore Fileset Changes
= yes" to prevent another Full backup from running, but ideally you
would run a new Full backup after updating the Fileset resource.

HTH,


I've already made that change, and the new Full's are running/have run.

I was just wondering if it would make sense for Bacula to become 
smarter about this,

and be helpful and not duplicate the files.







___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bacula Release 9.4.4

2019-05-29 Thread Kern Sibbald
Hello,

We are pleased to announce that you can find Bacula Release 9.4.4 on
Source Forge and www.bacula.org.  The binary deb and rpm packages should
be ready in a week or two.

Release 9.4.4

This is a bug fix release to 9.4.3.  It includes some fixes that fix
bad data records in Copy/Migration jobs or problems doing restores
of Copy/Migration jobs.

28May19
 - rpm: Fix mysql dependency for bacula-postgresql
 - Fix bug 2476 -- copy/migration jobs fail when waiting for a new Volume
 - Add copy-jobspan-label-wait-test to do_all
 - Add two new regression tests submitted by Martin Simmons for bug 2476
 - Remove bacula32.def and bacula64.def.
 - Add Michael Narigon as author for Mac heap implementation
 - Add tests/restart-jobmedia-test to do_all
 - Allow to hangup/blowup inside a file for tests
   slash - fix suggested by vondi
 - Add db_get_jobmedia_record() function
 - Check JobMedia validity after an incomplete job
 - baculum: Fix #2477 escaping backslashes in config in text directive types
 - baculum: Fix #2474 error 404 if document root path uses link with ending
 - baculum: Remove php database extensions dependency from web requirements
 - baculum: Fix removing path items selected to restore in restore wizard
 - baculum: Fix clearing restore path field after selecting backup in
restore
    wizard
 - baculum: Fix TPhpFatalErrorException exception on job view page with PHP
    version lower than 5.5

Bugs fixed/closed since last release:
2474 2476 2477


Thanks for the continuing community input and also for those who use Bacula,
Kern



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] The Cloud Parts do not match! (bacula-cloud-storage-9.4.2)

2019-05-26 Thread Kern Sibbald

  
  
Hello Mike,

Sorry, but I am 100% sure that Oracle cloud is not compatible with
Amazon S3.  Bacula was explicitly tested on Oracle (not by me) and
it doesn't work.  It requires a special Oracle driver that the
community does not have at this time.

Best regards,
Kern

On 5/26/19 12:08 PM, Mike wrote:


  
  
  
  Morning Kern,
  
  
  
  Thanks for the quick response. 
  
  
  
  1) Oracle Cloud
  
  2) I have not tried a restore but I repeatedly see this so
have doubt in the consistency of the backups. I have verified
the sd service is not restarting during the upload.  I only copy
backups from local stores to remote stores once a week but see
it every time. This has happened repeatedly over the past three
weeks.
  
  It feels like a timing issue.
  
  
  
  I'm happy to enable any debugging you suggest, as you imply
the files are being uploaded but the driver does not appear to
recognize all uploads (hence the low VolParts).
  
  
  
  Appreciate the help,
  
  Mike
  
  
  On Sun, May 26, 2019, at 19:37, Kern Sibbald wrote:
  
  
Hello,

 

 I have two questions:

 

 1. What cloud server are you using.  We support Amazon or
  100% Amazon compatible.  All S3 implementations are not the
  same or compatible with Amazon S3.

 

 2. Have you tried doing a restore?  Most of those errors
  look like they *may* be harmless in that

 they indicate that the Volume has more parts than what the
  Catalog reports.  This can happen when

 some of the parts are not uploaded to the cloud (i.e. they
  are still in the cache).  It could occur if you shutdown the
  SD while uploads are still in progress.

 

 Best regards,

 Kern

 

 

On 5/26/19 3:39 AM, Mike wrote:


  Morning Folks,
  
  
  
  I was keen to use the new cloud storage feature so I've
configured things up to use it as an offsite repository of
some backup jobs.
  
  I use the 'Next Pool' Pool parameter and then a Copy type
job on PoolUncopiedJobs
  
  
  
  I start a job and it seems to run OK but I'm seeing
dozens of errors similar to the following and the data from
'list volumes' is very different to what is actually present
on the S3 volume.
  
  
  
  ---
  
  10-May 07:06 ghost-sd JobId 12751: Warning:
cloud_dev.c:1748 For Volume "Offsite-0183":
  
  The Parts do not match! Metadata Volume=16915
Catalog=16889.
  
  Correcting Catalog
  
  ---
  
  Sometimes a single job will get multiple
  
  ---
  
  07-May 11:04 ghost-sd JobId 12608: Warning:
cloud_dev.c:1748 For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=6985
Catalog=6711.
  
  The Cloud Parts do not match! Metadata Volume=1817
Catalog=1574.
  
  
  
  Correcting Catalog
  
  07-May 11:47 ghost-sd JobId 12608: Warning:
cloud_dev.c:1748 For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=10080
Catalog=9986.
  
  The Cloud Parts do not match! Metadata Volume=1953
Catalog=1574.
  
  
  
  Correcting Catalog
  
  07-May 11:54 ghost-sd JobId 12608: Warning:
cloud_dev.c:1748 For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=10521
Catalog=10480.
  
  The Cloud Parts do not match! Metadata Volume=1976
Catalog=1574.
  
  
  
  Correcting Catalog
  
  
  
  

  
  

Director: 9.4.2 on Solaris

Storage: 9.4.2 on Solaris (disk based), 9.4.2 on Linux
  (cloud based)

Clients: 9.4.2 on Solaris, Windows and Linux


  
  

  
  Linux sd (ghost) is using Bacula community RPMs
  
  bacula-libs-9.4.2-1.el7.x86_64
  
  bacula-cloud-storage-9.4.2-1.el7.x86_64
  
 

Re: [Bacula-users] The Cloud Parts do not match! (bacula-cloud-storage-9.4.2)

2019-05-26 Thread Kern Sibbald

  
  
Hello,

I have two questions:

1. What cloud server are you using.  We support Amazon or 100%
Amazon compatible.  All S3 implementations are not the same or
compatible with Amazon S3.

2. Have you tried doing a restore?  Most of those errors look like
they *may* be harmless in that
they indicate that the Volume has more parts than what the Catalog
reports.  This can happen when
some of the parts are not uploaded to the cloud (i.e. they are still
in the cache).  It could occur if you shutdown the SD while uploads
are still in progress.

Best regards,
Kern

On 5/26/19 3:39 AM, Mike wrote:


  
  
  
  Morning Folks,
  
  
  
  I was keen to use the new cloud storage feature so I've
configured things up to use it as an offsite repository of some
backup jobs.
  
  I use the 'Next Pool' Pool parameter and then a Copy type job
on PoolUncopiedJobs
  
  
  
  I start a job and it seems to run OK but I'm seeing dozens of
errors similar to the following and the data from 'list volumes'
is very different to what is actually present on the S3 volume.
  
  
  
  ---
  
  10-May 07:06 ghost-sd JobId 12751: Warning: cloud_dev.c:1748
For Volume "Offsite-0183":
  
  The Parts do not match! Metadata Volume=16915 Catalog=16889.
  
  Correcting Catalog
  
  ---
  
  Sometimes a single job will get multiple
  
  ---
  
  07-May 11:04 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=6985 Catalog=6711.
  
  The Cloud Parts do not match! Metadata Volume=1817
Catalog=1574.
  
  
  
  Correcting Catalog
  
  07-May 11:47 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=10080 Catalog=9986.
  
  The Cloud Parts do not match! Metadata Volume=1953
Catalog=1574.
  
  
  
  Correcting Catalog
  
  07-May 11:54 ghost-sd JobId 12608: Warning: cloud_dev.c:1748
For Volume "Offsite-0182":
  
  The Parts do not match! Metadata Volume=10521 Catalog=10480.
  
  The Cloud Parts do not match! Metadata Volume=1976
Catalog=1574.
  
  
  
  Correcting Catalog
  
  
  
  

  
  

Director: 9.4.2 on Solaris

Storage: 9.4.2 on Solaris (disk based), 9.4.2 on Linux
  (cloud based)

Clients: 9.4.2 on Solaris, Windows and Linux


  
  

  
  Linux sd (ghost) is using Bacula community RPMs
  
  bacula-libs-9.4.2-1.el7.x86_64
  
  bacula-cloud-storage-9.4.2-1.el7.x86_64
  
  bacula-mysql-9.4.2-1.el7.x86_64
  
  
  
  I expect it's a bug in the new plugin but thought I'd see if
anyone had suggestions on how to proceed: it could be my
misconfiguration.
  
  Note I do have Truncate Cache = AfterUpload since I already
have a local copy of the job and restoring from cloud would be
last resort.
  
  Thanks for the help
  
  
  
  * list volumes
  
  
  
  Pool: Offsite
  
  +-+--+---+-+---+--+--+-+--+---+---+-+--+-++
  
  | MediaId | VolumeName   | VolStatus | Enabled |
VolBytes  | VolFiles | VolRetention | Recycle | Slot |
InChanger | MediaType | VolType | VolParts | LastWritten
| ExpiresIn  |
  
  +-+--+---+-+---+--+--+-+--+---+---+-+--+-++
  
  | 181 | Offsite-1    | Full  |   1 |  
262,087,761 |    0 |   15,552,000 |   1 |    0 |
0 | CloudType |  14 |    2,866 | 2019-05-05 11:43:59 |
15,011,184 |
  
  | 182 | Offsite-0182 | Full  |   1 |
1,073,696,800 |    0 |   15,552,000 |   1 |    0
| 0 | CloudType |  14 |   13,922 | 2019-05-07
12:44:02 | 15,187,587 |
  
  | 183 | Offsite-0183 | Append    |   1 |  
630,979,806 |    0 |   15,552,000 |   1 |    0 |
0 | CloudType |  14 |   17,827 | 2019-05-11 16:59:43 |
15,548,528 |
  
  | 184 | Offsite-0184 | Append    |   1 |
0 |    0 |   15,552,000 |   1 |    0 | 0 |
 

Re: [Bacula-users] Bacula director error messages

2019-05-10 Thread Kern Sibbald
Hello Martin,

This is an interesting reflection.  Do you think it is a timeout, or an
out and out bug where Bacula gets confused with additional
communications?   A bug would be a bit hard to understand, because the
SD often waits for a Volume to be mounted -- of course, there can
certainly be bug for a copy job where two separate Volumes are involved.

Do you have any way to easily reproduce this?

Best regards,
Kern

On 5/10/19 1:25 PM, Martin Simmons wrote:
> I'm pretty sure the "Connection reset by peer" error is a Bacula bug,
> triggered when a Copy job waits in the middle for a new tape to write.
>
> This is causing the Copying Error status.
>
> __Martin
>
>
>
>>>>>> On Fri, 10 May 2019 00:19:54 +0200, Andras Horvai said:
>> hi,
>>
>> anybody, any idea regarding this error? Why the termination status of the
>> previous job was: *** Copying Error *** ?
>>
>> Thanks,
>>
>> Andras
>>
>> On Wed, May 8, 2019 at 4:11 PM Andras Horvai 
>> wrote:
>>
>>> you got the point, here it is another error message:
>>>
>>> 06-May 12:01 backup2-dir JobId 1038: Using Device "LTO-6" to write.
>>> 06-May 12:07 backup2-sd JobId 1038: [SI0202] End of Volume "WORMW-1181" at
>>> 1424:28456 on device "LTO-6" (/dev/nst0). Write of 64512 bytes got -1.
>>> 06-May 12:07 backup2-sd JobId 1038: Re-read of last block succeeded.
>>> 06-May 12:07 backup2-sd JobId 1038: End of medium on Volume "WORMW-1181"
>>> Bytes=2,820,363,420,672 Blocks=43,718,430 at 06-May-2019 12:07.
>>> 06-May 12:08 backup2-dir JobId 1038: Created new Volume="WORMW-1182",
>>> Pool="TapeArchive", MediaType="LTO-6" in catalog.
>>> 06-May 12:08 backup2-sd JobId 1038: Please mount append Volume
>>> "WORMW-1182" or label a new one for:
>>> Job:  srv1-job.2019-05-06_09.00.01_21
>>> Storage:  "LTO-6" (/dev/nst0)
>>> Pool: TapeArchive
>>> Media type:   LTO-6
>>> 06-May 12:46 backup2-sd JobId 1038: Error: [SE0203] The Volume=WORMW-1182
>>> on device="LTO-6" (/dev/nst0) appears to be unlabeled.
>>> 06-May 12:47 backup2-sd JobId 1038: Labeled new Volume "WORMW-1182" on
>>> Tape device "LTO-6" (/dev/nst0).
>>> 06-May 12:47 backup2-sd JobId 1038: Wrote label to prelabeled Volume
>>> "WORMW-1182" on Tape device "LTO-6" (/dev/nst0)
>>> 06-May 12:47 backup2-sd JobId 1038: New volume "WORMW-1182" mounted on
>>> device "LTO-6" (/dev/nst0) at 06-May-2019 12:47.
>>> 06-May 12:56 backup2-sd JobId 1038: Fatal error: append.c:170 Error
>>> reading data header from FD. n=-2 msglen=0 ERR=Connection reset by peer
>>> 06-May 12:56 backup2-sd JobId 1038: Elapsed time=00:14:48, Transfer
>>> rate=68.06 M Bytes/second
>>> 06-May 12:56 backup2-sd JobId 1038: Sending spooled attrs to the Director.
>>> Despooling 27,981,780 bytes ...
>>>
>>> so why I got Connection reset by peer message? SD,FD,Director is on the
>>> same machine (in the case of Copy jobs)
>>>
>>> Thanks,
>>> Andras
>>>
>>> On Wed, May 8, 2019 at 3:10 PM Martin Simmons 
>>> wrote:
>>>
>>>> That look clean.
>>>>
>>>> Are there any messages for the "New Backup JobId" (1038)?  I find them
>>>> printed
>>>> after the "Termination:" line for the copy job.
>>>>
>>>> __Martin
>>>>
>>>>
>>>>>>>>> On Wed, 8 May 2019 14:32:31 +0200, Andras Horvai said:
>>>>> hi,
>>>>>
>>>>> here is the snipped part: :)
>>>>>
>>>>> 06-May 09:00 backup2-dir JobId 1037: Copying using JobId=1016
>>>>> Job=srv1-job.2019-05-04_02.00.00_59
>>>>> 06-May 12:01 backup2-dir JobId 1037: Start Copying JobId 1037,
>>>>> Job=ArchiveJob.2019-05-06_09.00.01_20
>>>>> 06-May 12:01 backup2-dir JobId 1037: Using Device "FileStorage" to read.
>>>>> 06-May 12:01 backup2-sd JobId 1037: Ready to read from volume
>>>> "FILEW-1006"
>>>>> on File device "FileStorage" (/backup).
>>>>> 06-May 12:01 backup2-sd JobId 1037: Forward spacing Volume "FILEW-1006"
>>>> to
>>>>> addr=531212699
>>>>> 06-May 12:01 backup2-sd JobId 1037: End of Volume "FILEW-1006" at
>>>>> 

Re: [Bacula-users] Virtual Full storage deadlock (was: Re: Virtual Full backups: Getting "No previous jobs found")

2019-05-10 Thread Kern Sibbald

  
  
Hello,

If you are doing any kind of "copy" of data (Migration, Copy,
VirtualFull, ...), it seems to me to be obvious, but perhaps I am
mistaken, you need two different Storage daemon device definitions
-- one to read a Volume, and one to write to a different Volume.  It
appears (I don't have enough information here) that this is not your
case.

Best regards,
Kern

On 5/10/19 2:58 AM, David Brodbeck
  wrote:


  
  

  

  
Still trying to get Progressive Virtual
  Full backups to work.


I solved the "no previous jobs found"
  problem by making only one job definition (instead of
  separate incremental and virtualfull definitions), and
  changing the job level to VirtualFull in the schedule
  for the times when I want to consolidate. It now
  correctly locates the jobs to consolidate. However,
  I'm getting deadlock when it tries to access storage,
  probably because I have Pool and Next Pool set to the
  same location.
  
  
  The documentation (https://www.bacula.org/9.4.x-manuals/en/main/Migration_Copy.html)
states that it should work:
  "Alternatively, you can set your Next Pool to
point to the current pool. This will cause Bacula to
read and write to Volumes in the current pool. In
general, this will work, because Bacula will not
allow reading and writing on the same Volume."
  
  
  This is what I'm trying to do, but it doesn't
work; the job stalls with "waiting on Storage." I
assume this is because my file pool only has one
device, so Bacula assumes it can't both read from
and write to it at the same time. I've found lots of
old (v5.x era) references to using vchanger to solve
this kind of problem, but I'm unsure if that's still
the best way to go. The current documentation is a
bit fragmentary on this and I'm hoping someone can
point me in the right direction.
  
  
  Here's the relevant configuration stanzas for my
storage:
  
  
  bacula-dir.conf:
  
Storage {
  Name = russell.math.ucsb.edu-sd
  Address = russell.math.ucsb.edu

  SDPort = 9103
  Password = 
  Device = DataCenter
  Media Type = DCFile
  Maximum Concurrent Jobs = 10
}
  
  Pool {
  
  
  Name = DataCenterPool
  Pool Type = Backup
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 60 days
  Maximum Volume Bytes = 50G 
  Maximum Volumes = 280

  Label Format = "DCVol-"
  Storage = russell.math.ucsb.edu-sd
}
  
  
  
  bacula-sd.conf:
  
Device {
  Name = DataCenter
  Device Type = File
  Media Type = DCFile
  Archive Device = /media/bacula/DataCenterPool

  LabelMedia = yes;
  Random Access = Yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
  Maximum Concurrent Jobs = 10
}
  
  
  
  
  

  
  -- 
  
David Brodbeck
  System Administrator, Department of Mathematics
University of California, Santa Barbara


  

  

  

  
  
  
  
  
  ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https:/

Re: [Bacula-users] [Bacula-devel] Bacula Community Release 9.4.3

2019-05-04 Thread Kern Sibbald

  
  
Hello Wanderlei,

Pushing the tags are always the last thing I do -- I guess in the
future, I should make 
the announcement the last thing I do.  

It seems to me that you checked just a few minutes prior to when I
got everything finalized.

Best regards,
Kern

On 5/3/19 6:24 PM, Wanderlei Huttel
  wrote:


  
  
Hello Kern
  
  I guess it was forgotten to include the release tag 9.4.3 in
  the git repository.
  

  

  

  

  
Best regards


Wanderlei Hüttel
http://www.bacula.com.br

  

  

  

  


  

  
  
  
Em sex, 3 de mai de 2019 às
  08:12, Kern Sibbald <k...@sibbald.com> escreveu:

Hello,
  
  We are pleased to announce Bacula Community Release version
  9.4.3.  It
  has been released to Source Forge and www.bacula.org. 
  The binary package
  will be build shortly and released as usual to the Binary
  Package Download
  area of www.bacula.org.
  
  This is a bug fix release for version 9.4.2.  It includes a
  number of bug
  fixes and patches.
  
  Baculum: there have been significant additions and changes to
  Baculum.
  If you want a web gui please check it out.
  
  S3 driver: If you are trying to build the S3 drivers, please
  remember to
  use the
  community supplied (from Bacula Enterprise) version of
  libs3.so found at:
  
  https://www.bacula.org/downloads/libs3-20181010.tar.gz
  
  As usual the binaries that correspond to this release will
  follow in
  a week or two.
  
  If there are no additional major bugs, this will be the last
  of the 9.4.x
  releases.  The next release will have a number of new
  features, and will
  require a major database upgrade (don't worry it will be easy
  -- just
  run update_bacula_tables)
  
  02May19
   - Fix Window bpipe-fd strncpy programming error
   - Change mysql my_bool to bool as it was removed from mysql
   - Improve assert message
   - examples: move backup-to-cdwriter.txt to
  move-backup-to-usb.txt
   - fix memory leak in DIR for copy-job
   - Skip empty lines when generating the FileSet from a command
   - Fix creation of bad JobMedia records in Incomplete Job
   - Add messages for Incomplete Jobs
   - Fix misplaced cancel check reported by Alain
   - Change round() to bround() to avoid library definition
  conflict
   - rpms: Fix bacula-cloud spec file
   - rpms: Add bacula-cloud spec file
   - rpms: Add missing isworm script
   - Use more appropriate computation for VolIndex when creating
  restore .bsr
   - Fix Daemon message "Message repeated X times" count
   - Fix Carsten's names
   - Make diff.pl adapt to
  different install locations
   - Fix #4598 Display JobIds used in the restore job log
   - Add smartalloc function to print the owner of a buffer
   - Avoid to use the same variable name for two different
  things in the
  cmd_parser
     class
   - Fix #4433 about 'UPDATE File SET MD5='...' WHERE FileId=0'
  error when
  using
     SpoolAttributes=no
   - Fix MaxVolumeBytes accounting after a mount request
   - Fix verify volume jobs with sparse files
   - Fix small memory leak with Console runscripts
   - Add 'prune jobs/files all' command
   - fix #4383 Sometime SD hangs when TLS and DEDUP are used
  together
  
   - baculum: Add status schedule page
   - baculum: Add status schedule API endpoint
   - baculum: Fix schedule directives setting
   - baculum: Update Portuguese translation file
   - baculum: Fix #2466 add plugin directive support in fileset
  resource
   - baculum: Fix updating whole Bacula config at once
   - baculum: Fix showing validation error if new config is
  incorrect
   - baculum: Fix setting multiple config resources at once
   - baculum: Fix renaming resources

[Bacula-users] Bacula Community Release 9.4.3

2019-05-03 Thread Kern Sibbald
Hello,

We are pleased to announce Bacula Community Release version 9.4.3.  It
has been released to Source Forge and www.bacula.org.  The binary package
will be build shortly and released as usual to the Binary Package Download
area of www.bacula.org.

This is a bug fix release for version 9.4.2.  It includes a number of bug
fixes and patches.

Baculum: there have been significant additions and changes to Baculum.
If you want a web gui please check it out.

S3 driver: If you are trying to build the S3 drivers, please remember to
use the
community supplied (from Bacula Enterprise) version of libs3.so found at:

https://www.bacula.org/downloads/libs3-20181010.tar.gz

As usual the binaries that correspond to this release will follow in
a week or two.

If there are no additional major bugs, this will be the last of the 9.4.x
releases.  The next release will have a number of new features, and will
require a major database upgrade (don't worry it will be easy -- just
run update_bacula_tables)

02May19
 - Fix Window bpipe-fd strncpy programming error
 - Change mysql my_bool to bool as it was removed from mysql
 - Improve assert message
 - examples: move backup-to-cdwriter.txt to move-backup-to-usb.txt
 - fix memory leak in DIR for copy-job
 - Skip empty lines when generating the FileSet from a command
 - Fix creation of bad JobMedia records in Incomplete Job
 - Add messages for Incomplete Jobs
 - Fix misplaced cancel check reported by Alain
 - Change round() to bround() to avoid library definition conflict
 - rpms: Fix bacula-cloud spec file
 - rpms: Add bacula-cloud spec file
 - rpms: Add missing isworm script
 - Use more appropriate computation for VolIndex when creating restore .bsr
 - Fix Daemon message "Message repeated X times" count
 - Fix Carsten's names
 - Make diff.pl adapt to different install locations
 - Fix #4598 Display JobIds used in the restore job log
 - Add smartalloc function to print the owner of a buffer
 - Avoid to use the same variable name for two different things in the
cmd_parser
   class
 - Fix #4433 about 'UPDATE File SET MD5='...' WHERE FileId=0' error when
using
   SpoolAttributes=no
 - Fix MaxVolumeBytes accounting after a mount request
 - Fix verify volume jobs with sparse files
 - Fix small memory leak with Console runscripts
 - Add 'prune jobs/files all' command
 - fix #4383 Sometime SD hangs when TLS and DEDUP are used together

 - baculum: Add status schedule page
 - baculum: Add status schedule API endpoint
 - baculum: Fix schedule directives setting
 - baculum: Update Portuguese translation file
 - baculum: Fix #2466 add plugin directive support in fileset resource
 - baculum: Fix updating whole Bacula config at once
 - baculum: Fix showing validation error if new config is incorrect
 - baculum: Fix setting multiple config resources at once
 - baculum: Fix renaming resources
 - baculum: Add links to resources in job log output
 - baculum: Update example web server config files and spec file
 - baculum: Fix list type directives on configure hosts page
 - baculum: Fix showing messages resource configuration
 - baculum: Add parent node property to directive list types
 - baculum: Add capability to define multiple drivetype and fstype
directives in
   fileset resource
 - baculum: Fix showing runscript subresource on job pages
 - baculum: Update new texts in Portuguese translation file
 - baculum: Add missing texts to translation files reported by Wanderlei
Huttel
 - baculum: Update Portuguese translations
 - baculum: Fix showing schedule resource configuration on job view and job
   history view pages
 - baculum: Fix returning one line output from bconsole
 - baculum: Fix restore wizard error when no fileset available for
normal user
   with limited access
 - baculum: Add capability to use many ACL Console directives in one config
   resource
 - baculum: Add client ls command to openapi file
 - baculum: Add text box list control to support directives that can be
defined
   multiple times in one resource
 - baculum: Add cancel button to last step new job wizard
 - baculum: Change PoolType field from text box into combo box
 - baculum: Fix adding new schedule run directives
 - baculum: Add support to multiple schedule run directives
 - baculum: Fix in API part saving job runscript config if RunsWhen=Always
 - baculum: Fix adding paths to empty include block
 - baculum: Fix saving job runscript config if RunsWhen=Always
 - baculum: New create backup job wizard
 - baculum: Add API endpoint to list files/dirs on client


Bugs fixed/closed since last release:
2466 4383 4433 4598

Thank you for using Bacula.  Enjoy ...

Best regards,
Kern



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Big job keeps failing after server replacement

2019-05-03 Thread Kern Sibbald
Hello,

Note: for Windows, the FD can probably backup the files, but it may not
be backing up everything so you might have
restore problems later.  There have been a *lot* of fixes for the
Windows FD since version 5.2.x

It seems that on 6 April, you were running the SD as bacula:bacula, and
now you are running it as bacula:tape.
That could be the source of your problem.

On 5/2/19 5:14 PM, Gary Stainburn wrote:
> Hi Kern,
>
> Thank you for your response.
>
> On Thursday 02 May 2019 14:23:54 Kern Sibbald wrote:
>> Hello,
>>
>> Sorry you are having problems.  I note the following things:
>>
>> 1. You are running on a *very* old Bacula version.
> I am aware of this, but it is the latest version that installs from the 
> Centos repositories, and more importantly is only a few revisions higher than 
> was on the old (dead) box ensuring that the database format etc would be 
> compatible.
>
>> 2. I am not sure that version of Bacula supports Windows 7, where you
>> are getting failures.
> This version has successfully worked on Win-Xp, Win7, Win8 and Win10.  I have 
> approc 150 workstations all with one of these versions. I also have various 
> ages of Linux server which all also work
>
>> 3. As for the errors, it looks like the SD does not have permission to
>> open /var/bacula/crownest
> When this was first set up I had forgotten to set the owner:group on the 
> directory. I did fix this, and the files that are now in that directory have 
> the correct owner:group, and were written by the SD (see OP). 
>
> I am aware that this is what the errors are saying, but that does not explain 
> how the files are still being created.
>
>
>> 4. Is /var/bacula/crownnest a network mount (this could explain the
>> failures).
> This is a file  storage device configured in the SD on the same box as the 
> Director.  The directory itself is local to that box
>
>> 5. It looks like it is taking a bit over 2 hours for the SD to mount the
>> volume for the backup, and
>>     this is approximately the network inactivity timeout period.  So
>> possibly the FD<->SD connection
>>     is timing out
> The client device and the storage device are on different sites, with a 30GB 
> connection, doing a 18GB full backup. This would explain the 2 hour run time.
>
>> 6. Try adding HeartBeatInterval = 300 to the Dir, FD, and SD.  I think
>> there are 5 places where it must
>>     be done (note this is the default in more recent Baculas).
> I will investigate where this needs to go and will apply it.
>
>> Recommendations:
>> 1. Make sure the SD either has full permissions on all disk files or
>> runs as root.
> The SD runs as bacula:tape which is the default configuration from the RPM's 
> and matches the old box.
>
> As you can see from the directory structure, this should be correct.
> [root@lou bacula]# ls -ld / /var/ /var/bacula/ /var/bacula/crownest/ 
> /var/bacula/crownest/* /var/bacula/hales/ /var/bacula/hales/*
> dr-xr-xr-x. 18 root   root  281 Apr  5 11:13 /
> drwxr-xr-x. 27 root   root 4096 Mar  8 13:59 /var/
> drwxr-xr-x. 14 bacula bacula   8192 Apr 30 11:28 /var/bacula/
> drwxr-xr-x.  2 bacula bacula   4096 May  2 15:43 /var/bacula/crownest/
> -rw-r-.  1 bacula tape   5368688828 Apr 30 17:20 
> /var/bacula/crownest/crownest72930
> -rw-r-.  1 bacula tape   5368688851 Apr 30 17:21 
> /var/bacula/crownest/crownest72931
> -rw-r-.  1 bacula tape   5368688789 Apr 30 17:21 
> /var/bacula/crownest/crownest72932
> [snip]
> -rw-r-.  1 bacula tape   1745617753 May  2 15:43 
> /var/bacula/crownest/crownest72965
> drwxr-xr-x.  3 bacula bacula  12288 May  1 23:41 /var/bacula/hales/
> -rw-r-.  1 bacula bacula 5368705600 Apr  6 03:29 
> /var/bacula/hales/hales68076
> -rw-r-.  1 bacula bacula 5368688843 Apr  6 03:29 
> /var/bacula/hales/hales68078
> -rw-r-.  1 bacula bacula 5368670610 Apr  6 03:51 
> /var/bacula/hales/hales68082
>
> (Hales is the storage device that was used previously)
>
>> 2. Make sure any network mounts (NFS, CIFS) are mounted prior to running
>> a job
> I do not use network mounts
>
>> 3. Add the Heart Beat Interval = 300 in all the required resources --
>> this may be documented  in one of the white papers, but is surely documented 
>> in the manual,
>> look in the index ...
> I have added this line to the Director, client and storage resources. I 
> cannot see where else it wuold be needed.
>
>> 4. When you get it running think seriously about upgrading.  Though
>> RedHat releases an older version
>>    of Bacula the builders do create packages for newer version -- or
>> look on www.bacula.org for binaries.
> I am aware that I am running an old version. Is there a documented upgrade 
> path?  I believe the Centos 8 is imminent, and would prefer to stick to 
> standard RPM's where possible
>



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Big job keeps failing after server replacement

2019-05-02 Thread Kern Sibbald
Hello,

Sorry you are having problems.  I note the following things:

1. You are running on a *very* old Bacula version.
2. I am not sure that version of Bacula supports Windows 7, where you
are getting failures.
3. As for the errors, it looks like the SD does not have permission to
open /var/bacula/crownest
4. Is /var/bacula/crownnest a network mount (this could explain the
failures).
5. It looks like it is taking a bit over 2 hours for the SD to mount the
volume for the backup, and
    this is approximately the network inactivity timeout period.  So
possibly the FD<->SD connection
    is timing out
6. Try adding HeartBeatInterval = 300 to the Dir, FD, and SD.  I think
there are 5 places where it must
    be done (note this is the default in more recent Baculas).

Recommendations:
1. Make sure the SD either has full permissions on all disk files or
runs as root.
2. Make sure any network mounts (NFS, CIFS) are mounted prior to running
a job
3. Add the Heart Beat Interval = 300 in all the required resources --
this may be documented
    in one of the white papers, but is surely documented in the manual,
look in the index ...
4. When you get it running think seriously about upgrading.  Though
RedHat releases an older version
   of Bacula the builders do create packages for newer version -- or
look on www.bacula.org for binaries.

Best regards,
Kern

On 5/2/19 1:14 PM, Gary Stainburn wrote:
> A few months back, the server running my Director and main storage died.  I 
> managed to boot using a live CD and successfully copied everything onto a new 
> Centos 7 box.  I restored the latest database backup, copied the config files 
> and rsynced the storage.
>
> Amazingly all just continued to work. More amazing seeing as the failed box 
> was a Fedora 19 setup and wa quite old.
>
> The only problem is I have a small number of large jobs that are constantly 
> failing.  Everything appears to work fine, and the client status shows the 
> job as completed OK. See below.
>
> However, the job itself fails, and gets rescheduled. In the case I'm looking 
> at it keeps running a 18GB backup.
>
> In order to try to narrow down the problem, as well as aid job scheduling I 
> have set up a new storage device and pointed this job at that, but it has not 
> made any difference.  Looking at the log file, I see two things
>
> Firstly, error messages about accessing the new storage folder and backup 
> volumes. This is odd as the backup is being successfully written to this 
> device. See below.
>
> Secondly, the job appears to be trying to connect to the FD once the job is 
> complete and is then failing.  I don't konw why this is the case.
>
> Does anyone have any suggestions of what I can do to fix this?
>
> The full job log is available at https://www1.ringways.co.uk/bacula.log
>
> I'm running standard Centos 7 RPM's.  
>
> bacula-console-bat-5.2.13-23.1.el7.x86_64
> bacula-libs-5.2.13-23.1.el7.x86_64
> bacula-console-5.2.13-23.1.el7.x86_64
> bacula-common-5.2.13-23.1.el7.x86_64
> bacula-storage-5.2.13-23.1.el7.x86_64
> bacula-libs-sql-5.2.13-23.1.el7.x86_64
> bacula-client-5.2.13-23.1.el7.x86_64
> bacula-director-5.2.13-23.1.el7.x86_64
> postgresql-docs-9.2.24-1.el7_5.x86_64
> postgresql-upgrade-9.2.24-1.el7_5.x86_64
> postgresql-contrib-9.2.24-1.el7_5.x86_64
> postgresql-devel-9.2.24-1.el7_5.x86_64
> postgresql-9.2.24-1.el7_5.x86_64
> postgresql-server-9.2.24-1.el7_5.x86_64
> postgresql-plperl-9.2.24-1.el7_5.x86_64
> postgresql-libs-9.2.24-1.el7_5.x86_64
>
>
> storage direcortory
> ***
> [root@lou bacula]# ls -ld crownest/ crownest/*
> drwxr-xr-x. 2 bacula bacula   4096 May  2 10:38 crownest/
> -rw-r-. 1 bacula tape   5368688828 Apr 30 17:20 crownest/crownest72930
> -rw-r-. 1 bacula tape   5368688851 Apr 30 17:21 crownest/crownest72931
> -rw-r-. 1 bacula tape   5368688789 Apr 30 17:21 crownest/crownest72932
> -rw-r-. 1 bacula tape   5368669086 Apr 30 21:03 crownest/crownest72933
> -rw-r-. 1 bacula tape   5368676895 Apr 30 22:59 crownest/crownest72934
> -rw-r-. 1 bacula tape   5368688828 Apr 30 23:00 crownest/crownest72935
> -rw-r-. 1 bacula tape   5368688851 Apr 30 23:01 crownest/crownest72936
> -rw-r-. 1 bacula tape   5368688795 Apr 30 23:01 crownest/crownest72937
> -rw-r-. 1 bacula tape   5368696610 May  1 04:24 crownest/crownest72938
> -rw-r-. 1 bacula tape   5368688851 May  1 04:25 crownest/crownest72939
> -rw-r-. 1 bacula tape   5368688851 May  1 04:26 crownest/crownest72940
> -rw-r-. 1 bacula tape   5368696566 May  1 14:03 crownest/crownest72941
> -rw-r-. 1 bacula tape   5368688838 May  1 14:04 crownest/crownest72942
> -rw-r-. 1 bacula tape   5368688851 May  1 14:04 crownest/crownest72943
> -rw-r-. 1 bacula tape   5368688801

  1   2   3   4   5   6   7   8   9   10   >