[Bacula-users] Bacula 13.0.0 now in the FreeBSD ports tree

2022-07-09 Thread Dan Langille

Hello,

Bacula 13 is now in the FreeBSD ports tree. Please proceed with caution. 
I have not tested it to see how stable it is.


It built cleanly based on the Bacula 11 port files, but I have not 
tested it.  Please open a PR at https://bugs.freebsd.org/bugzilla/ is 
you see a problem.


sysutils/bacula13-client-static did not build cleanly. That may come later.

===> Checking for items in STAGEDIR missing from pkg-plist
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: sbin/bacula-fd-static
Error: Missing: sbin/bconsole-static

Details at 
https://services.unixathome.org/poudriere/data/13amd64-dvl-testing/2022-07-10_00h33m03s/logs/errors/bacula13-client-static-13.0.0.log


--
Dan Langille - d...@langille.org
https://langille.org/


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-announce] Beta Release 11.3.2

2022-03-25 Thread Dan Langille



Dan Langille wrote on 3/25/22 9:16 PM:

Eric Bollengier via Bacula-announce wrote on 3/24/22 1:50 PM:

Hello,

We are pleased to announce the first BETA release of the next major 
Bacula
version 11.3.2 to both the Bacula website (www.bacula.org) and to 
SourceForge.




Now might be the time to add 11.3 to 
https://regress.bacula.org/viewProjects.php


I've just updated my test hosts to run 11.3 regression testing.


Oh, wait, they are showing up:

https://regress.bacula.org/index.php?project=Bacula-11.0=2022-03-25

Perhaps adding 11.3 is not needed. I hope those failure tests are helpful.

--
Dan Langille - d...@langille.org
https://langille.org/


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-announce] Beta Release 11.3.2

2022-03-25 Thread Dan Langille

Eric Bollengier via Bacula-announce wrote on 3/24/22 1:50 PM:

Hello,

We are pleased to announce the first BETA release of the next major 
Bacula
version 11.3.2 to both the Bacula website (www.bacula.org) and to 
SourceForge.




Now might be the time to add 11.3 to 
https://regress.bacula.org/viewProjects.php


I've just updated my test hosts to run 11.3 regression testing.

--
Dan Langille - d...@langille.org
https://langille.org/


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] S3 cloud plugin does not load on FreeBSD 13.0

2021-07-18 Thread Dan Langille
On Sun, Jul 18, 2021, at 6:32 AM, Ronny Forberger wrote:
> Hi,
> 
> I have a problem that I cannot load the s3 cloud plugin on FreeBSD 
> 13.0-RELEASE-p3 amd64.
> 
> The error in bacula-sd I am getting is the following:
> 
> > my-sd: init_dev.c:488-0 Driver=cloud entry point=800d4fe60
> > my-sd: cloud_dev.c:876-0 Open SD driver at 
> > /usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so
> > my-sd: cloud_dev.c:879-0 Driver=s3 handle=800d19408
> > my-sd: cloud_dev.c:881-0 Lookup "BaculaCloudDriver" in driver=s3
> > my-sd: cloud_dev.c:883-0 Driver=s3 entry point=0
> > my-sd: cloud_dev.c:889-0 Lookup of symbol "BaculaCloudDriver" 
> > driver=/usr/local/lib/bacula-sd-cloud-s3-driver-11.0.5.so failed: 
> > ERR=Undefined symbol "BaculaCloudDriver"

This line above ...

> > my-sd: message.c:1831-0 cloud_dev.c:1001 Could not open Cloud driver 
> > type=1 for Device=CloudStorage.
> > my-sd: message.c:1831-0 cloud_dev.c:1032 Cloud driver initialization 
> > error Cloud driver not properly loaded
> > my-sd: htable.c:67-0 malloc buf=801c71ae8 size=100 rem=76
> > my-sd: init_dev.c:235-0 init_dev allocated: 801c1cc28
> > my-sd: init_dev.c:437-0 init_dev: tape=0 
> > dev_name=/backup/backupdisk1/bacula-storage
> > my-sd: dev.c:1143-0 DEVICE::register_metrics called. 0x801c1cc28 
> > collector=0x80101c938
> > my-sd: stored.c:706-0 SD init done CloudStorage (0x801c1cc28)
> I comppiled the cloud-driver and the cloud-s3-driver with the following 
> commands:
> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored 
> > >   make install-cloud
> >
> > LT Compiling cloud_parts.c
> > LT Compiling cloud_transfer_mgr.c
> > LT Compiling cloud_dev.c
> > cloud_dev.c:1818:17: warning: address of array 'tpkt->m_hash64' will 
> > always evaluate to 'true' [-Wpointer-bool-conversion]
> >   if (tpkt->m_hash64) {
> >   ~~  ~~^~~~
> > 1 warning generated.
> > LT Compiling file_driver.c
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool 
> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib 
> > -L/usr/local/lib  -L/usr/local/lib -fstack-protector-strong -shared 
> > cloud_parts.lo cloud_transfer_mgr.lo cloud_dev.lo file_driver.lo -o 
> > bacula-sd-cloud-driver.la -R /usr/local/lib -rpath /usr/local/lib 
> > -module -export-dynamic -release 11.0.5
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >  
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool 
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 
> > bacula-sd-cloud-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
> >
> > $ /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/src/stored 
> > >   make install-s3-cloud
> >
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >  
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool 
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 
> > bacula-sd-cloud-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-driver.la
> > LT Compiling s3_driver.c
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool 
> > --silent --tag=CXX --mode=link /usr/bin/c++ -L/usr/local/lib 
> > -L/usr/local/lib  -L/usr/local/lib -fstack-protector-strong -shared 
> > cloud_parts.lo cloud_transfer_mgr.lo s3_driver.lo -o 
> > bacula-sd-cloud-s3-driver.la  -R /usr/local/lib -rpath /usr/local/lib 
> > -module -export-dynamic -release 11.0.5
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/autoconf/mkinstalldirs
> >  
> > /usr/local/lib
> > /usr/ports/sysutils/bacula11-server/work/bacula-11.0.5/libtool 
> > --silent --tag=CXX --mode=install /usr/bin/install -c -m 755 
> > bacula-sd-cloud-s3-driver.la /usr/local/lib
> > /bin/rm -f /usr/local/lib/bacula-sd-cloud-s3-driver.la
> >
> But when I start bacula-sd with bacula-sd -d 255, I am getting the above
> 
> > ERR=Undefined symbol "BaculaCloudDriver"
> error message.

Matches this line.  My thoughts: don't try to start bacula-sd until that 
compile time error is fixed.

> 
> Any ideas what cloud be wrong?

Let's see what the list comes up with over the next few days. If nothing comes 
of it, try the devel mailing list.

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-devel] Release 11.0.5

2021-06-04 Thread Dan Langille
On Fri, Jun 4, 2021, at 8:19 AM, Sven Hartge wrote:
> On 04.06.21 08:46, Eric Bollengier via Bacula-users wrote:
> 
> > It seems that the release process is very attractive these days,
> > the tags are usually pushed at the end of the procedure (after
> > few days or when preparing the next release. It's better to
> > be sure that everything is ok because we cannot overwrite tags
> > once they are cloned on other git repositories.
> > 
> > Of course, I can push them now, it seems that the release
> > is correct this time.
> 
> I am sorry to say, but that is really strange. Of all the projects I 
> follow, Bacula is the only with that interpretation of tag usage.
> 
> Everyone else tags a release as soon as it is released and then the 
> release it not changed afterwards.
> 
> If something is wrong, a new release is made and tagged as well.
> 
> Same goes for the provided archives. Please don't remove existing ones 
> and replace them with other archives with the same version.

A project should never do this. 

As a packager, you have now made my job much bigger than it needs to be.
This is a non-trivial situation. If there is a problem with the release,
create another release. Your downstream will have much more respect for you
than if you do otherwise.

Please, never do this.

Archives are archives, never to be modified. By definition.

> This makes life really difficult for distributions when you add a new 
> version and then suddenly it is revoked and replaces by a new release 
> with the same version number.
> 
> If you for example release 11.0.5 and we upload this to Debian and two 
> days later you remove the release and replace it, we would then need to 
> "invent" a version number like 11.0.5.1 to be able to upload the new 
> version.
> 
> Yes, it looks bad when you have to release several minor version over a 
> few days, but please think of the people downstream of you.

It should not be about the project. It should be about your consumers.

-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] postgres createdb for bacula fails

2021-05-03 Thread Dan Langille
On Mon, May 3, 2021, at 6:16 PM, kjohn...@eclypse.org wrote:

> The next step in my testing is to create an empty bacula database on the
> test system and restore it from the pg_dump file created on the conversion
> system.  I have learned that the bacula postgres database needs to have
> encoding SQL_ASCII.  

For what it's worth, SQL_ASCII was decided upon many years ago.

> Here's the difficulty.  As user bacula on the test system:
> 
> $ export LC_ALL=C
> $ createdb -E SQL_ASCII -T template0 bacula
> createdb: database creation failed: ERROR:  encoding "SQL_ASCII" does not
> match locale "en_US.UTF-8"
> DETAIL:  The chosen LC_CTYPE setting requires encoding "UTF8".
> 
> Probably, fixing this is something that everyone else knows.  Unfortunately,
> my Google searches find too much or almost nothing.  A specific search of
> the mailing list archive found a discussion of why SQL_ASCII was so
> important (circa 2009),

The decision was made earlier I think.

> but not an answer to this problem.  At least, not
> one that I recognized.  (same failure without the export, by the way).  The
> documentation for Bacula 7.4
> (https://www.bacula.org/7.4.x-manuals/en/main/Installing_Configuring_Post.ht
> ml) also states that SQL_ASCII is essential, but I did not recognize a
> solution to this problem there.

I think the problem is more PostgreSQL & environment specific. Here is my test
on FreeBSD 12.2 with PostgreSQL 12.6 - these commands are run as the postgres
user:

$ createdb -E SQL_ASCII -T template0 bacula
$ dropdb bacula
$ export LC_ALL=C
$ createdb -E SQL_ASCII -T template0 bacula
$ 

Works there. I suspect it is something specific to the environment.

What do you have for this value?

$ echo $locale

$ 


-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Restore not working on compressed backup

2021-05-02 Thread Dan Langille
On Wed, Apr 21, 2021, at 4:09 AM, jerome raoul wrote:
> Hello everyone,

> 

> I'm testing Bacula, I have compiled it from source with gzip and lzo to 
> compress my backup. This is working without any issue. But when I try to 
> restore one file on the same client using the default job “RestoreFiles”, the 
> file is restored but compressed. If I decompress it using a trick (printf 
> "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - /tmp/restore/myfile | gzip -dc 
> >/tmp/restore/un_myfile).

> I can read it. Of course, in log I got an error message: Error: attribs.c:621 
> File size of restored file /tmp/restore/myfile not correct. Original 239, 
> restored 173.

> Does someone have an idea of what is causing the issue? Bad configuration 
> when compiling, wrong option on restore?


Perhaps if you shared the backup job so we can see the settings you used...

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] regression testing - more volunteers wanted

2021-04-17 Thread Dan Langille

> On Apr 17, 2021, at 14:22, Andrea Venturoli  wrote:
> 
> On 4/17/21 2:14 PM, Dan Langille wrote:
> 
>> Looking at 
>> https://regress.bacula.org/index.php?project=Bacula-9.6=2021-04-07
>> I consistently get errors on langille_mysql57_bacula but not the others.
>> By all means, post to the developers channel if you get consistent failures. 
>> It is vital that they at least know the tests are failing.
>> I have noticed for a while that I'm regression testing Bacula-9.6 - 
>> eventually I should move to or add Bacula-11.0.
>> Thank you.
> 
> Thanks for your answer.
> 
> See https://regress.bacula.org/index.php?project=Bacula-11.0
> 
> (Notice I moved from FreeBSD 11 to 12).

There’s a good point. My regression testing is done in FreeBSD jails. I should 
install FreeBSD 11.4 jails as well. I hope to update the host to FreeBSD 13 
soon. Then I’ll copy the existing regression jails to new jails & update them 
to FreeBSD 13. Then we’ll have good coverage on all current versions of 
FreeBSD. 

> Shouldn't the developer see all from here?

Perhaps. Posting a regression failure over on the devs list is a good idea 
though. It is the dev mailing list & a regression test failure is a dev issue. 

Between us both, FreeBSD should be well covered. 


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] regression testing - more volunteers wanted

2021-04-17 Thread Dan Langille
On Thu, Apr 8, 2021, at 6:29 AM, Andrea Venturoli wrote:
> On 3/21/21 2:48 PM, Dan Langille wrote:
> 
> Hello.
> 
> > More diversity in OS versions is a goal. At present, only FreeBSD is 
> > consistently represented.
> 
> While we are at it, I'm one of those representing FreeBSD :)
> Since I moved my tests to version 11, I'm getting a few persistent failures:
> > The following tests FAILED:
> >  69 - disk:file-span-vol-test (Failed)
> >  85 - disk:backup-bacula-test (BAD_COMMAND)
> >  88 - disk:bconsole-test (Failed)
> > 172 - disk:truncate-test (Failed)
> 
> Should I investigate better? Report something in some way? Or is the 
> automatic posting of results enough?

Looking at 
https://regress.bacula.org/index.php?project=Bacula-9.6=2021-04-07

I consistently get errors on langille_mysql57_bacula but not the others.

By all means, post to the developers channel if you get consistent failures. It 
is vital that they at least know the tests are failing.

I have noticed for a while that I'm regression testing Bacula-9.6 - eventually 
I should move to or add Bacula-11.0.

Thank you.

-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Backing up Bacula itself

2021-03-29 Thread Dan Langille


On Mon, Mar 29, 2021, at 9:46 AM, Doruk Fisek via Bacula-users wrote:
> Hi,
> 
>  What do you recommend for backing up the Bacula server itself?
> 
>  We're backing up the Bacula VM at the virtualization level, I'd like
> to hear other (perhaps better) practices.

That's a good start.  It's similar to how I back up my FreeBSD jail which
runs Bacula.

The catalog is dumped to disk on a daily basis. That is backed up every day.

The dump is in plain text which makes it easier/faster to rsync somewhere else. 
That rsync is to multiple locations.

I rsync the configuration files away on a daily basis, to a few other locations.

-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Restoring encrypted files to a different host

2021-03-24 Thread Dan Langille
On Wed, Mar 24, 2021, at 3:37 PM, Shawn Rappaport wrote:
> What do I need to do in order to be able to restore from one server to the 
> other? Do I need to copy the private key from portal02-px to portal01-px and 
> update bacula-fd.conf on them as well? 

Yes, but note, I have never tried this.

> If so, what would I put in bacula-fd.conf?

Basically, the same as what you had in the other client for PKI Keypair

see

https://www.bacula.org/11.0.x-manuals/en/main/Data_Encryption.html

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] regression testing - more volunteers wanted

2021-03-24 Thread Dan Langille
On Mon, Mar 22, 2021, at 8:23 AM, Adolf Belka (gmail) wrote:
> Hi All,
> 
> I have run the regression tests and had a couple of failures. One of 
> those disappeared after re-running the test. The other I have re-run 
> with debug set and saved the output to a file. I also had a syntax error 
> from the regression script itself right at the end for the total time 
> printing.

What was the error?


-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] regression testing - more volunteers wanted

2021-03-24 Thread Dan Langille
On Tue, Mar 23, 2021, at 8:26 AM, Adolf Belka (gmail) wrote:
> Hi All,
> 
> Found the README.ctest file and the CDashboard url. Have successfully 
> run the experimental-disk script and the results have got onto the 
> CDashboard.
> 
> So everything seems to be working well except that the bacula configure 
> script does not recognise Arch Linux.

Looking at 
http://regress.bacula.org/index.php?project=Bacula-9.6=2021-03-24 I see

9.6.7-10Dec20-mysql-unknown-unknown

Are you referring to 'unknown-unknown' where as other results have 
'freebsd-12.2-RELEASE-p4'?

-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] regression testing - more volunteers wanted

2021-03-21 Thread Dan Langille
Hello,

Members of project communities often wonder how they can contribute without 
coding.

I have one suggestion: regression testing.

More diversity in OS versions is a goal. At present, only FreeBSD is 
consistently represented. That's not enough. Your OS should be there too.With 
daily regression testing on a wide variety of operating systems and versions, 
changes to code which introduce regressions in behavior can be quickly caught. 
Catching them early is key to getting them fixed.

It is not difficult to set up and it run automatically. No special skills 
required. If you are running Bacula
now, this is an easy way for you to contribute to the community.

The testing is run as non-root and requires no special privileges. I run my 
testing under my own login.

In my setup, I create a dedicated host (in a FreeBSD jail for this) but that is 
not required. The resources
required are not great.

Details at 
https://www.bacula.org/11.0.x-manuals/en/developers/Bacula_Regression_Testing.html

-- 
Dan Langille
d...@langille.org




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] ERR=20:"unable to get local issuer certificate"

2021-01-23 Thread Dan Langille
On Tue, Nov 10, 2020, at 2:11 PM, David Newman wrote:
> Director: FreeBSD 12.2, bacula-server-9.6.6 from pkgs
> Client: OpenBSD 6.8, bacula-client-9.6.5 from pkgs
> 
> After upgrading a bacula client's OS from OpenBSD 6.7 to 6.8, nightly
> backups run successfully but throw this warning:
> 
> ERR=20:"unable to get local issuer certificate"
> 
> This setup uses self-signed certificates and worked without errors or
> warnings before this OS upgrade.
> 
> There has been no bacula configuration change on either the client or
> director . A diff of the client bacula-fd.conf file (excerpted below)
> before and after the upgrade shows no change.
> 
> I tried revoking the old client cert and generating a new one, but this
> had no effect on the warning message.
> 
> I also tried command-line "openssl s_client -connect" commands both
> ways. Both connections worked on the respective ports 9101 and 9102.
> 
> Besides the bacula client configuration -- which hasn't changed, aside
> from pointing to new certs with the same filenames -- is there something
> else that needs tweaking on the client?
> 
> Many thanks.
> 
> dn
> 
> -
> 
> client bacula-fd.conf
> 
> Director {
>   Name = nye-dir
>  ..
> 
>   TLS Require = yes
>   TLS Enable = yes
>   TLS Verify Peer = yes
> 
>  # Allow only the Director to connect
>   TLS Allowed CN = "backups.example.com"
>   TLS CA Certificate File = /etc/bacula/cacert.pem
>   TLS Certificate = /etc/bacula/client.pem
>   TLS Key = /etc/bacula/client.key
> 
> }
> 
> ..
> 
> FileDaemon {
>   Name = client-fd
>   FDport = 9102  # where we listen for the director
>   WorkingDirectory = /var/db/bacula
>   Pid Directory = /var/run
>   Maximum Concurrent Jobs = 20
> 
>   TLS Require = yes
>   TLS Enable = yes
> 
>   TLS CA Certificate File = /etc/bacula/cacert.pem
>   TLS Certificate = /etc/bacula/client.pem
>   TLS Key = /etc/bacula/client.key
> 
> }

Did you solve this one?

-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Discrepancy between backup/restore size and files

2020-09-13 Thread Dan Langille
On Sat, Sep 12, 2020, at 6:23 AM, Ben Laurie wrote:
> I restored a Windows incremenental backup (to a FreeBSD machine).
> 
> The restore job looks like:
> 
>   Build OS:   amd64-portbld-freebsd12.1 freebsd 12.1-SYNTH
>   JobId:  92
>   Job:RestoreFiles.2020-09-12_10.36.22_08
>   Restore Client: b2-fd
>   Where:  /tmp/bacula-restores
>   Replace:Always
>   Start time: 12-Sep-2020 10:36:24
>   End time:   12-Sep-2020 10:45:48
>   Elapsed time:   9 mins 24 secs
>   Files Expected: 3,858
>   Files Restored: 3,858
>   Bytes Restored: 82,256,949,051 (82.25 GB)
>   Rate:   145845.7 KB/s
>   FD Errors:  0
>   FD termination status:  OK
>   SD termination status:  OK
>   Termination:Restore OK
> 
> But...
> 
> b2 /tmp/bacula-restores# du -sh *   
> 804Mc:
>  22Kj:
>  25Gk:
> 
> Why are the restored files less than a third of the size of the restore job?

Is zfs compression involved?

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] TLS issues

2020-08-03 Thread Dan Langille
On Mon, Aug 3, 2020, at 12:30 PM, Shawn Rappaport wrote:
> 
> Any idea why it would not be encrypted even though I am requiring TLS?

TLS is just transit.  Transport Layer Security.

https://www.bacula.org/9.6.x-manuals/en/main/Bacula_TLS_Communications_E.html

You want Data Encryption:

https://www.bacula.org/9.6.x-manuals/en/main/Data_Encryption.html

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] TLS issues

2020-08-01 Thread Dan Langille
run up to 10 jobs a the same time
>   Autochanger = File1 # point to ourself
>   TLS Enable = yes
>   TLS Require = yes
>   TLS CA Certificate File = /etc/ssl/bacula_ca.crt
>   TLS Certificate = /etc/ssl/bacula_server.crt
>   TLS Key = /etc/ssl/bacula_server.key
> }
> 
> I also added the TLS directives to *bacula-sd.conf*:
> 
> Storage { # definition of myself 
>   Name = xbacdirector01-lv.internal.shutterfly.com-sd
>   SDPort = 9103  # Director's port
>   WorkingDirectory = "/var/bacula"
>   Pid Directory = "/var/run"
>   Plugin Directory = "/usr/lib64"
>   Maximum Concurrent Jobs = 20
>   TLS Enable = yes
>   TLS Require = yes
>   TLS CA Certificate File = /etc/ssl/bacula_ca.crt
>   TLS Certificate = /etc/ssl/bacula_server.crt
>   TLS Key = /etc/ssl/bacula_server.key
> }
> 
> Director { 
>   Name = xbacdirector01-lv.internal.shutterfly.com-dir
>   Password = "password"
>   TLS Enable = yes
>   TLS Require = yes
>   TLS CA Certificate File = /etc/ssl/bacula_ca.crt
>   TLS Certificate = /etc/ssl/bacula_server.crt
>   TLS Key = /etc/ssl/bacula_server.key
> }
> 
> After those changes, I bounced Bacula and tried running bconsole. Here is the 
> error I'm receiving:
> 
> Connecting to Director xbacdirector01-lv.internal.shutterfly.com:9101 
> bconsole: tls.c:87-0 Error with certificate at depth: 0, issuer = 
> /C=US/ST=Arizona/L=Tempe/O=Shutterfly/OU=ops-syseng/CN=xbacdirector01-lv.internal.shutterfly.com/emailAddress=t...@shutterfly.com,
>  subject = 
> /C=US/ST=Arizona/L=Tempe/O=Shutterfly/OU=ops-syseng/CN=xbacdirector01-lv.internal.shutterfly.com/emailAddress=t...@shutterfly.com,
>  ERR=18:self signed certificate
> 31-Jul 14:41 bconsole JobId 0: Error: tls.c:87 Error with certificate at 
> depth: 0, issuer = 
> /C=US/ST=Arizona/L=Tempe/O=Shutterfly/OU=ops-syseng/CN=xbacdirector01-lv.internal.shutterfly.com/emailAddress=t...@shutterfly.com,
>  subject = 
> /C=US/ST=Arizona/L=Tempe/O=Shutterfly/OU=ops-syseng/CN=xbacdirector01-lv.internal.shutterfly.com/emailAddress=t...@shutterfly.com,
>  ERR=18:self signed certificate
> TLS negotiation failed
> Director authorization problem.
> Most likely the passwords do not agree.
> If you are using TLS, there may have been a certificate validation error 
> during the TLS handshake.
> For help, please see 
> http://www.bacula.org/rel-manual/en/problems/Bacula_Frequently_Asked_Que.html
> 
> 
> I'm guessing I didn't configure the certs properly but I don't know why. I 
> also tried creating things with the CN not being fully-qualified, i.e.: 
> CN=xbacdirector01-lv. That didn't make a difference. I'm pretty green when it 
> comes to configuring certs so I apologize if I'm making rookie mistakes.
> 
> Does anyone know what I'm doing wrong? Please let me know if you need more 
> details of my setup and configuration.
> 
> Thanks!
> 
> --Shawn
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

--
  Dan Langille
  d...@langille.org

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] BSR file not updated with every job

2020-08-01 Thread Dan Langille
On Tue, Jul 28, 2020, at 12:52 PM, Martin Simmons wrote:
> >>>>> On Tue, 28 Jul 2020 07:55:21 -0400, Dan Langille said:
> > 
> > On Mon, Jul 27, 2020, at 11:55 AM, Dan Langille wrote:
> > > On Sun, Jul 26, 2020, at 7:33 AM, Dan Langille wrote:
> > > > > On Jul 24, 2020, at 6:00 PM, Martin Simmons  
> > > > > wrote:
> > > > > 
> > > > >>>>>> On Fri, 24 Jul 2020 09:09:12 -0400, Dan Langille said:
> > > > >> 
> > > > >> I'm using Bacula 9.6.5 on FreeBSD 12.1
> > > > >> 
> > > > >> I have noticed a BSR file which is not getting updated after every 
> > > > >> job run.
> > > > >> ...
> > > > >> This is the most recent job, from today:
> > > > >> 
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: Start Backup JobId 314258, 
> > > > >> Job=tallboy_home.2020-07-24_03.05.00_08
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: There are no more Jobs 
> > > > >> associated with Volume "IncrAuto-7393". Marking it purged.
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: All records pruned from Volume 
> > > > >> "IncrAuto-7393"; marking it "Purged"
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: Recycled volume "IncrAuto-7393"
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: Using Device 
> > > > >> "vDrive-IncrFile-6" to write.
> > > > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Recycled volume 
> > > > >> "IncrAuto-7393" on File device "vDrive-IncrFile-6" 
> > > > >> (/usr/local/bacula/volumes/IncrFile), all previous data lost.
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: Max Volume jobs=1 exceeded. 
> > > > >> Marking Volume "IncrAuto-7393" as Used.
> > > > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Elapsed time=00:00:03, 
> > > > >> Transfer rate=0  Bytes/second
> > > > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Sending spooled attrs to 
> > > > >> the Director. Despooling 0 bytes ...
> > > > >> 24-Jul 03:05 bacula-dir JobId 314258: Bacula bacula-dir 9.6.5 
> > > > >> (11Jun20):
> > > > >>  Build OS:   amd64-portbld-freebsd12.1 freebsd 
> > > > >> 12.1-RELEASE-p7
> > > > >>  JobId:  314258
> > > > >>  Job:tallboy_home.2020-07-24_03.05.00_08
> > > > >>  Backup Level:   Incremental, since=2020-07-23 03:05:17
> > > > >>  Client: "tallboy-fd" 9.6.5 (11Jun20) 
> > > > >> amd64-portbld-freebsd12.1,freebsd,12.1-RELEASE-p7
> > > > >>  FileSet:"tallboy home" 2012-12-20 14:19:02
> > > > >>  Pool:   "IncrFile" (From Job IncPool override)
> > > > >>  Catalog:"MyCatalog" (From Client resource)
> > > > >>  Storage:"bacula-sd-01-IncrFile" (From Pool resource)
> > > > >>  Scheduled time: 24-Jul-2020 03:05:00
> > > > >>  Start time: 24-Jul-2020 03:05:17
> > > > >>  End time:   24-Jul-2020 03:05:21
> > > > >>  Elapsed time:   4 secs
> > > > >>  Priority:   10
> > > > >>  FD Files Written:   0
> > > > >>  SD Files Written:   0
> > > > >>  FD Bytes Written:   0 (0 B)
> > > > >>  SD Bytes Written:   0 (0 B)
> > > > > 
> > > > > This job wrote nothing to the volume, so there is nothing to add to 
> > > > > the BSR
> > > > > file.
> > > > 
> > > > When I look at this post now, that's easy to see. I find myself 
> > > > smirking to myself when it see it now. Thanks.
> > > > 
> > > > Why was this a problem to me? I'm backing up my BSR files with an rsync 
> > > > to other hosts.
> > > > 
> > > > There is a monitoring script to make sure the rcync'd files are not 
> > > > more than 48 hours old.
> > > > 
> > > > I have 'fixed' with with a crontab on the above referenced host:
> > > > 
> > > > 0   0   *   *  

Re: [Bacula-users] BSR file not updated with every job

2020-07-28 Thread Dan Langille
On Mon, Jul 27, 2020, at 11:55 AM, Dan Langille wrote:
> On Sun, Jul 26, 2020, at 7:33 AM, Dan Langille wrote:
> > > On Jul 24, 2020, at 6:00 PM, Martin Simmons  wrote:
> > > 
> > >>>>>> On Fri, 24 Jul 2020 09:09:12 -0400, Dan Langille said:
> > >> 
> > >> I'm using Bacula 9.6.5 on FreeBSD 12.1
> > >> 
> > >> I have noticed a BSR file which is not getting updated after every job 
> > >> run.
> > >> ...
> > >> This is the most recent job, from today:
> > >> 
> > >> 24-Jul 03:05 bacula-dir JobId 314258: Start Backup JobId 314258, 
> > >> Job=tallboy_home.2020-07-24_03.05.00_08
> > >> 24-Jul 03:05 bacula-dir JobId 314258: There are no more Jobs associated 
> > >> with Volume "IncrAuto-7393". Marking it purged.
> > >> 24-Jul 03:05 bacula-dir JobId 314258: All records pruned from Volume 
> > >> "IncrAuto-7393"; marking it "Purged"
> > >> 24-Jul 03:05 bacula-dir JobId 314258: Recycled volume "IncrAuto-7393"
> > >> 24-Jul 03:05 bacula-dir JobId 314258: Using Device "vDrive-IncrFile-6" 
> > >> to write.
> > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Recycled volume 
> > >> "IncrAuto-7393" on File device "vDrive-IncrFile-6" 
> > >> (/usr/local/bacula/volumes/IncrFile), all previous data lost.
> > >> 24-Jul 03:05 bacula-dir JobId 314258: Max Volume jobs=1 exceeded. 
> > >> Marking Volume "IncrAuto-7393" as Used.
> > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Elapsed time=00:00:03, 
> > >> Transfer rate=0  Bytes/second
> > >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Sending spooled attrs to the 
> > >> Director. Despooling 0 bytes ...
> > >> 24-Jul 03:05 bacula-dir JobId 314258: Bacula bacula-dir 9.6.5 (11Jun20):
> > >>  Build OS:   amd64-portbld-freebsd12.1 freebsd 
> > >> 12.1-RELEASE-p7
> > >>  JobId:  314258
> > >>  Job:tallboy_home.2020-07-24_03.05.00_08
> > >>  Backup Level:   Incremental, since=2020-07-23 03:05:17
> > >>  Client: "tallboy-fd" 9.6.5 (11Jun20) 
> > >> amd64-portbld-freebsd12.1,freebsd,12.1-RELEASE-p7
> > >>  FileSet:"tallboy home" 2012-12-20 14:19:02
> > >>  Pool:   "IncrFile" (From Job IncPool override)
> > >>  Catalog:"MyCatalog" (From Client resource)
> > >>  Storage:"bacula-sd-01-IncrFile" (From Pool resource)
> > >>  Scheduled time: 24-Jul-2020 03:05:00
> > >>  Start time: 24-Jul-2020 03:05:17
> > >>  End time:   24-Jul-2020 03:05:21
> > >>  Elapsed time:   4 secs
> > >>  Priority:   10
> > >>  FD Files Written:   0
> > >>  SD Files Written:   0
> > >>  FD Bytes Written:   0 (0 B)
> > >>  SD Bytes Written:   0 (0 B)
> > > 
> > > This job wrote nothing to the volume, so there is nothing to add to the 
> > > BSR
> > > file.
> > 
> > When I look at this post now, that's easy to see. I find myself 
> > smirking to myself when it see it now. Thanks.
> > 
> > Why was this a problem to me? I'm backing up my BSR files with an rsync 
> > to other hosts.
> > 
> > There is a monitoring script to make sure the rcync'd files are not 
> > more than 48 hours old.
> > 
> > I have 'fixed' with with a crontab on the above referenced host:
> > 
> > 0   0   *   *   *   touch ~/.backmeup
> > 
> > Cheers
> 
> Today we have:
> 
> -rw-r-  1 bacula  bacula   1035 Jul 26 03:05 tallboy-fd_tallboy_home.bsr
> 
> The file updated by cron is:
> 
> [dan@tallboy:~] $ ls -l .backmeup 
> -rw-r--r--  1 dan  dan  0 Jul 27 00:00 .backmeup
> 
> 
> The BSR file contains:
> 
> [dan@bacula:/usr/local/bacula/bsr] $ cat tallboy-fd_tallboy_home.bsr
> # 16-Jul-2020 16:19:59 - tallboy_home.2020-07-05_03.05.02_01 - Full, 
> since=2020-07-16 03:05:07
> Volume="HQ0024L4"
> MediaType="LTO4"
> Slot=3
> VolSessionId=32
> VolSessionTime=1594911018
> VolAddr=1791001367956-1791001368387
> FileIndex=1-91
> Volume="HQ0024L4"
> MediaType="LTO4"
> Slot=3
> VolSessionId=32
> VolSessionTime=1594911018
> VolAddr=1833951043364-1833951043364
> 

Re: [Bacula-users] BSR file not updated with every job

2020-07-27 Thread Dan Langille
On Mon, Jul 27, 2020, at 11:55 AM, Dan Langille wrote:

> There are tree other jobs on this host and their respective BSR files 
> are dated today.

*There are three other jobs ...



-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] BSR file not updated with every job

2020-07-27 Thread Dan Langille
On Sun, Jul 26, 2020, at 7:33 AM, Dan Langille wrote:
> > On Jul 24, 2020, at 6:00 PM, Martin Simmons  wrote:
> > 
> >>>>>> On Fri, 24 Jul 2020 09:09:12 -0400, Dan Langille said:
> >> 
> >> I'm using Bacula 9.6.5 on FreeBSD 12.1
> >> 
> >> I have noticed a BSR file which is not getting updated after every job run.
> >> ...
> >> This is the most recent job, from today:
> >> 
> >> 24-Jul 03:05 bacula-dir JobId 314258: Start Backup JobId 314258, 
> >> Job=tallboy_home.2020-07-24_03.05.00_08
> >> 24-Jul 03:05 bacula-dir JobId 314258: There are no more Jobs associated 
> >> with Volume "IncrAuto-7393". Marking it purged.
> >> 24-Jul 03:05 bacula-dir JobId 314258: All records pruned from Volume 
> >> "IncrAuto-7393"; marking it "Purged"
> >> 24-Jul 03:05 bacula-dir JobId 314258: Recycled volume "IncrAuto-7393"
> >> 24-Jul 03:05 bacula-dir JobId 314258: Using Device "vDrive-IncrFile-6" to 
> >> write.
> >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Recycled volume "IncrAuto-7393" 
> >> on File device "vDrive-IncrFile-6" (/usr/local/bacula/volumes/IncrFile), 
> >> all previous data lost.
> >> 24-Jul 03:05 bacula-dir JobId 314258: Max Volume jobs=1 exceeded. Marking 
> >> Volume "IncrAuto-7393" as Used.
> >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Elapsed time=00:00:03, Transfer 
> >> rate=0  Bytes/second
> >> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Sending spooled attrs to the 
> >> Director. Despooling 0 bytes ...
> >> 24-Jul 03:05 bacula-dir JobId 314258: Bacula bacula-dir 9.6.5 (11Jun20):
> >>  Build OS:   amd64-portbld-freebsd12.1 freebsd 12.1-RELEASE-p7
> >>  JobId:  314258
> >>  Job:tallboy_home.2020-07-24_03.05.00_08
> >>  Backup Level:   Incremental, since=2020-07-23 03:05:17
> >>  Client: "tallboy-fd" 9.6.5 (11Jun20) 
> >> amd64-portbld-freebsd12.1,freebsd,12.1-RELEASE-p7
> >>  FileSet:"tallboy home" 2012-12-20 14:19:02
> >>  Pool:   "IncrFile" (From Job IncPool override)
> >>  Catalog:"MyCatalog" (From Client resource)
> >>  Storage:"bacula-sd-01-IncrFile" (From Pool resource)
> >>  Scheduled time: 24-Jul-2020 03:05:00
> >>  Start time: 24-Jul-2020 03:05:17
> >>  End time:   24-Jul-2020 03:05:21
> >>  Elapsed time:   4 secs
> >>  Priority:   10
> >>  FD Files Written:   0
> >>  SD Files Written:   0
> >>  FD Bytes Written:   0 (0 B)
> >>  SD Bytes Written:   0 (0 B)
> > 
> > This job wrote nothing to the volume, so there is nothing to add to the BSR
> > file.
> 
> When I look at this post now, that's easy to see. I find myself 
> smirking to myself when it see it now. Thanks.
> 
> Why was this a problem to me? I'm backing up my BSR files with an rsync 
> to other hosts.
> 
> There is a monitoring script to make sure the rcync'd files are not 
> more than 48 hours old.
> 
> I have 'fixed' with with a crontab on the above referenced host:
> 
> 0   0   *   *   *   touch ~/.backmeup
> 
> Cheers

Today we have:

-rw-r-  1 bacula  bacula   1035 Jul 26 03:05 tallboy-fd_tallboy_home.bsr

The file updated by cron is:

[dan@tallboy:~] $ ls -l .backmeup 
-rw-r--r--  1 dan  dan  0 Jul 27 00:00 .backmeup


The BSR file contains:

[dan@bacula:/usr/local/bacula/bsr] $ cat tallboy-fd_tallboy_home.bsr
# 16-Jul-2020 16:19:59 - tallboy_home.2020-07-05_03.05.02_01 - Full, 
since=2020-07-16 03:05:07
Volume="HQ0024L4"
MediaType="LTO4"
Slot=3
VolSessionId=32
VolSessionTime=1594911018
VolAddr=1791001367956-1791001368387
FileIndex=1-91
Volume="HQ0024L4"
MediaType="LTO4"
Slot=3
VolSessionId=32
VolSessionTime=1594911018
VolAddr=1833951043364-1833951043364
FileIndex=91-130
# 18-Jul-2020 03:05:24 - tallboy_home.2020-07-18_03.05.01_50 - Incremental, 
since=2020-07-17 03:05:08
Volume="IncrAuto-11646"
MediaType="IncrFile"
VolSessionId=349
VolSessionTime=1594656307
VolAddr=257-14794
FileIndex=1-3
# 19-Jul-2020 03:05:14 - tallboy_home.2020-07-19_03.05.01_23 - Differential, 
since=2020-07-05 03:07:39
Volume="DiffAuto-1473"
MediaType="DiffFile"
VolSessionId=525
VolSessionTime=1594656307
VolAddr=235-19992
FileIndex=1-5
# 26-Jul-2020 03:05:12 - tallboy_home.2020-07-26_03.05.01_14 - Differential, 
since=2020-

Re: [Bacula-users] BSR file not updated with every job

2020-07-26 Thread Dan Langille
> On Jul 24, 2020, at 6:00 PM, Martin Simmons  wrote:
> 
>>>>>> On Fri, 24 Jul 2020 09:09:12 -0400, Dan Langille said:
>> 
>> I'm using Bacula 9.6.5 on FreeBSD 12.1
>> 
>> I have noticed a BSR file which is not getting updated after every job run.
>> ...
>> This is the most recent job, from today:
>> 
>> 24-Jul 03:05 bacula-dir JobId 314258: Start Backup JobId 314258, 
>> Job=tallboy_home.2020-07-24_03.05.00_08
>> 24-Jul 03:05 bacula-dir JobId 314258: There are no more Jobs associated with 
>> Volume "IncrAuto-7393". Marking it purged.
>> 24-Jul 03:05 bacula-dir JobId 314258: All records pruned from Volume 
>> "IncrAuto-7393"; marking it "Purged"
>> 24-Jul 03:05 bacula-dir JobId 314258: Recycled volume "IncrAuto-7393"
>> 24-Jul 03:05 bacula-dir JobId 314258: Using Device "vDrive-IncrFile-6" to 
>> write.
>> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Recycled volume "IncrAuto-7393" 
>> on File device "vDrive-IncrFile-6" (/usr/local/bacula/volumes/IncrFile), all 
>> previous data lost.
>> 24-Jul 03:05 bacula-dir JobId 314258: Max Volume jobs=1 exceeded. Marking 
>> Volume "IncrAuto-7393" as Used.
>> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Elapsed time=00:00:03, Transfer 
>> rate=0  Bytes/second
>> 24-Jul 03:05 bacula-sd-01-sd JobId 314258: Sending spooled attrs to the 
>> Director. Despooling 0 bytes ...
>> 24-Jul 03:05 bacula-dir JobId 314258: Bacula bacula-dir 9.6.5 (11Jun20):
>>  Build OS:   amd64-portbld-freebsd12.1 freebsd 12.1-RELEASE-p7
>>  JobId:  314258
>>  Job:tallboy_home.2020-07-24_03.05.00_08
>>  Backup Level:   Incremental, since=2020-07-23 03:05:17
>>  Client: "tallboy-fd" 9.6.5 (11Jun20) 
>> amd64-portbld-freebsd12.1,freebsd,12.1-RELEASE-p7
>>  FileSet:"tallboy home" 2012-12-20 14:19:02
>>  Pool:   "IncrFile" (From Job IncPool override)
>>  Catalog:"MyCatalog" (From Client resource)
>>  Storage:"bacula-sd-01-IncrFile" (From Pool resource)
>>  Scheduled time: 24-Jul-2020 03:05:00
>>  Start time: 24-Jul-2020 03:05:17
>>  End time:   24-Jul-2020 03:05:21
>>  Elapsed time:   4 secs
>>  Priority:   10
>>  FD Files Written:   0
>>  SD Files Written:   0
>>  FD Bytes Written:   0 (0 B)
>>  SD Bytes Written:   0 (0 B)
> 
> This job wrote nothing to the volume, so there is nothing to add to the BSR
> file.

When I look at this post now, that's easy to see. I find myself smirking to 
myself when it see it now. Thanks.

Why was this a problem to me? I'm backing up my BSR files with an rsync to 
other hosts.

There is a monitoring script to make sure the rcync'd files are not more than 
48 hours old.

I have 'fixed' with with a crontab on the above referenced host:

0   0   *   *   *   touch ~/.backmeup

Cheers

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] BSR file not updated with every job

2020-07-24 Thread Dan Langille
dex=91-130
# 18-Jul-2020 03:05:24 - tallboy_home.2020-07-18_03.05.01_50 - Incremental, 
since=2020-07-17 03:05:08
Volume="IncrAuto-11646"
MediaType="IncrFile"
VolSessionId=349
VolSessionTime=1594656307
VolAddr=257-14794
FileIndex=1-3
# 19-Jul-2020 03:05:14 - tallboy_home.2020-07-19_03.05.01_23 - Differential, 
since=2020-07-05 03:07:39
Volume="DiffAuto-1473"
MediaType="DiffFile"
VolSessionId=525
VolSessionTime=1594656307
VolAddr=235-19992
FileIndex=1-5
[dan@bacula:/usr/local/bacula/bsr] $ 

Ideas?

Thank you


-- 
  Dan Langille
  d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] SD fails to create new volumes: trace included

2019-12-02 Thread Dan Langille
Resending. Sorry, the previous email was incomplete.


On Tue, Jan 9, 2018, at 3:29 PM, Dan Langille wrote:
> > On Jan 9, 2018, at 3:07 PM, Dan Langille  wrote:
> > 
> >> On Jan 9, 2018, at 3:01 PM, Dan Langille  wrote:
> >> 
> >> I wonder if this line is related:
> >> 
> >> 
> >> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> >> 
> >> If no labeling, then no volume is created.
> >> 
> >> See 
> >> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-bacula-trace-L84
> >> 
> >> 
> >> 
> >> bacula-sd-01-sd: mount.c:182-269534 want vol=FullAuto-8867 devvol= 
> >> dev="vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile)
> >> bacula-sd-01-sd: mount.c:190-269534 Try open Vol=FullAuto-8867
> >> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
> >> DEVICE::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> >> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> >> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could 
> >> not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> >> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: mount.c:192-269534 Try autolabel Vol=FullAuto-8867
> >> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
> >> DEVICE::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> >> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> >> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could 
> >> not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> >> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: mount.c:198-269534 open_device failed: ERR=file_dev.c:190 
> >> Could not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> >> bacula-sd-01-sd: mount.c:215-269534 Open of File device 
> >> "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) Volume 
> >> "FullAuto-8867" failed: ERR=file_dev.c:190 Could not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> 
> >> bacula-sd-01-sd: mount.c:220-269534 Volume "FullAuto-8867" not loaded on 
> >> File device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile).
> > 
> > This is why it is not labelling, from bacula/src/stored/mount.c
> > 
> > int DCR::try_autolabel(bool opened)
> > {
> >   DCR *dcr = this;
> > 
> >   if (dev->poll && !dev->is_tape()) {
> >  Dmsg0(100, "No autolabel because polling.\n");
> >  return try_default;   /* if polling, don't try to create new 
> > labels */
> >   }
> > 
> 
> To get this backup running again:
> 
> I manually deleted the last volume which failed to create (it was in 
> the Catalog but not in the filesystem).
> 
> I manually labeled that Volume (which creates a Catalog entry and a 
> file on disk).
> 
> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-1319

This is still an issue.  This time it's on a new bacula-sd I have recently 
created. The bacula-sd configuration is based upon other SD instances I have.

*m
02-Dec 13:06 bacula-dir JobId 302468: Created new Volume="FullAuto-11080", 
Pool="FullFile-03", MediaType="FullFile03" in catalog.
02-Dec 13:06 bacula-sd-03 JobId 302468: Warning: mount.c:215 Open of File 
device &qu

Re: [Bacula-users] SD fails to create new volumes: trace included

2019-12-02 Thread Dan Langille
On Tue, Jan 9, 2018, at 3:29 PM, Dan Langille wrote:
> > On Jan 9, 2018, at 3:07 PM, Dan Langille  wrote:
> > 
> >> On Jan 9, 2018, at 3:01 PM, Dan Langille  wrote:
> >> 
> >> I wonder if this line is related:
> >> 
> >> 
> >> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> >> 
> >> If no labeling, then no volume is created.
> >> 
> >> See 
> >> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-bacula-trace-L84
> >> 
> >> 
> >> 
> >> bacula-sd-01-sd: mount.c:182-269534 want vol=FullAuto-8867 devvol= 
> >> dev="vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile)
> >> bacula-sd-01-sd: mount.c:190-269534 Try open Vol=FullAuto-8867
> >> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
> >> DEVICE::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> >> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> >> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could 
> >> not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> >> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: mount.c:192-269534 Try autolabel Vol=FullAuto-8867
> >> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
> >> DEVICE::open_device(DCR *, int)
> >> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> >> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> >> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could 
> >> not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> >> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> >> file_dev::open_device(DCR *, int)
> >> bacula-sd-01-sd: mount.c:198-269534 open_device failed: ERR=file_dev.c:190 
> >> Could not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> >> bacula-sd-01-sd: mount.c:215-269534 Open of File device 
> >> "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) Volume 
> >> "FullAuto-8867" failed: ERR=file_dev.c:190 Could not 
> >> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640):
> >>  ERR=No such file or directory
> >> 
> >> bacula-sd-01-sd: mount.c:220-269534 Volume "FullAuto-8867" not loaded on 
> >> File device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile).
> > 
> > This is why it is not labelling, from bacula/src/stored/mount.c
> > 
> > int DCR::try_autolabel(bool opened)
> > {
> >   DCR *dcr = this;
> > 
> >   if (dev->poll && !dev->is_tape()) {
> >  Dmsg0(100, "No autolabel because polling.\n");
> >  return try_default;   /* if polling, don't try to create new 
> > labels */
> >   }
> > 
> 
> To get this backup running again:
> 
> I manually deleted the last volume which failed to create (it was in 
> the Catalog but not in the filesystem).
> 
> I manually labeled that Volume (which creates a Catalog entry and a 
> file on disk).
> 
> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-1319

This is still an issue.  This time it's on a new bacula-sd I have recently 
created. The bacula-sd configuration is based upon other SD instances I have.

*m
02-Dec 13:06 bacula-dir JobId 302468: Created new Volume="FullAuto-11080", 
Pool="FullFile-03", MediaType="FullFile03" in catalog.
02-Dec 13:06 bacula-sd-03 JobId 302468: Warning: mount.c:215 Open of File 
device "vDrive-FullFile-1" (/usr/local/bacula/volumes/

Re: [Bacula-users] Is there a test for LTO-4 tapes or just pull them from service?

2019-05-07 Thread Dan Langille
On Mon, May 6, 2019, at 10:46 PM, John R Zoch wrote:
> I'm running Bacula 9.4.2 on an FreeNAS 11.2-U3 in an iocage jail. Attached is 
> an image from bat indicating a bunch of failed tapes. They passed the rewind 
> - write EOF test. 
> 

The rewind-write EOF test is more a test of Bacula and the tape drive, not of 
the tape itself.

That image from BAT shows the status of that tape in the Catalog. The Error 
could be for reasons other than a bad tape.

To know for sure, you need to look at the job email, or the bacula log on the 
system itself (provided there is one).

> 
> Is there something I should try to make them usable again (a test/format of 
> sorts) or is the Bacula error indication sufficient that I should just pull 
> them from service?
> 
> I've used "btape qfill" in the past but it only tests the first tape in the 
> library. I'm not certain qfill verifies the data it wrote to the tape to 
> provide some assurance that the tape is good.

Show us some of these errors and we'll know for sure if it's a tape error. It 
could be drive loading error. It wanted tape X but couldn't load it, for 
whatever reason.___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] MySQL 8.0 mysql.c:201:4: error: unknown type name 'my_bool'

2019-05-03 Thread Dan Langille
> On Apr 27, 2019, at 4:03 PM, Carsten Leonhardt  wrote:
> 
> Hi Dan,
> 
>> I am setting up regression testing for MySQL 8.0 and getting this error on 
>> the build:
> 
>> /usr/home/dan/src/bacula/regress/build/libtool --silent --tag=CXX 
>> --mode=link /usr/bin/c++-o libbaccats.la cats_null.lo -export-dynamic 
>> -rpath /usr/home/dan/src/bacula/regress/bin -release 9.4.3
>> Compiling mysql.c
>> mysql.c:201:4: error: unknown type name 'my_bool'
>>   my_bool reconnect = 1;
>>   ^
>> 1 error generated.
>> *** Error code 1
> 
> MySQL removed "my_bool" in version 8, a patch is needed. For example:
> 
> see https://patches.ubuntu.com/b/bacula/bacula_9.4.2-2ubuntu1.patch
> 
> There might be cleverer approaches.
> 
> Regards,
> 
> Carsten

That got me farther into the compile.  Now it's:

cats_test.c:533:75: error: too many arguments provided to function-like macro 
invocation
   ok(db_get_file_list(jcr, jcr->db_batch, buf, false, false, list_files, ),
  ^
../cats/protos.h:224:9: note: macro 'db_get_file_list' defined here
#define db_get_file_list(jcr, mdb, jobids, opts, result_handler, ctx) \
^
1 error generated.
*** Error code 1

— 
Dan Langille
http://langille <http://langille/>.org/





___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] bacula and SQLite

2019-05-01 Thread Dan Langille
On Wed, May 1, 2019, at 4:59 AM, Kern Sibbald wrote:
> Hello,
> 
> Just so everyone knows how I feel about SQLite -- it is a great program,
> written by a very good programmer.  It is probably closer to Postgres
> than to MySQL in its syntax. 

An aside...

There is a reason for that similarity.  When SQLite was being written,
PostgreSQL was the reference for what to do.

D. Richard Hipp gave the keynote talk at PGCon 2014.  I was present.

The slides are at: https://www.pgcon.org/2014/schedule/events/736.en.html

> However, it is really not an appropriate
> database for many reasons for Bacula, and maintaining the update tables
> scripts is very time consuming.  I will continue to keep the source code
> as long as Sven and others contribute what is necessary to keep it
> working.

In that talk, you will find these quotes:

“SQLite is not a replacement for PostgreSQL.
SQLite is a replacement for fopen().”

"Use PostgreSQL as an enterprise data depot, and use SQLite for application 
files."

The Catalog is definitely not an application file.

Please move from SQLite to PostgreSQL. You will be better off.

 


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] MySQL 8.0 mysql.c:201:4: error: unknown type name 'my_bool'

2019-04-27 Thread Dan Langille
Thank you. 

With that patch, we have MySQL 8 covered via regression testing. 

http://regress.bacula.org/index.php?project=Bacula-9.4=2019-04-27

FYI I apply that patch during testing. It needs to get into the code. 

--
 Dan Langille
 d...@langille.org



On Sat, Apr 27, 2019, at 4:20 PM, Carsten Leonhardt wrote:
> Hi Dan,
> 
> > I am setting up regression testing for MySQL 8.0 and getting this error on 
> > the build:
> 
> > /usr/home/dan/src/bacula/regress/build/libtool --silent --tag=CXX 
> > --mode=link /usr/bin/c++ -o libbaccats.la cats_null.lo -export-dynamic 
> > -rpath /usr/home/dan/src/bacula/regress/bin -release 9.4.3
> > Compiling mysql.c
> > mysql.c:201:4: error: unknown type name 'my_bool'
> > my_bool reconnect = 1;
> > ^
> > 1 error generated.
> > *** Error code 1
> 
> MySQL removed "my_bool" in version 8, a patch is needed. For example:
> 
> see https://patches.ubuntu.com/b/bacula/bacula_9.4.2-2ubuntu1.patch
> 
> There might be cleverer approaches.
> 
> Regards,
> 
> Carsten
> 
> 
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] MySQL 8.0 mysql.c:201:4: error: unknown type name 'my_bool'

2019-04-14 Thread Dan Langille
I am setting up regression testing for MySQL 8.0 and getting this error on the 
build:

Making libbacsql.la ...
/usr/home/dan/src/bacula/regress/build/libtool --silent --tag=CXX --mode=link 
/usr/bin/c++-o libbacsql.la bvfs.lo cats.lo sql.lo sql_cmds.lo 
sql_create.lo sql_delete.lo sql_find.lo sql_get.lo sql_list.lo sql_update.lo 
-export-dynamic -rpath /usr/home/dan/src/bacula/regress/bin -release 9.4.3 
Compiling cats_null.c
Making libbaccats.la ...
/usr/home/dan/src/bacula/regress/build/libtool --silent --tag=CXX --mode=link 
/usr/bin/c++-o libbaccats.la cats_null.lo -export-dynamic -rpath 
/usr/home/dan/src/bacula/regress/bin -release 9.4.3
Compiling mysql.c
mysql.c:201:4: error: unknown type name 'my_bool'
   my_bool reconnect = 1;
   ^
1 error generated.
*** Error code 1

mysql80-client-8.0.15
mysql80-server-8.0.15

   C++ Compiler: /usr/bin/c++ version
   Compiler flags:-g -O2 -Wall -x c++ -fno-strict-aliasing 
-fno-exceptions -fno-rtti
   Linker flags:  
   Libraries:-lpthread 
   Statically Linked Tools:  no
   Statically Linked FD: no
   Statically Linked SD: no
   Statically Linked DIR:no
   Statically Linked CONS:   no
   Database backends:MySQL
   Database port: 
   Database name:regress
   Database user:regress
   Database SSL options: 

   SMTP Host Address:localhost

   Director Port:8101
   File daemon Port: 8102
   Storage daemon Port:  8103

   Director User:
   Director Group:   
   Storage Daemon User:  
   Storage DaemonGroup:  
   File Daemon User: 
   File Daemon Group:

   Large file support:   yes
   Bacula conio support: yes -ltinfo
   readline support: no 
   TCP Wrappers support: yes -lwrap
   TLS support:  yes
   Encryption support:   yes
   ZLIB support: yes
   LZO support:  no
   enable-smartalloc:yes
   enable-lockmgr:   yes
   bat support:  no
   client-only:  no
   build-dird:   yes
   build-stored: yes
   Plugin support:   yes
   AFS support:  no
   ACL support:  yes
   XATTR support:yes
   systemd support:  no 
   Batch insert enabled: MySQL

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [External] Re: [Bacula-devel] IBAdmin

2019-01-29 Thread Dan Langille
> On Jan 28, 2019, at 1:19 PM, Radosław Korzeniewski 
>  wrote:
> 
> Hello Kern,
> 
> pt., 25 sty 2019 o 16:47 Kern Sibbald  <mailto:k...@sibbald.com>> napisał(a):
> Hello guys,
> 
> Interesting conversation.  I thought I would throw in some general comments 
> of my own.
> 
> - I really like seeing another GUI for Bacula, because it is something we 
> really need.
> 
> Thank you Kern, I really appreciate your support. You are the only one who 
> sees it as an opportunity and not a threat.

I think that's unfair.

I hope you have not taken the suggestions provided here are the result of 
viewing IBAdmin as a threat. They are not.

I hope you have not taken the suggestions for improvement as an attack on 
IBAdmin. They are not.

It is relatively straight forward, I should hope, to identify the parts of the 
database you need to read and the parts you need to update.

I recently did similar for another application:

It looks something like this:

create role freshsource_ro;
GRANT SELECT ON TABLE public.commit_log TO freshsource_ro;
GRANT SELECT ON TABLE public.commit_log_elements TO freshsource_ro;
GRANT SELECT ON TABLE public.element TO freshsource_ro;
GRANT SELECT ON TABLE public.latest_commits TO freshsource_ro;
GRANT SELECT ON TABLE public.repo TO freshsource_ro;
GRANT SELECT ON TABLE public.security_notice TO freshsource_ro;
GRANT SELECT ON TABLE public.system TO freshsource_ro;
GRANT SELECT ON TABLE public.users TO freshsource_ro;
GRANT UPDATE(cookie) ON TABLE public.users TO freshsource_ro;
GRANT UPDATE(lastlogin) ON TABLE public.users TO freshsource_ro;
GRANT SELECT ON TABLE public.watch_list TO freshsource_ro;
GRANT SELECT ON TABLE public.watch_list_element TO freshsource_ro;
GRANT SELECT ON TABLE public.watch_notice TO freshsource_ro;

Then a user is created and added to that freshsource_ro role:

create user freshsource_dev with password '[redacted]' IN ROLE freshsource_ro;

Nobody sees the application itself as threat.

These are straight forward security practices which are carried out in many 
organizations, both small and large.

Best wishes.

--
Dan Langille - BSDCan / PGCon
d...@langille.org



signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-devel] IBAdmin

2019-01-24 Thread Dan Langille
> On Jan 24, 2019, at 5:30 AM, Radosław Korzeniewski 
>  wrote:
> 
> Hello Dan,
> 
> śr., 23 sty 2019 o 23:59 Dan Langille  <mailto:d...@langille.org>> napisał(a):
>> On Jan 23, 2019, at 4:56 PM, Radosław Korzeniewski 
>> mailto:rados...@korzeniewski.net>> wrote:
>> 
>> 
>> Does IBAdmin work with a readonly connection to the Catalog database?
>> 
>> 
>> No.
> 
> What features will be available with a read-only connection?
> 
> 
> Why do you need a read-only database?

Isolation.

Bacula manipulates the Catalog. IBAdmin reports upon it.  Why would it need to 
modify my Catalog?

> All management is based on database objects. When you want to create or 
> change any configuration it will change/update database objects for you. When 
> you log in your session is stored in a database. These are the most basic 
> operations which require read-write access to database objects.

OK, but all of this must be in the Bacula Catalog? If so, why? Why not put 
IBAdmin-specific data in a separate database?

> I assume you want to test it, right?

Yes, I do, I am interested, but not if I must give it write access to my Bacula 
database.

--
Dan Langille - BSDCan / PGCon
d...@langille.org





signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-devel] IBAdmin

2019-01-23 Thread Dan Langille
> On Jan 23, 2019, at 4:56 PM, Radosław Korzeniewski 
>  wrote:
> 
> 
> Does IBAdmin work with a readonly connection to the Catalog database?
> 
> 
> No.

What features will be available with a read-only connection?



signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] [Bacula-devel] IBAdmin

2019-01-23 Thread Dan Langille
> On Jan 22, 2019, at 6:19 AM, Radosław Korzeniewski 
>  wrote:
> 
> Hello Bacula Users!
> 
> There are a lot of different tools for managing a Bacula system, some GUI, 
> some for reports only and every new week we have a new one in the wild (just 
> kidding)! So I thought I'll introduce you another one. :) More GUI for 
> Bacula, more fun.
> 
> The GUI is called IBAdmin (Inteos Backup Administration GUI) and is developed 
> by Inteos for its BEE customers. The IBAdmin application is available for 
> community (IBAdmin Community version) for a few months now with AGPL v3 
> license. IBAdmin Community support Bacula Enterprise Edition 8.x-10.x and 
> Bacula Community 9.2.x. You can check the code at 
> https://github.com/inteos/IBAdmin <https://github.com/inteos/IBAdmin>, its 
> Wiki at https://github.com/inteos/IBAdmin/wiki 
> <https://github.com/inteos/IBAdmin/wiki> and especially the installation 
> (https://github.com/inteos/IBAdmin/wiki/Installation-procedure 
> <https://github.com/inteos/IBAdmin/wiki/Installation-procedure>) or upgrade 
> (https://github.com/inteos/IBAdmin/wiki/Upgrade-procedure 
> <https://github.com/inteos/IBAdmin/wiki/Upgrade-procedure>) procedures if you 
> run a previous version. 
> 
> What is IBAdmin, someone may ask? IBAdmin is a new Bacula Enterprise or 
> Community administration GUI dedicated to Inteos Backup users. But currently 
> as a general purpose tool it can be used by any BEE or BC enthusiasts. The 
> main goal for IBAdmin is simplicity, so you don't need to know much about 
> Bacula or Bacula Enterprise to start using it, just an overall backup 
> architecture. Thanks to this any newbie learning curve should be extremely 
> short, i.e. all the configuration and management is hidden as IBAdmin. 
> IBAdmin allows you to prepare and manage your backup solution as a whole 
> appliance, something you can find at other products (sort of, some 
> functionality is still missing). The IBAdmin Community support file level 
> backup only for Bacula Community and BEE which is included in the latest 
> release. The latest release brings a lot of new features including an 
> advanced users management feature with roles based access control and 
> departments support as a multitenant solution.
> I hope you will find it useful.

Can you elaborate upon the systemd requirement please?

I see: "Without systemd you can't manage system daemons and its logs directly 
from application and this part of ibadmin won't work"

By manage, what do you mean? Stop, start, etc? What does IBAdmin do with the 
logs?

Does IBAdmin work with a readonly connection to the Catalog database?

When a new client is created, does IBAdmin create the bacula-fd.conf file?

Thank you.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Backing up to NetApp

2019-01-20 Thread Dan Langille
Have you backed up to NetApp?

At $WORK we are likely to soon have heaps of NetApp storage at our disposal.

Of course my thoughts turned to backups.

Do you use a NetApp appliance with Bacula as a destination for backups?

I know of the NetApp plugin for Bacula, but that is the wrong direction: that 
is for backing up the NetApp device.

I've never mounted remote storage for bacula-sd over any of NFS, CIF, Samba, 
etc.

I can't imagine NFS would be useful given the throughput.  Mind you, I don't 
yet know how much we'll be backing up, but it'll be more than 1

Have you?

--
Dan Langille - BSDCan / PGCon
d...@langille.org




signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] check_bacula Segmentation fault if host is down

2019-01-18 Thread Dan Langille
> On Jan 18, 2019, at 11:24 AM, Josh Fisher  wrote:
> 
> On 1/17/2019 12:17 PM, Dan Langille wrote:
> 
>> I was noticing this in my /var/log/messages:
>> 
>> Jan 17 17:04:00 slocum kernel: pid 52623 (check_bacula), uid 181: exited on 
>> signal 11
>> Jan 17 17:04:21 slocum kernel: pid 53805 (check_bacula), uid 181: exited on 
>> signal 11
>> 
>> I tracked it down to a host which was not up:
>> 
>> $ time ./check_bacula -H tape02 -D fd -M nagios-mon -K '[redacted]'
>> Segmentation fault
>> 
>> real 1m15.101s
>> user 0m0.006s
>> sys  0m0.000s
>> 
>> 
>> Could someone else please try to replicate this situation for me please?
>> 
>> This check is being run on FreeBSD 12.0-RELEASE-p2 with check_bacula from 
>> Bacula 9.2.2
>> 
>> Thank you.
>> 
> Have you tried with debug level increased, say using -d7 in the command line? 
> Maybe that would give a clue as to what it does prior to the segfault.
> 
This was interesting:


$ time ./check_bacula -H tape02 -D fd -M nagios-mon -K '[redacted]' -d7
Segmentation fault

real1m15.137s
user0m0.000s
sys 0m0.004s

Note the time, similar to the first test.

Let's up the debug level

$ time ./check_bacula -H tape02 -D fd -M nagios-mon -K '[redacted]' -d77
check_bacula: bsockcore.c:384-0 Could not connect to server File daemon 
tape02:9102. ERR=Operation timed out
check_bacula: bsockcore.c:197-0 Unable to connect to File daemon on 
tape02:9102. ERR=Operation timed out
Segmentation fault

real1m15.024s
user0m0.000s
sys 0m0.004s


Ok debugging info.

Let's bump up:

$ time ./check_bacula -H tape02 -D fd -M nagios-mon -K '[redacted]' -d777
check_bacula: bsockcore.c:299-0 Current 10.55.0.110:9102 All 10.55.0.110:9102
check_bacula: bsockcore.c:384-0 Could not connect to server File daemon 
tape02:9102. ERR=Host is down
check_bacula: bsockcore.c:197-0 Unable to connect to File daemon on 
tape02:9102. ERR=Host is down
Segmentation fault

real0m0.013s
user0m0.000s
sys 0m0.004s


What: zero time?

Further tests with -d777, -d77, and -d7 all finished in near-zero time.  I 
tried again without -d, and it failed in 75 seconds, like the others.

Try again with -d (a larger value than previously tried), near-zero time 
again.

Then I waited and tried again.

[dan@webserver:/usr/local/libexec/nagios] $ time ./check_bacula -H tape02 -D fd 
-M nagios-mon -K '[redacted' -d
check_bacula: bsockcore.c:299-0 Current 10.55.0.110:9102 All 10.55.0.110:9102
check_bacula: bsockcore.c:384-0 Could not connect to server File daemon 
tape02:9102. ERR=Operation timed out
check_bacula: bsockcore.c:197-0 Unable to connect to File daemon on 
tape02:9102. ERR=Operation timed out
check_bacula: watchdog.c:82-0 Initialising NicB-hacked watchdog thread
check_bacula: watchdog.c:197-0 Registered watchdog 800c38098, interval 300 one 
shot
check_bacula: watchdog.c:254-0 NicB-reworked watchdog thread entered
check_bacula: watchdog.c:296-0 pthread_cond_timedwait 60
check_bacula: btimers.c:177-0 Start bsock timer 800c135e8 tid=800c15000 for 300 
secs at 1547849285
check_bacula: btimers.c:212-0 Stop bsock timer 800c135e8 tid=800c15000 at 
1547849285.
check_bacula: watchdog.c:217-0 Unregistered watchdog 800c38098
check_bacula: watchdog.c:296-0 pthread_cond_timedwait 60
Segmentation fault

real1m15.035s
user0m0.000s
sys 0m0.004s


--
Dan Langille - BSDCan / PGCon
d...@langille.org





signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] check_bacula Segmentation fault if host is down

2019-01-17 Thread Dan Langille
I was noticing this in my /var/log/messages:

Jan 17 17:04:00 slocum kernel: pid 52623 (check_bacula), uid 181: exited on 
signal 11
Jan 17 17:04:21 slocum kernel: pid 53805 (check_bacula), uid 181: exited on 
signal 11

I tracked it down to a host which was not up:

$ time ./check_bacula -H tape02 -D fd -M nagios-mon -K '[redacted]'
Segmentation fault

real1m15.101s
user0m0.006s
sys 0m0.000s


Could someone else please try to replicate this situation for me please?

This check is being run on FreeBSD 12.0-RELEASE-p2 with check_bacula from 
Bacula 9.2.2

Thank you.

--
Dan Langille - BSDCan / PGCon
d...@langille.org




signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] FreeBSD: 7.x ports deprecated - expire 2019-03-01

2018-11-08 Thread Dan Langille
Hello,

I write with my plans for the Bacula 7.x packages on FreeBSD.

Bacula 7.x was last updated 20 months ago. Bacula 9.x came out shortly 
thereafter.

(re https://sourceforge.net/projects/bacula/files/bacula/)

On all the 7.x ports, I plan to set:

DEPRECATED=Unsupported, please install Bacula 9.x

and

EXPIRATION_DATE=2019-03-01

I chose that date because it is more than 3 months from now, and still gives 
users the FreeBSD quarterly ports branch until 2019Q2.

Those changes should appear later this week.

Thank you.

--
Dan Langille - BSDCan / PGCon
d...@langille.org




signature.asc
Description: Message signed with OpenPGP
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] FileSet estimate listing sorting

2018-09-24 Thread Dan Langille
> On Sep 23, 2018, at 5:27 AM, George Anchev via Bacula-users 
>  wrote:
> 
> On Sun, 23 Sep 2018 10:42:20 +0200 Kern Sibbald wrote:
> 
>> Direct the output the estimate listing to a file,
>> then sort it any way you want.
> 
> I actually do this but I am afraid it is not that
> simple because sorting it e.g. by filename (including
> path) or by date would require additional scripting.
> Hence the whole question.
> 
> I was hoping that the listing could be sorted by
> Bacula itself.

What use cases exist for sorting the estimate?




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Retaining files indefinitely ?

2018-08-29 Thread Dan Langille
> On Aug 28, 2018, at 2:54 AM, Chris Wilkinson  wrote:
> 
> One of my requirements for backups is that the last version of every file 
> should remain in the backup storage indefinitely or at least for a long time.
> 
> Could I achieve this by writing "File Retention = 10 years" in the pool or 
> client definition? 

Yes. But you'll also need Volume and Job retention also set to 10 years if you 
want to easily restore that data.

This will keep all records, not just the latest.

Yes, this will use more database records, but my view is that this is a cheap 
price to pay.

Also, as mentioned by others, does this requirement mean that a file deleted 6 
years ago needs to be kept for another 4 years?

You'll need to clarify that 10 years meets the "indefinitely" requirement.

Given the originally stated requirements, it seems your only solution is not to 
ever recycle volumes or allow the Catalog to be pruned.

> My concern is that, if this is possible, then it would stop automatic volume 
> truncation and recycling and eventually result in out of space storage errors.

Yes, you need to have enough storage.


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] any way to "collapse" back to a single director?

2018-08-14 Thread Dan Langille
> On Aug 11, 2018, at 10:47 AM, Phil Stracchino  wrote:
> 
> On 08/09/18 11:47, Matthew Arguin wrote:
>> But then you lose the historical stuff?  Was hoping for a way to sort of
>> migrate everything from one to another.  I will say that I don’t expect
>> that this is doable with out more work than it is worth.
> 
> 
> Moving the clients is easy.  Moving the historical data is hard.  As
> others have said, you must do it manually.  You would have to start out
> by changing the numeric IDs of every Job, every Volume, every Pool,
> every Client, every Fileset, every Schedule, every type of resource, in
> the catalog of the Director(s) you are planning to shut down to values
> not used by the one you're trying to keep, and you need to do it
> CONSISTENTLY, making sure every single record in the database refers to
> every other record it should by the correct *NEW* row IDs.  Then you
> need to import that data into the Catalog you're keeping without
> overwriting any data you already have.
> 
> None of this is anything you should be attempting unless you have strong
> SQL database skills and understand how the Bacula catalog tables relate
> to each other.  And of course, in all cases you should back up all of
> your catalog databases first.
> 
> 
> By comparison, you will probably find it a lot simpler to just add your
> Clients and their Filesets and Schedules to the Director you're keeping
> and restart it, then keep the other Directors around as backups in case
> you need them for restores until their historical data becomes outdated.

So far, thought has been given to merging the database.

Consider not merging them, but keeping them.

My idea:

* You can shutdown the other directors.  The historical data is in a database.

* You can keep the databases.

* A given director can using multiple Catalogs.

This might be a line of investigation.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Copy Job to Another Catalog

2018-07-20 Thread Dan Langille
> On Jul 20, 2018, at 6:26 PM, Dan Langille  wrote:
> 
> Signed PGP part
>> On Jul 19, 2018, at 1:41 PM, Jari Fredriksson  wrote:
>> 
>> Signed PGP part
>> 
>> 
>>> Heitor Faria  kirjoitti 18.7.2018 kello 20.42:
>>> 
>>> Dear Users,
>>> 
>>> I'm planning to deploy a Copy Job for Geographical Redundancy Disaster 
>>> Recovery (Site A, Site B).
>>> Failover site (B) has a secondary Bacula Director, Catalog and Storage 
>>> Daemon.
>>> Do you think it is possible to perform Copy jobs from Site A to Site B, 
>>> using the failover Catalog as the metadata repository?
>>> 
>> 
>> Hello
>> 
>> As far as I know a Copy job is not possible to be running on two storage 
>> daemons. That is what I learned when I had a need for it.
> 
> That was once the case, it is no longer.  I copy jobs between two bacula-sd 
> now.
> 

It changed in 7.0.0: 
http://www.bacula.org/9.0.x-manuals/en/main/New_Features_in_7_0_0.html#SECTION00611000

The documentation is incorrect: 
http://www.bacula.org/9.0.x-manuals/en/main/Migration_Copy.html#SECTION00283

"Migration is only implemented for a single Storage daemon. You cannot read on 
one Storage daemon and write on another."

--
Dan Langille - BSDCan / PGCon
d...@langille.org





signature.asc
Description: Message signed with OpenPGP
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Copy Job to Another Catalog

2018-07-20 Thread Dan Langille
> On Jul 19, 2018, at 1:41 PM, Jari Fredriksson  wrote:
> 
> Signed PGP part
> 
> 
>> Heitor Faria  kirjoitti 18.7.2018 kello 20.42:
>> 
>> Dear Users,
>> 
>> I'm planning to deploy a Copy Job for Geographical Redundancy Disaster 
>> Recovery (Site A, Site B).
>> Failover site (B) has a secondary Bacula Director, Catalog and Storage 
>> Daemon.
>> Do you think it is possible to perform Copy jobs from Site A to Site B, 
>> using the failover Catalog as the metadata repository?
>> 
> 
> Hello
> 
> As far as I know a Copy job is not possible to be running on two storage 
> daemons. That is what I learned when I had a need for it.

That was once the case, it is no longer.  I copy jobs between two bacula-sd now.



signature.asc
Description: Message signed with OpenPGP
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] disaster recovery - full OS?

2018-06-18 Thread Dan Langille
> On Jun 18, 2018, at 10:59 AM, czezz  wrote:
> 
> Hi,
> Does bacula offer full disaster recovery?

http://www.bacula.org/9.0.x-manuals/en/main/Disaster_Recovery_Using_Bac.html 
<http://www.bacula.org/9.0.x-manuals/en/main/Disaster_Recovery_Using_Bac.html>

That said, I always prefer to backup data and let me take care of setting up 
the system.  The backup may have occurred on one type of filesystem, parition 
layout, OS, and I want to restore to another, for example.

> What I have read up until now it seems that Bacula backups only files and 
> directories.
> It seems that in case full system restoration, fresh installation of OS is 
> required.
> 
> Is this correct?
> Is ther an option to completely image OS like eg. Clonezilla offers (I know 
> it is offline backup though)

Everything specializes in something.  Create your priorities, make your 
choices.  if getting a new system back up and running ASAP is your priority, 
restoring from backup is your biggest problem. Deployment is.

If you like what Clonezilla does, make sure it does everything you want.  For 
example, do you want, and does it do, incrementals, differentials, can it be 
used to restore the 400GB file which was just deleted.

Can it gives you a list of files in the backup, and their checksums, and 
timestamps so you can decide which one to restore?

This is just a small list of things to consider.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula h/w write fails, but tar writes w/out error?

2018-06-07 Thread Dan Langille
> On Jun 7, 2018, at 11:38 AM, Stieneke, Dan  wrote:
> 
> The job ate through 4 tapes, with only 2 – 60GB on each tape. Then it hit 
> recycle limits and was asking for more media.
>  
> These are used tapes, but I can’t see 4 consecutive tapes going bad at the 
> same time.
>  
> Incidentally, this is the same behavior I saw 4 months ago, and at that time 
> I did test bacula to a brand-new tape, which also failed quickly.


If it is all tapes, is the issue with the tape drive?
-- 
Dan Langille - BSDCan / PGCon
d...@langille.org

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] RunAfterJob

2018-06-06 Thread Dan Langille
> On Jun 6, 2018, at 10:48 AM, Donna Hofmeister  wrote:
> 
> In a job{} definition, is it possible/advisable/wise to have multiple 
> RunAfterJob statements?
> 
> If so, does N+1 wait on N to complete?  - d

I understand it is possible.

My preference would be just one. Make is a /bin/sh script and do everything in 
there.

Then you know.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Accidentally deleted a running jobid during a backup - Any hint on how to address?

2018-06-03 Thread Dan Langille
> On Jun 3, 2018, at 3:42 PM, Pedro Oliveira  <mailto:oliveira...@gmail.com>> wrote:
> 
> No dia 03/06/2018, às 16:13, Dan Langille  <mailto:d...@langille.org>> escreveu:
> 
>>> On Jun 3, 2018, at 1:23 AM, Ben Wong >> <mailto:benjamin.w...@blur.compbio.ucsf.edu>> wrote:
>>> 
>>> Hello, 
>>> 
>>> I was trying to clean up my failed jobs from my catalog but I happened to 
>>> accidentally delete the jobid of a running job.  Stupid, I know.  From what 
>>> I can see, the backup looks like it succeeded, it only appears failed due 
>>> to bacula being unable to find original jobid because I deleted it.  
>>> 
>>> Is there a simple way to retrieve that catalog data for the jobid I 
>>> deleted?  The backup is written across four LTO7 tape volumes.  I was 
>>> thinking of redoing the backup and not deleting the jobid but maybe there's 
>>> an option  where I don't have to rewrite all that data?  
>>> 
>>> Here's the error I get:  JobID 245 is the job I accidentally deleted.  
>>> 02-Jun 20:38 bacula-dir JobId 245: Warning: Error updating job record. 
>>> sql_update.c:202 Update failed: affected_rows=0 for UPDATE Job SET 
>>> JobStatus='T',EndTime='2018-06-02 
>>> 20:38:50',ClientId=8,JobBytes=40063503226963,ReadBytes=40063503226963,JobFiles=2684608,JobErrors=28,VolSessionId=8,VolSessionTime=1527615994,PoolId=4,FileSetId=8,JobTDate=1527997130,RealEndTime='2018-06-02
>>>  20:38:50',PriorJobId=0,HasBase=0,PurgedFiles=0 WHERE JobId=245
>>> 02-Jun 20:38 bacula-dir JobId 245: Warning: Error getting Job record for 
>>> Job report: ERR=sql_get.c:318 No Job found for JobId 245 
>> 
>> If you really need it, perhaps bextract will help.
>> 
>> Keep a fresh backup of your Catalog.

> Use Bscan to scan the volume with bsr file 
> 

Yes,  bscan, not bextract.  Sorry.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org <mailto:d...@langille.org>


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Accidentally deleted a running jobid during a backup - Any hint on how to address?

2018-06-03 Thread Dan Langille
> On Jun 3, 2018, at 1:23 AM, Ben Wong  
> wrote:
> 
> Hello, 
> 
> I was trying to clean up my failed jobs from my catalog but I happened to 
> accidentally delete the jobid of a running job.  Stupid, I know.  From what I 
> can see, the backup looks like it succeeded, it only appears failed due to 
> bacula being unable to find original jobid because I deleted it.  
> 
> Is there a simple way to retrieve that catalog data for the jobid I deleted?  
> The backup is written across four LTO7 tape volumes.  I was thinking of 
> redoing the backup and not deleting the jobid but maybe there's an option  
> where I don't have to rewrite all that data?  
> 
> Here's the error I get:  JobID 245 is the job I accidentally deleted.  
> 02-Jun 20:38 bacula-dir JobId 245: Warning: Error updating job record. 
> sql_update.c:202 Update failed: affected_rows=0 for UPDATE Job SET 
> JobStatus='T',EndTime='2018-06-02 
> 20:38:50',ClientId=8,JobBytes=40063503226963,ReadBytes=40063503226963,JobFiles=2684608,JobErrors=28,VolSessionId=8,VolSessionTime=1527615994,PoolId=4,FileSetId=8,JobTDate=1527997130,RealEndTime='2018-06-02
>  20:38:50',PriorJobId=0,HasBase=0,PurgedFiles=0 WHERE JobId=245
> 02-Jun 20:38 bacula-dir JobId 245: Warning: Error getting Job record for Job 
> report: ERR=sql_get.c:318 No Job found for JobId 245 

If you really need it, perhaps bextract will help.

Keep a fresh backup of your Catalog.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Intervention needed for backup

2018-05-26 Thread Dan Langille
> On May 26, 2018, at 5:39 PM, Heitor Faria <hei...@bacula.com.br> wrote:
> 
> Hello Dan and Donna,
> On May 25, 2018, at 1:31 PM, Donna Hofmeister <do...@allegro.com 
> <mailto:do...@allegro.com>> wrote:
> 
> 
> On Thu, May 24, 2018 at 9:26 AM, Donna Hofmeister <do...@allegro.com 
> <mailto:do...@allegro.com>> wrote:
> 
> Second interesting question -- given my "Label Format" statement, I would 
> expect my files to be named something like this: Vol-minerva.allegro.com 
> <http://vol-minerva.allegro.com/>-fd-File-20180517-2305-73-19. So why are my 
> files named: MinervaBackup.2018-05-23_23.05.00_53?
> 
> To answer my own question
> 
> I think the underlying issue was I had made several changes to the "label 
> format" statement while I've been testing Bacula...and had inadvertantly 
> created this situation.
> 
> To solve this, I wiped out my bacula storage environment and all seems to be 
> working well.
> 
> So -- reasking my second question -- if you change the "label format" 
> statement, is there additional steps after that?- donna 
> 
> 
> 
> Issue an update in bconsole.
> I think the update is not necessary when changing the Volume Format since 
> prior created volumes are already created and Bacula won't rename them.

How will bacula-dir know about the changed label format without issuing the 
update command?

> I also think Bacula could criticize erratic volume names when reloading. E.g. 
> a misspelled variable ${Poll} is accepted, and the user will only find the 
> error when a Job gets stalled unable to create new volumes.

Sanity testing? Sounds good.

> Finally, I think the Bacula configuration default Label Format should be 
> ${Pool}-${NumVols}. Vol- works, but it becomes ugly when you use multiple 
> Pools such as Daily, Weekly etc. since Volume numeration matches the MediaID. 
> Having the ${Pool} name on the Volume might also help to distinguish them in 
> a Bacula disaster recovery scenario.


I have never subscribed to the LOTS OF INFO IN THE VOLUME LABEL school of 
thought.  I use Pool names:

$ grep 'LabelFormat' *
pools.conf:  LabelFormat = "FullAuto-"
pools.conf:  LabelFormat = "DiffAuto-"
pools.conf:  LabelFormat = "IncrAuto-"
pools.conf:  LabelFormat = "IncrAutoNoNextPool-"
pools.conf:  LabelFormat = "FullAutoNoNextPool-"
pools.conf:  LabelFormat = "TwoHourly-"
pools.conf:  LabelFormat = "Daily-"
pools.conf:  LabelFormat = "Weekly-"
pools.conf:  LabelFormat = "Monthly-"

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Intervention needed for backup

2018-05-26 Thread Dan Langille
> On May 25, 2018, at 1:31 PM, Donna Hofmeister <do...@allegro.com> wrote:
> 
> 
> On Thu, May 24, 2018 at 9:26 AM, Donna Hofmeister <do...@allegro.com 
> <mailto:do...@allegro.com>> wrote:
> 
> Second interesting question -- given my "Label Format" statement, I would 
> expect my files to be named something like this: 
> Vol-minerva.allegro.com-fd-File-20180517-2305-73-19. So why are my files 
> named: MinervaBackup.2018-05-23_23.05.00_53?
> 
> To answer my own question
> 
> I think the underlying issue was I had made several changes to the "label 
> format" statement while I've been testing Bacula...and had inadvertantly 
> created this situation.
> 
> To solve this, I wiped out my bacula storage environment and all seems to be 
> working well.
> 
> So -- reasking my second question -- if you change the "label format" 
> statement, is there additional steps after that?- donna 
> 


Issue an update in bconsole.

The change will affect only future Volumes. Existing Volumes are never affected.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula 7+mysql

2018-05-26 Thread Dan Langille
> On May 23, 2018, at 1:02 PM, Chris Wilkinson <winstonia...@gmail.com> wrote:
> 
> This will seem a dumb question to the experienced Bacula user. I would prefer 
> to use mysql rather than the default sqlite. During installation (Debian 
> Stretch repo, Bacula v7.4.4) I said no to configuring sqlite expecting to be 
> offered the opportunity to configure mysql instead. That didn't happen. I 
> have created an empty bacula database with the appropriate user/permissions 
> but can't figure out how to make Bacula populate it or connect.

That sounds like a Debian packaging problem.

> 
> At present the director fails to start of course.
> 
> An extract from journalctl -xe follows indicating it is looking for sqlite.
> 
> -- Unit bacula-director.service has begun starting up.
> May 23 17:40:41 HOMESERVER bacula-dir[16139]: bacula-dir: dird.c:969-0 Could 
> not open Catalog "MyCatalog", databasMay 23 17:40:41 HOMESERVER 
> bacula-dir[16139]: bacula-dir: dird.c:974-0 sqlite.c:199 Database 
> /var/lib/bacula/XXX_DMay 23 17:40:41 HOMESERVER bacula-dir[16139]: 23-May 
> 17:40 bacula-dir ERROR TERMINATION
> May 23 17:40:41 HOMESERVER bacula-dir[16139]: Please correct configuration 
> file: /etc/bacula/bacula-dir.conf
> May 23 17:40:41 HOMESERVER systemd[1]: bacula-director.service: Control 
> process exited, code=exited status=1
> May 23 17:40:41 HOMESERVER systemd[1]: Failed to start Bacula Director Daemon 
> service.
> -- Subject: Unit bacula-director.service has failed
> -- Defined-By: systemd
> -- Support: https://www.debian.org/support <https://www.debian.org/support>
> --
> -- Unit bacula-director.service has failed.
> --
> -- The result is failed.
> May 23 17:40:41 HOMESERVER systemd[1]: bacula-director.service: Unit entered 
> failed state.
> May 23 17:40:41 HOMESERVER systemd[1]: bacula-director.service: Failed with 
> result 'exit-code'.
> May 23 17:47:01 HOMESERVER minissdpd[716]: 4 new devices added
> lines 2591-2644/2644 (END)
> 
> 
> This seems to indicate that bacula-dir is wrong but I haven't found an 
> example of what it should be.
> 
> Any help with that would be great.

Does this help? 
http://www.bacula.org/7.4.x-manuals/en/main/Installing_Configuring_MySQ.html 
<http://www.bacula.org/7.4.x-manuals/en/main/Installing_Configuring_MySQ.html>
Bacula will not create the required tables etc. You must do that.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] ERR=Operation timed out

2018-05-17 Thread Dan Langille
> On May 16, 2018, at 2:44 PM, Elam <elm.c...@gmail.com> wrote:
> 
> Hello everyone,
> 
> I have recently received the following error for a bacula job:
> 
> Fatal error: Network error with FD during Backup: ERR=Operation timed out
> Fatal error: No Job status returned from FD.
> 
> bacula starts the backup job, it runs for a few mins, ( I could see the 
> backup progress) but it terminate all of a sudden while reporting the above 
> error.
> 

This sounds like network issues.  What does 'status client' show for this FD?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] BSCAN - what to expect?

2018-04-08 Thread Dan Langille
> On Apr 7, 2018, at 5:30 PM, Thomas plancon <tplan...@bkaarchs.com> wrote:
> 
> Hi Dan,
> 
> Thanks for your interest!
> 
> So I let bscan run  for almost 48 hours and then thought I should try and see 
> if any records were added to the database: nothing was added except a job 
> number and the volume name! I cancelled bscan and went into bconsole to 
> verify and yes, nothing except as mentioned above.

One thing to consider: automated pruning.  I recall previous posts saying 
records would get added, but later get pruned because they are too old.

> I tried bls, and that was actually pulling file names off of the tape and I 
> redirected it to a text file. From that info I ran bextract with the name of 
> a project retrieved with bls. Bextract successfully restored the project to a 
> local directory!!!
> 
> So now I'm letting bls run on the tapes and collecting the list in text 
> files. Hoping I'll see the project I need to recover. Unfortunately, bls is 
> also a very slow process; 48 hours and still not through listing a LTO-2, 400 
> Gb, tape; I've got 3 tapes to go through!

I think that is because, like bscan, it reads the whole tape.

> It is frustrating because I'm not even sure the needed project is even on 
> these tapes. The backup was full, so I'm just deducing, and hoping, (that 
> could be on a t-shirt: deducing and hoping), that it is there somewhere.
> 
> I'll post again in a day or so with the status.

Good.


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



> 
> Tom Plancon
> 
> 
> 
> On 4/7/2018 3:57 PM, Dan Langille wrote:
>>> On Apr 3, 2018, at 11:56 AM, Tom Plancon <tplan...@bkaarchs.com> wrote:
>>> 
>>> Hi folks,
>>> 
>>> I'm running BSCAN to recover old data from LTO-2 tapes. The backup job 
>>> spanned 3 tapes/volumes. I ran the BSCAN command listing the volumes as 
>>> required with the first tape in the drive - this is NOT an autochanger. 
>>> BSCAN seemed to start OK, found the tape, dbase etc., and has been running 
>>> now for about 24hrs with the first tape still in the drive!
>> bscan does take a very long time. I have no personal experience with it 
>> though.
>> 
>>> So, my questions, will BSCAN ask for the next tape when the first is done? 
>>> Or, how will I know how to change it? Should it be taking this long for a 
>>> 400Gb tape? The tape drive moves for a few seconds, pauses a few seconds 
>>> and moves again, now for a full day!
>>> 
>>> I'm trying to recover a specific project in this old data, there is 
>>> probably over 100 projects saved on these tapes. Is there a better way to 
>>> do this?
>>> 
>>> Any help appreciated! Thanks much!
>> It's been a few days. Any progress to report?
>> 
>> 
> 


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Autochanger not changing

2018-04-07 Thread Dan Langille
> On Apr 1, 2018, at 11:12 AM, Sebastian Suchanek <bac...@suchanek.de> wrote:
> 
> Hi everyone,
> 
> while still setting up Bacula (v5.2.6), I've come accross a rather
> strange issue with my tape library (an Overland NEO2000) with a
> LTO4-FC-Drive. When I do "fill" test with btape over multiple tapes,
> everything works fine. Also, for example, when I do a "label barcodes"
> command in btape over multiple slots, the individual tapes are moved
> into the drive, get written and are put back into their original slot.
> 
> But when I run a "real" backup job which spans more than one tape, the
> job gets stuck as soon as the end of the first tape is reached. The last
> messages from the job are:
> 
> | [...]
> | 2018-04-01 14:16:42 tigersclaw-sd JobId 111: Volume "SU4103L4"
> previously written, moving to end of data.
> | 2018-04-01 15:55:32 tigersclaw-sd JobId 111: End of Volume 
> "SU4103L4"
> at 21:2548 on device "LTO4-Drive-1" (/dev/nst0). Write of 200 bytes
> got -1.
> 
> And after that, nothing happens anymore. Not even if I wait for half an
> hour.
> 
> Here's the (hopefully) relevant part of my bacula-sd.conf:
> 

[snip]

> 
> Any idea what exactly is going wrong here? What can I do to track down
> the reason for this error? What can I do to fix it?

It is usually permissions. It it nearly always permission.

I have documented my experiences here: https://dan.langille.org/

Search for mtx and mtx-loader.  There are several configuration step and low 
level testing at the command line.


This might be your best bet: 
https://dan.langille.org/2016/02/04/dell-tl4000-tape-library-running-btape-and-configuring-baculas-mtx-changer/

start slow.  Make sure mtx-changer works. If you need to customize it, move it 
away from /etc/bacula/scripts/mtx-changer ... I put mine at /usr/local/sbin 
well away from the installed one.

Make sure all the steps with mtx-changer work: load, unload, loaded, list

Then make sure they work as the bacula user (your SD should be running as 
bacula, not root; there is no need for bacula-sd to run as root).

To get mtx-changer working, first make sure mtx works.

Try what I tried and make sure it all works.

HTH


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] BSCAN - what to expect?

2018-04-07 Thread Dan Langille
> On Apr 3, 2018, at 11:56 AM, Tom Plancon  wrote:
> 
> Hi folks,
> 
> I'm running BSCAN to recover old data from LTO-2 tapes. The backup job 
> spanned 3 tapes/volumes. I ran the BSCAN command listing the volumes as 
> required with the first tape in the drive - this is NOT an autochanger. BSCAN 
> seemed to start OK, found the tape, dbase etc., and has been running now for 
> about 24hrs with the first tape still in the drive!

bscan does take a very long time. I have no personal experience with it though.

> So, my questions, will BSCAN ask for the next tape when the first is done? 
> Or, how will I know how to change it? Should it be taking this long for a 
> 400Gb tape? The tape drive moves for a few seconds, pauses a few seconds and 
> moves again, now for a full day! 
> 
> I'm trying to recover a specific project in this old data, there is probably 
> over 100 projects saved on these tapes. Is there a better way to do this?
> 
> Any help appreciated! Thanks much!

It's been a few days. Any progress to report?



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Using LibreSSL?

2018-03-09 Thread Dan Langille
> On Feb 9, 2018, at 1:34 PM, Martin Simmons <mar...@lispworks.com> wrote:
> 
>>>>>> On Wed, 7 Feb 2018 20:33:49 -0500, Dan Langille said:
>> 
>>> On Nov 30, 2017, at 11:41 AM, Martin Simmons <mar...@lispworks.com> wrote:
>>> 
>>> It looks like LibreSSL defines OPENSSL_VERSION_NUMBER as 0x2000L, i.e. 
>>> as
>>> if it was OpenSSL 2.0.  Bacula uses this variable to detect OpenSSL >= 1.1,
>>> which causes it to compile the 1.1 code when using LibreSSL, even though
>>> LibreSSL only claims to provide the API from OpenSSL 1.0.
>>> 
>>> Bacula probably needs to detect LibreSSL (e.g. look for
>>> LIBRESSL_VERSION_NUMBER) and treat that as OpenSSL 1.0 regardless of
>>> OPENSSL_VERSION_NUMBER.
>> 
>> This seems to fit closely with a patch provided to FreeBSD: 
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223994
>> 
>> I could file a bug and pass this directly to the Bacula project, and in the 
>> meantime, patch the FreeBSD port.
>> 
>> Both will take time and I'm preoccupied with conferences just now.
> 
> That patch looks incomplete though (there are a few other uses of
> OPENSSL_VERSION_NUMBER).

New patches have been uploaded to 
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223994

Does that look better?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Using LibreSSL?

2018-02-07 Thread Dan Langille
> On Nov 30, 2017, at 11:41 AM, Martin Simmons <mar...@lispworks.com> wrote:
> 
> It looks like LibreSSL defines OPENSSL_VERSION_NUMBER as 0x2000L, i.e. as
> if it was OpenSSL 2.0.  Bacula uses this variable to detect OpenSSL >= 1.1,
> which causes it to compile the 1.1 code when using LibreSSL, even though
> LibreSSL only claims to provide the API from OpenSSL 1.0.
> 
> Bacula probably needs to detect LibreSSL (e.g. look for
> LIBRESSL_VERSION_NUMBER) and treat that as OpenSSL 1.0 regardless of
> OPENSSL_VERSION_NUMBER.

This seems to fit closely with a patch provided to FreeBSD: 
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223994

I could file a bug and pass this directly to the Bacula project, and in the 
meantime, patch the FreeBSD port.

Both will take time and I'm preoccupied with conferences just now.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




> 
> 
> 
>>>>>> On Thu, 30 Nov 2017 09:53:33 -0500, Dan Langille said:
>> 
>> Are you using LibreSSL with Bacula 9.0.6?
>> 
>> I ask because of https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223994
>> 
>> re:
>> 
>> 
>> --- openssl.lo ---
>> Compiling openssl.c
>> --- crypto.lo ---
>> crypto.c:199:1: error: unknown type name 'DEFINE_STACK_OF'
>> DEFINE_STACK_OF(SignerInfo);
>> ^
>> crypto.c:200:1: error: unknown type name 'DEFINE_STACK_OF'
>> DEFINE_STACK_OF(RecipientInfo);
>> ^
>> crypto.c:334:21: error: use of undeclared identifier 'ASN1_STRING_get0_data';
>> did you mean 'ASN1_STRING_data'?
>>  ext_value_data = ASN1_STRING_get0_data(asn1_ext_val);
>>   ^
>>   ASN1_STRING_data
>> /usr/local/include/openssl/asn1.h:787:17: note: 'ASN1_STRING_data' declared
>> here
>> unsigned char * ASN1_STRING_data(ASN1_STRING *x);
>>   ^
>> crypto.c:334:43: error: cannot initialize a parameter of type 'ASN1_STRING *'
>> (aka 'asn1_string_st *') with an lvalue of type 'const ASN1_STRING *' (aka
>> 'const asn1_string_st *')
>>  ext_value_data = ASN1_STRING_get0_data(asn1_ext_val);
>> 
>> 
>> -- 
>> Dan Langille - BSDCan / PGCon
>> d...@langille.org
>> 
>> 
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> 
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-07 Thread Dan Langille
> On Feb 7, 2018, at 1:04 PM, Dimitri Maziuk <dmaz...@bmrb.wisc.edu> wrote:
> 
> On 02/07/2018 11:57 AM, Dan Langille wrote:
>> Either Bacula is in charge of the changer or it is not.
> 
> Only when it's running a job. If access is not concurrent, then it's
> very likely the tool not cleaning up after itself properly or something.

What I mean: either use Bacula or not use Bacula.

The other tool is taking hold of the device and not releasing.  That is 
interfering with backups.

So.. make a decision: Bacula or not.

Then go from there.

--
Dan Langille - BSDCan / PGCon
d...@langille.org





signature.asc
Description: Message signed with OpenPGP
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-07 Thread Dan Langille
Either Bacula is in charge of the changer or it is not.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


> On Feb 7, 2018, at 12:48 PM, Kenneth Garges <gar...@ucsc.edu> wrote:
> 
> That might be the issue. Somebody does something in the web interface that 
> takes the changer offline, clicks OK at the warning message. Then later when 
> it’s back online FreeBSD doesn’t figure that out and needs a reboot. 
> 
> Probably an issue that should be addressed with FreeBSD rather than Bacula.
> 
> 
> 
> 
>> On 6Feb 2018, at 6:50 PM, Alan Brown <a.br...@ucl.ac.uk> wrote:
>> 
>> On 07/02/18 02:01, Kenneth Garges wrote:
>>> Permissions and ownership are ok. I think I figured out the problem.
>>> 
>>> Both mtx and chio would work for a while, then stop working returning only 
>>> “Inappropriate ioctl for device” or other errors. The culprit I think is a 
>>> library management tool by Quantum. The library provides a web interface 
>>> that allows you to check status, move tapes around, and various utilities. 
>>> 
>>> Problem is if you use that web interface it gets the server (both Bacula SD 
>>> and Director on the same FreeBSD box) wedged such that mtx and chio always 
>>> fail.
>> 
>> Yes - and if it's about to do anything which takes the changer offline
>> the web ui gives a warning message and asked if you want to continue.
>> 
>>> Unknown to me, the operations staff had been using that web interface while 
>>> I was testing. After I asked them to stop my system seems to work reliably. 
>> 
>> I hope you changed the password and put the changer in its own IP subnet
>> after that.
>> 
>> I wrote some kludgy shell scripts (originally for Neo4000, then Neo 8000
>> and now Quantum i500 - portable as far as I know) which automate loading
>> and unloading of the changer (imported tapes are scattered randomly
>> around available slots to ensure even wear and tear) - these mean that
>> all operations staff have to do is open the magazine to load/unload
>> tapes when emailed to do so by Bacula.
>> 
>> Dan can probably improve them considerably.
>> 
>> As an adjunct to that another script works out which tapes are oldest
>> (if in the scratch pool) or about to expire (if none are in the scratch
>> pool) and asks staff to pull them out of the safe.
>> 
>> I also submitted some chages to the mtx-changer script a while ago which
>> check that the changer's actually ready before attempting to send it
>> commands. Kern's been sitting on them for a couple of years.
>> 
>> 
>> There are some pretty whizzy things you can do to monitor tape health
>> when a tape is in the changer using the sg_attr and sg_logs commands
>> that go _far _ beyond Tapealart or smartmontools capabilities.
>> 
>> I've submitted a number of updates to sg_utils and am in the process of
>> (slowly) reverse engineering what I can from the MAMs that's not well
>> documented using text output provided by veritape (proprietary, windows
>> tool from mptapes.com which reads MAMs).
>> 
>> Selfish motivation is (of course) to reduce the amount of time I have to
>> spend manually scanning tapes with the windows program/standalone
>> scanner then interpreting output. All of that information is available
>> when the tape is in the drive and with appropriate MAM reads at
>> load/unload it's possible to have the bacula server tell me when a tape
>> is approaching end of life (this is far more accurate than the metrics
>> bacula uses of simple load cycles.) - and more importantly, when tape
>> drives are going bad.
>> 
>> Having spent several man-years dealing with the fallout of bad LTO tapes
>> damaging drives and then those drives damaging tapes, I want to minimize
>> the pain if it ever happens again.
>> 
>> (LTO drives even return detailed information about the condition of each
>> of the heads if queried the right way. This data is interpreted by IBM
>> or HP's proprietary tools to give reports but they use the same queries
>> as sg_logs does to get it)
>> 
> 


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-06 Thread Dan Langille
> On Feb 6, 2018, at 9:01 PM, Kenneth Garges <gar...@ucsc.edu> wrote:
> 
> Permissions and ownership are ok. I think I figured out the problem.
> 
> Both mtx and chio would work for a while, then stop working returning only 
> “Inappropriate ioctl for device” or other errors. The culprit I think is a 
> library management tool by Quantum. The library provides a web interface that 
> allows you to check status, move tapes around, and various utilities. 
> 
> Problem is if you use that web interface it gets the server (both Bacula SD 
> and Director on the same FreeBSD box) wedged such that mtx and chio always 
> fail.
> 
> Unknown to me, the operations staff had been using that web interface while I 
> was testing. After I asked them to stop my system seems to work reliably. 
> 
> Thanks to all that chimed in on this thread. It was very helpful.

Wow, that would have been hard to track down.  How did you stumble across that?


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-06 Thread Dan Langille

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


> On Feb 6, 2018, at 7:07 AM, Alan Brown <a.br...@ucl.ac.uk> wrote:
> 
> On 03/02/18 13:45, Dan Langille wrote:
>>> *ahem*
>>> 
>>> MTX is _only_ designed to interface to scsi generic devices. Unless 
>>> /dev/pass42 is another name for the generic device you'll have problems.
>> Can confirm /dev/pass* devices on FreeBSD are the right thing to use. From 
>> https://dan.langille.org/2016/02/04/dell-tl4000-tape-library-running-btape-and-configuring-baculas-mtx-changer/
> 
> What about permissions and ownership vs what bacula-sd is running as?

Good point.

I see that's not covered in the above, but is here: 
https://dan.langille.org/2016/02/07/using-a-dell-tl4000-with-baculas-bacula-sd-on-freebsd-with-zfs/

Look for /etc/group
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-03 Thread Dan Langille
> On Feb 2, 2018, at 7:07 AM, Alan Brown  wrote:
> 
> On 31/01/18 03:23, Kenneth Garges wrote:
>> I’m having trouble getting a tape library to work. Individual tape drive 
>> seems ok but the library sometimes makes btape hang requiring a reboot. Or 
>> it fails.
> 
> 
> >  # Define a Virtual autochanger
> 
> Remove anything you're not actually using.  They're just going to add 
> confusion.
> 
>> # Quantum Scalar i80 tape library at UCSC
>> # Hostname of library: core-prod-jb-2
>> #
>> Autochanger {
>>   Name = "core-prod-jb-2"
>>   Device = "Drive1"
>> # Device = "Drive2"
>>   Changer Command = "/usr/local/etc/bacula/mtx-changer %c %o %S %a %d"
>> #  Changer Device = /dev/ch0
>>   Changer Device = /dev/pass42
>> }
> 
> *ahem*
> 
> MTX is _only_ designed to interface to scsi generic devices. Unless 
> /dev/pass42 is another name for the generic device you'll have problems.

Can confirm /dev/pass* devices on FreeBSD are the right thing to use. From 
https://dan.langille.org/2016/02/04/dell-tl4000-tape-library-running-btape-and-configuring-baculas-mtx-changer/

$ sudo camcontrol devlist
 at scbus0 target 0 lun 0 (pass0,sa0)
 at scbus0 target 0 lun 1 (pass1,ch0)
 at scbus0 target 1 lun 0 (pass2,sa1)
 at scbus0 target 1 lun 1 (pass3,ch1)
at scbus1 target 0 lun 0 (ada0,pass4)
at scbus3 target 0 lun 0 (ada1,pass5)

CAM is the Common Access Method for the SCSI/ATA subsystem: 
https://www.freebsd.org/cgi/man.cgi?query=cam=4=FreeBSD+11.0-RELEASE


$ sudo mtx -f /dev/pass0 inquiry
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULT3580-HH4 '
Revision: 'C7Q1'
Attached Changer API: No
 
$ sudo tapeinfo -f /dev/pass0
Product Type: Tape Drive
Vendor ID: 'IBM '
Product ID: 'ULT3580-HH4 '
Revision: 'C7Q1'
Attached Changer API: No
SerialNumber: '1K10012996'
MinBlock: 1
MaxBlock: 16777215
Ready: yes
BufferedMode: yes
Medium Type: 0x48
Density Code: 0x46
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
BOP: yes
Block Position: 0
Partition 0 Remaining Kbytes: -1
Partition 0 Size in Kbytes: -1
ActivePartition: 0
EarlyWarningSize: 0
 
$ sudo tapeinfo -f /dev/pass1
Product Type: Medium Changer
Vendor ID: 'IBM '
Product ID: '3573-TL '
Revision: 'B.60'
Attached Changer API: No
SerialNumber: '00X4U78G4272_LL0'
Ready: yes


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-02-01 Thread Dan Langille
> On Feb 1, 2018, at 6:42 PM, Kenneth Garges <gar...@ucsc.edu> wrote:
> 
> This is really weird. 
> 
> First mtx works:
> 
> # mtx -f /dev/pass42 status
>   Storage Changer /dev/pass42:2 Drives, 54 Slots ( 5 Import/Export )
> Data Transfer Element 0:Empty
> Data Transfer Element 1:Full (Storage Element 1 Loaded):VolumeTag = 000600
>   
>   Storage Element 1:Empty:VolumeTag=
>   Storage Element 2:Empty:VolumeTag=
>   Storage Element 3:Empty:VolumeTag=
>   Storage Element 4:Empty:VolumeTag=
>   Storage Element 5:Empty:VolumeTag=
>   Storage Element 6:Empty:VolumeTag=
>   Storage Element 7:Empty:VolumeTag=
>   Storage Element 8:Empty:VolumeTag=
>   Storage Element 9:Empty:VolumeTag=
>   Storage Element 10:Empty:VolumeTag=
>   Storage Element 11:Empty:VolumeTag=
>   Storage Element 12:Empty:VolumeTag=
>   Storage Element 13:Empty:VolumeTag=
>   Storage Element 14:Empty:VolumeTag=
>   Storage Element 15:Full :VolumeTag=B1  
>   Storage Element 16:Full :VolumeTag=B2  
>   Storage Element 17:Full :VolumeTag=B3  
>   Storage Element 18:Full :VolumeTag=B4  
>   Storage Element 19:Empty:VolumeTag=
>   Storage Element 20:Empty:VolumeTag=
>   Storage Element 21:Empty:VolumeTag=
>   Storage Element 22:Empty:VolumeTag=
>   Storage Element 23:Empty:VolumeTag=
>   Storage Element 24:Full :VolumeTag=B5  
>   Storage Element 25:Empty:VolumeTag=
>   Storage Element 26:Empty:VolumeTag=
>   Storage Element 27:Empty:VolumeTag=
>   Storage Element 28:Empty:VolumeTag=
>   Storage Element 29:Empty:VolumeTag=
>   Storage Element 30:Empty:VolumeTag=
>   Storage Element 31:Empty:VolumeTag=
>   Storage Element 32:Empty:VolumeTag=
>   Storage Element 33:Empty:VolumeTag=
>   Storage Element 34:Empty:VolumeTag=
>   Storage Element 35:Empty:VolumeTag=
>   Storage Element 36:Empty:VolumeTag=
>   Storage Element 37:Empty:VolumeTag=
>   Storage Element 38:Empty:VolumeTag=
>   Storage Element 39:Empty:VolumeTag=
>   Storage Element 40:Empty:VolumeTag=
>   Storage Element 41:Empty:VolumeTag=
>   Storage Element 42:Empty:VolumeTag=
>   Storage Element 43:Empty:VolumeTag=
>   Storage Element 44:Empty:VolumeTag=
>   Storage Element 45:Empty:VolumeTag=
>   Storage Element 46:Empty:VolumeTag=
>   Storage Element 47:Empty:VolumeTag=
>   Storage Element 48:Empty:VolumeTag=
>   Storage Element 49:Empty:VolumeTag=
>   Storage Element 50 IMPORT/EXPORT:Empty:VolumeTag=   
>  
>   Storage Element 51 IMPORT/EXPORT:Empty:VolumeTag=   
>  
>   Storage Element 52 IMPORT/EXPORT:Empty:VolumeTag=   
>  
>   Storage Element 53 IMPORT/EXPORT:Empty:VolumeTag=   
>  
>   Storage Element 54 IMPORT/EXPORT:Empty:VolumeTag=   
>      
> 
> Try it again immediately afterwards and it doesn’t:

I suspect timing.  Set the sleep option in mtx-changer's config

It can take a while for a tape to settle and for the tape drive to be ready.
-- 
Dan Langille - BSDCan 

Re: [Bacula-users] Problems getting a tape library to work under FreeBSD

2018-01-30 Thread Dan Langille
ote 1 EOF to "Drive1" (/dev/nsa0)
> btape: btape.c:582-0 Rewound "Drive1" (/dev/nsa0)
> btape: btape.c:1644-0 Now forward spacing 1 file.
> We should be in file 1. I am at file 1. This is correct!
> btape: btape.c:1656-0 Now forward spacing 2 files.
> We should be in file 3. I am at file 3. This is correct!
> btape: btape.c:582-0 Rewound "Drive1" (/dev/nsa0)
> btape: btape.c:1669-0 Now forward spacing 4 files.
> We should be in file 4. I am at file 4. This is correct!
> 
> btape: btape.c:1687-0 Now forward spacing 1 more file.
> We should be in file 5. I am at file 5. This is correct!
> 
> === End Forward space files test ===
> 
> 
> Ah, I see you have an autochanger configured.
> To test the autochanger you must have a blank tape
>  that I can write on in Slot 1.
> 
> Do you wish to continue with the Autochanger test? (y/n): y
> 
> 
> === Autochanger test ===
> 
> 3301 Issuing autochanger "loaded" command.
> 3991 Bad autochanger command: /usr/local/etc/bacula/mtx-changer /dev/pass42 
> loaded 1 /dev/nsa0 0
> 3991 result="mtx: Request Sense: Long Report=yes
> ": ERR=Child exited with code 1
> You must correct this error or the Autochanger will not work.
> *quit
> btape: dev.c:271-0 close_dev vol= fd=3 dev=802c6bc28 adata=0 dev="Drive1" 
> (/dev/nsa0)
> Pool   Maxsize  Maxused  Inuse
> NoPool  2568  0
> NAME1300  0
> FNAME   256   11  0
> MSG   645124  0
> EMSG   10243  0
> BSOCK  40960  0
> 
> # /usr/local/etc/bacula/mtx-changer /dev/pass42 loaded 1 /dev/nsa0 0
> mtx: Request Sense: Long Report=yes
> mtx: Request Sense: Valid Residual=no
> mtx: Request Sense: Error Code=0 (Unknown?!)
> mtx: Request Sense: Sense Key=No Sense
> mtx: Request Sense: FileMark=no
> mtx: Request Sense: EOM=no
> mtx: Request Sense: ILI=no
> mtx: Request Sense: Additional Sense Code = 00
> mtx: Request Sense: Additional Sense Qualifier = 00
> mtx: Request Sense: BPV=no
> mtx: Request Sense: Error in CDB=no
> mtx: Request Sense: SKSV=no
> INQUIRY command Failed
> 
> The FreeBSD version of mtx also fails. 
> 
> # /usr/local/sbin/mtx -f /dev/pass42 inquiry
> mtx: Request Sense: Long Report=yes
> mtx: Request Sense: Valid Residual=no
> mtx: Request Sense: Error Code=0 (Unknown?!)
> mtx: Request Sense: Sense Key=No Sense
> mtx: Request Sense: FileMark=no
> mtx: Request Sense: EOM=no
> mtx: Request Sense: ILI=no
> mtx: Request Sense: Additional Sense Code = 00
> mtx: Request Sense: Additional Sense Qualifier = 00
> mtx: Request Sense: BPV=no
> mtx: Request Sense: Error in CDB=no
> mtx: Request Sense: SKSV=no
> INQUIRY Command Failed
> # /usr/local/sbin/mtx -f /dev/pass42 status
> mtx: Request Sense: Long Report=yes
> mtx: Request Sense: Valid Residual=no
> mtx: Request Sense: Error Code=0 (Unknown?!)
> mtx: Request Sense: Sense Key=No Sense
> mtx: Request Sense: FileMark=no
> mtx: Request Sense: EOM=no
> mtx: Request Sense: ILI=no
> mtx: Request Sense: Additional Sense Code = 00
> mtx: Request Sense: Additional Sense Qualifier = 00
> mtx: Request Sense: BPV=no
> mtx: Request Sense: Error in CDB=no
> mtx: Request Sense: SKSV=no
> INQUIRY command Failed
> 
> 
> And using /dev/ch0 instead of /dev/pass42 doesn’t seem to help.
> 
> # /usr/local/sbin/mtx -f /dev/ch0 status
> cannot open SCSI device '/dev/ch0' - Input/output error
> 
> # /usr/local/etc/bacula/mtx-changer /dev/ch0 loaded 1 /dev/nsa0 0
> cannot open SCSI device '/dev/ch0' - Input/output error

I have documented most of my setup procedure on FreeBSD.

Getting mtx working first.  Then mtx-changers.  Verify permissions on devices 
for the bacula user.

These might help, based on https://dan.langille.org/?s=mtx-changer=

https://dan.langille.org/2016/02/04/dell-tl4000-tape-library-running-btape-and-configuring-baculas-mtx-changer/

This example has more on permissions:

https://www.freebsddiary.org/tape-library-integration.php

Hope that gets you started.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] problems with storage daemon

2018-01-11 Thread Dan Langille
> On Jan 11, 2018, at 11:56 AM, Jerry Lowry <michaiah2...@gmail.com> wrote:
> 
> Hi,  Last weekend I ran through a bunch of updates for my backup server.  Of 
> the many updates it updated the kernel and MariaDB ( 10.2.8 -> 10.2.12 ). I 
> have recently upgraded bacula to 9.0.6.  The problem is that the following 
> error:
>  10-Jan 20:36 distress-dir JobId 34724: Fatal error: Bad response to Storage 
> command: wanted 2000 OK storage
> , got 2902 Bad storage
> 
> I also get errors saying that the client can not connect to the storage 
> server:
> 
> 11-Jan 01:37 Denial-fd: BackupDenial.2018-01-11_00.05.01_44 Fatal error: 
> ../../lib/bnet.c:870 Unable to connect to Storage daemon on 
> distress.ACCOUNTING.EDT.LOCAL:9103. ERR=No error
> 11-Jan 01:38 distress-dir JobId 34726: Fatal error: Socket error on Storage 
> command: ERR=No data available
> 11-Jan 01:38 distress-dir JobId 34726: Fatal error: No Job status returned 
> from FD.
> 
> nothing was changed in the config files, just the updates were installed. I 
> can ping the storage server from all of the clients.
> 
> The jobs get to the point where they display "waiting for client"  but the 
> jobs never complete.
> 

When you did those update, did you keep all bacula-dir and all bacula-sd on the 
same exact version?

The general rule for software versions is:

bacula-dir = bacula-sd >= bacula-fd

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] SD fails to create new volumes: trace included

2018-01-09 Thread Dan Langille
> On Jan 9, 2018, at 3:07 PM, Dan Langille <d...@langille.org> wrote:
> 
>> On Jan 9, 2018, at 3:01 PM, Dan Langille <d...@langille.org> wrote:
>> 
>> I wonder if this line is related:
>> 
>> 
>> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
>> 
>> If no labeling, then no volume is created.
>> 
>> See 
>> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-bacula-trace-L84
>> 
>> 
>> 
>> bacula-sd-01-sd: mount.c:182-269534 want vol=FullAuto-8867 devvol= 
>> dev="vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile)
>> bacula-sd-01-sd: mount.c:190-269534 Try open Vol=FullAuto-8867
>> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
>> file_dev::open_device(DCR *, int)
>> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
>> DEVICE::open_device(DCR *, int)
>> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
>> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
>> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
>> ERR=No such file or directory
>> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
>> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
>> file_dev::open_device(DCR *, int)
>> bacula-sd-01-sd: mount.c:192-269534 Try autolabel Vol=FullAuto-8867
>> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
>> file_dev::open_device(DCR *, int)
>> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool 
>> DEVICE::open_device(DCR *, int)
>> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
>> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
>> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
>> ERR=No such file or directory
>> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
>> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
>> file_dev::open_device(DCR *, int)
>> bacula-sd-01-sd: mount.c:198-269534 open_device failed: ERR=file_dev.c:190 
>> Could not 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
>> ERR=No such file or directory
>> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
>> bacula-sd-01-sd: mount.c:215-269534 Open of File device "vDrive-FullFile-8" 
>> (/usr/local/bacula/volumes/FullFile) Volume "FullAuto-8867" failed: 
>> ERR=file_dev.c:190 Could not 
>> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
>> ERR=No such file or directory
>> 
>> bacula-sd-01-sd: mount.c:220-269534 Volume "FullAuto-8867" not loaded on 
>> File device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile).
> 
> This is why it is not labelling, from bacula/src/stored/mount.c
> 
> int DCR::try_autolabel(bool opened)
> {
>   DCR *dcr = this;
> 
>   if (dev->poll && !dev->is_tape()) {
>  Dmsg0(100, "No autolabel because polling.\n");
>  return try_default;   /* if polling, don't try to create new labels 
> */
>   }
> 

To get this backup running again:

I manually deleted the last volume which failed to create (it was in the 
Catalog but not in the filesystem).

I manually labeled that Volume (which creates a Catalog entry and a file on 
disk).

https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-1319

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] SD fails to create new volumes: trace included

2018-01-09 Thread Dan Langille

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


> On Jan 9, 2018, at 3:01 PM, Dan Langille <d...@langille.org> wrote:
> 
> I wonder if this line is related:
> 
> 
> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> 
> If no labeling, then no volume is created.
> 
> See 
> https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-bacula-trace-L84
> 
> 
> 
> bacula-sd-01-sd: mount.c:182-269534 want vol=FullAuto-8867 devvol= 
> dev="vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile)
> bacula-sd-01-sd: mount.c:190-269534 Try open Vol=FullAuto-8867
> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> file_dev::open_device(DCR *, int)
> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool DEVICE::open_device(DCR 
> *, int)
> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
> ERR=No such file or directory
> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> file_dev::open_device(DCR *, int)
> bacula-sd-01-sd: mount.c:192-269534 Try autolabel Vol=FullAuto-8867
> bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
> file_dev::open_device(DCR *, int)
> bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool DEVICE::open_device(DCR 
> *, int)
> bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
> bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
> bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
> ERR=No such file or directory
> bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
> bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
> file_dev::open_device(DCR *, int)
> bacula-sd-01-sd: mount.c:198-269534 open_device failed: ERR=file_dev.c:190 
> Could not 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
> ERR=No such file or directory
> bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
> bacula-sd-01-sd: mount.c:215-269534 Open of File device "vDrive-FullFile-8" 
> (/usr/local/bacula/volumes/FullFile) Volume "FullAuto-8867" failed: 
> ERR=file_dev.c:190 Could not 
> open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
> ERR=No such file or directory
> 
> bacula-sd-01-sd: mount.c:220-269534 Volume "FullAuto-8867" not loaded on File 
> device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile).

This is why it is not labelling, from bacula/src/stored/mount.c

int DCR::try_autolabel(bool opened)
{
   DCR *dcr = this;

   if (dev->poll && !dev->is_tape()) {
  Dmsg0(100, "No autolabel because polling.\n");
  return try_default;   /* if polling, don't try to create new labels */
   }


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] SD fails to create new volumes: trace included

2018-01-09 Thread Dan Langille
I wonder if this line is related:


bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.

If no labeling, then no volume is created.

See 
https://gist.github.com/dlangille/513cb50351f7bc6ab6d33a6859b20326#file-bacula-trace-L84



bacula-sd-01-sd: mount.c:182-269534 want vol=FullAuto-8867 devvol= 
dev="vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile)
bacula-sd-01-sd: mount.c:190-269534 Try open Vol=FullAuto-8867
bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
file_dev::open_device(DCR *, int)
bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool DEVICE::open_device(DCR 
*, int)
bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
ERR=No such file or directory
bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
file_dev::open_device(DCR *, int)
bacula-sd-01-sd: mount.c:192-269534 Try autolabel Vol=FullAuto-8867
bacula-sd-01-sd: file_dev.c:138-269534 Enter: virtual bool 
file_dev::open_device(DCR *, int)
bacula-sd-01-sd: dev.c:113-269534 Enter: virtual bool DEVICE::open_device(DCR 
*, int)
bacula-sd-01-sd: dev.c:325-269534 Enter: virtual bool DEVICE::mount(int)
bacula-sd-01-sd: file_dev.c:184-269534 open disk: mode=OPEN_READ_WRITE 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867, 0x2, 0640)
bacula-sd-01-sd: file_dev.c:191-269534 open failed: file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
ERR=No such file or directory
bacula-sd-01-sd: file_dev.c:211-269534 open dev: disk fd=-1 opened
bacula-sd-01-sd: file_dev.c:214-269534 Leave: virtual bool 
file_dev::open_device(DCR *, int)
bacula-sd-01-sd: mount.c:198-269534 open_device failed: ERR=file_dev.c:190 
Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
ERR=No such file or directory
bacula-sd-01-sd: mount.c:663-269534 No autolabel because polling.
bacula-sd-01-sd: mount.c:215-269534 Open of File device "vDrive-FullFile-8" 
(/usr/local/bacula/volumes/FullFile) Volume "FullAuto-8867" failed: 
ERR=file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8867,OPEN_READ_WRITE,0640): 
ERR=No such file or directory

bacula-sd-01-sd: mount.c:220-269534 Volume "FullAuto-8867" not loaded on File 
device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile).


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] SD fails to create new volumes: trace included

2018-01-09 Thread Dan Langille
> On Jan 9, 2018, at 12:20 PM, Bill Arlofski <waa-bac...@revpol.com> wrote:
> 
> On 01/09/2018 07:24 AM, Dan Langille wrote:
>> I'm confused as to why Volumes are not being created.  They do not exist on 
>> disk. This is not a newly configured system.
>> 
>> Is "No autolabel because polling" relevant? see below.
>> 
>> Said backup has been running for several days.  I have confirmed space is 
>> available (about 6TB free) and that files can be manually created.
>> 
>> Bacula 9.0.6 on FreeBSD with PostgreSQL 9.6.6
>> 
>> After running out of Volumes, I adjusted the configuration files to increase 
>> 'Maximum Volumes' from 5060 to 5070.. issued a reload in bconsole, now I see 
>> 10 of these:
> 
> Hi Dan,
> 
> After the reload, you will still need to do:
> 
> * update pool=
> 
> Before this you can do:
> 
> * list pools
> 
> And this should prove that the old MaxVols is still in effect. Then issue the
> update pool command and then re-check with list pools

List pools output (trimmed) before I change the pool:

| poolid | name   | numvols | maxvols | maxvolbytes| 
volretention | enabled | pooltype | labelformat |
| 22 | FullFile   |   5,083 |   5,083 | 10,737,418,240 |   
94,608,000 |   1 | Backup   | FullAuto-   |

Then I change: Maximum Volumes  = 5084

Issue reload and another list pools:

| poolid | name   | numvols | maxvols | maxvolbytes| 
volretention | enabled | pooltype | labelformat |
| 22 | FullFile   |   5,083 |   5,084 | 10,737,418,240 |   
94,608,000 |   1 | Backup   | FullAuto-   |

The 'update pool' is not required if just bumping maxvols.

Thank you.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] SD fails to create new volumes: trace included

2018-01-09 Thread Dan Langille
I'm confused as to why Volumes are not being created.  They do not exist on 
disk. This is not a newly configured system.

Is "No autolabel because polling" relevant? see below.

Said backup has been running for several days.  I have confirmed space is 
available (about 6TB free) and that files can be manually created.

Bacula 9.0.6 on FreeBSD with PostgreSQL 9.6.6

After running out of Volumes, I adjusted the configuration files to increase 
'Maximum Volumes' from 5060 to 5070.. issued a reload in bconsole, now I see 10 
of these:


[dan@bacula:~] $ bconsole 
Connecting to Director bacula.int.unixathome.org:9101
1000 OK: 103 bacula-dir Version: 9.0.6 (20 November 2017)
Enter a period to cancel a command.
*m
09-Jan 12:44 bacula-dir JobId 269534: Created new Volume="FullAuto-8846", 
Pool="FullFile", MediaType="FullFile" in catalog.
09-Jan 12:44 bacula-sd-01-sd JobId 269534: Warning: mount.c:215 Open of File 
device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) Volume 
"FullAuto-8846" failed: ERR=file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8846,OPEN_READ_WRITE,0640): 
ERR=No such file or directory

09-Jan 12:44 bacula-sd-01-sd JobId 269534: Marking Volume "FullAuto-8846" in 
Error in Catalog.
09-Jan 12:44 bacula-dir JobId 269534: Created new Volume="FullAuto-8847", 
Pool="FullFile", MediaType="FullFile" in catalog.
09-Jan 12:44 bacula-sd-01-sd JobId 269534: Warning: mount.c:215 Open of File 
device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) Volume 
"FullAuto-8847" failed: ERR=file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8847,OPEN_READ_WRITE,0640): 
ERR=No such file or directory

09-Jan 12:44 bacula-sd-01-sd JobId 269534: Marking Volume "FullAuto-8847" in 
Error in Catalog.

...

After that, the following command was issued, the pool 'max volume' was bumped 
again, and another reload command issued.

*setdebug trace=1 level=200 storage=bacula-sd-01-DailyBackups
Connecting to Storage daemon bacula-sd-01-DailyBackups at 
bacula-sd-01.int.unixathome.org:9103
3000 OK setdebug=200 trace=1 options= tags=
*m
You have no messages.

*m
09-Jan 14:06 bacula-dir JobId 269534: Created new Volume="FullAuto-8856", 
Pool="FullFile", MediaType="FullFile" in catalog.
09-Jan 14:06 bacula-sd-01-sd JobId 269534: Warning: mount.c:215 Open of File 
device "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) Volume 
"FullAuto-8856" failed: ERR=file_dev.c:190 Could not 
open(/usr/local/bacula/volumes/FullFile/FullAuto-8856,OPEN_READ_WRITE,0640): 
ERR=No such file or directory

09-Jan 14:06 bacula-sd-01-sd JobId 269534: Marking Volume "FullAuto-8856" in 
Error in Catalog.
*


Full output, including the trace, at 
https://gist.github.com/dlangille/379b9d7784548211bcab936d31e57255 including 
the 'setdebug trace=1 level=200' output of the SD.

Is "No autolabel because polling" relevant. I see that in the output.

'status storage' output includes:

Device File: "vDrive-FullFile-8" (/usr/local/bacula/volumes/FullFile) is not 
open.
   Device is BLOCKED waiting to create a volume for:
   Pool:FullFile
   Media type:  FullFile
   Drive 8 is not loaded.
   Available Space=7.902 TB
==


The configuration of that drive is:

Device {
  Name   = vDrive-FullFile-8
  Media Type = FullFile
  Archive Device = /usr/local/bacula/volumes/FullFile
  LabelMedia = yes
  Random Access  = yes
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen     = no

  Autochanger= yes
  Drive Index= 8

  Maximum Concurrent Jobs = 1
  Volume Poll Interval= 15
}


Ideas? I keep thinking it's something very clearly stated but I am overly 
looking it.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Database trouble on 9.0.4

2017-12-29 Thread Dan Langille
> On Dec 29, 2017, at 5:00 AM, Matthias Koch-Schirrmeister <m.k...@syspac.de> 
> wrote:
> 
> Good morning everybody,
> 
> a fresh installation of 9.0.4 on OpenBSD 6.2 is giving me various
> troubles. Amoong other things, the catalog backups fail:
> 
> 
> From bacula.log:
> 
>  JobId 21: BeforeJob: could not identify current directory:
> Permission denied
>  JobId 21: BeforeJob: could not identify current directory:
> Permission denied
>  JobId 21: BeforeJob: psql: could not find own program executable
>  JobId 21: BeforeJob: root doesn't have access to the catalog
> database
>  JobId 21: Error: Runscript: BeforeJob returned non-zero
> status=1. ERR=Child exited with code 1

I think this is file / directory permissions.

Check the RunBefore job, the directory it is in, and the directories it will 
access.

I think this is not specifically related to configuration or 9.0.4

Is this script dumping the database?  If so, it is running at the bacula-dir 
server.

If you want it running on the client, you want a slightly different directive, 
perhaps RunBeforeClient.

What is this script doing?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Bytes/block=64,512 for an LT04

2017-12-21 Thread Dan Langille
Hello,

I'm using Bacula 7.4.7 on FreeBSD 11

What Bytes/block are you using for your LTO4 tapes?

I ask because I just noticed:

###
Device tape is "LTO_0" (/dev/nsa0) mounted with:
Volume:  34L4
Pool:FullsLTO4
Media type:  LTO4
Slot 38 is loaded in drive 0.
Total Bytes=20,127,937,536 Blocks=312,002 Bytes/block=64,512
Positioned at File=20 Block=2,003
###

What are you using and why?

Thank you.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Recovery not possible

2017-12-19 Thread Dan Langille
> On Dec 18, 2017, at 9:59 AM, Claudia Koch <ck...@telecolumbus.net> wrote:
> 
> Hello list,
> 
> I am using Bacula 5.2.6 on Debian 8 with mysql. The database dump is 82
> GB (with --add-drop-database --add-drop-table --allow-keywords
> --single-transaction).
> 
> I had checked the database for the remains of old clients and deleted
> old entries with "purge jobid=".
> 
> My problem is that as far as possible (backups, database queries etc.).
> But I can't recover anymore.
> 
> After selecting the client, the tapes used are still displayed, but
> creating the file list does not come to an end.
> 
> The last select is a rather cryptic construct. A tmp-table is created,
> but apparently not filled. A CPU runs on 100% but the process never ends.
> 
> What is it possible to check what goes wrong? How can a database of this
> size (among other things, various mail servers with many small files are
> backed up) be cleaned up in such a way that it can be used afterwards?
> 
> I have had similar experiences with dbcheck in the past.

Never ends? How long have you waited? How many files are in that backup?
-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] Using LibreSSL?

2017-11-30 Thread Dan Langille
Are you using LibreSSL with Bacula 9.0.6?

I ask because of https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223994

re:


--- openssl.lo ---
Compiling openssl.c
--- crypto.lo ---
crypto.c:199:1: error: unknown type name 'DEFINE_STACK_OF'
DEFINE_STACK_OF(SignerInfo);
^
crypto.c:200:1: error: unknown type name 'DEFINE_STACK_OF'
DEFINE_STACK_OF(RecipientInfo);
^
crypto.c:334:21: error: use of undeclared identifier 'ASN1_STRING_get0_data';
did you mean 'ASN1_STRING_data'?
  ext_value_data = ASN1_STRING_get0_data(asn1_ext_val);
   ^
   ASN1_STRING_data
/usr/local/include/openssl/asn1.h:787:17: note: 'ASN1_STRING_data' declared
here
unsigned char * ASN1_STRING_data(ASN1_STRING *x);
   ^
crypto.c:334:43: error: cannot initialize a parameter of type 'ASN1_STRING *'
(aka 'asn1_string_st *') with an lvalue of type 'const ASN1_STRING *' (aka
'const asn1_string_st *')
  ext_value_data = ASN1_STRING_get0_data(asn1_ext_val);


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Errors migrating from mysql to postgres

2017-11-27 Thread Dan Langille
>> -Original Message-
>> From: Dan Langille [mailto:d...@langille.org] 
>> Sent: Monday, November 27, 2017 1:29 PM
>> To: Michel Figgins
>> Cc: bacula-users
>> Subject: Re: [Bacula-users] Errors migrating from mysql to postgres
>> 
>>> On Nov 27, 2017, at 2:13 PM, Michel Figgins <mfigg...@atlasallied.com> 
>>> wrote:
>>> 
>>> Hello,
>>> I'm trying to migrate from a mysql backend to a postgresql backend and 
>>> I'm getting an error importing the RestoreObject table.  Specifically I'm 
>>> getting :
>>> ERROR:  invalid input syntax for type bytea LINE 1: 
>>> ...bjectCompression) VALUES (1588,'job_metadata.xml','<B\0A\0C\...
>>> Googling around leads me to believe that Postgres isn't liking the 
>>> format of RestoreObject field, but I'm stumped as to how to change it.
>>> So my questions: Can anyone suggest a way to modify the INSERT 
>>> statement to one Postgres will understand? Do I actually need to import 
>>> this table, or will the entries get re-populated as successful backups are 
>>> completed?
>>> Additional Info:
>>> This is on Fedora 27, Bacula version is 9.0.5, the MySQL database is 
>>> MariaDB 10.2.9, the PostgreSQL database is 9.6.6.  The mysqldump command 
>>> used to generate the import file is :
>>> mysqldump -t -n -c -e --compatible=postgresql --skip-quote-names --skip-opt 
>>> --disable-keys --lock-tables -u bacula -pX bacula   | grep -v "INSERT 
>>> INTO Status"   | sed -e 's/-00-00 00:00:00/1970-01-01 00:00:00/g'   | 
>>> sed -e 's/\\0//' | sed -e "s#\\\'#\'\'#g" > bacula-export.sql
>>> The pqsl command used to import the dump is:
>>> psql -Ubacula bacula < bacula-export.sql
>> 
>> Can you confirm your PostgreSQL is SQL_ASCII?
>> 
>> $ psql -l 
>> List of databases
>>   Name   |  Owner   | Encoding  | Collate | Ctype |   Access 
>> privileges   
>> --+--+---+-+---+
>> --+--+---+-+---+---
>> bacula   | bacula   | SQL_ASCII | C   | C | 
>> 
>> 
>> 
> 
> 

> On Nov 27, 2017, at 7:25 PM, Michel Figgins <mfigg...@atlasallied.com> wrote:
> 
> Yes, it is:
> postgres=# \l
>  List of databases
>   Name|  Owner   | Encoding  |   Collate   |Ctype|   Access 
> privileges
> ---+--+---+-+-----+---
> bacula| bacula   | SQL_ASCII | C   | C   | =Tc/bacula 
>   +
>   |  |   | | | 
> bacula=C*T*c*/bacula
> 

The only other idea I have is escape notation.  That would give you: VALUES 
(1588,'job_metadata.xml',E'<B\0A\0C\

To answer your other question, that table is empty in my database.


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Errors migrating from mysql to postgres

2017-11-27 Thread Dan Langille
> On Nov 27, 2017, at 2:13 PM, Michel Figgins  wrote:
> 
> Hello,
>   I'm trying to migrate from a mysql backend to a postgresql backend and 
> I'm getting an error importing the RestoreObject table.  Specifically I'm 
> getting :
> ERROR:  invalid input syntax for type bytea
> LINE 1: ...bjectCompression) VALUES (1588,'job_metadata.xml','   Googling around leads me to believe that Postgres isn't liking the 
> format of RestoreObject field, but I'm stumped as to how to change it.
>   So my questions: Can anyone suggest a way to modify the INSERT 
> statement to one Postgres will understand? Do I actually need to import this 
> table, or will the entries get re-populated as successful backups are 
> completed?
> Additional Info:
> This is on Fedora 27, Bacula version is 9.0.5, the MySQL database is MariaDB 
> 10.2.9, the PostgreSQL database is 9.6.6.  The mysqldump command used to 
> generate the import file is :
> mysqldump -t -n -c -e --compatible=postgresql --skip-quote-names --skip-opt 
> --disable-keys --lock-tables -u bacula -pX bacula   | grep -v "INSERT 
> INTO Status"   | sed -e 's/-00-00 00:00:00/1970-01-01 00:00:00/g'   | sed 
> -e 's/\\0//' | sed -e "s#\\\'#\'\'#g" > bacula-export.sql
> The pqsl command used to import the dump is:
> psql -Ubacula bacula < bacula-export.sql

Can you confirm your PostgreSQL is SQL_ASCII?

$ psql -l 
 List of databases
   Name   |  Owner   | Encoding  | Collate | Ctype |   Access 
privileges   
--+--+---+-+---+---
 bacula   | bacula   | SQL_ASCII | C   | C | 


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Setting pg_dump version

2017-11-21 Thread Dan Langille
> On Nov 7, 2017, at 4:21 PM, Ian Douglas <i...@zti.co.za> wrote:
> 
> Hi all
> 
> I updated my version of Postgres. Foolishly did it while Bacula was busy with 
> a long backup, but that's another story.
> 
> I'm on Gentoo Linux and followed their upgrade guide.
> 
> Anyway, normal backups run fine. However the nightly catalogue backup fails 
> thusly:
> 
> 07-Nov 23:10 trooper-dir JobId 13399: shell command: run BeforeJob "/usr/
> libexec/bacula/make_catalog_backup.pl MyCatalog"
> 07-Nov 23:10 trooper-dir JobId 13399: BeforeJob: pg_dump: server version: 
> 9.6.4; pg_dump version: 9.5.8
> 07-Nov 23:10 trooper-dir JobId 13399: BeforeJob: pg_dump: aborting because of 
> server version mismatch
> 
> I have the following:
> 
> trooper bacula # ll /usr/bin/pg_dump*
> lrwxrwxrwx 1 root root 35 Oct  1 14:01 /usr/bin/pg_dump93 -> ../lib64/
> postgresql-9.3/bin/pg_dump
> lrwxrwxrwx 1 root root 35 Oct  1 13:54 /usr/bin/pg_dump94 -> ../lib64/
> postgresql-9.4/bin/pg_dump
> lrwxrwxrwx 1 root root 35 Oct  1 13:58 /usr/bin/pg_dump95 -> ../lib64/
> postgresql-9.5/bin/pg_dump
> lrwxrwxrwx 1 root root 35 Oct  1 13:51 /usr/bin/pg_dump96 -> ../lib64/
> postgresql-9.6/bin/pg_dump
> 
> So I'm guess that somewhere there is a config file that tells Bacula which 
> version to use. I've tried poking around on this machine, and asking Google, 
> all to no avail...
> 
> Can someone please tell me where or what I must change to tell the catalogue 
> backup to use the 9.6 version?

My guess: alter the script Bacula is using to dump, and change it from pg_dump 
to pg_dumpXX.

Or perhaps symlink pg_dump to the version you want to use.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] copy from disk (ZFS) to tape (LTO4) - considering spooling options

2017-11-14 Thread Dan Langille
I backup to disk then copy to LTO4 tape. All the servers in question are on a 
10Gb network.

The bacula-sd on disk has a 10x 5TB raidz3 ZFS array. I am not sure of my read 
speeds there, but I know there is no room to attach the LTO4 tape library on 
that server.

I have two spare servers which can be attached to the library:

* Dell R610 - cheaper to run - only SSDs in here.
* Dell R710 - about 2.6 times more expensive to run, no SSD.

When I copy to tape, I usually run multiple jobs at once. There are are two 
bigger jobs, and one or the other usually streams alone for a while.

Here are the scenarios I have mulling over in my head:

* use the R610, add a 128G SATA SSD drive for spooling. That will saturate the 
drive.
* use the R610, don't spool since it's a 10Gb network. This is contingent upon 
getting at least 120MB/s from the first bacula-sd.

* use the R710: it has about 5TB of disk space; I could copy to there, then 
spool to disk; then more copies of the backup exist. Also contingent upon 
getting at least 120MB/s from the first bacula-sd.

Do you have other ideas / suggestions / things to consider? Recommendations?

Thank you.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Feature Request - job complete status of with warnings on zero files zero bytes

2017-11-11 Thread Dan Langille
> On Nov 9, 2017, at 3:35 PM, Jim Richardson <j...@securit360.com> wrote:
> 
> 
> Any chance I can get a feature request in?  I would like to have all jobs 
> that complete with zero files and / or zero bytes to complete with Backup OK 
> – with warnings.  Currently these jobs complete with Backup OK. 
> 
>  


Instead, why not make add this check to your monitoring system?

I have this in use:

$ cat /usr/home/dan/bin/size-check-files
#!/bin/sh

BACKUPDIR="/usr/home/dan/backups"

# nothing to report
response=""

# all good
success=0

FILES="`/usr/bin/find ${BACKUPDIR} -size 0 | /usr/bin/grep -v /.NOBACKUP`"

# if any of these files are of size zero, bad.
for file in ${FILES}
do
  success=2
  response="$response ${file} has size zero. "
done

if [ $success -eq  0 ]
then
  echo OK. No zero-size files found.
else
  echo $response
fi

return $success

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] TAPE backups

2017-10-21 Thread Dan Langille
> On Oct 21, 2017, at 10:32 AM, Alan Brown  wrote:
> 
> 
> Straw poll time for people using tape-based systems
> 
> 
> I'd like to know who is currently using what (not historical usage)
> 
> 
> 1:   Who on this list is NOT using half inch tape (12.7mm) formats(*) -
> if so, what are you using?

To answer, I had to look that up.  I know it covers LTO, DLT and SDT.

> 2:   Who on this list is NOT using a robot or changer device?

I have one external DLT 7000 tape drive, kept in reserve.

> 3:  For the half inch tape formats, what are you using?

DLT IV - 292 tapes, kept for archive, will start disposal sometime.
SDLT - 166 tapes

 (The above: not writing any new tapes, these are scheduled for disposal)

LTO 4 - 45 tapes

> 4:  For the robot users: What are you using? (number of drives / number
> of slots)

Digital DLT MiniLibrary (Model DS-TL891-NE) 
http://www.freebsddiary.org/digital-tl891.php 
10 slots - one DLT 7000, perhaps 
two. I have a few spare DLT 7000 drives here

Compaq Storage Works MSL5026 Library
one SDLT tape drive: COMPAQ SuperDLT1 with 2 magazines for 30 slots (I think).

Dell PowerVault TL4000 - two LTO4 drives with 4 magazines for 48 slots

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Configuration for clients define director

2017-09-29 Thread Dan Langille
> On Sep 26, 2017, at 9:10 AM, Hicks, Daniel CTR OSD DMEA 
> <daniel.hicks@dmea.osd.mil> wrote:
> 
> Hello
> 
> I have been going through the configuration settings working on a script to 
> configure clients. I have not been able to find which switch is used to 
> define the director by name and address.
> 
> Using Bacula 9.0.3 on Centos7

What is your question?

In the bacula-fd.conf, you add a Director { } clause.

--
Dan Langille - BSDCan / PGCon
d...@langille.org





signature.asc
Description: Message signed with OpenPGP
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] malformed message error when testing new fileset

2017-09-23 Thread Dan Langille
> On Sep 22, 2017, at 11:30 AM, Adam Weremczuk <ad...@matrixscience.com> wrote:
> 
> Hello,
> 
> I have tried a couple of different ways of implementing "Exclude Dir 
> Containing" but it was failing when restarting director with e.g:
> 
> Config error: Keyword DirContaining not permitted in this resource
> or
> Config error: ExcludeDirContaining directive not permitted in Exclude.
> 
> With this in client's config file:

What do you mean by "client's config file"?  I assume it's a file included by 
bacula-dir.conf

> 
> FileSet {
>   Name = testclient_fileset
>   Enable VSS = yes
>   Include {
> Options {
>   signature = SHA1
> }
> File = "c:/backedup"
> Exclude Dir Containing = .excludeme
>   }
> }

I cannot see the error but here is a simple example from my configuration.

FileSet {
  Name = "dbclone daily databases"
  Include {
Options {
  signature=MD5
} 
Exclude Dir Containing = .NOBACKUP
File = /home/dan/backups
  }
}

I know that works.



> 
> director doesn't complain and starts fine but then a problem manifests itself 
> as below:
> 
> Connecting to Director bacula:9101
> 1000 OK: bacula_dir Version: 5.2.6 (21 February 2012)


Your directory is 5.2.6.

What version is your client?  It must be <= 5.2.6

> Enter a period to cancel a command.
> *estimate job=testclient_backup listing client=testclient_fd 
> fileset=testclient_fileset
> Using Catalog "bacula_catalog"
> Connecting to Client testclient_fd at testclient.domainname.com:9102
> Error sending include list.
> 
> 22-Sep 09:46 bacula_dir JobId 0: Error: getmsg.c:194 Malformed message: 
> Invalid FileSet command: valid FileSet command: valid FileSet command: valid 
> FileSet command: valid FileSet command: valid FileSet command: valid FileSet 
> command: valid FileSet command: valid FileSet command: valid FileSet command: 
> valid FileSet command: valid FileSet command: valid FileSet command: valid 
> FileSet command: valid FileSet command: valid FileSet command: valid FileSet 
> command: valid FileSet command: valid FileSet command: valid FileSet command: 
> valid FileSet command: valid FileSet command: val
> 22-Sep 09:46 bacula_dir JobId 0: Fatal error: Socket error on Include 
> command: ERR=No data available
> 
> Can somebody advise what's wrong with my config or how to troubleshoot it 
> further?


-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] print numbers without thousands separators

2017-09-23 Thread Dan Langille
> On Sep 22, 2017, at 5:26 PM, Dimitri Maziuk <dmaz...@bmrb.wisc.edu> wrote:
> 
> On 2017-09-22 09:47, Heitor Faria wrote:
>> Hello, John,
>>"I'm not sure I see the utility of copying and pasting any of the
>>other formatted numbers here."
>>I do.  Running calculations within a script, a tally of number of
>>files, job bytes, averages, etc.
>> Can't you just workaround using sed (sed 's/,//')?
> 
> Why? AFAIK vast majority of humans use nothing or space for thousands 
> separator and comma for decimal. Why can't bacula get it right?

Why? Because nobody wants it enough.

I'll fall back to the usual open source stance.  If someone really wanted it, 
it'd already be in the code.

Clearly, nobody wants it enough to do the work (or pay someone else).

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Incremental changed to Full

2017-09-11 Thread Dan Langille
> On Sep 11, 2017, at 10:54 AM, Maciolek, Mark <mark.macio...@unh.edu> wrote:
>  
> Hi,
>  
> The full took over 5 days (47Tb) so job 17 was in the schedule before it 
> finished, so I should change my schedule to do a Full the first Sunday of the 
> month and then only incremental starting on the second Sunday of the month?

Instead, I would use the Duplicate job directives previously mentioned.
-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] A device per pool

2017-09-09 Thread Dan Langille
> On Sep 8, 2017, at 10:39 AM, Dan Langille <d...@langille.org> wrote:
> 
>> On Sep 7, 2017, at 11:45 PM, Phil Stracchino <ph...@caerllewys.net 
>> <mailto:ph...@caerllewys.net>> wrote:
>> 
>> On 09/07/17 22:04, Dan Langille wrote:
>>> I have recently moved to one Storage device per pool.
>>> 
>>> Why?
>>> 
>>> So each pool is on a different ZFS dataset.
>> 
>> 
>> I'm curious.  What do you see as the advantage of this layout?
> 
> 
> Background: I'm using ZFS on FreeBSD.  I like keeping application specific 
> data
> in its own ZFS dataset. It allows for an easy snapshot and backup solution.
> 
> Storage space was the original motivation. One zpool was at 80%,
> the other is now at about 44%. I wanted to move some data from one zpool
> to another.
> 
> When moving to another pool, the mount point changes, which meant creating
> a new Storage device pointing at that mount point.
> 
> After moving the first Pool I figured: well, if it's this easy, why not do it 
> for each Bacula Pool?
> 
> Back to your question: A similar question was asked on Google+: 
> 
> https://plus.google.com/+DanLangille/posts/K2zfNo1ii22 
> <https://plus.google.com/+DanLangille/posts/K2zfNo1ii22>
> 
> ZFS can certainly and easily handle many files in the same directory. That 
> wasn't
> a basis for change.
> 
> My answer from there (with slight edits) appears below:
> 
> ###
> 
> If the Pool is no longer required, it is easy to delete. Dividing data up 
> this way
> it's always a good idea, because of the flexibility for future manipulation 
> it provides. 
> 
> If these were all on separate devices, you'd get better concurrent throughput.
> 
> Full backups are usually bigger, incremental backups are usually smaller, so 
> you 
> could just recordsize accordingly.
> 
> If you assigned each client to a different pool, deleting their backups when 
> the
> client leaves is now a simple matter of deleting the appropriate ZFS datasets.
> 
> If you want to move a pool to a different bacula-sd, you move that data set.
> 
> ###
> 
> The above was typed very early in my day and I wasn't quite awake.
> 
> Perhaps the highlights are:
> 
> - keep like data in like datasets
> - more datasets gives more flexibility
> - because I can (I was at vBSDCon and had the time to do this)
> 
> Ideas? Suggestions?

Over a discussion last night at dinner, we came up with two additional benefits:

* snapshot retention - If you want a different snap shot retention policy
  for different types of backups, having each pool on a different ZFS dataset 
allows this.

* zfs replication - if you want to replicate some, but not all of your backups, 
having only that
  data on a separate ZFS dataset allows for this.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] A device per pool

2017-09-08 Thread Dan Langille
> On Sep 7, 2017, at 11:45 PM, Phil Stracchino <ph...@caerllewys.net> wrote:
> 
> On 09/07/17 22:04, Dan Langille wrote:
>> I have recently moved to one Storage device per pool.
>> 
>> Why?
>> 
>> So each pool is on a different ZFS dataset.
> 
> 
> I'm curious.  What do you see as the advantage of this layout?


Background: I'm using ZFS on FreeBSD.  I like keeping application specific data
in its own ZFS dataset. It allows for an easy snapshot and backup solution.

Storage space was the original motivation. One zpool was at 80%,
the other is now at about 44%. I wanted to move some data from one zpool
to another.

When moving to another pool, the mount point changes, which meant creating
a new Storage device pointing at that mount point.

After moving the first Pool I figured: well, if it's this easy, why not do it 
for each Bacula Pool?

Back to your question: A similar question was asked on Google+: 

https://plus.google.com/+DanLangille/posts/K2zfNo1ii22 
<https://plus.google.com/+DanLangille/posts/K2zfNo1ii22>

ZFS can certainly and easily handle many files in the same directory. That 
wasn't
a basis for change.

My answer from there (with slight edits) appears below:

###

If the Pool is no longer required, it is easy to delete. Dividing data up this 
way
it's always a good idea, because of the flexibility for future manipulation it 
provides. 

If these were all on separate devices, you'd get better concurrent throughput.

Full backups are usually bigger, incremental backups are usually smaller, so 
you 
could just recordsize accordingly.

If you assigned each client to a different pool, deleting their backups when the
client leaves is now a simple matter of deleting the appropriate ZFS datasets.

If you want to move a pool to a different bacula-sd, you move that data set.

###

The above was typed very early in my day and I wasn't quite awake.

Perhaps the highlights are:

- keep like data in like datasets
- more datasets gives more flexibility
- because I can (I was at vBSDCon and had the time to do this)

Ideas? Suggestions?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] A device per pool

2017-09-08 Thread Dan Langille
> On Sep 8, 2017, at 6:47 AM, Martin Simmons <mar...@lispworks.com> wrote:
> 
> That's a useful article.

Thank you.

> There is a minor caveat when copying volume files: the volume label contains
> the PoolName and MediaType.  This is minor because it only matters if you need
> to bscan the volume before it is recycled.

I hope nobody ever has to use bscan. (using bscan means something has gone 
wrong;
this is not a criticism of the tool).

I don't think we can easily change that label.  For those reading, here is an 
older document
I found via Google. 
http://www.bacula.org/2.4.x-manuals/en/developers/Storage_Media_Output_Format.html
 
<http://www.bacula.org/2.4.x-manuals/en/developers/Storage_Media_Output_Format.html>

As you say, it won't be relevant until bscan time.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





> __Martin
> 
> 
> 
>>>>>> On Thu, 7 Sep 2017 22:04:35 -0400, Dan Langille said:
>> 
>> I have recently moved to one Storage device per pool.
>> 
>> Why?
>> 
>> So each pool is on a different ZFS dataset.
>> 
>> Once I figured out the process for the first one, the rest was mostly 
>> waiting for data to be copied.
>> 
>> I documented it here: 
>> https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/
>>  
>> <https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/>
>> 
>> The final zfs layout looks like this: 
>> https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e 
>> <https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e>
>> 
>> Here is the rough overview:
>> 
>> 
>> # in bacula-dir.conf
>> 
>> Pool {
>>  Name = DiffFile
>>  Pool Type= Backup
>>  Recycle  = yes
>>  AutoPrune= yes
>>  Volume Retention = 6 weeks
>>  Storage  = bacula-sd-01-DiffFile
>>  Next Pool= Differentials
>> 
>>  Maximum Volume Bytes = 10G
>>  Maximum Volume Jobs  = 1
>>  Maximum Volumes  = 300
>> 
>>  LabelFormat = "DiffAuto-"
>> }
>> 
>> # Definiton of file storage device
>> Storage {
>>  Name   = bacula-sd-01-DiffFile
>>  Address= bacula-sd-01.int.unixathome.org# N.B. Use a 
>> fully qualified name here
>>  SDPort = 9103
>>  Password   = "[redacted]"
>> 
>>  AutoChanger = Yes
>> 
>>  Device = VirtualDisk-DiffFile
>>  Media Type = DiffFile
>> 
>>  Maximum Concurrent Jobs = 27
>> }
>> 
>> # in bacula-sd.conf
>> 
>> # why do I do an autochanger? Some AutoChangers have multiple drives.
>> # It's easier to make them all similar.
>> # an example with 10 virtual drives is here: 
>> https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55 
>> <https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55>
>> 
>> Autochanger {
>>  Name = VirtualDisk-DiffFile
>> 
>>  Changer Device  = /dev/null
>>  Changer Command = /dev/null
>> 
>>  Device  = vDrive-DiffFile-0
>> }
>> 
>> Device {
>>  Name   = vDrive-DiffFile-0
>>  Media Type = DiffFile
>>  Archive Device = /usr/local/bacula/volumes/DiffFile
>>  LabelMedia = yes
>>  Random Access  = yes
>>  AutomaticMount = yes
>>  RemovableMedia = no
>>  AlwaysOpen = no
>> 
>>  Autochanger= yes
>>  Drive Index= 0
>> 
>>  Maximum Concurrent Jobs = 1
>>  Volume Poll Interval= 15
>> }
>> 
>> 
>> 
>> 
>> Dan Langille - BSDCan / PGCon
>> d...@langille.org
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] A device per pool

2017-09-07 Thread Dan Langille
I have recently moved to one Storage device per pool.

Why?

So each pool is on a different ZFS dataset.

Once I figured out the process for the first one, the rest was mostly waiting 
for data to be copied.

I documented it here: 
https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/
 
<https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/>

The final zfs layout looks like this: 
https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e 
<https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e>

Here is the rough overview:


# in bacula-dir.conf

Pool {
  Name = DiffFile
  Pool Type= Backup
  Recycle  = yes
  AutoPrune= yes
  Volume Retention = 6 weeks
  Storage  = bacula-sd-01-DiffFile
  Next Pool= Differentials

  Maximum Volume Bytes = 10G
  Maximum Volume Jobs  = 1
  Maximum Volumes  = 300

  LabelFormat = "DiffAuto-"
}

# Definiton of file storage device
Storage {
  Name   = bacula-sd-01-DiffFile
  Address= bacula-sd-01.int.unixathome.org# N.B. Use a 
fully qualified name here
  SDPort = 9103
  Password   = "[redacted]"

  AutoChanger = Yes

  Device = VirtualDisk-DiffFile
  Media Type = DiffFile

  Maximum Concurrent Jobs = 27
}

# in bacula-sd.conf

# why do I do an autochanger? Some AutoChangers have multiple drives.
# It's easier to make them all similar.
# an example with 10 virtual drives is here: 
https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55 
<https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55>

Autochanger {
  Name = VirtualDisk-DiffFile

  Changer Device  = /dev/null
  Changer Command = /dev/null

  Device  = vDrive-DiffFile-0
}

Device {
  Name   = vDrive-DiffFile-0
  Media Type = DiffFile
  Archive Device = /usr/local/bacula/volumes/DiffFile
  LabelMedia = yes
  Random Access  = yes
  AutomaticMount = yes
  RemovableMedia = no
  AlwaysOpen = no

  Autochanger= yes
  Drive Index= 0

  Maximum Concurrent Jobs = 1
  Volume Poll Interval= 15
}




Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Renamed and moved files are backed up again

2017-09-06 Thread Dan Langille
> On Sep 4, 2017, at 2:27 AM, Fabian Brod <fabian.b...@gmx.net> wrote:
> 
> Hello,
>  
> is it normal to back up bacula moved files or renamed files over and over 
> again?
> I have created a test configuration with base jobs but the files are backed 
> up again and again.
> Does anyone have a solution to the problem or a working configuration file?
>  
> To the procedure:
> 1. I'm starting a backup as a base job OK
> 2. a full backup using a base job OK
> 3. rename or move a file
> 4. Full Backup with Base Job
> The renamed or moved file is saved with every job again and again

Please help us understand.

What documenation led you to believe something different would happen?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 9.0.3 static client

2017-08-29 Thread Dan Langille

> On Aug 29, 2017, at 10:20 AM, Kern Sibbald <k...@sibbald.com> wrote:
> 
> On 08/28/2017 07:17 PM, Dan Langille wrote:
> 
>> Have you build the static client for 9.0.3?
>> 
>> In my testing, two things have come to light:
>> 
>> 1 - file names have changed: static-bacula-fd (was bacula-fd-static) & 
>> static-bconsole (was bconsole-static).
> I am very surprised, because I do not remember having changed these names.  
> It must be a result of backporting the Bacula Enterprise code, where they 
> probably changed the names -- I find the new names more appropriate.

I am incorrect.  The names did not change.  More below

>> 2 - binaries are not installed. Perhaps this is related to issue 1.
> I suspect that you must issue a specific make install-xxx command to install 
> those binaries.

I am in the process of packaging bacula9-server (and associated bacula9 ports) 
for FreeBSD.

The above misunderstandings arose because I had not completely converts the 
bacula-client-static port to work with bacula 9.  Packaging woes...

Please ignore this entire thread. Sorry for the noise.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 9.0.3 static client

2017-08-28 Thread Dan Langille
On Aug 28, 2017, at 1:42 PM, Dan Langille <d...@langille.org> wrote:
> 
>> On Aug 28, 2017, at 1:17 PM, Dan Langille <d...@langille.org 
>> <mailto:d...@langille.org>> wrote:
>> 
>> Have you build the static client for 9.0.3?
>> 
>> In my testing, two things have come to light:
>> 
>> 1 - file names have changed: static-bacula-fd (was bacula-fd-static) & 
>> static-bconsole (was bconsole-static).
>> 
>> 2 - binaries are not installed. Perhaps this is related to issue 1.
>> 
>> What has been your experience?
> 
> And my most recent test, after building bacula-bat worked..
> 
> https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_17h39m49s/logs/bacula-client-static-9.0.3.log
>  
> <https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_17h39m49s/logs/bacula-client-static-9.0.3.log>
> 
> I have no ide why building bacula-bat changed the outcome.

Please ignore my previous message.  As you can see, the build which succeeded 
was bacula client static for 7.x, not 9.x

My apologies for the noise.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 9.0.3 static client

2017-08-28 Thread Dan Langille
> On Aug 28, 2017, at 1:17 PM, Dan Langille <d...@langille.org> wrote:
> 
> Have you build the static client for 9.0.3?
> 
> In my testing, two things have come to light:
> 
> 1 - file names have changed: static-bacula-fd (was bacula-fd-static) & 
> static-bconsole (was bconsole-static).
> 
> 2 - binaries are not installed. Perhaps this is related to issue 1.
> 
> What has been your experience?

And my most recent test, after building bacula-bat worked..

https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_17h39m49s/logs/bacula-client-static-9.0.3.log
 
<https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_17h39m49s/logs/bacula-client-static-9.0.3.log>

I have no ide why building bacula-bat changed the outcome.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] 9.0.3 static client

2017-08-28 Thread Dan Langille
Have you build the static client for 9.0.3?

In my testing, two things have come to light:

1 - file names have changed: static-bacula-fd (was bacula-fd-static) & 
static-bconsole (was bconsole-static).

2 - binaries are not installed. Perhaps this is related to issue 1.

What has been your experience?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


Full build at 
https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_16h56m23s/logs/errors/bacula9-client-static-9.0.3.log
 
<https://services.unixathome.org/poudriere/data/110amd64-working/2017-08-28_16h56m23s/logs/errors/bacula9-client-static-9.0.3.log>

From the above URL:

Configuration on Mon Aug 28 16:56:50 UTC 2017:

   Host: amd64-portbld-freebsd11.0 -- freebsd 
11.0-RELEASE-p8
   Bacula version:   Bacula 9.0.3 (08 August 2017)
   Source code location: .
   Install binaries: /usr/local/sbin
   Install libraries:/usr/local/lib
   Install config files: /usr/local/etc
   Scripts directory:/usr/local/etc
   Archive directory:/tmp
   Working directory:/opt/bacula/working
   PID directory:/var/run
   Subsys directory: /var/run/subsys
   Man directory:/usr/local/man
   Data directory:   /usr/local/share
   Plugin directory: /usr/local/lib
   C Compiler:   cc version
   C++ Compiler: /usr/bin/c++ version
   Compiler flags:-O2 -pipe  -fstack-protector -fno-strict-aliasing 
-x c++ -fno-strict-aliasing -fno-exceptions -fno-rtti
   Linker flags:   -L/usr/local/lib -L/usr/local/lib 
-Wl,-rpath,/usr/local/lib -fstack-protector
   Libraries:-lpthread 
   Statically Linked Tools:  no
   Statically Linked FD: yes
   Statically Linked SD: no
   Statically Linked DIR:no
   Statically Linked CONS:   yes
   Database backends:None
   Database port: 
   Database name:bacula
   Database user:bacula
   Database SSL options: 

   Job Output Email: root@localhost
   Traceback Email:  root@localhost
   SMTP Host Address:localhost

   Director Port:9101
   File daemon Port: 9102
   Storage daemon Port:  9103

   Director User:
   Director Group:   
   Storage Daemon User:  
   Storage DaemonGroup:  
   File Daemon User: 
   File Daemon Group:

   Large file support:   yes
   Bacula conio support: yes -ltinfo
   readline support: no 
   TCP Wrappers support: no 
   TLS support:  yes
   Encryption support:   yes
   ZLIB support: yes
   LZO support:  yes
   enable-smartalloc:yes
   enable-lockmgr:   no
   bat support:  no
   client-only:  yes
   build-dird:   no
   build-stored: no
   Plugin support:   no
   AFS support:  no
   ACL support:  yes
   XATTR support:yes
   systemd support:  no 
   Batch insert enabled: None





--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula driver misconceptions

2017-08-25 Thread Dan Langille
> On Aug 23, 2017, at 11:27 AM, Phil Stracchino <ph...@caerllewys.net> wrote:
> 
> On 08/23/17 11:06, Dimitri Maziuk wrote:
>> On 2017-08-22 21:26, Heitor Faria wrote:
>> 
>>> It usually cost a new tape library, a new set of tapes and coffee.
>>> It is not that hard to migrate legacy backups if you use a backup rotation 
>>> strategy such as GFS and works in a common company with 1-5 maximum 
>>> montly/yearly retention backups.
>> 
>> I repeat, how many times have *you* done it, how many tapes, and how 
>> much did it cost you? In hardware, your time, and energy bills?
> 
> 
> I don't have the luxury of a controlled environment for my backup
> system.  One of the things I've learned as a result is that I greatly
> extend the life of my tape drive by *only powering it on* when I'm
> preparing to run monthly full backups.  (This also, practically
> speaking, rules out internal drives, period.)  If you want to get a long
> service life out of a modern tape drive you practically need to have a
> clean-room for it.  Requiring a controlled environment to get a useful
> service life is another hidden cost factor in LTO tape, and the higher
> the track densities go, the more sensitive the drives are going to be to
> environmental factors.

FWIW, I do the same, mostly for noise and power considerations. The
tape libraries sit about three feet from my desk.

Mine is for home use. Not a common situation.

> I'm REALLY hoping that by the time I need to upgrade from my LTO4 drive,
> there is a better and more robust alternative available.


I'm hoping my LTO4 lasts at least 3 more years.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Bacula driver misconceptions

2017-08-22 Thread Dan Langille
> On Aug 22, 2017, at 11:01 AM, Dimitri Maziuk <dmaz...@bmrb.wisc.edu> wrote:
> 
> On 2017-08-22 03:10, Kern Sibbald wrote:
>> Dimitri,
>> I have heard the same complaint from you a number of times: "Bacula sucks at 
>> disk as tapes".  It is my view that your statement is simply not true.
> 
> I think at the very basic level, a disk is a device that doesn't quack like a 
> tape and doesn't walk like a tape. What this means is if your software is 
> built on the tape paradigm, it can call a disk "a tape", and it will suck 
> because it actually is not a tape. It does not matter to me what API the code 
> calls to do addressing: the point is that a disk has multiple files on it 
> while a tape has bytes until EOT.
> 
> Don't get me wrong, this is not specific to bacula: every tape backup 
> software has the same conceptual limitation and has to employ some 
> workarounds for backups to disk. But this is the bacula list, we're not 
> talking about every software here.
> 
> Bacula, in fact, is much better than some, if you can fit all your backups on 
> a single filesystem: bacula's built-in disk backup setup is a breeze.
> 
> However if you want to backup to removable disks and put them on the shelf 
> when full, that's where things get less than perfect fast.

Is this a common use case?  I'm not challenging your approach. I am seeking 
knowledge about how Bacula gets used.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] HEADS UP: Bacula 5.2 removal from FreeBSD

2017-08-22 Thread Dan Langille
> On Aug 15, 2017, at 7:53 AM, Dan Langille <d...@langille.org> wrote:
> 
> Bacula 5.2.12 will be removed from the FreeBSD ports tree soon. This affects 
> 5 ports. Full list here: https://www.freshports.org/search.php?query=bacula5 
> <https://www.freshports.org/search.php?query=bacula5>
> 
> Bacula 5.2.12 was added to the FreeBSD ports tree in Sep 2012.  Bacula 7 was 
> added in Aug 2014.  Everyone should be on 7 by now.

The ports are now deprecated and will be removed from the tree on 2017-10-04

re: 
https://docs.freebsd.org/cgi/getmsg.cgi?fetch=1482374+0+current/svn-ports-head 
<https://docs.freebsd.org/cgi/getmsg.cgi?fetch=1482374+0+current/svn-ports-head>

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] 9.0.3's FD incompatible with 7.4's DIR

2017-08-21 Thread Dan Langille
> Kern has always said that "Older File Daemons *should* be compatible with the 
> x.x.x Director" and this has been true for years, it seems that with this 
> (9.0.3) latest version you need to align DIR with FD and SD.
> I've read all the release notes since 9.0.0, maybe this error is for the "New 
> Message Identification Format" ? 
> http://www.bacula.org/9.0.x-manuals/en/main/New_Features_in_9_0_0.html#SECTION003024000

The rule for versions has long been:

DIR = SD >= FD


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


[Bacula-users] HEADS UP: Bacula 5.2 removal from FreeBSD

2017-08-15 Thread Dan Langille
Bacula 5.2.12 will be removed from the FreeBSD ports tree soon. This affects 5 
ports. Full list here: https://www.freshports.org/search.php?query=bacula5 
<https://www.freshports.org/search.php?query=bacula5>

Bacula 5.2.12 was added to the FreeBSD ports tree in Sep 2012.  Bacula 7 was 
added in Aug 2014.  Everyone should be on 7 by now.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] job records have gone after database move

2017-08-04 Thread Dan Langille
> On Aug 3, 2017, at 4:44 AM, Andras Horvai <andras.hor...@gmail.com> wrote:
> 
> Hi,
> 
> I had to move my bacula backup server from one server to another because of 
> hardware
> failure. I am using mysql server as my catalog backend. Before installing the 
> new server
> I did a mysql dump on the old and then I did a mysql restore on the new.
> Seemed everything was fine. Then recently I had to restore from Archive.
> 
> I have archive jobs which are copy jobs. 
> I do weekly full backup to a file storage. Then these copy jobs move the last 
> full backup from
> file storage to tape (worm) storage. 
> 
> The retention period of the volume of archive is 100 years. This retention
> time is inherited by the jobs. The file retention time of these jobs is 2 
> months. So in theory,
> when I have to restore something from archive what is 2 years old I would be 
> able to do
> a full job restore... of that full backup which would be perfectly fine for 
> me but the problem:

As I understand it, and perhaps other can correct/confirm:

 - when the first retention period is met, the data can be purged from the 
Catalog.

Perhaps this data has been purged from the new database server after it was 
copied it.  It was
there, but now it has been purged. Has the 2 month period been met since that 
move?

> It seemed that after the move of the backup server I lost my archive job 
> records from the catalog.
> bacula did not find the archive jobs of the client in the catalog. I had to 
> use bscan on the tape
> volume, so I was able to restore.
> 
> Fortunately, I did not wipe the old system so I was able to run bconsole and 
> I checked
> that the archive records are present... in the old database 
> 
> Can anybody advise what to check in this case? What did I wrong?

I think you did nothing wrong.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


  1   2   3   4   5   6   7   8   9   10   >