Re: [Bacula-users] how to tell 9.6 director to use mysql? SOLVED

[Greg Woods]

On Mon, Jul 20, 2020 at 11:04 PM Davide Franco  wrote:

> Hello,
>
> I can produce Fedora 32 rpm packages if you think that can help you.
>

Thank you very much for the offer. But I think the alternatives system is
sufficient for my needs. It is even documented. The only problem is that my
Google search fu is not good enough to have found the document I needed
before posting to the list. Sorry for all the noise.

--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] how to tell 9.6 director to use mysql?

[Greg Woods]

On Mon, Jul 20, 2020 at 5:43 PM Dmitri Maziuk 
wrote:

> On 7/20/2020 6:31 PM, Gary R. Schmidt wrote:
> > On 21/07/2020 09:21, Greg Woods wrote:
> >>
> >> Has this always been the case?
> > Yes.
> >
>
> Wow. I could swear I installed a backend-specific library RPM and made a
> symlink or something in the version we're using. It all must've been a
> figment of my imagination I guess.
>

Maybe not, because I found the answer. I tried downgrading as I suggested
to myself above, but it didn't work. I was running the exact same RPM
packages for Bacula that I was running on my older system, and it still
complained that Bacula was compiled for Postgres.

It turns out that Fedora has this "alternatives" system. It defines, for
example, whether "/usr/bin/sendmail" will point to "sendmail" or "postfix"
for mail delivery. They have done this for Bacula too; there is a library
called "libbaccats.so" which is apparently linked to by their Bacula
binaries, and the alternatives system can be used to determine whether this
points to a MySQL, PostgreSQL, or SQLite3 version. I finally hit upon the
right Google search words to find this:
https://src.fedoraproject.org/rpms/bacula . Since I have always run my
Director on a Fedora machine, this explains why I didn't realize Bacula was
designed to be hard-compiled for a single database backend.

After reading this, I knew that PostgreSQL is the default, but it can be
changed by running "alternatives --config libbaccats.so" and selecting the
MySQL option.

The alternatives system is quite slick as can be seen here; it's easy to
use one set of Bacula packages no matter which database you are using; no
separate package or compiling from source necessary. But it's not always
easy to figure out which things have "alternatives" and how to use them.
Worse still, it appears that I must have gone down this road before, as
even on my older system, Postgres is the default but it had already been
set to use MySQL. I really wish my Google search fu was better, then I
could have avoided all this.

--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] how to tell 9.6 director to use mysql?

[Greg Woods]

On Mon, Jul 20, 2020 at 4:56 PM Gary R. Schmidt 
wrote:

> Bacula can only be built with support for one database, if you want to use
> MySQL you will either have to ask the distribution  maintainers to produce
> one, or build Bacula yourself from source.
>

Has this always been the case? If so, why is there a "dbdriver" parameter
in the director configuration at all, if it cannot actually be used?

I could certainly buy that the Fedora package maintainer decided to switch
from MySQL (which has always worked in every previous Fedora distribution
since I started using Bacula back with version 5) to PostgreSQL; this sort
of thing happens in Fedora. I did verify that the Bacula 9.4 packaged for
Fedora 31 seems to be set for MySQL, because if I try to specify "dbdriver
= postgresql" on my old 9.4/F31 box, then I get the converse error:

20-Jul 17:05 bacula-dir JobId 0: Warning: Dbdriver field within director
config file "postgresql" mismatched with the Database argument "MySQL"
passed during Bacula compilation.

Assuming this is all correct: I think there is nearly zero chance of
getting the Fedora people to produce a different package; Fedora is
notoriously short of package maintainers. So I could either compile 9.6
from source, or downgrade to the Fedora 31 version that worked with MySQL.
I have the additional issue to deal with that my Storage server is on a
Raspbian system which of course isn't going to get new Bacula versions
until the next OS upgrade, but I have found that generally different minor
versions work together just fine as long as it's the same major version.  I
realize it's not guaranteed unless DIR = SD >= FD.

Thanks for setting me straight, one way or the other I can move forward now.

--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] how to tell 9.6 director to use mysql?

[Greg Woods]

I am migrating my Bacula director and database from a Fedora 31 machine
running 9.4 to a new
Fedora 32 machine running 9.6. I am having what seems like a simple
configuration issue but I can't figure it out after about an hour of
Googling around and trying different things by trial and error. The error I
get is this:

[root@seveneves greg]# bacula-dir -t -c /etc/bacula/bacula-dir.conf
bacula-dir: dird.c:1248-0 Dbdriver field within director config file
"mysql" mismatched with the Database argument "PostgreSQL" passed during
Bacula compilation.
bacula-dir: dird.c:1273-0 Could not open Catalog "GregAndEva", database
"bacula".
bacula-dir: dird.c:1279-0 postgresql.c:330 Unable to connect to PostgreSQL
server. Database=bacula User=bacula
Possible causes: SQL server not running; password incorrect;
max_connections exceeded.
20-Jul 16:24 bacula-dir ERROR TERMINATION
Please correct configuration file: /etc/bacula/bacula-dir.conf

This is my Catalog definition:

Catalog {
  Name = GregAndEva
# Uncomment the following line if you want the dbi driver
  dbdriver = "mysql";
  dbname = "bacula"; dbuser = "bacula"; dbpassword = "MY BIG SECRET";
}

Unmodified from what worked on 9.4 . I also tried "dbdriver="dbi:mysql"
with the same result.

Could somebody just tell me what the proper incantation is here?

Thank you,
--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Cannot find any appendable volumes

[Greg Woods]

I am getting this error when running copy jobs. I have an "ARCHIVE-ALL" job
that searches for any client backup jobs that terminated normally and have
not yet been copied from online to archive backups, and runs copy jobs for
each. I have been using this system for 7 years now, and it has worked well
until today.

Here's the Pool definitions (File is the online pool, and Archive is a
vchanger with 6 removable disks as magazines):

Pool {
  Name = File
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle
Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 365 days # 1 year
  Maximum Volume Bytes = 50G # Limit Volume size to something
reasonable
  Maximum Volumes = 245# Limit number of Volumes in Pool
  Storage = BKUP
  Next Pool = Archive # Where to copy/migrate to
}
Pool {
  Name = Archive
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle
Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 365 days # one year
  Maximum Volume Bytes = 50G # Limit Volume size to something
reasonable
  Maximum Volumes = 136# Limit number of Volumes in Pool
  Storage = ARCH
}

As you can see, the volume retention in the Archive pool is set to 365
days., and Recycle is set to yes, which means  (if I understand this
correctly) that any volume that hasn't been written to in more than a year
should be available to recycle. So why is it suddenly telling me that there
are no appendable volumes after 7 years of running this configuration?

All of my client definitions look like this:

Client {
  Name = "myhost"
  Address = myhost.my.domain
  Catalog = "My Catalog" # there is only one
  Password = "big secret"
  File Retention = 365 days
  Job Retention = 365 days
}

Did I miss something? There are definitely some volumes in  the Archive
pool that have not been written to in more than a year; shouldn't those be
available for recycling?

Lastly, what command would I use to let Bacula know that a given volume can
be made available? Eventually I have to solve my configuration issue, but
for now I'd like to just mark some old volumes as available manually so
that my copy jobs can continue.

Thanks for any pointers,
--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] volumes in error status

[Greg Woods]

I have a small home setup (about 10 clients). I have one pool used for
backups which is just a directory full of disk files (50GB each). The other
pool is a vchanger used for copy jobs.

The problem I have is that for some reason, the directory holding the
on-disk volumes became unreachable (probably due to an automount error but
I'm not sure about that). This resulted in several volumes being set to
Error status. I know the data in those files is still good. Is there any
way to recover it? I tried just setting the volume status to "Full", but
that didn't work because bacula thinks the volume size is 1 byte. I could
set the status to "purged", which might allow the volumes to be re-used(?),
but of course then I would lose all the data on the volumes.

Thanks,
--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] help debugging new SD

[Greg Woods]

On Thu, Dec 5, 2019 at 12:45 PM Greg Woods  wrote:

>
> bstore: askdir.c:235-41356 get_volume_info failed: ERR=1000 OK
> VolName=BKUP.65 VolJobs=5 VolFiles=7 VolBlocks=491843 VolBytes=31729597852
> VolABytes=0 VolHoleBytes=0 VolHoles=0 VolMounts=164 VolErrors=0
> VolWrites=7981752 MaxVolBytes=53687091200 VolCapacityBytes=0
> VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0
> VolReadTime=5665051214 VolWriteTime=15363285212 EndFile=7
> EndBlock=2345686412 VolType=1 LabelType=0 MediaId=100 ScratchPoolId=0
> VolParts=0 VolCloudParts=0 LastPartBytes=0 Enabled=1
> bstore: askdir.c:398-41356 No vol. index 1 return false. dev="BKUP"
> (/bkup/bacula)
> bstore: askdir.c:406-41356 unlock_volumes at askdir.c:406 precnt=1
> bstore: mount.c:363-41356 not dir_find_next
> bstore: askdir.c:799-41356 enter dir_ask_sysop_to_create_appendable_volume
>
> I have checked the obvious and the stupid
>

Well, as it turns out I *thought* I had checked the obvious and the stupid,
but I missed one thing: version numbers. Turns out the new SD is 9.4, and
the director was 9.2 . Upgrading the director to 9.4 fixed the issue.

Sorry for the noise.

--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] help debugging new SD

[Greg Woods]

I have a small home installation (less than a dozen clients) that has the
director and MySQL on one host, and the SD on a Raspberry Pi. What I am
trying to do now is replace an old Fedora 28 Pi 2 with a new Raspbian Pi 4.
I'm having trouble getting the new SD to work.

I've got two storage devices: BKUP (disk volumes) and ARCH (uses a
vchanger). Ironically, the ARCH storage appears to be fine (although I
haven't actually tried to write to it yet). The volumes in the magazine
that is currently inserted show green in bat. But all the BKUP volumes show
red, so it's not surprising that it's not working. The problem is when I
run a normal backup job, it hangs, asking me to label a new volume because
no appendable volumes were found. But the catalog does list at least one
volume with Append status, and all the volumes have Recycle set to 1. The
catalog listing for all the BKUP volumes looks fine (no surprise there as
it's the same director and catalog as before). So I am trying to debug this
running the SD with -d 999, and here's the tail of what I get:

bstore: askdir.c:233-41356 ___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Storage Daemon Service not working at start-up (after os update)

[Greg Woods]

On Thu, Feb 21, 2019 at 11:32 AM Yılmaz Bilgili via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

>
>
> If I stop the system service and start it manually from command line
> with "bacula-sd -d 200 -c /opt/bacula/etc/bacula-sd.conf" command then I
> can get status of storage daemon and the backups are working as
> expected.
>

I had the same symptom on a Fedora system. It turned out that the storage
daemon was being started before the network interface was fully up. This is
common with systemd-based systems when something is either compiled from
source, or installed from a package that was not built by someone with a
systemd-based system.

For me, I fixed it by writing my own service override file. I put this in
/etc/systemd/system/bacula-sd.service.d/override.conf :

 [Unit]
After=network.target nss-lookup.target NetworkManger-wait-online.service
autofs.service

I put autofs.service in there because I am using autofs to mount
USB-attached disks that contain my Bacula volumes; you may not need that.
NetworkManager-wait-online.service is the critical piece (and of course you
may need to use systemctl to enable the NetworkManager-wait-online service).

There is also an easier but kludgy fix that usually works, which is to put
a sleep into the startup script to get it to wait a while for the network
to come up before starting bacula-sd. Not guaranteed to work every time,
but in practice this may be good enough. It does introduce a few seconds
delay in getting the system fully booted, but perhaps that's not a big deal
for you.

--Greg
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] trying to build a static bacula-fd

[Greg Woods]

On Fri, Feb 9, 2018 at 6:26 AM, Mike Eggleston  wrote:

>
> /bin/ld: cannot find -lz
> /bin/ld: cannot find -lpthread
> /bin/ld: cannot find -ldl
> /bin/ld: cannot find -ldl
> /bin/ld: cannot find -lssl
> /bin/ld: cannot find -lcrypto
> /bin/ld: cannot find -lstdc++
> /bin/ld: cannot find -lm
> /bin/ld: cannot find -lc


You are missing the packages that contain the libraries for static linking.
For instance, on my Fedora system, "dnf whatprovides */libssl.a" shows that
I would need the "openssl-static" package. You would need to do the
equivalent for your distro and install the packages that provide those
static libraries.

--Greg
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment

[Greg Woods]

On Mon, Jan 4, 2016 at 6:51 AM, Peter Keller  wrote:

>
>
> # bacula-dir -t
>
> If it comes back with a 0 return value, then your config file is of a good
> syntax. Then try:
>
> # bacula-dir -v -f -d100
>
>
>
Excellent suggestions. Also useful in debugging systemd unit files is the
journal. Something like:

# journalctl -xb | grep bacula

(shows all bacula-related entries since the last boot).

--Greg
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment

[Greg Woods]

On Thu, Dec 24, 2015 at 8:34 AM, Daniel Bareiro 
wrote:

> Greg, you mention having compiled the SD on Jessie, but I guess you've
> also compiled the Director, right? Since both must have the same
> version. Though I suppose that the Director may also be on another host
> that includes Bacula 7.0.5 in its repositories.
>

The latter is correct. Most of my machines are at Fedora 23. At release,
these had 7.0.5, although 7.2.0 update packages have since been released.
The one system that is not Fedora 23 is my Raspberry Pi that I use as the
storage server; that runs Debian Jessie. In order to be able to use the
7.0.5 director, I had to compile 7.0.5 storage server on Jessie. The
hardest part about this was creating a proper systemd unit file for
bacula-sd. I finally installed the bacula-sd package, saved off the systemd
unit file that it provided, then removed the package and installed the
compiled storage daemon. This all works, but it is very convoluted.


> About using Jessie, is it worth compile the Director and SD?
>

If you're going to compile one, you should probably compile both.

Now I have heard that Fedora 23 can be installed on a Raspberry Pi 2, and I
got one of those for Christmas, so I will be attempting that this weekend.
This isn't because I have anything against Debian, it's just that
maintenance will be a lot easier if I can use the distro's package
management system and easily update all the various Bacula components at
the same time. Running "dnf update bacula*" is way easier than compiling
from source and mucking about with systemd. And now I have had to exclude
bacula packages from updates on my systems to avoid having the file daemons
upgraded to a newer version than my storage server, so this becomes even
more of an issue for me now.

--Greg
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Question about versions to be installed in an Linux / Windows environment

[Greg Woods]

On Wed, Dec 23, 2015 at 5:10 AM, Daniel Bareiro 
wrote:

> am thinking of using Debian Jessie which includes Bacula 5.2.6 in its
> repository for director and storage daemon. But I remember a few cases
> where I have experienced incompatibilities if the client versions are
> far from the server version. Correct me if I'm wrong, please.
>

The requirement is FD <= (SD == DIR). The storage daemon and director must
be the same version, and must be newer than or the same as  any file
daemons of any of the clients. I do use a 7.0.5 director and storage daemon
(compiled from source on Debian Jessie in the SD case) with some clients
that have 5.2.6 file daemons and that works just fine.

--Greg
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula and NFS

[Greg Woods]

On Wed, Dec 16, 2015 at 9:58 AM, Bryn Hughes  wrote:

> The 'ERR=Connection refused' message means that the storage daemon itself
> isn't accepting connections.  This could be due to firewall rules or due to
> mismatched passwords between the storage daemon and the other components.


This can also happen due to a version mismatch. The requirement is

FD <= (SD == DIR)

(someone will no doubt correct me if I have screwed that up :-)

--Greg
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] authorization errors after upgrade

[Greg Woods]

I am trying to upgrade from 5.2.13 to 7.0.5, and I am getting authorization
errors between the director daemon and storage daemon. I am using the
identical config files in both versions, so I am sure that the passwords
match.

The director and storage run on different machines, but they are both 7.0.5
.

If it matters, the director is Fedora 23, and the storage is Debian Jessie
(on a Raspberry Pi). The director was installed from the release packages,
the storage compiled from source (as the Pi version of Jessie still has
5.2.13).

Are there changes needed in the 7.0.5 config files to make it work with
5.2.13 config files?

Thanks,
--Greg
--
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] SQL query for copy jobs

[Greg Woods]

Thanks for looking at this.

I am a bit redfaced right now, because it looks like I have to withdraw the
question. When I used "bat" to view the Jobs Run, and clicked "Filter Copy
Jobs", a lot of the confusing stuff was removed, and I could see that the
dates actually had a monotonically increasing progression. Which means
that, in fact, I wasn't getting duplicate jobs copied and the ones I
thought hadn't been copied actually were.


> To check if a job was already copied you have to verify priorjobid column
> in job table. The original job will have a priorjobid=0 and a copied job
> will have a priorjobid= of the original.
>

Yes, that is exactly what the "ArchiveAll" job is counting on.


>
> Your SQL is very weird to me. You copy jobs from pool File to the Pool
> File?
>

Sorry for the confusion there.

 Pool {
  Name = File
  Pool Type = Backup
  Recycle = yes   # Bacula can automatically recycle
Volumes
  AutoPrune = yes # Prune expired volumes
  Volume Retention = 90 days # 3 months
  Maximum Volume Bytes = 50G # Limit Volume size to something
reasonable
  Maximum Volumes = 70# Limit number of Volumes in Pool
  Storage = BKUP
  Next Pool = Archive # Where to copy/migrate to
}

The Pool definition for the File pool lists "Next Pool = Archive", so the
copy jobs really were being written to the Archive pool.

Sorry for the noise everyone.

--Greg
--
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991=/4140___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] SQL query for copy jobs

[Greg Woods]

I have a job called ARCHIVE-ALL that looks like this:

JobDefs {
  Name = "ArchiveJob"
  Type = Copy
  Level = Full
  Selection Type = SQLQuery
  Messages = Standard
  File Set = Copy
  Pool = File
}

# Copy (to archive) all not-already-copied successful backup jobs
Job {
   JobDefs = ArchiveJob
   Name = "ARCHIVE-ALL"
   Client = bstore
   Selection Pattern = "select JobId from Job j where PoolId=2 and
Type='B'and Jobstatus='T' and (select count(*) from Job where PriorJobId =
j.JobId)  = 0"
}

PoolId=2 corresponds to my "File" pool, which is where all the backups are
written to initially. I also have an Archive pool where the copy jobs get
written.

This worked well for over a year, but recently, the SQL query is matching a
whole bunch of jobs that have already been copied, and it is failing to
match some (like a Full backup I did on a client just today) that need to
be copied.

I am theorizing that the reason it stopped working correctly is that a lot
of the jobs have now been expired, and a recent round of full backups
required that some jobs from the File pool had to be purged. Does this
reset the PriorJobId to zero for any jobs that had a PriorJobId pointing to
a job that is being purged?

Does anybody have an SQL query that will do what I want (which is to copy
all the backup jobs from the File pool to the Archive pool that have not
yet been copied?

Thanks,
--Greg
--
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991=/4140___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bacula-users replyTo address

[Greg Woods]

On Wed, Apr 8, 2015 at 5:24 AM, Robert Oschwald robertoschw...@gmail.com
wrote:


 Is it possible for the admins to set the reply-to address to the
 list-address so replies are posted to the list instead of the author?


Possible? Probably. Desireable? This is one of those religious wars  that
comes up on every mailing list eventually. I personally hate reply to
list; I would much rather accidentally reply to author when I meant reply
to list than accidentally send a reply to the list that was intended for
only one person. Others will argue just as vociferously the other way.

--Greg
--
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15utm_medium=emailutm_campaign=VA_SF___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Is this an Inter-version Compatibility Issue???

[Greg Woods]

On Wed, Jan 7, 2015 at 9:49 AM, John Florian john.flor...@dart.biz wrote:


 My Fedora 21 workstation has:



 bacula-client-7.0.5-2.fc21.x86_64

 bacula-common-7.0.5-2.fc21.x86_64

 bacula-libs-7.0.5-2.fc21.x86_64





I ran into the same problem. I solved it by removing the bacula packages,
then installing the fc20 RPMs. You probably will have to download the RPMs,
then installl them from the local RPM files you downloaded.

My problem is that my SD server is a Raspberry Pi, and there are no Bacula
7 packages for it. I'd have to compile from source. It was easier to
downgrade the f21 system than to do this. Eventually I may try it, because
the system that the Bacula director runs on is going to need upgrading soon
too (for reasons having nothing to do with Bacula).

--Greg
--
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Problem with schedule

[Greg Woods]

On Sun, Jan 4, 2015 at 11:58 PM, Florian florian.spl...@web.de wrote:

  Hello everyone.

 I seem to be having a strange issue with the schedule of my backups.
 This is the schedule I use:

 Schedule {
   Name = MonthlyCycle
   Run = Full on 1 at 3:15
   Run = Differential sun at 3:15
   Run = Incremental mon-sat at 3:15
 }

 I do not know why, but on the first of every month, bacula tries to do
 both the Full backup AND the incremental


I think you need to explicitly tell bacula that you don't want the
incremental run on the 1st. For example, I have this:

 Schedule {
   Name = worldsys
   Run = Level=Incremental on 1-2 at 00:05
   Run = Level=Incremental on 4-31 at 00:05
   Run = Level=Differential on 3 at 00:05
}

I am using Bacula in a small home operation, so I run my Full backups by
hand, once every 6 months, but I want to run a Differential instead of an
Incremental once a month. The above schedule says to run the incremental on
all days except the 3rd, and run a differential on the 3rd.

--Greg

-- 

--Greg
--
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bacula backups failing

[Greg Woods]

Maybe it's removing the existing /var/bacula/.my.cnf file and then trying
to write a new one? What are the permissions on /var/bacula?

--Greg
--
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] symbol lookup error: /usr/lib/libbaccats-7.0.2.so: undefined symbol: mysql_thread_safe

[Greg Woods]

On Sun, May 4, 2014 at 6:44 AM, Kern Sibbald k...@sibbald.com wrote:

 MariaDB is not 100% compatible with MySQL, and thus possibly not
 compatible with Bacula.


Just as a data point: I am using Bacula with the director and database
running on Fedora 20. This distro is using Mariadb. It works fine with
Bacula, although this is only a small home installation, I can't speak to
how well it would work with a large enterprise installation.


--Greg
--
Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
unparalleled scalability from the best Selenium testing platform available.
Simple to use. Nothing to install. Get started now for free.
http://p.sf.net/sfu/SauceLabs___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about how retention works

[Greg Woods]

Good, we're getting somewhere.

On Fri, 2013-12-20 at 11:45 -0500, Dan Langille wrote:

 Let's confirm this.  Let's look at this job:

{SQL stuff snipped}

That is what I was eventually going to get around to doing, so thanks
for doing that for me. So it looks like, if the same file is written to
Volumes in multiple jobs, then there are multiple File records for it.

  Building directory tree for JobId(s)
  56,3603,3609,3616,3622,3710,3716,3722,3832,3839 ...
  ++
  
  For one or more of the JobIds selected, no files were found,
  so file selection is not possible.
 
 Ahh, look there: 'one or more'
 
 Perhaps you are able to restore by regex because you're using the jobs 
 which are not passed the File retention period?

I'll have to check that, but I'm pretty sure I was able to restore the
entire /etc/.* tree, and the Full backup is definitely more than 30 days
old and got pruned. But I'll verify that.

But there is also the bscan command, the man page for which states:

   The  purpose  of bscan is to read (scan) a Bacula Volume and to
recreate or update the database contents
   with the information found on the Volume.  This is done in a
non-destructive way. This permits restoring
   database entries that have been lost by pruning, purging,
deleting, or a database corruption problem.

This implies that there is information on the Volumes that duplicates
what's in the database, but it doesn't say whether or not that includes
File records. So there is also the bls command, the man page for which
implies that it can list the files from a volume. So I tried it. It's
slow on my poor Raspberry Pi (the bacula-sd server), but it works:

bls: butil.c:287 Using device: BKUP for reading.
20-Dec 10:47 bls JobId 0: Ready to read from volume BKUP.2 on device
BKUP (/bkup/bacula).
bls JobId 0: drwxrwxr-x   2 root 499 4096 2013-12-07
11:34:48  /var/lib/VirtualBox/
bls JobId 0: -rw-r--r--   1 989  986 3072 2013-12-08
00:32:01  /var/spool/asterisk/astdb.sqlite3
bls JobId 0: drwxr-x---   7 989  986 4096 2013-12-08
00:32:01  /var/spool/asterisk/
bls JobId 0: drwxrwx---   2 51   51  4096 2013-12-08
00:14:38  /var/spool/clientmqueue/
bls JobId 0: drwx--x---   3 root lp  4096 2013-12-07
09:04:26  /var/spool/cups/
bls JobId 0: drwx--   2 root man 4096 2013-12-08
00:14:39  /var/spool/mqueue/
bls JobId 0: -rw---   1 root root   9 2013-12-08
01:01:01  /var/spool/anacron/cron.weekly
bls JobId 0: -rw---   1 root root   9 2013-12-08
01:01:01  /var/spool/anacron/cron.monthly
bls JobId 0: -rw---   1 root root   9 2013-12-08
01:01:01  /var/spool/anacron/cron.daily
[]

This does not contain information about which client those files came
from though. But that information would be available from Job records if
those are still present in the database.

So it would appear that, in fact, there is information on the Volumes
that duplicates the File records in the database, which means I may well
have been restoring files for which there were no longer any File
records in the database. I'll play some SQL games this weekend and see
if I can find an explicit example for which there is no File record but
I can still restore the file.

I hope some others out there have found this to be as informative as I
have.

--Greg


 
  Most likely your retention policy pruned the files.
  
  Do you want to restore all the files? (yes|no): no
  
  Regexp matching files to restore? (empty to abort):
  
  At this point, I can type in a regex. Then it shows the usual 
  parameters
  followed by yes/no/mod, which I can then modify in the usual way, and
  then, although it takes quite a while, it will restore the files that
  matched the regex.
  
  
  Does what you've observed make sense when you consider my claim of
  'first one counts'?
  
  I'm not sure I can say it makes sense to me until I find some time to
  poke around in the database to see what is really going on.
  
  The one thing I am afraid of is having a Volume in the main pool 
  getting
  recycled causing the File records for files in the archive pool getting
  pruned.
 
 I think that is why I keep File and Job retention to 3 years.  To avoid 
 this situation.  I let Volume retention dictate how long I keep File and 
 Job records.  i.e. up to 3 years.
 



--
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET,  PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bacula-users mailing list

Re: [Bacula-users] question about how retention works

[Greg Woods]

On Fri, 2013-12-20 at 13:14 -0500, Dan Langille wrote:

 Read http://www.bacula.org/en/dev-manual/main/main/Restore_Command.html
 
 'If a Job's file records have been pruned from the catalog, the restore 
 command will be unable to find any files to restore. Bacula will ask if 
 you want to restore all of them or if you want to use a regular 
 expression to restore only a selection while reading media. See 
 FileRegex optionFileRegex and below for more details on this.'

That pretty much implies that it *is* possible to restore files for
which there are no more file records in the catalog. And that is a VERY
good thing. I would hate to have all my backup data become useless due
to a database crash (yes, I do back up my database outside of Bacula
just in case, but still...)

--Greg



--
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET,  PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about how retention works

[Greg Woods]

On Thu, 2013-12-19 at 11:02 -0500, Dan Langille wrote:
 There was a lot in the above and I couldn't parse it all 
 just now. 

Sorry about that; I suppose it would help if I just explicitly stated
what I really want. I have a main backup pool where all the backup jobs
write. This pool is manually copied to the archive pool using Copy jobs.
What I want is for volumes in the main pool to be recycled relatively
quickly (because space is limited), while the files are retained much
longer in the archive pool. I'd like restore attempts to automatically
figure out when the files are still available in the backup pool and use
it when possible, but get them from the archive pool if necessary.
That's what I'm working toward.


  Instead, I'll point you to something I wrote a few years ago. 
   If you have questions after that, please ask again.
 
 http://dan.langille.org/2010/02/08/bacula-retention-periods/

OK, good. It's good that we're having this discussion. If I understand
any of this at all (which I may not), there are a couple of things in
there that seem not to match my experience so far. For instance, in that
document it says you can’t restore a given file after the File
retention passed. But that's not really true. You can't restore it in
the usual way, by marking files, presumably because the list of
available files for marking is created using those File records that
aren't there any more. But you *can* restore those files by using a
regular expression; I've had to do this because I have lost a lot of my
File records by initially setting File Retention too low. I'm not sure
how it actually does this, but there must be some information written to
the backup volume itself that can be used to determine what files are
there. It's much slower than a database access, but I have restored
directory trees and individual files this way.

The document also says Volumes are eligible for recycling once ALL
retention periods have passed.  But that doesn't seem to be true
either; I have apparently seen Bacula recycle some volumes even though
the Job retention period has not passed. Which means, presumably, that
there are Job records in the database that point to backup files that
aren't there any more, since the Volume has been recycled. 


 Perhaps the Recycling Algorithm will help:
 
 http://www.bacula.org/en/dev-manual/main/main/Automatic_Volume_Recycling.html


That does help, and I've looked at this before, but it's not always
clear (to me) exactly how it works, even after reading that.

I notice that the paragraph on Automatic Pruning says that you can't
restore files once the File records are gone, but I know that's not true
because I have done it. I suspect the documentation may be a few
versions old and I can't necessarily trust everything it says.

It also states that when volume recycling happens, all Jobs that are
older than the VolumeRetention period will be pruned, presumably
regardless of what I set for Job Retention, which leads to some of my
confusion about what  these retention periods really mean. It's even
more confusing (to me) when you throw in Copy jobs, so that the same
files may be on multiple Volumes in different pools.

--Greg



--
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET,  PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about how retention works

[Greg Woods]

On Thu, 2013-12-19 at 12:38 -0500, Dan Langille wrote:

 I set my FILE and JOB retentions high.  3 years.  Then I set my VOLUME 
 retention lower.  Whichever retention period expires first, that's the 
 one which counts. == I will refer to that as 'first one counts' later 
 in this email.
 
 Therefore, in your archive pool, set your VOLUME retention to 3 years, 
 for example.
 
 And in your main pool, set it to 5 weeks, for example.
 
 Does that help?

Yes. This is more-or-less what I have now. I have File Retention = Job
Retention = 365 days. In the main pool, Volume Retention = 30 days, and
in the archive pool, Volume Retention = 365 days. I'm just trying to
make sure that this is really going to accomplish what I want.

The one big question remaining is how Copy jobs affect this. If I use a
Copy job to make a copy of one or more jobs (usually all the jobs for a
given client) from the main pool to the archive pool, what happens to
the File records? Are there now two File records for every file on the
backup? For me that's fine, but in an enterprise setup with thousands of
clients, effectively doubling the size of the database would be a pretty
big deal, so it wouldn't surprise me to learn that there are some
efficiency improvements there. That's what leads me to wonder what
really happens when a Volume in the main pool is recycled. The Recycle
Algorithm document suggests that the File records associated with that
Volume are pruned, so does that or does that not mean that the
corresponding jobs in the archive pool no longer have File records? I
may have to poke around in the database a little to see if I can answer
that question for myself.


  I notice that the paragraph on Automatic Pruning says that you can't
  restore files once the File records are gone, but I know that's not 
  true
  because I have done it. I suspect the documentation may be a few
  versions old and I can't necessarily trust everything it says.
 
 You've done it a different way.  Not the 'usual way'.

I guess that depends on how you define the usual way, so I'll clarify.
This is what I see, running restore out of bconsole. I first select
most recent backup for a client, and go through the usual process of
selecting the client and the File Set, and then I see:

You have selected the following JobIds:
56,3603,3609,3616,3622,3710,3716,3722,3832,3839

Building directory tree for JobId(s)
56,3603,3609,3616,3622,3710,3716,3722,3832,3839 ...
++

For one or more of the JobIds selected, no files were found,
so file selection is not possible.
Most likely your retention policy pruned the files.

Do you want to restore all the files? (yes|no): no

Regexp matching files to restore? (empty to abort): 

At this point, I can type in a regex. Then it shows the usual parameters
followed by yes/no/mod, which I can then modify in the usual way, and
then, although it takes quite a while, it will restore the files that
matched the regex.

 
 Does what you've observed make sense when you consider my claim of 
 'first one counts'? 

I'm not sure I can say it makes sense to me until I find some time to
poke around in the database to see what is really going on.

The one thing I am afraid of is having a Volume in the main pool getting
recycled causing the File records for files in the archive pool getting
pruned.

--Greg



--
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET,  PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] question about how retention works

[Greg Woods]

There is some stuff I still don't understand here, and I'd like to avoid
learning the hard way. For instance, I never realized that File
Retention referred to the individual file records in the database, not
to the actual files on backup. So I ended up losing many of my File
records (learning the hard way), when what I really wanted was for the
volumes containing those files to be made available for re-use when
necessary. So now I *think* I have accomplished this by setting Job
Retention and File Retention to 365 days, and Volume Retention to 30
days. I would assume that, after 30 days since the last time a Volume
was written, it will be available for recycling, but I won't lose the
database records when that happens. In my case, I'm only using Bacula
for a dozen or so systems and VMs at home as clients, so I'm not overly
concerned with the size of the database.

What good does it do to have database records for files that are no
longer available, you may well ask? I use Copy jobs to copy my online
backup media to my archive backup media. The former is just one big
disk, the latter is a series of disks inserted in an external USB caddy,
implemented with vchanger. I periodically run Copy jobs to copy backups
from the main disk to the vchanger, so that if the main disk volume gets
recycled, the files still exist on the vchanger. Since these are
separate Pools, I can and do set the Volume Retention differently for
each. But my impression is that the Copy jobs don't really create
separate File records. File Retention seems to be set only on a per
client basis, so if I want to be able to do full-feature restores from
the vchanger, I need to keep the File records around even if the volume
in the main backup Pool has been recycled.

Am I understanding how this works correctly?

Next question is about volume purging. How does Bacula determine when it
is time to recycle a volume? Will it do this automatically after the
Volume Retention period has expired? How exactly is the Volume Retention
period measured? (Meaning, if I set it to 30 days, it's 30 days from
when?)  When a volume expires, does it immediately become unusable, or
only when space is needed? When space is needed, will Bacula
automatically use a volume that is past its retention period, or do I
have to do something manually to make the expired volumes available?

Sorry to be so dense here. I have spent a lot of time reading the online
docs, and I've been using Bacula since August, but some of this is still
not clear to me.

Thanks,
--Greg



--
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET,  PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database

[Greg Woods]

On Sat, 2013-11-23 at 15:18 -0700, Greg Woods wrote:
  I
 played games trying to give Mysql more resources in my.cnf, but I don't
 think that's really where the problem is. 

It turns out that *is* where a problem is, but there were absolutely no
error messages anywhere to indicate this, other than the generic cannot
connect to MySQL server error. I did check the bacula logs, the system
log files and the mysqld log file.

I think the problem was that I was using the Selection Type =
PoolUncopiedJobs, and the number of jobs to be copied had gotten larger
than previously and caused some limit to be exceeded. I  increased a
number of resource limits in my.cnf, but it still doesn't work.

I don't see any smoking gun in the bacula or system logs to tell me what
the problem is. The only error message I see anywhere is Cannot connect
to database, either the server is not running or your authorization is
incorrect. However, I know the authorization is correct and the server
is running, so it's got to be some kind of resource exhaustion. I did
try running a copy for a single client. That also works, also pointing
to resource exhaustion. The problem with that is that it also copies
jobs that have already been copied. What I'd really like is to be able
to copy all jobs for a single client that have not already been copied,
so I can do all of them a few jobs at a time, but there doesn't seem to
be any way to do that. Maybe with some kind of SQL query? I could maybe
develop one, but it would take a lot of time to learn enough about how
the database is structured to be able to do that, so I want to have
someone who knows say that this is possible before I invest all that
time.

Another approach is to accelerate my move of Bacula and MythTV to a new
machine with more RAM, which I was going to do anyway. Then I can
allocate more memory to the database. But of course I don't know for
sure that that will fix the issue either.

--Greg



--
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing 
conversations that shape the rapidly evolving mobile landscape. Sign up now. 
http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database

[Greg Woods]

On Sun, 2013-11-24 at 11:32 -0700, Greg Woods wrote:
 . I  increased a
 number of resource limits in my.cnf, but it still doesn't work.

Got a teeny bit farther. Now, with increased limits, bacula-dir no
longer segfaults, and I no longer get the database connection error. The
problem now is that all the copy jobs are spawned, but the first one is
just hanging there doing nothing. Job log looks normal so far:


Log records for job 3496

2013-11-24 11:54:20


bdir


Copying using JobId=2014 Job=orolo.2013-11-02_19.00.09_28


Bootstrap records written to /var/spool/bacula/bdir.restore.1.bsr
2013-11-24 11:54:23


bdir


Start Copying JobId 3496, Job=ARCHIVE-ALL.2013-11-24_11.54.20_06

That was 20 minutes ago and it's still just sitting there. The Media tab
in bat shows none of the archive volumes are being written to (although
they are online).  I'll let it sit there for a while longer just in
case, but it kind of looks like I have to find a way to limit the number
of jobs spawned at once while also avoiding copying jobs that have
already been copied.

--Greg



--
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing 
conversations that shape the rapidly evolving mobile landscape. Sign up now. 
http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] ERROR: Msg delivery error: Unable to store data in database

[Greg Woods]

Isn't there some way I can debug this? I have spent my entire Saturday
trying to fix this unsuccessfully. My backup jobs work, but copy jobs
fail. I really haven't changed anything in the Bacula or Mysql
configurations before today in quite a while (that I know of). Today I
played games trying to give Mysql more resources in my.cnf, but I don't
think that's really where the problem is. It looks like an
authorization/grants issue but I'm damned if I can figure out what grant
is needed or why this has suddenly started happening when it has been
working fine for months.  I'll put in the time, I'll read the docs, if
somebody can just point me to somewhere that will help me make some
progress. I have to give up for today because I am totally frustrated. I
will post relevant config files if that will help.

Today I tried updating Bacula and Mysql on the director server (now at
5.2.13-17 and  5.5.34-1 respectively) but it didn't make any difference.
I am running 64-bit Fedora 18 if that matters (and yes, I know it will
be end-of-life soon but as of now it's still supported).

--Greg

On Thu, 2013-11-21 at 11:13 -0700, Greg Woods wrote:
 On the surface this is obvious, but I don't see why this should suddenly
 be happening. I haven't modified the configurations for either
 bacula-dir or mysql in some time, yet suddenly now the director can't
 access the database. Even weirder, this is only happening for Copy jobs,
 normal backups work fine. Manually accessing the database from the
 bacula account command line also works fine.
 
 Anybody ever seen anything like this or have any tips for debugging
 this?
 
 Thanks,
 --Greg
 
 Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05  Message
 delivery ERROR: Msg delivery error: Unable to store data in database.
 Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05  Message
 delivery ERROR: Msg delivery error: Unable to store data in database.
 Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05  Message delivery
 ERROR: Msg delivery error: Unable to store data in database.
 Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05  Message delivery
 ERROR: Msg delivery error: Unable to store data in database.
 Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 bdir: ABORTING
 due to ERROR in mem_pool.c:167
 Nov 21 11:05:33 confusion bacula-dir[14226]: obuf is NULL
 Nov 21 11:05:33 confusion bacula-dir[14226]: bacula-dir: message.c:1270:
 void e_msg(const char*, int, int, int, const char*, ...): Assertion `p!
 =__null' failed.
 Nov 21 11:05:33 confusion bacula-dir[14226]: Bacula interrupted by
 signal 6: IOT trap
 Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! bacula-dir, bdir
 got signal 6 - IOT trap. Attempting traceback.
 Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! exepath=/usr/sbin/
 Nov 21 11:05:33 confusion bacula-dir: bdir: ABORTING due to ERROR in
 mem_pool.c:167
 Nov 21 11:05:33 confusion bacula-dir: Bacula interrupted by signal 6:
 IOT trap
 Nov 21 11:05:33 confusion bacula-dir[14226]:
 Calling: /usr/sbin/btraceback /usr/sbin/bacula-dir
 14226 /var/spool/bacula
 Nov 21 11:05:33 confusion bacula-dir[14226]: It looks like the traceback
 worked...



--
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing 
conversations that shape the rapidly evolving mobile landscape. Sign up now. 
http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] ERROR: Msg delivery error: Unable to store data in database

[Greg Woods]

On the surface this is obvious, but I don't see why this should suddenly
be happening. I haven't modified the configurations for either
bacula-dir or mysql in some time, yet suddenly now the director can't
access the database. Even weirder, this is only happening for Copy jobs,
normal backups work fine. Manually accessing the database from the
bacula account command line also works fine.

Anybody ever seen anything like this or have any tips for debugging
this?

Thanks,
--Greg

Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05  Message
delivery ERROR: Msg delivery error: Unable to store data in database.
Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05  Message
delivery ERROR: Msg delivery error: Unable to store data in database.
Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05  Message delivery
ERROR: Msg delivery error: Unable to store data in database.
Nov 21 11:05:33 confusion bacula-dir: 21-Nov 11:05  Message delivery
ERROR: Msg delivery error: Unable to store data in database.
Nov 21 11:05:33 confusion bacula-dir[14226]: 21-Nov 11:05 bdir: ABORTING
due to ERROR in mem_pool.c:167
Nov 21 11:05:33 confusion bacula-dir[14226]: obuf is NULL
Nov 21 11:05:33 confusion bacula-dir[14226]: bacula-dir: message.c:1270:
void e_msg(const char*, int, int, int, const char*, ...): Assertion `p!
=__null' failed.
Nov 21 11:05:33 confusion bacula-dir[14226]: Bacula interrupted by
signal 6: IOT trap
Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! bacula-dir, bdir
got signal 6 - IOT trap. Attempting traceback.
Nov 21 11:05:33 confusion bacula-dir[14226]: Kaboom! exepath=/usr/sbin/
Nov 21 11:05:33 confusion bacula-dir: bdir: ABORTING due to ERROR in
mem_pool.c:167
Nov 21 11:05:33 confusion bacula-dir: Bacula interrupted by signal 6:
IOT trap
Nov 21 11:05:33 confusion bacula-dir[14226]:
Calling: /usr/sbin/btraceback /usr/sbin/bacula-dir
14226 /var/spool/bacula
Nov 21 11:05:33 confusion bacula-dir[14226]: It looks like the traceback
worked...




--
Shape the Mobile Experience: Free Subscription
Software experts and developers: Be at the forefront of tech innovation.
Intel(R) Software Adrenaline delivers strategic insight and game-changing 
conversations that shape the rapidly evolving mobile landscape. Sign up now. 
http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] copy, delete, purge question

[Greg Woods]

I just want to make sure I understand how this is supposed to work
before I do something that might cause me to lose backups.

Suppose I take a full backup for a couple of clients to a backup pool.
Then I successfully run a copy job which copies these to an archive
pool. Am I correct in assuming that, if I now delete the original backup
jobs, any volumes in the backup pool that contain only files from the
deleted jobs will be marked Purged and be available for re-use? And that
the copy jobs will now be marked as backups and I would be able to
restore files from those jobs from the volumes in the archive pool?

That's what I want to happen; I want to be able to delete jobs that I
have archive copies of, to free up space in the backup pool, but still
be able to restore files from the archived jobs. (The backup pool is
always online so that I can schedule backups 24x7, the archive pool is
much larger, and uses removable drives that are only connected while in
use, so they can't be taken out by a power surge etc. and can be stored
off site to guard against total disaster).

--Greg



--
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register 
http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] vchanger (one more time) SOLVED

[Greg Woods]

On Sat, 2013-09-21 at 11:23 -0600, Greg Woods wrote:

 Unless I forgot something else, this isn't a permissions problem. The
 problem is that initmag does not work:
 
 #   vchanger -u bacula -g bacula /etc/bacula/ARCH.conf initmag 1
 magazine bay 1 has no magazine mounted

[...]

 Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID
 returned -3

I thought about this, and it looked as though perhaps the uuid library
package I had installed was not actually working. It links OK, but it
doesn't work when called. So I tried Josh Fisher's suggested fix for the
linking issue:

 The quick and dirty way is to comment out the vchanger_LDADD line in
the Makefile.am in the src directory and replace the uuidlookup.c file
with a version having a single function as follows:

A GetMountpointFromUUID function that does nothing and always returns
failure (look for the code and complete instructions in the USB tape
drives? thread if you need it). This disables the ability to declare
magazines by UUID, but since I was already automounting by UUID anyway,
it was merely a syntactic change to switch to defining magazines by
their mount points instead. So in ARCH.conf, instead of:

magazine = UUID:fea7e396-427b-49c4-af1e-195a6e6036dd

I now have:

magazine = /wd/mag/fea7e396-427b-49c4-af1e-195a6e6036dd

...and now initmag worked, and I was able to move on. I was able to
successfully create and run Copy jobs, and I am a happy camper. The Copy
jobs are horribly slow on the Raspberry Pi (copying from one USB disk to
another), but I can live with that. 

Thanks to everyone who offered advice.

--Greg




--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] vchanger

[Greg Woods]

On Fri, 2013-09-20 at 19:20 -0600, Greg Woods wrote:
 I'm having some trouble getting vchanger to work, partly because the
 documentation that comes with it seems not to be up to date.

Ugh, never mind. I've been spending so much time on this and doing so
many searches that I got my versions of the Howto confused. It looks
like the one that actually comes with the source code is not the one I
was actually reading. Now that I have the right Howto, I'll see how it
goes from here. Sorry for the noise.

--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] vchanger (one more time)

[Greg Woods]

I apologize for yet another posting, but I am just having a HELL of a
time trying to get vchanger to work. I have been following the
instructions in the Howto that came with the source code. I have the
automounting working, and the permissions are all correct (the bacula
user owns the work_dir and the automount_dir, as well as the root
directory of my first magazine drive:

# ls -ld /var/spool/bacula/mag /wd/mag /wd/mag/*
drwxr-xr-x 2 bacula bacula 4096 Sep 21 11:06 /var/spool/bacula/mag
drwxr-xr-x 3 bacula bacula0 Sep 21 11:06 /wd/mag
drwxrwxrwx 3 bacula bacula 4096 Sep 20
19:06 /wd/mag/fea7e396-427b-49c4-af1e-195a6e6036dd

Unless I forgot something else, this isn't a permissions problem. The
problem is that initmag does not work:

#   vchanger -u bacula -g bacula /etc/bacula/ARCH.conf initmag 1
magazine bay 1 has no magazine mounted

The initmag command does trigger a successful automount before the
error occurs.

Obviously I cannot get past this.

Another oddity is, if I run the listmags command, even with no disk
drive inserted, I get this:

# vchanger -u bacula -g bacula /etc/bacula/ARCH.conf listmags
1::0:

According to the Howto, the 0 should be a -1 when there is no disk
inserted.

I also end up with this in the work_dir:

# ls -l /var/spool/bacula/mag
total 4
-rw-r--r-- 1 bacula bacula 1 Sep 21 11:15 bay1
-rw-r--r-- 1 bacula bacula 0 Sep 21 10:23 state0

(the bay1 file contains only a single newline).

I must have made some stupid mistake in configuring vchanger, but I'm
damned if I can see it. Here is my ARCH.conf file:

changer_name = ARCH
slots_per_magazine = 34
virtual_drives = 1
magazine_bays = 1
automount_dir = /wd/mag
work_dir = /var/spool/bacula/mag
log_level = LOG_DEBUG
logfile = /var/log/ARCH.log
magazine = UUID:fea7e396-427b-49c4-af1e-195a6e6036dd
magazine = UUID:c15c828b-0237-4b05-aa85-a9e3102ffe42
magazine = UUID:fe3c5faf-7ea2-4051-8a92-bc4c6c94b0ab
magazine = UUID:6c7279d5-f287-4e3b-8a33-51e398a3c932

Here's what's in the log file after the initmag command:

Sat Sep 21 11:20:43 2013 Bay 1 was previously unloaded
Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID
returned -3
Sat Sep 21 11:20:44 2013 Init: magazine
UUID:fea7e396-427b-49c4-af1e-195a6e6036dd not mounted
Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID
returned -3
Sat Sep 21 11:20:44 2013 Init: magazine
UUID:c15c828b-0237-4b05-aa85-a9e3102ffe42 not mounted
Sat Sep 21 11:20:44 2013 MagazineState::set() - GetMountpointFromUUID
returned -3
Sat Sep 21 11:20:44 2013 Init: magazine
UUID:fe3c5faf-7ea2-4051-8a92-bc4c6c94b0ab not mounted
Sat Sep 21 11:20:45 2013 MagazineState::set() - GetMountpointFromUUID
returned -3
Sat Sep 21 11:20:45 2013 Init: magazine
UUID:6c7279d5-f287-4e3b-8a33-51e398a3c932 not mounted
Sat Sep 21 11:20:45 2013 confirm bay 1 loaded with mag 0 = 
Sat Sep 21 11:20:45 2013 Init: drive 0 last state was unloaded
Sat Sep 21 11:20:45 2013 Info: drive 0 is unloaded
Sat Sep 21 11:20:45 2013 magazine bay 1 has no magazine mounted


I have been struggling with this for two days now, so any ideas would be
gratefully accepted.

Thanks,
--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] USB tape drives?

[Greg Woods]

On Fri, 2013-09-20 at 12:17 -0600, Greg Woods wrote:
 On Tue, 2013-09-17 at 13:13 +0300, Guy wrote:
  Yes I do this with vchanger... 
 
 I've run into a wall trying to get vchanger to compile on Raspbian (a
 limited version of Debian for the Raspberry Pi).

I did solve the compile issue. With a lot more Googling (I really wish I
were better at searching for things), I did find the -dev packages. For
some unknown reason, they are not in the repository, I had to download
them manually and install them. When I did that and re-ran configure and
make, it does compile.

Now on to see if I can get it to work.

--Greg




--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] USB tape drives?

[Greg Woods]

On Tue, 2013-09-17 at 13:13 +0300, Guy wrote:
 Yes I do this with vchanger... 

I've run into a wall trying to get vchanger to compile on Raspbian (a
limited version of Debian for the Raspberry Pi).

When I run configure, it notes that I do not have libuuid. I believe
this is correct; there is a shared library (.so) version, but there is
no development (.a) version. However, it still tries to use -luuid
anyway, and I get linker errors:

libtool: link: g++ -g -O2 -o vchanger vchanger-vchanger_common.o
vchanger-compat_util.o vchanger-junctions.o vchanger-uuidlookup.o
vchanger-util.o vchanger-vconf.o vchanger-loghandler.o
vchanger-diskchanger.o vchanger-changerstate.o vchanger-vchanger.o
-L/lib/arm-linux-gnueabihf -luuid -lblkid
/usr/bin/ld: cannot find -luuid
/usr/bin/ld: cannot find -lblkid
collect2: ld returned 1 exit status
make: *** [vchanger] Error 1

I tried manually editing the Makefile in the src directory to comment
out this line:

#vchanger_LDADD = -L/usr/local/lib -luuid -lblkid

I didn't really expect that to work, since manually editing Makefiles
produced by configure seldom does, and it didn't:

vchanger-uuidlookup.o: In function `GetMountpointFromUUID':
/usr/local/src/vchanger/src/uuidlookup.c:236: undefined reference to
`blkid_get_devname'
collect2: ld returned 1 exit status

Is there any hope of getting vchanger to compile? A set of configure
options that will prevent the linker errors, perhaps?

Thanks,
--Greg

P.S. It doesn't appear that -dev packages are available for these
libraries:

r# dpkg -l '*uuid*'
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name   Version  Architecture Description
+++-==---=
ii  libuuid1:armhf 2.20.1-5.3   armhfUniversally Unique ID
library
ii  uuid-runtime   2.20.1-5.3   armhfruntime components for the
Univer
root@bstore:/usr/local/src/vchanger# dkpg -l '*blkid*
 ^C
root@bstore:/usr/local/src/vchanger# ^^'
dkpg -l '*blkid*'
bash: dkpg: command not found
root@bstore:/usr/local/src/vchanger# dpkg -l '*blkid*'
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name   Version  Architecture Description
+++-==---=
ii  libblkid1:armh 2.20.1-5.3   armhfblock device id library



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] vchanger

[Greg Woods]

I'm having some trouble getting vchanger to work, partly because the
documentation that comes with it seems not to be up to date. For
instance, the keywords described in the howto HTML file that comes with
the source are invalid keywords. The example config file has the right
keywords, but I'm having difficulty getting vchanger to work. 

When I get to testing vchanger, the list command works; it shows there
are 34 slots. But label barcodes appears not to work correctly:

*label barcodes
Automatically selected Catalog: GregAndEva
Using Catalog GregAndEva
The defined Storage resources are:
 1: BKUP
 2: ARCH
Select Storage resource (1-2): 2
Enter autochanger drive[0]: 
Connecting to Storage daemon ARCH at bstore.gregandeva.net:9103 ...
3306 Issuing autochanger slots command.
Device ARCH has 34 slots.
Connecting to Storage daemon ARCH at bstore.gregandeva.net:9103 ...
3306 Issuing autochanger list command.
No Volumes found to label, or no barcodes.
*

The inserted magazine disk does automount correctly, but there isn't
anything there except the lost+found directory. Nothing in the
documentation tells me how the index file is created or the
one-file-per-slot files or what name those files should have or if I am
supposed to create them myself or if vchanger is supposed to create them
and if so, how.

Is there any up-to-date documentation that explains how to set up and
configure vchanger? Is there something obvious that I have done wrong? I
can post the relevant parts of the config files if necessary.

Thanks,
--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] USB tape drives?

[Greg Woods]

On Sat, 2013-09-14 at 14:02 -0600, Greg Woods wrote:

 My question is whether there is any such thing as a USB tape drive that
 is known to work with Bacula. 

It's clear from the responses I got that I left out an important detail,
since all the responses were telling me why I should use something other
than a USB tape drive.

This is a cheap home setup. My storage server is a Raspberry Pi. So I do
not have SATA bays,  SCSI interfaces, or eSATA interfaces available. The
only connection for peripherals is USB. My current storage device is a
4TB USB drive, of the green type that shuts itself down automatically
when inactive. But it is still connected, because I want to be able to
fire off an incremental backup for a laptop or desktop whenever I want,
without having to fiddle with hardware connections.

I want a secondary backup device that can be used for migration and/or
copy jobs. For this one, I don't want the media always online, because
of course I want to insulate myself against software going nuts, power
surges, etc. (yes I have a UPS and those are very good but they are not
perfect; ideally, I'd like to copy my full backups to media that I can
store offsite).

At this point, I can see three options: getting a USB tape drive, using
a second hard drive that I manually connect and mount as needed, or
using another machine in the house as a secondary storage server (more
complicated to set up and maintain) so that other hardware options
become available.

--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] USB tape drives?

[Greg Woods]

On Mon, 2013-09-16 at 18:12 +0300, Guy wrote:
 I use SATA drives and pretend they are tape drives by using vchanger.

I will look into doing something like this with one of those USB-to-SATA
drive docks that was recommended. I presume I will be able to find some
way to pretend that the drives are 2TB capacity tapes, so that I can
label the drives and Bacula will keep track of which archives are on
which drive, and tell me which one I need to insert should I ever need
to do a restore from an archive. I figure once every few weeks, I'll
manually run some copy jobs to copy backup jobs to the archive drives.

On the 4TB active drive, I just created via the label command a bunch
of files and made them 50GB capacity, which does cause Bacula to write
50GB to each file, and it will automatically choose the next file to
write to when one is full. I have done some restores (including one that
was actually not just a test) from these backups successfully.

For me, it doesn't matter that USB is slow. I know that of course. I
knew I wasn't going to get great performance from the storage server as
soon as I decided to try using a Raspberry Pi. At that time, I was
looking for a way to avoid having to have yet another always-up server
in the house, and the always-up servers I already have are already
running enough stuff that I didn't want to add to them. So getting a
server for 50 bucks and very low power use seemed like a good thing to
try, and it has worked very well so far. The amount of data I write is
low enough that performance is not a major issue. Slow but reliable is
fine for my purposes.

Thanks to everyone who chimed in on this.

--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] USB tape drives?

[Greg Woods]

Pardon me if this is a dumb or frequently-asked question, but I have not
found a definitive answer searching the list archives, Google, or the
Bacula documentation.

My question is whether there is any such thing as a USB tape drive that
is known to work with Bacula. When reading about tape drives, I see
phrases such as DLT and LTO, but the only USB tape drives I have
been able to find available are DAT 160. I know next to nothing about
tape technology, but I'd like to have a tape device so that I can
migrate my backups from an attached hard drive to tape, so that I have
copies of my backups that are not on a device that is always connected.

Thanks,
--Greg



--
LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint
2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes
Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. 
http://pubads.g.doubleclick.net/gampad/clk?id=64545871iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] backing up NTFS file systems

[Greg Woods]

On Sun, 2013-09-08 at 09:32 -0600, compdoc wrote:


 But since you mention the commands ls -lt and ls -lc, which are not
 windows or dos commands, I'm assuming that your NTFS volumes are attached to
 a nix OS?  That may have more to do with the problem than being NTFS
 volumes.

Well, yes, I did say:


 I expect this isn't an issue with Bacula specifically, but rather an
 artifact of how Linux deals with M$ file systems. 

I am just wondering what exactly the issue is, and whether or not there
is a workaround.

This is a home setup. At first, I thought using something like Bacula
for my house might be overkill, but it turns out that I have three Linux
laptops (work, personal, and wife's) two desktops (mine and wife's) plus
two other desktops that act as servers. One of the servers has a Windows
7 VM that is up all the time. All three laptops also have a dual boot
Windows 7 system, and two of the laptops also have Windows 7 virtual
machines. So that's a total of (I have to count them :-) 7 Linux OS's
and 8 Windows OS's to back up. So I used a Raspberry Pi with a 4TB
green drive (that shuts itself down when idle) as a storage server
(oops, make that 8 Linux OS's to back up).

Except for the one Windows VM that is up all the time (on which I did
install bacula-fd), all the other Windows systems only get used
occasionally, as needed, so I would really rather back them up via the
Linux OS that gets used all the time. So the dual-boot systems have
their file systems mounted under Linux, and I mount the virtual machine
file systems also (one has to use an offset to mount the partitions
within the virtual disk image). Their files are included in the backups
of Linux OS, and can be restored that way.

This works well except for the fact that some of the incrementals turn
into full backups of the NTFS file systems, and I would like to find out
why that happens. I suppose I could work around the problem by
installing bacula-fd on 7 more Windows clients, but that's a lot more
hassle than just mounting the NTFS file systems under Linux and backing
the files up that way. Hopefully this explains what I'm trying to do and
why.

--Greg




--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] backing up NTFS file systems

[Greg Woods]

This mostly seems to work, but with one annoying oddity. When I run an
incremental backup, it seems to want to back up just about the entire
NTFS file system, even though commands like ls -lt and ls -lc don't
show the files or their metadata as having been modified.

I expect this isn't an issue with Bacula specifically, but rather an
artifact of how Linux deals with M$ file systems. Has anyone else seen
this? Have a workaround?

Thanks,
--Greg



--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Post starting bacula-dir does not succeed error on line 29

[Greg Woods]

On Wed, 2013-08-28 at 06:24 -0700, Benny_pixelweb wrote:
 Also when i run a backup 
 
 there are manny errors now.
 
 17: Error: Read error on file 
 /sys/devices/pci:00/:00:1e.0/:01:04.2/resource1. ERR=Input/output 
 error
 28-Aug 15:06 bacula-fd JobId 17: Error: Read error on file 
 /sys/devices/pci:00/:00:1e.0/:01:04.2/resource2. ERR=Input/output 
 error


/sys is not a real file system, it's full of entries into the kernel, so
you don't need or want to back up /sys. It's completely recreated every
time the system reboots. 

You can use df to see where things are mounted. As a general rule, you
don't need/want to back up things that are mounted as tmpfs, or things
that are not mounted at all (that don't show up in the df output).
You only want to back up file systems that are mounted on physical
devices (/dev/something). Whether a given file system really needs to be
backed up depends a lot on what you have stored there; you will have to
decide that for yourself based on your own use of the system. For
instance, I have one system that has a lot (several TB) of videos stored
on it. These take up a huge amount of room and are not that important,
so I do not back them up (I do not include those file systems in my
FileSet). If the drive they are stored on fails, I know I will lose
them, but I accept that. This type of decision, nobody can advise you
on.

--Greg



--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58040911iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Post starting bacula-dir does not succeed error on line 29

[Greg Woods]

On Mon, 2013-08-26 at 13:34 -0700, Benny_pixelweb wrote:
 Hi Ana,
 
 i have set the root / also as you can see in the screenshot now its 2.69Gb 
 backup.
 
 But that could not be all my data anny suggestions?

Each file system has to be declared separately. If all you have in your
FileSet resource Include section is File = /, then you are just going
to get the root partition. You need to also have File declarations for
any other mounted partitions that you want backed up. 

You might notice messages in the console that say things like /home is
a separate file system, will not descend into it or something like
that. In that example, if you wanted /home backed up, you would need
File = /home in your Include section as well.

--Greg



--
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58040911iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] modifying FileSet

[Greg Woods]

Last night I modified the FileSet for my main house server. I got the
Windows version of Bacula running inside the Windows 7 virtual machine,
so that I don't have to copy the entire 12GB virtual disk image on every
incremental backup of the server, plus I get the ability to restore
individual files. So what I changed in the FileSet was only to add an
exclusion for the virtual disk image file.

This resulted in the next incremental backup turning into a Full. Is
that expected behavior? Does any modification of the FileSet invalidate
all the previously-stored catalog entries for that client?

--Greg


--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

On Thu, 2013-08-08 at 11:08 +0100, Martin Simmons wrote:
  On Wed, 07 Aug 2013 19:11:33 -0600, Greg Woods said:

  |13 | anathem   | 2013-08-02 20:37:18 | B| F |  800,022
  | 137,247,853,895 | T |
  |43 | anathem   | 2013-08-07 14:32:19 | B| I |   27,916
  |   2,345,766,355 | T |
  |44 | anathem   | 2013-08-07 14:41:45 | B| I |  773,678
  | 136,134,397,714 | T |
  
  I was under the impression that an incremental would only back up files
  modified since the last backup of any kind, so what made the second
  incremental act like a full?
 
 Look strange to me too -- it would be interesting to see the logs of these
 backups.

Thanks for taking an interest. This has now become a more-or-less fatal
problem for me as all my Incrementals are getting translated into Full.
At least one of my servers has 735GB of stuff (music, videos, photos,
RPM repositories) that takes over 21 hours to do a full backup on, so
it's important to get Incrementals working the way they should (most of
that stuff never changes, so once backed up once, I shouldn't have to
write it again until the next Full).

I have a feeling that this is related to my attempts to modularize the
director configuration, which I have just done in the past couple of
days. I think the time of that change corresponds with when the problem
started, to it's likely related somehow. 

That is, I have a /etc/bacula/clients subdirectory in which I have a
file for each client that defines the File Set and Schedule for each
client (which do vary from client to client of course), and a template
file for the parts of the configuration that are the same for every
client except for the client name. The actual configuration gets put
together with shell scripts run out of bacula-dir.conf, so it looks like
this:

@|sh -c 'for client in  `ls /etc/bacula/clients` ; do sed -e
s/CLIENT-NAME/${client}/ /etc/bacula/bacula-client.conf;
cat /etc/bacula/clients/${client} ; done'

Here's what's in bacula-client.conf:

Job {
  Name = CLIENT-NAME
  JobDefs = DefaultJob
  Type = Backup
  Schedule = CLIENT-NAME
  Client = CLIENT-NAME
  File Set = CLIENT-NAME
}
Job {
  Name = CLIENT-NAME-Restore
  Type = Restore
  Client=CLIENT-NAME
  FileSet=CLIENT-NAME
  Storage = BKUP
  Pool = File
  Messages = Standard
  Where = /tmp/bacula-restores
}
Client {
  Name = CLIENT-NAME
  Address = CLIENT-NAME.gregandeva.net
  Catalog = GregAndEva
  Password = bkup$ucks
  File Retention = 200
  Job Retention = 200
}

For completeness, the DefaultJob is defined as:

JobDefs {
  Name = DefaultJob
  Type = Backup
  Level = Incremental
  FileSet = Full Set
  Storage = BKUP
  Messages = Standard
  Pool = File
  Priority = 10
  Accurate = yes
  Write Bootstrap = /var/spool/bacula/%c.bsr
}

And a sample client file:

FileSet {
  Name = anathem
  Include {
Options {
  signature = MD5
}
   File = /
   File = /xp
   File = /local
   File = /games
   File = /home
  }

#
# If you backup the root directory, the following two excluded
#   files can be useful
#
  Exclude {
File = /var/spool/bacula
File = /tmp
File = /proc
File = /tmp
File = /.journal
File = /.fsck
  }
}


Schedule {
   Name = anathem
}

Maybe another pair of eyes can spot where I messed this up. (If I run
bacula-dir -t -c /etc/bacula/bacula-dir.conf, it completes
successfully. I can't seem to find any way to have it run the
preprocessor and show me what the actual generated bacula-dir.conf
file would look like, although I have manually run the shell command and
the output looks good).

Thanks,
--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

   Password = 

Well, that was stupid of me. I am now going to have to change my
password  in all of the *#()@! bacula-fd.conf files. Boot to the head

:-)

--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

On Thu, 2013-08-08 at 12:12 -0600, Greg Woods wrote:
 On Thu, 2013-08-08 at 11:08 +0100, Martin Simmons wrote:
   On Wed, 07 Aug 2013 19:11:33 -0600, Greg Woods said:
 
   |13 | anathem   | 2013-08-02 20:37:18 | B| F |  800,022
   | 137,247,853,895 | T |
   |43 | anathem   | 2013-08-07 14:32:19 | B| I |   27,916
   |   2,345,766,355 | T |
   |44 | anathem   | 2013-08-07 14:41:45 | B| I |  773,678
   | 136,134,397,714 | T |
   

I left out one important detail. I can see that my Job specifications
are all Full backups by default. The last job in the above list was
invoked from bconsole by typing run Level=Incremental anathem. That
results in a job that lists itself as an Incremental but acts like a
Full. Similarly, my Schedule stanzas look like:

Schedule {
   Name = confusion
   Run = Level=Incremental on 5-31 at 00:05
}

The relevant output from list jobs is:

|27 | confusion | 2013-08-04 06:16:04 | B| F |  726,854
|  27,626,867,742 | T |

|38 | confusion | 2013-08-07 00:05:58 | B| I |  684
|   1,786,383,078 | T |
|47 | confusion | 2013-08-08 00:11:09 | B| I |  726,862
|  28,140,663,999 | T |

Again, after the first incremental, subsequent incrementals list as
incrementals but act like Fulls.

--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

On Thu, 2013-08-08 at 14:39 -0400, John Drescher wrote:
 Make sure the times of the folders (modified and ctime) also any
 attributes are not changing between backups. 

I suppose it's good to check the obvious and the stupid first, but I
don't think that's it. I don't know how to check to see if attributes
are changing, but it seems unlikely that would be happening on all my
desktops and servers. Here's one example:

|27 | confusion | 2013-08-04 06:16:04 | B| F |  726,854
|  27,626,867,742 | T |

|38 | confusion | 2013-08-07 00:05:58 | B| I |  684
|   1,786,383,078 | T |
|47 | confusion | 2013-08-08 00:11:09 | B| I |  726,862
|  28,140,663,999 | T |

FileSet {
  Name = confusion
  Include {
Options {
  signature = MD5
}
   File = /
   File = /old
  }

#
# If you backup the root directory, the following two excluded
#   files can be useful
#
  Exclude {
File = /var/spool/bacula
File = /tmp
File = /proc
File = /tmp
File = /.journal
File = /.fsck
  }
}

[root@confusion /]# find / -xdev -mtime -2 -print | wc -l
409
[root@confusion /]# find /old -xdev -mtime -2 -print | wc -l
0
[root@confusion /]# find /old -xdev -ctime -2 -print | wc -l
0
[root@confusion /]# find / -xdev -ctime -2 -print | wc -l
719

Some of the ctime and mtime changes are likely for the same files, but
in any case, it doesn't look like there should have been more than 1100
files or so on today's incremental, instead there were 726,000. Anything
else I can use besides find to see what ought to be in an
incremental? 

I'm sure it's either something I've done wrong or something I don't
understand about how Bacula is supposed to work.

Just for grins, I tried explicitly entering a Job stanza for the
incremental backups, like this:

Job {
  Name = CLIENT-NAME-Incremental
  JobDefs = DefaultJob
  Type = Backup
  Level = Incremental
  Schedule = CLIENT-NAME
  Client = CLIENT-NAME
  File Set = CLIENT-NAME
}

If after this change, I do estimate and select confusion-Incremental,
it still says 726,000 files will be backed up.

--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

OK, I think I have it figured out, and it's not pretty. I suspect that
most of the backups I have already done are useless and I will have to
repeat all of the Full backups and start all over again. 

The answer came when I tried to do a restore. I thought, OK, if I messed
up the director configuration, why not restore the version of
bacula-dir.conf prior to my most recent set of changes and see what
happens if I use that? The problem was that I was unable to find this
file on the Full backup. It's simply not there. And here's the problem,
in my template client:

Client {
  Name = CLIENT-NAME
  Address = CLIENT-NAME.gregandeva.net
  Catalog = GregAndEva
  Password = bkup$ucks
  File Retention = 200
  Job Retention = 200
}


I thought I was specifying retention times of 200 days, but I'll bet I'm
not, and in fact, my Full backups have been pruned and everything
removed from the Catalog. This is particularly bad because I've already
got my backup storage drive about 25% full, and it's full of stuff that
is useless because without the catalog entries, it can never be
accessed. So that means I'll just have to wipe the backup drive clean,
purge all of the current records in the catalog, and redo all the full
backups (which literally took days to complete). AUUUGH! Lucy pulled
the football away again :-)

So what's the easiest way to clear everything out so I can start over?
Drop the database and reload it from the bacula.sql script?

--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] question about backup levels

[Greg Woods]

On Thu, 2013-08-08 at 17:36 -0400, John Drescher wrote:

 You will have to also clean up your volumes.

I'm planning on deleting all the jobs, then purging all the volumes. If
my understanding of what a purge does is correct, then the data that is
currently written to the volumes (which in my case are just disk files)
will not actually be deleted, but the same space will be reused once all
of the unused volumes have been filled.

--Greg



--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] question about backup levels

[Greg Woods]

I'm a new Bacula user, having just set up a system for backing up the
machines in my house (the storage server is a Raspberry Pi with a 4TB
external disk drive attached to it).

My question concerns backup levels. This afternoon, I think while
learning to use bat, I accidentally scheduled two consecutive
incremental backups on my desktop. The first one ran as expected, but
the second one looks like it basically turned into a full backup, even
though it still appears in the job listing as an incremental:

|13 | anathem   | 2013-08-02 20:37:18 | B| F |  800,022
| 137,247,853,895 | T |
|43 | anathem   | 2013-08-07 14:32:19 | B| I |   27,916
|   2,345,766,355 | T |
|44 | anathem   | 2013-08-07 14:41:45 | B| I |  773,678
| 136,134,397,714 | T |

I was under the impression that an incremental would only back up files
modified since the last backup of any kind, so what made the second
incremental act like a full?

--Greg


--
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with 2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users