Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Kern Sibbald
Hello,

Thanks for informing us.

There was a problem with Ubuntu 16.04 where they used a linking option 
(something like -Bsymbolic...) that caused all Bacula default values to 
be 0.  This showed up rather dramatically because the default number of 
consoles was then set to zero, and bconsole (as well as other consoles 
such as bat) did not work.  If you did not get a new corrected version, 
but instead applied a workaround to explicitly set the number of console 
connections, you might be in for more surprises where default values are 
no longer what you expect.

I recommend trying to get a corrected version.  It is in one of their 
PPAs (I think that is what they call them). They also ran into another 
quite serious bug causing the performance of MySQL to be something like 
10 times slower than normal. This was due to changes made by the MySQL 
project.  The MySQL problem does not apply to you, but for those who may 
be using MySQL, please get the corrected version of Bacula.

Best regards,
Kern

On 10/08/2016 12:03 AM, Jeremy Parrish wrote:
> I have found the solution to my problem, so I am posting in case it
> helps someone else out in the future.
>
> The problem was indeed related to the lack of defaults in the Ubuntu
> 16.04 package. The solution was to make sure "Catalog Files = yes" was
> set in the Pool resource.
>
> After adding that to the director's config, I ran a new job, and I was
> indeed able to select files to restore afterward.
>
> -Jeremy
>
> On 10/7/16 3:26 PM, Jeremy Parrish wrote:
>> Thanks for the info. Unfortunately, this is not the problem in my case.
>> I upgraded to the Postgres 9.5 db and dumped the old one right away. It
>> appears that no new records are getting inserted into the File table.
>> The latest JobId in that table is very old (I have not verified, but I
>> assume this would be the last job that ran before the upgrade). I wonder
>> what could be causing the new File records to not get inserted properly...
>>
>> -Jeremy
>>
>> On 10/7/16 2:21 PM, Marcio Vogel Merlone dos Santos wrote:
>>> Hi,
>>>
>>> Yes, I found the culprit. After the upgrade process I ended up with
>>> *two* PostgreSQL server instances (trying to) running over the same
>>> dataset: the old and the upgraded one. Removed the old and got it
>>> working fine.
>>>
>>> Hope this helps.
>>>
>>> Regards.
>>>
>>>
>>> Em 07/10/2016 12:50, Jeremy Parrish escreveu:
 I am in the same boat... apparently never tried doing a restore since
 upgrading to Ubuntu 16.04. I am guessing there is another config option 
 that
 is not getting its default value set properly. Did you ever find a solution
 for this?

 -Jeremy



 --
 View this message in context: 
 http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84656.html
 Sent from the Bacula - Users mailing list archive at Nabble.com.

 --
 Check out the vibrant tech community on one of the world's most
 engaging tech sites, SlashDot.org! http://sdm.link/slashdot
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users
>>> --
>>> *Marcio Merlone*
>>> TI - Administrador de redes
>>>
>>> *A1 Engenharia - Unidade Corporativa*
>>> Fone:   +55 41 3616-3797
>>> Cel:+55 41 9689-0036
>>>
>>> http://www.a1.ind.br/ 
>>>
>>>
>>> --
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>>>
>>>
>>>
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>


--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! 

Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Jeremy Parrish
I have found the solution to my problem, so I am posting in case it 
helps someone else out in the future.

The problem was indeed related to the lack of defaults in the Ubuntu 
16.04 package. The solution was to make sure "Catalog Files = yes" was 
set in the Pool resource.

After adding that to the director's config, I ran a new job, and I was 
indeed able to select files to restore afterward.

-Jeremy

On 10/7/16 3:26 PM, Jeremy Parrish wrote:
> Thanks for the info. Unfortunately, this is not the problem in my case.
> I upgraded to the Postgres 9.5 db and dumped the old one right away. It
> appears that no new records are getting inserted into the File table.
> The latest JobId in that table is very old (I have not verified, but I
> assume this would be the last job that ran before the upgrade). I wonder
> what could be causing the new File records to not get inserted properly...
>
> -Jeremy
>
> On 10/7/16 2:21 PM, Marcio Vogel Merlone dos Santos wrote:
>> Hi,
>>
>> Yes, I found the culprit. After the upgrade process I ended up with
>> *two* PostgreSQL server instances (trying to) running over the same
>> dataset: the old and the upgraded one. Removed the old and got it
>> working fine.
>>
>> Hope this helps.
>>
>> Regards.
>>
>>
>> Em 07/10/2016 12:50, Jeremy Parrish escreveu:
>>> I am in the same boat... apparently never tried doing a restore since
>>> upgrading to Ubuntu 16.04. I am guessing there is another config option that
>>> is not getting its default value set properly. Did you ever find a solution
>>> for this?
>>>
>>> -Jeremy
>>>
>>>
>>>
>>> --
>>> View this message in context: 
>>> http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84656.html
>>> Sent from the Bacula - Users mailing list archive at Nabble.com.
>>>
>>> --
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>> --
>> *Marcio Merlone*
>> TI - Administrador de redes
>>
>> *A1 Engenharia - Unidade Corporativa*
>> Fone:+55 41 3616-3797
>> Cel: +55 41 9689-0036
>>
>> http://www.a1.ind.br/ 
>>
>>
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>>
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Jeremy Parrish
Thanks for the info. Unfortunately, this is not the problem in my case. 
I upgraded to the Postgres 9.5 db and dumped the old one right away. It 
appears that no new records are getting inserted into the File table. 
The latest JobId in that table is very old (I have not verified, but I 
assume this would be the last job that ran before the upgrade). I wonder 
what could be causing the new File records to not get inserted properly...

-Jeremy

On 10/7/16 2:21 PM, Marcio Vogel Merlone dos Santos wrote:
> Hi,
>
> Yes, I found the culprit. After the upgrade process I ended up with
> *two* PostgreSQL server instances (trying to) running over the same
> dataset: the old and the upgraded one. Removed the old and got it
> working fine.
>
> Hope this helps.
>
> Regards.
>
>
> Em 07/10/2016 12:50, Jeremy Parrish escreveu:
>> I am in the same boat... apparently never tried doing a restore since
>> upgrading to Ubuntu 16.04. I am guessing there is another config option that
>> is not getting its default value set properly. Did you ever find a solution
>> for this?
>>
>> -Jeremy
>>
>>
>>
>> --
>> View this message in context: 
>> http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84656.html
>> Sent from the Bacula - Users mailing list archive at Nabble.com.
>>
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
> --
> *Marcio Merlone*
> TI - Administrador de redes
>
> *A1 Engenharia - Unidade Corporativa*
> Fone: +55 41 3616-3797
> Cel:  +55 41 9689-0036
>
> http://www.a1.ind.br/ 
>
>
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Marcio Vogel Merlone dos Santos

Hi,

Yes, I found the culprit. After the upgrade process I ended up with 
*two* PostgreSQL server instances (trying to) running over the same 
dataset: the old and the upgraded one. Removed the old and got it 
working fine.


Hope this helps.

Regards.


Em 07/10/2016 12:50, Jeremy Parrish escreveu:

I am in the same boat... apparently never tried doing a restore since
upgrading to Ubuntu 16.04. I am guessing there is another config option that
is not getting its default value set properly. Did you ever find a solution
for this?

-Jeremy



--
View this message in context: 
http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84656.html
Sent from the Bacula - Users mailing list archive at Nabble.com.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


--
*Marcio Merlone*
TI - Administrador de redes

*A1 Engenharia - Unidade Corporativa*
Fone:   +55 41 3616-3797
Cel:+55 41 9689-0036

http://www.a1.ind.br/ 
--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Jeremy Parrish
I am in the same boat... apparently never tried doing a restore since
upgrading to Ubuntu 16.04. I am guessing there is another config option that
is not getting its default value set properly. Did you ever find a solution
for this? 

-Jeremy



--
View this message in context: 
http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84656.html
Sent from the Bacula - Users mailing list archive at Nabble.com.

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-10-07 Thread Jeremy Parrish
I am in the same boat... apparently never tried doing a restore since
upgrading to Ubuntu 16.04. I am guessing there is another config option that
is not getting its default value set properly. Did you ever find a solution
for this?

-Jeremy



--
View this message in context: 
http://bacula.10910.n7.nabble.com/Files-pruned-from-catalog-or-never-inserted-For-one-or-more-of-the-JobIds-selected-no-files-were-fou-tp84088p84655.html
Sent from the Bacula - Users mailing list archive at Nabble.com.

--
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-06-27 Thread Ana Emília M . Arruda
Hello Marcio,

Which was the version immediately previous to the upgrade to 7.05? In some
cases, there is a need of database upgrade.

Fortunately, there are ways to restore individual files by the use of bls
and bextract utilities.

With bls you can have a list of all the files backed up in one volume, the
you can use bextract with "-i" option and inform here a list of files to be
restored.

Also, you can use bscan to recover the jobs and media information from your
volumes into your catalog again.

Best regards,
Ana




On Thu, Jun 16, 2016 at 8:00 PM, Marcio Vogel Merlone dos Santos <
marcio.merl...@a1.ind.br> wrote:

> Hi,
>
> I am running a bacula server for years now, it has been upgraded since its
> 2.x version and now I got it to 7.0.5+dfsg-4build1 on a Ubuntu server 16.04
> with a postgresql database.
>
> Since its upgrade to 7.x I could not yet test a restore (shame on me, I
> know), and now a user asked for some yesterday's files. Guess what: "For
> one or more of the JobIds selected, no files were found, so file selection
> is not possible.". That happens to all clients and filesets.
>
> I searched Google and found some old and recent messages regarding this
> situation and none seems to apply to this case. Some info:
>
> Client {
> Name= orion-fd
> Address = 10.0.0.1
> FDPort  = 9102
> Catalog = CatalogoA1Sede
> Password= "aaa"
> *File Retention  = 6 months*
> Job Retention   = 1 year
> AutoPrune   = yes
> Maximum Concurrent Jobs = 1
> }
>
> Last full job for this client was on 2016-06-01, so much less than 6
> months. Yesterday's INC was like this:
>
> 15-Jun 22:17 phobos-dir JobId 39191: Start Backup JobId 39191, 
> Job=JobOrionFab.2016-06-15_22.00.01_35
> 15-Jun 22:17 phobos-dir JobId 39191: Using Device "LTO" to write.
> 15-Jun 22:22 phobos-sd JobId 39191: Elapsed time=00:04:50, Transfer 
> rate=52.40 M Bytes/second
> 15-Jun 22:22 phobos-dir JobId 39191: Bacula phobos-dir 7.0.5 (28Jul14):
>   Build OS:   x86_64-pc-linux-gnu ubuntu 16.04
>   JobId:  39191
>   Job:JobOrionFab.2016-06-15_22.00.01_35
>   Backup Level:   Incremental, since=2016-06-14 22:18:51
>   Client: "orion-fd" 5.2.5 (26Jan12) 
> x86_64-pc-linux-gnu,ubuntu,12.04
>   FileSet:"FAB" 2016-01-04 15:27:59
>   Pool:   "PoolDiarioSemanal" (From Run Pool override)
>   Catalog:"CatalogoA1Sede" (From Client resource)
>   Storage:"LTO" (From Pool resource)
>   Scheduled time: 15-Jun-2016 22:00:01
>   Start time: 15-Jun-2016 22:17:30
>   End time:   15-Jun-2016 22:22:21
>   Elapsed time:   4 mins 51 secs
>   Priority:   10  FD Files Written:   1,314
>   SD Files Written:   1,314
>   FD Bytes Written:   15,197,809,897 (15.19 GB)
>   SD Bytes Written:   15,198,047,213 (15.19 GB)
>   Rate:   52226.2 KB/s
>   Software Compression:   None
>   VSS:no
>   Encryption: no
>   Accurate:   no
>   Volume name(s): PDS-0002
>   Volume Session Id:  90
>   Volume Session Time:1465823891
>   Last Volume Bytes:  1,086,222,670,848 (1.086 TB)
>   Non-fatal FD errors:0
>   SD Errors:  0
>   FD termination status:  OK
>   SD termination status:  OK
>   Termination:Backup OK
>
> 15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Jobs older than 1 year .
> 15-Jun 22:22 phobos-dir JobId 39191: No Jobs found to prune.
> 15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Files.
> 15-Jun 22:22 phobos-dir JobId 39191: No Files found to prune.
> 15-Jun 22:22 phobos-dir JobId 39191: End auto prune.
>
> No files or jobs were pruned, looks good.
>
> As per this thread
> http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg41594.html
> I run those queries on db:
>
> select jobfiles from job where jobid=39191;
> Result: 1314, which matches "SD Files Written" above. Alas:
>
> select count(*) from file where jobid=39191;
> Result: 0
>
> When upgraded from 5.x to 7.x I had to add the "Maximum Concurrent Jobs"
> directive to all client and job defs otherwise it would wait for max
> something. Perhaps I missed something else? Has the file records been
> pruned from db or never been there?
>
> While dbcheck runs, can anybody kindly help me find what's wrong or what
> happened?
>
> Dir conf:
> #
> Director {
> Name   = phobos-dir
> DIRport= 9101# where we listen for UA
> connections
> QueryFile  = "/etc/bacula/scripts/query.sql"
> WorkingDirectory   = "/var/lib/bacula"
> PidDirectory   = "/var/run/bacula"
> Maximum Concurrent Jobs = 2
> Password   = "aaa" # Console password
> Messages   = Daemon
> # DirAddress   = 10.0.0.2
>   

[Bacula-users] Files pruned from catalog or never inserted - For one or more of the JobIds selected, no files were found, so file selection is not possible.

2016-06-16 Thread Marcio Vogel Merlone dos Santos

Hi,

I am running a bacula server for years now, it has been upgraded since 
its 2.x version and now I got it to 7.0.5+dfsg-4build1 on a Ubuntu 
server 16.04 with a postgresql database.


Since its upgrade to 7.x I could not yet test a restore (shame on me, I 
know), and now a user asked for some yesterday's files. Guess what: "For 
one or more of the JobIds selected, no files were found, so file 
selection is not possible.". That happens to all clients and filesets.


I searched Google and found some old and recent messages regarding this 
situation and none seems to apply to this case. Some info:


Client {
Name= orion-fd
Address = 10.0.0.1
FDPort  = 9102
Catalog = CatalogoA1Sede
Password= "aaa"
*File Retention  = 6 months*
Job Retention   = 1 year
AutoPrune   = yes
Maximum Concurrent Jobs = 1
}

Last full job for this client was on 2016-06-01, so much less than 6 
months. Yesterday's INC was like this:


15-Jun 22:17 phobos-dir JobId 39191: Start Backup JobId 39191, 
Job=JobOrionFab.2016-06-15_22.00.01_35
15-Jun 22:17 phobos-dir JobId 39191: Using Device "LTO" to write.
15-Jun 22:22 phobos-sd JobId 39191: Elapsed time=00:04:50, Transfer rate=52.40 
M Bytes/second
15-Jun 22:22 phobos-dir JobId 39191: Bacula phobos-dir 7.0.5 (28Jul14):
  Build OS:   x86_64-pc-linux-gnu ubuntu 16.04
  JobId:  39191
  Job:JobOrionFab.2016-06-15_22.00.01_35
  Backup Level:   Incremental, since=2016-06-14 22:18:51
  Client: "orion-fd" 5.2.5 (26Jan12) 
x86_64-pc-linux-gnu,ubuntu,12.04
  FileSet:"FAB" 2016-01-04 15:27:59
  Pool:   "PoolDiarioSemanal" (From Run Pool override)
  Catalog:"CatalogoA1Sede" (From Client resource)
  Storage:"LTO" (From Pool resource)
  Scheduled time: 15-Jun-2016 22:00:01
  Start time: 15-Jun-2016 22:17:30
  End time:   15-Jun-2016 22:22:21
  Elapsed time:   4 mins 51 secs
  Priority:   10  FD Files Written:   1,314
  SD Files Written:   1,314
  FD Bytes Written:   15,197,809,897 (15.19 GB)
  SD Bytes Written:   15,198,047,213 (15.19 GB)
  Rate:   52226.2 KB/s
  Software Compression:   None
  VSS:no
  Encryption: no
  Accurate:   no
  Volume name(s): PDS-0002
  Volume Session Id:  90
  Volume Session Time:1465823891
  Last Volume Bytes:  1,086,222,670,848 (1.086 TB)
  Non-fatal FD errors:0
  SD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:Backup OK

15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Jobs older than 1 year .
15-Jun 22:22 phobos-dir JobId 39191: No Jobs found to prune.
15-Jun 22:22 phobos-dir JobId 39191: Begin pruning Files.
15-Jun 22:22 phobos-dir JobId 39191: No Files found to prune.
15-Jun 22:22 phobos-dir JobId 39191: End auto prune.

No files or jobs were pruned, looks good.

As per this thread 
http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg41594.html 
I run those queries on db:


select jobfiles from job where jobid=39191;
Result: 1314, which matches "SD Files Written" above. Alas:

select count(*) from file where jobid=39191;
Result: 0

When upgraded from 5.x to 7.x I had to add the "Maximum Concurrent Jobs" 
directive to all client and job defs otherwise it would wait for max 
something. Perhaps I missed something else? Has the file records been 
pruned from db or never been there?


While dbcheck runs, can anybody kindly help me find what's wrong or what 
happened?


Dir conf:
#
Director {
Name   = phobos-dir
DIRport= 9101# where we listen for UA 
connections

QueryFile  = "/etc/bacula/scripts/query.sql"
WorkingDirectory   = "/var/lib/bacula"
PidDirectory   = "/var/run/bacula"
Maximum Concurrent Jobs = 2
Password   = "aaa" # Console password
Messages   = Daemon
# DirAddress   = 10.0.0.2
MaximumConsoleConnections = 20
}

JobDefs {
Name = "jobDiarioSemanal"
Type = Backup
Level= Incremental
Client   = phobos-fd
FileSet  = Config
Schedule = SchDefault
Storage  = LTO
Pool = PoolDiarioSemanal
Messages = Standard
Priority = 10
Write Bootstrap  = "/var/lib/bacula/%c.bsr"
Allow Duplicate Jobs = no
Cancel Lower Level Duplicates = yes
Enabled  = yes
Maximum Concurrent Jobs= 1
}

JobDefs {
Name = "jobMensal"
Type = Backup
Level  = Full
Client = phobos-fd
FileSet= Config
Schedule = SchMensal
Storage = LTO
Pool = PoolDiarioSemanal