[Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you*must* be using a PostgreSQL that was built with the*|--|enable-thread-safety* option, otherwise you will get *data corruption*. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) 2) else since the update, i have a client with a state of dir inserting attribute for 3 hours :( This Client have 1,8 To and 16 Millions of files. An idea if something change since ?? No grant privileges on Tables restoreobject* , like this urls say ( http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg51149.html ) everything works well apart this status DIR inserting attribute for one big client and libpq.a not installed. I'm a little worried. Thank for any information. -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
On 2012-11-29 07:38, Olivier Delestre wrote: Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you MUST be using a PostgreSQL that was built with the --ENABLE-THREAD-SAFETY option, otherwise you will get DATA CORRUPTION. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? Different operating systems install this file in different places. Here is how I found it on a FreeBSD system: $ locate libpq.a /usr/local/lib/libpq.a And running the command: $ nm /usr/local/lib/libpq.a | grep pthread_mutex_lock U pthread_mutex_lock U pthread_mutex_lock i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) Do you have any unanswered questions? 2) else since the update, i have a client with a state of dir inserting attribute for 3 hours :( This Client have 1,8 To and 16 Millions of files. An idea if something change since ?? That's a lot of inserts to do.It can take a while to insert 16 million rows. No grant privileges on Tables restoreobject* , like this urls say ( http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg51149.html ) I'm not sure why you mention this... :) everything works well apart this status DIR inserting attribute for one big client and libpq.a not installed. I'm a little worried. Are you sure that libpq.a is not installed? Please check again... Thank for any information. -- Dan Langille - http://langille.org/ -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
On 2012-11-29 08:34, Olivier Delestre wrote: Le 29/11/2012 13:55, Dan Langille a écrit : On 2012-11-29 07:38, Olivier Delestre wrote: Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you MUST be using a PostgreSQL that was built with the --ENABLE-THREAD-SAFETY option, otherwise you will get DATA CORRUPTION. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? Different operating systems install this file in different places. Here is how I found it on a FreeBSD system: $ locate libpq.a /usr/local/lib/libpq.a And running the command: $ nm /usr/local/lib/libpq.a | grep pthread_mutex_lock U pthread_mutex_lock U pthread_mutex_lock i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) Do you have any unanswered questions? I found /usr/lib/libpq.a if i installed postgresql-server-dev-9.1 on my test machine. what use is it ? I'm sorry, but I don't know for sure. I'm sure it is used by that package. :) 2) else since the update, i have a client with a state of dir inserting attribute for 3 hours :( This Client have 1,8 To and 16 Millions of files. An idea if something change since ?? That's a lot of inserts to do.It can take a while to insert 16 million rows. No grant privileges on Tables restoreobject* , like this urls say ( http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg51149.html ) I'm not sure why you mention this... :) these are things that I could etrangent notes for the update... Not important ? ... I don't know. everything works well apart this status DIR inserting attribute for one big client and libpq.a not installed. I'm a little worried. Are you sure that libpq.a is not installed? Please check again... In prod, i very sure. Thank for any information. I search ... I found a notion of Spool attributes ?? It sounds like your system is already spooling attributes. Perhaps if you pasted your Job and JobDefs (if applicable) for the Job in question. -- Dan Langille - http://langille.org/ -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
On 2012-11-29 09:04, Olivier Delestre wrote:
Le 29/11/2012 14:51, Dan Langille a écrit :
On 2012-11-29 08:34, Olivier Delestre wrote:
Le 29/11/2012 13:55, Dan Langille a écrit :
On 2012-11-29 07:38, Olivier Delestre wrote:
Hi,
I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on
ubuntu
12.04.1 LTS.
I asked about :
1) The documentation of bacula that saying :
If you configure the Batch Insert code in Bacula (attribute
inserts
are
10 times faster), you MUST be using a PostgreSQL that was built
with
the --ENABLE-THREAD-SAFETY option, otherwise you will get
DATA CORRUPTION. Most major Linux distros have thread safety
turned
on, but
it is better to check. One way is to see if the PostgreSQL
library
that
Bacula will be linked against references pthreads. This can be
done
with a command such as:
nm /usr/lib/libpq.a | grep pthread_mutex_lock
But, no such file. What is its usefulness ?
Different operating systems install this file in different places.
Here is
how I found it on a FreeBSD system:
$ locate libpq.a
/usr/local/lib/libpq.a
And running the command:
$ nm /usr/local/lib/libpq.a | grep pthread_mutex_lock
U pthread_mutex_lock
U pthread_mutex_lock
i found it by installing postgresql-server-dev-9.1. Not installed
from
scratch install or upgrade...
I think the package bacula 5.2.5 is compiled with
enable-batch-insert
and postgresql 5.2.5 with enable-thread-safety ( obtain source
package
with apt-get source xxx )
Do you have any unanswered questions?
I found /usr/lib/libpq.a if i installed postgresql-server-dev-9.1
on
my test machine. what use is it ?
I'm sorry, but I don't know for sure. I'm sure it is used by that
package. :)
2) else since the update, i have a client with a state of dir
inserting attribute for 3 hours :(
This Client have 1,8 To and 16 Millions of files. An idea if
something
change since ??
That's a lot of inserts to do.It can take a while to insert 16
million rows.
No grant privileges on Tables restoreobject* , like this urls say
(
http://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg51149.html
)
I'm not sure why you mention this... :)
these are things that I could etrangent notes for the update...
Not important ? ...
I don't know.
everything works well apart this status DIR inserting attribute
for
one big client and libpq.a not installed. I'm a little worried.
Are you sure that libpq.a is not installed? Please check again...
In prod, i very sure.
Thank for any information.
I search ... I found a notion of Spool attributes ??
It sounds like your system is already spooling attributes.
Perhaps if you pasted your Job and JobDefs (if applicable) for the
Job
in question.
../..
JobDefs {
Name = DefaultJob
Type = Backup
Level = Incremental
Schedule = WeeklyCycle
Storage = File
Messages = Standard
#
Reschedule On Error = yes
Reschedule Interval = 1h
Reschedule Times = 2
# Fenetre = Max start delay + Max Run Time
# duree de demarrage max du job de 24h
Max Start Delay = 24h
# temps execution max du job 36 ou 12h
Full Max Run Time = 36h
Differential Max Run Time = 36h
Incremental Max Run Time = 36h
## Incremental Max Run Time = 12h
Priority = 10
}
#
# chargement des clients
#
@|sh -c 'for f in /etc/bacula/Conf.d/Client/*.conf ; do echo @${f} ;
done'
#
../..
Job {
Name = mail
Enabled = yes
Client = mail-fd
JobDefs = DefaultJob
FileSet = fileset_mail
Pool = mail
Storage = mail
Schedule = Weekly-mail
Full Max Run Time = 47h
## debut modif le 20100512
# Incremental Max Run Time = 14h
Incremental Max Run Time = 16h
Reschedule On Error = no
Reschedule Times = 1
## fin modif le 20100512
ClientRunBeforeJob = /etc/bacula/scripts/before.sh
## ClientRunAfterJob = /etc/bacula/scripts/after.sh
RunScript {
RunsWhen = After
RunsOnFailure = yes
RunsOnClient = yes
RunsOnSuccess = yes
Command = /etc/bacula/scripts/after.sh
}
##
Messages = MAIL-mail
Write Bootstrap = /var/lib/bacula/mail.bsr
}
If you see something .
I am out of time just now. Hopefully someone else on the list can take
over.
(remember to do a reply-all)
--
Dan Langille - http://langille.org/
--
Keep yourself connected to Go Parallel:
VERIFY Test and improve your parallel project with help from experts
and peers. http://goparallel.sourceforge.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
On Thu, 29 Nov 2012 08:51:36 -0500, Dan Langille said: On 2012-11-29 08:34, Olivier Delestre wrote: Le 29/11/2012 13:55, Dan Langille a écrit : On 2012-11-29 07:38, Olivier Delestre wrote: Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you MUST be using a PostgreSQL that was built with the --ENABLE-THREAD-SAFETY option, otherwise you will get DATA CORRUPTION. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? Different operating systems install this file in different places. Here is how I found it on a FreeBSD system: $ locate libpq.a /usr/local/lib/libpq.a And running the command: $ nm /usr/local/lib/libpq.a | grep pthread_mutex_lock U pthread_mutex_lock U pthread_mutex_lock i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) Do you have any unanswered questions? I found /usr/lib/libpq.a if i installed postgresql-server-dev-9.1 on my test machine. what use is it ? I'm sorry, but I don't know for sure. I'm sure it is used by that package. :) The Bacula documentation assumes static libraries (.a), but Linux distros typically don't install them by default. In some Linux distros, the static libraries are in the dev(el) package. I suspect Olivier has /usr/lib/libpq.so* instead, so objdump -T /usr/lib/libpq.so* | grep pthread_mutex_lock should be check instead. -- Martin Simmons LispWorks Ltd http://www.lispworks.com/ -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
On 2012-11-29 09:59, Olivier Delestre wrote: and for the very long dir inserting attributes ? an idea ? You are inserting 14 million rows into a database. :) That will take some time. -- Dan Langille - http://langille.org/ -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
Le 29/11/2012 16:04, Dan Langille a écrit : On 2012-11-29 09:59, Olivier Delestre wrote: and for the very long dir inserting attributes ? an idea ? You are inserting 14 million rows into a database. :) That will take some time. xxx-dir Bacula xxx-dir 5.0.3 (04Aug10): 05-Nov-2012 12:51:40 Build OS: x86_64-pc-linux-gnu ubuntu 10.04 JobId: 21582 Job:mail.2012-11-03_20.05.00_20 Backup Level: Full Client: mail-fd 2.0.3 (06Mar07) x86_64-redhat-linux-gnu,redhat,Enterprise release FileSet:fileset_mail 2012-10-06 20:05:00 Pool: mail (From Job resource) Catalog:MyCatalog (From Client resource) Storage:mail (From Job resource) Scheduled time: 03-Nov-2012 20:05:00 Start time: 03-Nov-2012 20:08:04 End time: 05-Nov-2012 12:51:40 Elapsed time: 1 day 16 hours 43 mins 36 secs Priority: 10 FD Files Written: 16,263,062 SD Files Written: 16,263,062 FD Bytes Written: 1,766,373,115,882 (1.766 TB) SD Bytes Written: 1,768,544,870,912 (1.768 TB) Rate: 12047.6 KB/s Software Compression: None VSS:no Encryption: no Accurate: no Volume name(s): mail-2365 Volume Session Id: 500 Volume Session Time:1351086368 Last Volume Bytes: 1,770,395,265,139 (1.770 TB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination:Backup OK -- and with bacula 5.2.5 xxx-dir Bacula xxx-dir 5.2.5 (26Jan12): Build OS: x86_64-pc-linux-gnu ubuntu 12.04 JobId: 22581 Job:mail.2012-11-24_20.05.00_24 Backup Level: Full Client: mail-fd 2.0.3 (06Mar07) x86_64-redhat-linux-gnu,redhat,Enterprise release FileSet:fileset_mail 2012-10-06 20:05:00 Pool: mail (From Job resource) Catalog:MyCatalog (From Client resource) Storage:mail (From Job resource) Scheduled time: 24-Nov-2012 20:05:00 Start time: 24-Nov-2012 20:05:27 End time: 26-Nov-2012 18:51:50 Elapsed time: 1 day 22 hours 46 mins 23 secs Priority: 10 FD Files Written: 16,375,543 SD Files Written: 16,375,543 FD Bytes Written: 1,859,952,150,039 (1.859 TB) SD Bytes Written: 1,862,120,679,979 (1.862 TB) Rate: 11046.0 KB/s Software Compression: None VSS:no Encryption: no Accurate: no Volume name(s): mail-2080 Volume Session Id: 220 Volume Session Time:1353416210 Last Volume Bytes: 1,864,046,644,698 (1.864 TB) Non-fatal FD errors:0 SD Errors: 0 FD termination status: OK SD termination status: OK Termination:Backup OK The Full next is saturday. I'm waiting to see the result. -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 5.2.5 - Postgresql 9.1 - libpq.a - dir inserting attributes ??
Zitat von Olivier Delestre olivier.deles...@univ-rouen.fr: Hi, I upgrade bacula 5.0.3 ( psql 8.4 ) to 5.2.5 ( psql 9.1 ) on ubuntu 12.04.1 LTS. I asked about : 1) The documentation of bacula that saying : If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you*must* be using a PostgreSQL that was built with the*|--|enable-thread-safety* option, otherwise you will get *data corruption*. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as: nm /usr/lib/libpq.a | grep pthread_mutex_lock But, no such file. What is its usefulness ? i found it by installing postgresql-server-dev-9.1. Not installed from scratch install or upgrade... I think the package bacula 5.2.5 is compiled with enable-batch-insert and postgresql 5.2.5 with enable-thread-safety ( obtain source package with apt-get source xxx ) Debian/Ubuntu does a little different because they split Bacula in dynamic loadable parts to use the same main binaries with .so modules for different databases. In /usr/lib/bacula/ you can find the modules used and they include libpthread.so, so you are done. 2) else since the update, i have a client with a state of dir inserting attribute for 3 hours :( This Client have 1,8 To and 16 Millions of files. An idea if something change since ?? If you have done an inplace upgrade from 10.04 carefully check your database schema and index. As far as i remember there was a (Ubuntu) bug regarding Bacula database when doing OS upgrade. Also have a look here: http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server (checkpoint_segments) Regards Andreas -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
