Hi,
Le dimanche 20 décembre 2009 19:15:06, Kern Sibbald a écrit :
> Hello Eric,
>
> I've been a bit busy, but just the same worried about what Bruno is
> describing here, and interestingly enough, on my machine, I am also
> finding that for MySQL batch insert it turned off, yet for PostgreSQL it
> is turned on.
>
> After a bit of experimenting, I think the problem is that there are several
> problems with the ./configure script.
>
> First, for MySQL, the library that is found is
> /usr/lib64/libmysqlclient_r.so, which is OK, but our test for
> pthread_mutex_lock uses nm on the library, which is simply broken. It is
> not an object file, but rather a shared object, so it simply imports other
> shared objects.
>
> The second problem is that for PostgreSQL, it is using /usr/lib/libpq.a,
> which does show pthread_mutex_lock, but in my opinion, we should
> probably /usr/lib/libpq.so, and obviously if we do the test will fail.
Postgres gives an API call to determine if the client is thread-safe or not,
perhaps MySQL do the same (i will check).
> I think there is one or more bugs hidden here ...
I have seen also problem with SQLite3 detection on my ArchLinux.
Bye
> Kern
>
> On Thursday 17 December 2009 08:11:09 Bruno Friedmann wrote:
> > Hello Kern, and thank you for your precious comment
> > about the mysql pthread_mutex_lock availability.
> >
> > I cut all none interesting part. And add what I've found
> > I'm working now on openSUSE 11.2 and I want to extend the bacula.spec of
> > Scott to include the su112 tag and a way to build with and without
> > batch-insert-enabled binary.
> >
> > After several check, none of the rpm build have batch insert inside.
> >
> > So to be sure, I've reget the tar.gz original sources try to reconfigure
> > with simple flags
> >
> > ./configure --enable-smartalloc --with-tcp-wrappers
> > --with-libiconv-prefix --with-libintl-prefix --enable-largefile --with-x
> > --enable-ipv6
> > --enable-acl --enable-xattr --enable-batch-insert --with-python
> > --with-qwt --enable-gnome --enable-bat --with-sqlite3 ( or --with-mysql
> > or
> > --with-postgresql )
> >
> > In every try I only get Enable Batch insert No
> >
> > Configuration on Thu Dec 17 07:59:22 CET 2009:
> >
> > Host: i686-pc-linux-gnu -- suse 11.2
> > Bacula version: Bacula 3.0.2 (18 July 2009)
> > Source code location: .
> > Install binaries: /sbin
> > Install libraries: /usr/lib
> > Install config files: /etc/bacula
> > Scripts directory: /etc/bacula
> > Archive directory: /tmp
> > Working directory: /var/bacula/working
> > PID directory: /var/run
> > Subsys directory: /var/lock/subsys
> > Man directory: ${datarootdir}/man
> > Data directory: /usr/share
> > Plugin directory: /usr/lib
> > C Compiler: gcc Linux)
> > C++ Compiler: /usr/bin/g++ Linux)
> > Compiler flags: -g -O2 -Wall -fno-strict-aliasing
> > -fno-exceptions -fno-rtti Linker flags:
> > Libraries: -lpthread -ldl
> > Statically Linked Tools: no
> > Statically Linked FD: no
> > Statically Linked SD: no
> > Statically Linked DIR: no
> > Statically Linked CONS: no
> > Database type: MySQL
> > Database port:
> > Database lib: -L/usr/lib -lmysqlclient_r -lz
> > Database name: bacula
> > Database user: bacula
> >
> > Job Output Email: r...@localhost
> > Traceback Email: r...@localhost
> > SMTP Host Address: localhost
> >
> > Director Port: 9101
> > File daemon Port: 9102
> > Storage daemon Port: 9103
> >
> > Director User:
> > Director Group:
> > Storage Daemon User:
> > Storage DaemonGroup:
> > File Daemon User:
> > File Daemon Group:
> >
> > SQL binaries Directory /usr/bin
> >
> > Large file support: yes
> > Bacula conio support: yes -lncurses
> > readline support: no
> > TCP Wrappers support: yes -lwrap
> > TLS support: yes
> > Encryption support: yes
> > ZLIB support: yes
> > enable-smartalloc: yes
> > enable-lockmgr: no
> > bat support: yes -L/usr/lib
> > enable-gnome: yes Version 2.x
> > enable-bwx-console: no
> > enable-tray-monitor: no
> > client-only: no
> > build-dird: yes
> > build-stored: yes
> > ACL support: yes
> > XATTR support: yes
> > Python support: yes -lpthread -ldl -lutil -lm -lpython2.6
> > -lutil -lrt Batch insert enabled: no
> >
> >
> > I put here the pg_config and mysql_config results
> > mysql_config
> > Usage: /usr/bin/mysql_config [OPTIONS]
> > Options:
> > --cflags [-I/usr/include/mysql -fomit-frame-pointer
> > -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
> > -fasynchronous-unwind-tables -g -DPIC -fPIC -DUNDEF_HAVE_INITGROUPS
> > -fno-strict-aliasing -DUNIV_LINUX] --include
> > [-I/usr/include/mysql]
> > --libs [-L/usr/lib/mysql -lmysqlclient -lz -lcrypt
> > -lnsl -lm -L/usr/lib -L/usr/lib64 -lssl -lcrypto] --libs_r
> > [-L/usr/lib/mysql -lmysqlclient_r -lz -lpthread -lcrypt -lnsl -lm
> > -lpthread -L/usr/lib -L/usr/lib64 -lssl -lcrypto]
> > --plugindir [/usr/lib/mysql/plugin]
> > --socket [/var/run/mysql/mysql.sock]
> > --port [0]
> > --version [5.1.36]
> > --libmysqld-libs [-L/usr/lib/mysql -lmysqld -ldl -lz -lpthread
> > -lcrypt -lnsl -lm -lpthread -lwrap -lrt -L/usr/lib -L/usr/lib64 -lssl
> > -lcrypto]
> >
> > pg_config
> > BINDIR = /usr/bin
> > DOCDIR = /usr/share/doc/packages/postgresql
> > HTMLDIR = /usr/share/doc/packages/postgresql
> > INCLUDEDIR = /usr/include/pgsql
> > PKGINCLUDEDIR = /usr/include/pgsql
> > INCLUDEDIR-SERVER = /usr/include/pgsql/server
> > LIBDIR = /usr/lib
> > PKGLIBDIR = /usr/lib/postgresql
> > LOCALEDIR = /usr/share/locale
> > MANDIR = /usr/share/man
> > SHAREDIR = /usr/share/postgresql
> > SYSCONFDIR = /usr/etc/postgresql
> > PGXS = /usr/lib/postgresql/pgxs/src/makefiles/pgxs.mk
> > CONFIGURE = '--prefix=/usr' '--libdir=/usr/lib' '--bindir=/usr/bin'
> > '--includedir=/usr/include/pgsql' '--datadir=/usr/share/postgresql'
> > '--mandir=/usr/share/man' '--docdir=/usr/share/doc/packages'
> > '--disable-rpath' '--enable-nls' '--enable-thread-safety'
> > '--enable-integer-datetimes' '--without-python' '--without-perl'
> > '--without-tcl' '--with-openssl' '--with-pam' '--with-krb5'
> > '--with-gssapi' '--with-ldap' '--with-libxml' '--with-libxslt'
> > '--with-system-tzdata=/usr/share/zoneinfo' 'CFLAGS=-fomit-frame-pointer
> > -fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector
> > -funwind-tables -fasynchronous-unwind-tables '
> > CC = gcc
> > CPPFLAGS = -D_GNU_SOURCE -I/usr/include/libxml2
> > CFLAGS = -fomit-frame-pointer -fmessage-length=0 -O2 -Wall
> > -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables
> > -fasynchronous-unwind-tables -Wall -Wmissing-prototypes -Wpointer-arith
> > -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv
> > CFLAGS_SL = -fpic
> > LDFLAGS = -Wl,--as-needed
> > LDFLAGS_SL =
> > LIBS = -lpgport -lxslt -lxml2 -lpam -lssl -lcrypto -lkrb5 -lcom_err
> > -lgssapi_krb5 -lz -lreadline -lcrypt -ldl -lm VERSION = PostgreSQL 8.4.1
> >
> >
> > The two main database are quiet new 5.1 and 8.4.1.
> >
> > I really don't understand why I didn't get batch enable insert to on
> > openSUSE 11.2, and need some advice to find why and/or where to look. So
> > perharps i could complain in bugzilla.novell.com
> >
> >
> > ps : I'm using the default new desktop kernel (mainly preempt+pae)
> > uname -a
> > Linux c-3po 2.6.31.5-0.1-desktop #1 SMP PREEMPT 2009-10-26 15:49:03 +0100
> > i686 i686 i386 GNU/Linux
> >
> >
> >
> > rest of mail is for archive.
> >
> > On 12/16/2009 08:24 AM, Kern Sibbald wrote:
> > > Hello Bruno,
> > >
> > > On Wednesday 16 December 2009 07:18:48 Bruno Friedmann wrote:
> > >> On 12/14/2009 10:00 PM, John Jorgensen wrote:
> > >>>>>>>> "bruno" == Bruno Friedmann <[email protected]> writes:
> > >>>
> > >>> bruno> ok I've try and only get this on the two proposed
> > >>>
> > >>> bruno> strings /usr/sbin/bacula-dir | grep batch
> > >>> bruno> _Z27db_write_batch_file_recordsP3JCR
> > >>> bruno> _Z23db_open_batch_connexionP3JCRP4B_DB
> > >>>
> > >>> I think that the string that Eric suggested searching for might
> > >>> have changed from one release of bacula to another.
> > >>>
> > >>> So in case you're as far behind the times as our site (we're
> > >>> still running Bacula 2.4.3) ...
> > >>>
> > >>> While I'm pretty sure our old server is doing batch inserts,
> > >>> grep'ing for "batch" produces only:
> > >>>
> > >>> 558 backup $strings /usr/local/bacula/sbin/bacula-dir | grep -i
> > >>> batch Cannot select %s in batch mode.
> > >>>
> > >>> Searching for "temporary" hits though:
> > >>>
> > >>> 559 backup $strings /usr/local/bacula/sbin/bacula-dir | grep -i
> > >>> temporary CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER NOT
> > >>> NULL, PurgedFiles SMALLINT, FileSetId INTEGER, JobFiles INTEGER,
> > >>> JobStatus char(1)) CREATE TEMPORARY TABLE temp (JobId INTEGER NOT
> > >>> NULL,JobTDate BIGINT,ClientId INTEGER,Level CHAR,JobFiles
> > >>> INTEGER,JobBytes BIGINT,StartTime TEXT,VolumeName TEXT,StartFile
> > >>> INTEGER,VolSessionId INTEGER,VolSessionTime INTEGER) CREATE TEMPORARY
> > >>> TABLE temp1 (JobId INTEGER NOT NULL,JobTDate BIGINT) 560 backup $
> > >>>
> > >>> As does looking for the message that I see in bconsole while the
> > >>> directory is unbatching the attributes:
> > >>>
> > >>> 560 backup $strings /usr/local/bacula/sbin/bacula-dir | grep -i
> > >>> inserting Inserting first %s res: %s index=%d
> > >>> Inserting %s res: %s index=%d pass=%d
> > >>> Dir inserting Attributes
> > >>
> > >> Thanks John on a 3.0.3 build the temporary search give more
> > >> instructions strings /usr/sbin/bacula-dir | grep -i temporary
> > >> CREATE TEMPORARY TABLE cpy_tmp AS SELECT MIN(JobId) AS JobId FROM Job
> > >> WHERE Type='%c' AND ( PriorJobId IN (%s) OR PriorJobId IN ( SELECT
> > >> PriorJobId FROM Job WHERE JobId IN (%s) AND Type='B' ) ) GROUP BY
> > >> PriorJobId
> > >>
> > >> and in libbacsql.so
> > >>
> > >> strings libbacsql.so.1.0.0 | grep -i temporary
> > >> CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER UNSIGNED NOT NULL,
> > >> PurgedFiles TINYINT, FileSetId INTEGER UNSIGNED, JobFiles INTEGER
> > >> UNSIGNED, JobStatus BINARY(1))
> > >> CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER NOT NULL,
> > >> PurgedFiles SMALLINT, FileSetId INTEGER, JobFiles INTEGER, JobStatus
> > >> char(1)) CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER UNSIGNED
> > >> NOT NULL, PurgedFiles TINYINT, FileSetId INTEGER UNSIGNED, JobFiles
> > >> INTEGER UNSIGNED, JobStatus CHAR)
> > >> CREATE TEMPORARY TABLE temp (JobId INTEGER UNSIGNED NOT NULL,JobTDate
> > >> BIGINT UNSIGNED,ClientId INTEGER UNSIGNED,Level CHAR,JobFiles INTEGER
> > >> UNSIGNED,JobBytes BIGINT UNSIGNED,StartTime TEXT,VolumeName
> > >> TEXT,StartFile INTEGER UNSIGNED,VolSessionId INTEGER
> > >> UNSIGNED,VolSessionTime INTEGER UNSIGNED)
> > >> CREATE TEMPORARY TABLE temp (JobId INTEGER NOT NULL,JobTDate
> > >> BIGINT,ClientId INTEGER,Level CHAR,JobFiles INTEGER,JobBytes
> > >> BIGINT,StartTime TEXT,VolumeName TEXT,StartFile INTEGER,VolSessionId
> > >> INTEGER,VolSessionTime INTEGER) CREATE TEMPORARY TABLE temp1 (JobId
> > >> INTEGER UNSIGNED NOT NULL,JobTDate BIGINT UNSIGNED) CREATE TEMPORARY
> > >> TABLE temp1 (JobId INTEGER NOT NULL,JobTDate BIGINT)
> > >>
> > >> So in this case I can deduct that there's batch-insert enable ..
> > >
> > > None of the above look to me like the Batch Insert code. I am not the
> > > expert, but these are not the SQL statements that Eric said would be
> > > present in Batch Insert.
> > >
> > >> BUT when I check against a package newly build with
> > >> --disable-batch-inssert confirmed by the result of config
> > >>
> > >> Configuration on Wed Dec 16 06:33:02 CET 2009:
> > >>
> > >> Host: i686-pc-linux-gnu -- suse 11.2
> > >> Bacula version: Bacula 3.0.3 (18 October 2009)
> > >> Source code location: .
> > >> Install binaries: /usr/sbin
> > >> Install libraries: /usr/lib
> > >> Install config files: /etc/bacula
> > >> Scripts directory: /usr/lib/bacula
> > >> Archive directory: /tmp
> > >> Working directory: /var/lib/bacula
> > >> PID directory: /var/run
> > >> Subsys directory: /var/lock/subsys
> > >> Man directory: /usr/share/man
> > >> Data directory: /usr/share
> > >> Plugin directory: /usr/lib/bacula
> > >> C Compiler: gcc Linux)
> > >> C++ Compiler: /usr/bin/g++ Linux)
> > >> Compiler flags: -O2 -g -msse -mmmx -march=i586
> > >> -mtune=i686 -funroll-loops -mfpmath=sse -fmessage-length=0
> > >> -fno-strict-aliasing -pipe -fstack-protector -Wall -D_FORTIFY_SOURCE=0
> > >> -fno-strict-aliasing -fno-exceptions -fno-rtti
> > >>
> > >> Linker flags: -I/usr/lib -L/usr/lib/termcap
> > >>
> > >> Libraries: -lpthread -ldl
> > >>
> > >> Statically Linked Tools: no
> > >>
> > >> Statically Linked FD: no
> > >>
> > >> Statically Linked SD: no
> > >>
> > >> Statically Linked DIR: no
> > >>
> > >> Statically Linked CONS: no
> > >>
> > >> Database type: MySQL
> > >>
> > >> Database port:
> > >>
> > >> Database lib: -L/usr/lib -lmysqlclient_r -lz
> > >>
> > >> Database name: bacula
> > >>
> > >> Database user: bacula
> > >>
> > >>
> > >> Job Output Email: r...@localhost
> > >> Traceback Email: r...@localhost
> > >> SMTP Host Address: localhost
> > >>
> > >> Director Port: 9101
> > >> File daemon Port: 9102
> > >> Storage daemon Port: 9103
> > >>
> > >> Director User: bacula
> > >> Director Group: bacula
> > >> Storage Daemon User: bacula
> > >> Storage DaemonGroup: disk
> > >> File Daemon User: root
> > >> File Daemon Group: bacula
> > >>
> > >> SQL binaries Directory /usr/bin
> > >>
> > >> Large file support: yes
> > >> Bacula conio support: yes -ltermcap
> > >> readline support: no
> > >> TCP Wrappers support: yes -lwrap
> > >> TLS support: yes
> > >> Encryption support: yes
> > >> ZLIB support: yes
> > >> enable-smartalloc: yes
> > >> enable-lockmgr: no
> > >> bat support: no
> > >> enable-gnome: no
> > >> enable-bwx-console: no
> > >> enable-tray-monitor: no
> > >> client-only: no
> > >> build-dird: yes
> > >> build-stored: yes
> > >> ACL support: yes
> > >> XATTR support: yes
> > >> Python support: yes -lpthread -ldl -lutil -lm -lpython2.6
> > >> -lutil -lrt Batch insert enabled: no
> > >>
> > >>
> > >> I'm getting the same string inside bacula-dir and libbacsql.so ?
> > >>
> > >> CREATE TEMPORARY TABLE cpy_tmp AS SELECT MIN(JobId) AS JobId FROM Job
> > >> WHERE Type='%c' AND ( PriorJobId IN (%s) OR PriorJobId IN ( SELECT
> > >> PriorJobId FROM Job WHERE JobId IN (%s) AND Type='B' ) ) GROUP BY
> > >> PriorJobId
> > >>
> > >> strings libbacsql.so.1.0.0 | grep -i temporary
> > >> CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER UNSIGNED NOT NULL,
> > >> PurgedFiles TINYINT, FileSetId INTEGER UNSIGNED, JobFiles INTEGER
> > >> UNSIGNED, JobStatus BINARY(1))
> > >> CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER NOT NULL,
> > >> PurgedFiles SMALLINT, FileSetId INTEGER, JobFiles INTEGER, JobStatus
> > >> char(1)) CREATE TEMPORARY TABLE DelCandidates (JobId INTEGER UNSIGNED
> > >> NOT NULL, PurgedFiles TINYINT, FileSetId INTEGER UNSIGNED, JobFiles
> > >> INTEGER UNSIGNED, JobStatus CHAR)
> > >> CREATE TEMPORARY TABLE temp (JobId INTEGER UNSIGNED NOT NULL,JobTDate
> > >> BIGINT UNSIGNED,ClientId INTEGER UNSIGNED,Level CHAR,JobFiles INTEGER
> > >> UNSIGNED,JobBytes BIGINT UNSIGNED,StartTime TEXT,VolumeName
> > >> TEXT,StartFile INTEGER UNSIGNED,VolSessionId INTEGER
> > >> UNSIGNED,VolSessionTime INTEGER UNSIGNED)
> > >> CREATE TEMPORARY TABLE temp (JobId INTEGER NOT NULL,JobTDate
> > >> BIGINT,ClientId INTEGER,Level CHAR,JobFiles INTEGER,JobBytes
> > >> BIGINT,StartTime TEXT,VolumeName TEXT,StartFile INTEGER,VolSessionId
> > >> INTEGER,VolSessionTime INTEGER) CREATE TEMPORARY TABLE temp1 (JobId
> > >> INTEGER UNSIGNED NOT NULL,JobTDate BIGINT UNSIGNED) CREATE TEMPORARY
> > >> TABLE temp1 (JobId INTEGER NOT NULL,JobTDate BIGINT)
> > >>
> > >>
> > >> So there's two solutions, the strings are always present, or there's
> > >> somewhere a default in sources or build ....
> > >>
> > >> Eric can you recheck about the string on a 3.0.3 pure git trunk ?
> > >
> > > It is pretty clear from the code that there will be differences in the
> > > SQL statements that are in the binary files when Batch Insert is on and
> > > when it is off. If you are getting the same code with Batch Insert on
> > > and off, then the problem lies in MySQL not having the multithread
> > > option turned on when it was built (i.e. it has no references to
> > > pthread_mutex_lock in the MySQL client library that Bacula links in).
> > > If that is the case, batch insert will be flagged as "no" in the
> > > output of ./configure.
> > >
> > > If you are getting the same strings in both bacula-dir and
> > > libbacsql.so, then you are mixing code. That is you have built
> > > bacula-dir statically and dynamically. You probably need to do "make
> > > uninstall" between make installs, but make uninstall is so seldom used,
> > > I am not 100% sure it handles uninstalling the shared objects.
> > >
> > > The CREATE TEMPORARY TABLE cpy_tmp should appear only in bacula-dir and
> > > not in libbacsq.so
> > >
> > >> What make me in doubt is that bacula-dir/libbacsql with ou without
> > >> batch enabled have the same md5sum ? b9156e6cb7902024c790add0291216bc
> > >> bacula-dir (without bi)
> > >> b9156e6cb7902024c790add0291216bc /usr/sbin/bacula-dir
> > >> f01fe8fb42648dae586762b44987bfe1 libbacsql.so.1.0.0 (without bi)
> > >> f01fe8fb42648dae586762b44987bfe1 /usr/lib/libbacsql.so.1.0.0
> > >>
> > >> What are the result for you ?
> > >
> > > Given what I wrote above, you are clearly not comparing Bacula with and
> > > without Batch Insert.
> > >
> > > Regards,
> > >
> > > Kern
>
------------------------------------------------------------------------------
This SF.Net email is sponsored by the Verizon Developer Community
Take advantage of Verizon's best-in-class app development support
A streamlined, 14 day to market process makes app distribution fast and easy
Join now and get one step closer to millions of Verizon customers
http://p.sf.net/sfu/verizon-dev2dev
_______________________________________________
Bacula-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-devel
