Re: [Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
Ethan Funk et...@redmountainradio.com was heard to say: Even with libdbi linked against the libmysqlclient_r library, the API is not thread safe with out the mysql_thread_start() and mysql_thread_end() calls on a **shared** connection, so including my change would not **hurt**. Linking agains the non _r library and you can't even run different connection on different threads. Would it be feasible to tell the libdbi driver how your app uses threads by means of a driver option? Something like mysql_one_conn_per_thread? This way, the driver could use some optimizations like yours which apply only under these particular conditions. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.net Dev2Dev email is sponsored by: Show off your parallel programming skills. Enter the Intel(R) Threading Challenge 2010. http://p.sf.net/sfu/intel-thread-sfd ___ Libdbi-drivers-devel mailing list Libdbi-drivers-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
Re: [Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
Ethan Funk writes:
2. Modify the dbd_mysql.c source file, changing dbd_disconnect as follows:
int dbd_disconnect(dbi_conn_t *conn) {
if (conn-connection) mysql_close((MYSQL *)conn-connection);
// added the next three lines to resolve memory leak in threadsafe
mysqlclient library: assumes each thread has it's own connection
if(mysql_thread_safe())
mysql_thread_end();
return 0;
}
Is this generally applicable, or would that hurt if several threads
share a connection? I wonder if this should be applied to the driver
sources.
3. build and link the dbd_mysql driver. It MUST be linked agains the _r
version of the mysqlclientlib. This was really tricky on OS X.
I've added an --enable-mysql-threadsafe option to ./configure which
allows to optionally link against libmysqlclient_r instead of
libmysqlclient.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
This SF.net Dev2Dev email is sponsored by:
Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
___
Libdbi-drivers-devel mailing list
Libdbi-drivers-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
Re: [Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
My understanding of what's going on under the hood of the mysqlclient_r library
is fuzzy a best. I based my code changes on some very poor documentation
regarding threading from the MySQL folks. The indication was that you call
mysql_thread_start() when you begin using the library in a thread and then
mysql_thread_end() when you are finished. However, I'm told
mysql_thread_start() is called for you when you open a connection on the _r
version of the library so I figured calling mysql_thread_end() on connection
close with in libdbi would balance that nicely if there are no other
mysql_thread_start() calls.
The problem is, if you use the same connection on more than one thread, you
need to call mysql_thread_start() at the start and mysql_thread_end() at the
end of each threads usage, and there is no way for this to be done with in the
lidbdi API. There is no way to tell libdbi about the new thread usage such
that it can pass that down to libmysqlclient, so I just deal with it at the
connection level.
Even with libdbi linked against the libmysqlclient_r library, the API is not
thread safe with out the mysql_thread_start() and mysql_thread_end() calls on a
**shared** connection, so including my change would not **hurt**. Linking
agains the non _r library and you can't even run different connection on
different threads.
Ethan...
On Sep 1, 2010, at 3:20 PM, Markus Hoenicka wrote:
Ethan Funk writes:
2. Modify the dbd_mysql.c source file, changing dbd_disconnect as follows:
int dbd_disconnect(dbi_conn_t *conn) {
if (conn-connection) mysql_close((MYSQL *)conn-connection);
// added the next three lines to resolve memory leak in threadsafe
mysqlclient library: assumes each thread has it's own connection
if(mysql_thread_safe())
mysql_thread_end();
return 0;
}
Is this generally applicable, or would that hurt if several threads
share a connection? I wonder if this should be applied to the driver
sources.
3. build and link the dbd_mysql driver. It MUST be linked agains the _r
version of the mysqlclientlib. This was really tricky on OS X.
I've added an --enable-mysql-threadsafe option to ./configure which
allows to optionally link against libmysqlclient_r instead of
libmysqlclient.
regards,
Markus
--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38
--
This SF.net Dev2Dev email is sponsored by:
Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
___
Libdbi-drivers-devel mailing list
Libdbi-drivers-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
Re: [Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
I successfully use libdbi in production with MySQL and Postgres servers in multithreading, but by multithreading I mean that I am using one connection per thread, and a variable pool of threads, with connections to MySQL and Postgres on the same libdbi instance. A server going down does not cause any issue. However, you might be encountering an issue with the dbi_conn_error, which last time I checked was not thread safe, because it is sharing a static char * between all connections! I guess I should have filed a bug. Here's how I get around it: // dbi_conn_error is not thread safe, it uses a static char * // internally which could be freed twice if the function is not locked boost::mutex::scoped_lock lock(sErrorMutex); const char *dbError = NULL; int errNum = dbi_conn_error(mDbConn, dbError); Logger::logX( LOG_WARNING, 1504 - Failed to connect to DB with connection string \%s\, reason: %d - %s, dbAccess-getConnection().c_str(), errNum, dbError ); Anyway, why don't you gdb your crash, it will give you a better hint of where the problem actually is. Regards, Emmanuel. On Thu, Aug 19, 2010 at 10:24 AM, Markus Hoenicka markus.hoeni...@mhoenicka.de wrote: Shaofeng Yang breath...@gmail.com was heard to say: Greeting, I am working on a thread pool with libdbi and libdbi-driver-mysql. i saw a problem that program crashes if mysql server goes away for one thread. i am using the thread safe version of libdbi and the design of my thread pool doesn't have problems. The problem comes out from libdbi mysql driver, which is compiled against libmysqlclient not libmysqlclient_r by default. I am just wondering what i should do to make libdbi mysql driver thread-safe. Can I just change the makefile.am to use -lmysqlclient_r instead of -lmysqlclient? Is dbd_mysql.c thread-safe? Hi, I assume you connect to the server before starting your threads. Would it be possible to use the new interface (available in CVS and in the eventually to be released libdbi version 0.9) which allows to manage independent instances of libdbi in a single process? You could use that to connect to the server in each thread separately, so if one of them goes away it should not affect the other instances. I'm not familiar with thread-safe programming, so I have no idea if libdbi is currently thread-safe. Someone more familiar with this issue should speak up. As for libmysqlclient and libmysqlclient_r: if these libraries use the same API, I guess you can simply change Makefile.am to use the latter. If that works, we should probably make this a configurable build option. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev ___ Libdbi-drivers-devel mailing list Libdbi-drivers-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel -- This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev ___ Libdbi-drivers-devel mailing list Libdbi-drivers-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
Re: [Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
On 08/19/2010 10:24 AM, Markus Hoenicka wrote: I'm not familiar with thread-safe programming, so I have no idea if libdbi is currently thread-safe. Someone more familiar with this issue should speak up. I don't know if it is thread safe, I haven't really looked into that really. I just assumed that it was not, and lock a mutex around all dbi_conn_query() and dbi_comm_ping() calls. This has worked perfectly in operational systems for years, with extensive use of threads (postgresql only). -- --Kjell -- This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev ___ Libdbi-drivers-devel mailing list Libdbi-drivers-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
[Libdbi-drivers-devel] mysql libdbi driver compiled against libmysqlclient not libmysqlclient_r
Greeting, I am working on a thread pool with libdbi and libdbi-driver-mysql. i saw a problem that program crashes if mysql server goes away for one thread. i am using the thread safe version of libdbi and the design of my thread pool doesn't have problems. The problem comes out from libdbi mysql driver, which is compiled against libmysqlclient not libmysqlclient_r by default. I am just wondering what i should do to make libdbi mysql driver thread-safe. Can I just change the makefile.am to use -lmysqlclient_r instead of -lmysqlclient? Is dbd_mysql.c thread-safe? Thanks, James -- This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev ___ Libdbi-drivers-devel mailing list Libdbi-drivers-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-drivers-devel
