Re: [libdbi-users] Found and fixed: dbd_mysql thread unsafe problem
Markus, Unfortunately, I have no experience maintaining code for packages. The best I can do is a few lines in a Makefile, and I don't understand automake scripts at all. I can fix all sorts of things in c source code, but I would likely break the test and build process. So I wouldn't trust me as a maintainer. Thanks for your work on libdbi over the years. I have been using it for well over a decade. Ethan... On Tue, 2019-11-19 at 00:20 +0100, Markus Hoenicka wrote: > Hi Ethan and anyone else who might be listening, > I'm sorry for the sluggish response time. I'm afraid I'm the one > you're looking for. I used to maintain libdbi and the drivers for a > couple of years while I was able to do some programming. I continued > as sort of a caretaker for another while even though there was no > time left for pretty much anything outside my dayjob. I still use > libdbi everyday but I'd have to read the manual how to check in a > patch these days. So long story short - if anyone is willing to take > over development, or at least to act as a caretaker, please step > forward. If you have the skills to check in and test your patch, let > me know. I can add you as a developer to the project. > regards,Markus > On 2019-11-16 21:09, Ethan Funk was heard to say: > > I found and tested a fix for the mysql thread safety problem I > > havebeen troubleshooting. > > Commenting out line 154 of dbd_mysql.c succeeds in fixing the > > problemwhen libdbi-mysql is build against recent libmysqlclient > > libraries.This is the were mysql_server_end() is called after each > > dbi instanceif finished. In a multi-threaded application, multiple > > dbi instancescan come and go in various threads. With recent > > libmysqlclient libraryimplementations, mysql_server_end() should > > only be called ONCE, at theend of the entire program/process, not > > multiple times when a thread isfinished with a dbi intsnace. > > libmysqlclient's mysql_server_end() function performs some > > finallibmysqlclient memory clean-up. So if it isn't called, and > > theapplication ends, it's not really a problem - the OS will clean > > up thememory allocation. > > A few other thread safety notes that might be helpful to others. To > > bethread safe: > > 1. Each program thread must use it's own libdbi instance > > (asdocumented in libdbi). > > 2. The dbi_conn_connect function must be protected by a mutex, > > sinceit results in an underlying libmysqlclient call that is not > > threadsafe, using memory objects allocated once by the > > libmysqlclientlibrary when it was first initialized. This is > > exactly the memory thatwas causing trouble (freed) when > > mysql_server_end() is called bylibdbi_shutdown_r in onw thread when > > the libdbi library is still inuse by other threads prior to program > > termination. > > So... is libdbi and the database drivers still maintained by > > someone?How can I get this fix included so it will be present when > > folksinstall the driver for mysql on their various systems? > > Thanks,Ethan... > > On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote: > > > I think I found the problem in the mysql dbd driver code.: > > > dbd_finalize() calls mysql_server_end(), which is deprecated in > > > the > > > current mysqlclient21 library, but results in a call to > > > areplacement > > > function which is intended to be called only once, at the end of > > > a > > > process. With the call being made inside dbd_finalize(), the > > > mysql_server_end() gets called each time my application > > > codefinished > > > with a libdbi instance. Reading between the lines in the MySQL > > > documentation, it looks think the mysql_server_end() call to > > > it'snew > > > counter part frees memory which is allocated by a pthread_once() > > > function triggered by mysql_library_init(), which itself is > > > calledby > > > mysql_init() via the libdbi driver on each connection > > > attempt. Somy > > > code's first libdbi connection results in the memory > > > beingallocated, > > > with subsequent calls in other threads being ignored. But > > > dbd_finalize() gets called at the end of EVERY libdbi > > > instanceclosure, > > > so the memory if freed over and over again. I don't see a > > > mechanism > > > for a once-per-process driver shutdown callback in libdbi, so I > > > wantto > > > try commenting out mysql_servefr_end() in dbd_finalize(). This > > > will > > > leak memory which is never freed at the end of the process, but > > > so > > > what. It is only allocated once per process, so the > > > operatingsystem > > > process shutdown will free the memory. > > > Unfortunately, I can't seem to build my modified code for > > > testingwith > > > the libdi_dbd project code. "./configure" is giving me the > > > following > > > error: > > > configure: error: Invalid libdbi directory - include files > > > notfound. > > > This is after configure has already indicated that yes, it > > > haslocated > > > the dbi header files, which I installed with the
Re: [libdbi-users] Found and fixed: dbd_mysql thread unsafe problem
Hi Ethan and anyone else who might be listening, I'm sorry for the sluggish response time. I'm afraid I'm the one you're looking for. I used to maintain libdbi and the drivers for a couple of years while I was able to do some programming. I continued as sort of a caretaker for another while even though there was no time left for pretty much anything outside my dayjob. I still use libdbi everyday but I'd have to read the manual how to check in a patch these days. So long story short - if anyone is willing to take over development, or at least to act as a caretaker, please step forward. If you have the skills to check in and test your patch, let me know. I can add you as a developer to the project. regards, Markus On 2019-11-16 21:09, Ethan Funk was heard to say: I found and tested a fix for the mysql thread safety problem I have been troubleshooting. Commenting out line 154 of dbd_mysql.c succeeds in fixing the problem when libdbi-mysql is build against recent libmysqlclient libraries. This is the were mysql_server_end() is called after each dbi instance if finished. In a multi-threaded application, multiple dbi instances can come and go in various threads. With recent libmysqlclient library implementations, mysql_server_end() should only be called ONCE, at the end of the entire program/process, not multiple times when a thread is finished with a dbi intsnace. libmysqlclient's mysql_server_end() function performs some final libmysqlclient memory clean-up. So if it isn't called, and the application ends, it's not really a problem - the OS will clean up the memory allocation. A few other thread safety notes that might be helpful to others. To be thread safe: 1. Each program thread must use it's own libdbi instance (as documented in libdbi). 2. The dbi_conn_connect function must be protected by a mutex, since it results in an underlying libmysqlclient call that is not thread safe, using memory objects allocated once by the libmysqlclient library when it was first initialized. This is exactly the memory that was causing trouble (freed) when mysql_server_end() is called by libdbi_shutdown_r in onw thread when the libdbi library is still in use by other threads prior to program termination. So... is libdbi and the database drivers still maintained by someone? How can I get this fix included so it will be present when folks install the driver for mysql on their various systems? Thanks, Ethan... On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote: I think I found the problem in the mysql dbd driver code.: dbd_finalize() calls mysql_server_end(), which is deprecated in the current mysqlclient21 library, but results in a call to a replacement function which is intended to be called only once, at the end of a process. With the call being made inside dbd_finalize(), the mysql_server_end() gets called each time my application code finished with a libdbi instance. Reading between the lines in the MySQL documentation, it looks think the mysql_server_end() call to it's new counter part frees memory which is allocated by a pthread_once() function triggered by mysql_library_init(), which itself is called by mysql_init() via the libdbi driver on each connection attempt. So my code's first libdbi connection results in the memory being allocated, with subsequent calls in other threads being ignored. But dbd_finalize() gets called at the end of EVERY libdbi instance closure, so the memory if freed over and over again. I don't see a mechanism for a once-per-process driver shutdown callback in libdbi, so I want to try commenting out mysql_servefr_end() in dbd_finalize(). This will leak memory which is never freed at the end of the process, but so what. It is only allocated once per process, so the operating system process shutdown will free the memory. Unfortunately, I can't seem to build my modified code for testing with the libdi_dbd project code. "./configure" is giving me the following error: configure: error: Invalid libdbi directory - include files not found. This is after configure has already indicated that yes, it has located the dbi header files, which I installed with the libdbi-dev package with the Ubuntu apt command. How might I be able to fix this error, and proceed with testing my code fix? Thanks, Ethan... On Wed, 2019-11-13 at 14:19 -0700, Ethan Funk wrote: I am having a problem with libdbd_mysql on linux (Ubuntu 19.10) reagding thread fasety. There used to be two versions of libmysqlclient, one was thread safe, one was not. However, now there seems to be only one client library, which I understood was thread safe: libmysqlclient.so.21. Any insight into the dump below would be greatly appreciated! Ethan... Here is a valgrind dump of a read-already-freed error that occurs in what I will call thread #3 when it attempts to connect to MySQL. This thread has it's own libdbi instance, yet mysql_real_connect is trying to read memory allocated
[libdbi-users] Found and fixed: dbd_mysql thread unsafe problem
I found and tested a fix for the mysql thread safety problem I have been troubleshooting. Commenting out line 154 of dbd_mysql.c succeeds in fixing the problem when libdbi-mysql is build against recent libmysqlclient libraries. This is the were mysql_server_end() is called after each dbi instance if finished. In a multi-threaded application, multiple dbi instances can come and go in various threads. With recent libmysqlclient library implementations, mysql_server_end() should only be called ONCE, at the end of the entire program/process, not multiple times when a thread is finished with a dbi intsnace. libmysqlclient's mysql_server_end() function performs some final libmysqlclient memory clean-up. So if it isn't called, and the application ends, it's not really a problem - the OS will clean up the memory allocation. A few other thread safety notes that might be helpful to others. To be thread safe: 1. Each program thread must use it's own libdbi instance (as documented in libdbi). 2. The dbi_conn_connect function must be protected by a mutex, since it results in an underlying libmysqlclient call that is not thread safe, using memory objects allocated once by the libmysqlclient library when it was first initialized. This is exactly the memory that was causing trouble (freed) when mysql_server_end() is called by libdbi_shutdown_r in onw thread when the libdbi library is still in use by other threads prior to program termination. So... is libdbi and the database drivers still maintained by someone? How can I get this fix included so it will be present when folks install the driver for mysql on their various systems? Thanks, Ethan... On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote: > I think I found the problem in the mysql dbd driver code.: > > dbd_finalize() calls mysql_server_end(), which is deprecated in the > current mysqlclient21 library, but results in a call to a replacement > function which is intended to be called only once, at the end of a > process. With the call being made inside dbd_finalize(), the > mysql_server_end() gets called each time my application code finished > with a libdbi instance. Reading between the lines in the MySQL > documentation, it looks think the mysql_server_end() call to it's new > counter part frees memory which is allocated by a pthread_once() > function triggered by mysql_library_init(), which itself is called by > mysql_init() via the libdbi driver on each connection attempt. So my > code's first libdbi connection results in the memory being allocated, > with subsequent calls in other threads being ignored. But > dbd_finalize() gets called at the end of EVERY libdbi instance closure, > so the memory if freed over and over again. I don't see a mechanism > for a once-per-process driver shutdown callback in libdbi, so I want to > try commenting out mysql_servefr_end() in dbd_finalize(). This will > leak memory which is never freed at the end of the process, but so > what. It is only allocated once per process, so the operating system > process shutdown will free the memory. > > Unfortunately, I can't seem to build my modified code for testing with > the libdi_dbd project code. "./configure" is giving me the following > error: > configure: error: Invalid libdbi directory - include files not found. > > This is after configure has already indicated that yes, it has located > the dbi header files, which I installed with the libdbi-dev package > with the Ubuntu apt command. > > How might I be able to fix this error, and proceed with testing my code > fix? > > Thanks, > Ethan... > > On Wed, 2019-11-13 at 14:19 -0700, Ethan Funk wrote: > > I am having a problem with libdbd_mysql on linux (Ubuntu 19.10) > > reagding thread fasety. > > There used to be two versions of libmysqlclient, one was thread safe, > > one was not. However, > > now there seems to be only one client library, which I understood was > > thread safe: libmysqlclient.so.21. > > > > Any insight into the dump below would be greatly appreciated! > > > > Ethan... > > > > Here is a valgrind dump of a read-already-freed error that occurs in > > what I will call thread #3 > > when it attempts to connect to MySQL. This thread has it's own libdbi > > instance, yet mysql_real_connect > > is trying to read memory allocated by the first thread (thread #1) that > > first called mysql_real_connect, > > after that same memory was freed by another thread (thread #2) that > > called mysql_server_end as it was > > finishing. Each thread has it's own libdbi instance, so why is > > libmysqlclient sharring memory between > > threads? It's looking like the libmysqlclient isn't thread safe. > > > > ==2828== Invalid read of size 1 > > ==2828==at 0x4A3A042: strlen (vg_replace_strmem.c:461) > > ==2828==by 0x141A46C1: ??? (in /usr/lib/x86_64-linux- > > gnu/libmysqlclient.so.21.1.17) > > ==2828==by 0x141A5468: ??? (in /usr/lib/x86_64-linux- > > gnu/libmysqlclient.so.21.1.17) > > ==2828==
