Hi Ethan and anyone else who might be listening,

I'm sorry for the sluggish response time. I'm afraid I'm the one you're looking for. I used to maintain libdbi and the drivers for a couple of years while I was able to do some programming. I continued as sort of a caretaker for another while even though there was no time left for pretty much anything outside my dayjob. I still use libdbi everyday but I'd have to read the manual how to check in a patch these days. So long story short - if anyone is willing to take over development, or at least to act as a caretaker, please step forward. If you have the skills to check in and test your patch, let me know. I can add you as a developer to the project.

regards,
Markus

On 2019-11-16 21:09, Ethan Funk was heard to say:
I found and tested a fix for the mysql thread safety problem I have
been troubleshooting.

Commenting out line 154 of dbd_mysql.c succeeds in fixing the problem
when libdbi-mysql is build against recent libmysqlclient libraries.
This is the were mysql_server_end() is called after each dbi instance
if finished. In a multi-threaded application, multiple dbi instances
can come and go in various threads. With recent libmysqlclient library
implementations, mysql_server_end() should only be called ONCE, at the
end of the entire program/process, not multiple times when a thread is
finished with a dbi intsnace.

libmysqlclient's mysql_server_end() function performs some final
libmysqlclient memory clean-up. So if it isn't called, and the
application ends, it's not really a problem - the OS will clean up the
memory allocation.

A few other thread safety notes that might be helpful to others. To be
thread safe:

1. Each program thread must use it's own libdbi instance (as
documented in libdbi).

2. The dbi_conn_connect function must be protected by a mutex, since
it results in an underlying libmysqlclient call that is not thread
safe, using memory objects allocated once by the libmysqlclient
library when it was first initialized. This is exactly the memory that
was causing trouble (freed) when mysql_server_end() is called by
libdbi_shutdown_r in onw thread when the libdbi library is still in
use by other threads prior to program termination.

So... is libdbi and the database drivers still maintained by someone?
How can I get this fix included so it will be present when folks
install the driver for mysql on their various systems?

Thanks,
Ethan...

On Fri, 2019-11-15 at 09:35 -0700, Ethan Funk wrote:

I think I found the problem in the mysql dbd driver code.:

dbd_finalize() calls mysql_server_end(), which is deprecated in the

current mysqlclient21 library, but results in a call to a
replacement

function which is intended to be called only once, at the end of a

process.  With the call being made inside dbd_finalize(), the

mysql_server_end() gets called each time my application code
finished

with a libdbi instance.  Reading between the lines in the MySQL

documentation, it looks think the mysql_server_end() call to it's
new

counter part frees memory which is allocated by a pthread_once()

function triggered by mysql_library_init(), which itself is called
by

mysql_init() via the libdbi driver on each connection attempt.  So
my

code's first libdbi connection results in the memory being
allocated,

with subsequent calls in other threads being ignored.  But

dbd_finalize() gets called at the end of EVERY libdbi instance
closure,

so the memory if freed over and over again.  I don't see a mechanism

for a once-per-process driver shutdown callback in libdbi, so I want
to

try commenting out mysql_servefr_end() in dbd_finalize().  This will

leak memory which is never freed at the end of the process, but so

what.  It is only allocated once per process, so the operating
system

process shutdown will free the memory.

Unfortunately, I can't seem to build my modified code for testing
with

the libdi_dbd project code. "./configure" is giving me the following

error:

configure: error: Invalid libdbi directory - include files not
found.

This is after configure has already indicated that yes, it has
located

the dbi header files, which I installed with the libdbi-dev package

with the Ubuntu apt command.

How might I be able to fix this error, and proceed with testing my
code

fix?

Thanks,

Ethan...

On Wed, 2019-11-13 at 14:19 -0700, Ethan Funk wrote:

I am having a problem with libdbd_mysql on linux (Ubuntu 19.10)

reagding thread fasety.

There used to be two versions of libmysqlclient, one was thread
safe,

one was not. However,

now there seems to be only one client library, which I understood
was

thread safe: libmysqlclient.so.21.

Any insight into the dump below would be greatly appreciated!

Ethan...

Here is a valgrind dump of a read-already-freed error that occurs
in

what I will call thread #3

when it attempts to connect to MySQL.  This thread has it's own
libdbi

instance, yet mysql_real_connect

is trying to read memory allocated by the first thread (thread #1)
that

first called mysql_real_connect,

after that same memory was freed by another thread (thread #2)
that

called mysql_server_end as it was

finishing. Each thread has it's own libdbi instance, so why is

libmysqlclient sharring memory between

threads? It's looking like the libmysqlclient isn't thread safe.

==2828== Invalid read of size 1

==2828==    at 0x4A3A042: strlen (vg_replace_strmem.c:461)

==2828==    by 0x141A46C1: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A5468: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A54A8: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A373E: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x4FE041E: __pthread_once_slow (pthread_once.c:116)

==2828==    by 0x141A320D: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A41B2: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A429E: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1415472B: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1415492A: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141577A8: mysql_real_connect (in /usr/lib/x86_64-

linux-gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x12F47435: dbd_connect (in /usr/lib/x86_64-linux-

gnu/dbd/libdbdmysql.so)

==2828==    by 0x4DC37AE: dbi_conn_connect (in
/usr/lib/x86_64-linux-

gnu/libdbi.so.1.1.0)

==2828==    by 0x134BFA: dbSetupConnection (database.c:211)

==2828==    by 0x1357F6: DeleteLogEntry (database.c:440)

==2828==    by 0x131ECE: taskCallProcInThread (tasks.c:73)

==2828==    by 0x4FD7668: start_thread (pthread_create.c:479)

==2828==    by 0x567B322: clone (clone.S:95)

Memory was freed in a thread (lets call it thread #2) that ran and

completed. It created it's own

dbi instance, did it's thing, and then closed it's MySQL
connection and

shutdown the instance.

So why is it freeing memory deep in the mysql driver that it did
not

allocate?

==2828==  Address 0x1343e698 is 424 bytes inside a block of size
4,088

free'd

==2828==    at 0x4A37F90: free (vg_replace_malloc.c:540)

==2828==    by 0x141A9B8A: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A8756: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x14148982: mysql_server_end (in
/usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x12F4684C: dbd_finalize (in /usr/lib/x86_64-linux-

gnu/dbd/libdbdmysql.so)

==2828==    by 0x4DC2E4D: dbi_shutdown_r (in
/usr/lib/x86_64-linux-

gnu/libdbi.so.1.1.0)

==2828==    by 0x134440: dbi_instance_free (database.c:56)

==2828==    by 0x4FD6600: __nptl_deallocate_tsd.part.0

(pthread_create.c:301)

==2828==    by 0x4FD7689: __nptl_deallocate_tsd
(pthread_create.c:256)

==2828==    by 0x4FD7689: start_thread (pthread_create.c:490)

==2828==    by 0x567B322: clone (clone.S:95)

This is in the first thread in the program to make use of
libdbi/MySQL,

and it is still

running/connected to MySQL using a unique libdbi intance.

==2828==  Block was alloc'd at

==2828==    at 0x4A36E96: malloc (vg_replace_malloc.c:309)

==2828==    by 0x141A9A30: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A9AEB: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A4E1C: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1419CFA0: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1419D95E: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1416FD29: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A2A66: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A3771: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x4FE041E: __pthread_once_slow (pthread_once.c:116)

==2828==    by 0x141A320D: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A41B2: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141A429E: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1415472B: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x1415492A: ??? (in /usr/lib/x86_64-linux-

gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x141577A8: mysql_real_connect (in /usr/lib/x86_64-

linux-gnu/libmysqlclient.so.21.1.17)

==2828==    by 0x12F47435: dbd_connect (in /usr/lib/x86_64-linux-

gnu/dbd/libdbdmysql.so)

==2828==    by 0x4DC37AE: dbi_conn_connect (in
/usr/lib/x86_64-linux-

gnu/libdbi.so.1.1.0)

==2828==    by 0x134BFA: dbSetupConnection (database.c:211)

==2828==    by 0x13D2F4: GetItemMetaData (database.c:2960)

==2828==
_______________________________________________
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users

--
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38


_______________________________________________
libdbi-users mailing list
libdbi-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libdbi-users

Reply via email to