How many user IDs are you looking up? It would be much more efficient to use the SQL IN operator and pass the set of IDs to the database.
https://www.w3schools.com/sql/sql_in.asp > On 27 Sep 2020, at 11:23 pm, Charlie <[email protected]> wrote: > > The threads are started in the following function : > > def get_users_fullname(site, ids): > users = {} > threads = [] > for id in ids: > t = SQLThread (id) > threads.append(t) > t.start() > for t in threads: > t.join() > users[str(t.id <http://t.id/>)] = t._fullname > return users > > I have to collect a lot of data from 5 sql servers. I tried without > multithreading and i was very long to process, then i use threading module > and the process time was reduced by 10. > > I tried to give the sql cursor to the thread to avoid reconnecting to the sql > server but it didn't work, maybe it didn't use the good method > > Le dimanche 27 septembre 2020 à 13:28:34 UTC+2, Graham Dumpleton a écrit : > You never actually start the thread by calling start() on the thread object. > > Do be aware that creating a background thread per request like this to do a > database operation is generally a bad idea. If hit with a large number of > requests you could very easily overwhelm the number of allowed database > connections. You also aren't cleaning up the state of the thread by calling > join() on it afterwards either. > > Is there a reason you aren't doing this within the context of the request > rather than in a background thread? > > Graham > > >> On 27 Sep 2020, at 9:23 pm, Charlie <[email protected] >> <applewebdata://2349FF30-FE2D-4D86-9F24-3334FAE9BF81>> wrote: >> > >> My code is : >> >> from threading import Thread >> >> class SQLThread(Thread): >> >> def __init__(self, id): >> print(' SQLThread ') >> Thread.__init__(self) >> self.id <http://self.id/> = id >> self._fullname = None >> >> def run(self): >> try: >> print('run') >> connection = MySQLdb.connect(MYSQL_SERVER, >> MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB, connect_timeout=2) >> cursor = connection.cursor() >> self._fullname = get_user_fullname(cursor, self.id >> <http://self.id/>) >> connection.close() >> except MySQLdb.Error as e: >> print('SQLThread: %s' % e) >> >> Sorry, i thought the threads never run because 'run' is not written in the >> apache log file whereas ' SQLThread' is written. Do you know why ? >> Le dimanche 27 septembre 2020 à 12:04:03 UTC+2, Graham Dumpleton a écrit : >> Would need to see an example of the code you are talking about to understand >> better what you are talking about. In particular, how are you creating the >> threads and where? How are you waiting on the threads to complete, etc? >> >> Also, are you using manually configure Apache/mod_wsgi or are you using >> mod_wsgi-express? If manually configure Apache/mod_wsgi, how are you >> configuring mod_wsgi? >> >> Graham >> >> >>> On 27 Sep 2020, at 8:01 pm, Charlie <charlie77...@ <>gmail.com >>> <http://gmail.com/>> wrote: >>> >> >>> I use threading module for concurrent SQL queries. Using a simple python >>> script, all threads run normaly but when i use the same code in a WSGI app, >>> the threads never start >>> >>> Thanks for your help >>> >>> Charlie >>> >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to modwsgi+u...@ <>googlegroups.com <http://googlegroups.com/>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/modwsgi/b2e6594a-0477-4b4e-a2b6-bba162922296n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/modwsgi/b2e6594a-0477-4b4e-a2b6-bba162922296n%40googlegroups.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] >> <applewebdata://2349FF30-FE2D-4D86-9F24-3334FAE9BF81>. > >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/modwsgi/899fb259-674c-4b02-bded-906274f18321n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/modwsgi/899fb259-674c-4b02-bded-906274f18321n%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/modwsgi/3418ff45-33ed-480f-801c-e58cdbf4ab96n%40googlegroups.com > > <https://groups.google.com/d/msgid/modwsgi/3418ff45-33ed-480f-801c-e58cdbf4ab96n%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/modwsgi/8B9E566E-39AD-44CF-95A8-2C809C938E4A%40gmail.com.
