Yep, that's where I've stopped with my research for now. Mysqlclient 
handles relatively ok for now although I think as well it's a problem with 
network there, wouldn't be the first time. Eithter way, thanks for your 
support, I'll get back with full details if/when I completely understand 
what's happening and how to configure the whole thing. 

Best,
Josip

On Monday, July 17, 2017 at 3:30:16 PM UTC+2, Mike Bayer wrote:
>
> On Mon, Jul 17, 2017 at 4:23 AM, josip povreslo 
> <josip.p...@gmail.com <javascript:>> wrote: 
> > To add more information to it: 
> > 
> > We are running our dB on AWS RDS, this is what MySQL say in logs when we 
> > switched to mysql-connector: 
>
> Oh.  That is an *extremely* different situation.    Amazon RDS is a 
> MySQL variant that runs in an entirely Amazon-specific environment. 
> They are likely performing some kind of simple rate limiting on your 
> service, or otherwise having their own kinds of outages.    It is not 
> normal that they should have basic problems like this with off the 
> shelf MySQL client libraries, and issues like "can't connect" are not 
> normal in the MySQL world unless there are network issues.  I don't 
> have experience with RDS so you'd need to check with their support. 
>
>
>
> > 
> > [Note] Aborted connection 9191 to db: 'connstr' (Got an error reading 
> > communication packets) 
> > 
> > 
> > Best Regards, 
> > Josip 
> > 
> > 
> > On Monday, July 17, 2017 at 9:47:50 AM UTC+2, josip povreslo wrote: 
> >> 
> >> Mike, 
> >> 
> >> Just as an info, mysql-connector is a total NO for us, when I switched 
> to 
> >> mysql-connector we saw bigger problems and those happens much more 
> often. 
> >> PyMySQL is the last thing to switch to and try. For this processing 
> piece 
> >> I'm thinking to replace SQLAlchemy with a direct database driver usage, 
> do 
> >> you think that's the catch or this is a more general issue with the 
> drivers 
> >> itself? 
> >> 
> >> Best, 
> >> Josip 
> >> 
> >> On Friday, July 14, 2017 at 11:57:12 PM UTC+2, Mike Bayer wrote: 
> >>> 
> >>> that "mysqldb" URL is not mysql-connector - it's MySQL-Python (old and 
> >>> unmaintained) or mysqlclient (much better, if you have it installed 
> >>> and not conflicting w/ MySQL-Python).   You'd need to specify 
> >>> "mysql+mysqlconnector://" to use that DBAPI, but also I'd recommend 
> >>> trying "mysql+pymysql://" to see if you get better results. 
> >>> 
> >>> as far as "can't connect to server" you should make sure you are not 
> >>> surpassing your max_connections limit, though it should be emitting 
> >>> different error message for that. 
> >>> 
> >>> The Queue.Empty error is not the "error", it has to do with how the 
> >>> QueuePool works internally to test that the pool is empty, and Python 
> >>> 3 can't help but report every exception in the chain.   The error is 
> >>> the can't connect to server. 
> >>> 
> >>> 
> >>> 
> >>> On Fri, Jul 14, 2017 at 5:41 PM, josip povreslo 
> >>> <josip.p...@gmail.com> wrote: 
> >>> > In addition to the previous email, what I do is the following: 
> >>> > 
> >>> > When request comes in: 
> >>> > self.mysql_engine = 
> >>> > 
> >>> > 
> create_engine('mysql+mysqldb://{}:{}@{}/{}'.format(self.db_params['dbuser'], 
>
> >>> > self.db_params['dbpass'], self.db_params['db_hosts'][db_host_index], 
> >>> > self.db_params['dbname']), pool_recycle=3600, 
> >>> > connect_args={'connect_timeout': 2}) 
> >>> > 
> >>> > And then when need to interact with the dB: 
> >>> > 
> >>> > connection = self.mysql_engine.connect() 
> >>> > connection.execute("raw query") 
> >>> > connection.close() 
> >>> > 
> >>> > 
> >>> > Best Regards, 
> >>> > Josip 
> >>> > 
> >>> > On Friday, July 14, 2017 at 11:36:26 PM UTC+2, josip povreslo wrote: 
> >>> >> 
> >>> >> Hello, 
> >>> >> 
> >>> >> As mentioned in the subject, we have the following packages 
> installed: 
> >>> >> 
> >>> >> Python 3.5 
> >>> >> mysql-connector==2.1.6 
> >>> >> SQLAlchemy==1.1.11 
> >>> >> 
> >>> >> Our service is working until we get a bit more traffic (40 - 50 
> >>> >> req/min), 
> >>> >> although I'm still not 100% if it's a pattern, however, then we get 
> >>> >> this 
> >>> >> error: 
> >>> >> 
> >>> >> Traceback (most recent call last): 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 1122, in _do_get 
> >>> >>     return self._pool.get(wait, self._timeout) 
> >>> >>   File 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/queue.py", 
> >>> >> line 
> >>> >> 145, in get 
> >>> >>     raise Empty 
> >>> >> sqlalchemy.util.queue.Empty 
> >>> >> During handling of the above exception, another exception occurred: 
> >>> >> Traceback (most recent call last): 
> >>> >>   File 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", 
> >>> >> line 
> >>> >> 2147, in _wrap_pool_connect 
> >>> >>     return fn() 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 328, in unique_connection 
> >>> >>     return _ConnectionFairy._checkout(self) 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 766, in _checkout 
> >>> >>     fairy = _ConnectionRecord.checkout(pool) 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 516, in checkout 
> >>> >>     rec = pool._do_get() 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 1138, in _do_get 
> >>> >>     self._dec_overflow() 
> >>> >>   File 
> >>> >> 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/langhelpers.py", 
> >>> >> line 66, in __exit__ 
> >>> >>     compat.reraise(exc_type, exc_value, exc_tb) 
> >>> >>   File 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/compat.py", 
> >>> >> line 
> >>> >> 187, in reraise 
> >>> >>     raise value 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 1135, in _do_get 
> >>> >>     return self._create_connection() 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 333, in _create_connection 
> >>> >>     return _ConnectionRecord(self) 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 461, in __init__ 
> >>> >>     self.__connect(first_connect_check=True) 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", 
> >>> >> line 
> >>> >> 651, in __connect 
> >>> >>     connection = pool._invoke_creator(self) 
> >>> >>   File 
> >>> >> 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/strategies.py", 
> >>> >> line 105, in connect 
> >>> >>     return dialect.connect(*cargs, **cparams) 
> >>> >>   File 
> >>> >> 
> >>> >> 
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", 
> >>> >> line 393, in connect 
> >>> >>     return self.dbapi.connect(*cargs, **cparams) 
> >>> >>   File 
> "/usr/local/lib64/python3.5/site-packages/MySQLdb/__init__.py", 
> >>> >> line 81, in Connect 
> >>> >>     return Connection(*args, **kwargs) 
> >>> >>   File 
> >>> >> "/usr/local/lib64/python3.5/site-packages/MySQLdb/connections.py", 
> >>> >> line 191, in __init__ 
> >>> >>     super(Connection, self).__init__(*args, **kwargs2) 
> >>> >> _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL 
> >>> >> server 
> >>> >> on 'dbhost_obviously' (4)") 
> >>> >> 
> >>> >> We haven't been seeing this error up until recently when we ported 
> our 
> >>> >> service to Python 3.5. I've read most of the articles related to 
> this 
> >>> >> queue 
> >>> >> empty error and most of those are from '12 or '13 and mentioning 
> >>> >> immature 
> >>> >> mysql drivers. Is this still the case, should we search for an 
> >>> >> alternative 
> >>> >> to mysql-connector or alternative to something else? Any help or 
> >>> >> guidance is 
> >>> >> appreciated! 
> >>> >> 
> >>> >> Thanks! 
> >>> >> 
> >>> >> Best, 
> >>> >> Josip 
> >>> > 
> >>> > -- 
> >>> > SQLAlchemy - 
> >>> > The Python SQL Toolkit and Object Relational Mapper 
> >>> > 
> >>> > http://www.sqlalchemy.org/ 
> >>> > 
> >>> > To post example code, please provide an MCVE: Minimal, Complete, and 
> >>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a 
> full 
> >>> > description. 
> >>> > --- 
> >>> > You received this message because you are subscribed to the Google 
> >>> > Groups 
> >>> > "sqlalchemy" group. 
> >>> > To unsubscribe from this group and stop receiving emails from it, 
> send 
> >>> > an 
> >>> > email to sqlalchemy+...@googlegroups.com. 
> >>> > To post to this group, send email to sqlal...@googlegroups.com. 
> >>> > Visit this group at https://groups.google.com/group/sqlalchemy. 
> >>> > For more options, visit https://groups.google.com/d/optout. 
> > 
> > -- 
> > SQLAlchemy - 
> > The Python SQL Toolkit and Object Relational Mapper 
> > 
> > http://www.sqlalchemy.org/ 
> > 
> > To post example code, please provide an MCVE: Minimal, Complete, and 
> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full 
> > description. 
> > --- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "sqlalchemy" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to sqlalchemy+...@googlegroups.com <javascript:>. 
> > To post to this group, send email to sqlal...@googlegroups.com 
> <javascript:>. 
> > Visit this group at https://groups.google.com/group/sqlalchemy. 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Reply via email to