Mike,
Just as an info, mysql-connector is a total NO for us, when I switched to
mysql-connector we saw bigger problems and those happens much more often.
PyMySQL is the last thing to switch to and try. For this processing piece
I'm thinking to replace SQLAlchemy with a direct database driver usage, do
you think that's the catch or this is a more general issue with the drivers
itself?
Best,
Josip
On Friday, July 14, 2017 at 11:57:12 PM UTC+2, Mike Bayer wrote:
>
> that "mysqldb" URL is not mysql-connector - it's MySQL-Python (old and
> unmaintained) or mysqlclient (much better, if you have it installed
> and not conflicting w/ MySQL-Python). You'd need to specify
> "mysql+mysqlconnector://" to use that DBAPI, but also I'd recommend
> trying "mysql+pymysql://" to see if you get better results.
>
> as far as "can't connect to server" you should make sure you are not
> surpassing your max_connections limit, though it should be emitting
> different error message for that.
>
> The Queue.Empty error is not the "error", it has to do with how the
> QueuePool works internally to test that the pool is empty, and Python
> 3 can't help but report every exception in the chain. The error is
> the can't connect to server.
>
>
>
> On Fri, Jul 14, 2017 at 5:41 PM, josip povreslo
> <josip.p...@gmail.com <javascript:>> wrote:
> > In addition to the previous email, what I do is the following:
> >
> > When request comes in:
> > self.mysql_engine =
> >
> create_engine('mysql+mysqldb://{}:{}@{}/{}'.format(self.db_params['dbuser'],
>
> > self.db_params['dbpass'], self.db_params['db_hosts'][db_host_index],
> > self.db_params['dbname']), pool_recycle=3600,
> > connect_args={'connect_timeout': 2})
> >
> > And then when need to interact with the dB:
> >
> > connection = self.mysql_engine.connect()
> > connection.execute("raw query")
> > connection.close()
> >
> >
> > Best Regards,
> > Josip
> >
> > On Friday, July 14, 2017 at 11:36:26 PM UTC+2, josip povreslo wrote:
> >>
> >> Hello,
> >>
> >> As mentioned in the subject, we have the following packages installed:
> >>
> >> Python 3.5
> >> mysql-connector==2.1.6
> >> SQLAlchemy==1.1.11
> >>
> >> Our service is working until we get a bit more traffic (40 - 50
> req/min),
> >> although I'm still not 100% if it's a pattern, however, then we get
> this
> >> error:
> >>
> >> Traceback (most recent call last):
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 1122, in _do_get
> >> return self._pool.get(wait, self._timeout)
> >> File
> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/queue.py",
> line
> >> 145, in get
> >> raise Empty
> >> sqlalchemy.util.queue.Empty
> >> During handling of the above exception, another exception occurred:
> >> Traceback (most recent call last):
> >> File
> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/base.py",
> line
> >> 2147, in _wrap_pool_connect
> >> return fn()
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 328, in unique_connection
> >> return _ConnectionFairy._checkout(self)
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 766, in _checkout
> >> fairy = _ConnectionRecord.checkout(pool)
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 516, in checkout
> >> rec = pool._do_get()
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 1138, in _do_get
> >> self._dec_overflow()
> >> File
> >>
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/langhelpers.py",
> >> line 66, in __exit__
> >> compat.reraise(exc_type, exc_value, exc_tb)
> >> File
> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/compat.py",
> line
> >> 187, in reraise
> >> raise value
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 1135, in _do_get
> >> return self._create_connection()
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 333, in _create_connection
> >> return _ConnectionRecord(self)
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 461, in __init__
> >> self.__connect(first_connect_check=True)
> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
> line
> >> 651, in __connect
> >> connection = pool._invoke_creator(self)
> >> File
> >>
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/strategies.py",
> >> line 105, in connect
> >> return dialect.connect(*cargs, **cparams)
> >> File
> >>
> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/default.py",
> >> line 393, in connect
> >> return self.dbapi.connect(*cargs, **cparams)
> >> File "/usr/local/lib64/python3.5/site-packages/MySQLdb/__init__.py",
> >> line 81, in Connect
> >> return Connection(*args, **kwargs)
> >> File
> "/usr/local/lib64/python3.5/site-packages/MySQLdb/connections.py",
> >> line 191, in __init__
> >> super(Connection, self).__init__(*args, **kwargs2)
> >> _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL
> server
> >> on 'dbhost_obviously' (4)")
> >>
> >> We haven't been seeing this error up until recently when we ported our
> >> service to Python 3.5. I've read most of the articles related to this
> queue
> >> empty error and most of those are from '12 or '13 and mentioning
> immature
> >> mysql drivers. Is this still the case, should we search for an
> alternative
> >> to mysql-connector or alternative to something else? Any help or
> guidance is
> >> appreciated!
> >>
> >> Thanks!
> >>
> >> Best,
> >> Josip
> >
> > --
> > SQLAlchemy -
> > The Python SQL Toolkit and Object Relational Mapper
> >
> > http://www.sqlalchemy.org/
> >
> > To post example code, please provide an MCVE: Minimal, Complete, and
> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full
> > description.
> > ---
> > You received this message because you are subscribed to the Google
> Groups
> > "sqlalchemy" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to sqlalchemy+...@googlegroups.com <javascript:>.
> > To post to this group, send email to sqlal...@googlegroups.com
> <javascript:>.
> > Visit this group at https://groups.google.com/group/sqlalchemy.
> > For more options, visit https://groups.google.com/d/optout.
>
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
