To add more information to it:
We are running our dB on AWS RDS, this is what MySQL say in logs when we
switched to mysql-connector:
[Note] Aborted connection 9191 to db: 'connstr' (Got an error reading
communication packets)
Best Regards,
Josip
On Monday, July 17, 2017 at 9:47:50 AM UTC+2, josip povreslo wrote:
>
> Mike,
>
> Just as an info, mysql-connector is a total NO for us, when I switched to
> mysql-connector we saw bigger problems and those happens much more often.
> PyMySQL is the last thing to switch to and try. For this processing piece
> I'm thinking to replace SQLAlchemy with a direct database driver usage, do
> you think that's the catch or this is a more general issue with the drivers
> itself?
>
> Best,
> Josip
>
> On Friday, July 14, 2017 at 11:57:12 PM UTC+2, Mike Bayer wrote:
>>
>> that "mysqldb" URL is not mysql-connector - it's MySQL-Python (old and
>> unmaintained) or mysqlclient (much better, if you have it installed
>> and not conflicting w/ MySQL-Python). You'd need to specify
>> "mysql+mysqlconnector://" to use that DBAPI, but also I'd recommend
>> trying "mysql+pymysql://" to see if you get better results.
>>
>> as far as "can't connect to server" you should make sure you are not
>> surpassing your max_connections limit, though it should be emitting
>> different error message for that.
>>
>> The Queue.Empty error is not the "error", it has to do with how the
>> QueuePool works internally to test that the pool is empty, and Python
>> 3 can't help but report every exception in the chain. The error is
>> the can't connect to server.
>>
>>
>>
>> On Fri, Jul 14, 2017 at 5:41 PM, josip povreslo
>> <josip.p...@gmail.com> wrote:
>> > In addition to the previous email, what I do is the following:
>> >
>> > When request comes in:
>> > self.mysql_engine =
>> >
>> create_engine('mysql+mysqldb://{}:{}@{}/{}'.format(self.db_params['dbuser'],
>>
>> > self.db_params['dbpass'], self.db_params['db_hosts'][db_host_index],
>> > self.db_params['dbname']), pool_recycle=3600,
>> > connect_args={'connect_timeout': 2})
>> >
>> > And then when need to interact with the dB:
>> >
>> > connection = self.mysql_engine.connect()
>> > connection.execute("raw query")
>> > connection.close()
>> >
>> >
>> > Best Regards,
>> > Josip
>> >
>> > On Friday, July 14, 2017 at 11:36:26 PM UTC+2, josip povreslo wrote:
>> >>
>> >> Hello,
>> >>
>> >> As mentioned in the subject, we have the following packages installed:
>> >>
>> >> Python 3.5
>> >> mysql-connector==2.1.6
>> >> SQLAlchemy==1.1.11
>> >>
>> >> Our service is working until we get a bit more traffic (40 - 50
>> req/min),
>> >> although I'm still not 100% if it's a pattern, however, then we get
>> this
>> >> error:
>> >>
>> >> Traceback (most recent call last):
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 1122, in _do_get
>> >> return self._pool.get(wait, self._timeout)
>> >> File
>> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/queue.py",
>> line
>> >> 145, in get
>> >> raise Empty
>> >> sqlalchemy.util.queue.Empty
>> >> During handling of the above exception, another exception occurred:
>> >> Traceback (most recent call last):
>> >> File
>> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/base.py",
>> line
>> >> 2147, in _wrap_pool_connect
>> >> return fn()
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 328, in unique_connection
>> >> return _ConnectionFairy._checkout(self)
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 766, in _checkout
>> >> fairy = _ConnectionRecord.checkout(pool)
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 516, in checkout
>> >> rec = pool._do_get()
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 1138, in _do_get
>> >> self._dec_overflow()
>> >> File
>> >>
>> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/langhelpers.py",
>> >> line 66, in __exit__
>> >> compat.reraise(exc_type, exc_value, exc_tb)
>> >> File
>> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/compat.py",
>> line
>> >> 187, in reraise
>> >> raise value
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 1135, in _do_get
>> >> return self._create_connection()
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 333, in _create_connection
>> >> return _ConnectionRecord(self)
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 461, in __init__
>> >> self.__connect(first_connect_check=True)
>> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py",
>> line
>> >> 651, in __connect
>> >> connection = pool._invoke_creator(self)
>> >> File
>> >>
>> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/strategies.py",
>> >> line 105, in connect
>> >> return dialect.connect(*cargs, **cparams)
>> >> File
>> >>
>> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/default.py",
>> >> line 393, in connect
>> >> return self.dbapi.connect(*cargs, **cparams)
>> >> File "/usr/local/lib64/python3.5/site-packages/MySQLdb/__init__.py",
>> >> line 81, in Connect
>> >> return Connection(*args, **kwargs)
>> >> File
>> "/usr/local/lib64/python3.5/site-packages/MySQLdb/connections.py",
>> >> line 191, in __init__
>> >> super(Connection, self).__init__(*args, **kwargs2)
>> >> _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL
>> server
>> >> on 'dbhost_obviously' (4)")
>> >>
>> >> We haven't been seeing this error up until recently when we ported our
>> >> service to Python 3.5. I've read most of the articles related to this
>> queue
>> >> empty error and most of those are from '12 or '13 and mentioning
>> immature
>> >> mysql drivers. Is this still the case, should we search for an
>> alternative
>> >> to mysql-connector or alternative to something else? Any help or
>> guidance is
>> >> appreciated!
>> >>
>> >> Thanks!
>> >>
>> >> Best,
>> >> Josip
>> >
>> > --
>> > SQLAlchemy -
>> > The Python SQL Toolkit and Object Relational Mapper
>> >
>> > http://www.sqlalchemy.org/
>> >
>> > To post example code, please provide an MCVE: Minimal, Complete, and
>> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full
>> > description.
>> > ---
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "sqlalchemy" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to sqlalchemy+...@googlegroups.com.
>> > To post to this group, send email to sqlal...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/sqlalchemy.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>
--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
http://www.sqlalchemy.org/
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable
Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
