On Mar 13, 2014, at 11:51 PM, Ni Wesley <[email protected]> wrote:
> Hi all,
> I hit a problem that cannot use the connection pool effectively.
>
> messages_writer = DBWriter('messages')
> connection_writer = DBWriter('sessions_details')
> messages_details_writer = DBWriter('messages_details')
>
>
> In other place, I call the connection_writer to do rapid db operations.
> For example, keep operatons rapidly.
>
> And, I watch from the client by netstat -anpto|grep mysql_server_ip|wc -l
>
> I see the number is always 1 or 2.
the code you sent does not do anything concurrently? it only has code
referring to "gevent", but that code is all commented out.
if you'd like to see a connection pool open more than two connections, run this
program:
from sqlalchemy import create_engine
e = create_engine("mysql://scott:tiger@localhost/test")
conn1 = e.connect()
conn2 = e.connect()
conn3 = e.connect()
raw_input('take a look at your netstat now')
run your netstat in a separate window at the prompt, and you'll see three
connections open.
Add this:
conn4 = e.connect()
then you'll see four. Simple right?
that's all I can give you. There is nothing mysterious going on, so as to why
your program isn't concurrent, I have no idea, you aren't really sharing the
actual program here (and you don't have to, I don't have time to debug a whole
gevent program).
But SQLAlchemy likely has nothing to do with the issue. If it does, you'd need
to boil it down to a simple test case that any of us can run which confirms a
specific issue in SQLAlchemy. Maybe QueuePool locks in some funny way with
gevent (noone has reported that, but it does have some mutexes in there on
checkout and I've no idea what mutexes do when you're using gevent). Try
using NullPool instead then. That's about it. Or just use a straight MySQL
connection and cursor, this program is so simplistic running just three SQL
statements, I think you'll find if you run it using no SQLAlchemy at all, it
will do exactly the same thing with the same concurrency/performance issues.
> Actually, the other script who is calling db opeartion is slow, why the tcp
> connections not increased to improve the performance?
As I said I've not used gevent very much, maybe gevent has a community that can
be consulted also.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.
