On 11/27/2015 05:20 AM, Hongxin Song wrote: > In multithreaded scenarios , it is a serious problem , because the > server will be oom
that's part of CPython, first see: http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm so once you use 2G of RAM, that's permanent. So don't use 2G of RAM. Keep in mind the DBAPI fetches all 1M rows at once no matter what, even if you don't fetch them. But the ORM object might be much bigger than your row. The two primary options are to 1. use yield_per() to reduce ORM-level memory use and 2. query only for windows of data at a time to reduce *all* memory use, but more complexity at the query level. Some discussion is at: http://stackoverflow.com/questions/7389759/memory-efficient-built-in-sqlalchemy-iterator-generator/7390660#7390660 > > 在 2015年11月27日星期五 UTC+8下午4:44:50，Hongxin Song写道： > > > <https://lh3.googleusercontent.com/-JTTCeJzu3V0/VlgX4JxTCkI/AAAAAAAAAAo/K9fx7wj3LUc/s1600/694275B6-CFE4-49AE-81C1-51F80C67630A.png> > > > when I query a lot of record from mysql by session.query, which is > one million more, > > and the size of memory reaches to 2G Byte , I want to release the > memory *immediately ， * > > *but , try to some kind of method , it does not work * > * > * > *please help !* > > > <https://lh3.googleusercontent.com/-JTTCeJzu3V0/VlgX4JxTCkI/AAAAAAAAAAo/K9fx7wj3LUc/s1600/694275B6-CFE4-49AE-81C1-51F80C67630A.png> > > * > * > > * > * > * > | > 在此输入代码... > | > * > * > session_factory = sessionmaker( > autocommit=False, autoflush=True, > expire_on_commit=False, bind=engine > ) > > * > * > Session = scoped_session(session_factory) > > Base = declarative_base() > Base.query = Session.query_property() > > class ORMBase(object): > @classmethod > def getlist(cls, **kw): > return cls.query.filter_by(**kw).all() > * > > -- > You received this message because you are subscribed to the Google > Groups "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to sqlalchemy+unsubscr...@googlegroups.com > <mailto:sqlalchemy+unsubscr...@googlegroups.com>. > To post to this group, send email to sqlalchemy@googlegroups.com > <mailto:sqlalchemy@googlegroups.com>. > Visit this group at http://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
