one more thing here, i noticed now that the query formed by sa when we do an
eager load has got some problems (may be i am not doing the right thing)
here's the problem
i have entities A, B. where A -> B (1:N relationship)
i form a query like this
clauses = []
clauses.append(A.c.col1 == 'xyz')
clauses.append(B.c.col == 'xcv')
qry = session.query(B).filter_by(*clauses)
eager_qry = qry.options(sqlalchemy.eagerload('a')
eager_qry.all()
the sql shows:
select ... from A, B left outer join A as alias on alias.key == B.key
why is A included for join two times ? i understand eager load might be
creating the outer join but looks like because i am having a clause on A, A
is included in the first join as well.
what is the right way to use it so that i can get rid off first join and
eager load A.
this is creating a huge result set.
On Wed, Jun 18, 2008 at 8:10 PM, Arun Kumar PG <[EMAIL PROTECTED]>
wrote:
> thanks for the instant reply guys!
>
> as my app is on production so i cannot afford to bring things down right
> away for 0.4/0.5 migration. eventually, i will be going to (in next month)
> use 0.4/0.5. so for the time being (at least for the next one month) i am
> looking for the best solution on 0.3.x so that users are not affected.
>
> michael, as you mentioned about explicit cleaning of session, i am doing
> that currently. let me quickly mention the flow of request so that you guys
> can have more information:
>
> - search request comes
> - if orm mapping is not created it's get created now (only happens one
> time)
> - new session is created and attached to the current thread (this is done
> so that different DAOs can access the same session from the current thread)
> - all orm queries are fired.. results processed
> - finally, current thread is accessed again, session attached earlier is
> accessed, session.clear() invoked and del session done.
>
> what's the best way to deal with the problem now...
>
> thanks,
>
> - A
>
>
>
>
> On Wed, Jun 18, 2008 at 7:49 PM, Michael Bayer <[EMAIL PROTECTED]>
> wrote:
>
>>
>>
>> On Jun 18, 2008, at 9:59 AM, Arun Kumar PG wrote:
>>
>> > hi all folks,
>> >
>> > i have a search form that allows user to search for records. i am
>> > eager loading 4 attributes on the master object which results in 4
>> > left outer joins in the sa's sql query. the problem is that when i
>> > look at the memory consumption using top command it looks crazy.
>> >
>> > the memory shoots up by 50-60 MB instantly (some times even 100+
>> > MB). i executed the query on db directly and the results are
>> > returned in 3 secs (close to around 60,000 rows). sa is spending a
>> > good amount of time processing the results and while it is doing
>> > that i see abnormal memory growth. also the cpu is used almost 98%
>> > during this time.
>> >
>> > the interesting thing is that after processing the request the
>> > memory does not comes down. it stays there only. i dont know why its
>> > not gc'ed.
>> >
>> > my environment:
>> > - mysql 4.1
>> > - sa 3.9
>> > - python 2.4
>> >
>> > is there any chance that memory is getting leaked as i don't see
>> > memory come down even after some time.
>> >
>>
>> The Session in 0.3 does not lose references to any data loaded
>> automatically, it has to be cleaned out manually using
>> session.expunge(obj) or session.clear(). From 0.4 on forward the
>> Session is weak referencing so that unreferenced, clean objects fall
>> out of scope automatically. 0.4 also eager loads many rows about 30%
>> faster than 0.3 and 0.5 is then about 15% faster than 0.4. ORMs in
>> general are designed for rich in-memory functionality and are not
>> optimized for loads of many tens of thousands of rows, so for better
>> performance overall consider non-ORM access to these rows.
>>
>> >>
>>
>
>
> --
> cheers,
>
> - a
--
cheers,
- a
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
