Hmm... it looks to me like your situation is a perfect storm.
* You're loading lots of objects.
* You're loading big objects.
* You are guaranteeing an N+1 selects problem with that for loop.
No one of these situations should create a major problem, but all three
together are a guaranteed disappointment.
That said, I'm surprised that 2000 rows would be that much of a problem...
how big are these objects really?
Also, memory wise, there should be almost no difference between using one
session or multiple. And no, I would not open multiple as in your second
example.
As for the serialization error, that might be a bug if LoadPair wasn't
serializable.
Clinton
On Mon, Jan 18, 2010 at 8:24 AM, Dave Rafkind <dave.rafk...@gmail.com>wrote:
> Ok, that's some good information, I understand that you should marshall
> your objects with care. Unfortunately for me at some point I will have to
> marshall every single one of my database rows into an object graph. However
> what I took from your advice was not to be so stingy in re-using the session
> objects. So if I do something like this:
>
> for (id : ids) {
> SqlSession session2 = sessionMapper.openSession();
> MyObject o = session2.select("getOne", id.getActualId());
> session2.close();
> }
>
> Now memory usage is pretty good! However now performance is terrible. In
> attempting to rectify this I tried to put in a custom cache implementation,
> logging cache access to see how it was performing. I noticed it was never
> actually using the cache (never doing cache.putObject).
> I noticed that according to this discussion:
>
>
> http://markmail.org/search/?q=select+commit+cache+list%3Aorg.apache.ibatis.user-java#query:select%20commit%20cache%20list%3Aorg.apache.ibatis.user-java+page:1+mid:4mmwki3dnp57gweu+state:results
>
> ...commits don't occur (and thus cache fill doesn't occur) unless a
> transaction is committed. So for a pure read-only use case of the db the
> cache won't be very useful, right?
>
> Additionally, putting in a "session2.commit()" before the call to close()
> causes the following error (which I assume, but don't know is caused by
> committing when nothing actually needs to be comitted:)
>
> ### Error committing transaction. Cause:
> org.apache.ibatis.cache.CacheException: Error serializing object. Cause:
> java.io.NotSerializableException:
> org.apache.ibatis.executor.loader.ResultLoaderRegistry$LoadPair
>
>
>
> On Thu, Jan 14, 2010 at 2:44 PM, Clinton Begin <clinton.be...@gmail.com>wrote:
>
>> By nested results, yes, I mean collections and associations. And by
>> "flattening" I mean avoiding the use of those.
>>
>> iBATIS exhibits this behavior, the same way any ORM would, because the
>> object instances need to be cached to preserve object identity. So as
>> you're result set is being read through, each unique object is stored.
>> iBATIS isn't quite as effcient as something like Hibernate can be with
>> circular references though, in that depending on how you map it out, you may
>> end up with multiple instances of the same data (parent/child relationships
>> mapped with resultMap).
>>
>> So with iBATIS, the most memory efficient approach is to use nested select
>> associations/collections. But for query performance, nested resultMaps are
>> ideal. I often find I need to use a combination of both to get the best
>> optimization. But if I often don't load lists of complete objects either.
>> If I'm loading a large list, I'll use a lighter weight representation, and
>> then only load the complete object graph for individual instances. I've
>> never found a case where this wasn't a good idea anyway. Even when working
>> with something like Rails, a rich domain ORM, I would often write optimized
>> lightweight queries for large lists of flatter objects.
>>
>> The memory should not be significantly more than will ultimately be
>> required to store your final result set. And any additional memory used
>> should be released upon the closing of the SqlSession.
>>
>> If the memory isn't being released at the end of the session, that's a
>> different story... but otherwise, this is normal behavior.
>>
>> Clinton
>>
>>
>> On Thu, Jan 14, 2010 at 11:20 AM, Dave Rafkind <dave.rafk...@gmail.com>wrote:
>>
>>> Thanks for the reply. What do you mean by nested result maps or selects?
>>> Do you mean collections or associations with their own selects and result
>>> maps? Why would ibatis exhibit this behavior in that case?
>>>
>>> And by flattening, you mean the same kind of stuff used to avoid the n+1
>>> select problem?
>>>
>>> On Thu, Jan 14, 2010 at 1:02 PM, Clinton Begin
>>> <clinton.be...@gmail.com>wrote:
>>>
>>>> If it uses nested result maps or nested selects, I'm afraid you're out
>>>> of luck. You'll need to reduce the query results, or flatten out the
>>>> results.
>>>>
>>>> Clinton
>>>>
>>>> On 2010-01-14, Dave Rafkind <dave.rafk...@gmail.com> wrote:
>>>> > Hi ibatis list, I'm new to ibatis so perhaps this is a noob question.
>>>> I'm
>>>> > using Ibatis 3 (ibatis-3-core-3.0.0.216.jar) with a somewhat
>>>> complicated
>>>> > schema (plenty of circular links etc).
>>>> >
>>>> > I'm doing something like this:
>>>> >
>>>> > List<MyIdObject> ids = session.selectList("getAll");
>>>> >
>>>> > for (id : ids) {
>>>> > MyObject o = session.select("getOne", id.getActualId());
>>>> > }
>>>> >
>>>> > The first query returns a list about 2k big, and the second query in
>>>> the for
>>>> > loop returns objects that are somewhat large (have several collections
>>>> in
>>>> > them, a discriminator, etc).
>>>> >
>>>> > The problem I have is that as the for loop marches on it uses an
>>>> > ever-increasing amount of memory. I would assuming that when the
>>>> objects in
>>>> > the body of the for loop go out of scope they can get garbage
>>>> collected, but
>>>> > apparently that never happens; is there some weird interaction with
>>>> the
>>>> > "first-level cache"? Should I be going about this a different way?
>>>> >
>>>> > Thanks!
>>>> > Dave
>>>> >
>>>>
>>>> --
>>>> Sent from my mobile device
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-java-unsubscr...@ibatis.apache.org
>>>> For additional commands, e-mail: user-java-h...@ibatis.apache.org
>>>>
>>>>
>>>
>>
>
