Well another option is to change the way batch load sql is created/handled which is something that has come up many times before, and someone was just discussing on the list last few weeks.
Given a batch size, Hibernate currently generates a non-unintuitive number of loaders. Lets look at an example number. You gave 16 as the batch size. That means Hibernate would generate loaders for sizes: 16,10,9,8,7,6,5,4,3,2,1 The idea being how to efficiently load batches when the number of things to load is less that 16. Deferring initialization of these batch loaders is one option, but that just really delays the memory consumption. Personally, if it were my app I'd rather that sizing be done up front rather than over the course of the next few hours or weeks as the system continues to run. Its been proposed a number of times to instead generate just a single loader for loading that number of items. This would mean generating a single SQL statement that has parameter holders for the full batch size and then somehow "filling out" the empty slots when the number of things to load is less that 16. If this could be made workable across all dialects, I personally think it would be the best approach as it probably gets close to the start up sizings you claimed but would never grow beyond that. But we would need to discuss this to see if it is a real possibility. Lets work an example with batch-size = 3. That would produce SQL like "select a.x, a.y, a.z from a a where a.x = ? or a.x = ? or a.x = ?" So for a case where we have 2 things to batch load, how do we best leverage that single statement? Which really comes down to how we handle the the 3rd parameter binding. I have seen simply reusing one of the keys as a proposal, but that will not work. Null should work, but would it work in all cases against all dialects? On 05/16/2012 03:14 PM, Andrej Golovnin wrote: > Hi Scott, > >> >> Nice find! Could you drill into one of the LockMode loaders and see what >> the larger objects referenced by that are? >> > > Largest objects are of type BatchingEntityLoader. > > The problem has to do with the value of hibernate.default_batch_fetch_size. > In our case we use 16 as the value. In this case Hibernate creates 11 > EntityLoaders > for every entity class involved in a collection association. So the most > memory > is consumed by generated SQL statements. If I compare memory snapshots, > I see ca. 100MB more of String objects in JBoss 7 as in JBoss 4. > > Changing the value of hibernate.default_batch_fetch_size from 16 to 4 > reduces the size of SessionFactoryImpl in our case from ca. 370MB to 180MB. > And if I use patched version of Hibernate as suggested by me, the size of > SessionFactoryImpl is ca. 80MB. So we will now investigate, wether it is > really needed to set hibernate.default_batch_fetch_size to 16 or maybe > the lower value would also give us decent performance. > > Best regards > Andrej Golovnin > _______________________________________________ > hibernate-dev mailing list > hibernate-dev@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/hibernate-dev -- st...@hibernate.org http://hibernate.org _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
