actually, thinking about it more, not so sure the "reusing keys" approach would not work.
On Wed 16 May 2012 04:35:36 PM CDT, Steve Ebersole wrote: > Well another option is to change the way batch load sql is > created/handled which is something that has come up many times before, > and someone was just discussing on the list last few weeks. > > Given a batch size, Hibernate currently generates a non-unintuitive > number of loaders. Lets look at an example number. You gave 16 as the > batch size. That means Hibernate would generate loaders for sizes: > 16,10,9,8,7,6,5,4,3,2,1 > > The idea being how to efficiently load batches when the number of > things to load is less that 16. > > Deferring initialization of these batch loaders is one option, but > that just really delays the memory consumption. Personally, if it were > my app I'd rather that sizing be done up front rather than over the > course of the next few hours or weeks as the system continues to run. > > Its been proposed a number of times to instead generate just a single > loader for loading that number of items. This would mean generating a > single SQL statement that has parameter holders for the full batch > size and then somehow "filling out" the empty slots when the number of > things to load is less that 16. If this could be made workable across > all dialects, I personally think it would be the best approach as it > probably gets close to the start up sizings you claimed but would > never grow beyond that. But we would need to discuss this to see if it > is a real possibility. Lets work an example with batch-size = 3. That > would produce SQL like "select a.x, a.y, a.z from a a where a.x = ? or > a.x = ? or a.x = ?" > > So for a case where we have 2 things to batch load, how do we best > leverage that single statement? Which really comes down to how we > handle the the 3rd parameter binding. > > I have seen simply reusing one of the keys as a proposal, but that > will not work. Null should work, but would it work in all cases > against all dialects? > > > On 05/16/2012 03:14 PM, Andrej Golovnin wrote: >> Hi Scott, >> >>> >>> Nice find! Could you drill into one of the LockMode loaders and see >>> what the larger objects referenced by that are? >>> >> >> Largest objects are of type BatchingEntityLoader. >> >> The problem has to do with the value of >> hibernate.default_batch_fetch_size. >> In our case we use 16 as the value. In this case Hibernate creates 11 >> EntityLoaders >> for every entity class involved in a collection association. So the >> most memory >> is consumed by generated SQL statements. If I compare memory snapshots, >> I see ca. 100MB more of String objects in JBoss 7 as in JBoss 4. >> >> Changing the value of hibernate.default_batch_fetch_size from 16 to 4 >> reduces the size of SessionFactoryImpl in our case from ca. 370MB to >> 180MB. >> And if I use patched version of Hibernate as suggested by me, the >> size of >> SessionFactoryImpl is ca. 80MB. So we will now investigate, wether it is >> really needed to set hibernate.default_batch_fetch_size to 16 or maybe >> the lower value would also give us decent performance. >> >> Best regards >> Andrej Golovnin >> _______________________________________________ >> hibernate-dev mailing list >> hibernate-dev@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/hibernate-dev > -- st...@hibernate.org http://hibernate.org _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
