As already mentioned, I plan to take a look at this next week. I also thought we could share most of the info between loaders but Steve was thinking it might not be possible without massive changes.
I'll take a look and report back to all of you. I might be way over my head as I don't know the codebase very well but I already did a very similar work for HV with great results so it's worth a shot. -- Guillaume On Fri, May 4, 2018 at 10:31 PM, Gail Badner <gbad...@redhat.com> wrote: > I *think* that most, if not all, of the loaders for a particular entity > could be reworked to share most of the same information. > > IIRC, the only real differences are the batch size and the actual query > that gets generated. > > I'm happy to look into this, but I don't want to duplicate what others are > doing. It a bit difficult collaborating on this since our time zones don't > overlap very well (and, unfortunately, I'm not a morning person). > > I have some other work I need to attend to today. Please let me know if I > should dig into this next week. > > On Fri, May 4, 2018 at 12:30 PM, Guillaume Smet <guillaume.s...@gmail.com> > wrote: > >> Don't know about the history but with the entity I checked out in the >> OP's case, each EntityLoader is approximately 200 KB. >> >> The entity contains a lot of columns so that's no surprise. >> >> What doesn't help is that I don't think the 13 EntityLoaders/entity share >> a lot of things so they are just adding up. >> >> On Fri, May 4, 2018 at 9:15 PM, Sanne Grinovero <sa...@hibernate.org> >> wrote: >> >>> On 4 May 2018 at 19:45, Gail Badner <gbad...@redhat.com> wrote: >>> > I don't think the batching strategies actually changed when we moved >>> to load >>> > plans. >>> > >>> > Is the problem that each loader is taking considerably more memory? >>> >>> Yes that's right. >>> >>> > >>> > On Fri, May 4, 2018 at 5:41 AM, Guillaume Smet < >>> guillaume.s...@gmail.com> >>> > wrote: >>> >> >>> >> On Fri, May 4, 2018 at 11:48 AM, Sanne Grinovero <sa...@hibernate.org >>> > >>> >> wrote: >>> >> >>> >> > thanks for investigating. Steve in chat mentioned the option of >>> >> > creating these EntityLoaders lazily: as while they need to be >>> reused, >>> >> > it's possible many of them are never really used. >>> >> > >>> >> > I think that's a very interesting observation, is that what you >>> plan to >>> >> > explore? >>> >> > >>> >> >>> >> Yes, it's one of the things we should explore. >>> >> >>> >> >>> >> > I see two difficulties: >>> >> > - you'd need thredsafe code, yet we don't want barriers on this >>> area. >>> >> > Will probably need features such as lazy set, yet these would need >>> >> > even more objects allocated if you're not careful. >>> >> > - we always prefer fail-at-boot validation, so we'd need to make >>> sure >>> >> > even if you don't build them all, we can at least make sure we'll be >>> >> > able to build them later. >>> >> > >>> >> >>> >> Well, the idea is to prototype something on the memory usage front, >>> then >>> >> we >>> >> can discuss further what we want to do with it and if it's worth >>> refining >>> >> it and spending more time on it. >>> >> >>> >> If we end up choosing this path, be sure we'll be bugging you for >>> advice >>> >> on >>> >> concurrency and performance :). >>> >> >>> >> >>> >> > Additionally there's the idea to see if any such Loader instance >>> could >>> >> > be reused - probably using a cache which we clear at end of >>> bootstrap >>> >> > - yet since Steve mentioned this would be a significant change let's >>> >> > not try to push it in 5.3. >>> >> > >>> >> > Finally, a simple one: I have the impression that loader's Map could >>> >> > be made into a plain array, or a custom structure which would then >>> be >>> >> > ideal to support lazy creation of loaders. With the exception of two >>> >> > special strings - which will needt to be treated specially - all >>> other >>> >> > keys are enums. >>> >> > >>> >> >>> >> Yeah, I'll play with a few ideas and see what I can do. I'm not yet >>> sure >>> >> if >>> >> not knowing the code base will be a thing in my favor or not. At >>> least I >>> >> don't have preconceived ideas :]. >>> >> >>> >> >>> >> > Whatever you choose to try, let's keep significant changes for 6 >>> only :) >>> >> > >>> >> >>> >> Steve was saying that this whole thing wouldn't be an issue in 6. >>> >> >>> >> -- >>> >> Guillaume >>> >> _______________________________________________ >>> >> hibernate-dev mailing list >>> >> hibernate-dev@lists.jboss.org >>> >> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> > >>> > >>> _______________________________________________ >>> hibernate-dev mailing list >>> hibernate-dev@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/hibernate-dev >>> >> >> > _______________________________________________ hibernate-dev mailing list hibernate-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/hibernate-dev
