On Tuesday, December 3, 2013 2:38:35 PM UTC-6, Jeremy Evans wrote:
>
> On Mon, Dec 2, 2013 at 9:00 PM, Nels Nelson wrote:
>
>> Thank you! I completely failed to find that in the docs. I'm sure it's
>> there, and I just missed it. It is exactly ideal, and it works perfectly.
>>
>
> Great!
>
Oops, I spoke too soon. So, the after_load method applies exclusively to
the artist node's albums association after the album association loads, and
is defined with a proc on the association definition. What I really need
is something that runs after the artist node loads, and then iterates over
the album association members for that node. Removing the
after_initializemethod and instead trying to use the
after_load method on the album association resulted only in my custom code
not getting invoked at all, resulting in the perceived performance
improvement.
[...] only the main model instance seems to get loaded into cache, and none
>> of its non-rcte tree associations -- that is, ideally I would prefer that
>> the non-child many-to-many associations get automatically loaded along with
>> the model instance.
>>
>
> I'm not exactly sure what you are describing (example code would help).
> Is it that the model instance is cached without any associations, or
> cached with some associations and not others (even though those
> associations were present in @associations at time of caching), or that in
> the uncached case @associations is not prepopulated with some associations
> that you want to be prepopulated.
>
>
>>
>> use Dataset#eager on the dataset that creates the objects, or the
>>> tactical_eager_loading plugin, in conjuction with Dataset#all to load the
>>> objects.
>>>
>>
>> Is it possible to do this automatically? [*] That is, upon retrieving the
>> children of a node, and dealing with an individual instance within that
>> set, is it possible to specify that all of that node instance's normal
>> many-to-many association members (non-rcte tree associations) get
>> implicitly included in the query which fetches those children?
>>
>
> Again, I'm not exactly sure what you are asking for here. If you want to
> eagerly load associations for associated objects, you can use the :eager
> option. Note that like usual eager loading, this is an additional query.
> Something like this after loading the rcte_tree plugin may help:
>
> one_to_many :children, :clone=>:children, :eager=>:other_assoc
>
> Note that if you use the tactical_eager_loading plugin, you shouldn't need
> to do this, as it will eager load implicitly in most cases. In either the
> :eager or tactical_eager_loading case, obj.children.each{|c| c.other_assoc}
> should be only 2 queries regardless of the number of children).
>
> The query you post does not make sense in rcte_tree context, since it
>>> appears to be using a join table, and the rcte_tree plugin does not.
>>>
>>
>> Yes. So, Artists could be considered to be nodes in the tree, and albums
>> are normal non-hierarchical many-to-many associations, joined through an
>> artist_to_album table and entity.
>>
>
> That makes sense.
>
Cool! I'll try to provide a simplified example.
class Artist < Sequel::Model
plugin :rcte_tree
plugin :caching, GloballyDefinedConcurrentHashMapInstance
plugin :tactical_eager_loading
many_to_many :albums,
:class => Album,
:join_table => :artists_to_albums,
:left_key => :artist_id,
:right_key => :album_id
def after_initialize
for influenced in self.children
for album in influenced.albums
something_special_with(album)
end
end
end
end
Many Artists, may be associated to many Albums. Artists are nodes in a
tree. Upon loading of an Artist node's children Artist nodes, I would like
each of the children Artist nodes to have included each of their respective
associated Albums so that they are available immediately, and without
additional SQL queries.
So when I do,
jimi = Artist.where(:name => 'Jimi Hendrix')
# Meanwhile, the after_initialize hook method is executed.
(For the sake of this example, please ignore the fact that in the real
world, artists may of course have more than one influencer.)
It would be nice if at most only two queries were executed here. Only one
query would be even better, but I can never keep straight whether or not
the children association can be loaded eagerly using rcte_tree -- let alone
loaded eagerly while also eagerly loaded with their respective albums
association members. I'm pretty sure the answer is no.
Furthermore, it would be even nicer if after_initialize only ran one time
ever, or until the model instances were marked as modified, or until the
cached instances were explicitly cleared. Unfortunately, the only instance
that winds up making it into the cache is the instance that gets assigned
to jimi. And of course, none of jimi's albums make it into the cache
either.
I hope this helps clear things up. I'm having trouble being as clear as
I'd like to be.
I did try using the suggested
one_to_many :children, :clone=>:children, :eager=>:albums
on the Artist model definition, but not only does it not seem to achieve
the desired performance improvements, it also causes some weird errors when
attempting to access a count of the cloned Artist.children association. I
can investigate more about that, if need be.
... a given node's branch is programmatically pre-order traversed, and
>> information retrieved. I may have to rework this, because it is bit of a
>> performance pain point for the application.
>>
>
> That sounds interesting. If you want to provide details, I may be able to
> give advice in that area.
>
The algorithm is an adaptation from another program, and it is rather
convoluted. Basically, it performs a pre-order traversal of a branch
starting from a given node, and then does some complex book-keeping along
the way, foregoing recursive pre-order traversals of certain nodes meeting
specific criteria.
The reason this routine is so slow is because some hundreds of queries and
after_initialize hooks get executed against the datasource backend, even if
there are only a couple dozen nodes in the branch.
I'm pretty certain that if the loading and caching problem with the example
above is solved, then this routine's performance will improve as well.
Furthermore, if I were able to leverage the descendants dataset of the
given branch node, and somehow managed to devise a filter based on the
algorithm's criteria, then I imagine this bottleneck would be solved
completely.
But this is a bit of a rabbit hole for now.
However, the associations with which I am concerned are not the children
>> associations. They are just the normal many-to-many associations on the
>> individual nodes.
>>
>
> OK. I think the standard association :eager option or the
> tactical_eager_loading plugin should help in those cases, modulo caching
> issues.
>
I am having difficulty finding example of how to apply the :eager option to
a many_to_many association definition. Does this involve defining and
specifying a proc? Or is it as simple as
many_to_many :albums, :eager => true
Thanks again for your time, Jeremy. I hope that you are not the only one
in this forum who ever replies to questions? :D
Cheers,
-Nels
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sequel-talk.
For more options, visit https://groups.google.com/groups/opt_out.
