Luke Cheers for that info, should prive useful...
Guess I could implement a method for each property that pulls back all that property values for all the volumes, and then match them up in prefetch. Cheers Gav On 29 Mar 2013 12:49, "Luke Kanies" <[email protected]> wrote: > Hi Gavin, > > Yeah, there isn't as much consistency in API as I would like. > > The way it should work, I believe, is something like: > > * The Type should support an 'instances' method that returns a complete > list of all instances of that type that exist on the host itself. This > method should not require a catalog. > > * The provider should support a 'prefetch' method, which accepts a list of > resources from the catalog and updates them with the correct provider > instance, with appropriate data already filled in to reflect the system > state. > > * Optionally, many providers implement (I think) their own 'instances' > method, which returns all instances and doesn't require a catalog, to > implement both of the above methods. > > I just looked at the Provider base class, though, and it's not at all > clear from this. Part of the problem is that the system is smart enough to > skip providers that don't support prefetch, and the way we denote 'doesn't > support prefetch' is that it doesn't have the method. Given that, I left > the method off of the base class. Because 'instances' isn't used for this > kind of test, the base class ships with a stub method that just throws an > error. > > So yeah, all that's confusing, and could really use some additional work > to make it easier to understand, and to bring some consistency. > > On Mar 29, 2013, at 12:37 PM, fatmcgav <[email protected]> wrote: > > Luke > > No worries. > > For the 2 I've converted soo far, it's made quite a performance > difference. Thankfully can pull back all the resources with one NetApp api > call, so it's drop the resource time from 5+ seconds to sub 1 second. > However was a bit confused initially, as different providers seem to use > different models... Some with just prefetch, some with just instances, some > with both... > > However I'm just about to start the more complex netapp_volume provider, > which has 4+ properties that have different api calls on top of the base > resource list call. This is by far the slowest provider out of them with an > average of 20+ seconds per run. > So not sure of the best way to handle as yet. > > Cheers > Gav > On 29 Mar 2013 12:11, "Luke Kanies" <[email protected]> wrote: > >> Hi Gavin, >> >> I'm glad to see you've sorted it out, sorry I didn't jump in before it >> was all resolved. >> >> How has prefetching worked out for you, in terms of performance? >> >> On Mar 28, 2013, at 3:28 PM, Gavin Williams <[email protected]> wrote: >> >> Afternoon >> >> Managed to find my issue.. Variable name re-use :( >> >> Defined 'qtrees' as an empty array on line 3, and then populated it with >> a whole load of device output on line 18 :( >> >> So have defined a 'qtree_instances' array on the outside to contain my >> output, with qtrees being used on the inside to hold the NetApp filer >> response... Latest code in >> Github<https://github.com/fatmcgav/fatmcgav-netapp/commit/1d51b1267466176db1a3d4e6ae32d9340a06fb56> >> . >> >> Cheers >> Gavin >> >> On Thursday, 28 March 2013 11:44:39 UTC, Gavin Williams wrote: >>> >>> Morning all >>> >>> Quick update... Looks like I managed to hack around the issue by adding >>> the following: >>> >>> ... >>> >>> ap qtree_info >>> >>> # Check if it is a NaElement >>> next unless qtree_info.respond_to?(:child_**get_string) >>> >>> # Pull out the qtree name. >>> name = qtree_info.child_get_string("**qtree") >>> ... >>> >>> >>> However this shows that I'm getting a total of 78 items processed, >>> whereas the original array only contains 53 items... >>> The additional items being processed are all like: '* >>> #<Puppet::Type::Netapp_qtree::ProviderNetapp_qtree:*'. >>> Have updated the gist with latest code and log file. >>> >>> Would like to understand where these are coming from, and if it's >>> something I'm doing incorrectly? >>> >>> In the mean-time, following fixing that bug, the provider now seems to >>> work as expected :) >>> >>> Cheers >>> Gavin >>> >>> >>> On Wednesday, 27 March 2013 17:32:23 UTC, Gavin Williams wrote: >>>> >>>> Afternoon all >>>> >>>> I've started working on converting a couple more of my NetApp network >>>> device providers to use a prefetch/flush model, as can see performance >>>> gains available, etc... >>>> >>>> Anyways, I'm having issues with my netapp_qtree provider. It would >>>> appear that somehow, an additional *Puppet::Type...* row is getting >>>> into an array and breaking things... >>>> >>>> Have created a gist here <https://gist.github.com/fatmcgav/5256240>with >>>> the details, as the log file is quite long. >>>> Also includes the *instances *and *prefetch* def's for my netapp_qtree >>>> provider... >>>> >>>> As you can see on Line 337 of the log, the array contains 40 items, >>>> however on line 734 *self.instances* is trying to process item 41?!?! >>>> What's also strange is that the item contents look like a Puppet Type ( >>>> *#<Puppet::Type::Netapp_qtree*)**, whereas all the others in the array >>>> are NetApp specific items (*#<NaElement:*). >>>> >>>> So, any ideas??? >>>> >>>> Cheers >>>> Gavin >>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> >> -- >> Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | >> +1-615-594-8199 >> Join us at PuppetConf 2013, August 22-23 in San Francisco - >> http://bit.ly/pupconf13 >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Puppet Developers" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en >> . >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/puppet-dev?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-dev?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > -- > Luke Kanies | http://about.me/lak | http://puppetlabs.com/ | > +1-615-594-8199 > Join us at PuppetConf 2013, August 22-23 in San Francisco - > http://bit.ly/pupconf13 > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Puppet Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/puppet-dev/74JW491YSAk/unsubscribe?hl=en > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/puppet-dev?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
