Agreed. The SLE architecture is layered for a reason: it gives you 3 different choices of where to perform a calculation: once per report, once per template, or once per (hit) tag. I'm pretty confident we can find the right place for Rich's expensive calculation.
-tk On Thu, Apr 5, 2018 at 7:38 AM, gjr80 <[email protected]> wrote: > Agree with the last para. > > There is probably a lot that can be done in terms of restructuring some of > my old SLEs given that some are 5 years old (I regularly cringe at some of > the code I used back then). Needs a clearer head than I have just now in > the wee hours. > > Getting back to Rich's issue, perhaps if you want to show us your SLE code > we might be able to give you a few pointers as to how it can be refactored > to operate more efficiently. > > Gary > > On Friday, 6 April 2018 00:27:51 UTC+10, Tom Keffer wrote: >> >> I think we're all on the same page here. >> >> The Standard report typically involves 14 file generations, so the >> function get_extension_list() will be invoked 14 times. If that's taking a >> long time, I would suggest trying to restructuring the SLE so the >> get_extension_list() invocation is not so expensive. For example, perhaps >> the parameters for the invocation could be stored in the object, but not >> actually do anything unless the tag is used (lazy evaluation). Or, if all >> 14 of those expensive invocations are identical, put it in __init__(), >> instead of get_extension_list(), then do it just once and store the results. >> >> Of course, if the expensive calculation needs to be done for every >> template and they are all different, then there's no way around it. But, I >> find it hard to imagine such a case. >> >> -tk >> >> On Thu, Apr 5, 2018 at 6:58 AM, gjr80 <[email protected]> wrote: >> >>> Tom, >>> >>> Perhaps I am not understanding you correctly, but I have run into this >>> issue a number of times with a processing intensive SLE. Whether or not any >>> of the tags created by the SLE are used, the SLE is run in its entirety for >>> each template listed in the skin. For example, I have a SLE that returns >>> some rain related tags and it takes about 1 second for get_extension_list() >>> to execute. I included this SLE in the search_list_extensions list in the >>> Standard skin.conf. None of the tags returned by the SLE are used in any >>> template in the Standard skin, yet during the report cycle the SLE is >>> invoked 15 times in total taking a little over 15 seconds in total. >>> >>> I can understand that the TimeBinder 'chain' is more efficient in that >>> time consuming tasks such as db queries only occur if the tag is 'hit', but >>> not every SLE uses the TimeBinder chain approach. I think that is the issue >>> Rich is seeing (and I have encountered in the past) >>> >>> Gary >>> >>> On Thursday, 5 April 2018 22:47:06 UTC+10, Tom Keffer wrote: >>>> >>>> Hello, Rich. >>>> >>>> I'm afraid I'm not quite understanding your situation. >>>> >>>> There are 3 different occasions when your SLE may come into play: >>>> >>>> 1. When the Cheetah report generator starts up. This is when your >>>> SLE gets instantiated, that is, its __init__ function, if any, gets >>>> invoked. This happens only once per report generation. >>>> 2. When a template gets generated. This is when >>>> get_extension_list() is invoked, which happens once per template. For >>>> the >>>> standard search list that comes with weeWX, this is when the top of the >>>> tag >>>> chain (class TimeBinder), described in How tags work >>>> <http://weewx.com/docs/customizing.htm#how_tags_work>, gets >>>> created, however, the chain itself does not actually get chased. >>>> 3. When looking for a tag. The Cheetah template engine will probe >>>> all the search list extensions looking for a specific tag. It does this >>>> by >>>> calling __getattr__ repeatedly with the tag name as an argument --- a >>>> cheap >>>> call. If it finds a hit, then this is typically when the expensive stuff >>>> starts happening. Note that this probing will happen *for every tag*, >>>> whether your tag or not. But, the rest of the tag chain gets invoked >>>> only >>>> if Cheetah finds a hit. For the search list that comes with weeWX >>>> nothing >>>> expensive happens until the very end of the tag chain. That's when the >>>> database lookups start happening. >>>> >>>> You say your expensive calculation is happening 14 times. If you have >>>> 14 of your tags sprinkled throughout the templates, then this is >>>> unavoidable: 14 requests for your tag, 14 calculations. However, if you >>>> expect all 14 tags to produce the same results (an example would be >>>> $station.altitude) then you should be doing your calculation farther up the >>>> stack described above. Perhaps you should be doing the calculation in >>>> get_extension_list() or even __init__. >>>> >>>> I hope this make sense. If not, give us some more specifics. >>>> >>>> Cheers, >>>> >>>> -tk >>>> >>>> >>>> On Wed, Apr 4, 2018 at 10:26 PM, gjr80 <[email protected]> wrote: >>>> >>>>> Hi Rich, >>>>> >>>>> Unfortunately the SLEs are applied to each report in a skin. Can you >>>>> pull out the template concerned into its own skin and use the SLE in that >>>>> skin only? >>>>> >>>>> Gary >>>>> >>>>> >>>>> On Thursday, 5 April 2018 14:59:13 UTC+10, Rich Altmaier wrote: >>>>>> >>>>>> I'm creating a Report extension which is fairly expensive to >>>>>> calculate. When I add my extension to skin.conf, with the usual >>>>>> search_list_extensions under CheetahGenerator, I find my extension is >>>>>> invoked about 14 times, which costs many seconds. Is it possible to >>>>>> make >>>>>> a Report extension apply to a selected file generation, such as in >>>>>> generating index.html only? >>>>>> Thank you! >>>>>> Rich >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
