Wow, you guys are giving a lot of ideas there, thanks so much! First to supply info: -I have only one tag usage presently, in the index.html.tmpl file. -the cost comes from DB queries, for which I wanted the db_lookup class, which is passed to get_extension_list. -I am creating the TimespanBinder, to cover a rainstorm period. Thereby allowing various stats over the storm period, obviously $last_rain_storm.rain.sum, but could be other interesting things like $last_rain_storm.outTemp.max, etc. So I followed the example in setting the calculated TimespanBinder into a variable which is added to the dictionary returned by get_extension_list.
Some clarifying questions: -I didn't understand __init__, so that looks useful, but how can I obtain the db_lookup method there? -mentioning "once per tag" or do code when tag is used, what is the technique to detect or run on tag usage, to calculate the TimespanBinder? I can also fall back to saving the complex results in a temp DB and reusing without recalculating. Thanks! Rich On Thursday, April 5, 2018 at 7:44:59 AM UTC-7, Tom Keffer wrote: > > Agreed. The SLE architecture is layered for a reason: it gives you 3 > different choices of where to perform a calculation: once per report, once > per template, or once per (hit) tag. I'm pretty confident we can find the > right place for Rich's expensive calculation. > > -tk > > On Thu, Apr 5, 2018 at 7:38 AM, gjr80 <[email protected] <javascript:>> > wrote: > >> Agree with the last para. >> >> There is probably a lot that can be done in terms of restructuring some >> of my old SLEs given that some are 5 years old (I regularly cringe at some >> of the code I used back then). Needs a clearer head than I have just now in >> the wee hours. >> >> Getting back to Rich's issue, perhaps if you want to show us your SLE >> code we might be able to give you a few pointers as to how it can be >> refactored to operate more efficiently. >> >> Gary >> >> On Friday, 6 April 2018 00:27:51 UTC+10, Tom Keffer wrote: >>> >>> I think we're all on the same page here. >>> >>> The Standard report typically involves 14 file generations, so the >>> function get_extension_list() will be invoked 14 times. If that's taking a >>> long time, I would suggest trying to restructuring the SLE so the >>> get_extension_list() invocation is not so expensive. For example, perhaps >>> the parameters for the invocation could be stored in the object, but not >>> actually do anything unless the tag is used (lazy evaluation). Or, if all >>> 14 of those expensive invocations are identical, put it in __init__(), >>> instead of get_extension_list(), then do it just once and store the results. >>> >>> Of course, if the expensive calculation needs to be done for every >>> template and they are all different, then there's no way around it. But, I >>> find it hard to imagine such a case. >>> >>> -tk >>> >>> On Thu, Apr 5, 2018 at 6:58 AM, gjr80 <[email protected]> wrote: >>> >>>> Tom, >>>> >>>> Perhaps I am not understanding you correctly, but I have run into this >>>> issue a number of times with a processing intensive SLE. Whether or not >>>> any >>>> of the tags created by the SLE are used, the SLE is run in its entirety >>>> for >>>> each template listed in the skin. For example, I have a SLE that returns >>>> some rain related tags and it takes about 1 second for >>>> get_extension_list() >>>> to execute. I included this SLE in the search_list_extensions list in the >>>> Standard skin.conf. None of the tags returned by the SLE are used in any >>>> template in the Standard skin, yet during the report cycle the SLE is >>>> invoked 15 times in total taking a little over 15 seconds in total. >>>> >>>> I can understand that the TimeBinder 'chain' is more efficient in that >>>> time consuming tasks such as db queries only occur if the tag is 'hit', >>>> but >>>> not every SLE uses the TimeBinder chain approach. I think that is the >>>> issue >>>> Rich is seeing (and I have encountered in the past) >>>> >>>> Gary >>>> >>>> On Thursday, 5 April 2018 22:47:06 UTC+10, Tom Keffer wrote: >>>>> >>>>> Hello, Rich. >>>>> >>>>> I'm afraid I'm not quite understanding your situation. >>>>> >>>>> There are 3 different occasions when your SLE may come into play: >>>>> >>>>> 1. When the Cheetah report generator starts up. This is when your >>>>> SLE gets instantiated, that is, its __init__ function, if any, gets >>>>> invoked. This happens only once per report generation. >>>>> 2. When a template gets generated. This is when >>>>> get_extension_list() is invoked, which happens once per template. For >>>>> the >>>>> standard search list that comes with weeWX, this is when the top of >>>>> the tag >>>>> chain (class TimeBinder), described in How tags work >>>>> <http://weewx.com/docs/customizing.htm#how_tags_work>, gets >>>>> created, however, the chain itself does not actually get chased. >>>>> 3. When looking for a tag. The Cheetah template engine will probe >>>>> all the search list extensions looking for a specific tag. It does >>>>> this by >>>>> calling __getattr__ repeatedly with the tag name as an argument --- a >>>>> cheap >>>>> call. If it finds a hit, then this is typically when the expensive >>>>> stuff >>>>> starts happening. Note that this probing will happen *for every >>>>> tag*, whether your tag or not. But, the rest of the tag chain gets >>>>> invoked only if Cheetah finds a hit. For the search list that comes >>>>> with >>>>> weeWX nothing expensive happens until the very end of the tag chain. >>>>> That's >>>>> when the database lookups start happening. >>>>> >>>>> You say your expensive calculation is happening 14 times. If you have >>>>> 14 of your tags sprinkled throughout the templates, then this is >>>>> unavoidable: 14 requests for your tag, 14 calculations. However, if you >>>>> expect all 14 tags to produce the same results (an example would be >>>>> $station.altitude) then you should be doing your calculation farther up >>>>> the >>>>> stack described above. Perhaps you should be doing the calculation in >>>>> get_extension_list() or even __init__. >>>>> >>>>> I hope this make sense. If not, give us some more specifics. >>>>> >>>>> Cheers, >>>>> >>>>> -tk >>>>> >>>>> >>>>> On Wed, Apr 4, 2018 at 10:26 PM, gjr80 <[email protected]> wrote: >>>>> >>>>>> Hi Rich, >>>>>> >>>>>> Unfortunately the SLEs are applied to each report in a skin. Can you >>>>>> pull out the template concerned into its own skin and use the SLE in >>>>>> that >>>>>> skin only? >>>>>> >>>>>> Gary >>>>>> >>>>>> >>>>>> On Thursday, 5 April 2018 14:59:13 UTC+10, Rich Altmaier wrote: >>>>>>> >>>>>>> I'm creating a Report extension which is fairly expensive to >>>>>>> calculate. When I add my extension to skin.conf, with the usual >>>>>>> search_list_extensions under CheetahGenerator, I find my extension is >>>>>>> invoked about 14 times, which costs many seconds. Is it possible to >>>>>>> make >>>>>>> a Report extension apply to a selected file generation, such as in >>>>>>> generating index.html only? >>>>>>> Thank you! >>>>>>> Rich >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
