Soren, Can you share the data set, or a dummy one for us to look at performance improvements?
I presume you have reviewed you filters in respect of their performance here <https://tiddlywiki.com/#Performance>? Tones On Friday, 23 July 2021 at 10:17:24 UTC+10 Soren Bjornstad wrote: > Tones, thanks for putting this together, but your solution is slower than > mine as far as I can tell (no easy way to instrument that since it has a > very different number of filters, but the difference is apparent from > clicking around a bit). > > Saq, I can't get your filter to work and I can't see how it would work in > theory either now that I look at it – the !!condition in the subfilter is > getting the condition field on the tiddler in which the tab list is being > rendered, whereas it needs to be the *tab* tiddler that's coming in on > the input of the subfilter operator…right? Or am I just being dense? In my > :reduce version it has a different meaning because :reduce actually > resets the currentTiddler variable to match the input value currently being > processed <https://tiddlywiki.com/#reduce%20Operator>. > > With the additional optimization work I did the other night, including > optimizing the filters on each tab as Saq pointed out, I would say it's now > performing acceptably with my current version, if only barely on a slower > computer. So if nobody has any other ideas, I think I can leave it where it > is for now. > > > On Wednesday, July 21, 2021 at 8:58:05 AM UTC-5 TW Tones wrote: > >> Oops; >> >> I must add >> the tab template contains >> >> <$tiddler tiddler=<<currentTab>>> >> <$transclude mode="block" /> >> </$tiddler> >> >> and each tab contains >> <$list filter={{!!filter}}> >> >> </$list> >> >> The tabs work stand alone >> >> Tones >> >> On Wednesday, 21 July 2021 at 23:55:11 UTC+10 TW Tones wrote: >> >>> Soren, >>> >>> Given this thread, I am not sure if I am on track here, but reading your >>> Original post it seems so. >>> >>> As I understand it you want to use a filter in a tab tiddler to >>> determine if it should have the tab displayed. >>> >>> The thing is the tabs macro is not designed for conditional tab display, >>> it simply wants a list of tiddlers. If this could all be rammed into a >>> single filter then all would be fine. However you not only want to make use >>> of another variable or in this case a field but that field contains a >>> filter. This is asking too much of the filter syntax. >>> >>> What I have done in my solution was to generate the list of tiddlers >>> that would be tabs according to you conditions and feed only the result to >>> the tabs macro. >>> >>> - In my small test not performance issues, and I do not expect their >>> to be >>> - I think what is happening in your case if the reduce etc... is >>> adding unnecessary complexity to the performance of the tabs macro. >>> >>> >>> Here is the core logic tested on tiddlywiki.com and pre-release >>> incidentally. >>> >>> \define tabs-with-content() >>> <$list filter="[tag[Tag]]" > >>> <$list filter="[all[current]get[filter]]" variable=found-filter> >>> <$list >>> filter="[subfilter<found-filter>count[]!match[0]then<currentTiddler>]"> >>> >>> </$list> >>> </$list> >>> </$list> >>> \end >>> >>> <$wikify name=tabs-with-content text="<<tabs-with-content>>"> >>> <$macrocall $name=tabs tabsList=<<tabs-with-content>> >>> template="tabtemplate"/> >>> </$wikify> >>> >>> If this does not make sense I will package the tiddlers in this test for >>> you. >>> >>> The above example may be a good test case if we wanted to invent a >>> method to make filters more sophisticated basically permit ones that allow >>> you to avoid the nested lists I used, however as stated previously dont be >>> shy using nested lists, because each level of nesting allows to generate a >>> new variable and keep the old ones in this case found-filter while keeping >>> currentTiddler from the first filter. >>> >>> My preference would be a shorthand method when presenting variables as a >>> parameter, in this case <<tabs-with-content>> such that it is first >>> wikified rather than using the ungainly wikify widget and its need to wrap >>> the macro call. Eg a a parameter ((tabs-with-content)) or in filters >>> (tabs-with-content) >>> Alternatively TiddlyWiki would detect the need for wikification of that >>> variable/macro and do it itself. >>> >>> >>> Regards >>> tones >>> >>> >>> On Wednesday, 21 July 2021 at 22:15:12 UTC+10 Soren Bjornstad wrote: >>> >>>> Saq, the intermittent refresh every 5 seconds updates a clock and >>>> Pomodoro timer. The reason it's puzzling is that it doesn't appear by >>>> default in the public edition. If you're curious, tick the "clock" >>>> checkbox >>>> in the FeatureFlags tiddler. >>>> >>>> I'll give the :filter version a try later. I suspect I didn't think to >>>> use filter and subfilter together in one run! >>>> >>>> On Wednesday, July 21, 2021 at 5:57:42 AM UTC-5 [email protected] >>>> wrote: >>>> >>>>> Hi Soren, glad that was helpful. When I worked on performance >>>>> improvements for Streams a few months ago, the biggest gains were from >>>>> simplifying the widget and DOM structure, and replacing macros with >>>>> transclusions, rather than from optimizing filters. >>>>> >>>>> I took a brief look at your wiki just now and the one odd thing I see >>>>> is that there seems to be an intermittent refresh being triggered every >>>>> few >>>>> seconds. Any idea what that is about? To see what I mean, create a new >>>>> tiddler with the following in the text and then observe the developer >>>>> console: >>>>> <$log t=<<currentTiddler>>/> >>>>> >>>>> This should work on the current pre-release but probably isn't any >>>>> faster: >>>>> [tag[$:/sib/refexplorer/tabs]] :filter[subfilter{!!condition}] +[join[ >>>>> ]] >>>>> >>>>> I suspect that any real gains would come from optimizing the condition >>>>> filters, as they likely make up the bulk of the time needed for the above >>>>> filter expression. For filter operators that are indexed, the first time >>>>> they run they will take a while but subsequent runs should be faster. >>>>> https://tiddlywiki.com/#Performance >>>>> >>>>> On Wednesday, July 21, 2021 at 4:20:13 AM UTC+2 Soren Bjornstad wrote: >>>>> >>>>>> Thanks Saq, that view is super useful and I never knew it existed! >>>>>> >>>>>> Armed with that information, I spent most of this evening tinkering >>>>>> on improving some of the other things that have been slowing my wiki >>>>>> down, >>>>>> including adding an Idea tag so I don't have to define it as the >>>>>> negation >>>>>> of a dozen other tags and replacing TiddlyTables with a similar HTML >>>>>> table >>>>>> that needs fewer filters, macros, and transclusions in the most critical >>>>>> paths. (More work needs to be done to bring the rest of the Reference >>>>>> Explorer in line and refactor it to catch it up with a bunch of other >>>>>> changes I've made recently.) >>>>>> >>>>>> The tabs filter discussed above was and remains the #1 contributor to >>>>>> filter processing time when clicking around the wiki, often something >>>>>> like >>>>>> 30% of the total time, so I'm still interested in suggestions others may >>>>>> have for improving that. I didn't want to publish the wiki in its >>>>>> present >>>>>> state previously since it was so slow it would be difficult for people >>>>>> to >>>>>> use, but I've ameliorated that enough that I am now comfortable >>>>>> republishing it. As mentioned in my previous post, you can start looking >>>>>> at >>>>>> $:/sib/refexplorer/ReferenceExplorer, but now it is a working example >>>>>> rather than something you would have to tweak into a working example by >>>>>> yourself. >>>>>> >>>>>> On Tuesday, July 20, 2021 at 9:29:49 AM UTC-5 [email protected] >>>>>> wrote: >>>>>> >>>>>>> @Soren I recommend enabling performance instrumentation and checking >>>>>>> to see which filter runs are the culprits. >>>>>>> >>>>>>> >>>>>>> https://tiddlywiki.com/#%24%3A%2Fconfig%2FPerformance%2FInstrumentation:%24%3A%2Fconfig%2FPerformance%2FInstrumentation%20%5B%5BPerformance%20Instrumentation%5D%5D >>>>>>> >>>>>>> Note that you need to enable it by setting the config tiddler to >>>>>>> "yes", then save and reload. >>>>>>> >>>>>>> Also suggest posting a public wiki where the issue (and the >>>>>>> condition filters) can be seen to facilitate debugging. >>>>>>> On Tuesday, July 20, 2021 at 4:05:15 PM UTC+2 Soren Bjornstad wrote: >>>>>>> >>>>>>>> Important note, the condition fields aren't defined in the current >>>>>>>> public version, so you'd have to add them in to get a working test. >>>>>>>> >>>>>>>> On Tuesday, July 20, 2021 at 9:03:42 AM UTC-5 Soren Bjornstad wrote: >>>>>>>> >>>>>>>>> Tones, >>>>>>>>> >>>>>>>>> :filter was my first thought, but I couldn't figure out how the >>>>>>>>> data would flow through it. Perhaps I was missing something, looking >>>>>>>>> forward to seeing your version. >>>>>>>>> >>>>>>>>> If it helps to see the context, have a peek at >>>>>>>>> https://zettelkasten.sorenbjornstad.com/#%24%3A%2Fsib%2Frefexplorer%2FReferenceExplorer. >>>>>>>>> >>>>>>>>> The snippet above (or a replacement) would go inside the ref-explorer >>>>>>>>> macro >>>>>>>>> definition. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tuesday, July 20, 2021 at 8:07:09 AM UTC-5 TW Tones wrote: >>>>>>>>> >>>>>>>>>> Soren, >>>>>>>>>> >>>>>>>>>> Its late here but I have done something similar in the past >>>>>>>>>> without a performance hit, and will try and create a solution >>>>>>>>>> tomorrow, >>>>>>>>>> However I think the answer best answer may be through the use of a >>>>>>>>>> filter >>>>>>>>>> run as in 5.3.23+ however I am sure I succeeded in something similar >>>>>>>>>> a few >>>>>>>>>> versions ago. >>>>>>>>>> >>>>>>>>>> No need for reduce and accumulators I think. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Tones >>>>>>>>>> >>>>>>>>>> On Tuesday, 20 July 2021 at 13:09:18 UTC+10 Soren Bjornstad wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I want to determine what tabs of the Reference Explorer in my >>>>>>>>>>> Zettelkasten to show on a template, such that if there aren't any >>>>>>>>>>> results >>>>>>>>>>> on that tab, the tab doesn't appear at all. The results of the tab >>>>>>>>>>> are >>>>>>>>>>> produced based on a filter (of course), so I figured I would also >>>>>>>>>>> determine >>>>>>>>>>> whether the tab appears by running a filter. The filter is stored >>>>>>>>>>> in a >>>>>>>>>>> field in the tab tiddler. >>>>>>>>>>> >>>>>>>>>>> That is, I have a series of tiddlers with a certain tag (say >>>>>>>>>>> *Tab*), and each of these tiddlers contains a filter in some >>>>>>>>>>> field (say *condition*). For each Tiddler tagged Tab, if and >>>>>>>>>>> only if the filter Tiddler!!condition, run with the current tiddler >>>>>>>>>>> as >>>>>>>>>>> input, has more than zero results, I want to display the tab. >>>>>>>>>>> >>>>>>>>>>> I came up with the following: >>>>>>>>>>> >>>>>>>>>>> <$set name="tabList" value={{{ [tag[Tab]] >>>>>>>>>>> :reduce[<storyTiddler>subfilter{!!condition}then<currentTiddler>addprefix[ >>>>>>>>>>> >>>>>>>>>>> ]addprefix<accumulator>] }}}> >>>>>>>>>>> <$macrocall $name="tabs" tabsList=<<tabList>>/> >>>>>>>>>>> </$set> >>>>>>>>>>> >>>>>>>>>>> This produces the correct result (well, as long as there are no >>>>>>>>>>> spaces in the titles of the tiddlers tagged *Tab*; I'm OK >>>>>>>>>>> assuming that since there indeed aren't any). The problem is that >>>>>>>>>>> it is >>>>>>>>>>> horrendously slow to run all these filters. On my dev machine it is >>>>>>>>>>> tolerable, but this is a machine specced for serious processing >>>>>>>>>>> power. On >>>>>>>>>>> my MacBook Air it now takes 1–2 seconds to open a new tiddler, even >>>>>>>>>>> without >>>>>>>>>>> anything currently open! >>>>>>>>>>> >>>>>>>>>>> Probably I am just asking TW to do too much on the fly here, but >>>>>>>>>>> before I start rethinking the project too hard, can anyone think of >>>>>>>>>>> obvious >>>>>>>>>>> optimizations I might be missing here? The filters involved are >>>>>>>>>>> moderately >>>>>>>>>>> complex (the basic pattern for each is to gather together links[], >>>>>>>>>>> backlinks[], and tagging[] for the story tiddler, then filter some >>>>>>>>>>> things >>>>>>>>>>> out of that using + and !*operator*[]'s). >>>>>>>>>>> >>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c9a172e1-fcd9-4fe3-8098-3a41976be865n%40googlegroups.com.
