Tones, Here's the reference explorer tiddler, as mentioned earlier in the thread: https://zettelkasten.sorenbjornstad.com/#%24%3A%2Fsib%2Frefexplorer%2FReferenceExplorer
Most of the data set is public in the same wiki…if you just click around in some random non-system tiddlers, they all have the Ideas section of the Reference Explorer showing. The public version may perform just a little better than my private one since it has about a third fewer tiddlers, but won't be too far off. I had not remembered to look at the #Performance tiddler, but I don't believe any of the suggestions in there are applicable here. On Friday, July 23, 2021 at 12:18:06 AM UTC-5 TW Tones wrote: > Soren, > > Can you share the data set, or a dummy one for us to look at performance > improvements? > > I presume you have reviewed you filters in respect of their performance > here <https://tiddlywiki.com/#Performance>? > Tones > > On Friday, 23 July 2021 at 10:17:24 UTC+10 Soren Bjornstad wrote: > >> Tones, thanks for putting this together, but your solution is slower than >> mine as far as I can tell (no easy way to instrument that since it has a >> very different number of filters, but the difference is apparent from >> clicking around a bit). >> >> Saq, I can't get your filter to work and I can't see how it would work in >> theory either now that I look at it – the !!condition in the subfilter is >> getting the condition field on the tiddler in which the tab list is being >> rendered, whereas it needs to be the *tab* tiddler that's coming in on >> the input of the subfilter operator…right? Or am I just being dense? In my >> :reduce version it has a different meaning because :reduce actually >> resets the currentTiddler variable to match the input value currently being >> processed <https://tiddlywiki.com/#reduce%20Operator>. >> >> With the additional optimization work I did the other night, including >> optimizing the filters on each tab as Saq pointed out, I would say it's now >> performing acceptably with my current version, if only barely on a slower >> computer. So if nobody has any other ideas, I think I can leave it where it >> is for now. >> >> >> On Wednesday, July 21, 2021 at 8:58:05 AM UTC-5 TW Tones wrote: >> >>> Oops; >>> >>> I must add >>> the tab template contains >>> >>> <$tiddler tiddler=<<currentTab>>> >>> <$transclude mode="block" /> >>> </$tiddler> >>> >>> and each tab contains >>> <$list filter={{!!filter}}> >>> >>> </$list> >>> >>> The tabs work stand alone >>> >>> Tones >>> >>> On Wednesday, 21 July 2021 at 23:55:11 UTC+10 TW Tones wrote: >>> >>>> Soren, >>>> >>>> Given this thread, I am not sure if I am on track here, but reading >>>> your Original post it seems so. >>>> >>>> As I understand it you want to use a filter in a tab tiddler to >>>> determine if it should have the tab displayed. >>>> >>>> The thing is the tabs macro is not designed for conditional tab >>>> display, it simply wants a list of tiddlers. If this could all be rammed >>>> into a single filter then all would be fine. However you not only want to >>>> make use of another variable or in this case a field but that field >>>> contains a filter. This is asking too much of the filter syntax. >>>> >>>> What I have done in my solution was to generate the list of tiddlers >>>> that would be tabs according to you conditions and feed only the result to >>>> the tabs macro. >>>> >>>> - In my small test not performance issues, and I do not expect >>>> their to be >>>> - I think what is happening in your case if the reduce etc... is >>>> adding unnecessary complexity to the performance of the tabs macro. >>>> >>>> >>>> Here is the core logic tested on tiddlywiki.com and pre-release >>>> incidentally. >>>> >>>> \define tabs-with-content() >>>> <$list filter="[tag[Tag]]" > >>>> <$list filter="[all[current]get[filter]]" variable=found-filter> >>>> <$list >>>> filter="[subfilter<found-filter>count[]!match[0]then<currentTiddler>]"> >>>> >>>> </$list> >>>> </$list> >>>> </$list> >>>> \end >>>> >>>> <$wikify name=tabs-with-content text="<<tabs-with-content>>"> >>>> <$macrocall $name=tabs tabsList=<<tabs-with-content>> >>>> template="tabtemplate"/> >>>> </$wikify> >>>> >>>> If this does not make sense I will package the tiddlers in this test >>>> for you. >>>> >>>> The above example may be a good test case if we wanted to invent a >>>> method to make filters more sophisticated basically permit ones that allow >>>> you to avoid the nested lists I used, however as stated previously dont be >>>> shy using nested lists, because each level of nesting allows to generate a >>>> new variable and keep the old ones in this case found-filter while keeping >>>> currentTiddler from the first filter. >>>> >>>> My preference would be a shorthand method when presenting variables as >>>> a parameter, in this case <<tabs-with-content>> such that it is first >>>> wikified rather than using the ungainly wikify widget and its need to wrap >>>> the macro call. Eg a a parameter ((tabs-with-content)) or in filters >>>> (tabs-with-content) >>>> Alternatively TiddlyWiki would detect the need for wikification of that >>>> variable/macro and do it itself. >>>> >>>> >>>> Regards >>>> tones >>>> >>>> >>>> On Wednesday, 21 July 2021 at 22:15:12 UTC+10 Soren Bjornstad wrote: >>>> >>>>> Saq, the intermittent refresh every 5 seconds updates a clock and >>>>> Pomodoro timer. The reason it's puzzling is that it doesn't appear by >>>>> default in the public edition. If you're curious, tick the "clock" >>>>> checkbox >>>>> in the FeatureFlags tiddler. >>>>> >>>>> I'll give the :filter version a try later. I suspect I didn't think to >>>>> use filter and subfilter together in one run! >>>>> >>>>> On Wednesday, July 21, 2021 at 5:57:42 AM UTC-5 [email protected] >>>>> wrote: >>>>> >>>>>> Hi Soren, glad that was helpful. When I worked on performance >>>>>> improvements for Streams a few months ago, the biggest gains were from >>>>>> simplifying the widget and DOM structure, and replacing macros with >>>>>> transclusions, rather than from optimizing filters. >>>>>> >>>>>> I took a brief look at your wiki just now and the one odd thing I see >>>>>> is that there seems to be an intermittent refresh being triggered every >>>>>> few >>>>>> seconds. Any idea what that is about? To see what I mean, create a new >>>>>> tiddler with the following in the text and then observe the developer >>>>>> console: >>>>>> <$log t=<<currentTiddler>>/> >>>>>> >>>>>> This should work on the current pre-release but probably isn't any >>>>>> faster: >>>>>> [tag[$:/sib/refexplorer/tabs]] :filter[subfilter{!!condition}] >>>>>> +[join[ ]] >>>>>> >>>>>> I suspect that any real gains would come from optimizing the >>>>>> condition filters, as they likely make up the bulk of the time needed >>>>>> for >>>>>> the above filter expression. For filter operators that are indexed, the >>>>>> first time they run they will take a while but subsequent runs should be >>>>>> faster. >>>>>> https://tiddlywiki.com/#Performance >>>>>> >>>>>> On Wednesday, July 21, 2021 at 4:20:13 AM UTC+2 Soren Bjornstad wrote: >>>>>> >>>>>>> Thanks Saq, that view is super useful and I never knew it existed! >>>>>>> >>>>>>> Armed with that information, I spent most of this evening tinkering >>>>>>> on improving some of the other things that have been slowing my wiki >>>>>>> down, >>>>>>> including adding an Idea tag so I don't have to define it as the >>>>>>> negation >>>>>>> of a dozen other tags and replacing TiddlyTables with a similar HTML >>>>>>> table >>>>>>> that needs fewer filters, macros, and transclusions in the most >>>>>>> critical >>>>>>> paths. (More work needs to be done to bring the rest of the Reference >>>>>>> Explorer in line and refactor it to catch it up with a bunch of other >>>>>>> changes I've made recently.) >>>>>>> >>>>>>> The tabs filter discussed above was and remains the #1 contributor >>>>>>> to filter processing time when clicking around the wiki, often >>>>>>> something >>>>>>> like 30% of the total time, so I'm still interested in suggestions >>>>>>> others >>>>>>> may have for improving that. I didn't want to publish the wiki in its >>>>>>> present state previously since it was so slow it would be difficult for >>>>>>> people to use, but I've ameliorated that enough that I am now >>>>>>> comfortable >>>>>>> republishing it. As mentioned in my previous post, you can start >>>>>>> looking at >>>>>>> $:/sib/refexplorer/ReferenceExplorer, but now it is a working example >>>>>>> rather than something you would have to tweak into a working example by >>>>>>> yourself. >>>>>>> >>>>>>> On Tuesday, July 20, 2021 at 9:29:49 AM UTC-5 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> @Soren I recommend enabling performance instrumentation and >>>>>>>> checking to see which filter runs are the culprits. >>>>>>>> >>>>>>>> >>>>>>>> https://tiddlywiki.com/#%24%3A%2Fconfig%2FPerformance%2FInstrumentation:%24%3A%2Fconfig%2FPerformance%2FInstrumentation%20%5B%5BPerformance%20Instrumentation%5D%5D >>>>>>>> >>>>>>>> Note that you need to enable it by setting the config tiddler to >>>>>>>> "yes", then save and reload. >>>>>>>> >>>>>>>> Also suggest posting a public wiki where the issue (and the >>>>>>>> condition filters) can be seen to facilitate debugging. >>>>>>>> On Tuesday, July 20, 2021 at 4:05:15 PM UTC+2 Soren Bjornstad wrote: >>>>>>>> >>>>>>>>> Important note, the condition fields aren't defined in the current >>>>>>>>> public version, so you'd have to add them in to get a working test. >>>>>>>>> >>>>>>>>> On Tuesday, July 20, 2021 at 9:03:42 AM UTC-5 Soren Bjornstad >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Tones, >>>>>>>>>> >>>>>>>>>> :filter was my first thought, but I couldn't figure out how the >>>>>>>>>> data would flow through it. Perhaps I was missing something, looking >>>>>>>>>> forward to seeing your version. >>>>>>>>>> >>>>>>>>>> If it helps to see the context, have a peek at >>>>>>>>>> https://zettelkasten.sorenbjornstad.com/#%24%3A%2Fsib%2Frefexplorer%2FReferenceExplorer. >>>>>>>>>> >>>>>>>>>> The snippet above (or a replacement) would go inside the >>>>>>>>>> ref-explorer macro >>>>>>>>>> definition. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, July 20, 2021 at 8:07:09 AM UTC-5 TW Tones wrote: >>>>>>>>>> >>>>>>>>>>> Soren, >>>>>>>>>>> >>>>>>>>>>> Its late here but I have done something similar in the past >>>>>>>>>>> without a performance hit, and will try and create a solution >>>>>>>>>>> tomorrow, >>>>>>>>>>> However I think the answer best answer may be through the use of a >>>>>>>>>>> filter >>>>>>>>>>> run as in 5.3.23+ however I am sure I succeeded in something >>>>>>>>>>> similar a few >>>>>>>>>>> versions ago. >>>>>>>>>>> >>>>>>>>>>> No need for reduce and accumulators I think. >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> Tones >>>>>>>>>>> >>>>>>>>>>> On Tuesday, 20 July 2021 at 13:09:18 UTC+10 Soren Bjornstad >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I want to determine what tabs of the Reference Explorer in my >>>>>>>>>>>> Zettelkasten to show on a template, such that if there aren't any >>>>>>>>>>>> results >>>>>>>>>>>> on that tab, the tab doesn't appear at all. The results of the tab >>>>>>>>>>>> are >>>>>>>>>>>> produced based on a filter (of course), so I figured I would also >>>>>>>>>>>> determine >>>>>>>>>>>> whether the tab appears by running a filter. The filter is stored >>>>>>>>>>>> in a >>>>>>>>>>>> field in the tab tiddler. >>>>>>>>>>>> >>>>>>>>>>>> That is, I have a series of tiddlers with a certain tag (say >>>>>>>>>>>> *Tab*), and each of these tiddlers contains a filter in some >>>>>>>>>>>> field (say *condition*). For each Tiddler tagged Tab, if and >>>>>>>>>>>> only if the filter Tiddler!!condition, run with the current >>>>>>>>>>>> tiddler as >>>>>>>>>>>> input, has more than zero results, I want to display the tab. >>>>>>>>>>>> >>>>>>>>>>>> I came up with the following: >>>>>>>>>>>> >>>>>>>>>>>> <$set name="tabList" value={{{ [tag[Tab]] >>>>>>>>>>>> :reduce[<storyTiddler>subfilter{!!condition}then<currentTiddler>addprefix[ >>>>>>>>>>>> >>>>>>>>>>>> ]addprefix<accumulator>] }}}> >>>>>>>>>>>> <$macrocall $name="tabs" tabsList=<<tabList>>/> >>>>>>>>>>>> </$set> >>>>>>>>>>>> >>>>>>>>>>>> This produces the correct result (well, as long as there are no >>>>>>>>>>>> spaces in the titles of the tiddlers tagged *Tab*; I'm OK >>>>>>>>>>>> assuming that since there indeed aren't any). The problem is that >>>>>>>>>>>> it is >>>>>>>>>>>> horrendously slow to run all these filters. On my dev machine it >>>>>>>>>>>> is >>>>>>>>>>>> tolerable, but this is a machine specced for serious processing >>>>>>>>>>>> power. On >>>>>>>>>>>> my MacBook Air it now takes 1–2 seconds to open a new tiddler, >>>>>>>>>>>> even without >>>>>>>>>>>> anything currently open! >>>>>>>>>>>> >>>>>>>>>>>> Probably I am just asking TW to do too much on the fly here, >>>>>>>>>>>> but before I start rethinking the project too hard, can anyone >>>>>>>>>>>> think of >>>>>>>>>>>> obvious optimizations I might be missing here? The filters >>>>>>>>>>>> involved are >>>>>>>>>>>> moderately complex (the basic pattern for each is to gather >>>>>>>>>>>> together >>>>>>>>>>>> links[], backlinks[], and tagging[] for the story tiddler, then >>>>>>>>>>>> filter some >>>>>>>>>>>> things out of that using + and !*operator*[]'s). >>>>>>>>>>>> >>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/65a85989-8364-420d-9ae3-5b8de899e044n%40googlegroups.com.
