[tw] Re: [TW5] will the search get quicker?

[Jon]

Thanks for that Tobias - I'll look forward to the improvements.
Regards
Jon

On Monday, 2 March 2015 11:54:54 UTC, Tobias Beer wrote:
>
> Hi Jon,
>
> There are some open issues that would very much improve (perceived) search 
> performance, namely...
>
>- *a textlength / minlength filter*
>- so that search only kicks in after x characters being entered, e.g. 3
>   - https://github.com/Jermolene/TiddlyWiki5/issues/1493
>   - 
>   
> https://github.com/Jermolene/TiddlyWiki5/commit/2305385c08152c050946e4502735045a8e30aac1#commitcomment-9685869
>   - *some form of refresh delay*
>   - so that TiddlyWiki only starts searching after you pause typing
>   - https://github.com/Jermolene/TiddlyWiki5/issues/1494
>
> Best wishes, Tobias.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

Re: [tw] Re: [TW5] will the search get quicker?

[Tobias Beer]

Hi Andreas,
 

> if you double the tiddler count, the search will take 4  times the time, 
> provided all tiddlers are equal length.
>

Not following. Why would that be?

Best wishes, Tobias.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Tobias Beer]

Hi Jon,

There are some open issues that would very much improve (perceived) search 
performance, namely...

   - *a textlength / minlength filter*
   - so that search only kicks in after x characters being entered, e.g. 3
  - https://github.com/Jermolene/TiddlyWiki5/issues/1493
  - 
  
https://github.com/Jermolene/TiddlyWiki5/commit/2305385c08152c050946e4502735045a8e30aac1#commitcomment-9685869
  - *some form of refresh delay*
  - so that TiddlyWiki only starts searching after you pause typing
  - https://github.com/Jermolene/TiddlyWiki5/issues/1494
   
Best wishes, Tobias.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Jon]

It's good to know there's potential for improvement down the road.
Thanks
Jon

On Monday, 23 February 2015 20:48:11 UTC, Jon wrote:
>
> Hi,
>
> I use Microsoft OneNote at work and one thing that strikes me is the speed 
> of the search facility as the results appear instantly as you type.
>
> I know TW is a very different kettle of fish but is the search facility 
> likely to become quicker in future releases or is the current speed as good 
> as it gets?
>
> I have two wikis, one of 4MB and another which is 8MB and the lag when 
> typing in the search box is far worse in the bigger file and quite annoying 
> and will no doubt get worse as the file size increases.
>
> Thanks, Jon.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

Re: [tw] Re: [TW5] will the search get quicker?

[Jeremy Ruston]

Echoing Andreas' points, it's worth stressing that very little work has yet
gone into optimising the implementation of TW5 (there's been a lot of
emphasis on optimising the architecture eg selective updating of the DOM).
So, there's generally a lot of scope for optimisation.

Having said that, I suspect that it's not actually the searching that's
taking the time in the cases reported here; it's more likely to be the
refreshing of the sidebar and story river (particularly the "recent" tab,
which is inherently slow to render).

Picking up some other points:

* It's sub-optimal that searching occurs when only one or two characters
have been typed in the search box (we've recently been discussing fixes for
this over on GitHub)
* Obviously, it's impossible for the browser to search lazily loaded
tiddlers without loading their content from the server. The only way around
that is to have server side support for searching (which I'd certainly like
to add).
* The lack of UI feedback during time-consuming operations is definitely
sub-optimal. The problem here is that these operations block the browser,
making it impossible to display feedback
* The number of binary tiddlers shouldn't make much difference to searches;
the core only searches text tiddlers (whether wikitext, plain text or other
UTF-8 encoded text format)

So, going back to Jon's original question, the answer is "yes", search
performance will improve as we address the issues above.

Best wishes

Jeremy.




On Tue, Feb 24, 2015 at 1:52 PM, Andreas Hahn 
wrote:

> Hi,
>
> Mat is right though: TiddlyWiki is not optimized for speed. In almost all
> cases TiddlyWiki prefers structural quality, correctness and simplicity of
> implementation over speed. Also a lot of features are only implemented just
> so they are available, but noone optimized them yet.
> The search for instance (correct me if I am wrong with the following)
> applies a regex to all tiddlers in which it searches, which in simple cases
> (like most searches) means quadratic runtime at minimum. (So to explain it:
> if you double the tiddler count, the search will take 4 times the time,
> provided all tiddlers are equal length).
> So to answer the question: I guess Jeremy is aware that at some point the
> search becomes very slow in larger TW files and that there are significant
> speedups possible, however they will require work and are currently not
> important enough to be worked on. So it will probably get quicker, when TW
> gets more mature.
>
> /Andreas
>
> P.S.: Also, I have a question: Provided I would have lazy loading set up
> for most of my tiddlers. Does doing a search mean that all tiddlers will be
> loaded when I do the search ? Because .. It sure seems that way.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tiddlywiki+unsubscr...@googlegroups.com.
> To post to this group, send email to tiddlywiki@googlegroups.com.
> Visit this group at http://groups.google.com/group/tiddlywiki.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Jeremy Ruston
mailto:jeremy.rus...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

Re: [tw] Re: [TW5] will the search get quicker?

[Andreas Hahn]

Hi,

Mat is right though: TiddlyWiki is not optimized for speed. In almost 
all cases TiddlyWiki prefers structural quality, correctness and 
simplicity of implementation over speed. Also a lot of features are only 
implemented just so they are available, but noone optimized them yet.
The search for instance (correct me if I am wrong with the following) 
applies a regex to all tiddlers in which it searches, which in simple 
cases (like most searches) means quadratic runtime at minimum. (So to 
explain it: if you double the tiddler count, the search will take 4 
times the time, provided all tiddlers are equal length).
So to answer the question: I guess Jeremy is aware that at some point 
the search becomes very slow in larger TW files and that there are 
significant speedups possible, however they will require work and are 
currently not important enough to be worked on. So it will probably get 
quicker, when TW gets more mature.


/Andreas

P.S.: Also, I have a question: Provided I would have lazy loading set up 
for most of my tiddlers. Does doing a search mean that all tiddlers will 
be loaded when I do the search ? Because .. It sure seems that way.


--
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

Re: [tw] Re: [TW5] will the search get quicker?

[Alex Hough]

Richard,

There is a bit of lag your the calculus TW, but I think part of it is an
illusion. it it were facebook there'd be a little turning cog indicating
that something is happening. its perhaps all about the perception of
speed

Alex

On 24 February 2015 at 09:55, Mat  wrote:

> Speed has been a recurring issue since inception.
>
> We really should find a best, common, *test engine* for this and apply it
> to an empty + a "full" TW, with + without embedded images, perhaps
> different browsers etc and see what actually makes sense. And ideally an
> engine that users can feed their own TW's into for comparison. Actually,
> such a service, even if only an external link, would be a great addition to
> tw .com IMO.
>
> For online TW's (starting with http(s)) maybe googles PageSpeed Insights
>  is the simplest?
>
> Could perhap yslow be used?
>
>
>
>
>
>
>
>
>
> On Tuesday, February 24, 2015 at 8:32:24 AM UTC+1, Linus Johnsson wrote:
>>
>> I've also noticed some performance issues while searching in my TW. It
>> currently contains 1102 tiddlers and is about 3 Mb, with no embedded files.
>> There is a huge difference between my desktop PC (Intel i7-3770K) and my
>> 2013 MacBook Pro (i5), so I gather that CPU power is crucial. I use Firefox
>> exclusively. My overall feel is that TW does not scale that well, which is
>> troubling.
>>
>> As an aside, I think that the input lag while editing (sometimes several
>> seconds) is far worse. It increases about linearly with the number of open
>> tiddlers. Having the "Recent" tab open also makes it much worse. Since I
>> use transclusion a lot, perhaps some kind of cascading effects may be to
>> blame, although it does not seem to make much difference whether the
>> tiddler I'm editing is transcluded or not.
>>
>> Best regards,
>> Linus
>>
>> Den måndag 23 februari 2015 kl. 23:36:58 UTC+1 skrev Jon:
>>>
>>> Hi Richard,
>>>
>>> no, not at all. I'll look into my file and remove the embedded jpg
>>> images and I'll see what that does.
>>>
>>> Regards
>>> Jon
>>>
>>> On Monday, 23 February 2015 22:32:36 UTC, Richard Smith wrote:

 Hi Jon,

 I hope you understand that I'm not trying to 'defend' TW, but just to
 help you/us see if there is a problem of some kind that needs fixing.

 The same file I mentioned has 727 svg images but only 5 jpg, so certain
 file types could be an issue, I suppose.

 You can get a copy of the file I mentioned here https://dl.
 dropboxusercontent.com/u/83055414/Active_Calculus_v0.1.html which
 might help to determine whether it's your file or your set-up that's
 slowing things down.

 Regards,
 Richard

>>>  --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tiddlywiki+unsubscr...@googlegroups.com.
> To post to this group, send email to tiddlywiki@googlegroups.com.
> Visit this group at http://groups.google.com/group/tiddlywiki.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Mat]

Speed has been a recurring issue since inception.

We really should find a best, common, *test engine* for this and apply it 
to an empty + a "full" TW, with + without embedded images, perhaps 
different browsers etc and see what actually makes sense. And ideally an 
engine that users can feed their own TW's into for comparison. Actually, 
such a service, even if only an external link, would be a great addition to 
tw .com IMO.

For online TW's (starting with http(s)) maybe googles PageSpeed Insights 
 is the simplest?

Could perhap yslow be used?








On Tuesday, February 24, 2015 at 8:32:24 AM UTC+1, Linus Johnsson wrote:
>
> I've also noticed some performance issues while searching in my TW. It 
> currently contains 1102 tiddlers and is about 3 Mb, with no embedded files. 
> There is a huge difference between my desktop PC (Intel i7-3770K) and my 
> 2013 MacBook Pro (i5), so I gather that CPU power is crucial. I use Firefox 
> exclusively. My overall feel is that TW does not scale that well, which is 
> troubling.
>
> As an aside, I think that the input lag while editing (sometimes several 
> seconds) is far worse. It increases about linearly with the number of open 
> tiddlers. Having the "Recent" tab open also makes it much worse. Since I 
> use transclusion a lot, perhaps some kind of cascading effects may be to 
> blame, although it does not seem to make much difference whether the 
> tiddler I'm editing is transcluded or not.
>
> Best regards,
> Linus
>
> Den måndag 23 februari 2015 kl. 23:36:58 UTC+1 skrev Jon:
>>
>> Hi Richard,
>>
>> no, not at all. I'll look into my file and remove the embedded jpg images 
>> and I'll see what that does.
>>
>> Regards
>> Jon
>>
>> On Monday, 23 February 2015 22:32:36 UTC, Richard Smith wrote:
>>>
>>> Hi Jon,
>>>
>>> I hope you understand that I'm not trying to 'defend' TW, but just to 
>>> help you/us see if there is a problem of some kind that needs fixing. 
>>>
>>> The same file I mentioned has 727 svg images but only 5 jpg, so certain 
>>> file types could be an issue, I suppose. 
>>>
>>> You can get a copy of the file I mentioned here 
>>> https://dl.dropboxusercontent.com/u/83055414/Active_Calculus_v0.1.html 
>>> which might help to determine whether it's your file or your set-up that's 
>>> slowing things down.
>>>
>>> Regards,
>>> Richard
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Linus Johnsson]

I've also noticed some performance issues while searching in my TW. It 
currently contains 1102 tiddlers and is about 3 Mb, with no embedded files. 
There is a huge difference between my desktop PC (Intel i7-3770K) and my 
2013 MacBook Pro (i5), so I gather that CPU power is crucial. I use Firefox 
exclusively. My overall feel is that TW does not scale that well, which is 
troubling.

As an aside, I think that the input lag while editing (sometimes several 
seconds) is far worse. It increases about linearly with the number of open 
tiddlers. Having the "Recent" tab open also makes it much worse. Since I 
use transclusion a lot, perhaps some kind of cascading effects may be to 
blame, although it does not seem to make much difference whether the 
tiddler I'm editing is transcluded or not.

Best regards,
Linus

Den måndag 23 februari 2015 kl. 23:36:58 UTC+1 skrev Jon:
>
> Hi Richard,
>
> no, not at all. I'll look into my file and remove the embedded jpg images 
> and I'll see what that does.
>
> Regards
> Jon
>
> On Monday, 23 February 2015 22:32:36 UTC, Richard Smith wrote:
>>
>> Hi Jon,
>>
>> I hope you understand that I'm not trying to 'defend' TW, but just to 
>> help you/us see if there is a problem of some kind that needs fixing. 
>>
>> The same file I mentioned has 727 svg images but only 5 jpg, so certain 
>> file types could be an issue, I suppose. 
>>
>> You can get a copy of the file I mentioned here 
>> https://dl.dropboxusercontent.com/u/83055414/Active_Calculus_v0.1.html 
>> which might help to determine whether it's your file or your set-up that's 
>> slowing things down.
>>
>> Regards,
>> Richard
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Jon]

Hi Richard,

no, not at all. I'll look into my file and remove the embedded jpg images 
and I'll see what that does.

Regards
Jon

On Monday, 23 February 2015 22:32:36 UTC, Richard Smith wrote:
>
> Hi Jon,
>
> I hope you understand that I'm not trying to 'defend' TW, but just to help 
> you/us see if there is a problem of some kind that needs fixing. 
>
> The same file I mentioned has 727 svg images but only 5 jpg, so certain 
> file types could be an issue, I suppose. 
>
> You can get a copy of the file I mentioned here 
> https://dl.dropboxusercontent.com/u/83055414/Active_Calculus_v0.1.html 
> which might help to determine whether it's your file or your set-up that's 
> slowing things down.
>
> Regards,
> Richard
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Richard Smith]

Hi Jon,

I hope you understand that I'm not trying to 'defend' TW, but just to help 
you/us see if there is a problem of some kind that needs fixing. 

The same file I mentioned has 727 svg images but only 5 jpg, so certain 
file types could be an issue, I suppose. 

You can get a copy of the file I mentioned 
here https://dl.dropboxusercontent.com/u/83055414/Active_Calculus_v0.1.html 
which might help to determine whether it's your file or your set-up that's 
slowing things down.

Regards,
Richard

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[Jon]

Hi Richard,

My 8MB file only has ~580 tiddlers which is a tiny fraction of your file 
which sounds like it's still very responsive.

I'm only doing regular searches and I'm running Windows 7/Firefox 

Mine must be bloated with embedded image files so I'll take these out and 
hopefully that will speed things up.

Thanks,
Jon

On Monday, 23 February 2015 21:43:43 UTC, RichardWilliamSmith wrote:
>
> Hi Jon,
>
> I know some issues with the search have been discussed in the past, 
> specifically with regards to limiting the maximum number of results and 
> waiting until there's a minimal string before searching. Having said that, 
> I have personally found the search to be quite quick.
>
> I just checked and my biggest file is ~17MB with ~9000 tiddlers and the 
> search is still very quick (Firefox under OSX yosemite, TW5.1.7 running as 
> a single file) - typing just the letter 'a', for example, returns 9090 
> matches in less than a second. There's a slight lag on entering text in the 
> search box itself but it's not noticeable enough to be annoying. 
>
> Are the searches you're doing just plain-text matching or something more 
> exotic?
> What is the set-up that you're using? (browser/OS/standalone or node etc.)
> How many tiddlers do your files have in them?
>
> Regards,
> Richard
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.

[tw] Re: [TW5] will the search get quicker?

[RichardWilliamSmith]

Hi Jon,

I know some issues with the search have been discussed in the past, 
specifically with regards to limiting the maximum number of results and 
waiting until there's a minimal string before searching. Having said that, 
I have personally found the search to be quite quick.

I just checked and my biggest file is ~17MB with ~9000 tiddlers and the 
search is still very quick (Firefox under OSX yosemite, TW5.1.7 running as 
a single file) - typing just the letter 'a', for example, returns 9090 
matches in less than a second. There's a slight lag on entering text in the 
search box itself but it's not noticeable enough to be annoying. 

Are the searches you're doing just plain-text matching or something more 
exotic?
What is the set-up that you're using? (browser/OS/standalone or node etc.)
How many tiddlers do your files have in them?

Regards,
Richard

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywiki@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.