Hi Jeremy,
The system/shadow tiddlers were excluded because originally I wasn't
renaming the tiddlers, thus causing unwanted interactions on the search
wiki. I guess that with the renaming (simple prefix with $:/<wiki id>/)
there is little risk of interaction, but I'm not sure to know about all
the possible kinds of "interactions" in TW. Anyway I can do some tests
with them, but if you or any other TW expert could tell me which aspects
to be careful about when merging all the tiddlers, that could be
helpful. I know at least that I can't include plugin tiddlers, and
currently I also exclude theme and javascript tiddlers to be safe (?).
Indexing all public wikis seems more problematic to me, as some authors
might not want that: additionally to my emails on the TW list I have
sent an individual email (and later a reminder) to every known author in
my list, so at this point I have to assume that those who did not answer
are not interested. Since my system requires storing their tiddlers in
the search wiki, it wouldn't be appropriate to do it without their
consent. It would be different if the content was not made public (even
hidden as system tiddlers), but I don't see how to do differently since
it is the TW search system which does all the work.
About the particular case of is[missing], I did a manual grep search on
my local machine among the 29 wikis in my list (tagged CommunityWikis,
including the non-indexed wikis; 5715 tiddlers in total) and found 25
occurrences (more precisely lines containing at least one occurrence);
see the detailed results attached. By comparison I found:
- 135 is[current]
- 19 is[image]
- 5 is[orphan]
- 67 is[shadow]
- 389 is[system]
- 44 is[tiddler]
- 7 is[tag]
I was even imagining that this could be automatized, actually: some kind
of asynchronous search triggered by some particular tag/tiddler in an
indexed wiki, for which you would see the results the day after in a
special tiddler of the search wiki... But that seems to me like a
terribly complicated system for a very specific use case! (and that's
not even addressing the potentially excessive workload on my machine)
Regards
Erwan
On 07/02/15 13:35, Jeremy Ruston wrote:
Hi Erwan
Well done for your work on this, it's good to see this sort of
experimentation.
Something that's been cropping up over in GitHub discussions is that
it might be useful to be able to search across all community wikis to
see if certain constructions are being used heavily in the wild. For
example, we've realised that the `[is[missing]]` filter has the wrong
semantics, which we'd like to fix but it would be useful to have some
idea if it is widely used it in its present form.
So I guess I'm asking for the ability to search system and shadow
tiddlers, and perhaps to broaden the search to all publicly accessible
TW5 wikis. If only there was a way to get Google/DuckDuckGo to return
a list all TW5 documents that it has crawled...
Best wishes
Jeremy.
On Sat, Feb 7, 2015 at 1:04 PM, Tobias Beer <[email protected]
<mailto:[email protected]>> wrote:
Hi Erwan,
Will be curious how your aggregator will handle all the redirects
and renaming I'm pushing though right now. ^^
Best wishes, Tobias.
--
You received this message because you are subscribed to the Google
Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
--
Jeremy Ruston
mailto:[email protected] <mailto:[email protected]>
--
You received this message because you are subscribed to the Google
Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
ae-railroad/tiddlers/$__core.tid: "text": "/*\\\r\ntitle:
$:/core/modules/filters/is/missing.js\r\ntype:
application/javascript\r\nmodule-type: isfilteroperator\r\n\r\nFilter function
for [is[missing]]\r\n\r\n\\*/\r\n(function(){\r\n\r\n/*jslint node: true,
browser: true */\r\n/*global $tw: false */\r\n\"use
strict\";\r\n\r\n/*\r\nExport our filter function\r\n*/\r\nexports.missing =
function(source,prefix,options) {\r\n\tvar results = [];\r\n\tif(prefix ===
\"!\") {\r\n\t\tsource(function(tiddler,title)
{\r\n\t\t\tif(options.wiki.tiddlerExists(title))
{\r\n\t\t\t\tresults.push(title);\r\n\t\t\t}\r\n\t\t});\r\n\t} else
{\r\n\t\tsource(function(tiddler,title)
{\r\n\t\t\tif(!options.wiki.tiddlerExists(title))
{\r\n\t\t\t\tresults.push(title);\r\n\t\t\t}\r\n\t\t});\r\n\t}\r\n\treturn
results;\r\n};\r\n\r\n})();\r\n",
fontawesome/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid:<$list
filter="[all[current]is[missing]tagging[]sort[title]]">
tb5dev/tiddlers/1388-fix-is-missing-filter.tid:summary: fix `[is[missing]]` —
filter didn't check against `options.wiki.getMissingTitles()`
tb5dev/tiddlers/$__.tb_ui_ViewTemplate_empty.tid:<$list
filter="[all[current]!is[missing]field:text[]]">
tb5dev/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid: <$list
filter="[all[current]is[missing]tagging[]sort[title]limit[1]]">
tb5dev/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid:<<taggly
subfilter:"all[current]is[missing]sort[title]">>
tb5/tiddlers/$__core_modules_filters_is_missing.js.tid:Filter function for
[is[missing]]
tb5/tiddlers/filter_ is missing.tid:This wiki implements a different
''is[missing]'' filter than the TiddlyWiki core. This version, in fact,
effectively overrides the core filter which returns whether or not the tiddlers
in the input set exist or not.
tb5/tiddlers/filter_ is missing.tid:; `is[missing]`
tb5/tiddlers/filter_ is missing.tid:Compared to the core, `is[missing]` is
__not__ synonymous with `!is[tiddler]`, or vice versa.
tb5/tiddlers/filter_ is missing.tid:<<` "{{{ [[DoesntExist]] [[Welcome]]
+[is[missing]] }}}">>
tb5/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid: <$list
filter="[all[current]is[missing]] [all[current]text[]]
[all[current]tag[$:/tags/tagging]] +[tagging[]limit[1]]">
tb5/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid:<<taggly
subfilter:"all[current]is[missing]] [all[current]text[]]
[all[current]tag[$:/tags/tagging]] +[">>
tbdemo/tiddlers/$__core.tid: "text": "/*\\\ntitle:
$:/core/modules/filters/is/missing.js\ntype:
application/javascript\nmodule-type: isfilteroperator\n\nFilter function for
[is[missing]]\n\n\\*/\n(function(){\n\n/*jslint node: true, browser: true
*/\n/*global $tw: false */\n\"use strict\";\n\n/*\nExport our filter
function\n*/\nexports.missing = function(source,prefix,options) {\n\tvar
results = [];\n\tif(prefix === \"!\") {\n\t\tsource(function(tiddler,title)
{\n\t\t\tif(options.wiki.tiddlerExists(title))
{\n\t\t\t\tresults.push(title);\n\t\t\t}\n\t\t});\n\t} else
{\n\t\tsource(function(tiddler,title)
{\n\t\t\tif(!options.wiki.tiddlerExists(title))
{\n\t\t\t\tresults.push(title);\n\t\t\t}\n\t\t});\n\t}\n\treturn
results;\n};\n\n})();\n",
tbdemo/tiddlers/each_list.tid:<$list
filter="[each:list[list]is[missing]sort[]]">
tbdemo/tiddlers/each_list.tid:<$list
filter="[each:list[list]is[missing]sort[]]">
tbdemo/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid: <$list
filter="[all[current]is[missing]tagging[]limit[1]]">
tbdemo/tiddlers/$__.tb_ui_ViewTemplate_tagging.tid:<<taggly
subfilter:"all[current]is[missing]">>
TiddlyMap/tiddlers/$__core.tid: "text": "/*\\\ntitle:
$:/core/modules/filters/is/missing.js\ntype:
application/javascript\nmodule-type: isfilteroperator\n\nFilter function for
[is[missing]]\n\n\\*/\n(function(){\n\n/*jslint node: true, browser: true
*/\n/*global $tw: false */\n\"use strict\";\n\n/*\nExport our filter
function\n*/\nexports.missing = function(source,prefix,options) {\n\tvar
results = [];\n\tif(prefix === \"!\") {\n\t\tsource(function(tiddler,title)
{\n\t\t\tif(options.wiki.tiddlerExists(title))
{\n\t\t\t\tresults.push(title);\n\t\t\t}\n\t\t});\n\t} else
{\n\t\tsource(function(tiddler,title)
{\n\t\t\tif(!options.wiki.tiddlerExists(title))
{\n\t\t\t\tresults.push(title);\n\t\t\t}\n\t\t});\n\t}\n\treturn
results;\n};\n\n})();\n",
TiddlyQuickly/tiddlers/Creating custom lists of tiddlers using widgets and
filters (slide 9).tid:<$list filter="[!is[system]sort[title]]
[is[missing]sort[title]] +[sort[title]]"><$link><$view
field="title"/></$link><br></$list>
tiddlywiki.com/tiddlers/FilterOperator_ is.tid:* `[is[missing]]` -
MissingTiddlers that are referenced but undefined
tiddlywiki.com/tiddlers/FilterOperator_ is.tid:|`[is[missing]]` |Returns an
empty list (see note below) |
welford/tiddlers/$__core.tid: "text": "/*\\\r\ntitle:
$:/core/modules/filters/is/missing.js\r\ntype:
application/javascript\r\nmodule-type: isfilteroperator\r\n\r\nFilter function
for [is[missing]]\r\n\r\n\\*/\r\n(function(){\r\n\r\n/*jslint node: true,
browser: true */\r\n/*global $tw: false */\r\n\"use
strict\";\r\n\r\n/*\r\nExport our filter function\r\n*/\r\nexports.missing =
function(source,prefix,options) {\r\n\tvar results = [];\r\n\tif(prefix ===
\"!\") {\r\n\t\tsource(function(tiddler,title)
{\r\n\t\t\tif(options.wiki.tiddlerExists(title))
{\r\n\t\t\t\tresults.push(title);\r\n\t\t\t}\r\n\t\t});\r\n\t} else
{\r\n\t\tsource(function(tiddler,title)
{\r\n\t\t\tif(!options.wiki.tiddlerExists(title))
{\r\n\t\t\t\tresults.push(title);\r\n\t\t\t}\r\n\t\t});\r\n\t}\r\n\treturn
results;\r\n};\r\n\r\n})();\r\n",
welford/tiddlers/Filters.tid:``[is[missing]]``
welford/tiddlers/Filters.tid:<<list-links "[is[missing]]">>
