Re: [tw5] Fetching tiddlers from one wiki to another

[Ste]

This is sounding exciting!  All I need to do now is get my IT to host:( 
:D

On Thursday, 22 April 2021 at 09:23:21 UTC+1 Jan wrote:

> Hi Saq, 
> I think that your plugin is great for educational scenarios - and there 
> CORS won't be a problem because as student's  and teacher's wikis are 
> likely to be hosted in the same domain. 
> I found the $filter in the widget - thanks for the hint.
>
> I would like to build / use a set of wikis where  
> -the teacher (me) has a wiki has a wiki which is serving material and 
> tasks. 
> -the students work on the tasks an build a portfolio doing this. There 
> your plugin would be great to import the tiddlers with the current material
> -the teacher has a anotherwiki to automatically collect the tiddlers 
> marked to be sent in as solved tasks. (Again your plugin could be very 
> usefull.)
> (What is still missing in my thought is an easy way for the sutdents to 
> interact in this construction)
>
> So thanks for helping me with your valuable tool.
> Jan
>
>
>
>
>
>
> I found the way to implement a filter which is great. 
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/188dfe58-a1f2-4919-9509-e21f6dda9eb6n%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Jan Johannpeter]

Hi Saq,
I think that your plugin is great for educational scenarios - and there 
CORS won't be a problem because as student's  and teacher's wikis are 
likely to be hosted in the same domain.

I found the $filter in the widget - thanks for the hint.

I would like to build / use a set of wikis where
-the teacher (me) has a wiki has a wiki which is serving material and 
tasks.
-the students work on the tasks an build a portfolio doing this. There 
your plugin would be great to import the tiddlers with the current 
material
-the teacher has a anotherwiki to automatically collect the tiddlers 
marked to be sent in as solved tasks. (Again your plugin could be very 
usefull.)
(What is still missing in my thought is an easy way for the sutdents to 
interact in this construction)


So thanks for helping me with your valuable tool.
Jan






I found the way to implement a filter which is great.






--
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/JYHYRQ.LHYJLLEJQS1K%40gmail.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Jean-Pierre Rivière]

Thank you for your answer. Well, now the POC is working from this link with 
chromium and firefox, both on linux. 

If I get back the bug, I'll look at the console. I should have thought 
about it. I had tried a few things before discovering the "imteractive 
import" tiddlers but not this time.

As for loading tiddlers, to be honest, this is not the proper way to 
describe what I want. The tiddlers I want to show suppose a lot of macros 
and data tiddlers for their rendering. What I would like is getting a 
snapshot of their final html code within the requiring wiki, which would 
have imported the sylesheet beforehand. This being stated, I also have some 
plain tiddlers I would like to see. importing them within the $:/temp/ 
hierarchy would be fine. All those tiddlers are helpers for reference 
documentation. They are too numerous to include. Thiy must be seen on an ad 
hoc basis and get ridden off ASAP.


Le mercredi 21 avril 2021 à 15:34:26 UTC+2, saq.i...@gmail.com a écrit :

>  
>
>> I have got a look at 
>> https://saqimtiaz.github.io/sq-tw/temp/get-remote-tiddlers.html but in 
>> the "Interactive import" tiddler, the button reads "fetch Undefined widget 
>> 'action-getremotetiddler'" and does nothing. I tested it on a recent 
>> chromium.
>>
>>
> That is pretty odd, it works for me in both current Chrome and FF on 
> Windows 10. Do you see any error messages in the browser console?
>  
>
>>  As for me, I would like to be able to transclude tiddlers from other 
>> wikis, but not import them per se.
>>
>
> That is not the aim with this POC.  TiddlyWiki does have facilities to 
> load remote .tid files as tiddlers which might be closer to what you need. 
> It makes use of the _canonical_uri field but has the same limitations 
> regarding CORS.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/dc67cd1b-457b-48e7-aa33-09c1c1baf282n%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

 

> I have got a look at 
> https://saqimtiaz.github.io/sq-tw/temp/get-remote-tiddlers.html but in 
> the "Interactive import" tiddler, the button reads "fetch Undefined widget 
> 'action-getremotetiddler'" and does nothing. I tested it on a recent 
> chromium.
>
>
That is pretty odd, it works for me in both current Chrome and FF on 
Windows 10. Do you see any error messages in the browser console?
 

>  As for me, I would like to be able to transclude tiddlers from other 
> wikis, but not import them per se.
>

That is not the aim with this POC.  TiddlyWiki does have facilities to load 
remote .tid files as tiddlers which might be closer to what you need. It 
makes use of the _canonical_uri field but has the same limitations 
regarding CORS.

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/75328118-9525-46a4-b7a2-6ab1fceed642n%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Jean-Pierre Rivière]

I have got a look 
at https://saqimtiaz.github.io/sq-tw/temp/get-remote-tiddlers.html but in 
the "Interactive import" tiddler, the button reads "fetch Undefined widget 
'action-getremotetiddler'" and does nothing. I tested it on a recent 
chromium.

I can't see any plugin or macro for the magic. And I'm not so sure at what 
is happening. As for me, I would like to be able to transclude tiddlers 
from other wikis, but not import them per se. Icbnnot see if that would be 
possible within the scope of this projects (and it should also be noted the 
alrady announced technical limitations liki CORS).

regards,

-- 
Jean-Pierre

Le mercredi 21 avril 2021 à 07:23:57 UTC+2, saq.i...@gmail.com a écrit :

> Hi Jan,
>
> I'll reiterate from my first post that this is just a quick and messy POC 
> with a very narrow usage window because of the CORS requirement.
>
> Also note that apparently Twederation is supposed to be able to silently 
> import content in the background as well, as pointed out by Jeremy above. 
> If that works for your intended workflow, it may be a better choice for you 
> as it wont impose the same requirement to serve the wiki files with the 
> correct CORS headers.
>
> The demo I posted does support using a filter to only import certain 
> tiddlers. The other two requirements you mention are feasible though not 
> implemented in the demo. I am still at the exploration stage of seeing what 
> is possible, and need to work on the project proposal I have in mind which 
> involves using TW in an LMS context too. If that goes well I will revisit 
> this topic later this year.
>
> Regards,
> Saq
>
> On Saturday, April 17, 2021 at 4:09:09 PM UTC+2 Jan wrote:
>
>> Hi Saq,
>> thank you very much for this widget! This is a great help, I have looked 
>> for something like this for a long time. 
>> This is a very important puzzlepiece to use TW as an interactive Learning 
>> Management System. 
>> For this usecase it is great that it has a silent mode, which makes it 
>> mor unobstrusive than the fetch-function of Twederation.
>> For the interactity of an LMS I would love three other features:
>>
>> -Can it handle filters to import only certain tiddlers? So far I could 
>> not see how to implement them in the action-widget.
>> -It would be great if it could tag them imported tiddlers like "temp" or 
>> "imported"
>> -It would be very important to have an "Overwrite Filter" determining 
>> which tiddlers can be overwritten by the imported tiddlers. (For example al 
>> those whose modifier is not the current user).
>>
>> Best wishes  and nany thanks for this great improvement!
>> Jan
>>
>>
>>
>>
>>
>> Am So, 4. Apr, 2021 um 10:49 VORMITTAGS schrieb Saq Imtiaz <
>> saq.i...@gmail.com>:
>>
>> Hi Jeremy,
>>
>> A little background: I have a couple of ideas in mind for projects to do 
>> with education with some of the organizations I work with, for which I am 
>> considering TiddlyWiki. One of them would be very similar to the syncing 
>> educator and student notebooks project that was based on TiddlyWiki classic 
>> over 10 years ago. The other involves providing a sandbox for exploration 
>> and developing skills to do with using and potentially creating/customizing 
>> digital tools to aid one's own learning. While TiddlyWiki sounds like a 
>> good fit, it will also be essential that it is hard for the students to 
>> break the tool itself.
>>
>> Both projects, if they proceed, will entail quite a bit of custom coding 
>> to get the user experience just right, so as a first step I wanted to 
>> understand what was possible today in terms of browser restrictions and 
>> loading content dynamically in TW5. As mentioned in my original post, I 
>> realize the use case for this particular prototype is very narrow. However 
>> I find that sharing ideas and prototypes is rarely a bad idea as it can 
>> often inspire other ideas in the community, so I try my best to do so 
>> whenever possible.
>>  
>>
>>> Those restrictions still exist. For example, the XMLHttpRequest approach 
>>> described here generally won’t work from a file: URI, which was a major 
>>> design goal for the plugin library and Twederation.
>>>
>>
>> That's interesting as I've found it to work without problems to fetch 
>> content into a local file wiki as long as the remote content is served with 
>> the correct CORS headers (and the request is made without the 
>> X-Requested-With header). Or do you mean when the remote content is also 
>> accessed via a file: URI ? I have not tested that scenario.
>>
>> Hosting all the wikis with CORS support is an easy requirement to satisfy 
>> for the projects I have in mind.
>>  
>>
>>> It’s definitely time we explored dynamic content loading via 
>>> XMLHttpRequest in more detail. The constraints get less onerous as we gain 
>>> better HTTP/HTTPS solutions.
>>>
>>
>> Agreed. I quite miss the possibilities offered by the quite rich 
>> ecosystem of adaptors that we had in TiddlyWiki Classic and even the 

Re: [tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

Hi Jan,

I'll reiterate from my first post that this is just a quick and messy POC 
with a very narrow usage window because of the CORS requirement.

Also note that apparently Twederation is supposed to be able to silently 
import content in the background as well, as pointed out by Jeremy above. 
If that works for your intended workflow, it may be a better choice for you 
as it wont impose the same requirement to serve the wiki files with the 
correct CORS headers.

The demo I posted does support using a filter to only import certain 
tiddlers. The other two requirements you mention are feasible though not 
implemented in the demo. I am still at the exploration stage of seeing what 
is possible, and need to work on the project proposal I have in mind which 
involves using TW in an LMS context too. If that goes well I will revisit 
this topic later this year.

Regards,
Saq

On Saturday, April 17, 2021 at 4:09:09 PM UTC+2 Jan wrote:

> Hi Saq,
> thank you very much for this widget! This is a great help, I have looked 
> for something like this for a long time. 
> This is a very important puzzlepiece to use TW as an interactive Learning 
> Management System. 
> For this usecase it is great that it has a silent mode, which makes it mor 
> unobstrusive than the fetch-function of Twederation.
> For the interactity of an LMS I would love three other features:
>
> -Can it handle filters to import only certain tiddlers? So far I could not 
> see how to implement them in the action-widget.
> -It would be great if it could tag them imported tiddlers like "temp" or 
> "imported"
> -It would be very important to have an "Overwrite Filter" determining 
> which tiddlers can be overwritten by the imported tiddlers. (For example al 
> those whose modifier is not the current user).
>
> Best wishes  and nany thanks for this great improvement!
> Jan
>
>
>
>
>
> Am So, 4. Apr, 2021 um 10:49 VORMITTAGS schrieb Saq Imtiaz <
> saq.i...@gmail.com>:
>
> Hi Jeremy,
>
> A little background: I have a couple of ideas in mind for projects to do 
> with education with some of the organizations I work with, for which I am 
> considering TiddlyWiki. One of them would be very similar to the syncing 
> educator and student notebooks project that was based on TiddlyWiki classic 
> over 10 years ago. The other involves providing a sandbox for exploration 
> and developing skills to do with using and potentially creating/customizing 
> digital tools to aid one's own learning. While TiddlyWiki sounds like a 
> good fit, it will also be essential that it is hard for the students to 
> break the tool itself.
>
> Both projects, if they proceed, will entail quite a bit of custom coding 
> to get the user experience just right, so as a first step I wanted to 
> understand what was possible today in terms of browser restrictions and 
> loading content dynamically in TW5. As mentioned in my original post, I 
> realize the use case for this particular prototype is very narrow. However 
> I find that sharing ideas and prototypes is rarely a bad idea as it can 
> often inspire other ideas in the community, so I try my best to do so 
> whenever possible.
>  
>
>> Those restrictions still exist. For example, the XMLHttpRequest approach 
>> described here generally won’t work from a file: URI, which was a major 
>> design goal for the plugin library and Twederation.
>>
>
> That's interesting as I've found it to work without problems to fetch 
> content into a local file wiki as long as the remote content is served with 
> the correct CORS headers (and the request is made without the 
> X-Requested-With header). Or do you mean when the remote content is also 
> accessed via a file: URI ? I have not tested that scenario.
>
> Hosting all the wikis with CORS support is an easy requirement to satisfy 
> for the projects I have in mind.
>  
>
>> It’s definitely time we explored dynamic content loading via 
>> XMLHttpRequest in more detail. The constraints get less onerous as we gain 
>> better HTTP/HTTPS solutions.
>>
>
> Agreed. I quite miss the possibilities offered by the quite rich ecosystem 
> of adaptors that we had in TiddlyWiki Classic and even the sync mechanism, 
> despite all the clunkiness inherent in the design and the later 
> difficulties imposed by greater browser restrictions. 
>  
> Regards,
> Saq
>
> -- 
>
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywiki+...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywiki/7ea4cca2-c07a-486d-a60c-b59c9838c0bdn%40googlegroups.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group 

Re: [tw5] Fetching tiddlers from one wiki to another

[TW Tones]

Folks,

I have long sought something similar, and wonder if the following approach 
would help with remote tiddlers, loading/importing and performance.

   - Provide a utility to save filtered tiddlers (json) to another filename 
   on the host, not index.html or wikiname.html, a bit like rss
  - This is the source wiki publishing a subset of tiddler for remote 
  load, it can be added to the save process.
   - Provide a plugin that given a url (inc file://) to the above address 
   will silently import (although log it) the tiddlers found in the JSON file 
   if they have changed, in the current wiki.

I am confident this is possible, I have seen the various requirements 
satisfied previously I just to not have the skills to code it, needing a 
saver etc... 

This same mechaisium could be used to generate multiple files to use a 
filter to publish a library. If necessary this can be done through a zip 
file, however it would be nice if the remote wiki could unzip and import. I 
am a strong believer that a library or importing remote tiddlers would be a 
far simpler way to intergrrating tiddlywiki than other suggestions so far, 
which with all due respect to those at the leading edge can be hard to 
follow.

My current workaround is a package Icon (tiddlers in a json format) that I 
can drag from one Wiki in an iFrame to another to install or update.

Tones
On Sunday, 18 April 2021 at 00:09:09 UTC+10 Jan wrote:

> Hi Saq,
> thank you very much for this widget! This is a great help, I have looked 
> for something like this for a long time. 
> This is a very important puzzlepiece to use TW as an interactive Learning 
> Management System. 
> For this usecase it is great that it has a silent mode, which makes it mor 
> unobstrusive than the fetch-function of Twederation.
> For the interactity of an LMS I would love three other features:
>
> -Can it handle filters to import only certain tiddlers? So far I could not 
> see how to implement them in the action-widget.
> -It would be great if it could tag them imported tiddlers like "temp" or 
> "imported"
> -It would be very important to have an "Overwrite Filter" determining 
> which tiddlers can be overwritten by the imported tiddlers. (For example al 
> those whose modifier is not the current user).
>
> Best wishes  and nany thanks for this great improvement!
> Jan
>
>
>
>
>
> Am So, 4. Apr, 2021 um 10:49 VORMITTAGS schrieb Saq Imtiaz <
> saq.i...@gmail.com>:
>
> Hi Jeremy,
>
> A little background: I have a couple of ideas in mind for projects to do 
> with education with some of the organizations I work with, for which I am 
> considering TiddlyWiki. One of them would be very similar to the syncing 
> educator and student notebooks project that was based on TiddlyWiki classic 
> over 10 years ago. The other involves providing a sandbox for exploration 
> and developing skills to do with using and potentially creating/customizing 
> digital tools to aid one's own learning. While TiddlyWiki sounds like a 
> good fit, it will also be essential that it is hard for the students to 
> break the tool itself.
>
> Both projects, if they proceed, will entail quite a bit of custom coding 
> to get the user experience just right, so as a first step I wanted to 
> understand what was possible today in terms of browser restrictions and 
> loading content dynamically in TW5. As mentioned in my original post, I 
> realize the use case for this particular prototype is very narrow. However 
> I find that sharing ideas and prototypes is rarely a bad idea as it can 
> often inspire other ideas in the community, so I try my best to do so 
> whenever possible.
>  
>
>> Those restrictions still exist. For example, the XMLHttpRequest approach 
>> described here generally won’t work from a file: URI, which was a major 
>> design goal for the plugin library and Twederation.
>>
>
> That's interesting as I've found it to work without problems to fetch 
> content into a local file wiki as long as the remote content is served with 
> the correct CORS headers (and the request is made without the 
> X-Requested-With header). Or do you mean when the remote content is also 
> accessed via a file: URI ? I have not tested that scenario.
>
> Hosting all the wikis with CORS support is an easy requirement to satisfy 
> for the projects I have in mind.
>  
>
>> It’s definitely time we explored dynamic content loading via 
>> XMLHttpRequest in more detail. The constraints get less onerous as we gain 
>> better HTTP/HTTPS solutions.
>>
>
> Agreed. I quite miss the possibilities offered by the quite rich ecosystem 
> of adaptors that we had in TiddlyWiki Classic and even the sync mechanism, 
> despite all the clunkiness inherent in the design and the later 
> difficulties imposed by greater browser restrictions. 
>  
> Regards,
> Saq
>
> -- 
>
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving 

Re: [tw5] Fetching tiddlers from one wiki to another

[Jan Johannpeter]

Hi Saq,
thank you very much for this widget! This is a great help, I have 
looked for something like this for a long time.
This is a very important puzzlepiece to use TW as an interactive 
Learning Management System.
For this usecase it is great that it has a silent mode, which makes it 
mor unobstrusive than the fetch-function of Twederation.

For the interactity of an LMS I would love three other features:

-Can it handle filters to import only certain tiddlers? So far I could 
not see how to implement them in the action-widget.
-It would be great if it could tag them imported tiddlers like "temp" 
or "imported"
-It would be very important to have an "Overwrite Filter" determining 
which tiddlers can be overwritten by the imported tiddlers. (For 
example al those whose modifier is not the current user).


Best wishes  and nany thanks for this great improvement!
Jan





Am So, 4. Apr, 2021 um 10:49 VORMITTAGS schrieb Saq Imtiaz 
:

Hi Jeremy,

A little background: I have a couple of ideas in mind for projects to 
do with education with some of the organizations I work with, for 
which I am considering TiddlyWiki. One of them would be very similar 
to the syncing educator and student notebooks project that was based 
on TiddlyWiki classic over 10 years ago. The other involves providing 
a sandbox for exploration and developing skills to do with using and 
potentially creating/customizing digital tools to aid one's own 
learning. While TiddlyWiki sounds like a good fit, it will also be 
essential that it is hard for the students to break the tool itself.


Both projects, if they proceed, will entail quite a bit of custom 
coding to get the user experience just right, so as a first step I 
wanted to understand what was possible today in terms of browser 
restrictions and loading content dynamically in TW5. As mentioned in 
my original post, I realize the use case for this particular 
prototype is very narrow. However I find that sharing ideas and 
prototypes is rarely a bad idea as it can often inspire other ideas 
in the community, so I try my best to do so whenever possible.


Those restrictions still exist. For example, the XMLHttpRequest 
approach described here generally won’t work from a file: URI, 
which was a major design goal for the plugin library and Twederation.


That's interesting as I've found it to work without problems to fetch 
content into a local file wiki as long as the remote content is 
served with the correct CORS headers (and the request is made without 
the X-Requested-With header). Or do you mean when the remote content 
is also accessed via a file: URI ? I have not tested that scenario.


Hosting all the wikis with CORS support is an easy requirement to 
satisfy for the projects I have in mind.


It’s definitely time we explored dynamic content loading via 
XMLHttpRequest in more detail. The constraints get less onerous as 
we gain better HTTP/HTTPS solutions.


Agreed. I quite miss the possibilities offered by the quite rich 
ecosystem of adaptors that we had in TiddlyWiki Classic and even the 
sync mechanism, despite all the clunkiness inherent in the design and 
the later difficulties imposed by greater browser restrictions.


Regards,
Saq
 --
 You received this message because you are subscribed to the Google 
Groups "TiddlyWiki" group.
 To unsubscribe from this group and stop receiving emails from it, 
send an email to tiddlywiki+unsubscr...@googlegroups.com 
.
 To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/7ea4cca2-c07a-486d-a60c-b59c9838c0bdn%40googlegroups.com 
.


--
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/UMOPRQ.4ES1X6YF4FAM%40gmail.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

Hi Jeremy,

A little background: I have a couple of ideas in mind for projects to do 
with education with some of the organizations I work with, for which I am 
considering TiddlyWiki. One of them would be very similar to the syncing 
educator and student notebooks project that was based on TiddlyWiki classic 
over 10 years ago. The other involves providing a sandbox for exploration 
and developing skills to do with using and potentially creating/customizing 
digital tools to aid one's own learning. While TiddlyWiki sounds like a 
good fit, it will also be essential that it is hard for the students to 
break the tool itself.

Both projects, if they proceed, will entail quite a bit of custom coding to 
get the user experience just right, so as a first step I wanted to 
understand what was possible today in terms of browser restrictions and 
loading content dynamically in TW5. As mentioned in my original post, I 
realize the use case for this particular prototype is very narrow. However 
I find that sharing ideas and prototypes is rarely a bad idea as it can 
often inspire other ideas in the community, so I try my best to do so 
whenever possible.
 

> Those restrictions still exist. For example, the XMLHttpRequest approach 
> described here generally won’t work from a file: URI, which was a major 
> design goal for the plugin library and Twederation.
>

That's interesting as I've found it to work without problems to fetch 
content into a local file wiki as long as the remote content is served with 
the correct CORS headers (and the request is made without the 
X-Requested-With header). Or do you mean when the remote content is also 
accessed via a file: URI ? I have not tested that scenario.

Hosting all the wikis with CORS support is an easy requirement to satisfy 
for the projects I have in mind.
 

> It’s definitely time we explored dynamic content loading via 
> XMLHttpRequest in more detail. The constraints get less onerous as we gain 
> better HTTP/HTTPS solutions.
>

Agreed. I quite miss the possibilities offered by the quite rich ecosystem 
of adaptors that we had in TiddlyWiki Classic and even the sync mechanism, 
despite all the clunkiness inherent in the design and the later 
difficulties imposed by greater browser restrictions. 
 
Regards,
Saq

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/7ea4cca2-c07a-486d-a60c-b59c9838c0bdn%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Jeremy Ruston]

Hi Saq

>  I briefly looked at twederation but it didn't meet this particular 
> requirement which would be very helpful for my intended purposes: 
> "...preferably without needing to have extra plugins installed on the wiki 
> that is the content source".
> 
> It also wasn't clear to me from the ReadMe or the demo whether fetching 
> tiddlers silently in the background with no user interaction was possible, 
> which is a must for the workflow I will need.

It did indeed fetch without user interaction. The Twederation prototype was all 
about using iframes to get around CORS restrictions.

Those restrictions still exist. For example, the XMLHttpRequest approach 
described here generally won’t work from a file: URI, which was a major design 
goal for the plugin library and Twederation.

> This is really just a quick hack to understand if what I am envisioning is 
> technically possible before I start working out the workflow. Now I need to 
> find the time to flesh out the desired user workflow which is the real 
> challenge and then I'll return to the technical implementation, which may 
> involve use of the core syncer. I decided to share this demo in case it was 
> interesting for others.

It’s definitely time we explored dynamic content loading via XMLHttpRequest in 
more detail. The constraints get less onerous as we gain better HTTP/HTTPS 
solutions.

Best wishes

Jeremy

>  
> Cheers,
> Saq
> 
> On Sunday, April 4, 2021 at 6:39:13 PM UTC+2 Ste wrote:
> Jumps in, mentions twederation 
> , jumps out again. 
> 
> On Sunday, 4 April 2021 at 10:28:31 UTC+1 Mohammad wrote:
> On Sun, Apr 4, 2021 at 1:38 PM Saq Imtiaz > wrote:
> Hi Mohammad,
> 
> Hi Saq,
> 
> So a lot of things are still up in the air. As I mentioned this is a quick 
> prototype without a lot of pre-planning and I haven't had the chance to 
> figure out a good workflow yet even for my specific needs. I'll try to answer 
> your questions as well as I can at this point,
>  
> but some things we may need to discuss and figure out together.
> 
> Sure!
>  
> One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis 
> ) it uses index tiddlers from 
> target wikis to search them
> The question is: Can I use Get Remote Tiddlers plugin to automatically update 
> the indeed tiddlers in my main wiki i.e to fetch latest changes?
> 
> Yes that should be possible as long as the wikis are on the same domain or 
> served with CORS headers. The catch is that for every wiki, the code will be 
> fetching the entire TW file to extract the tiddlers, which means more 
> bandwidth used and also possibly some lag in the wiki depending on the number 
> of wikis being fetched and the hardware. I want to experiment with using the 
> same technique with an uglified TW with Flibbles' plugin, or a TW with the 
> core in an external js file. Especially the second option should load 
> considerably faster.
> 
> I did not know the whole wiki is fetched, then yes for many external wikis or 
> large wikis we will have some time lags! I use the Flibbles uglify, I am very 
> happy with that, but for this case still the empty.html is around 1.2MB, the 
> external.js seems promising here!
> 
> Another option that might make the most sense for Searchwikis is to generate 
> and save the index as a separate file. This would need far less bandwidth to 
> fetch and should overall be considerably faster. For wikis served by Github 
> pages we could even try to implement an option to save the index file 
> automatically each time the wiki is saved.
> 
> Great! Right now I have to open each target wiki separately, update the 
> index, export it, and import to the central wiki!
>  
> A more radical question is can " Get Remote Tiddlers " deprecate searchwikis 
> and lets users search other wikis from a central wiki?
> 
> Fetching tiddlers from a remote wiki already entails loading the entire wiki 
> and extracting all its tiddlers and creating an array of tiddler titles. So 
> yes, theoretically you could skip the part of Searchwikis that creates the 
> index and just use Get Remote Tiddlers to create the index for the other 
> wikis. However, the same caveats apply in terms of bandwidth usage and that 
> with many wikis being fetched in the background it could slow down the wiki 
> when that is happening. Using TW files with an external core.js file would be 
> very helpful here.
> 
> Having to rely on CORS headers being present is indeed a limitation but this 
> overall approach (rather than the code I posted specifically) opens some 
> interesting possibilities as well.
> 
> If you're interested, there's a small discussion here about enabling CORS on 
> TiddlyHost: https://github.com/simonbaird/tiddlyhost/issues/112 
> 
> 
> I will!
> 
> I am also curious to see if TW on Fission opens up for similar workflows.
> 
> Cheers,
> Saq
> 

Re: [tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

Thanks @Ste

 I briefly looked at twederation but it didn't meet this particular 
requirement which would be very helpful for my intended purposes: 
"...preferably without needing to have extra plugins installed on the wiki 
that is the content source".

It also wasn't clear to me from the ReadMe or the demo whether fetching 
tiddlers silently in the background with no user interaction was possible, 
which is a must for the workflow I will need.

This is really just a quick hack to understand if what I am envisioning is 
technically possible before I start working out the workflow. Now I need to 
find the time to flesh out the desired user workflow which is the real 
challenge and then I'll return to the technical implementation, which may 
involve use of the core syncer. I decided to share this demo in case it was 
interesting for others.
 
Cheers,
Saq

On Sunday, April 4, 2021 at 6:39:13 PM UTC+2 Ste wrote:

> Jumps in, mentions twederation 
> , jumps out again. 
>
> On Sunday, 4 April 2021 at 10:28:31 UTC+1 Mohammad wrote:
>
>> On Sun, Apr 4, 2021 at 1:38 PM Saq Imtiaz  wrote:
>>
>>> Hi Mohammad,
>>
>>
>> Hi Saq,
>>
>>>
>>> So a lot of things are still up in the air. As I mentioned this is a 
>>> quick prototype without a lot of pre-planning and I haven't had the chance 
>>> to figure out a good workflow yet even for my specific needs. I'll try to 
>>> answer your questions as well as I can at this point,
>>>
>>  
>>
>>> but some things we may need to discuss and figure out together.
>>>
>>
>> Sure!
>>
>>>  
>>>
 One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis) 
 it uses index tiddlers from target wikis to search them
 The question is: Can I use Get Remote Tiddlers plugin to automatically 
 update the indeed tiddlers in my main wiki i.e to fetch latest changes?

>>>
>>> Yes that should be possible as long as the wikis are on the same domain 
>>> or served with CORS headers. The catch is that for every wiki, the code 
>>> will be fetching the entire TW file to extract the tiddlers, which means 
>>> more bandwidth used and also possibly some lag in the wiki depending on the 
>>> number of wikis being fetched and the hardware. I want to experiment with 
>>> using the same technique with an uglified TW with Flibbles' plugin, or a TW 
>>> with the core in an external js file. Especially the second option should 
>>> load considerably faster.
>>>
>>
>> I did not know the whole wiki is fetched, then yes for many external 
>> wikis or large wikis we will have some time lags! I use the Flibbles 
>> uglify, I am very happy with that, but for this case still the empty.html 
>> is around 1.2MB, the external.js seems promising here!
>>
>>>
>>> Another option that might make the most sense for Searchwikis is to 
>>> generate and save the index as a separate file. This would need far less 
>>> bandwidth to fetch and should overall be considerably faster. For wikis 
>>> served by Github pages we could even try to implement an option to save the 
>>> index file automatically each time the wiki is saved.
>>>
>>
>> Great! Right now I have to open each target wiki separately, update the 
>> index, export it, and import to the central wiki!
>>
>>>  
>>>
 A more radical question is can " Get Remote Tiddlers " deprecate 
 searchwikis and lets users search other wikis from a central wiki?

>>>
>>> Fetching tiddlers from a remote wiki already entails loading the entire 
>>> wiki and extracting all its tiddlers and creating an array of tiddler 
>>> titles. So yes, theoretically you could skip the part of Searchwikis that 
>>> creates the index and just use Get Remote Tiddlers to create the index for 
>>> the other wikis. However, the same caveats apply in terms of bandwidth 
>>> usage and that with many wikis being fetched in the background it could 
>>> slow down the wiki when that is happening. Using TW files with an external 
>>> core.js file would be very helpful here.
>>>
>>> Having to rely on CORS headers being present is indeed a limitation but 
>>> this overall approach (rather than the code I posted specifically) opens 
>>> some interesting possibilities as well.
>>>
>>> If you're interested, there's a small discussion here about enabling 
>>> CORS on TiddlyHost: https://github.com/simonbaird/tiddlyhost/issues/112
>>>
>>
>> I will!
>>
>>>
>>> I am also curious to see if TW on Fission opens up for similar workflows.
>>>
>>> Cheers,
>>> Saq
>>>
>>
>>
>> Thank you Saq!
>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "TiddlyWiki" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to tiddlywiki+...@googlegroups.com.
>>>
>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/tiddlywiki/a30adfd0-8d9f-42d8-aea6-e0824c777db6n%40googlegroups.com
>>>  
>>> 

Re: [tw5] Fetching tiddlers from one wiki to another

[Ste]

Jumps in, mentions twederation 
, jumps out again. 

On Sunday, 4 April 2021 at 10:28:31 UTC+1 Mohammad wrote:

> On Sun, Apr 4, 2021 at 1:38 PM Saq Imtiaz  wrote:
>
>> Hi Mohammad,
>
>
> Hi Saq,
>
>>
>> So a lot of things are still up in the air. As I mentioned this is a 
>> quick prototype without a lot of pre-planning and I haven't had the chance 
>> to figure out a good workflow yet even for my specific needs. I'll try to 
>> answer your questions as well as I can at this point,
>>
>  
>
>> but some things we may need to discuss and figure out together.
>>
>
> Sure!
>
>>  
>>
>>> One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis) 
>>> it uses index tiddlers from target wikis to search them
>>> The question is: Can I use Get Remote Tiddlers plugin to automatically 
>>> update the indeed tiddlers in my main wiki i.e to fetch latest changes?
>>>
>>
>> Yes that should be possible as long as the wikis are on the same domain 
>> or served with CORS headers. The catch is that for every wiki, the code 
>> will be fetching the entire TW file to extract the tiddlers, which means 
>> more bandwidth used and also possibly some lag in the wiki depending on the 
>> number of wikis being fetched and the hardware. I want to experiment with 
>> using the same technique with an uglified TW with Flibbles' plugin, or a TW 
>> with the core in an external js file. Especially the second option should 
>> load considerably faster.
>>
>
> I did not know the whole wiki is fetched, then yes for many external wikis 
> or large wikis we will have some time lags! I use the Flibbles uglify, I am 
> very happy with that, but for this case still the empty.html is around 
> 1.2MB, the external.js seems promising here!
>
>>
>> Another option that might make the most sense for Searchwikis is to 
>> generate and save the index as a separate file. This would need far less 
>> bandwidth to fetch and should overall be considerably faster. For wikis 
>> served by Github pages we could even try to implement an option to save the 
>> index file automatically each time the wiki is saved.
>>
>
> Great! Right now I have to open each target wiki separately, update the 
> index, export it, and import to the central wiki!
>
>>  
>>
>>> A more radical question is can " Get Remote Tiddlers " deprecate 
>>> searchwikis and lets users search other wikis from a central wiki?
>>>
>>
>> Fetching tiddlers from a remote wiki already entails loading the entire 
>> wiki and extracting all its tiddlers and creating an array of tiddler 
>> titles. So yes, theoretically you could skip the part of Searchwikis that 
>> creates the index and just use Get Remote Tiddlers to create the index for 
>> the other wikis. However, the same caveats apply in terms of bandwidth 
>> usage and that with many wikis being fetched in the background it could 
>> slow down the wiki when that is happening. Using TW files with an external 
>> core.js file would be very helpful here.
>>
>> Having to rely on CORS headers being present is indeed a limitation but 
>> this overall approach (rather than the code I posted specifically) opens 
>> some interesting possibilities as well.
>>
>> If you're interested, there's a small discussion here about enabling CORS 
>> on TiddlyHost: https://github.com/simonbaird/tiddlyhost/issues/112
>>
>
> I will!
>
>>
>> I am also curious to see if TW on Fission opens up for similar workflows.
>>
>> Cheers,
>> Saq
>>
>
>
> Thank you Saq!
>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "TiddlyWiki" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to tiddlywiki+...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/tiddlywiki/a30adfd0-8d9f-42d8-aea6-e0824c777db6n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/c8eb6ed1-0046-441a-9d82-0c022fed248en%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Mohammad Rahmani]

On Sun, Apr 4, 2021 at 1:38 PM Saq Imtiaz  wrote:

> Hi Mohammad,


Hi Saq,

>
> So a lot of things are still up in the air. As I mentioned this is a quick
> prototype without a lot of pre-planning and I haven't had the chance to
> figure out a good workflow yet even for my specific needs. I'll try to
> answer your questions as well as I can at this point,
>


> but some things we may need to discuss and figure out together.
>

Sure!

>
>
>> One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis)
>> it uses index tiddlers from target wikis to search them
>> The question is: Can I use Get Remote Tiddlers plugin to automatically
>> update the indeed tiddlers in my main wiki i.e to fetch latest changes?
>>
>
> Yes that should be possible as long as the wikis are on the same domain or
> served with CORS headers. The catch is that for every wiki, the code will
> be fetching the entire TW file to extract the tiddlers, which means more
> bandwidth used and also possibly some lag in the wiki depending on the
> number of wikis being fetched and the hardware. I want to experiment with
> using the same technique with an uglified TW with Flibbles' plugin, or a TW
> with the core in an external js file. Especially the second option should
> load considerably faster.
>

I did not know the whole wiki is fetched, then yes for many external wikis
or large wikis we will have some time lags! I use the Flibbles uglify, I am
very happy with that, but for this case still the empty.html is around
1.2MB, the external.js seems promising here!

>
> Another option that might make the most sense for Searchwikis is to
> generate and save the index as a separate file. This would need far less
> bandwidth to fetch and should overall be considerably faster. For wikis
> served by Github pages we could even try to implement an option to save the
> index file automatically each time the wiki is saved.
>

Great! Right now I have to open each target wiki separately, update the
index, export it, and import to the central wiki!

>
>
>> A more radical question is can " Get Remote Tiddlers " deprecate
>> searchwikis and lets users search other wikis from a central wiki?
>>
>
> Fetching tiddlers from a remote wiki already entails loading the entire
> wiki and extracting all its tiddlers and creating an array of tiddler
> titles. So yes, theoretically you could skip the part of Searchwikis that
> creates the index and just use Get Remote Tiddlers to create the index for
> the other wikis. However, the same caveats apply in terms of bandwidth
> usage and that with many wikis being fetched in the background it could
> slow down the wiki when that is happening. Using TW files with an external
> core.js file would be very helpful here.
>
> Having to rely on CORS headers being present is indeed a limitation but
> this overall approach (rather than the code I posted specifically) opens
> some interesting possibilities as well.
>
> If you're interested, there's a small discussion here about enabling CORS
> on TiddlyHost: https://github.com/simonbaird/tiddlyhost/issues/112
>

I will!

>
> I am also curious to see if TW on Fission opens up for similar workflows.
>
> Cheers,
> Saq
>


Thank you Saq!

> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tiddlywiki+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tiddlywiki/a30adfd0-8d9f-42d8-aea6-e0824c777db6n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/CAAV1gMApi7STpRFofsx3Uq3inNk6B1R0u%3Dsbp%2BEqQa2xXurzJw%40mail.gmail.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

Hi Mohammad,

So a lot of things are still up in the air. As I mentioned this is a quick 
prototype without a lot of pre-planning and I haven't had the chance to 
figure out a good workflow yet even for my specific needs. I'll try to 
answer your questions as well as I can at this point, but some things we 
may need to discuss and figure out together.
 

> One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis) 
> it uses index tiddlers from target wikis to search them
> The question is: Can I use Get Remote Tiddlers plugin to automatically 
> update the indeed tiddlers in my main wiki i.e to fetch latest changes?
>

Yes that should be possible as long as the wikis are on the same domain or 
served with CORS headers. The catch is that for every wiki, the code will 
be fetching the entire TW file to extract the tiddlers, which means more 
bandwidth used and also possibly some lag in the wiki depending on the 
number of wikis being fetched and the hardware. I want to experiment with 
using the same technique with an uglified TW with Flibbles' plugin, or a TW 
with the core in an external js file. Especially the second option should 
load considerably faster.

Another option that might make the most sense for Searchwikis is to 
generate and save the index as a separate file. This would need far less 
bandwidth to fetch and should overall be considerably faster. For wikis 
served by Github pages we could even try to implement an option to save the 
index file automatically each time the wiki is saved.
 

> A more radical question is can " Get Remote Tiddlers " deprecate 
> searchwikis and lets users search other wikis from a central wiki?
>

Fetching tiddlers from a remote wiki already entails loading the entire 
wiki and extracting all its tiddlers and creating an array of tiddler 
titles. So yes, theoretically you could skip the part of Searchwikis that 
creates the index and just use Get Remote Tiddlers to create the index for 
the other wikis. However, the same caveats apply in terms of bandwidth 
usage and that with many wikis being fetched in the background it could 
slow down the wiki when that is happening. Using TW files with an external 
core.js file would be very helpful here.

Having to rely on CORS headers being present is indeed a limitation but 
this overall approach (rather than the code I posted specifically) opens 
some interesting possibilities as well.

If you're interested, there's a small discussion here about enabling CORS 
on TiddlyHost: https://github.com/simonbaird/tiddlyhost/issues/112

I am also curious to see if TW on Fission opens up for similar workflows.

Cheers,
Saq

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/a30adfd0-8d9f-42d8-aea6-e0824c777db6n%40googlegroups.com.

Re: [tw5] Fetching tiddlers from one wiki to another

[Mohammad Rahmani]

Hi Saq,
 This is very exciting and opens a lot of new windows for Tiddlywiki.

One question: In Searchwikis (https://github.com/kookma/TW-Searchwikis) it
uses index tiddlers from target wikis to search them
The question is: Can I use Get Remote Tiddlers plugin to automatically
update the indeed tiddlers in my main wiki i.e to fetch latest changes?

A more radical question is can " Get Remote Tiddlers " deprecate
searchwikis and lets users search other wikis from a central wiki?


Best wishes
Mohammad


On Sun, Apr 4, 2021 at 10:24 AM Saq Imtiaz  wrote:

> I've been contemplating a project where if I were to use TiddlyWiki, I
> would need to be able to fetch content from other wikis on startup, and
> preferably without needing to have extra plugins installed on the wiki that
> is the content source.
>
> I've created a quick and messy proof of concept. This has only a very
> narrow use case in that it requires that both wikis either be on the same
> domain, or that the wiki that is the source of content be served with CORS
> headers. I am sharing it in case anyone else finds it of interest.
>
> Warning: this is very raw and incomplete. I spent about 40 minutes hacking
> this into place - with a lot of déjà vu moments thinking of working with
> FND and prototyping a plugin update system back in the TW 2.x days - and
> then maybe just as long preparing a demo:
>
> https://saqimtiaz.github.io/sq-tw/temp/get-remote-tiddlers.html
>
> What does make this a little exciting is that Github pages supports CORS
> and with a little luck we might get CORS support at TiddlyHost too. So it
> opens the door to some of the workflow that was commonplace in the TWC
> days, like auto-importing content, updates to certain tiddlers, plugin auto
> updates etc.
>
> In preparing the demo I also realized that it would be useful to be able
> to bypass the cache when fetching the content.
>
> Cheers,
> Saq
>
> --
> You received this message because you are subscribed to the Google Groups
> "TiddlyWiki" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tiddlywiki+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tiddlywiki/20ade4fa-4f1e-4569-8133-3bf3cd4ad1e4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/CAAV1gMAXRjnFzd9Jq%2BFyexfLzf-PdrNtq9NbFupFF6wBJ8FbvQ%40mail.gmail.com.

[tw5] Fetching tiddlers from one wiki to another

[Saq Imtiaz]

I've been contemplating a project where if I were to use TiddlyWiki, I 
would need to be able to fetch content from other wikis on startup, and 
preferably without needing to have extra plugins installed on the wiki that 
is the content source.

I've created a quick and messy proof of concept. This has only a very 
narrow use case in that it requires that both wikis either be on the same 
domain, or that the wiki that is the source of content be served with CORS 
headers. I am sharing it in case anyone else finds it of interest.

Warning: this is very raw and incomplete. I spent about 40 minutes hacking 
this into place - with a lot of déjà vu moments thinking of working with 
FND and prototyping a plugin update system back in the TW 2.x days - and 
then maybe just as long preparing a demo:

https://saqimtiaz.github.io/sq-tw/temp/get-remote-tiddlers.html

What does make this a little exciting is that Github pages supports CORS 
and with a little luck we might get CORS support at TiddlyHost too. So it 
opens the door to some of the workflow that was commonplace in the TWC 
days, like auto-importing content, updates to certain tiddlers, plugin auto 
updates etc.

In preparing the demo I also realized that it would be useful to be able to 
bypass the cache when fetching the content. 

Cheers,
Saq

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywiki+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywiki/20ade4fa-4f1e-4569-8133-3bf3cd4ad1e4n%40googlegroups.com.