Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Evan Balster]

Hey, Jeremy —

With what you've told me about the preboot approach on the call, it's 
starting to sound a bit better for modloader's needs.  Learning that the 
serialization DOM format is fixed across all TW5 platforms helps me to 
visualize a solution.

I'm wondering if this pre-boot approach will be incompatible with encrypted 
wikis, though.  I have at least one of those that I'd really like to use 
patches with, if possible.  Are plugins (including core) encrypted, or are 
they excluded from that mechanism?


* The proposed modifications don’t have any anticipated benefit beyond the 
> usage by the modloader plugin


I will continue to voice my dissent to this argument.  There are other 
useful mechanisms that might want to add code before startup.  For example, 
a plugin could fetch some widget script via HTTPS in order to ensure that 
client wikis are always using the latest version.  Arguably this could be 
done in pre-boot like the modloader, too, but it's an example of something 
that needs to deal with similar constraints.



On Tuesday, 13 February 2018 05:57:07 UTC-6, Jeremy Ruston wrote:
>
> Hi Evan
>
> The other problem is that my plugin's requirement — "minimize the number 
> of modules executing before me" — is at odds with the dependency-oriented 
> nature of startup modules.  Modloader doesn't depend on any functionality 
> other than what's provided by the bootloader.
>
>
> “Minimize the number of modules executing before me” is the key phrase 
> that triggers my concern about an arms race; I’ve seen it so many times 
> before from CSS to TSRs, and introducing another category is never a good 
> long term answer. The pre-boot approach is attractive precisely because 
> it’s outside the normal mechanisms of TiddlyWiki, and hence can be used to 
> patch/hack those mechanisms. It is there to help avoid the kind of 
> patching-the-rug-you’re-standing-on situation we face here.
>
> Aren’t (the executions before startup) just the cascade of modules 
>> requiring one another?
>>
>
> Yes.  The requirement for my plugin is "run before most modules execute".  
> There's no best practice for startup modules to discourage them from 
> loading modules at execution time, and I'm not sure there should be(?), 
> hence the conundrum.
>
> You'll notice my modloader plugin eclipses startup.js to move its 
> require() call into its startup() method.  If I don't do this, it's 
> impossible to modify a large portion of the core (including all widgets).  
> Unfortunately, this eclipsing is exactly the kind of thing the modloader is 
> designed to avoid.
>
>
> I’ve no objection to moving existing require() calls into a different 
> scope.
>
> It might be a useful exercise to try to write the documentation for 
>> bootloader modules, and see whether one can explain the difference clearly 
>> without referencing the particular use case that has prompted this 
>> discussion.
>>
>
> "bootloader modules run during the boot process, allowing them to add, 
> remove and modify tiddlers in the store prior to the wiki's startup 
> process.  This is useful for injecting plugins, code and content that 
> TiddlyWiki isn't capable of refreshing at runtime.  Injected content may be 
> dynamically generated or loaded from an external source.  These modules are 
> encouraged to minimize their dependency upon other modules in the wiki and 
> to avoid any dependency on the TiddlyWiki core.
>
> bootloader modules may safely revise any content in the tiddler store with 
> the exception of other bootloader modules and their dependencies.”
>
>
> And that sounds pretty much like a description of the existing 
> $tw.preloadTiddlers mechanism. The preboot approach offers the interesting 
> possibility of patching boot.js, too.
>
> I’m happy to discuss further but my position remains:
>
> * The proposed modifications don’t have any anticipated benefit beyond the 
> usage by the modloader plugin
> * The proposed modifications are relatively expensive in terms of time and 
> attention from me and the rest of the core team
> * The modloader plugin already exists and works, with some limitations
> * The preboot approach already exists as an alternative mechanism that 
> could be used instead, with different limitations
>
> Best wishes
>
> Jeremy
>
>
>  
>
> On Monday, 12 February 2018 16:53:53 UTC-6, Jeremy Ruston wrote:
>>
>> Hi Evan
>>
>> Thanks, I’ll give your arguments more thought, and hope you can join the 
>> hangout tomorrow to discuss further.
>>
>> This is almost, but not quite, the issue.  My modloader *is* the first 
>> module to have any method called on it.  However, TiddlyWiki currently 
>> *loads* (via $tw.module.execute) a very large number of modules prior to 
>> making any method calls.  There is no control over the ordering of these 
>> executions, and they are subsequently impossible to change at boot-time 
>> without overwriting the core.
>>
>>
>> Aren’t those executions just the cascade of modules requiring one another

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Jeremy Ruston]

Hi Evan

> The other problem is that my plugin's requirement — "minimize the number of 
> modules executing before me" — is at odds with the dependency-oriented nature 
> of startup modules.  Modloader doesn't depend on any functionality other than 
> what's provided by the bootloader.

“Minimize the number of modules executing before me” is the key phrase that 
triggers my concern about an arms race; I’ve seen it so many times before from 
CSS to TSRs, and introducing another category is never a good long term answer. 
The pre-boot approach is attractive precisely because it’s outside the normal 
mechanisms of TiddlyWiki, and hence can be used to patch/hack those mechanisms. 
It is there to help avoid the kind of patching-the-rug-you’re-standing-on 
situation we face here.

> Aren’t (the executions before startup) just the cascade of modules requiring 
> one another?
> 
> Yes.  The requirement for my plugin is "run before most modules execute".  
> There's no best practice for startup modules to discourage them from loading 
> modules at execution time, and I'm not sure there should be(?), hence the 
> conundrum.
> 
> You'll notice my modloader plugin eclipses startup.js to move its require() 
> call into its startup() method.  If I don't do this, it's impossible to 
> modify a large portion of the core (including all widgets).  Unfortunately, 
> this eclipsing is exactly the kind of thing the modloader is designed to 
> avoid.

I’ve no objection to moving existing require() calls into a different scope.

> It might be a useful exercise to try to write the documentation for 
> bootloader modules, and see whether one can explain the difference clearly 
> without referencing the particular use case that has prompted this discussion.
> 
> "bootloader modules run during the boot process, allowing them to add, remove 
> and modify tiddlers in the store prior to the wiki's startup process.  This 
> is useful for injecting plugins, code and content that TiddlyWiki isn't 
> capable of refreshing at runtime.  Injected content may be dynamically 
> generated or loaded from an external source.  These modules are encouraged to 
> minimize their dependency upon other modules in the wiki and to avoid any 
> dependency on the TiddlyWiki core.
> 
> bootloader modules may safely revise any content in the tiddler store with 
> the exception of other bootloader modules and their dependencies.”

And that sounds pretty much like a description of the existing 
$tw.preloadTiddlers mechanism. The preboot approach offers the interesting 
possibility of patching boot.js, too.

I’m happy to discuss further but my position remains:

* The proposed modifications don’t have any anticipated benefit beyond the 
usage by the modloader plugin
* The proposed modifications are relatively expensive in terms of time and 
attention from me and the rest of the core team
* The modloader plugin already exists and works, with some limitations
* The preboot approach already exists as an alternative mechanism that could be 
used instead, with different limitations

Best wishes

Jeremy

> 
>  
> 
> On Monday, 12 February 2018 16:53:53 UTC-6, Jeremy Ruston wrote:
> Hi Evan
> 
> Thanks, I’ll give your arguments more thought, and hope you can join the 
> hangout tomorrow to discuss further.
> 
>> This is almost, but not quite, the issue.  My modloader is the first module 
>> to have any method called on it.  However, TiddlyWiki currently loads (via 
>> $tw.module.execute) a very large number of modules prior to making any 
>> method calls.  There is no control over the ordering of these executions, 
>> and they are subsequently impossible to change at boot-time without 
>> overwriting the core.
> 
> Aren’t those executions just the cascade of modules requiring one another?
> 
>> As I mentioned before, I think there's potentially a meaningful difference 
>> between startup modules and "bootloader" behaviors.  Loading content and 
>> code into the tiddler store, versus setting up APIs, DOMs and run-time 
>> functionality.  It's much more likely that a loader module would want to 
>> affect startup behavior than another loader module.
> 
> Bootloader modules feel a bit like an admission of defeat with respect to the 
> startup module mechanism.
> 
> It might be a useful exercise to try to write the documentation for 
> bootloader modules, and see whether one can explain the difference clearly 
> without referencing the particular use case that has prompted this discussion.
> 
> I worry that they are a narrow solution to your specific problem. We could 
> also be looking at something buggyj has already started: making it easier for 
> plugins to splice in raw markup code, and possibly adding more hooks to the 
> boot kernel (eg, an explicit “patch-tiddler” hook).
> 
>> Anyway, the modloader doesn't need this functionality, strictly speaking.  
>> It just extends its capabilities a bit, so that the whole core can be 
>> patched by installed plugins.  The mod

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Evan Balster]

Hey, Jeremy —

Bootloader modules feel a bit like an admission of defeat with respect to 
> the startup module mechanism.
>

Potentially this functionality could be an extension of the startup 
mechanism.  Trouble is, in order to load and execute them one-by-one you'd 
need to be able to prioritize startup modules without executing them, 
meaning that their dependency information would need to be migrated to 
fields.   This would break compatibility with any plugins that define 
startup modules.

The other problem is that my plugin's requirement — "minimize the number of 
modules executing before me" — is at odds with the dependency-oriented 
nature of startup modules.  Modloader doesn't depend on any functionality 
other than what's provided by the bootloader.


Aren’t (the executions before startup) just the cascade of modules 
> requiring one another?
>

Yes.  The requirement for my plugin is "run before most modules execute".  
There's no best practice for startup modules to discourage them from 
loading modules at execution time, and I'm not sure there should be(?), 
hence the conundrum.

You'll notice my modloader plugin eclipses startup.js to move its require() 
call into its startup() method.  If I don't do this, it's impossible to 
modify a large portion of the core (including all widgets).  Unfortunately, 
this eclipsing is exactly the kind of thing the modloader is designed to 
avoid.


It might be a useful exercise to try to write the documentation for 
> bootloader modules, and see whether one can explain the difference clearly 
> without referencing the particular use case that has prompted this 
> discussion.
>

"bootloader modules run during the boot process, allowing them to add, 
remove and modify tiddlers in the store prior to the wiki's startup 
process.  This is useful for injecting plugins, code and content that 
TiddlyWiki isn't capable of refreshing at runtime.  Injected content may be 
dynamically generated or loaded from an external source.  These modules are 
encouraged to minimize their dependency upon other modules in the wiki and 
to avoid any dependency on the TiddlyWiki core.

bootloader modules may safely revise any content in the tiddler store with 
the exception of other bootloader modules and their dependencies."

 

On Monday, 12 February 2018 16:53:53 UTC-6, Jeremy Ruston wrote:
>
> Hi Evan
>
> Thanks, I’ll give your arguments more thought, and hope you can join the 
> hangout tomorrow to discuss further.
>
> This is almost, but not quite, the issue.  My modloader *is* the first 
> module to have any method called on it.  However, TiddlyWiki currently 
> *loads* (via $tw.module.execute) a very large number of modules prior to 
> making any method calls.  There is no control over the ordering of these 
> executions, and they are subsequently impossible to change at boot-time 
> without overwriting the core.
>
>
> Aren’t those executions just the cascade of modules requiring one another?
>
> As I mentioned before, I think there's potentially a meaningful difference 
> between startup modules and "bootloader" behaviors.  Loading content and 
> code into the tiddler store, versus setting up APIs, DOMs and run-time 
> functionality.  It's much more likely that a loader module would want to 
> affect startup behavior than another loader module.
>
>
> Bootloader modules feel a bit like an admission of defeat with respect to 
> the startup module mechanism.
>
> It might be a useful exercise to try to write the documentation for 
> bootloader modules, and see whether one can explain the difference clearly 
> without referencing the particular use case that has prompted this 
> discussion.
>
> I worry that they are a narrow solution to your specific problem. We could 
> also be looking at something buggyj has already started: making it easier 
> for plugins to splice in raw markup code, and possibly adding more hooks to 
> the boot kernel (eg, an explicit “patch-tiddler” hook).
>
> Anyway, the modloader doesn't *need* this functionality, strictly 
> speaking.  It just extends its capabilities a bit, so that the *whole* core 
> can be patched by installed plugins.  The modloader currently issues a 
> warning when trying (and failing) to apply a patch of this nature so it 
> shouldn't catch developers by surprise.
>
>
> Excellent.
>
> The design intent of the modloader is to maintain compatibility with 
> future versions of TiddlyWiki.  Duplicating bootloader behavior (which is 
> subject to change) would detrimental enough to that goal that I'd rather 
> stick with the current constraints.
>
>
> But I think the code that you’d need to duplicate is code that’s only 
> going to change if the underlying serialisation formats change? If that 
> happens all bets are off with respect to future versions of TiddlyWiki 
> maintaining backwards compatibility with this February 2018 incarnation of 
> the plugin.
>
> Best wishes
>
> Jeremy.
>
>
>
> On Monday, 12 February 2018 15:10:16 UTC-6,

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Jeremy Ruston]

Hi Evan

Thanks, I’ll give your arguments more thought, and hope you can join the 
hangout tomorrow to discuss further.

> This is almost, but not quite, the issue.  My modloader is the first module 
> to have any method called on it.  However, TiddlyWiki currently loads (via 
> $tw.module.execute) a very large number of modules prior to making any method 
> calls.  There is no control over the ordering of these executions, and they 
> are subsequently impossible to change at boot-time without overwriting the 
> core.

Aren’t those executions just the cascade of modules requiring one another?

> As I mentioned before, I think there's potentially a meaningful difference 
> between startup modules and "bootloader" behaviors.  Loading content and code 
> into the tiddler store, versus setting up APIs, DOMs and run-time 
> functionality.  It's much more likely that a loader module would want to 
> affect startup behavior than another loader module.

Bootloader modules feel a bit like an admission of defeat with respect to the 
startup module mechanism.

It might be a useful exercise to try to write the documentation for bootloader 
modules, and see whether one can explain the difference clearly without 
referencing the particular use case that has prompted this discussion.

I worry that they are a narrow solution to your specific problem. We could also 
be looking at something buggyj has already started: making it easier for 
plugins to splice in raw markup code, and possibly adding more hooks to the 
boot kernel (eg, an explicit “patch-tiddler” hook).

> Anyway, the modloader doesn't need this functionality, strictly speaking.  It 
> just extends its capabilities a bit, so that the whole core can be patched by 
> installed plugins.  The modloader currently issues a warning when trying (and 
> failing) to apply a patch of this nature so it shouldn't catch developers by 
> surprise.

Excellent.

> The design intent of the modloader is to maintain compatibility with future 
> versions of TiddlyWiki.  Duplicating bootloader behavior (which is subject to 
> change) would detrimental enough to that goal that I'd rather stick with the 
> current constraints.

But I think the code that you’d need to duplicate is code that’s only going to 
change if the underlying serialisation formats change? If that happens all bets 
are off with respect to future versions of TiddlyWiki maintaining backwards 
compatibility with this February 2018 incarnation of the plugin.

Best wishes

Jeremy.

> 
> 
> On Monday, 12 February 2018 15:10:16 UTC-6, Jeremy Ruston wrote:
> Hi Evan
> 
> We’ve got to the point where the changes you are proposing to the core are 
> quite deep, and have lots of implications for the rest of the core. It’s 
> going to mean a lot of work on verification and testing from me, even if 
> somebody else writes the code. I don’t think that burden is justified at 
> present.
> 
> Better to work with the plugin for a while, and explore the capabilities and 
> consequences before we commit to a new mechanism in the core.
> 
>> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach here. 
>>  The modloader process 
>> 
>>  uses quite a lot of $:/boot library functionality to identify and inspect 
>> shadow tiddlers as part of the patching process, and to generate the output 
>> plugin.
> 
> At this point, I don’t see any problem with your plugin duplicating a few 
> hundred lines of boot.js; eliminating such duplication would come firmly 
> under the heading of optimisation.
> 
>> I also worry that working on the pre-boot DOM would create incompatibilities 
>> with certain TiddlyWiki environments (but I don't understand this low level 
>> stuff very well). 
> 
> You can see all the code that depends on this stuff. The approach is solid.
> 
>> The current modloader can manipulate almost everything in the wiki except 
>> for startup modules and the modules they require, hence the request.
> 
> Yes, but your proposed solution is just an escalation in an ongoing arms race 
> to be the first module to run.
> 
>> In most cases, bootloader modules (including modloader) are going to want to 
>> generate any output to $:/temp or similar, where they won't create 
>> persistent changes in the wiki.
> 
> But equally, there’s no limit to the functionality in the boot kernel that 
> experimental plugins might want to tweak. Better to be based on a pre-boot 
> hook, and have the capability to hook anything, than to be dependent on the 
> hooks anticipated and provided by the core.
> 
> Best wishes
> 
> Jeremy.
> 
>> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach here. 
>>  The modloader process 
>> 
>>  uses quite a lot of $:/boot library functionality to identify and inspect 
>> shadow tiddlers a

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Evan Balster]

Hey, Jeremy —
 

> Yes, but your proposed solution is just an escalation in an ongoing arms 
> race to be the first module to run.


This is almost, but not quite, the issue.  My modloader *is* the first 
module to have any method called on it.  However, TiddlyWiki currently 
*loads* (via $tw.module.execute) a very large number of modules prior to 
making any method calls.  There is no control over the ordering of these 
executions, and they are subsequently impossible to change at boot-time 
without overwriting the core.

As I mentioned before, I think there's potentially a meaningful difference 
between startup modules and "bootloader" behaviors.  Loading content and 
code into the tiddler store, versus setting up APIs, DOMs and run-time 
functionality.  It's much more likely that a loader module would want to 
affect startup behavior than another loader module.


We’ve got to the point where the changes you are proposing to the core are 
> quite deep, and have lots of implications for the rest of the core. It’s 
> going to mean a lot of work on verification and testing from me, even if 
> somebody else writes the code. I don’t think that burden is justified at 
> present.
>

A minimal implementation of bootloader modules would look something like 
this:

// Run bootloader modules
$tw.modules.forEachModuleOfType("bootloader",function(title,module) {
if(module.boot) module.boot();
});

This would go right before "startup" modules.  Lack of prioritization and 
"one by one" execution is less than ideal but still preferable to the 
current constraints.

Anyway, the modloader doesn't *need* this functionality, strictly 
speaking.  It just extends its capabilities a bit, so that the *whole* core 
can be patched by installed plugins.  The modloader currently issues a 
warning when trying (and failing) to apply a patch of this nature so it 
shouldn't catch developers by surprise.


At this point, I don’t see any problem with your plugin duplicating a few 
> hundred lines of boot.js.
>

The design intent of the modloader is to maintain compatibility with future 
versions of TiddlyWiki.  Duplicating bootloader behavior (which is subject 
to change) would detrimental enough to that goal that I'd rather stick with 
the current constraints.


On Monday, 12 February 2018 15:10:16 UTC-6, Jeremy Ruston wrote:
>
> Hi Evan
>
> We’ve got to the point where the changes you are proposing to the core are 
> quite deep, and have lots of implications for the rest of the core. It’s 
> going to mean a lot of work on verification and testing from me, even if 
> somebody else writes the code. I don’t think that burden is justified at 
> present.
>
> Better to work with the plugin for a while, and explore the capabilities 
> and consequences before we commit to a new mechanism in the core.
>
> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach 
> here.  The modloader process 
> 
>  uses 
> quite a lot of $:/boot library functionality to identify and inspect shadow 
> tiddlers as part of the patching process, and to generate the output plugin.
>
>
> At this point, I don’t see any problem with your plugin duplicating a few 
> hundred lines of boot.js; eliminating such duplication would come firmly 
> under the heading of optimisation.
>
> I also worry that working on the pre-boot DOM would create 
> incompatibilities with certain TiddlyWiki environments (but I don't 
> understand this low level stuff very well). 
>
>
> You can see all the code that depends on this stuff. The approach is solid.
>
> The current modloader can manipulate almost everything in the wiki 
> *except *for startup modules and the modules they require, hence the 
> request.
>
>
> Yes, but your proposed solution is just an escalation in an ongoing arms 
> race to be the first module to run.
>
> In most cases, bootloader modules (including modloader) are going to want 
> to generate any output to $:/temp or similar, where they won't create 
> persistent changes in the wiki.
>
>
> But equally, there’s no limit to the functionality in the boot kernel that 
> experimental plugins might want to tweak. Better to be based on a pre-boot 
> hook, and have the capability to hook anything, than to be dependent on the 
> hooks anticipated and provided by the core.
>
> Best wishes
>
> Jeremy.
>
> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach 
> here.  The modloader process 
> 
>  uses 
> quite a lot of $:/boot library functionality to identify and inspect shadow 
> tiddlers as part of the patching process, and to generate the output 
> plugin.  I also worry that working on the pre-boot DOM would create 
> incompatibilities with certain TiddlyWiki environments (but I don't 
> understand this low level stuff very well).  The current mod

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Jeremy Ruston]

Hi Evan

We’ve got to the point where the changes you are proposing to the core are 
quite deep, and have lots of implications for the rest of the core. It’s going 
to mean a lot of work on verification and testing from me, even if somebody 
else writes the code. I don’t think that burden is justified at present.

Better to work with the plugin for a while, and explore the capabilities and 
consequences before we commit to a new mechanism in the core.

> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach here.  
> The modloader process 
> 
>  uses quite a lot of $:/boot library functionality to identify and inspect 
> shadow tiddlers as part of the patching process, and to generate the output 
> plugin.

At this point, I don’t see any problem with your plugin duplicating a few 
hundred lines of boot.js; eliminating such duplication would come firmly under 
the heading of optimisation.

> I also worry that working on the pre-boot DOM would create incompatibilities 
> with certain TiddlyWiki environments (but I don't understand this low level 
> stuff very well).

You can see all the code that depends on this stuff. The approach is solid.

> The current modloader can manipulate almost everything in the wiki except for 
> startup modules and the modules they require, hence the request.

Yes, but your proposed solution is just an escalation in an ongoing arms race 
to be the first module to run.

> In most cases, bootloader modules (including modloader) are going to want to 
> generate any output to $:/temp or similar, where they won't create persistent 
> changes in the wiki.

But equally, there’s no limit to the functionality in the boot kernel that 
experimental plugins might want to tweak. Better to be based on a pre-boot 
hook, and have the capability to hook anything, than to be dependent on the 
hooks anticipated and provided by the core.

Best wishes

Jeremy.

> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach here.  
> The modloader process 
> 
>  uses quite a lot of $:/boot library functionality to identify and inspect 
> shadow tiddlers as part of the patching process, and to generate the output 
> plugin.  I also worry that working on the pre-boot DOM would create 
> incompatibilities with certain TiddlyWiki environments (but I don't 
> understand this low level stuff very well).  The current modloader can 
> manipulate almost everything in the wiki except for startup modules and the 
> modules they require, hence the request.
> 
> In most cases, bootloader modules (including modloader) are going to want to 
> generate any output to $:/temp or similar, where they won't create persistent 
> changes in the wiki.
> 
> I expect we'll end up discussing in more depth during tomorrow's call.
> 
> On Monday, 12 February 2018 14:09:17 UTC-6, Jeremy Ruston wrote:
> Here’s the .tid of the $:/PatchCore.js tiddler, too.
> 
> Best wishes
> 
> Jeremy
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikidev+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to tiddlywikidev@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/tiddlywikidev 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/98a6ce14-9ba5-470d-b396-365f08ddd806%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/5123518B-8F9C-4C58-9997-FF2BCB9964D7%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Evan Balster]

Hey, Jeremy —

So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach 
here.  The modloader process 

 uses 
quite a lot of $:/boot library functionality to identify and inspect shadow 
tiddlers as part of the patching process, and to generate the output 
plugin.  I also worry that working on the pre-boot DOM would create 
incompatibilities with certain TiddlyWiki environments (but I don't 
understand this low level stuff very well).  The current modloader can 
manipulate almost everything in the wiki *except *for startup modules and 
the modules they require, hence the request.

In most cases, bootloader modules (including modloader) are going to want 
to generate any output to $:/temp or similar, where they won't create 
persistent changes in the wiki.

I expect we'll end up discussing in more depth during tomorrow's call.

On Monday, 12 February 2018 14:09:17 UTC-6, Jeremy Ruston wrote:
>
> Here’s the .tid of the $:/PatchCore.js tiddler, too.
>
> Best wishes
>
> Jeremy
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/98a6ce14-9ba5-470d-b396-365f08ddd806%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Jeremy Ruston]

Here’s the .tid of the $:/PatchCore.js tiddler, too.Best wishesJeremy



-- 
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/BE3A2D59-D25D-4CFA-A458-496E5F4FDD9F%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


$__PatchCore.js.tid
Description: Binary data
On 12 Feb 2018, at 20:00, Jeremy Ruston  wrote:Hi EvanI worked up the technique I mentioned, and it may be suitable for you.Working in Node.js, create a file called “patchcore.js" in a test wiki with the following content:/*\title: $:/PatchCore.jstype: application/_javascript_library: yesDemo of patching the core plugin\*/(function(){/*jslint node: true, browser: true *//*global $tw: false */"use strict";var preCore = document.querySelector("div[title='$:/core'] > pre"),	pluginText = preCore.textContent;pluginText = pluginText.replace(/a non-linear personal web notebook/mg,"this is a patch");preCore.textContent = pluginText;window.$tw = window.$tw || {};$tw.preloadTiddlers = $tw.preloadTiddlers || [];$tw.preloadTiddlers.push({title: "HelloJeremy",text: "Hello there Jeremy"});})();What I was suggesting would take a little bit more code: instead of writing the modified plugin text back to the DOM, you could instead neutralise the DOM element (by targetting the DIV and removing the title attribute) and insert the tiddler via $tw.preloadTiddlers.Best wishesJeremyOn 12 Feb 2018, at 19:08, Evan Balster  wrote:Addendum:  If startup modules aren't collectively preloaded before they are run, boot modules may be unnecessary.  However, this would force a change in the prioritization mechanism (using tiddler fields rather than export variables).I think there might also be a meaningful difference in scope between bootloader functionality (modifying the tiddler store) and startup functionality (preparing functionality), but that's a loose opinion.On Monday, 12 February 2018 13:05:06 UTC-6, Evan Balster wrote:Hey, all —(I had a Twitter discussion with Jeremy yesterday about this topic which quickly overflowed the boundaries of that medium.)To facilitate development of my Mod-Loader plugin, I'm requesting a new module-type that runs during boot with maximal priority.  This module-type would be used for modules that need to add, remove or modify items in the tiddler store which become "permanent" during the boot process (IE, other _javascript_ modules).While the modloader is my main concern, I've given some thought to other use-cases for boot-time modification of the tiddler store.  The obvious one is "loader extensions":  That is, any plugin that loads tiddlers from a non-default source.  A boot module could load some content from a network or file source at boot-time, from the client-side.Boot modules could work in a fashion similar to startup modules, running first.  I suspect it might be preferable for boot modules to be loaded and run one-by-one and prioritized by field values rather than _javascript_-defined properties, though.  This would allow them to modify one another or inject other boot modules into the sequence (potentially by adding or modifying plugins).Jeremy suggested yesterday that high-priority code like this could be installed as raw markup tiddlers, but I'm a bit skeptical of this approach.  I think it's important that modules like this have access to tiddler store functionality, and remain portable across the many different formats TiddlyWiki can be packaged into.  I also think it should remain possible to package these boot modules into plugins.-- You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikidev+unsubscr...@googlegroups.com.To post to this group, send email to tiddlywikidev@googlegroups.com.Visit this group at https://groups.google.com/group/tiddlywikidev.To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/2a76bfeb-b709-46a9-9797-ab4acb82c9a8%40googlegroups.com.For more options, visit https://groups.google.com/d/optout.



-- 
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/BE3A2D59-D25D-4CFA-A458-496E5F4FDD9F%4

Re: [twdev] Re: Feature Request: high-priority "bootloader" module-type

[Jeremy Ruston]

Hi Evan

I worked up the technique I mentioned, and it may be suitable for you.

Working in Node.js, create a file called “patchcore.js" in a test wiki with the 
following content:

/*\
title: $:/PatchCore.js
type: application/javascript
library: yes

Demo of patching the core plugin

\*/
(function(){

/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";

var preCore = document.querySelector("div[title='$:/core'] > pre"),
pluginText = preCore.textContent;

pluginText = pluginText.replace(/a non-linear personal web notebook/mg,"this is 
a patch");
preCore.textContent = pluginText;

window.$tw = window.$tw || {};
$tw.preloadTiddlers = $tw.preloadTiddlers || [];
$tw.preloadTiddlers.push({title: "HelloJeremy",text: "Hello there Jeremy"});


})();

What I was suggesting would take a little bit more code: instead of writing the 
modified plugin text back to the DOM, you could instead neutralise the DOM 
element (by targetting the DIV and removing the title attribute) and insert the 
tiddler via $tw.preloadTiddlers.

Best wishes

Jeremy

> On 12 Feb 2018, at 19:08, Evan Balster  wrote:
> 
> Addendum:  If startup modules aren't collectively preloaded before they are 
> run, boot modules may be unnecessary.  However, this would force a change in 
> the prioritization mechanism (using tiddler fields rather than export 
> variables).
> 
> I think there might also be a meaningful difference in scope between 
> bootloader functionality (modifying the tiddler store) and startup 
> functionality (preparing functionality), but that's a loose opinion.
> 
> On Monday, 12 February 2018 13:05:06 UTC-6, Evan Balster wrote:
> Hey, all —
> 
> (I had a Twitter discussion 
>  with Jeremy 
> yesterday about this topic which quickly overflowed the boundaries of that 
> medium.)
> 
> To facilitate development of my Mod-Loader plugin 
> , I'm 
> requesting a new module-type that runs during boot with maximal priority.  
> This module-type would be used for modules that need to add, remove or modify 
> items in the tiddler store which become "permanent" during the boot process 
> (IE, other JavaScript modules).
> 
> While the modloader is my main concern, I've given some thought to other 
> use-cases for boot-time modification of the tiddler store.  The obvious one 
> is "loader extensions":  That is, any plugin that loads tiddlers from a 
> non-default source.  A boot module could load some content from a network or 
> file source at boot-time, from the client-side.
> 
> 
> Boot modules could work in a fashion similar to startup modules, running 
> first.  I suspect it might be preferable for boot modules to be loaded and 
> run one-by-one and prioritized by field values rather than JavaScript-defined 
> properties, though.  This would allow them to modify one another or inject 
> other boot modules into the sequence (potentially by adding or modifying 
> plugins).
> 
> Jeremy suggested yesterday that high-priority code like this could be 
> installed as raw markup tiddlers, but I'm a bit skeptical of this approach.  
> I think it's important that modules like this have access to tiddler store 
> functionality, and remain portable across the many different formats 
> TiddlyWiki can be packaged into.  I also think it should remain possible to 
> package these boot modules into plugins.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikidev+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to tiddlywikidev@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/tiddlywikidev 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/2a76bfeb-b709-46a9-9797-ab4acb82c9a8%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/1BB1CFD5-3C49-4611-A44E-B405F10379D8%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[twdev] Re: Feature Request: high-priority "bootloader" module-type

[Evan Balster]

Addendum:  If startup modules aren't collectively preloaded before they are 
run, boot modules may be unnecessary.  However, this would force a change 
in the prioritization mechanism (using tiddler fields rather than export 
variables).

I think there might also be a meaningful difference in scope between 
bootloader functionality (modifying the tiddler store) and startup 
functionality (preparing functionality), but that's a loose opinion.

On Monday, 12 February 2018 13:05:06 UTC-6, Evan Balster wrote:
>
> Hey, all —
>
> *(I had a Twitter discussion 
>  with Jeremy 
> yesterday about this topic which quickly overflowed the boundaries of that 
> medium.)*
>
> To facilitate development of my Mod-Loader plugin 
> , I'm 
> requesting a new module-type that runs during boot with maximal priority.  
> This module-type would be used for modules that need to add, remove or 
> modify items in the tiddler store which become "permanent" during the boot 
> process (IE, other JavaScript modules).
>
> While the modloader is my main concern, I've given some thought to other 
> use-cases for boot-time modification of the tiddler store.  The obvious one 
> is "loader extensions":  That is, any plugin that loads tiddlers from a 
> non-default source.  A boot module could load some content from a network 
> or file source at boot-time, from the client-side.
>
>
> Boot modules could work in a fashion similar to startup modules, running 
> first.  I suspect it might be preferable for boot modules to be loaded and 
> run one-by-one and prioritized by field values rather than 
> JavaScript-defined properties, though.  This would allow them to modify one 
> another or inject other boot modules into the sequence (potentially by 
> adding or modifying plugins).
>
> Jeremy suggested yesterday that high-priority code like this could be 
> installed as raw markup tiddlers, but I'm a bit skeptical of this 
> approach.  I think it's important that modules like this have access to 
> tiddler store functionality, and remain portable across the many different 
> formats TiddlyWiki can be packaged into.  I also think it should remain 
> possible to package these boot modules into plugins.
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/2a76bfeb-b709-46a9-9797-ab4acb82c9a8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.