Hey, Jeremy —

With what you've told me about the preboot approach on the call, it's 
starting to sound a bit better for modloader's needs.  Learning that the 
serialization DOM format is fixed across all TW5 platforms helps me to 
visualize a solution.

I'm wondering if this pre-boot approach will be incompatible with encrypted 
wikis, though.  I have at least one of those that I'd really like to use 
patches with, if possible.  Are plugins (including core) encrypted, or are 
they excluded from that mechanism?


* The proposed modifications don’t have any anticipated benefit beyond the 
> usage by the modloader plugin


I will continue to voice my dissent to this argument.  There are other 
useful mechanisms that might want to add code before startup.  For example, 
a plugin could fetch some widget script via HTTPS in order to ensure that 
client wikis are always using the latest version.  Arguably this could be 
done in pre-boot like the modloader, too, but it's an example of something 
that needs to deal with similar constraints.



On Tuesday, 13 February 2018 05:57:07 UTC-6, Jeremy Ruston wrote:
>
> Hi Evan
>
> The other problem is that my plugin's requirement — "minimize the number 
> of modules executing before me" — is at odds with the dependency-oriented 
> nature of startup modules.  Modloader doesn't depend on any functionality 
> other than what's provided by the bootloader.
>
>
> “Minimize the number of modules executing before me” is the key phrase 
> that triggers my concern about an arms race; I’ve seen it so many times 
> before from CSS to TSRs, and introducing another category is never a good 
> long term answer. The pre-boot approach is attractive precisely because 
> it’s outside the normal mechanisms of TiddlyWiki, and hence can be used to 
> patch/hack those mechanisms. It is there to help avoid the kind of 
> patching-the-rug-you’re-standing-on situation we face here.
>
> Aren’t (the executions before startup) just the cascade of modules 
>> requiring one another?
>>
>
> Yes.  The requirement for my plugin is "run before most modules execute".  
> There's no best practice for startup modules to discourage them from 
> loading modules at execution time, and I'm not sure there should be(?), 
> hence the conundrum.
>
> You'll notice my modloader plugin eclipses startup.js to move its 
> require() call into its startup() method.  If I don't do this, it's 
> impossible to modify a large portion of the core (including all widgets).  
> Unfortunately, this eclipsing is exactly the kind of thing the modloader is 
> designed to avoid.
>
>
> I’ve no objection to moving existing require() calls into a different 
> scope.
>
> It might be a useful exercise to try to write the documentation for 
>> bootloader modules, and see whether one can explain the difference clearly 
>> without referencing the particular use case that has prompted this 
>> discussion.
>>
>
> "bootloader modules run during the boot process, allowing them to add, 
> remove and modify tiddlers in the store prior to the wiki's startup 
> process.  This is useful for injecting plugins, code and content that 
> TiddlyWiki isn't capable of refreshing at runtime.  Injected content may be 
> dynamically generated or loaded from an external source.  These modules are 
> encouraged to minimize their dependency upon other modules in the wiki and 
> to avoid any dependency on the TiddlyWiki core.
>
> bootloader modules may safely revise any content in the tiddler store with 
> the exception of other bootloader modules and their dependencies.”
>
>
> And that sounds pretty much like a description of the existing 
> $tw.preloadTiddlers mechanism. The preboot approach offers the interesting 
> possibility of patching boot.js, too.
>
> I’m happy to discuss further but my position remains:
>
> * The proposed modifications don’t have any anticipated benefit beyond the 
> usage by the modloader plugin
> * The proposed modifications are relatively expensive in terms of time and 
> attention from me and the rest of the core team
> * The modloader plugin already exists and works, with some limitations
> * The preboot approach already exists as an alternative mechanism that 
> could be used instead, with different limitations
>
> Best wishes
>
> Jeremy
>
>
>  
>
> On Monday, 12 February 2018 16:53:53 UTC-6, Jeremy Ruston wrote:
>>
>> Hi Evan
>>
>> Thanks, I’ll give your arguments more thought, and hope you can join the 
>> hangout tomorrow to discuss further.
>>
>> This is almost, but not quite, the issue.  My modloader *is* the first 
>> module to have any method called on it.  However, TiddlyWiki currently 
>> *loads* (via $tw.module.execute) a very large number of modules prior to 
>> making any method calls.  There is no control over the ordering of these 
>> executions, and they are subsequently impossible to change at boot-time 
>> without overwriting the core.
>>
>>
>> Aren’t those executions just the cascade of modules requiring one another?
>>
>> As I mentioned before, I think there's potentially a meaningful 
>> difference between startup modules and "bootloader" behaviors.  Loading 
>> content and code into the tiddler store, versus setting up APIs, DOMs and 
>> run-time functionality.  It's much more likely that a loader module would 
>> want to affect startup behavior than another loader module.
>>
>>
>> Bootloader modules feel a bit like an admission of defeat with respect to 
>> the startup module mechanism.
>>
>> It might be a useful exercise to try to write the documentation for 
>> bootloader modules, and see whether one can explain the difference clearly 
>> without referencing the particular use case that has prompted this 
>> discussion.
>>
>> I worry that they are a narrow solution to your specific problem. We 
>> could also be looking at something buggyj has already started: making it 
>> easier for plugins to splice in raw markup code, and possibly adding more 
>> hooks to the boot kernel (eg, an explicit “patch-tiddler” hook).
>>
>> Anyway, the modloader doesn't *need* this functionality, strictly 
>> speaking.  It just extends its capabilities a bit, so that the *whole* core 
>> can be patched by installed plugins.  The modloader currently issues a 
>> warning when trying (and failing) to apply a patch of this nature so it 
>> shouldn't catch developers by surprise.
>>
>>
>> Excellent.
>>
>> The design intent of the modloader is to maintain compatibility with 
>> future versions of TiddlyWiki.  Duplicating bootloader behavior (which is 
>> subject to change) would detrimental enough to that goal that I'd rather 
>> stick with the current constraints.
>>
>>
>> But I think the code that you’d need to duplicate is code that’s only 
>> going to change if the underlying serialisation formats change? If that 
>> happens all bets are off with respect to future versions of TiddlyWiki 
>> maintaining backwards compatibility with this February 2018 incarnation of 
>> the plugin.
>>
>> Best wishes
>>
>> Jeremy.
>>
>>
>>
>> On Monday, 12 February 2018 15:10:16 UTC-6, Jeremy Ruston wrote:
>>>
>>> Hi Evan
>>>
>>> We’ve got to the point where the changes you are proposing to the core 
>>> are quite deep, and have lots of implications for the rest of the core. 
>>> It’s going to mean a lot of work on verification and testing from me, even 
>>> if somebody else writes the code. I don’t think that burden is justified at 
>>> present.
>>>
>>> Better to work with the plugin for a while, and explore the capabilities 
>>> and consequences before we commit to a new mechanism in the core.
>>>
>>> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach 
>>> here.  The modloader process 
>>> <http://evanbalster.com/tiddlywiki/formulas.html#%24%3A%2Fplugins%2Febalster%2Fmodloader%2Floader.js>
>>>  uses 
>>> quite a lot of $:/boot library functionality to identify and inspect shadow 
>>> tiddlers as part of the patching process, and to generate the output plugin.
>>>
>>>
>>> At this point, I don’t see any problem with your plugin duplicating a 
>>> few hundred lines of boot.js; eliminating such duplication would come 
>>> firmly under the heading of optimisation.
>>>
>>> I also worry that working on the pre-boot DOM would create 
>>> incompatibilities with certain TiddlyWiki environments (but I don't 
>>> understand this low level stuff very well). 
>>>
>>>
>>> You can see all the code that depends on this stuff. The approach is 
>>> solid.
>>>
>>> The current modloader can manipulate almost everything in the wiki 
>>> *except *for startup modules and the modules they require, hence the 
>>> request.
>>>
>>>
>>> Yes, but your proposed solution is just an escalation in an ongoing arms 
>>> race to be the first module to run.
>>>
>>> In most cases, bootloader modules (including modloader) are going to 
>>> want to generate any output to $:/temp or similar, where they won't create 
>>> persistent changes in the wiki.
>>>
>>>
>>> But equally, there’s no limit to the functionality in the boot kernel 
>>> that experimental plugins might want to tweak. Better to be based on a 
>>> pre-boot hook, and have the capability to hook anything, than to be 
>>> dependent on the hooks anticipated and provided by the core.
>>>
>>> Best wishes
>>>
>>> Jeremy.
>>>
>>> So, as mentioned in the OP I'm pretty leery of the "pre-boot" approach 
>>> here.  The modloader process 
>>> <http://evanbalster.com/tiddlywiki/formulas.html#%24%3A%2Fplugins%2Febalster%2Fmodloader%2Floader.js>
>>>  uses 
>>> quite a lot of $:/boot library functionality to identify and inspect shadow 
>>> tiddlers as part of the patching process, and to generate the output 
>>> plugin.  I also worry that working on the pre-boot DOM would create 
>>> incompatibilities with certain TiddlyWiki environments (but I don't 
>>> understand this low level stuff very well).  The current modloader can 
>>> manipulate almost everything in the wiki *except *for startup modules 
>>> and the modules they require, hence the request.
>>>
>>> In most cases, bootloader modules (including modloader) are going to 
>>> want to generate any output to $:/temp or similar, where they won't create 
>>> persistent changes in the wiki.
>>>
>>> I expect we'll end up discussing in more depth during tomorrow's call.
>>>
>>> On Monday, 12 February 2018 14:09:17 UTC-6, Jeremy Ruston wrote:
>>>>
>>>> Here’s the .tid of the $:/PatchCore.js tiddler, too.
>>>>
>>>> Best wishes
>>>>
>>>> Jeremy
>>>>
>>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "TiddlyWikiDev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to tiddlywikide...@googlegroups.com.
>>> To post to this group, send email to tiddly...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/tiddlywikidev.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/tiddlywikidev/98a6ce14-9ba5-470d-b396-365f08ddd806%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/tiddlywikidev/98a6ce14-9ba5-470d-b396-365f08ddd806%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "TiddlyWikiDev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to tiddlywikide...@googlegroups.com.
>> To post to this group, send email to tiddly...@googlegroups. 
>> <http://googlegroups.com/>com <http://googlegroups.com/>.
>> Visit this group at https://groups.google.com/group/tiddlywikidev.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/tiddlywikidev/cf2e1886-225d-49a0-9232-c646ee5352f7%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/tiddlywikidev/cf2e1886-225d-49a0-9232-c646ee5352f7%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "TiddlyWikiDev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to tiddlywikide...@googlegroups.com <javascript:>.
> To post to this group, send email to tiddly...@googlegroups.com 
> <javascript:>.
> Visit this group at https://groups.google.com/group/tiddlywikidev.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/tiddlywikidev/9a8c3a1b-e53c-42b1-9cde-c3a4bbcc7759%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/tiddlywikidev/9a8c3a1b-e53c-42b1-9cde-c3a4bbcc7759%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to tiddlywikidev+unsubscr...@googlegroups.com.
To post to this group, send email to tiddlywikidev@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/tiddlywikidev/9e519051-0aca-44c1-b2d1-9f71c01eb4bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to