Have you tried the 5.1.20 prerelease ? It's supposed to have indexing optimizations that will allow larger TW files. If so, it will be a real game changer.
Good luck On Friday, July 5, 2019 at 8:00:00 PM UTC-7, h0p3 wrote: > > What does the future hold for large Tiddlywikis? What can I do right now > to start optimizing my TW to be usable while still huge. I am grateful to > the TW contributors and those who made one my browser engines. This has to > be one of the best FOSS communities I've ever had the privilege of > participating in. I need the advice of experts. > > There's only so much optimization TW5 and the browsers which run it can > achieve. We're hitting the limits of single-threaded performance, and I am > not convinced there are many huge gains left for javascript in the browser > on high-end x86_64 CPUs (I desperately hope I'm wrong). Performance for > large TWs is looking a bit grim. I'm trying to look down the road 10 years > from now and ask myself if my Tiddlywiki is going to be the right tool for > the job. It's an amazing rapid-prototyping tool, but maybe it can't be made > performant enough. What do you think? > > Even though I'm still a noob at it, I use TW a lot > <https://philosopher.life/>; I'm an addict. In 3 years, I've amassed 10k > tiddlers of almost pure text in a 30MB TW (that's ~60 novels in length), > and I don't see myself slowing down. I don't do anything fancy, but this > tool is heavily integrated into my life. I'm a unificationist too: part of > the strength of this tool is that I don't have to separate it into > unconnected documents. I want to search, navigate, hyperlink, and construct > the whole. Unfortunately, the tool is getting slower and slower for me. I'm > pretty worried I need to move away from Tiddlywiki. > > The standout property of Tiddlywiki is that I get to serve a > self-modifying IDE+Product as a single html file (though I no longer > develop it without Bob). The client's browser does most of the calculation, > and I don't have to rely upon having a server which does anything more than > dishing out static files; I'm not beholden to centralized webservers > (though I still use github for now). I adore how it is censorship > resistant, cryptographically signable, easy to distribute, and it runs on > almost any device (though, at this point, my wiki barely runs on a phone). > It's perfect for P2P-serverlessness. There is nothing else like it in this > respect unless I'm handing someone a complete VM or container, but that > doesn't work nicely in a browser. There is no replacement as far as I can > tell. > > I've probably made plenty of mistakes in attempting to optimize, but I'm > trying. I try to stick to hardlinks, and I do my best not to generate > anything dynamically when I can. There's only so much optimizing one can do > for complex filter expressions. I even bend over backward to do what I call > "Firmcoding" in which dynamically generated lists are pre-computed into > static indexes (basically, this enables link references to function, limits > clientside computation, and if I have to move out of Tiddlywiki, all of my > linking structures are still hardcoded). What kinds of precomputing can I > do here to help the client side? I've only started really using tags this > past year, and I just ripped out all of the tags to find almost no > performance gains either. Do I have too many tiddlers? Do I need to start > finding ways to molecularize (rather than atomize) content into larger > tiddler bodies and then build specialized parsing for those large tiddlers? > I'm growing desperate. > > I feel like I'm going to puke. I've seen this coming for a while, and I've > been burying my head in the sand hoping it wasn't true. This is my evolving > horcrux-pensieve, and now I feel like a hermit crab who might have to find > another shell. I'm heartbroken at the thought. This is the best damn tool I > have ever used in my entire life; I can't bear to lose it. NO! NO! > NOO!!!!!!!!! /hissy-fit. > > Is there no hope for me here? Will the next TW-X be built with WASM? What > other toolset do you recommend for me? Where should this pilgrim go, and > what should he do? > -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywiki@googlegroups.com. Visit this group at https://groups.google.com/group/tiddlywiki. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/b8bf9b7d-d3ec-4369-b1db-07f216b78a76%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
