Hi Jeremy, Indeed, lazy loading of tiddlers would be a wonderful addition to server mode.
Eventhough I never thought of storing a TW copy of Wikipedia, I know that a mere database of contacts or documents quickly involves generating thousands of tiddlers. In fact, I'm ready to bet that anytime one finds a need for scripting the creation of tiddlers, s(he) would end up creating thousands of them. A practical use for me would be to convert my ePub library (a few hundred books) into a (searchable ?) TW5 library. Cheers, XC. -- Xavier Cazin On Mon, May 12, 2014 at 1:32 PM, Jeremy Ruston <[email protected]>wrote: > Hi Matthew > > Very interesting experiment. Over time I would like to optimise TW to cope > well with large data sets - I can imagine lifelogging applications where > 100,000 tiddlers wouldn't be unusual. > > One issue is that we don't currently support lazy loading on the server; > it loads every tiddler at startup. We can subsequently build a TW HTML file > that uses lazy loading to load tiddlers from the server - see > http://tiddlywiki.com/static/LazyLoading.html > > Best wishes > > Jeremy > > > > > > On Mon, May 12, 2014 at 11:24 AM, Matthew DeAbreu < > [email protected]> wrote: > >> I actually had a similar thought, I downloaded an XML dump of the >> Wikipedia database from herehttp:// >> en.m.wikipedia.org/wiki/Wikipedia:Database_download and then I wrote a >> fairly simple python script that would traverse the XML document tree >> creating individual files for each article (that wasn't a redirect) and >> formatting them like a Tiddler. Then I was able to use the NodeJS version >> of TiddlyWiki to start upk using this directory of tiddlers. >> >> The first thing I noticed was how massive the memory footprint was and >> how slow the site was to load (plus I only took the first 10,000 articles). >> So TW is unfortunately not fast enough or strong enough to load the entire >> Wikipedia database, but if you have your own Media wiki installation that >> you would like to convert as others have pointed out there are ways to dump >> the database that can work and TW should be able to handle it if you have a >> smaller number of articles. Though you may have to convert the syntax >> yourself unless someone makes a plugin to add Media wiki syntax support. >> >> If you or anyone else wants the python script just let me know and I'll >> post it. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "TiddlyWiki" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/tiddlywiki. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Jeremy Ruston > mailto:[email protected] > > -- > You received this message because you are subscribed to the Google Groups > "TiddlyWiki" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/tiddlywiki. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "TiddlyWiki" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/tiddlywiki. For more options, visit https://groups.google.com/d/optout.
