On Fri, Feb 22, 2013 at 4:41 AM, Johnuniq <[email protected]> wrote: > On Feb 20, 2013 at 3:54 pm, Tim Starling wrote: >> The idea of storing a database in a large string literal could >> be made to be fairly efficient and user-friendly if a helper >> module was written to do parsing and a binary search. > > I have implemented the above suggestion with some promising results. > Packing a large table in a string and unpacking it on demand appears > to work well, and the data is accessed as if it were stored in a > standard table. Using the table from Wiktionary Module:Languages > mentioned earlier in this thread, testing shows that accessing the > packed data is 20 times faster. Info is at > > http://test2.wikipedia.org/wiki/User_talk:Johnuniq#Big_tables
Note that https://gerrit.wikimedia.org/r/#/c/50299/ added a mw.loadData() function that should solve the problem for normal tables. It works like require, but can only handle simple data (no functions, tables with metatables, or tables with tables as keys), the returned data structure is made read-only, and it avoids having to re-execute the module chunk on every #invoke. Speaking of which, I need to update the documentation. -- Brad Jorsch Software Engineer Wikimedia Foundation _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
