Re: Speed up a slow loop
It occurs to me that any word which is valid in Boggle must consist of an uninterrupted run of letters which are *right next to each other* on the board. Hence, any digraph which *doesn't* consist of a pair of neighboring letters, *cannot* occur in any valid word. Since the "filter out all words with LettersNotOnTheBoard" idea worked, it may be that an analogous "filter out" operation can speed things up more. When setting up a new board... -- Construct a list of all 576 ( = 26x26 ) digraphs—"aa,ab,ac, … zx,zy,zz". Or, more likely, construct that list ahead of time, and store the list in a stack property.-- Remove all digraphs which contain at least one letter that's not on the board.-- Make a list of all pairs of neighboring letters on the board. Since you don't know which direction an arbitrary digraph is running in, you'll want to include ZA as well as AZ.-- Remove all the digraphs present on the board from the ListOfAllDigraphs.-- Replace all the "," in the ListOfAllDigraphs with "|". (i.e., replace commas with pipes) -- Put "[" & ListOfAllAbsentDigraphs & "]" into RegexPattern.-- Filter WordList without RegexPattern.-- ???-- Profit. Hmmm... since this starts out with a complete list of all 576 digraphs, there's no need to do a separate regex filter on individual letters; filtering out the full list of unused digraphs should encompass filtering out unused letters. Hence, just one regex filter should do the job. I think. On a 4x4 Boggle board, there are 72 neighboring pairs, hence a maximum of 144 valid digraphs; on a 5x5 board, there are 110 neighing pairs, hence a max of 220 digraphs. Hopefully, this won't add too much overhead to board setup. "Bewitched" + "Charlie's Angels" - Charlie = "At Arm's Length" Read the webcomic at [ http://www.atarmslength.net ]! If you like "At Arm's Length", support it at [ http://www.patreon.com/DarkwingDude ]. ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
> On 9 Mar 2022, at 4:00 am, Alex Tweedly wrote: > > I guess I'm not convinced about VM space thrashing being the problem, > certainly not due to the wordlist. It's 2.5 Mb - i.e. 0.02% of the real > RAM in a Pixel 5. There may be something else in the app making the > total VM space very large - but the wordlist wouldn't do it on its own. I have to agree … although the relevant RAM size is the amount allocated to the app, not the total RAM. The OS presumably takes up most of the RAM by itself. But of course just a single copy of the list in memory is not a problem or Jacque wouldn’t be able to do the filtering. Turning the list into an array will add indexing overhead very similar to what using a database would do (I suspect extremely similar in fact!) but I would have thought we are still only talking 10MB at most. Possibly more, the faster-than-light magic that happens with arrays must be trading off memory for cpu time somehow The symptoms definitely sound to me like VM thrashing, and reducing the list to around 20K words evidently removes the issue, which suggests multiple copies of the array are being created somewhere in (Jacque’s original ) search algorithm. > > >> An alternative is to use an sqlite database, always an option which should >> be considered when handling large data. This completely removes the issue of >> having the wordlist in memory, and at the same time provides an extremely >> fast search engine, so was worth exploring as a solution to both problems. >> Storing the words in a db single table with an index on initial letter and >> word length, or as lots of tables, one for each initial and length (!), both >> return a result for a search for a word in a small fraction of a >> millisecond, so definitely this would be a viable solution which would >> handle all letter distributions. > Sounds cool. I'd like to see an example of how this would be created and > used (I'm very much a SQL novice). I can’t claim to be an expert myself (see below) but it’s not hard and the LC tutorial together with online sqlite cheat sheets make it pretty straightforward. The hardest bit is getting into the db mindset which a bit different from procedural coding. The LC library makes it very easy to use sqlite in stacks. Creating the db from the data is easy, it only needs some thought on what tables and columns are needed - just like designing a DataGrid in LC. Making simple queries of the db is plain language-like. For example to find all words matching a crossword pattern - - T - - - - containing N you execute a query such as SELECT FROM words WHERE (length=7) AND (word LIKE ‘__T’) AND (word LIKE ‘%N%’); [ wildcard underscore _ matches a single character, % matches an arbitrary string … sound familiar? LIKE is used for comparing strings] This outputs the 89 answers in a few milliseconds [most of which look to me like real words. But LETTERN ?? WITCHEN ?? SOWPODS really is ridiculous] Making more complex queries is where things get arcane. >> There is however a downside: both methods produce a db file size of 7.1 MB >> for the SOWPODS wordlist, which rather bloats the app footprint, even when >> you discount the text file version of the wordlist which no longer needs to >> be stored. > You could do the opposite. Store the wordlist (700Kb compressed), and > decompress/populate into the SQLite database on initial run. But unfortunately populating the database is very slow, taking minutes. [Note to myself: hey wait a minute, me, a database with only 275K rows is not that big in sqlite terms. Why is it taking so long? Because I am using a dumb repeat loop inserting one word at a time, akin to adding one word at a time to a field and then updating the UI. There is a better way. Physician, heal thyself!] >> But now, speaking of databases, I have a question. I have an update to my >> nsScriptDatabase stack which I want to upload to the Sample Stacks. Because >> it really should be compiled to a standalone, I really need to upload a pair >> of stacks, a launch stack and one to hold data which can one modified by the >> user. But it would seem a sample stack must be a single item. What to do? > > Yep -this is a big drawback of the Sample Stacks system. > > Maybe you could make a single app, which contains the two desired stacks > as custom props. Ah thank you, and a very interesting approach which I would never have thought of. Prompted me to think of another way: include the data stack as a substack when used in the IDE, but on openstack when not in the development environment clone the data stack to a writeable directory. That should work. Neville ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
On 3/8/22 3:22 PM, Quentin Long via use-livecode wrote: Does it*matter* whether a genuine word isn't on the board? If the board has no "L" in it, any word with an "L" won't be on the board, right? Right, but what happens is that your eye tricks you and you skip a tile inadvertently or use the same one twice. In Boggle, words are only valid if the tiles touch consecutively, and you can only use a particular tile once per word. So it's easy to use the same "e" twice, or to skip a tile to get to an "e". Don't believe everything you (think) you see. -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Requirements for display PDF in a browser on Windows?
I have definitely opened PDF in the browser in the past and it opened the default "chrome html 5" pdf reader. everything seems to have worked. And I don't see why it would no longer work now. you can google "browser test" from within the browser and it will tell you some info about the chromium based browser. Some of it may shock you. Cheers On Tue, Mar 8, 2022 at 3:08 PM Klaus major-k via use-livecode < use-livecode@lists.runrev.com> wrote: > Hi, > > > Am 08.03.2022 um 19:54 schrieb Klaus major-k via use-livecode < > use-livecode@lists.runrev.com>: > > > > Hi all, > > > > the subject says it all, of course in regard of LC. > > > > Is the web-plugin of Acrobat reader still neccessary? > > Or do current Windows Browser display PDF files out of the box? > > and/or more important, what web engine is the browser widget using on > Windows? > > > Thanks for any hint! > > Best > > Klaus > > -- > Klaus Major > https://www.major-k.de > https://www.major-k.de/bass > kl...@major-k.de > > > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your > subscription preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode > ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
> On 9 Mar 2022, at 4:00 am,Alex Tweedly wrote: > > Oh, well, if you've got the fast boardwalk code in there anyway, that > opens up another possibility (with one caveat). > > The boardwalk will find all valid words on the board. So that list of > words can be used as the "dictionary" for the user's guesses - and now > you have a "dictionary" of only a few hundred words, rather than than > the many thousands of either the original or the filtered dictionaries. > Really only a few hundred potential words? My back-of-a-very-small-envelope guesstimate was about 2000. I’ll be interested to see how many there are, because my envelope isn’t big enough to work it out, and the walking algorithm which I am really impressed can be so fast. But anyway checking even a few hundred potential words (the vast majority of which would be non-words so the worst case) is going to be slower than checking a much smaller number of mostly valid user candidates. Both can one done in idle time so not a problem I guess. Neville ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
sez Alex Tweedly:> The caveat - a user guess which isn't in the word list found by the > boardwalk can be either "not a word" or "a word, but not present on the > board", and you don't know which. Does it *matter* whether a genuine word isn't on the board? If the board has no "L" in it, any word with an "L" won't be on the board, right? "Bewitched" + "Charlie's Angels" - Charlie = "At Arm's Length" Read the webcomic at [ http://www.atarmslength.net ]! If you like "At Arm's Length", support it at [ http://www.patreon.com/DarkwingDude ]. ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Requirements for display PDF in a browser on Windows?
Hi, > Am 08.03.2022 um 19:54 schrieb Klaus major-k via use-livecode > : > > Hi all, > > the subject says it all, of course in regard of LC. > > Is the web-plugin of Acrobat reader still neccessary? > Or do current Windows Browser display PDF files out of the box? and/or more important, what web engine is the browser widget using on Windows? > Thanks for any hint! Best Klaus -- Klaus Major https://www.major-k.de https://www.major-k.de/bass kl...@major-k.de ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: moveStack is not triggering
I'll be damned. Yes it does. Breakpoints do not work for some reason. Probably because the mouse is still down when dragging. Bob S > On Mar 8, 2022, at 11:21 , Niggemann, Bernd via use-livecode > wrote: > > Hi Bob, > > try this in the card script of a new stack > > on moveStack > beep > end moveStack > > That beeps when I move the stack. > > I think the debugger does not fire when it is invoked in a moveStack > handler as you test for in your bug report > https://quality.livecode.com/show_bug.cgi?id=23607 > > Kind regards > Bernd > > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
On 3/8/22 1:27 AM, Neville Smythe via use-livecode wrote: Jacque seems to have found a satisfactory solution to her problem with the help of the excellent suggestions we have seen here. Since there seems to have been a lot of interest, I thought I would take the opportunity to report on an alternative method I had been exploring with Jacque via email. And it was appreciated. Neville and I had an off-list conversation where he really put some effort into solving the problem. In the long run I decided it would be better to keep everything in LCS if possible since the timings were fixed, but his help was one more example of how great our community is. -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
On 3/8/22 10:30 AM, Alex Tweedly via use-livecode wrote: I guess I'm not convinced about VM space thrashing being the problem, certainly not due to the wordlist. It's 2.5 Mb - i.e. 0.02% of the real RAM in a Pixel 5. There may be something else in the app making the total VM space very large - but the wordlist wouldn't do it on its own. I'm not sure either. A few days ago I did some tests trying to track the total amount of RAM that was used by looking at the memory stats in the developer options. I started the game, did some quick word lookups, quit and removed from memory, then checked usage. It isn't very specific but while my app was running the Pixel 6 had a maximum usage of 368MB after three tests, and the Pixel 5 maxed at 255MB. This isn't near the top of the list, other apps use more. On the Pixel 5 today, with only background apps running, I'm using 55% of RAM which leaves 3.5GB free. The Pixel 6 is using 70% with 2.4GB free. The OS on the 6 is bigger and I have more background processes running there. In trying to reduce the memory footprint I wanted to see how much RAM my script locals were storing. The easiest way was to get the value of all of them, write them consecutively to a file, and let Finder tell me the size. I found that Alex's brilliant script was holding 5MB of data that I didn't need, so I emptied out the variables that were only used to calculate the board and kept the single one I needed for lookups. That file was 3K. I need to test more today. Last night I thought I was all set, got a great board and quickly found 125 words. Everything was snappy and worked great. Then suddenly the app froze solid, nothing responded, and I had to quit to the launcher. Which was a shame because it was a really good board. That was before I wiped out the extra 5MB of unnecessary variable data, and there may be more I can do trim it down. But you'd think 2-3GB of free RAM could accomodate it. If I can't figure it out soon, I'll give it to all of you to see if you can. -- Jacqueline Landman Gay | jac...@hyperactivesw.com HyperActive Software | http://www.hyperactivesw.com ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: moveStack is not triggering
Hi Bob, try this in the card script of a new stack on moveStack beep end moveStack That beeps when I move the stack. I think the debugger does not fire when it is invoked in a moveStack handler as you test for in your bug report https://quality.livecode.com/show_bug.cgi?id=23607 Kind regards Bernd ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Requirements for display PDF in a browser on Windows?
Hi all, the subject says it all, of course in regard of LC. Is the web-plugin of Acrobat reader still neccessary? Or do current Windows Browser display PDF files out of the box? Thanks for any hint! Best Klaus -- Klaus Major https://www.major-k.de https://www.major-k.de/bass kl...@major-k.de ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
moveStack is not triggering
Hi all. Anyone else using the moveStack message? I have a handler in the card script which is where the dictionary says the message is sent. Moving the stack does not trigger the message. Bob S ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
On 08/03/2022 07:27, Neville Smythe via use-livecode wrote: I believe there were two problems: a) the original search search algorithm was too slow, and b) when implemented on certain devices the app went into an unusable state. Both problems were caused by the wordlist being stored being too large (275K words for a file size of 2.5 MB). The second issue we are pretty sure was Virtual Memory swap space thrashing. I guess I'm not convinced about VM space thrashing being the problem, certainly not due to the wordlist. It's 2.5 Mb - i.e. 0.02% of the real RAM in a Pixel 5. There may be something else in the app making the total VM space very large - but the wordlist wouldn't do it on its own. An alternative is to use an sqlite database, always an option which should be considered when handling large data. This completely removes the issue of having the wordlist in memory, and at the same time provides an extremely fast search engine, so was worth exploring as a solution to both problems. Storing the words in a db single table with an index on initial letter and word length, or as lots of tables, one for each initial and length (!), both return a result for a search for a word in a small fraction of a millisecond, so definitely this would be a viable solution which would handle all letter distributions. Sounds cool. I'd like to see an example of how this would be created and used (I'm very much a SQL novice). There is however a downside: both methods produce a db file size of 7.1 MB for the SOWPODS wordlist, which rather bloats the app footprint, even when you discount the text file version of the wordlist which no longer needs to be stored. You could do the opposite. Store the wordlist (700Kb compressed), and decompress/populate into the SQLite database on initial run. But now, speaking of databases, I have a question. I have an update to my nsScriptDatabase stack which I want to upload to the Sample Stacks. Because it really should be compiled to a standalone, I really need to upload a pair of stacks, a launch stack and one to hold data which can one modified by the user. But it would seem a sample stack must be a single item. What to do? Yep -this is a big drawback of the Sample Stacks system. Maybe you could make a single app, which contains the two desired stacks as custom props. (Sometimes it's easier to type code than to describe it :-) In your preparation, you'd do: put URL ("binfile:launchstack.livecode") into tmp set the cLaunch of me to compress(tmp) put URL ("binfile:userdatastack.livecode") into tmp set the cUser of me to compress(tmp) And then when the user downloads your sample stack, it does the inverse to create the two local stacks, with an info box telling them about it. Alex. ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
Re: Speed up a slow loop
Apart from searching I am sure it will provide many other great learning opportunities as well. Looking forward to seeing how it all pans out. Mark > On Mar 6, 2022, at 8:18 PM, J. Landman Gay via use-livecode > wrote: > > On 3/6/22 9:40 AM, William Prothero via use-livecode wrote: >> I've been following this thread in wonder at the creative ways of optimizing >> this word search. I hope that the code will be published somewhere, >> available to all, with credits to the multiple idea authors. > > I can upload it to Sample Stacks when it's done, there's nothing proprietary > about it. There's no point to submitting it to any of the app stores since 1) > the name "Boggle" is copyrighted as well as the game itself, and 2) there are > a gazillion word search games there anyway. > > I wrote it for myself and anyone else who wants to play with it. > > -- > Jacqueline Landman Gay | jac...@hyperactivesw.com > HyperActive Software | http://www.hyperactivesw.com > > ___ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode ___ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode