Hello Richard,
I had to leave for awhile but now here is the updated info:
Using put len(arrayEncode(myArray)) I got the following:
there are actually 11 different arrays - one for each of the word lengths from 2 to 12 letters.
The 3-ltr word array is 17672
The 9-ltr word array is 1159610 (there are a lot more 9-ltr words than 3-ltr words.) I didn't do all the others, but that will give you an idea. 7 and 8-ltr words will be close to 9-ltr words in size

I haven't yet had time to test with later versions, but once I do, I will let you know. The problem is not the time it takes to load the arrays. As I mentioned, it loads all 11 in just over 1 second. Once they are loaded, I access the data directly from the arrays and not from the fields. The number elements (lines) in the arrays varies. The 3-ltr array has 1292 lines - I guess that means elements.
The 9-ltr array has 40,727 lines or elements.
Each line is in a format like this:
apt,APT,PAT,TAP where "apt" is the key, which only needs to be found once and it shows all the 3-ltr words that can be made with those letters. Of course, that speeds up the search process tremendously.

The time (about .4 seconds) I mentioned for finding the 596 words found in ELEPHANTS is what I tested today. But I know that even on a slower computer using the stack built with Rev 2.9, the time was very close to that - about 1/2 of a second.

About 5 or 6 years ago when I was using 2.9 and then upgraded to 3.5, I not only spoke to someone at RunRev (forgot who) but I also spoke to Jerry Daniels, of Daniel and Mara, on the phone about several RunRev things and we discussed the array issue. He acknowledged that with the release of 3.5 there were some serious array issues, that RunRev knew about them and Jerry hoped that they would soon be fixed. They were not fixed with the release of RunRev 4.0 and the only version of LC that I've used extensively is 6.1.1 and the array issues are still not fixed in that release - although I would say it is a big improvement over RunRev 3.5.

The crazy thing is this: I haven't made any significant changes to my stack over the years - mostly cosmetic stuff and the array sizes are what they've always been. RunRev 2.9 never flinched, brought up the data in the arrays almost instantly. But since then it has been worse than trying to get my little grandson to wear a shirt.

I'm working on other projects now, but wanted to incorporate some of the anagram features in my latest project. I am very reluctant to do so because the complete sluggishness of it makes it way too frustrating to try and program.

Someday soon I will get around to trying LC 7.0 or 7.1 or whatever is there and HOPEFULLY the issue will be solved.

In the meantime, thank you and Jacqueline SO MUCH for your kind attention and offers of help.

All the best,
Larry

----- Original Message ----- From: "Richard Gaskin" <[email protected]>
To: <[email protected]>
Sent: Friday, September 19, 2014 1:19 PM
Subject: Re: large arrays


larry wrote:

> As I mentioned, it choked starting with Rev 3.5 and every iteration
> of Rev or LC that I've tried since, up to the current 6.1.1 (rc 4)

The current shipping version is 6.6.3, and the latest test version lower than 7 is 6.7RC2.

Please let us know if you see any significant change with either version.


> I have a hidden stack with my word fields in it - for words in length
> from 2 to 12.  The word lists are optimized with a key for faster
> searching.
> When the program loads, it puts each of the 11 word list fields into
> one array - which takes about 1 second on my XP with 4gb of ram.

If the problem is the time it takes to load the array, and the loading is happening from fields, the problem may not be in the arrays and perhaps in the field accesses.


> The word lists vary in length (3-ltr words is 1292 lines & the
> largest, 9-ltr words, is 40,727 lines.)  Each line in all the lists
> averages less than 100 characters.
> The size of the field for 9-ltr words is 796kb if copied into a txt
> document.

So there are 11 elements in the array?

If you run the array through arrayEncode and obtain its length, what is the size reported?

   put len(arrayEncode(tSomeArray))


> If I put in a 9 letter word such as ELEPHANTS, the program will find
> all anagrams (words found within the letters of ELEPHANTS) in about
> 4/10 of a second.

Is that the old time or the new time? If new, what is the old time (or vice versa)?


> I spoke to someone at RunRev years ago and they acknowledged that
> there was an issue with arrays.

It would be interesting to know exactly what they'd said. While arrays have undergone revision a few times over the years, they tend to be very fast in every version I've used.

Hopefully once we understand more of the details of your code we'll pin down the bottleneck and get your performance back on par.

--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web
 ____________________________________________________________________
 [email protected]                http://www.FourthWorld.com

_______________________________________________
use-livecode mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode


_______________________________________________
use-livecode mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Reply via email to