Hi Marty.
Lotta prior posts on this so I will just summarize. Parsing data out into an
array can be a huge pain. It can be cumbersome to parse large amounts of data
into arrays. However there are techniques which can make it a great deal
faster.
First, read your data into manageable chunks. If you know your data is on
average 500 characters per line, and you want to read roughly 1000 lines at a
time, do something like:
read from fileName for 500 * 1000
put it into theData
read from file fileName until return
put it after theData
Now you have a manageable chunk to work with. Next, just repeating for each
line may seem the way to go, and some recommend it:
repeat for each line theLine in theData
put 1 into theColumnCount
repeat for each item theItem in theLine
put "column" & theColumnCount into theColumnName
put theItem into myArray[theRecordCount][theColumnName]
end repeat
end repeat
Others (myself included) think that simply working with line 1 of theData and
then deleting line 1 of theData is the way to go. The reason for this is that
each time your refer to a line, say line 256 of theData, the engine has to scan
through the block of data until it counts 255 returns, then get the data
between that and the next return. Well 256 not so bad. 256,783, bad. By just
getting line 1 each time and then deleting it, the engine only has to scan up
to the first carriage return each time.
So if it is a one time parsing, you may as well go ahead and create your array,
and then store it as a property and work with that. If you have to parse large
amounts of data each and every time you load your app, well there are better
ways for sure.
Also, PLEASE consider learning how to work with databases if you have large
data sets. It seems a pain at first, but once you get the hang of it, things
will go much better, especially searching and getting subsets of data.
Bob
On Dec 2, 2011, at 1:04 PM, Marty Knapp wrote:
> I'm using a form datagrid and have it successfully working with a tab
> delimited list that I cycle through and create an array, then use the
> FillInData handler in the datagrid to fill it out.
>
> I am now trying to set this up for large amounts of data. I've read the brief
> tutorial in the DG manual about using the "dgNumberOfRecords" and
> "GetDataForLine" but it's geared for accessing data from a database. I did
> look at the sample stack, but again it's geared for database access.
>
> The dgNumberOfRecords should be set to the total number of records in my data
> set, correct?
>
> In my scenario, how do I use the "GetDataForLine" command?
>
> Am I wasting time by converting my tab-delimited file to an array?
>
> I have no database experience and very little knowledge of arrays, so please
> bear with me! Any sample code, or sample stacks are appreciated.
>
> Thanks,
> Marty Knapp
>
> _______________________________________________
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
