Thank you Matthias, I wasn’t aware of that ability to open the file and read its contents straight from the disk.
A quick experiment looks positive - no hanging and fast access - so I just need to work on the loop. Much obliged. Best, Keith > On 3 Nov 2020, at 13:37, matthias rebbe via use-livecode > <firstname.lastname@example.org> wrote: > > Couldn't you read the file partially > > you would open the file for read > > open file tFile for read > > then in a repeat loop you could read e.g. 10000 lines and process the data > > read from file tFile for 10000 lines > > > Just a thought. > > > - > Matthias Rebbe > Life Is Too Short For Boring Code > >> Am 03.11.2020 um 13:14 schrieb Keith Clarke via use-livecode >> <email@example.com>: >> >> Hi folks, >> I’ve a 3GB CSV file with 18M rows from which I need to create a subset CSV >> file for the rows containing tString. The following script results in >> LiveCode becoming unresponsive, so I’m wondering if I’m missing an obvious >> trick or what alternative approach I might take... >> on mouseUp pButtonNumber >> >> -- Select CSV File >> >> answer file prompt as sheet >> >> put it into tFile >> >> put tFile into field "File" >> >> -- Process CSV extract >> >> put URL ("file:" & tFile) into tText >> >> put line 1 of tText into field "List" >> >> filter lines of tText with "*tString*" >> >> put tText into field tList >> >> end mouseUp >> >> I don’t normally deal with large data sets, so maybe LiveCode isn’t the best >> tool for this but Excel 365 seems to only offer CSV file import, not query. >> >> Any thoughts greatly appreciated. >> >> Thanks & regards, >> Keith >> _______________________________________________ >> use-livecode mailing list >> firstname.lastname@example.org >> Please visit this url to subscribe, unsubscribe and manage your subscription >> preferences: >> http://lists.runrev.com/mailman/listinfo/use-livecode > > > _______________________________________________ > use-livecode mailing list > email@example.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode _______________________________________________ use-livecode mailing list firstname.lastname@example.org Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode