Thank you Matthias, I wasn’t aware of that ability to open the file and read
its contents straight from the disk.
A quick experiment looks positive - no hanging and fast access - so I just need
to work on the loop.
Much obliged.
Best,
Keith
> On 3 Nov 2020, at 13:37, matthias rebbe via use-livecode
> <use-livecode@lists.runrev.com> wrote:
>
> Couldn't you read the file partially
>
> you would open the file for read
>
> open file tFile for read
>
> then in a repeat loop you could read e.g. 10000 lines and process the data
>
> read from file tFile for 10000 lines
>
>
> Just a thought.
>
>
> -
> Matthias Rebbe
> Life Is Too Short For Boring Code
>
>> Am 03.11.2020 um 13:14 schrieb Keith Clarke via use-livecode
>> <use-livecode@lists.runrev.com>:
>>
>> Hi folks,
>> I’ve a 3GB CSV file with 18M rows from which I need to create a subset CSV
>> file for the rows containing tString. The following script results in
>> LiveCode becoming unresponsive, so I’m wondering if I’m missing an obvious
>> trick or what alternative approach I might take...
>> on mouseUp pButtonNumber
>>
>> -- Select CSV File
>>
>> answer file prompt as sheet
>>
>> put it into tFile
>>
>> put tFile into field "File"
>>
>> -- Process CSV extract
>>
>> put URL ("file:" & tFile) into tText
>>
>> put line 1 of tText into field "List"
>>
>> filter lines of tText with "*tString*"
>>
>> put tText into field tList
>>
>> end mouseUp
>>
>> I don’t normally deal with large data sets, so maybe LiveCode isn’t the best
>> tool for this but Excel 365 seems to only offer CSV file import, not query.
>>
>> Any thoughts greatly appreciated.
>>
>> Thanks & regards,
>> Keith
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
