Re: Array Split vs Combine

2021-03-10 Thread Monte Goulding via use-livecode
Ideally we wouldn’t introduce an additional iteration of the array to pre-calculate but yes there’s probably a few ways to go. It could also be possible to flag a string with a buffer allocation strategy on creation so the cases where the engine creates a string then repeatedly appends to it can

Re: Array Split vs Combine

2021-03-10 Thread Sean Cole (Pi) via use-livecode
Monte, Would it be possible to precalculate (ish) how big a string buffer is required from the size (memalloc) of the array? Then, if later down the process, it works out it doesn't have enough, it can add a bunch more memory to the buffer, thus reducing the frequency of buffer resizing. Or maybe d

Re: Array Split vs Combine

2021-03-10 Thread Monte Goulding via use-livecode
It’s probably most likely to do with the string buffer needing to be constantly resized as the array is iterated for combine. Some googling suggests Windows may have issues with this. Our strategy for growing string buffers at the moment is to allocate just enough for the string. Mark would need

Re: Array Split vs Combine

2021-03-10 Thread Sean Cole (Pi) via use-livecode
The code for 'Split': void MCArraysExecSplit(MCExecContext& ctxt, MCStringRef p_string, MCStringRef p_element_delimiter, MCStringRef p_key_delimiter, MCArrayRef& r_array) { if (MCStringSplit(p_string, p_element_delimiter, p_key_delimiter, ctxt . GetStringComparisonType(), r_array)) return; ctxt .

Re: Array Split vs Combine

2021-03-10 Thread Tom Glod via use-livecode
fo.net > > > > -Original Message- > > From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On > > Behalf > > Of Sean Cole (Pi) via use-livecode > > Sent: Tuesday, March 09, 2021 11:56 AM > > To: How to use LiveCode > > Cc: Sean Cole

Re: Array Split vs Combine

2021-03-10 Thread Bob Sneidar via use-livecode
Now THAT is fascinating, considering the Windows performance issues with file access reported in the past. Could it be that combine is somehow caching data to virtual memory? Bob S On Mar 9, 2021, at 1:05 PM, Sean Cole (Pi) via use-livecode mailto:use-livecode@lists.runrev.com>> wrote: It's

Re: Array Split vs Combine

2021-03-09 Thread Sean Cole (Pi) via use-livecode
> Evergreen Information Services > rdim...@evergreeninfo.net > > -Original Message- > From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On > Behalf > Of Sean Cole (Pi) via use-livecode > Sent: Tuesday, March 09, 2021 11:56 AM > To: How to use Live

RE: Array Split vs Combine

2021-03-09 Thread Ralph DiMola via use-livecode
09, 2021 11:56 AM To: How to use LiveCode Cc: Sean Cole (Pi) Subject: Array Split vs Combine Hi all, I have a tsv I split out as an array, process, then combine back to a tsv. The tsv is 53MB (55,635,198 bytes) and has 228320 lines with 21 columns (55406878 chars). To load the tsv file : 434ms

Re: Array Split vs Combine

2021-03-09 Thread Tore Nilsen via use-livecode
I got 248 393 257 also on a Mac Tore Nilsen > 9. mar. 2021 kl. 18:40 skrev Sean Cole (Pi) via use-livecode > : > > Extraordinary. Thanks Alex. This is helpful. This is on windows and a > windows server. I will try on a Mac and Linux and see if it has the same > issue. > > Sean > > On Tu

Re: Array Split vs Combine

2021-03-09 Thread Sean Cole (Pi) via use-livecode
Extraordinary. Thanks Alex. This is helpful. This is on windows and a windows server. I will try on a Mac and Linux and see if it has the same issue. Sean On Tue, 9 Mar 2021 at 17:25, Alex Tweedly via use-livecode < use-livecode@lists.runrev.com> wrote: > > On 09/03/2021 16:56, Sean Cole (Pi) vi

Re: Array Split vs Combine

2021-03-09 Thread Bob Sneidar via use-livecode
I get 483, 4325, 4280. It actually takes less time for me. Bob S > On Mar 9, 2021, at 9:25 AM, Alex Tweedly via use-livecode > wrote: > > > On 09/03/2021 16:56, Sean Cole (Pi) via use-livecode wrote: >> Hi all, >> >> I have a tsv I split out as an array, process, then combine back to a tsv

Re: Array Split vs Combine

2021-03-09 Thread Alex Tweedly via use-livecode
On 09/03/2021 16:56, Sean Cole (Pi) via use-livecode wrote: Hi all, I have a tsv I split out as an array, process, then combine back to a tsv. The tsv is 53MB (55,635,198 bytes) and has 228320 lines with 21 columns (55406878 chars). To load the tsv file : 434ms To split the tsv to array : 506

Array Split vs Combine

2021-03-09 Thread Sean Cole (Pi) via use-livecode
Hi all, I have a tsv I split out as an array, process, then combine back to a tsv. The tsv is 53MB (55,635,198 bytes) and has 228320 lines with 21 columns (55406878 chars). To load the tsv file : 434ms To split the tsv to array : 506ms To combine the array to a tsv : 2m 58s 214ms Why does it tak