subject:"On Performance of Array Access"

Re: On Performance of Array Access

2018-08-31 Thread Richard Gaskin via use-livecode

jbv wrote:

> Once something "suitable for including in a 9.0.x maintenance release"
> is available, could you please include it to LC server, and especially
> the version of LC9 server installed on on-rev sage server ?

Here's a potentially quick CPU and RAM reduction opportunity for LC Server:
https://quality.livecode.com/show_bug.cgi?id=14115

With a CGI like LiveCode, it's launched with each request.

Depending on the system, up to 3/4 of the boot sequence is loading and 
initializing fonts.

I appreciate that LC Server now has the ability to generate custom 
graphics, some of which may need to specify fonts.  But I'd wager than 
less than 0.1% of scripts written for LC Server do that.

Comment #5 there suggests maybe adding a -f flag in the command line 
call we add to our .htaccess files to bypass the font init almost no one 
is using.

If we could get approval for that, we could probably find someone in the 
community to add the parsing of that flag from the input options and the 
IF statement before the font init.

By itself it won't make the scripts themselves faster.  But it'll make 
the engine init faster, and much leaner since it won't have the font 
info in memory.

On Dreamhost, for example, running LC Server is unpredictable because of 
its memory requirements and the full suite of fonts they have on their 
systems.

--
 Richard Gaskin
 Fourth World Systems
 Software Design and Development for the Desktop, Mobile, and the Web

 ambassa...@fourthworld.comhttp://www.FourthWorld.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

2018-08-31 Thread Tom Glod via use-livecode

Great work Mark! ...great news!

On Fri, Aug 31, 2018 at 5:41 PM Mark Talluto via use-livecode <
use-livecode@lists.runrev.com> wrote:

> I have only one complaint…You should have told me to put a diaper on
> before reading this. Aside from my inability to control myself with all
> this wonderful news…Thank you!
>
> Best regards,
>
> Mark Talluto
> livecloud.io 
> nursenotes.net 
> canelasoftware.com 
>
>
> > On Aug 31, 2018, at 6:58 AM, Mark Waddingham via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >
> > Generally, I don't tend to like to 'jump the gun' on anything related to
> optimization lest it is not what it seems when running in the real world
> but...
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

2018-08-31 Thread Mark Talluto via use-livecode

I have only one complaint…You should have told me to put a diaper on before 
reading this. Aside from my inability to control myself with all this wonderful 
news…Thank you!

Best regards,

Mark Talluto
livecloud.io 
nursenotes.net 
canelasoftware.com 


> On Aug 31, 2018, at 6:58 AM, Mark Waddingham via use-livecode 
>  wrote:
> 
> Generally, I don't tend to like to 'jump the gun' on anything related to 
> optimization lest it is not what it seems when running in the real world 
> but...

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

2018-08-31 Thread jbv via use-livecode

Mark,

Thank you so much for following up on this thread.

Once something "suitable for including in a 9.0.x maintenance release"
is available, could you please include it to LC server, and especially
the version of LC9 server installed on on-rev sage server ?
I'm in the process of improving for a client the speed a server-side
script that makes heavy use of arrays, and was originally written with
LC 4.- and then improved with LC 6.- .
Thanks in advance.

jbv





___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

2018-08-31 Thread Curry Kenworthy via use-livecode




Mark W:

> 6.7.11: 1117ms
> 9.0.1:  4020ms
> PR6671: 1017ms

Now THAT'S the kind of news that makes a Friday special! Honestly I was 
quite disappointed lately in 9's array speed, but this gives me a lot of 
hope, and it also helps in the practical details of code planning for 
projects. Thank you Mark.


Arrays are not the only issue, but it's a great start and one of the 
areas I was going to advocate. Even repeats with math calculations are 
also a problem in 9, for example. But I have full confidence, as I said 
on this list recently, that 9+ should be able to beat 6 in most areas 
when optimized. I will test your build and post my test stack and 
results here as soon as I get a chance.


Best wishes,

Curry Kenworthy

Custom Software Development
LiveCode Training and Consulting
http://livecodeconsulting.com/


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

2018-08-31 Thread Richard Gaskin via use-livecode


Impressive. Thank you.

--
 Richard Gaskin
 Fourth World Systems

Mark Waddingham wrote:
Generally, I don't tend to like to 'jump the gun' on anything related to 
optimization lest it is not what it seems when running in the real world 
but...


I'm becoming increasingly confident that a recent foray of mine into yet 
another attempt to improve the speed of array access in LC9 might have 
actually born some quite tasty fruit. Pleasingly the code changes 
required are small, highly localized and quite possibly more than 
suitable for including in a 9.0.x maintenance release.


The current WIP PR can be found here:

   https://github.com/livecode/livecode/pull/6671

Amusingly this didn't even start as an attempt at optimization, but an 
attempt to see if I could do anything about heap fragmentation!


The patch itself does three things:

   1) Minimizes the in-memory size of the core types - all 'book-keeping' 
records are now <=32 bytes on 64-bit systems


   2) Makes use of tArray[x]...[z] type expressions much more efficient 
when both evaluating and mutating


   3) Makes use of integer indicies in arrays much more efficient

I've been running it on a number of benchmarks, the two most notable 
follow.


LARGE ARRAY CONSTRUCTION

The following simple code example was supplied in Bug 17434 by Richard 
as it appears to show a memory leak (its actually heap fragmentation, 
but that's another story):


 put 100 into tMax
repeat with i = 1 to tMax
repeat with j = 1 to tMax
repeat with k = 1 to tMax
put any item of "bob,carol,ted,alice" into 
sA[i][j][k]
end repeat
end repeat
end repeat

The code constructs an 100x100x100 matrix (rendered as a nested array, 
rather than a flat one).


Timings in various versions (on my 2018 MBP laptop) are:

   6.7.11: 1117ms
   9.0.1:  4020ms
   PR6671: 1017ms

This would seem to be a not too uncommon a pattern of code - judging by 
JBV's recent post on here (which was essentially doing the same thing).


We can modify the above slightly to make it more focussed on array 
performance (i.e. taking out the need to construct / copy a random 
portion of a string):


 put 100 into tMax
repeat with i = 1 to tMax
repeat with j = 1 to tMax
repeat with k = 1 to tMax
put "bob,carol,ted,alice" into sA[i][j][k]
end repeat
end repeat
end repeat

Timings for this are:

   6.7.11: 1055ms
   9.0.1:  3281ms
   PR6671: 497ms

ARRAY RECORD FILTERING

The following code example was derived by me for my recent LCG talk 
based on a real-world code supplied by Mark Talluto. It iterates over a 
sequence of arrays, creating a new one where each record only has a 
subset of the keys. The keys to keep are taken from a stringlist:


private function _BenchmarkArrayFilterRecords pRecords, pColumnNames
local tFilteredRecords, tFilteredRecord
repeat for each element tRecord in pRecords
   repeat for each item tColumnName in pColumnNames
  put tRecord[tColumnName] into tFilteredRecord[tColumnName]
   end repeat
   put tFilteredRecord into \
 tFilteredRecords[the number of elements in tFilteredRecords 
+ 1]

end repeat
return tFilteredRecords
end _BenchmarkArrayFilterRecords

Timings for 10 iterations of the above are:

   6.7.11: 16872ms
   9.0.1:  8305ms
   PR6671: 4315ms

The following is the same as the above, except the keys to keep are 
taken from the keys of an array:


private function _BenchmarkArrayFilterRecordsSet pRecords, 
pColumnNameSet

local tFilteredRecords, tFilteredRecord
repeat for each element tRecord in pRecords
   repeat for each key tColumnName in pColumnNameSet
  put tRecord[tColumnName] into tFilteredRecord[tColumnName]
   end repeat
   put tFilteredRecord into \
 tFilteredRecords[the number of elements in tFilteredRecords 
+ 1]

end repeat
return tFilteredRecords
end _BenchmarkArrayFilterRecordsSet

Timings for 10 iterations of the above are:

   6.7.11: 16508ms
   9.0.1:  6397ms
   PR6671: 3001ms

REAL WORLD CASE

Now, I'm always a little skeptical about using synthetic benchmarks for 
performance. However, both of the above are actually real-world 
examples. Furthermore, when running a rather large LCS project on an 
engine with PR6671, I got a 2x speed up - one particular input took 
3mins to process, rather than 6mins (one phase of processing actually 
saw a 5x speed up!).


Anyway, thought this might make some potentially pleasing Friday 
afternoon news :)


Warmest Regards,

Mark.



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:

Re: On Performance of Array Access

2018-08-31 Thread Brian Milby via use-livecode

Cool. I subscribed to the PR :)

I’ll need to run this on the array stress test I did last week.

Thanks,
Brian
On Aug 31, 2018, 8:58 AM -0500, Mark Waddingham via use-livecode 
, wrote:
> Generally, I don't tend to like to 'jump the gun' on anything related to
> optimization lest it is not what it seems when running in the real world
> but...
>
> I'm becoming increasingly confident that a recent foray of mine into yet
> another attempt to improve the speed of array access in LC9 might have
> actually born some quite tasty fruit. Pleasingly the code changes
> required are small, highly localized and quite possibly more than
> suitable for including in a 9.0.x maintenance release.
>
> The current WIP PR can be found here:
>
> https://github.com/livecode/livecode/pull/6671
>
> Amusingly this didn't even start as an attempt at optimization, but an
> attempt to see if I could do anything about heap fragmentation!
>
> The patch itself does three things:
>
> 1) Minimizes the in-memory size of the core types - all 'book-keeping'
> records are now <=32 bytes on 64-bit systems
>
> 2) Makes use of tArray[x]...[z] type expressions much more efficient
> when both evaluating and mutating
>
> 3) Makes use of integer indicies in arrays much more efficient
>
> I've been running it on a number of benchmarks, the two most notable
> follow.
>
> LARGE ARRAY CONSTRUCTION
>
> The following simple code example was supplied in Bug 17434 by Richard
> as it appears to show a memory leak (its actually heap fragmentation,
> but that's another story):
>
> put 100 into tMax
> repeat with i = 1 to tMax
> repeat with j = 1 to tMax
> repeat with k = 1 to tMax
> put any item of "bob,carol,ted,alice" into sA[i][j][k]
> end repeat
> end repeat
> end repeat
>
> The code constructs an 100x100x100 matrix (rendered as a nested array,
> rather than a flat one).
>
> Timings in various versions (on my 2018 MBP laptop) are:
>
> 6.7.11: 1117ms
> 9.0.1: 4020ms
> PR6671: 1017ms
>
> This would seem to be a not too uncommon a pattern of code - judging by
> JBV's recent post on here (which was essentially doing the same thing).
>
> We can modify the above slightly to make it more focussed on array
> performance (i.e. taking out the need to construct / copy a random
> portion of a string):
>
> put 100 into tMax
> repeat with i = 1 to tMax
> repeat with j = 1 to tMax
> repeat with k = 1 to tMax
> put "bob,carol,ted,alice" into sA[i][j][k]
> end repeat
> end repeat
> end repeat
>
> Timings for this are:
>
> 6.7.11: 1055ms
> 9.0.1: 3281ms
> PR6671: 497ms
>
> ARRAY RECORD FILTERING
>
> The following code example was derived by me for my recent LCG talk
> based on a real-world code supplied by Mark Talluto. It iterates over a
> sequence of arrays, creating a new one where each record only has a
> subset of the keys. The keys to keep are taken from a stringlist:
>
> private function _BenchmarkArrayFilterRecords pRecords, pColumnNames
> local tFilteredRecords, tFilteredRecord
> repeat for each element tRecord in pRecords
> repeat for each item tColumnName in pColumnNames
> put tRecord[tColumnName] into tFilteredRecord[tColumnName]
> end repeat
> put tFilteredRecord into \
> tFilteredRecords[the number of elements in tFilteredRecords
> + 1]
> end repeat
> return tFilteredRecords
> end _BenchmarkArrayFilterRecords
>
> Timings for 10 iterations of the above are:
>
> 6.7.11: 16872ms
> 9.0.1: 8305ms
> PR6671: 4315ms
>
> The following is the same as the above, except the keys to keep are
> taken from the keys of an array:
>
> private function _BenchmarkArrayFilterRecordsSet pRecords,
> pColumnNameSet
> local tFilteredRecords, tFilteredRecord
> repeat for each element tRecord in pRecords
> repeat for each key tColumnName in pColumnNameSet
> put tRecord[tColumnName] into tFilteredRecord[tColumnName]
> end repeat
> put tFilteredRecord into \
> tFilteredRecords[the number of elements in tFilteredRecords
> + 1]
> end repeat
> return tFilteredRecords
> end _BenchmarkArrayFilterRecordsSet
>
> Timings for 10 iterations of the above are:
>
> 6.7.11: 16508ms
> 9.0.1: 6397ms
> PR6671: 3001ms
>
> REAL WORLD CASE
>
> Now, I'm always a little skeptical about using synthetic benchmarks for
> performance. However, both of the above are actually real-world
> examples. Furthermore, when running a rather large LCS project on an
> engine with PR6671, I got a 2x speed up - one particular input took
> 3mins to process, rather than 6mins (one phase of processing actually
> saw a 5x speed up!).
>
> Anyway, thought this might make some potentially pleasing Friday
> afternoon news :)
>
> Warmest Regards,
>
> Mark.
>
> --
> Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

On Performance of Array Access

2018-08-31 Thread Mark Waddingham via use-livecode

Generally, I don't tend to like to 'jump the gun' on anything related to 
optimization lest it is not what it seems when running in the real world 
but...


I'm becoming increasingly confident that a recent foray of mine into yet 
another attempt to improve the speed of array access in LC9 might have 
actually born some quite tasty fruit. Pleasingly the code changes 
required are small, highly localized and quite possibly more than 
suitable for including in a 9.0.x maintenance release.


The current WIP PR can be found here:

  https://github.com/livecode/livecode/pull/6671

Amusingly this didn't even start as an attempt at optimization, but an 
attempt to see if I could do anything about heap fragmentation!


The patch itself does three things:

  1) Minimizes the in-memory size of the core types - all 'book-keeping' 
records are now <=32 bytes on 64-bit systems


  2) Makes use of tArray[x]...[z] type expressions much more efficient 
when both evaluating and mutating


  3) Makes use of integer indicies in arrays much more efficient

I've been running it on a number of benchmarks, the two most notable 
follow.


LARGE ARRAY CONSTRUCTION

The following simple code example was supplied in Bug 17434 by Richard 
as it appears to show a memory leak (its actually heap fragmentation, 
but that's another story):


put 100 into tMax
repeat with i = 1 to tMax
repeat with j = 1 to tMax
repeat with k = 1 to tMax
put any item of "bob,carol,ted,alice" into 
sA[i][j][k]
end repeat
end repeat
end repeat

The code constructs an 100x100x100 matrix (rendered as a nested array, 
rather than a flat one).


Timings in various versions (on my 2018 MBP laptop) are:

  6.7.11: 1117ms
  9.0.1:  4020ms
  PR6671: 1017ms

This would seem to be a not too uncommon a pattern of code - judging by 
JBV's recent post on here (which was essentially doing the same thing).


We can modify the above slightly to make it more focussed on array 
performance (i.e. taking out the need to construct / copy a random 
portion of a string):


put 100 into tMax
repeat with i = 1 to tMax
repeat with j = 1 to tMax
repeat with k = 1 to tMax
put "bob,carol,ted,alice" into sA[i][j][k]
end repeat
end repeat
end repeat

Timings for this are:

  6.7.11: 1055ms
  9.0.1:  3281ms
  PR6671: 497ms

ARRAY RECORD FILTERING

The following code example was derived by me for my recent LCG talk 
based on a real-world code supplied by Mark Talluto. It iterates over a 
sequence of arrays, creating a new one where each record only has a 
subset of the keys. The keys to keep are taken from a stringlist:


private function _BenchmarkArrayFilterRecords pRecords, pColumnNames
   local tFilteredRecords, tFilteredRecord
   repeat for each element tRecord in pRecords
  repeat for each item tColumnName in pColumnNames
 put tRecord[tColumnName] into tFilteredRecord[tColumnName]
  end repeat
  put tFilteredRecord into \
tFilteredRecords[the number of elements in tFilteredRecords 
+ 1]

   end repeat
   return tFilteredRecords
end _BenchmarkArrayFilterRecords

Timings for 10 iterations of the above are:

  6.7.11: 16872ms
  9.0.1:  8305ms
  PR6671: 4315ms

The following is the same as the above, except the keys to keep are 
taken from the keys of an array:


private function _BenchmarkArrayFilterRecordsSet pRecords, 
pColumnNameSet

   local tFilteredRecords, tFilteredRecord
   repeat for each element tRecord in pRecords
  repeat for each key tColumnName in pColumnNameSet
 put tRecord[tColumnName] into tFilteredRecord[tColumnName]
  end repeat
  put tFilteredRecord into \
tFilteredRecords[the number of elements in tFilteredRecords 
+ 1]

   end repeat
   return tFilteredRecords
end _BenchmarkArrayFilterRecordsSet

Timings for 10 iterations of the above are:

  6.7.11: 16508ms
  9.0.1:  6397ms
  PR6671: 3001ms

REAL WORLD CASE

Now, I'm always a little skeptical about using synthetic benchmarks for 
performance. However, both of the above are actually real-world 
examples. Furthermore, when running a rather large LCS project on an 
engine with PR6671, I got a 2x speed up - one particular input took 
3mins to process, rather than 6mins (one phase of processing actually 
saw a 5x speed up!).


Anyway, thought this might make some potentially pleasing Friday 
afternoon news :)


Warmest Regards,

Mark.

--
Mark Waddingham ~ m...@livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: On Performance of Array Access

Re: On Performance of Array Access

Re: On Performance of Array Access

Re: On Performance of Array Access

Re: On Performance of Array Access

Re: On Performance of Array Access

Re: On Performance of Array Access

On Performance of Array Access

8 matches

Site Navigation

Mail list logo

Footer information