Mark,
Just had a thought on how you might want to optimize things here. I'm
guessing that with the array of directories, you are using the
DirectoryPut() API for setting values. While this API does provide a level
of convenience, it does have some downsides for situations like this.
Specifically, this ends up creating a new string object for every
directory name you set. So, for 20,000 records with 150 fields, your
result set will end up creating 3,000,000 string objects just for the value
names in the result set. This is overhead over and above the actual data.
As an optimization, you could create a single string object for each field
at the start, then use SendMessage() to invoke the directory PUT method
using the cached names. This will not create a new name object for each
value added. Since garbage collection overhead is heavily determined by
the number of live objects, eliminating almost 3 million objects from
consideration should be a pretty good performance boost. This will also
result in many fewer GC cycles on top of that.
Rick
On Tue, Oct 16, 2012 at 1:05 PM, Staffan Tylen <staffan.ty...@gmail.com>wrote:
> Mark, some progress. Throughout I've used a 'real' database for testing
> this, but I now decided to create a test database just like yours and with
> that I get the following results:
>
> 3.744000 OO_ARRAY_OF_ARRAYS
> 11.201000 OO_ARRAY_OF_DIRECTORIES
> 19.126000 OO_STEM_OF_STEMS
>
> So why this difference? The major difference between the databases that I
> can think of is that the 'real' database contains a LARGE number of NULL
> values, while the test database doesn't have any. I will try to create a
> different test database that includes NULLs and see what happens.
>
> Staffan
>
>
>
>
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Oorexx-users mailing list
> Oorexx-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-users
>
>
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Oorexx-users mailing list
Oorexx-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-users
