One more thing...are you making judicious use of ReleaseLocalReference
while you are adding values? That also can contribute to garbage
collection problems.
Rick
On Tue, Oct 16, 2012 at 1:25 PM, Rick McGuire <object.r...@gmail.com> wrote:
> Mark,
>
> Just had a thought on how you might want to optimize things here. I'm
> guessing that with the array of directories, you are using the
> DirectoryPut() API for setting values. While this API does provide a level
> of convenience, it does have some downsides for situations like this.
> Specifically, this ends up creating a new string object for every
> directory name you set. So, for 20,000 records with 150 fields, your
> result set will end up creating 3,000,000 string objects just for the value
> names in the result set. This is overhead over and above the actual data.
> As an optimization, you could create a single string object for each field
> at the start, then use SendMessage() to invoke the directory PUT method
> using the cached names. This will not create a new name object for each
> value added. Since garbage collection overhead is heavily determined by
> the number of live objects, eliminating almost 3 million objects from
> consideration should be a pretty good performance boost. This will also
> result in many fewer GC cycles on top of that.
>
> Rick
>
> On Tue, Oct 16, 2012 at 1:05 PM, Staffan Tylen <staffan.ty...@gmail.com>wrote:
>
>> Mark, some progress. Throughout I've used a 'real' database for testing
>> this, but I now decided to create a test database just like yours and with
>> that I get the following results:
>>
>> 3.744000 OO_ARRAY_OF_ARRAYS
>> 11.201000 OO_ARRAY_OF_DIRECTORIES
>> 19.126000 OO_STEM_OF_STEMS
>>
>> So why this difference? The major difference between the databases that I
>> can think of is that the 'real' database contains a LARGE number of NULL
>> values, while the test database doesn't have any. I will try to create a
>> different test database that includes NULLs and see what happens.
>>
>> Staffan
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Don't let slow site performance ruin your business. Deploy New Relic APM
>> Deploy New Relic app performance management and know exactly
>> what is happening inside your Ruby, Python, PHP, Java, and .NET app
>> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
>> http://p.sf.net/sfu/newrelic-dev2dev
>> _______________________________________________
>> Oorexx-users mailing list
>> Oorexx-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/oorexx-users
>>
>>
>
------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Oorexx-users mailing list
Oorexx-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-users
