Nick Ing-Simmons wrote:
Reinhard Pagitsch <[EMAIL PROTECTED]> writes:Hello,
I am parsing a binary file,extract the informations I want , and put them in an AV*
I am doing this in the way:
AV* res;
SV* res1;
res1 = newSVpvn(data, strlen(data)); av_push(res,res1);
After the file was parsed I return the AV* to a method of my module. It works perfect on little files.
But if I parse a huge file (> 200MB) Perl uses more and more memory, ca. 800 MB.
Can anyone tell me how to avoid this?
Depending on what is in your SVs a 4-to-1 expansion of on-disc
to in-perl-data structure is not impossible.
In addition to the char * for there string an SV also has other fields (length etc.) and then there is the array of pointers in the AV that points at the SVs.
So if you have a lot of small strings this may mount up.
Also note that if you do something like this:
my @data = xs_call(...);
there are (at least) two copies - one on the stack
and one in @data.
You can get that down to one by using a reference to an array.
I use a reverence to an array, I belive: RETVAL = newRV_noinc ((SV*) res);
With such things it is important to make sure that memory is being freed when you have done with it.
How can I do this? Have you, please a short sample?
Is it essential to read whole file at one go?I think yes, but maybe there would be a better way to get the informations I want/need.
I thougth to read the file with pure perl and check if the data is allready in the array,
but I have no idea how I can do this via a module instead of a perl script. Maybe a script would be the better way.
mit freundlichen GrÃÃen, with my best regards, Reinhard
QA
ISIS Information Systems
Austria tel: (+43) 2236 27551 150 Fax: 2236 21081
Visit our web site http://www.isis-papyrus.com
---------------------------------------------------------------
This e-mail is only intended for the recipient and not legally binding. Unauthorised use, publication, reproduction or disclosure of the content of this e-mail is not permitted.
---------------------------------------------------------------
