Reinhard Pagitsch <[EMAIL PROTECTED]> writes: >Hello, > >I am parsing a binary file,extract the informations I want , and put >them in an AV* >I am doing this in the way: >AV* res; >SV* res1; > >res1 = newSVpvn(data, strlen(data)); >av_push(res,res1); > >After the file was parsed I return the AV* to a method of my module. It >works perfect on little files. >But if I parse a huge file (> 200MB) Perl uses more and more memory, ca. >800 MB. >Can anyone tell me how to avoid this?
Depending on what is in your SVs a 4-to-1 expansion of on-disc to in-perl-data structure is not impossible. In addition to the char * for there string an SV also has other fields (length etc.) and then there is the array of pointers in the AV that points at the SVs. So if you have a lot of small strings this may mount up. Also note that if you do something like this: my @data = xs_call(...); there are (at least) two copies - one on the stack and one in @data. You can get that down to one by using a reference to an array. With such things it is important to make sure that memory is being freed when you have done with it. Is it essential to read whole file at one go?
