I got it working without loops, now it takes 1/3 the time to load, or so.
Now I'm back with the memory limitation, please try to fix this!
The hack of pdlhash isn't in 2.4.7, I will look if i get it to work.
Ingo
On 10/20/2010 07:47 PM, Chris Marshall wrote:
> On 10/20/2010 11:12 AM, Ingo Schmid wrote:
>> So you suggest something like
>>
>> $all=readflex('data',[{Dims=>[32+2*$xsize,$records],Type=>'float'}])
> Yes. That reduces the number of file reads
> by a factor of $records*2 (the number of calls
> to readflex by a factor of $records).
>
>> Then
>>
>> $hdr=$all(:31,)
>> $data=$all(32:,)->reshape(2,$xsize,$records)
> You can use substr() and unpack with your
> templates to extract the needed index offsets
> from the $hdr.
>
> The reduction in file IO and readflex calls should
> improve performance significantly. Even if th
> data(,x,header(field_y),header(field_z), ...)+=line
>
> part is still done by perl loop. I would still
> expect it to run much faster.
>
I created an index piddle, moved the line to the end to thread over it
and did
$ind=pdl(header(field_y,),header(field_z,), ...)
$data(indexND($ind)+=line
It took me the whole day to figure out how to reshape(), mv() and slice.
> Good luck,
> Chris
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
