For just this reason if I don't need to initialize the piddle I tend
to use new_from_specification, even if it is marked internal only.
use PDL;
use Benchmark;
my $pdl;
my $nelem = 1_000_000;
timethese(
100000,
{
zeroes => sub { $pdl = zeroes( byte, $nelem ) },
new_from_specification => sub {
$pdl = PDL->new_from_specification( byte, $nelem );
},
} );
results in:
Benchmark: timing 100000 iterations of new_from_specification, zeroes...
new_from_specification: 1 wallclock secs ( 0.83 usr + 0.00 sys =
0.83 CPU) @ 120481.93/s (n=100000)
zeroes: 74 wallclock secs (74.34 usr + 0.01 sys =
74.35 CPU) @ 1344.99/s (n=100000)
Maybe we can remove the "internal only" flag on new_from_specification
( or wrap it with something more intuitively named)?
On Thu, Sep 18, 2014 at 8:52 AM, Chris Marshall <[email protected]> wrote:
> Hi Roey-
>
> You haven't missed anything. zeros() is a pretty high level
> implementation and definitely is not optimized for anything
> like byte operation. I would guess that a lot of the difference
> is just from a byte loop in PDL::zeros() versus a memory
> copy.
>
> Have you tried just pre-allocating a zero piddle and then
> copying it to create each new buffer?
>
> $zero_1M = zeros(byte, 1_000_000);
> $buffer = $zero_1M->copy;
>
> Also, please feel free to put a ticket on our sf.net Feature
> Request tracker requesting improved performance with
> a sample code to demonstrate the timing difference.
>
> --Chris
>
>
> On Thu, Sep 18, 2014 at 7:22 AM, Roey Almog (Infoneto Ltd)
> <[email protected]> wrote:
>> Hi,
>>
>> I am a bit new to PDL so if the answer is obvious I have missed it some
>> how...
>>
>> I have around 1 million vectors of about 1 million elements long each
>> element is one byte in size
>>
>> The information is streamed in, one vector at a time with input data - the
>> data itself is not read from a file its real life information collected so
>> every time I run the program I get something else.
>>
>> I found that initializing the vectors with zeros takes long time, seems that
>> zeros iterates on all the cells settings them to zero
>>
>> so I created any empty pdl with zeros and each time I need to create a PDL I
>> use this methos
>>
>> # this is done once
>> my $pdl_template = zeros(byte, $size);
>> my $buffer_template = $pdl_template->get_dataref;
>>
>>
>> # now when getting the information
>> my $pdl = PDL->new_from_specification(byte, $size);
>> my $ptr_buffer = $pdl ->get_dataref;
>> $$ptr_buffer = ${$buffer_template};
>> $pdl->upd_data;
>>
>> This is about 7 times faster than zeros, however it would be better if zeros
>> (and also ones) had some optimization to clear an empty vector even faster
>> using c's memset function when applicable (bytes. shorts integer etc.)
>>
>> As this seems very useful I wonder if I missed something ?
>>
>> Thanks
>> Roey
>>
>> _______________________________________________
>> Perldl mailing list
>> [email protected]
>> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
>>
>
> _______________________________________________
> Perldl mailing list
> [email protected]
> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
