Thanks. This certainly works great. I guess I more or less could have known this but that I also wanted to probe whether there was not a far superior approach to tackle these kind of problems. I guess I could wrap this behaviour in a simple MemoryBuffer type.
On Tuesday, June 3, 2014 1:05:09 AM UTC+2, Simon Kornblith wrote: > > There may be a more elegant solution, but the sledgehammer approach would > be to replace the reshape operation with: > > tmp = pointer_to_array(pointer(buffer), currentdims) > > Obviously don't access tmp after resize! but before setting its value or > else Julia may segfault. > > Simon > > On Monday, June 2, 2014 11:34:13 PM UTC+2, Jutho wrote: >> >> Dear Julia users, >> >> I often need to use large temporary multidimensional arrays within loops. >> These temporary arrays don't necessarily have a fixed size, so allocating >> them just once before the start of the loop is not really an option. >> Allocating them within the loop triggers a lot of gc activity, which can >> have a non-negible impact on the performance. >> >> I was now trying to use the following strategy: >> buffer=zeros(T,prod(firstdims)) >> tmp=reshape(buffer,firstdims) >> # initialize tmp >> for i in iter >> # do something with tmp of previous iteration >> >> # from here on, previous tmp is no longer needed >> resize!(buffer,prod(currentdims)) >> tmp=reshape(buffer,currentdims) >> # initialise new tmp >> end >> >> The idea would be that by using resize!, memory would only be allocated >> if the current length of tmp exceeds any length encountered before. In >> addition, I think resize! is directly using memory allocation calls in C >> and is not passing along the GC. Unfortunately, I get the following error: >> >> *ERROR: cannot resize array with shared data* >> >> * in resize! at array.jl:496* >> >> which makes sense. the old buffer is still used in the previous value of >> tmp, and I have not expressed that I will no longer need tmp. Is there a >> possibility to express this and to make this work, or is there a better >> suggestion of how to tackle this kind of problem? >> >>
