Thanks for the suggestions. It seems best to do the simplest first, and then
optimize later if memory management is taking a significant cost. So I think
I’ll stick with reshape! and Array{Array{Float64,1},1}.
On 29 Dec 2013, at 7:36 am, Stefan Karpinski <[email protected]> wrote:
> If you make a mistake, segfault.
>
>
> On Sat, Dec 28, 2013 at 3:35 PM, Toivo Henningsson <[email protected]>
> wrote:
> So what happens if you use Tim's sneaky workaround and resize the 1d array? I
> suppose that the pointer is no longer valid...
>
>
> On Saturday, 28 December 2013 18:25:50 UTC+1, Stefan Karpinski wrote:
> The issue was bounds check elimination, which is already a problem for 1d
> arrays. Currently it's very hard to eliminate them because arrays can get
> resized out from under you at any point.
>
> > On Dec 28, 2013, at 10:08 AM, Tim Holy <[email protected]> wrote:
> >
> > Holding columns in separate entries is a great way. However, if you need to
> > do
> > linear algebra on the matrix at intermediate stages during its growth, then
> > you'll have a lot of needless copying occurring while you convert the
> > column-
> > storage into a matrix.
> >
> > In such circumstances, there's a sneaky workaround:
> >
> > reshape1(a::Vector, dims::Dims) = pointer_to_array(pointer(a), dims)
> >
> > a = zeros(3)
> > c = ones(3)
> > append!(a, c)
> > A = reshape1(a, (3, div(length(a),3)))
> > c += 1
> > append!(a, c)
> > A = reshape1(a, (3, div(length(a),3)))
> >
> > Using pointer_to_array circumvents the ordinary protections built into
> > resize!
> > There's still allocation occurring (it has to build a new Array "wrapper"
> > on
> > each iteration), but it avoids copying any data, and for large amounts of
> > data
> > this is a big win.
> >
> > Even better would be to generalize resize! to support the final dimension
> > of
> > any array. I seem to remember Stefan had a reason why this might be
> > problematic, but I confess I forget what it is.
> >
> > --Tim
> >
> >
> >> On Friday, December 27, 2013 05:45:15 PM Sheehan Olver wrote:
> >> What's the "best" way of constructing an array that can grow adaptively?
> >> For example, it has fixed m rows but the number of columns grows as an
> >> algorithm proceeds. Unfortunately,
> >>
> >> resize!
> >>
> >> doesn't work for 2d arrays. It does work for Array{Array{Float64,1},1},
> >> but not sure that's optimal.
>
