Using append! is definitely an improvement, thanks!
The reason I included the sizehint! call was that this resizes the vector
*once*, and without requiring to copy any elements, and I assumed those
saved copies would quickly win over having to iterate the collection an
extra time to get the total number of elements. I’m glad to be proven wrong
:)
function vcat_prealloc(y)
result = Array(eltype(y[1]), sum(map(length, y)))
for a in y
append!(result, a)
end
end
function vcat_dynamic(y)
result = Array(eltype(y[1]), 0)
for a in y
append!(result, a)
end
end
arr_of_arrs = Vector{Int}[rand(1:10, rand(1:10)) for _ in 1:10_000]
julia> @benchmark vcat_dynamic(arr_of_arrs)
================ Benchmark Results ========================
Time per evaluation: 441.11 μs [405.40 μs, 476.82 μs]
Proportion of time in GC: 0.00% [0.00%, 0.00%]
Memory allocated: 1.00 mb
Number of allocations: 16 allocations
Number of samples: 100
Number of evaluations: 100
Time spent benchmarking: 0.07 s
julia> @benchmark vcat_prealloc(arr_of_arrs)
================ Benchmark Results ========================
Time per evaluation: 716.85 μs [652.07 μs, 781.62 μs]
Proportion of time in GC: 0.00% [0.00%, 0.00%]
Memory allocated: 933.78 kb
Number of allocations: 7 allocations
Number of samples: 100
Number of evaluations: 100
Time spent benchmarking: 0.10 s
// T
On Monday, November 23, 2015 at 11:51:25 AM UTC+1, Dan wrote:
Using `append!` instead of `push!` and letting efficient dynamic
> reallocation of vector do the resizing:
>
> function vcat_nosplat2a(y)
> result = Array(eltype(y[1]), 0)
> for a in y
> append!(result, a)
> end
> result
> end
>
> (@benchmark shows way less allocations and 2-3x time)
> BTW the splat version is just as quick, but perhaps allocation on stack is
> problematic (anybody check the limit?)
>
> On Monday, November 23, 2015 at 12:07:18 PM UTC+2, Tomas Lycken wrote:
>>
>> That doesn’t quite seem to do what you want, Mauro:
>>
>> julia> arr_of_arr = Vector{Int}[[1],[2,3],[4,5]]
>> 3-element Array{Array{Int64,1},1}:
>> [1]
>> [2,3]
>> [4,5]
>>
>> julia> vcat_nosplat(y) = eltype(y[1])[el[1] for el in y]
>> vcat_nosplat (generic function with 1 method)
>>
>> julia> vcat_nosplat(arr_of_arr)
>> 3-element Array{Int64,1}:
>> 1
>> 2
>> 4
>>
>> It’s quite trivial to achieve the desired result with only a few lines of
>> code, though:
>>
>> julia> function vcat_nosplat2(y)
>> result = Array(eltype(y[1]), 0)
>> sizehint!(result, sum(map(length, y))) #skip if iterating is more
>> expensive than reallcoation
>>
>> for a in y
>> for x in a
>> push!(result, x)
>> end
>> end
>>
>> result
>> end
>> vcat_nosplat2 (generic function with 1 method)
>>
>> julia> vcat_nosplat2(arr_of_arr)
>> 5-element Array{Int64,1}:
>> 1
>> 2
>> 3
>> 4
>> 5
>>
>> // T
>>
>> On Sunday, November 22, 2015 at 9:12:55 PM UTC+1, Mauro wrote:
>>
>> In ODE.jl, I've used
>>>
>>> vcat_nosplat(y) = eltype(y[1])[el[1] for el in y] # Does vcat(y...)
>>> without the splatting
>>>
>>> I think the eltype might not be needed. There may be better ways
>>> though.
>>>
>>> On Sun, 2015-11-22 at 14:04, Cedric St-Jean <[email protected]>
>>> wrote:
>>> > I have a big vector of vectors. Is there any way to vcat/hcat them
>>> without
>>> > splatting?
>>> >
>>> > arr_of_arr = Vector[[1],[2,3],[4,5]]
>>> > vcat(arr_of_arr...)
>>> >
>>> > I'm asking because splatting big arrays is a performance issue (and
>>> IIRC it
>>> > blows the stack at some point).
>>>
>>
>>
>
