Copilot commented on code in PR #615:
URL: https://github.com/apache/arrow-julia/pull/615#discussion_r4488732354
##########
src/FlatBuffers/table.jl:
##########
@@ -110,7 +110,12 @@ function Array{T}(t::Table, off) where {T}
a = vector(t, off)
S = T <: Table ? UOffsetT : T <: Struct ? NTuple{structsizeof(T),UInt8} : T
ptr = convert(Ptr{S}, pointer(bytes(t), a + 1))
- data = unsafe_wrap(Base.Array, ptr, vectorlen(t, off))
+ len = vectorlen(t, off)
+ data = if UInt(ptr) % Base.datatype_alignment(S) == 0
+ unsafe_wrap(Base.Array, ptr, len)
+ else
+ copy(reinterpret(S, @view bytes(t)[(a + 1):(a + len * sizeof(S))]))
Review Comment:
For a batch with N Utf8View/BinaryView columns and a misaligned counts
payload, this copies the entire N-element vector for every column: `build` in
`src/table.jl:904` accesses `rb.variadicBufferCounts` anew, and the property
getter constructs a new `FlatBuffers.Array` each time. This introduces
quadratic metadata copying (about 800 MB for 10,000 view columns), even when
all strings are inline. Cache the aligned counts once per batch and reuse them,
or decode individual counts without copying the whole vector on each access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
