DrChainsaw commented on code in PR #436:
URL: https://github.com/apache/arrow-julia/pull/436#discussion_r1192989113
##########
src/table.jl:
##########
@@ -521,6 +521,11 @@ function uncompress(ptr::Ptr{UInt8}, buffer, compression)
len = unsafe_load(convert(Ptr{Int64}, ptr))
ptr += 8 # skip past uncompressed length as Int64
encodedbytes = unsafe_wrap(Array, ptr, buffer.length - 8)
+ if len === -1
Review Comment:
I don't know the logic why the writer would do this. I encountered the issue
when trying to read files generated from the official java arrow package. When
stepping through the debugger during reading in java I encountered this lines
which I just copied over to the Julia implementation:
https://github.com/apache/arrow/blob/febd0ff144cfb8b2baffb1cb0be57ca40dc7cc77/java/vector/src/main/java/org/apache/arrow/vector/compression/AbstractCompressionCodec.java#L72-L75
In the linked issue there is an example file which also reads in pyarrow. I
haven't made any effort to find the corresponding line in there though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
