baumgold commented on code in PR #397:
URL: https://github.com/apache/arrow-julia/pull/397#discussion_r1134725578
##########
src/Arrow.jl:
##########
@@ -82,6 +89,19 @@ function __init__()
lz4 = LZ4FrameCompressor(; compressionlevel=4)
CodecLz4.TranscodingStreams.initialize(lz4)
push!(LZ4_FRAME_COMPRESSOR, lz4)
+ # Locks
+ push!(LZ4_FRAME_COMPRESSOR_LOCK, ReentrantLock())
+ push!(ZSTD_COMPRESSOR_LOCK, ReentrantLock())
+ # Decompressors
+ zstdd = ZstdDecompressor()
+ CodecZstd.TranscodingStreams.initialize(zstdd)
Review Comment:
Maybe we should be a bit lazier about initializing these as I believe
they're quite expensive. Shall we initialize them only as needed? For
example, if a user is only decompressing (reading) data then there's no reason
to pay the penalty of initializing compressors (and vice versa). Same idea for
LZ4 vs ZSTD - if a user only uses one then no need to initialize both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
