Hi Chris, > The deserialization system should not assume a copy is necessary >> <https://github.com/apache/arrow/blob/ecba35cac76185f59c55048b82e895e5f8300140/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java#L381>. >> >> > This is one of many ways to reconstruct an arrow record batch. We > frequently reconstruct without any copies. It'd be great if you looked to > contribute some of the improvements you believe are needed back to the > project. >
+1. If I didn't say this on the previous thread. IIRC, there is nothing about the VectorLoader [1] that assumes copies, this just needs to be pushed further down the stack. My opinion is that a better design for the Arrow JVM bindings would be to > have each record batch be potentially allocated but remove allocators from > the vectors themselves. Could you expand on this? What problems to allocators on Vectors present? Lastly, if you are running benchmarks, please checkout performance tuning section of the README [2] which includes environment variables that would be set under production scenarios (I had a little trouble following the clojure call but it does look like it is calling "get" on the Float8Vector?). -Micah [1] https://github.com/apache/arrow/blob/ecba35cac76185f59c55048b82e895e5f8300140/java/vector/src/main/java/org/apache/arrow/vector/VectorLoader.java [2] https://github.com/apache/arrow/tree/master/java#performance-tuning
