headius commented on issue #35589:
URL: https://github.com/apache/arrow/issues/35589#issuecomment-1548072166
Success! Though I think it would be better to set up the proper
jar-dependencies logic instead of hand-requiring these scripts.
After installing arrow-vector and arrow-memory-netty like this:
```
$ mvn dependency:get -DgroupId=org.apache.arrow -DartifactId=arrow-vector
-Dversion=12.0.0
...
$ mvn dependency:get -DgroupId=org.apache.arrow
-DartifactId=arrow-memory-netty -Dversion=12.0.
...
```
I was able to run the following script (the slf4j errors are likely because
I just don't have the right jars for it loaded):
```ruby
require
'~/.m2/repository/org/apache/arrow/arrow-vector/12.0.0/arrow-vector-12.0.0.jar'
require
'~/.m2/repository/org/apache/arrow/arrow-memory-core/12.0.0/arrow-memory-core-12.0.0.jar'
require
'~/.m2/repository/org/apache/arrow/arrow-memory-netty/12.0.0/arrow-memory-netty-12.0.0.jar'
require
'~/.m2/repository/org/apache/arrow/arrow-format/12.0.0/arrow-format-12.0.0.jar'
require
'~/.m2/repository/io/netty/netty-buffer/4.1.90.Final/netty-buffer-4.1.90.Final.jar'
require
'~/.m2/repository/io/netty/netty-common/4.1.90.Final/netty-common-4.1.90.Final.jar'
require
'~/.m2/repository/com/google/flatbuffers/flatbuffers-java/1.12.0/flatbuffers-java-1.12.0.jar'
require '~/.m2/repository/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar'
require
'~/.m2/repository/org/slf4j/jul-to-slf4j/1.7.25/jul-to-slf4j-1.7.25.jar'
require
'~/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.25/jcl-over-slf4j-1.7.25.jar'
java_import org.apache.arrow.memory.RootAllocator
java_import org.apache.arrow.vector.IntVector
begin
allocator = RootAllocator.new
int_vector = IntVector.new("fixed-size-primitive-layout", allocator)
int_vector.allocate_new(3)
int_vector.set(0,1)
int_vector.set_null(1)
int_vector.set(2,2)
int_vector.set_value_count(3);
puts "Vector created in memory: #{int_vector}"
ensure
int_vector.close rescue nil
allocator.close rescue nil
end
```
Running it with the requisite `--add-opens` flag that the arrow Java
bindings need:
```
$ jruby -J--add-opens -Jjava.base/java.nio=ALL-UNNAMED arrow-vector.rb
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Vector created in memory: [1, null, 2]
```
So that's a basic start!
A few things to improve before moving forward:
* Get jars loading using the aforementioned jar-dependencies, so we can just
use `require_jar` to pull in arrow-vector and arrow-memory-netty with all
dependencies.
* The add-opens is probably a requirement because the netty memory
implementation needs access to the internals of
Java's NIO ByteBuffer class. There may be alternative ways that don't need
to open up ByteBuffer, or we can make this
work in JRuby by adding the `--add-opens` flag to `.jruby.java_opts` which
gets loaded automatically:
```
[] jruby-arrow-vector $ cat .jruby.java_opts
--add-opens java.base/java.nio=ALL-UNNAMED
[] jruby-arrow-vector $ jruby arrow-vector.rb
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Vector created in memory: [1, null, 2]
```
Something using JRuby's FFI or Project Panama's native memory access might
make sense.
* I'm not sure who is best to work on this! Someone familiar with the Java
API would be good, but also someone familiar with Ruby :-)
I'm glad this was reasonably easy to get working. How do you want to proceed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
