Hi Bryan, Thanks for bringing this up. +1 for the change.
I am not clear what is the right place to override the jvm property. It is possible that when we try to override it (possibly in a static block), the old property value has already been read by netty library. To avoid this problem, do we need to control the order of class loading? Best, Liya Fan On Mon, Nov 18, 2019 at 3:17 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > This sounds reasonable to me. At this point I think having our consumers > have a better experience is more important then library purity concerns > I've had in the past. > > Do we need to handle jdk8 as a special case? Do you think it pays to try > to find an alternate library that doesn't require special flags for > whatever we are using this functionality for? > > Thanks, > Micah > > On Sunday, November 17, 2019, Bryan Cutler <cutl...@gmail.com> wrote: > > > After ARROW-3191 [1], consumers of Arrow Java with a JDK 9 and above are > > required to set the JVM property "io.netty.tryReflectionSetAccessible= > > true" > > at startup, each time Arrow code is run, as documented at [2]. Not doing > > this will result in the error "java.lang.UnsupportedOperationException: > > sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available". > > This is due to a part of the Netty codebase, and I'm not sure if there > any > > way around it, but I don't think it's the correct behavior for Arrow > > out-of-the-box to fail with a 3rd party error by default. This has come > up > > before in our own unit testing [3], and most recently when upgrading > Arrow > > in Spark [4]. > > > > I'd like to propose that Arrow Java change to the following behavior: > > > > 1) check to see if the property io.netty.tryReflectionSetAccessible has > > been set > > 2) if not set, automatically set to "true" > > 3) else if set to "false", catch the Netty error and prepend the error > > message with the suggested setting of "true" > > > > What do other devs think? > > > > [1] https://issues.apache.org/jira/browse/ARROW-3191 > > [2] https://github.com/apache/arrow/tree/master/java#java-properties > > [3] https://issues.apache.org/jira/browse/ARROW-5412 > > [4] https://github.com/apache/spark/pull/26552 > > >
