srowen commented on issue #26552: [SPARK-29923][SQL][TESTS] Set io.netty.tryReflectionSetAccessible for Arrow on JDK9+ URL: https://github.com/apache/spark/pull/26552#issuecomment-554636446 OK, at least we need to mark this as a release-notes item, until it's undone. BTW this isn't really an Arrow behavior, but a Netty one: http://git.yunmaozj.com/opensource/netty/commit/e72c197aa3a88e97c10793ee7870775ab8807f92 I think it's not great to default to not-working on JDK 9+ vs print a warning. But it happened 2 years ago. I guess Spark doesn't trigger the same kind of issue in its use of Netty. We can try to set this system property at startup in Spark, though I don't know if we can do it early enough. I don't know if Arrow can or wants to try to set this, likewise? like if not set, set to true? This isn't ideal for JDK 9+ users as several things won't work unless their deployment also sets this value - and same could be true of any Arrow user. See also https://gitter.im/netty/netty?at=5ce3e8d513e9854e334005d7 which suggests using a different allocator avoids this, but I don't know if it's directly helpful. I think the issue is that Arrow is reaching into its PlatformDependent class directly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
