vanzin commented on issue #24502: [SPARK-27610] - Yarn external shuffle service fails to start when spark.shuffle.io.mode=EPOLL URL: https://github.com/apache/spark/pull/24502#issuecomment-488159709 I'm still a little at a loss about how this works. Following the netty code, given the file name of the library, the translated name for e.g. `io/netty/channel/epoll/Native` will be `org/spark/project/io/netty/channel/epoll/Native`, because of this code: ``` // Package names must be sanitized, in JNI packages names are separated by '/' characters. for (; temp != packageNameEnd; ++temp) { if (*temp == '_') { *temp = '/'; } } ``` So the native library should fail to register because that class does not exist (see `netty_unix_util_register_natives`). That shouldn't be hard to fix (just change the relocated name to not have an underscore), but it makes me wonder how you're verifying that this fix actually works.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
