[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15942976#comment-15942976 ]
Saisai Shao edited comment on SPARK-20059 at 3/27/17 10:02 AM: --------------------------------------------------------------- [~sowen], it probably is not the same issue. In SPARK-20019, jars are added through SQL command in to Hive's SessionState, the code path is different. And in SPARK-11421, AFAIK, the target is to add jars to the current classloader in the runtime. Here the issue is during start, jars specified through {{--jars}} should be added to Spark's child classloader in yarn cluster mode. They're probably all classloader related issues, but I think they're targeted to the different area, and touched the different code path. was (Author: jerryshao): [~sowen], it probably is not the same issue. In SPARK-20019, jars are added through SQL command in to Hive's SessionState, the code path is different. And in SPARK-11421, AFAIK, the target is to add jars to the current classloader in the runtime. Here the issue is during start, jars specified through {{--jars}} should be added to Spark's child classloader in yarn cluster mode. They're all probably all classloader related issues, but I think they're targeted to the different area, and touched the different code path. > HbaseCredentialProvider uses wrong classloader > ---------------------------------------------- > > Key: SPARK-20059 > URL: https://issues.apache.org/jira/browse/SPARK-20059 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 2.1.0, 2.2.0 > Reporter: Saisai Shao > > {{HBaseCredentialProvider}} uses system classloader instead of child > classloader, which will make HBase jars specified with {{--jars}} fail to > work, so here we should use the right class loader. > Besides in yarn cluster mode jars specified with {{--jars}} is not added into > client's class path, which will make it fail to load HBase jars and issue > tokens in our scenario. Also some customized credential provider cannot be > registered into client. > So here I will fix this two issues. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org