GitHub user sarutak opened a pull request:
https://github.com/apache/spark/pull/10358
[SPARK-12366][SQL][REPL] ExecutorClassLoader should skip finding classes
generated by Janino
I think, ExecutorClassLoader should not load classes generated by Janino so
ExecutorClassLoader should skip to find those classes to avoid useless RPC and
getting error messages. Otherwise, we can get error messages when we use REPL
and generated code.
Error messages are like as follows ( @jaceklaskowski reported ).
```
scala> val df =
sqlContext.read.json("examples/src/main/resources/people.json")
df: org.apache.spark.sql.DataFrame = [age: bigint, name: string]
scala> df.show
ERROR TransportRequestHandler: Error opening stream
/classes/org/apache/spark/sql/catalyst/expressions/GeneratedClass.class for
request from /172.20.4.141:55136
java.lang.IllegalArgumentException: requirement failed: File not found:
/classes/org/apache/spark/sql/catalyst/expressions/GeneratedClass.class
at scala.Predef$.require(Predef.scala:219)
at
org.apache.spark.rpc.netty.NettyStreamManager.openStream(NettyStreamManager.scala:60)
at
org.apache.spark.network.server.TransportRequestHandler.processStreamRequest(TransportRequestHandler.java:136)
at
org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:106)
at
org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:104)
at
org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:51)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sarutak/spark SPARK-12366
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10358.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10358
----
commit 9269b0ba9d4788e8aee3154bd838014551f0aaa0
Author: Kousuke Saruta <[email protected]>
Date: 2015-12-17T11:45:20Z
Fixed ExecutorClassLoader so that ignore finding classes related to
GeneratedClass
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
