xiaoyanxie commented on PR #2395: URL: https://github.com/apache/auron/pull/2395#issuecomment-5187990961
Hi @ShreyeshArangath, could you please help to review this PR? This PR fixes #2386. The problem is in `NativeConverters.deserializeExpression`. It used a plain `ObjectInputStream`. The default `resolveClass` resolves each class by `VM.latestUserDefinedLoader()`, which selects a class loader from the call stack. When Auron is provided by `spark.jars`, that loader cannot see the Auron classes. So the scalar subquery graph is only partly deserialized, and a `ClassCastException` is thrown. The fix is to resolve the classes with an explicit class loader. Spark's own `JavaDeserializationStream` does the same thing. This PR also adds a TPC-DS CI job for the runtime bloom filter. The job fails on the commit before the fix, and passes with the fix. If you prefer, I can move the CI job into a separate PR. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
