GitHub user rednaxelafx opened a pull request:
https://github.com/apache/spark/pull/22506
[SPARK-25494][SQL] Upgrade Spark's use of Janino to 3.0.10
## What changes were proposed in this pull request?
This PR upgrades Spark's use of Janino from 3.0.9 to 3.0.10.
Note that 3.0.10 is a out-of-band release specifically for fixing an
integer overflow issue in Janino's `ClassFile` reader. It is otherwise exactly
the same as 3.0.9, so it's a low risk and compatible upgrade.
The integer overflow issue affects Spark SQL's codegen stats collection:
when a generated Class file is huge, especially when the constant pool size is
above `Short.MAX_VALUE`, Janino's `ClassFile reader` will throw an exception
when Spark wants to parse the generated Class file to collect stats. So we'll
miss the stats of some huge Class files.
The related Janino issue is:
https://github.com/janino-compiler/janino/issues/58
## How was this patch tested?
Existing codegen tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rednaxelafx/apache-spark upgrade-janino
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22506.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22506
----
commit c3f8a6b41c9409339bf62fb17cd4cd905853d97f
Author: Kris Mok <kris.mok@...>
Date: 2018-09-20T23:08:27Z
bump Janino to 3.0.10
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
