Sean Owen created SPARK-22128:
---------------------------------
Summary: Update paranamer to 2.8 to avoid BytecodeReadingParanamer
ArrayIndexOutOfBoundsException with Scala 2.12 + Java 8 lambda
Key: SPARK-22128
URL: https://issues.apache.org/jira/browse/SPARK-22128
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 2.3.0
Reporter: Sean Owen
Assignee: Sean Owen
Priority: Minor
Minor, but probably needs a JIRA.
After updating to Scala 2.12 I encountered this issue pretty quickly in tests:
https://github.com/paul-hammant/paranamer/issues/17
{code}
java.lang.ArrayIndexOutOfBoundsException: ....
at
com.thoughtworks.paranamer.BytecodeReadingParanamer$ClassReader.accept(BytecodeReadingParanamer.java:563)
...
{code}
Spark depends on jackson-module-paranamer 2.6.7 to match the other jackson
deps, and that depends on paranamer 2.6. The bug above is fixed in 2.8.
However I noticed that, really, Spark uses jackson-module-scala 2.6.7.1, and
that in turn actually depends on jackson-module-paranamer 2.7.9 (kind of odd),
which already depends on 2.8.
So it seemed prudent to just not manually manage this dependency down. Then we
still need to manage paranamer to 2.8 because Avro 1.7 pulls in 2.3.
But it all seems to work in a quick test. And it's necessary to get 2.12
working.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
