PJ Fanning created SPARK-40911:
----------------------------------
Summary: Upgrade jackson-module-scala to 2.14.0
Key: SPARK-40911
URL: https://issues.apache.org/jira/browse/SPARK-40911
Project: Spark
Issue Type: Improvement
Components: Java API
Affects Versions: 3.3.0
Reporter: PJ Fanning
This 2.14.0 release is still a few weeks a way. There is an rc2 release but
there will probably be an rc3 before a full release.
The reason I marked the Jira component as 'Java API' is that this issue will
affect Java users more than Scala users.
I raised this separately to SPARK-40666 because I have a different reason for
this issue. SPARK-40666 can probably already be closed as the CVE is fixed in
jackson-databind 2.13.4.2.
There are performance issues in jackson-module-scala 2.13.x that may affect
some Spark users. Specifically, the per issue is
[https://github.com/FasterXML/jackson-module-scala/issues/576]
Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala
2.13, you should be able to use Scala3 compiled classes with
jackson-module-scala. Scala3 compiled classes are harder to recognise using
runtime reflection (and Jackson is built around runtime reflection). Scala2
compiled classes have specific annotations. With Scala3 compiled classes, we
need to look for .tasty files. This lookup can be slow if you have a lot of
jars (or big jars). Issue
[576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an
issue where this .tasty lookup is done every time you try to
serialize/deserialize a Java class with an ObjectMapper that has the
DefaultScalaModule registered.
For Scala usage, it may be worth turning off this .tasty file support
altogether. This is another enhancement in jackson-module-scala (but not in the
RC2 release).
I will follow up and update this issue when the v2.14.0 release is ready.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
