[
https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-11081.
--------------------------------
Resolution: Not A Problem
Ok, I filed SPARK-12154 to track it.
> Make spark-core pull in Jersey and javax.ws.rs dependencies separately for
> easier overriding
> --------------------------------------------------------------------------------------------
>
> Key: SPARK-11081
> URL: https://issues.apache.org/jira/browse/SPARK-11081
> Project: Spark
> Issue Type: Improvement
> Components: Build, Spark Core
> Reporter: Mingyu Kim
>
> As seen from this thread
> (https://mail-archives.apache.org/mod_mbox/spark-user/201510.mbox/%3CCALte62yD8H3=2KVMiFs7NZjn929oJ133JkPLrNEj=vrx-d2...@mail.gmail.com%3E),
> Spark is incompatible with Jersey 2 especially when Spark is embedded in an
> application running with Jersey.
> There was an in-depth discussion on options for shading and making it easier
> for users to be able to use Jersey 2 with Spark applications:
> https://github.com/apache/spark/pull/9615
> To recap the discussion, Jersey 1 has two issues:
> 1. It has classes listed in META-INF/services/ files that would be loaded
> even if Jersey 2 was being loaded on the classpath in a higher precedence.
> This means that Jersey 2 would attempt to use Jersey 1 implementations in
> some places regardless of user attempts to override the dependency with
> things like userClassPathFirst.
> 2. Jersey 1 packages javax.ws.rs classes inside itself, making it hard to
> exclude just javax.ws.rs APIs and replace them with ones that Jersey 2 is
> compatible with.
> Also discussed was the fact that plain old shading doesn't work here, since
> you would need to shade lines in META-INF/services as well, not just classes.
> Not to mention that shading JAX-RS annotations is tricky as well.
> To recap the discussion as what needs to happen Spark-side, we need to:
> 1. Create a "org.spark-project.jersey" artifact (loosely speaking) which is
> the Jersey 1 jar minus all the javax.ws.rs stuff (no need to actually
> shade/namespace the classes that way, just the artifact name)
> 2. Put all the javax.ws.rs stuff extracted from step 1 into its own artifact,
> say "org.spark-project.javax.ws.rs". (META-INF/services/javax.ws.rs* files
> live in this artifact as well)
> 3. Spark-core's pom depends on org.spark-project artifacts from step 1 and 2
> 4. Spark assembly excludes META-INF/services/javax.ws.rs.* - it turns out
> these files aren't actually necessary for Jersey 1 to function properly in
> general (we need to test this more however)
> Now a user that wants to depend on Jersey 2, and is depending on Spark maven
> artifacts, would do the following in their application
> 1. Provide my own dependency on Jersey 2 and its transitive javax.ws.rs
> dependencies
> 2. In my application's dependencies, exclude org.spark-project.javax.ws.rs
> from spark-core. We keep org.spark-project.jersey because spark-core needs
> it, but it will use the javax.ws.rs classes that my application is providing.
> 3. Set spark.executor.userClassPathFirst=true and ship Jersey 2 and new
> javax.ws.rs jars to the executors
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
