[
https://issues.apache.org/jira/browse/SPARK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372091#comment-16372091
]
Joseph K. Bradley commented on SPARK-15526:
-------------------------------------------
[~srowen] I just found that PMML is now listed as a dependency in a lot (all?)
of the Spark modules: catalyst, mllib-local, etc. I haven't yet tested to see
if these 2 PRs caused that change, but it seems likely.
> Shade JPMML
> -----------
>
> Key: SPARK-15526
> URL: https://issues.apache.org/jira/browse/SPARK-15526
> Project: Spark
> Issue Type: Dependency upgrade
> Components: ML, MLlib
> Affects Versions: 2.0.0
> Reporter: Villu Ruusmann
> Assignee: Sean Owen
> Priority: Minor
> Fix For: 2.3.0
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> The Spark-MLlib module depends on the JPMML-Model library
> (org.jpmml:pmml-model:1.2.7) for its PMML export capabilities. The
> JPMML-Model library is included in the Apache Spark assembly, which makes it
> very difficult to build and deploy competing PMML exporters that may wish to
> depend on different versions (typically much newer) of the same library.
> JPMML-Model library classes are not part of Apache Spark public APIs, so it
> shouldn't be a problem if they are relocated by prepending a prefix
> "org.spark_project" to their package names using Maven Shade Plugin. The
> requested treatment is identical to how Google Guava and Jetty dependencies
> are shaded in the final assembly.
> This issue is raised in relation to the JPMML-SparkML project
> (https://github.com/jpmml/jpmml-sparkml), which provides PMML export
> capabilities for Spark ML Pipelines. Currently, application developers who
> wish to use it must tweak their application classpath, which assumes
> familiarity with build internals.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
