[
https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382178#comment-15382178
]
Nick Pentreath commented on SPARK-16365:
----------------------------------------
Fair point Sean about the lists. However, I've seen many emails (and it's
increased a lot recently, at least anecdotally as far as I see) on user lists
around deploying Spark models. Each time there is some discussion, and I
normally respond with something along the lines of "the options are (a) your
own personal format; (b) parse Spark's format and (c) PMML".
The discussion usually gets lost or peters out on the lists. I'd like to try
gather some more complete discussion and thoughts here in one unified place.
I'm happy to shoot out an email to both lists with a link here to encourage
people to chime in.
The end result may be something in the spectrum from "do nothing" to "make it
easier to have pluggable export formats for models" to "do something more
full-blown in Spark mllib-local". I just want users (and especially the ML
devs) to get on the same page here about medium-to-long term goals for Spark
ML, precisely so we don't end up in some sort of half-way situation.
This applies for both the "local linear algebra?" and "local models/pipelines?"
questions in my mind. I think we need some good discussion upfront before doing
anything on this (if indeed that is the ultimate decision).
> Ideas for moving "mllib-local" forward
> --------------------------------------
>
> Key: SPARK-16365
> URL: https://issues.apache.org/jira/browse/SPARK-16365
> Project: Spark
> Issue Type: Brainstorming
> Components: ML
> Reporter: Nick Pentreath
>
> Since SPARK-13944 is all done, we should all think about what the "next
> steps" might be for {{mllib-local}}. E.g., it could be "improve Spark's
> linear algebra", or "investigate how we will implement local models/pipelines
> in Spark", etc.
> This ticket is for comments, ideas, brainstormings and PoCs. The separation
> of linalg into a standalone project turned out to be significantly more
> complex than originally expected. So I vote we devote sufficient discussion
> and time to planning out the next move :)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
