Eron, Thanks for sending out this list! We can make some of the critical ones public for 1.5, but they will be marked DeveloperApi since they may require changes in the future. Just made the JIRA: [ https://issues.apache.org/jira/browse/SPARK-9704] and I'll send a PR soon.
Joseph On Mon, Aug 3, 2015 at 4:51 PM, Eron Wright <ewri...@live.com> wrote: > > Hello, > > In developing new *third-party* *pipeline components* for Spark ML 1.4 > (see dl4j-spark-ml), I encountered a few gaps in the earlier effort to make > the ML Developer APIs public (SPARK-5995). I plan to file issues after > we discuss on this thread. The below is a list of types that are > presently private but might best be made public. > > 1. *VectorUDT*. To define a relation with a vector field, > VectorUDT must be instantiated. > 2. *SchemaUtils*. Third-party pipeline components have a need for > checking column types and appending columns. > 3. *Identifiable trait*. The trait generates a unique identifier for > the associated pipeline component. Nice to have a consistent format by > reusing the trait. > 4. *ProbabilisticClassifier*. Third-party components should leverage > the complex logic around computing only selected columns. > 5. *Shared Params* (HasLabel, HasFeatures). This is covered in > SPARK-7146 but reiterating it here. > > Thanks, > Eron Wright >
