Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/3637#issuecomment-70029316
@etrain About abstracting the optimized transform() in
LogisticRegressionModel: Here are a few possibilities. What are your thoughts?
* Goals:
* Optimize transform: Make it fast, and make it output only the desired
columns.
* Easy development
* Support Classifier, Regressor, and ProbabilisticClassifier
* (currently) Developers implement predictX methods for each output column
X. They may override transform() to optimize speed.
* Pros: predictX is easy to understand.
* Cons: An optimized transform() is annoying to write.
* Developers implement more basic transformation methods, such as
features2raw, raw2pred, raw2prob.
* Pros: Abstract classes may implement optimized transform().
* Cons: Different types of predictors require different methods:
* Predictor and Regressor: features2pred
* Classifier: features2raw, raw2pred
* ProbabilisticClassifier: raw2prob
* Developers implement a single predict() method which takes parameters for
what columns to output (returning tuple or some type with None for missing
values). Abstract classes take the outputs they want and put them into columns.
* Pros: Developers only write 1 method and can optimize it as much as
they want. It could be more optimized than the previous 2 options; e.g., if
LogisticRegressionModel only wants the prediction, then it never has to
construct intermediate results such as the vector of raw predictions.
* Cons: predict() will have a different signature for different
abstractions, based on the possible output columns.
@etrain Enumeration for labels: It could be hard to do since Enumeration
doesnât cooperate with Java. Also, ML attributes should be able to hold this
metadata. I like strong typing too...but itâs hard to make it work well for
all the APIs.
@tomerk Parameterizing sharedParams to eliminate need for setters: This
would make class declarations even longer, with the Estimator/Transformer type
specified for every Param mixed in. Also, there are some parameters which
should not have setters; e.g., a LogisticRegressionModel should have a
getRegParam() method, but it should not have a setRegParam() method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
