Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-135629255
Couple issues. Some were already mentioned by @feynmanliang :
1. Broadcasting the entire model. For example, we might just need some
pieces of it. For linear models, we only need the weights, but not summary
statistics.
2. Using trait. What if we have two objects to broadcast? For example, if
we have a local matrix factorization model. Are we going to make them a tuple
then broadcast?
3. Models are mutable, and we didn't put a contract and say users shouldn't
modify them. After this PR, if users modify the model after RDD prediction,
local prediction will see changes but RDD prediction will not. Have we
discussed this yet?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
