Github user staple commented on the pull request:
https://github.com/apache/spark/pull/2347#issuecomment-55747592
Hi, per the discussion in https://github.com/apache/spark/pull/2362 the
plan is to continue caching before deserialization from python rather than
after, in order to minimize the cached rdd memory footprint.
This means that, without further work, warning messages will be logged for
every python mllib regression and kmeans run. I added a patch that suppresses
these warning messages during python runs in a way that I think is fairly
unobtrusive. Please let me know what you think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
