[
https://issues.apache.org/jira/browse/MAHOUT-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14582660#comment-14582660
]
ASF GitHub Bot commented on MAHOUT-1660:
----------------------------------------
Github user dlyubimov commented on a diff in the pull request:
https://github.com/apache/mahout/pull/135#discussion_r32277536
--- Diff:
spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmSpark.scala
---
@@ -78,8 +78,8 @@ class CheckpointedDrmSpark[K: ClassTag](
}
def cache() = {
- if (!cached) {
- rdd.persist(_cacheStorageLevel)
+ if (!cached && _cacheStorageLevel != StorageLevel.NONE) {
--- End diff --
NONE means yes, it's not cached.
The material effect of setting "NONE" cache level in spark, as far as i
understand, is exactly as not setting anything at all; the difference is
once you set it to anything, you cannot change it. So this means if i call
"NONE" it would lock me to that choice with spark (which did not make much
sense to me) otherwise it doesn't matter.
On Thu, Jun 11, 2015 at 3:56 PM, Andrew Musselman <[email protected]>
wrote:
> In
>
spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmSpark.scala
> <https://github.com/apache/mahout/pull/135#discussion_r32277244>:
>
> > @@ -78,8 +78,8 @@ class CheckpointedDrmSpark[K: ClassTag](
> > }
> >
> > def cache() = {
> > - if (!cached) {
> > - rdd.persist(_cacheStorageLevel)
> > + if (!cached && _cacheStorageLevel != StorageLevel.NONE) {
>
> Will this be confusing if someone tries to call cache() with NONE set and
> it's not cached?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/mahout/pull/135/files#r32277244>.
>
> Hadoop1HDFSUtil.readDRMHEader should be taking Hadoop conf
> ----------------------------------------------------------
>
> Key: MAHOUT-1660
> URL: https://issues.apache.org/jira/browse/MAHOUT-1660
> Project: Mahout
> Issue Type: Bug
> Components: spark
> Affects Versions: 0.10.0
> Reporter: Suneel Marthi
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.10.2
>
>
> Hadoop1HDFSUtil.readDRMHEader should be taking Hadoop configuration from
> Context and not ignore it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
