[
https://issues.apache.org/jira/browse/SPARK-8400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706182#comment-14706182
]
Xiangrui Meng commented on SPARK-8400:
--------------------------------------
Sorry for my late reply! We check numBlocks in LocalIndexEncoder. However, I'm
not sure whether this happens before any data shuffling. It might be better to
check numUserBlocks and numItemBlocks directly.
> ml.ALS doesn't handle -1 block size
> -----------------------------------
>
> Key: SPARK-8400
> URL: https://issues.apache.org/jira/browse/SPARK-8400
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 1.3.1
> Reporter: Xiangrui Meng
> Assignee: Bryan Cutler
>
> Under spark.mllib, if number blocks is set to -1, we set the block size
> automatically based on the input partition size. However, this behavior is
> not preserved in the spark.ml API. If user sets -1 in Spark 1.3, it will not
> work, but no error messages will show.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
