[
https://issues.apache.org/jira/browse/MAHOUT-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15247992#comment-15247992
]
ASF GitHub Bot commented on MAHOUT-1572:
----------------------------------------
Github user dlyubimov commented on the pull request:
https://github.com/apache/mahout/pull/10#issuecomment-211983401
yes this is not the way to do it
actually it seems vectors are not reporting density faithfully, i had to do
a more thorough analysis of data density elsewhere before doing things. this
probably needs to be patched with a more thorough technique.
+1
> blockify() to detect (naively) the data sparsity in the loaded data
> --------------------------------------------------------------------
>
> Key: MAHOUT-1572
> URL: https://issues.apache.org/jira/browse/MAHOUT-1572
> Project: Mahout
> Issue Type: Bug
> Reporter: Dmitriy Lyubimov
> Fix For: 0.10.0
>
>
> per [~ssc]:
> .bq a dense matrix is converted into a SparseRowMatrix with dense row vectors
> by blockify(), after serialization this becomes a dense matrix in sparse
> format (triggering OOMs)!
> i guess we can look at first row vector and go on to either DenseMatrix or
> SparseRowMatrix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
