Github user johnc1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/17459#discussion_r109320468
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrix.scala
---
@@ -113,6 +114,67 @@ class IndexedRowMatrix @Since("1.0.0") (
}
/**
+ * Converts to BlockMatrix. Creates blocks of `DenseMatrix` with size
1024 x 1024.
+ */
+ def toBlockMatrixDense(): BlockMatrix = {
--- End diff --
I have been going back and forth on this myself. I think converting to a
BlockMatrix backed by dense matrices is better default behavior than one backed
by sparse matrices, but the the current implementation of toBlockMatrix
advertises that it converts to a BlockMatrix backed SparseMatrices, and I
thought changing that could negatively affect people who want that behavior. I
suppose we could add a default argument to toBlockMatrix like `isSparse = true`
so that it would not break anyone's code but people would be able to convert to
dense version if they wanted. What do you think of that?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
