Karen Feng created SPARK-33043:
----------------------------------
Summary: RowMatrix is incompatible with
spark.driver.maxResultSize=0
Key: SPARK-33043
URL: https://issues.apache.org/jira/browse/SPARK-33043
Project: Spark
Issue Type: Bug
Components: MLlib
Affects Versions: 3.0.1, 3.0.0
Reporter: Karen Feng
RowMatrix does not work if spark.driver.maxResultSize=0, as this requirement
breaks:
{code:java}
require(maxDriverResultSizeInBytes > aggregatedObjectSizeInBytes, s"Cannot
aggregate object of size $aggregatedObjectSizeInBytes Bytes, " + s"as
it's bigger than maxResultSize ($maxDriverResultSizeInBytes Bytes)")
{code}
[https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala#L795.]
This check should likely only happen if maxDriverResultSizeInBytes > 0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
