Davies Liu created SPARK-11767:
----------------------------------
Summary: Easy to OOM when cache large column
Key: SPARK-11767
URL: https://issues.apache.org/jira/browse/SPARK-11767
Project: Spark
Issue Type: Improvement
Reporter: Davies Liu
The default batch size (10000) does not work well the large column (with
serialized size about 100k), it's easy to OOM when unrolling the rows.
We should limit the serialized size of batch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
