[ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16998577#comment-16998577 ]
Imran Rashid commented on SPARK-6235: ------------------------------------- [~sammysheep] Spark's ML library uses the same jira project, if that is what you meant -- but I don't know of what specifically has already been implemented in spark to deal w/ large PCA or if there is another specific issue just for that. I'd suggest you first ask u...@spark.apache.org, since the first question is if there is another way of dealing with this > Address various 2G limits > ------------------------- > > Key: SPARK-6235 > URL: https://issues.apache.org/jira/browse/SPARK-6235 > Project: Spark > Issue Type: Umbrella > Components: Shuffle, Spark Core > Reporter: Reynold Xin > Priority: Major > Fix For: 2.4.0 > > Attachments: SPARK-6235_Design_V0.02.pdf > > > An umbrella ticket to track the various 2G limit we have in Spark, due to the > use of byte arrays and ByteBuffers. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org