[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-29 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533906#comment-15533906 ] Mike Dusenberry commented on SYSTEMML-946: -- [~mboehm7] I've created SYSTEMML-993 with this

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-29 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533828#comment-15533828 ] Mike Dusenberry commented on SYSTEMML-946: -- I think this still needs to be explored further.

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512045#comment-15512045 ] Mike Dusenberry commented on SYSTEMML-946: -- The DataFrames were previously computed and saved

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Matthias Boehm (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511348#comment-15511348 ] Matthias Boehm commented on SYSTEMML-946: - ok, I'll have a look into this. > OOM on spark

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511329#comment-15511329 ] Mike Dusenberry commented on SYSTEMML-946: -- [~mboehm7] This is using a slightly modified

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Matthias Boehm (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15511091#comment-15511091 ] Matthias Boehm commented on SYSTEMML-946: - what is the script you're execution here? > OOM on

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510944#comment-15510944 ] Mike Dusenberry commented on SYSTEMML-946: -- There was an additional GC limit error: {code}

[jira] [Commented] (SYSTEMML-946) OOM on spark dataframe-matrix / csv-matrix conversion

2016-09-21 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SYSTEMML-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510929#comment-15510929 ] Mike Dusenberry commented on SYSTEMML-946: -- [~mboehm7] With this update, I was able to get