[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-02 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161372323 @mateiz Thank you for your answer! Will try to do it soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-02 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161365442 It might be nice to only expose a smaller # of storage levels in Python, i.e. call them memory_only and memory_and_disk, but always use the serialized ones underneath.

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161212331 @davies Thank you for showing me the original JIRA by @mateiz . It sounds like it does not make sense to keep data as deserialized Java objects since data is seriali

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161206421 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161206417 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161206170 **[Test build #47041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47041/consoleFull)** for PR 10092 at commit [`d52deb5`](https://g

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161201435 **[Test build #47041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47041/consoleFull)** for PR 10092 at commit [`d52deb5`](https://gi

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/10092#issuecomment-161200705 This is on purpose, see https://issues.apache.org/jira/browse/SPARK-2014 cc @mateiz --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-12091] [PYSPARK] [Minor] Default storag...

2015-12-01 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/10092 [SPARK-12091] [PYSPARK] [Minor] Default storage level of persist/cache The current default storage level of Python persist API is MEMORY_ONLY_SER. This is different from the default level MEMORY