[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 @goldmedal, are you online now? how about fixing the PR title to say something like .. "Supports RDD of strings as input in spark.read.csv in PySpark"? ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 In a quick look, both tests failures: ``` File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/readwriter.py", line 303, in parquet return

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 umm.. I test it fine using Python 3.4 in my local. I'm not sure why did it test fail sometime... :( --- - To unsubscribe,

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82203/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82203/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82202/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82202/ Test FAILed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82201 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82201/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82201/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82203/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19339 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82202/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82198/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82201/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82198/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82198/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 This is so weird. I run it fine using Python 3.5.2 but it seems to have some problem using Python 3.4. Let me try Python 3.4 in my local. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 Hm to me .. I'd actually leave it for now. I am less sure if we should fix it now as we could sweep it out when we remove the deprecated ones later together and, for the current status, it

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82195/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82195/ Test FAILed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 ok, so maybe I create another JIRA for this issue? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 Yea, let's do it separately. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19339 As it relies on a deprecated API, I think it is also good to replace pyspark json to use Dataset. But I think it is better in another PR. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 @HyukjinKwon I think your way works fine after fixing a variable name bug (`_jsqlContext` >> `_jssql_ctx`). Should we need to modify the json part to be consistent with the csv part? ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82195/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82148/ Test PASSed. ---

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82148/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19339 **[Test build #82148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82148/testReport)** for PR 19339 at commit

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19339 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19339 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19339: [SPARK-22112][PYSPARK] Add an API to create a DataFrame ...

2017-09-25 Thread goldmedal
Github user goldmedal commented on the issue: https://github.com/apache/spark/pull/19339 @HyukjinKwon @viirya Could you review this PR? Thanks! :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org