[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87142/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20524 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87147/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87146/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20524 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87148/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87142/testReport)** for PR 19041 at commit [`19c0e77`](https://github.com/apache/spark/commit/19c0e77ecc06d6955de22023ca03b8a1dc30fe10). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20524 **[Test build #87147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87147/testReport)** for PR 20524 at commit [`34df0cd`](https://github.com/apache/spark/commit/34df0cdcc99794f212e6cbeb55c35e3f0321c770). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87145/testReport)** for PR 20477 at commit [`c4bfbf4`](https://github.com/apache/spark/commit/c4bfbf4753d244540865b94382e35ad191818980). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87146/testReport)** for PR 20477 at commit [`c0c5895`](https://github.com/apache/spark/commit/c0c58959a1cd7e38e8055e317931475af71f67cc). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87144/testReport)** for PR 20445 at commit [`f0ce5df`](https://github.com/apache/spark/commit/f0ce5df92ba79be9d868ffaa823607fc1fb5dd5c). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87148/testReport)** for PR 20525 at commit [`2764b1c`](https://github.com/apache/spark/commit/2764b1c0aa43a104393da909f388861209220d4f). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20464: [SPARK-23291][SQL][R] R's substr should not reduce start...
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/20464 I think @felixcheung has the most context here, so I'd suggest we wait for his comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87149/testReport)** for PR 20525 at commit [`95db6d6`](https://github.com/apache/spark/commit/95db6d64051ae5cd747f7348f13dcf6637a40dd0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/657/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/20525#discussion_r166540812
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala
---
@@ -32,6 +33,24 @@ class FileFormatWriterSuite extends QueryTest with
SharedSQLContext {
}
}
+ test("SPARK-23271 empty dataframe when saved in parquet should write a
metadata only file") {
+withTempDir { inputPath =>
+ withTempPath { outputPath =>
+val anySchema = StructType(StructField("anyName", StringType) ::
Nil)
+val df = spark.read.schema(anySchema).csv(inputPath.toString)
+df.write.parquet(outputPath.toString)
+val partFiles = outputPath.listFiles()
+ .filter(f => f.isFile && !f.getName.startsWith(".") &&
!f.getName.startsWith("_"))
+assert(partFiles.length === 1)
+
+// Now read the file.
+val df1 = spark.read.parquet(outputPath.toString)
--- End diff --
@gatorsmile Thank you. Fixed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
