[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20579 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-21 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r176232893 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Loggin

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-21 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r176187422 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Loggin

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r176163585 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Logging

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175952404 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -77,7 +77,6 @@ class ParquetFileFor

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175952408 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Loggin

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175950101 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -77,7 +77,6 @@ class ParquetFileForma

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175950061 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -719,4 +720,27 @@ object DataSource extends Logging

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175858198 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -546,6 +546,10 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175854733 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -546,6 +546,10 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175154988 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -542,6 +542,11 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175118143 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -542,6 +542,11 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-16 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175019613 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -542,6 +542,11 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-16 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r175013773 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -542,6 +542,11 @@ case class DataSource(

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-03-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r173625828 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167664375 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167660095 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -68,6 +68,16 @@ class ParquetFileFo

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167661193 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Qu

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167659522 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167625448 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Query

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-12 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167623896 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -68,6 +68,16 @@ class ParquetFileFor

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-11 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167465371 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -72,6 +72,29 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-11 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167463979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -68,6 +68,16 @@ class ParquetFileFo

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-11 Thread mannharleen
Github user mannharleen commented on a diff in the pull request: https://github.com/apache/spark/pull/20579#discussion_r167463659 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -68,6 +68,16 @@ class ParquetFileFo

[GitHub] spark pull request #20579: [SPARK-23372][SQL] Writing empty struct in parque...

2018-02-11 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/20579 [SPARK-23372][SQL] Writing empty struct in parquet fails during execution. It should fail earlier in the processing. ## What changes were proposed in this pull request? Running spark.em