[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[marmbrus]

Github user marmbrus commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191991412
  
Changes like this should probably also update the programming guide.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/11464


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191899042
  
Thanks - merging this in master.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191692108
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52384/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191692104
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191691777
  
**[Test build #52384 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52384/consoleFull)**
 for PR 11464 at commit 
[`d2ace37`](https://github.com/apache/spark/commit/d2ace37bea1cd8a4c8e1cfb4b1b7fb2ace67c005).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191691003
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191691005
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52385/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191690696
  
**[Test build #52385 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52385/consoleFull)**
 for PR 11464 at commit 
[`e4d4cfb`](https://github.com/apache/spark/commit/e4d4cfbc5a65ad95c4eee31ea5a2f09de7e5934a).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191648914
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52378/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191648913
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191648669
  
**[Test build #52378 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52378/consoleFull)**
 for PR 11464 at commit 
[`2817c9a`](https://github.com/apache/spark/commit/2817c9a774f54c5f79828137bdb5a95c391c90c3).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191646312
  
**[Test build #52385 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52385/consoleFull)**
 for PR 11464 at commit 
[`e4d4cfb`](https://github.com/apache/spark/commit/e4d4cfbc5a65ad95c4eee31ea5a2f09de7e5934a).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191644382
  
test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191644225
  
**[Test build #52384 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52384/consoleFull)**
 for PR 11464 at commit 
[`d2ace37`](https://github.com/apache/spark/commit/d2ace37bea1cd8a4c8e1cfb4b1b7fb2ace67c005).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54844054
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
 ---
@@ -396,6 +402,46 @@ class CSVSuite extends QueryTest with SharedSQLContext 
with SQLTestUtils {
 }
   }
 
+  test("SPARK-13543 Set explicitly the output as uncompressed") {
--- End diff --

Yea - i think that's better


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54843475
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRelation.scala
 ---
@@ -148,6 +148,19 @@ private[sql] class ParquetRelation(
 .get(ParquetRelation.METASTORE_SCHEMA)
 .map(DataType.fromJson(_).asInstanceOf[StructType])
 
+  private val compressionCodec: Option[String] = parameters
+.get("compression")
+.map { codecName =>
+  // Validate if given compression codec is supported or not.
+  val shortParquetCompressionCodecNames = 
ParquetRelation.shortParquetCompressionCodecNames
+  if 
(!shortParquetCompressionCodecNames.contains(codecName.toLowerCase)) {
+val availableCodecs = 
shortParquetCompressionCodecNames.keys.map(_.toLowerCase)
+throw new IllegalArgumentException(s"Codec [$codecName] " +
+  s"is not available. Available codecs are 
${availableCodecs.mkString(", ")}.")
+  }
--- End diff --

Ops. This indentation is weird.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54843218
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
 ---
@@ -396,6 +402,46 @@ class CSVSuite extends QueryTest with SharedSQLContext 
with SQLTestUtils {
 }
   }
 
+  test("SPARK-13543 Set explicitly the output as uncompressed") {
--- End diff --

Yes, which one would be better?
Should I just write like `write the uncompressed output via option`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54843031
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
 ---
@@ -396,6 +402,46 @@ class CSVSuite extends QueryTest with SharedSQLContext 
with SQLTestUtils {
 }
   }
 
+  test("SPARK-13543 Set explicitly the output as uncompressed") {
--- End diff --

is it to test whether uncompressed mode would work?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54843017
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
 ---
@@ -396,6 +402,46 @@ class CSVSuite extends QueryTest with SharedSQLContext 
with SQLTestUtils {
 }
   }
 
+  test("SPARK-13543 Set explicitly the output as uncompressed") {
--- End diff --

what exactly does this test do? it's not super clear here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[davies]

Github user davies commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191613880
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191601135
  
@davies can you help review this? Thanks.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191599891
  
**[Test build #52378 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52378/consoleFull)**
 for PR 11464 at commit 
[`2817c9a`](https://github.com/apache/spark/commit/2817c9a774f54c5f79828137bdb5a95c391c90c3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191599397
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191598834
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191598839
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52368/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191598375
  
**[Test build #52368 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52368/consoleFull)**
 for PR 11464 at commit 
[`2817c9a`](https://github.com/apache/spark/commit/2817c9a774f54c5f79828137bdb5a95c391c90c3).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191593337
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52360/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191593336
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191593158
  
**[Test build #52360 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52360/consoleFull)**
 for PR 11464 at commit 
[`4e212b1`](https://github.com/apache/spark/commit/4e212b16eb484563ca5b56dd9ae970b6a8337c21).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191578507
  
@rxin I think this is ready to be reviewed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191578259
  
**[Test build #52368 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52368/consoleFull)**
 for PR 11464 at commit 
[`2817c9a`](https://github.com/apache/spark/commit/2817c9a774f54c5f79828137bdb5a95c391c90c3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191571895
  
Hm.. This will not explicitly set `uncompressed` or `none` for JSON, CSV 
and TEXT. Let me correct them and add some tests.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191568621
  
**[Test build #52360 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52360/consoleFull)**
 for PR 11464 at commit 
[`4e212b1`](https://github.com/apache/spark/commit/4e212b16eb484563ca5b56dd9ae970b6a8337c21).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191565840
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52349/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191565838
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191565716
  
**[Test build #52349 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52349/consoleFull)**
 for PR 11464 at commit 
[`baf7a63`](https://github.com/apache/spark/commit/baf7a63337c49c4104240ad89a7cd257dca83eca).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191537772
  
**[Test build #52349 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52349/consoleFull)**
 for PR 11464 at commit 
[`baf7a63`](https://github.com/apache/spark/commit/baf7a63337c49c4104240ad89a7cd257dca83eca).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191503747
  
@rxin Sure.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191501409
  
There is a conflict. Maybe just make stuff consistent here?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191497497
  
@rxin Could you maybe merge this first before dealing with consistent 
compression name stuff if it looks good?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191204786
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52312/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191204783
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191203835
  
**[Test build #52312 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52312/consoleFull)**
 for PR 11464 at commit 
[`12e7275`](https://github.com/apache/spark/commit/12e7275047cf13d2e4a529a2e2c34a5867078758).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191168259
  
I will handle the consistent stuff in a new PR based on 
https://github.com/apache/spark/pull/11464.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191164322
  
**[Test build #52312 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52312/consoleFull)**
 for PR 11464 at commit 
[`12e7275`](https://github.com/apache/spark/commit/12e7275047cf13d2e4a529a2e2c34a5867078758).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191156213
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52307/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191156211
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191155207
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52306/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191155563
  
**[Test build #52307 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52307/consoleFull)**
 for PR 11464 at commit 
[`9c53d10`](https://github.com/apache/spark/commit/9c53d10f1a5ba136b6599d33c4b9d9070c720287).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191155204
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191154325
  
**[Test build #52306 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52306/consoleFull)**
 for PR 11464 at commit 
[`d107d50`](https://github.com/apache/spark/commit/d107d50bd2bc659d3f5441f8e1741c8356b6cc6e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191148542
  
@rxin I think I should change the codec names to lower cases 
[here](https://github.com/HyukjinKwon/spark/blob/SPARK-13543/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala#L367-L371).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191144806
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191144807
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/52305/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191144578
  
**[Test build #52305 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52305/consoleFull)**
 for PR 11464 at commit 
[`a96e510`](https://github.com/apache/spark/commit/a96e510295af1ed2bae25a05ee36d8f97b3e6b41).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191114850
  
How does this relate to https://github.com/apache/spark/pull/11408?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[rxin]

Github user rxin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11464#discussion_r54689173
  
--- Diff: python/pyspark/sql/readwriter.py ---
@@ -487,6 +487,12 @@ def parquet(self, path, mode=None, partitionBy=None):
 * ``error`` (default case): Throw an exception if data already 
exists.
 :param partitionBy: names of partitioning columns
 
+You can set the following Parquet-specific option(s) for writing 
Parquet files:
+* ``compression`` (default ``None``): compression codec to use 
when saving to file.
--- End diff --

maybe just put this as an argument?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191113649
  
**[Test build #52307 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52307/consoleFull)**
 for PR 11464 at commit 
[`9c53d10`](https://github.com/apache/spark/commit/9c53d10f1a5ba136b6599d33c4b9d9070c720287).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191112753
  
**[Test build #52306 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52306/consoleFull)**
 for PR 11464 at commit 
[`d107d50`](https://github.com/apache/spark/commit/d107d50bd2bc659d3f5441f8e1741c8356b6cc6e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191112794
  
cc @rxin


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

Github user HyukjinKwon commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191112297
  
retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11464#issuecomment-191110693
  
**[Test build #52305 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/52305/consoleFull)**
 for PR 11464 at commit 
[`a96e510`](https://github.com/apache/spark/commit/a96e510295af1ed2bae25a05ee36d8f97b3e6b41).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13543][SQL] Support for specifying comp...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

https://github.com/apache/spark/pull/11464

[SPARK-13543][SQL] Support for specifying compression codec for Parquet/ORC 
via option()

## What changes were proposed in this pull request?

This PR adds the support to specify compression codecs for both ORC and 
Parquet.

## How was this patch tested?

unittests within IDE and code style tests with `dev/run_tests`.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/HyukjinKwon/spark SPARK-13543

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/11464.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #11464


commit 9a43e721b0dd57c3b2919aac6d6eead78c38f769
Author: hyukjinkwon 
Date:   2016-03-02T00:10:51Z

Add the support to specify compression codec for Parquet and Orc

commit 9ce0ab707a9e54769b2d55feacd366893a0daf6e
Author: hyukjinkwon 
Date:   2016-03-02T05:26:58Z

Add some tests for testing compressions

commit 59a77d4eac7bd91fbe997d476808a94df66c2ec3
Author: hyukjinkwon 
Date:   2016-03-02T05:27:26Z

Merge upstream

commit 04e4a5167c49b6359f0cfac29d7f2f0c84bd497d
Author: hyukjinkwon 
Date:   2016-03-02T07:27:43Z

Add tests and some comments




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org