[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204763098 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala --- @@ -68,4 +70,25 @@ class AvroOptions(

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204762586 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala --- @@ -68,4 +70,25 @@ class AvroOptions(

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-24 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204757826 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala --- @@ -68,4 +70,25 @@ class AvroOptions( .map(_.toBoolean)

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204609385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204609328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204608704 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204608456 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204608411 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204601582 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala --- @@ -68,4 +70,25 @@ class AvroOptions(

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204601282 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -896,4 +896,33 @@ class AvroSuite extends QueryTest with

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204300978 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -133,8 +134,7 @@ private[avro] class AvroFileFormat

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204252552 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -133,8 +134,7 @@ private[avro] class AvroFileFormat extends

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204251485 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -133,8 +134,7 @@ private[avro] class AvroFileFormat

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204251513 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -889,4 +889,30 @@ class AvroSuite extends QueryTest with

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204243507 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -889,4 +889,30 @@ class AvroSuite extends QueryTest with

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-22 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/21837 [SPARK-24881][SQL] New Avro options - compression and compressionLevel ## What changes were proposed in this pull request? In the PR, I added new options for Avro datasource -