Re: [PR] [SPARK-46971][SQL] When the `compression` is null, a `NullPointException` should not be thrown [spark]

2024-05-19 Thread via GitHub


github-actions[bot] closed pull request #45015: [SPARK-46971][SQL] When the 
`compression` is null, a `NullPointException` should not be thrown
URL: https://github.com/apache/spark/pull/45015


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



Re: [PR] [SPARK-46971][SQL] When the `compression` is null, a `NullPointException` should not be thrown [spark]

2024-05-18 Thread via GitHub


github-actions[bot] commented on PR #45015:
URL: https://github.com/apache/spark/pull/45015#issuecomment-2119040244

   We're closing this PR because it hasn't been updated in a while. This isn't 
a judgement on the merit of the PR in any way. It's just a way of keeping the 
PR queue manageable.
   If you'd like to revive this PR, please reopen it and ask a committer to 
remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



Re: [PR] [SPARK-46971][SQL] When the `compression` is null, a `NullPointException` should not be thrown [spark]

2024-02-07 Thread via GitHub


LuciferYang commented on code in PR #45015:
URL: https://github.com/apache/spark/pull/45015#discussion_r1482479360


##
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala:
##
@@ -118,7 +119,11 @@ private[sql] class AvroOptions(
* taken into account. If the former one is not set too, the `snappy` codec 
is used by default.
*/
   val compression: String = {
-parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+val v = 
parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+if (v == null) {

Review Comment:
   Would there be compatibility issues if null values are prohibited in 
`CaseInsensitiveMap`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



Re: [PR] [SPARK-46971][SQL] When the `compression` is null, a `NullPointException` should not be thrown [spark]

2024-02-05 Thread via GitHub


dongjoon-hyun commented on code in PR #45015:
URL: https://github.com/apache/spark/pull/45015#discussion_r1478853391


##
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala:
##
@@ -118,7 +119,11 @@ private[sql] class AvroOptions(
* taken into account. If the former one is not set too, the `snappy` codec 
is used by default.
*/
   val compression: String = {
-parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+val v = 
parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+if (v == null) {

Review Comment:
   +1 for @HyukjinKwon 's comment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



Re: [PR] [SPARK-46971][SQL] When the `compression` is null, a `NullPointException` should not be thrown [spark]

2024-02-04 Thread via GitHub


HyukjinKwon commented on code in PR #45015:
URL: https://github.com/apache/spark/pull/45015#discussion_r1477698620


##
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroOptions.scala:
##
@@ -118,7 +119,11 @@ private[sql] class AvroOptions(
* taken into account. If the former one is not set too, the `snappy` codec 
is used by default.
*/
   val compression: String = {
-parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+val v = 
parameters.get(COMPRESSION).getOrElse(SQLConf.get.avroCompressionCodec)
+if (v == null) {

Review Comment:
   This problem is actually here and there, not only `AvroOptions`. I believe a 
lot of `CSVOptions` also has this problem.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org