dongjoon-hyun commented on a change in pull request #31133:
URL: https://github.com/apache/spark/pull/31133#discussion_r568424773
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##########
@@ -388,6 +394,9 @@ private[hive] object HiveTableUtil {
private[hive] object DeserializerLock
private[hive] object HadoopTableReader extends HiveInspectors with Logging {
+
+ val avroTableProperties =
AvroTableProperties.values().map(_.getPropName()).toSet
Review comment:
SPARK-26836 is filed for 2.3.1 ~ 3.2.0 and this patch depends on
`org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.AvroTableProperties`. I'm
wondering if this patch is applicable with old Spark branches with old Hive
libraries. For example,
- Apache Spark master : 2.3.8
- Apache Spark branch-3.1: 2.3.7
- Apache Spark branch-3.0: 2.3.7 and 1.2.1.spark2
- Apache Spark branch-2.4: 1.2.1.spark2
What is the minimum version of Hive library to use this? Is there a known
bug for this `AvroTableProperties` classes?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
