vinothchandar commented on issue #770: remove com.databricks:spark-avro to build spark avro schema by itself URL: https://github.com/apache/incubator-hudi/pull/770#issuecomment-532739103 I have a slightly different strategy. We can move to [spark 2.4](https://github.com/apache/spark/blob/branch-2.4/pom.xml) and match its parquet (1.10.1), avro (1.8.2) versions. Match parquet-avro to parquet version. As long as Spark 2.3 can work with parquet-avro 1.10.1, (we bundle this), it should be fine? Also @umehrot2 , is supporting 2.3 a must or can we drop Hudi support for lower than 2.4 versions? Hudi community is ok per se to just support 2.4. if so, then we can also drop com.databricks from the code and use org.apache.spark.avro (which is only in version 2.4) cc @bvaradar @bhasudha who are looking into the spark 2.4 move
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
