[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-606288517 >And then I used Hudi 0.5.1 (uses spark-avro) to update the values of these two types and it works. Thanks for confirming! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-606181565 @zhedoubushishi any thoughts? I will timeout and review land this if this sounds good.. we can add a blocker for 0.6.0 to get the backwards compatibility resolved This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-604128720 Sorry did not mean to hijack this fix.. Just trying to understand how it ll break compatibility while we are here.. All this schema namespace business is only before writing parquet files right... Once you are able to write parquet, it should be readable by parquet-avro for merging? (which has nothing to do with apache-spark-avro or databricks-spark-avro)... what causes the breakage? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-603506326 yes I am also wondering if we can do something about incompatibility.. that a large issue.. we don't want our users to go through this.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-602881385 >>So anyone who has written data using databricks-avro will face issues reading. By this you mean, reading for merging data (i.e during ingestion/writing) or querying via Spark/Hive/Presto? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema
vinothchandar commented on issue #1406: [HUDI-713] Fix conversion of Spark array of struct type to Avro schema URL: https://github.com/apache/incubator-hudi/pull/1406#issuecomment-599591998 @umehrot2 are you interested in reviewing this? :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
