akshat0395 commented on PR #4012: URL: https://github.com/apache/hive/pull/4012#issuecomment-1439897739
Thanks @ayushtkn, @zabetak for the input. I've done some further testing and analysis for the same. - I've ran a Java API compatibility test between the current avro version i.e 1.8.2 used by Hive and the target version 1.11.1. The compatibility is ~ 95%, There are few new APIs added which are safe to use and doesnt have impact on hive and there are few APIs that are deprecated, The deprecated API are already replaced as part of [HIVE-24324 ](https://issues.apache.org/jira/browse/HIVE-24324), PR: [1621](https://github.com/apache/hive/pull/1621) as @ayushtkn also mentioned. In conclusion compatibility wise the upgrade seems safe - Manual Tests: - I've triggered build and ran test with the upgraded version in an cluster and it passed. This PR also passed the test - I've create an avro table with different datatypes with a build that have avro 1.8.2 `CREATE TABLE avro_table STORED AS AVRO TBLPROPERTIES ('avro.schema.literal'='{ "type": "record", "name": "my_record", "fields": [ {"name": "tinyint_col", "type": "int"}, {"name": "smallint_col", "type": "int"}, {"name": "int_col", "type": "int"}, {"name": "bigint_col", "type": "long"}, {"name": "float_col", "type": "float"}, {"name": "double_col", "type": "double"}, {"name": "decimal_col", "type": {"type": "bytes", "logicalType": "decimal", "precision": 10, "scale": 2}}, {"name": "string_col", "type": "string"}, {"name": "varchar_col", "type": "string"}, {"name": "char_col", "type": "string"}, {"name": "binary_col", "type": "bytes"}, {"name": "boolean_col", "type": "boolean"}, {"name": "array_col", "type": {"type": "array", "items": "string"}}, {"name": "map_col", "type": {"type": "map", "values": "int"}}, {"name": "struct_col", "type": {"type": "record", "name": "my_struct", "fields": [{"name": "field1", "type": "string"}, {"name": "field2", "type": "int"}]}} ] }');` I've performed some basic Insert, Modify, and aggregation query on the current version, post that I've performed the same set of queries with the upgraded version and both seems to work without any issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org