[
https://issues.apache.org/jira/browse/HIVE-26954?focusedWorklogId=846913&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-846913
]
ASF GitHub Bot logged work on HIVE-26954:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Feb/23 11:55
Start Date: 22/Feb/23 11:55
Worklog Time Spent: 10m
Work Description: akshat0395 commented on PR #4012:
URL: https://github.com/apache/hive/pull/4012#issuecomment-1439897739
Thanks @ayushtkn, @zabetak for the input. I've done some further testing and
analysis for the same.
- I've ran a Java API compatibility test between the current avro version
i.e 1.8.2 used by Hive and the target version 1.11.1. The compatibility is ~
95%, There are few new APIs added which are safe to use and doesnt have impact
on hive and there are few APIs that are deprecated, The deprecated API are
already replaced as part of [HIVE-24324
](https://issues.apache.org/jira/browse/HIVE-24324), PR:
[1621](https://github.com/apache/hive/pull/1621) as @ayushtkn also mentioned.
In conclusion compatibility wise the upgrade seems safe
- Manual Tests:
- I've triggered build and ran test with the upgraded version in an cluster
and it passed. This PR also passed the test
- I've create an avro table with different datatypes with a build that have
avro 1.8.2
`CREATE TABLE avro_table
STORED AS AVRO
TBLPROPERTIES ('avro.schema.literal'='{
"type": "record",
"name": "my_record",
"fields": [
{"name": "tinyint_col", "type": "int"},
{"name": "smallint_col", "type": "int"},
{"name": "int_col", "type": "int"},
{"name": "bigint_col", "type": "long"},
{"name": "float_col", "type": "float"},
{"name": "double_col", "type": "double"},
{"name": "decimal_col", "type": {"type": "bytes", "logicalType":
"decimal", "precision": 10, "scale": 2}},
{"name": "string_col", "type": "string"},
{"name": "varchar_col", "type": "string"},
{"name": "char_col", "type": "string"},
{"name": "binary_col", "type": "bytes"},
{"name": "boolean_col", "type": "boolean"},
{"name": "array_col", "type": {"type": "array", "items": "string"}},
{"name": "map_col", "type": {"type": "map", "values": "int"}},
{"name": "struct_col", "type": {"type": "record", "name": "my_struct",
"fields": [{"name": "field1", "type": "string"}, {"name": "field2", "type":
"int"}]}}
]
}');`
I've performed some basic Insert, Modify, and aggregation query on the
current version, post that I've performed the same set of queries with the
upgraded version and both seems to work without any issues.
Issue Time Tracking
-------------------
Worklog Id: (was: 846913)
Time Spent: 1.5h (was: 1h 20m)
> Upgrade Avro to 1.11.1
> ----------------------
>
> Key: HIVE-26954
> URL: https://issues.apache.org/jira/browse/HIVE-26954
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 4.0.0
> Reporter: Akshat Mathur
> Assignee: Akshat Mathur
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Upgrade Avro dependencies to 1.11.1
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
