[
https://issues.apache.org/jira/browse/AVRO-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848251#comment-17848251
]
Oscar Westra van Holthe - Kind commented on AVRO-3980:
------------------------------------------------------
Hi [~jarilouvem],
Your concerns about the existing data are valid, I think. To move forward, it
is now important to understand how the data was written. Are you certain of the
schema(s) used?
Let's start with an example of data that decodes correctly (if there is any),
and an example where this fails.
Then, with the write schema from the line {{{}Schema schema =
obj.getSchema();{}}}, we'll need to manually attempt to decode the data.
> Error to deserialize field of type Long after upgrade from 1.11.1 to 1.11.3
> ---------------------------------------------------------------------------
>
> Key: AVRO-3980
> URL: https://issues.apache.org/jira/browse/AVRO-3980
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
> Affects Versions: 1.11.3
> Reporter: Jari Louvem
> Priority: Critical
>
> After we upgraded Avro library and avro-maven-plugin from version 1.11.1 to
> 1.11.3 and
> we started to get the error "cannot read collections larger than 2147483639
> items in java library".
>
> This error is generated by SystemLimitException.checkMaxCollectionLength.
> The data that we are trying to deserialize (using avro 1.11.3) was serealized
> using avro 1.11.1.
> The object that we are trying to deserealize is:
> {
> "name": "statuses",
> "type": {
> "type": "array",
> "items": "com.entity.avro.StatusAvro"
> }
> }
> {
> "name": "statuses",
> "type": {
> "type": "array",
> "items": {
> "name": "StatusAvro",
> "type": "record",
> "namespace": "com.entity.avro",
> "fields": [
> {
> "name": "status",
> "type": [
> "null",
> "string"
> ]
> },
> {
> "name": "reason",
> "type": [
> "null",
> "string"
> ]
> },
> {
> "name": "validFor",
> "type": "com.entity.avro.ValidForAvro"
> }
> ]
> }
> }
> }
> {
> "name": "validFor",
> "type": {
> "name": "ValidForAvro",
> "type": "record",
> "namespace": "com.entity.avro",
> "fields": [
> {
> "name": "start",
> "type": "long"
> },
> {
> "name": "end",
> "type": "long"
> }
> ]
> }
> }
> This is an example of the objects listed above:
> "statuses": [
> {
> "status": "INIT",
> "reason": "Final_New_Reason",
> "validFor": {
> "start": "2020-01-30T11:45:00.839Z",
> "end": "2030-01-23T06:58:21.563Z"
> }
> }
> ]
> The problem is that the array has only one item as shown above, so why is it
> throwing an error of the collection is too long?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
