[ https://issues.apache.org/jira/browse/AVRO-3980 ]
Jari Louvem deleted comment on AVRO-3980:
-----------------------------------
was (Author: JIRAUSER305220):
* How was the data written?
There is the schema definition in .avsc that are converted to java pojos using
avro-maven-plugin. Then, this pojo is serialized this way:
public static <T> ByteBuffer serialize(Object obj, Class<T>
targetType)
{
ByteArrayOutputStream out = new ByteArrayOutputStream();
BinaryEncoder encoder = EncoderFactory.get().binaryEncoder(out,
null);
DatumWriter<T> writer = new
SpecificDatumWriter<>(((GenericContainer)
targetType.getDeclaredConstructor().newInstance()).getSchema());
writer.write((T) obj, encoder);
encoder.flush();
out.close();
byte[] serializedBytes = out.toByteArray();
return ByteBuffer.wrap(serializedBytes);
} * The data is serialized and saved in a database inside a call to a
rest API call.
* How was the data read?
public static <T> T deserialize(ByteBuffer byteBuffer, Class<T>
targetType)
{
byte[] arr = getByteArrayFromByteBuffer(byteBuffer);
SpecificDatumReader<T> reader = new
SpecificDatumReader<T>(((GenericContainer)
targetType.getDeclaredConstructor().newInstance()).getSchema());
ByteArrayInputStream bis = new ByteArrayInputStream(arr);
Decoder decoder = DecoderFactory.get().binaryDecoder(bis,
null);
return
reader.read(targetType.getDeclaredConstructor().newInstance(), decoder);
}
> Error to deserialize field of type Long after upgrade from 1.11.1 to 1.11.3
> ---------------------------------------------------------------------------
>
> Key: AVRO-3980
> URL: https://issues.apache.org/jira/browse/AVRO-3980
> Project: Apache Avro
> Issue Type: Bug
> Components: java, logical types
> Affects Versions: 1.11.3
> Reporter: Jari Louvem
> Priority: Critical
>
> After we upgraded Avro library and avro-maven-plugin from version 1.11.1 to
> 1.11.3 and
> we started to get the error "cannot read collections larger than 2147483639
> items in java library".
>
> This error is generated by SystemLimitException.checkMaxCollectionLength.
> The data that we are trying to deserialize (using avro 1.11.3) was serealized
> using avro 1.11.1.
> The object that we are trying to deserealize is:
> {
> "name": "statuses",
> "type": {
> "type": "array",
> "items": "com.entity.avro.StatusAvro"
> }
> }
> {
> "name": "statuses",
> "type": {
> "type": "array",
> "items": {
> "name": "StatusAvro",
> "type": "record",
> "namespace": "com.entity.avro",
> "fields": [
> {
> "name": "status",
> "type": [
> "null",
> "string"
> ]
> },
> {
> "name": "reason",
> "type": [
> "null",
> "string"
> ]
> },
> {
> "name": "validFor",
> "type": "com.entity.avro.ValidForAvro"
> }
> ]
> }
> }
> }
> {
> "name": "validFor",
> "type": {
> "name": "ValidForAvro",
> "type": "record",
> "namespace": "com.entity.avro",
> "fields": [
> {
> "name": "start",
> "type": "long"
> },
> {
> "name": "end",
> "type": "long"
> }
> ]
> }
> }
> This is an example of the objects listed above:
> "statuses": [
> {
> "status": "INIT",
> "reason": "Final_New_Reason",
> "validFor": {
> "start": "2020-01-30T11:45:00.839Z",
> "end": "2030-01-23T06:58:21.563Z"
> }
> }
> ]
> The problem is that the array has only one item as shown above, so why is it
> throwing an error of the collection is too long?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
