[
https://issues.apache.org/jira/browse/AVRO-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742742#action_12742742
]
Ravi Gummadi commented on AVRO-61:
----------------------------------
The testcase in this patch passes only with the patch of AVRO-88. So made
AVRO-88 as a blocker for this JIRA.
> Support for blocking arrays/maps in python
> ------------------------------------------
>
> Key: AVRO-61
> URL: https://issues.apache.org/jira/browse/AVRO-61
> Project: Avro
> Issue Type: Bug
> Components: python
> Reporter: Thiruvalluvan M. G.
> Assignee: Ravi Gummadi
> Attachments: AVRO-61.patch
>
>
> We have introduced a blocking version of writer in Java that enables readers
> to efficiently skip large arrays and maps. The avro format encoded arrays and
> maps by encoding the number of elements by the the elements themselves. Zero
> element count indicates that the array/map has ended. The change we
> introduced is that if the element count is negative, it is followed by the
> byte-count of the encoded elements that follow. The reader, on seeing a
> negative element count should flip the sign to get the actual number of
> elements. In addition, if it is interested in supporting fast skip, it should
> use the byte count to skip the elements en-bloc instead of decoding them
> individually. If it does not want to support fast skip, it has to just read
> the byte count and ignore its value.
> The changes are already made in Java's ValueReader to support this. Similar
> changes need to done in Python as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.