[
https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484736#comment-15484736
]
Chamikara Jayalath commented on BEAM-570:
-----------------------------------------
Java seems to be supporting bzip2 [1] but seems like Python won't be able to
add that since the Avro implementation doesn't support that [2]. So we should
just add support for snappy and close this JIRA issue.
[1]
https://github.com/apache/incubator-beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/AvroSource.java
[2] https://github.com/cavorite/python-avro/blob/master/src/avro/datafile.py#L46
> Update AvroSource to support more compression types
> ---------------------------------------------------
>
> Key: BEAM-570
> URL: https://issues.apache.org/jira/browse/BEAM-570
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py
> Reporter: Chamikara Jayalath
> Assignee: Chamikara Jayalath
>
> Python AvroSource [1] currently only support 'deflate' compression. We should
> update it to support other compression types supported by the Avro library
> (e.g.: snappy, bzip2).
> [1]
> https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/avroio.py
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
