[ https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547423#comment-15547423 ]
ASF GitHub Bot commented on BEAM-570: ------------------------------------- GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/1053 [BEAM-570] Title of the pull request - Getting rid of CompressionTypes.ZLIB and CompressionTypes.NO_COMPRESSION. - Introducing BZIP2 compression in analogy to Dataflow Java's BZIP2, towards resolution of https://issues.apache.org/jira/browse/BEAM-570. - Introducing SNAPPY codec support for AVRO conciseness and in order to fully resolve https://issues.apache.org/jira/browse/BEAM-570. - Moving avroio from compression_type to codec as per various discussions. - A few cleanups in avroio. - Making textio more DRY and doing a few cleanups. - Raising exceptions when splitting is requested for compressed source since that should never happen (guaranteed by the service for the supported compression types). - Using cStringIO instead of StringIO in various places as decided in some other discussions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/katsiapis/incubator-beam bz2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1053.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1053 ---- commit bd44e76b80e4edf4f922b9a26f7b359c4ede2008 Author: Gus Katsiapis <katsia...@katsiapis-linux.mtv.corp.google.com> Date: 2016-10-05T02:41:07Z Several enhancements to Dataflow (part 2 of 2). ---- > Update AvroSource to support more compression types > --------------------------------------------------- > > Key: BEAM-570 > URL: https://issues.apache.org/jira/browse/BEAM-570 > Project: Beam > Issue Type: Improvement > Components: sdk-py > Reporter: Chamikara Jayalath > Assignee: Chamikara Jayalath > > Python AvroSource [1] currently only support 'deflate' compression. We should > update it to support other compression types supported by the Avro library > (e.g.: snappy, bzip2). > [1] > https://github.com/apache/incubator-beam/blob/python-sdk/sdks/python/apache_beam/io/avroio.py -- This message was sent by Atlassian JIRA (v6.3.4#6332)