[GitHub] incubator-beam pull request #940: [BEAM-] Making Dataflow Py...

2016-09-09 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/940 [BEAM-] Making Dataflow Python Materialized PCollection representation more efficient (3 of several). Be sure to do all of the following to help us incorporate your contribution

[GitHub] incubator-beam pull request #946: [BEAM-625] Making Dataflow Python Material...

2016-09-12 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/946 [BEAM-625] Making Dataflow Python Materialized PCollection representation more efficient (4 of several). - Refactoring code in avroio.py to allow for re-use. - Making sure that

[GitHub] incubator-beam pull request #940: [BEAM-625] Making Dataflow Python Material...

2016-09-12 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/940 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #946: [BEAM-625] Making Dataflow Python Material...

2016-09-13 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/946 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #976: [BEAM-625] Making Dataflow Python Material...

2016-09-19 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/976 [BEAM-625] Making Dataflow Python Materialized PCollection representation more efficient (5 of several). Changing ToStringCoder to BytesCoder since the former can't be used with so

[GitHub] incubator-beam pull request #976: [BEAM-625] Making Dataflow Python Material...

2016-09-20 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/976 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #989: [BEAM-625] Better documentation for Compre...

2016-09-22 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/989 [BEAM-625] Better documentation for CompressionTypes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/katsiapis/incubator-beam

[GitHub] incubator-beam pull request #989: [BEAM-989] Better documentation for Compre...

2016-09-22 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/989 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1053: [BEAM-570] Title of the pull request

2016-10-04 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/1053 [BEAM-570] Title of the pull request - Getting rid of CompressionTypes.ZLIB and CompressionTypes.NO_COMPRESSION. - Introducing BZIP2 compression in analogy to Dataflow Java's

[GitHub] incubator-beam pull request #1053: [BEAM-570] Several enhancements and depre...

2016-10-17 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/1053 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1391: [BEAM-1010] A few improvements to Apache ...

2016-11-18 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/1391 [BEAM-1010] A few improvements to Apache Beam Python's FileIO. - Ensuring that AUTO compression works properly for FileSinks. - Introducing __enter__ and __exit__ in _Compresse

[GitHub] incubator-beam pull request #1391: [BEAM-1010] A few improvements to Apache ...

2016-11-18 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/1391 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1392: [BEAM-101] A few improvements to Apache B...

2016-11-18 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/1392 [BEAM-101] A few improvements to Apache Beam Python's FileIO. - Ensuring that AUTO compression works properly for FileSinks. - Introducing __enter__ and __exit__ in _CompressedFi

[GitHub] incubator-beam pull request #1392: [BEAM-1010] A few improvements to Apache ...

2016-11-21 Thread katsiapis
Github user katsiapis closed the pull request at: https://github.com/apache/incubator-beam/pull/1392 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1672: [Beam-625] Peak memory reduction in Avro ...

2016-12-20 Thread katsiapis
GitHub user katsiapis opened a pull request: https://github.com/apache/incubator-beam/pull/1672 [Beam-625] Peak memory reduction in Avro and Pickler. Decreasing the number of copies of things in scope for reduced peak memory utilization in Avro and Pickler. You can merge this pull

[jira] [Created] (BEAM-625) Make Dataflow Python Materialized PCollection representation more efficient

2016-09-09 Thread Konstantinos Katsiapis (JIRA)
Konstantinos Katsiapis created BEAM-625: --- Summary: Make Dataflow Python Materialized PCollection representation more efficient Key: BEAM-625 URL: https://issues.apache.org/jira/browse/BEAM-625

[jira] [Commented] (BEAM-570) Update AvroSource to support more compression types

2016-09-12 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15484712#comment-15484712 ] Konstantinos Katsiapis commented on BEAM-570: - According to the

[jira] [Created] (BEAM-629) Cleanup SDK regardign lint warnings (and lint warning suppression).

2016-09-13 Thread Konstantinos Katsiapis (JIRA)
Konstantinos Katsiapis created BEAM-629: --- Summary: Cleanup SDK regardign lint warnings (and lint warning suppression). Key: BEAM-629 URL: https://issues.apache.org/jira/browse/BEAM-629 Project

[jira] [Resolved] (BEAM-625) Make Dataflow Python Materialized PCollection representation more efficient

2016-09-29 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Katsiapis resolved BEAM-625. - This is fixed as of Google Cloud Dataflow 0.4.2 https://cloud.google.com/dataflow

[jira] [Commented] (BEAM-570) Update AvroSource to support more compression types

2016-10-13 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573243#comment-15573243 ] Konstantinos Katsiapis commented on BEAM-570: - Thanks Frances, PR h

[jira] [Commented] (BEAM-570) Update AvroSource to support more compression types

2016-10-17 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582808#comment-15582808 ] Konstantinos Katsiapis commented on BEAM-570: - [~chamikara] I am unabl

[jira] [Commented] (BEAM-570) Update AvroSource to support more compression types

2016-10-17 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582770#comment-15582770 ] Konstantinos Katsiapis commented on BEAM-570: - Support is now in for both

[jira] [Created] (BEAM-1010) Custom FileSinks should respect AUTO compression

2016-11-18 Thread Konstantinos Katsiapis (JIRA)
Konstantinos Katsiapis created BEAM-1010: Summary: Custom FileSinks should respect AUTO compression Key: BEAM-1010 URL: https://issues.apache.org/jira/browse/BEAM-1010 Project: Beam

[jira] [Updated] (BEAM-1010) Custom FileSinks should respect AUTO compression

2016-11-18 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Katsiapis updated BEAM-1010: - Description: Currently AUTO compression is respected by Native FileSinks but not

[jira] [Created] (BEAM-1016) Use Content-Type and Content-Encoding (as opposed to overriding Content-Type) for compressed files

2016-11-19 Thread Konstantinos Katsiapis (JIRA)
Konstantinos Katsiapis created BEAM-1016: Summary: Use Content-Type and Content-Encoding (as opposed to overriding Content-Type) for compressed files Key: BEAM-1016 URL: https://issues.apache.org/jira

[jira] [Updated] (BEAM-1016) Use Content-Type and Content-Encoding (as opposed to overriding Content-Type) for compressed files

2016-11-19 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Katsiapis updated BEAM-1016: - Priority: Minor (was: Major) > Use Content-Type and Content-Encoding (as oppo

[jira] [Resolved] (BEAM-1010) Custom FileSinks should respect AUTO compression

2016-11-21 Thread Konstantinos Katsiapis (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantinos Katsiapis resolved BEAM-1010. -- Resolution: Fixed Fix Version/s: 0.4.0-incubating This was fixed by the