[GitHub] incubator-beam pull request #1544: Handle empty batches in GcsIO batch metho...

2016-12-09 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/1544 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1548: Fix template_runner_test on Windows

2016-12-07 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1548 Fix template_runner_test on Windows You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam fix-template

[GitHub] incubator-beam pull request #1544: Handle empty batches in GcsIO batch metho...

2016-12-07 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1544 Handle empty batches in GcsIO batch methods You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam

[GitHub] incubator-beam pull request #1496: Add labels to lambdas in write finalizati...

2016-12-06 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/1496 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1380: Fix issue where batch GCS renames were no...

2016-12-06 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/1380 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1379: Improve GcsIO throughput by 10x

2016-12-06 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/1379 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1496: Add labels to lambdas in write finalizati...

2016-12-02 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1496 Add labels to lambdas in write finalization You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam

[GitHub] incubator-beam pull request #1379: Speed up GcsIO throughput by 10x

2016-11-17 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1379 Speed up GcsIO throughput by 10x This change increases the read buffer used from 1M to 16M. Previously, the speed of reading an incompressible file were: (50 MB: 3.17 MB/s

[GitHub] incubator-beam pull request #1337: Use batch GCS operations during FileSink ...

2016-11-10 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1337 Use batch GCS operations during FileSink write finalization You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen

[GitHub] incubator-beam pull request #1129: Pin oauth2client to <4.0.0dev

2016-10-19 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/1129 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1129: Pin oauth2client to <4.0.0dev.

2016-10-18 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/1129 Pin oauth2client to <4.0.0dev. You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam pin-oau

[GitHub] incubator-beam pull request #991: Allow .whl files to be staged with --extra...

2016-09-27 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/991 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #991: Allow .whl files to be staged with --extra...

2016-09-22 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/991 Allow .whl files to be staged with --extra_package You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam

[GitHub] incubator-beam pull request #865: Support mode attribute on GCS files

2016-09-09 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/865 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #939: Use cStringIO instead of StringIO

2016-09-09 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/939 Use cStringIO instead of StringIO You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam fix-stringio

[GitHub] incubator-beam pull request #860: Allow Google Cloud Dataflow workflows to u...

2016-08-23 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/860 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #865: Support mode attribute on GCS files

2016-08-22 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/865 Support mode attribute on GCS files You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam gcsio-mode

[GitHub] incubator-beam pull request #846: Move native TextFileWriter to use GcsIO fo...

2016-08-19 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/846 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #846: Move native TextFileWriter to use GcsIO fo...

2016-08-18 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/846 Move native TextFileWriter to use GcsIO for writing You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator

[GitHub] incubator-beam pull request #830: Improve error handling in gcsio.py

2016-08-16 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/830 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #726: Make TextFileReader observable

2016-07-28 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/726 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #730: Fix SDK name and version sent to the Cloud...

2016-07-25 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/730 Fix SDK name and version sent to the Cloud Dataflow service You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen

[GitHub] incubator-beam pull request #678: [BEAM-460] Add size-estimation support to ...

2016-07-20 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/678 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #694: Remove "beamhead-02" container workaround

2016-07-20 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/694 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #686: Check type of coder for step feeding into ...

2016-07-20 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/686 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #694: Remove "beamhead-02" container workaround

2016-07-19 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/694 Remove "beamhead-02" container workaround You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-be

[GitHub] incubator-beam pull request #686: Check type of coder for step feeding into ...

2016-07-18 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/686 Check type of coder for step feeding into GroupByKey in Dataflow runner This prevents submission of invalid pipelines, such as one where a GroupByKey operation happens directly after

[GitHub] incubator-beam pull request #678: [BEAM-460] Add size-estimation support to ...

2016-07-18 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/678 [BEAM-460] Add size-estimation support to Python SDK Coders This change facilitates runner support for size-estimation counters. You can merge this pull request into a Git repository

[GitHub] incubator-beam pull request #669: Make step encodings consistently use Windo...

2016-07-15 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/669 Make step encodings consistently use WindowedValueCoders This change is needed for size-estimation aggregators. You can merge this pull request into a Git repository by running

[GitHub] incubator-beam pull request #644: Fix typo in Dataflow runner monitoring mes...

2016-07-15 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/644 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #653: Fix min and max timestamp on 32-bit machin...

2016-07-14 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/653 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #653: Fix min and max timestamp on 32-bit machin...

2016-07-14 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/653 Fix min and max timestamp on 32-bit machines You can merge this pull request into a Git repository by running: $ git pull https://github.com/charlesccychen/incubator-beam fix

[GitHub] incubator-beam pull request #644: Fix typo in Dataflow runner monitoring mes...

2016-07-12 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/644 Fix typo in Dataflow runner monitoring message Fixes spelling of "access" in monitoring message. You can merge this pull request into a Git repository by running:

[GitHub] incubator-beam pull request #622: Update Python aggregator example to match ...

2016-07-11 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/622 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #621: Allow ".tar" files in extra_packages

2016-07-11 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/621 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #622: Update Python aggregator example to match ...

2016-07-10 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/622 Update Python aggregator example to match Java usage Aggregations are performed in a per-DoFn/step scope. This change updates the examples to match the semantics, which are shared

[GitHub] incubator-beam pull request #621: Allow ".tar" files in extra_packages

2016-07-10 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/621 Allow ".tar" files in extra_packages This change allows uncompressed ".tar" package tarballs to be staged and installed on worker startup. You can mer

[GitHub] incubator-beam pull request #491: [BEAM-356] Update juliaset example to supp...

2016-06-24 Thread charlesccychen
Github user charlesccychen closed the pull request at: https://github.com/apache/incubator-beam/pull/491 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #491: [BEAM-356] Update juliaset example to supp...

2016-06-17 Thread charlesccychen
GitHub user charlesccychen opened a pull request: https://github.com/apache/incubator-beam/pull/491 [BEAM-356] Update juliaset example to support custom commands on "pip install" This change updates the juliaset example to support custom commands run when the workfl