[GitHub] incubator-beam pull request #436: Raise error rather than overwrite data whe...

2016-06-08 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/436 Raise error rather than overwrite data when sharding is not respected This is particularly relevant when withoutSharding() is used due to [BEAM-159]. You can merge this pull request

incubator-beam-site git commit: Remove duplicate paragraph.

2016-05-27 Thread robertwb
Repository: incubator-beam-site Updated Branches: refs/heads/asf-site 87d9a07fe -> edd0705b8 Remove duplicate paragraph. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/edd0705b Tree:

incubator-beam-site git commit: Fix blog post typo.

2016-05-27 Thread robertwb
Repository: incubator-beam-site Updated Branches: refs/heads/asf-site edd0705b8 -> 0c5c6471b Fix blog post typo. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/0c5c6471 Tree:

[GitHub] incubator-beam pull request: Optimize the Count CombineFn

2016-03-29 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/88 Optimize the Count CombineFn Previously the accumulator was stored as a Long. This uses a singleton long[] to avoid the boxing and unboxing on every increment

[2/3] incubator-beam git commit: Update docstring.

2016-07-25 Thread robertwb
Update docstring. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/76f3864b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/76f3864b Diff:

[3/3] incubator-beam git commit: Closes #730

2016-07-25 Thread robertwb
Closes #730 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/26ff6579 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/26ff6579 Diff:

[1/3] incubator-beam git commit: Fix SDK name and version sent to the Cloud Dataflow service

2016-07-25 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 153916fe9 -> 26ff65795 Fix SDK name and version sent to the Cloud Dataflow service Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[1/2] incubator-beam git commit: Made checksum_output optional in bigshuffle.py.

2016-07-12 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 95a591e05 -> 67a769a9a Made checksum_output optional in bigshuffle.py. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2b782ded Tree:

[2/2] incubator-beam git commit: Closes #625

2016-07-12 Thread robertwb
Closes #625 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/67a769a9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/67a769a9 Diff:

[2/2] incubator-beam git commit: Closes #641

2016-07-14 Thread robertwb
Closes #641 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e8c39c79 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e8c39c79 Diff:

[1/2] incubator-beam git commit: Add type hints to bigshuffle to avoid pickle overhead.

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 67a769a9a -> e8c39c798 Add type hints to bigshuffle to avoid pickle overhead. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f858ea93

[2/2] incubator-beam git commit: Fix typo in Dataflow runner monitoring message

2016-07-14 Thread robertwb
Fix typo in Dataflow runner monitoring message Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ffbaccaa Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ffbaccaa Diff:

[2/2] incubator-beam git commit: Closes #652

2016-07-14 Thread robertwb
Closes #652 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2b9d81fc Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2b9d81fc Diff:

[1/2] incubator-beam git commit: Added cy_combiners.py to the list of Cythonized Python SDK files.

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 52c0d893e -> 2b9d81fcc Added cy_combiners.py to the list of Cythonized Python SDK files. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #641: Add type hints to bigshuffle to avoid pick...

2016-07-14 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/641 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: Closes #644

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk e8c39c798 -> 52c0d893e Closes #644 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/52c0d893 Tree:

[1/2] incubator-beam git commit: Temporarily reverting pickler changes (@4e2d8ab).

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 2b9d81fcc -> 7c0c27ac0 Temporarily reverting pickler changes (@4e2d8ab). It is causing infinite recursion when some tests are directly invoked outside the testing framework. Added one such test to tox tests for testing

[2/2] incubator-beam git commit: Closes #645

2016-07-14 Thread robertwb
Closes #645 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7c0c27ac Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7c0c27ac Diff:

[1/2] incubator-beam git commit: DoOutputsTuple cleanup

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 7c0c27ac0 -> 762a2930a DoOutputsTuple cleanup This avoids the same (logical) PCollection from being re-created with a different producer. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #658

2016-07-14 Thread robertwb
Closes #658 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/762a2930 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/762a2930 Diff:

[GitHub] incubator-beam pull request #658: DoOutputsTuple cleanup

2016-07-14 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/658 DoOutputsTuple cleanup Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #657: Accept runners by fully qualified name.

2016-07-14 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/657 Accept runners by fully qualified name. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[3/3] incubator-beam git commit: Closes #657

2016-07-14 Thread robertwb
Closes #657 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3b695068 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3b695068 Diff:

[1/3] incubator-beam git commit: Accept runners by fully qualified name.

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 762a2930a -> 3b6950689 Accept runners by fully qualified name. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/84fe8954 Tree:

[1/2] incubator-beam git commit: Closes #653

2016-07-14 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 3b6950689 -> c8cef2cba Closes #653 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c8cef2cb Tree:

[2/2] incubator-beam git commit: Fix min and max timestamp on 32-bit machines

2016-07-14 Thread robertwb
Fix min and max timestamp on 32-bit machines Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/afc68bc3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/afc68bc3 Diff:

[2/2] incubator-beam git commit: Closes #650

2016-07-14 Thread robertwb
Closes #650 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a1a51c3c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a1a51c3c Diff:

[1/2] incubator-beam git commit: Update some of the example tests to use assert_that

2016-07-14 Thread robertwb
;""Test for the autocomplete example.""" -import collections import unittest - import apache_beam as beam from apache_beam.examples.complete import autocomplete -from apache_beam.pvalue import AsIter - -# TODO(robertwb): Move to testing utilities. - - -def ass

[GitHub] incubator-beam pull request #706: Simple DoFnRunner optimizatons.

2016-07-21 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/706 Simple DoFnRunner optimizatons. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[3/4] incubator-beam git commit: Allow passing logging context directly.

2016-07-25 Thread robertwb
Allow passing logging context directly. This is better than passing a logger module with a specific class. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/de35f282 Tree:

[2/4] incubator-beam git commit: Receiver and LoggingContext adapters.

2016-07-25 Thread robertwb
thon + from apache_beam.utils.windowed_value cimport WindowedValue + cdef type SideOutputValue, TimestampedValue @@ -31,10 +34,10 @@ cdef class DoFnRunner(Receiver): cdef object window_fn cdef object context # TODO(robertwb): Make this a DoFnContext cdef object tagged_receivers -

[4/4] incubator-beam git commit: Closes #721

2016-07-25 Thread robertwb
Closes #721 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/153916fe Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/153916fe Diff:

[GitHub] incubator-beam pull request #721: Make DoFnRunner a Receiver.

2016-07-25 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/721 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #741: Top improvements

2016-07-27 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/741 Top improvements Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM

[2/2] incubator-beam git commit: Closes #726

2016-07-28 Thread robertwb
Closes #726 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53ab635c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53ab635c Diff:

[1/2] incubator-beam git commit: Make TextFileReader observable

2016-07-28 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 26ff65795 -> 53ab635c7 Make TextFileReader observable This allows future implementation of size tracking for elements in side input sources. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[3/3] incubator-beam git commit: Closes #741

2016-07-28 Thread robertwb
Closes #741 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b4716d9d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b4716d9d Diff:

[1/3] incubator-beam git commit: Better top implementation.

2016-07-28 Thread robertwb
ing). - compare_by_id = {} - - def __init__(self, n, compare, _compare_id=None): # pylint: disable=invalid-name + # TODO(robertwb): Allow taking a key rather than a compare. + def __init__(self, n, compare): self._n = n +self._buffer_size = min(2 * n, n + 1000) self._com

[1/2] incubator-beam git commit: Optimize Map and Flatmap when there are no side inputs.

2016-07-28 Thread robertwb
elf.windowing, True) - state = InMemoryUnmergedState() + self.driver = create_trigger_driver(self.windowing, True) + self.state_type = InMemoryUnmergedState + +def process(self, context): + k, vs = context.element + state = self.state_type() # TODO(robertwb): Condit

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

2016-07-28 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/748 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #741: Top improvements

2016-07-28 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[2/2] incubator-beam git commit: Closes #748

2016-07-28 Thread robertwb
Closes #748 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/351c3831 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/351c3831 Diff:

[04/12] incubator-beam git commit: Fix multi-input named PTransforms.

2016-07-23 Thread robertwb
Fix multi-input named PTransforms. Now delegate the __ror__ logic entirely for the naming wrapper. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/937cf69e Tree:

[11/12] incubator-beam git commit: Final cleanup pass.

2016-07-23 Thread robertwb
Final cleanup pass. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e3c078fe Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e3c078fe Diff:

[09/12] incubator-beam git commit: Fix label-sensitive test.

2016-07-23 Thread robertwb
Fix label-sensitive test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2a59a121 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2a59a121 Diff:

[08/12] incubator-beam git commit: Fixes examples

2016-07-23 Thread robertwb
Fixes examples Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2ff5630b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2ff5630b Diff:

[10/12] incubator-beam git commit: Lint fixes.

2016-07-23 Thread robertwb
Lint fixes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b15d35ca Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b15d35ca Diff:

[03/12] incubator-beam git commit: Move names out of transform constructors.

2016-07-23 Thread robertwb
Move names out of transform constructors. sed -i -r 's/[|] (\S+)[(](["'"'"'][^"'"'"']+.)(, +|([)]))/| \2 >> \1(\4/g' Small number of tests will need to be fixed by hand. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/4] incubator-beam git commit: Make save_main_session optional

2016-07-23 Thread robertwb
Make save_main_session optional Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2e36602c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2e36602c Diff:

[4/4] incubator-beam git commit: Closes #723

2016-07-23 Thread robertwb
Closes #723 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9fe102a5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9fe102a5 Diff:

[1/4] incubator-beam git commit: Refactor setup.py to separate strings/versions

2016-07-23 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 47042ce89 -> 9fe102a5c Refactor setup.py to separate strings/versions Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/af6d3804 Tree:

[3/4] incubator-beam git commit: Refactor examples to use save_main_session

2016-07-23 Thread robertwb
Refactor examples to use save_main_session Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3d64a8c5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3d64a8c5 Diff:

[06/12] incubator-beam git commit: fix pipeline test

2016-07-23 Thread robertwb
fix pipeline test Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/362f2e9e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/362f2e9e Diff:

[02/12] incubator-beam git commit: Move names out of transform constructors.

2016-07-23 Thread robertwb
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/031c4cce/sdks/python/apache_beam/transforms/combiners_test.py -- diff --git a/sdks/python/apache_beam/transforms/combiners_test.py

[05/12] incubator-beam git commit: fixup: failing tests expecting name

2016-07-23 Thread robertwb
fixup: failing tests expecting name Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c5b5b14d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c5b5b14d Diff:

[01/12] incubator-beam git commit: Fix error messages for externally named PTransforms.

2016-07-23 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 9fe102a5c -> 38d9dea2e Fix error messages for externally named PTransforms. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7c186ce4

[12/12] incubator-beam git commit: Closes #718

2016-07-23 Thread robertwb
Closes #718 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/38d9dea2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/38d9dea2 Diff:

[07/12] incubator-beam git commit: Cleanup and fix combiners_test.

2016-07-23 Thread robertwb
Cleanup and fix combiners_test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/01830510 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/01830510 Diff:

[GitHub] incubator-beam pull request #718: Change all examples and tests to use the n...

2016-07-23 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/718 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #705: Minor cdef value changes.

2016-07-21 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/705 Minor cdef value changes. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #721: Make DoFnRunner a Receiver.

2016-07-22 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/721 Make DoFnRunner a Receiver. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #705: Minor cdef value changes.

2016-07-22 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/705 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #702: Add tests for WindowedValue.

2016-07-22 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/702 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #706: Simple DoFnRunner optimizatons.

2016-07-22 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/706 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[2/2] incubator-beam git commit: Add tests for WindowedValue.

2016-07-21 Thread robertwb
Add tests for WindowedValue. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/441fad31 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/441fad31 Diff:

[1/2] incubator-beam git commit: Closes #702

2016-07-21 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk e5006991d -> 09b4daf23 Closes #702 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/09b4daf2 Tree:

[1/4] incubator-beam git commit: Remove expensive per-element-step logging context.

2016-07-21 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 37f426faf -> d78b38d1d Remove expensive per-element-step logging context. This is 3-4% of the total runtime. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[4/4] incubator-beam git commit: Restore (faster) logging context.

2016-07-21 Thread robertwb
/apache_beam/runners/common.pxd index f01a362..7191659 100644 --- a/sdks/python/apache_beam/runners/common.pxd +++ b/sdks/python/apache_beam/runners/common.pxd @@ -27,7 +27,7 @@ cdef class DoFnRunner(object): cdef object window_fn cdef object context # TODO(robertwb): Make this a DoFnContext

[3/4] incubator-beam git commit: Closes #706

2016-07-21 Thread robertwb
Closes #706 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d78b38d1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d78b38d1 Diff:

[2/4] incubator-beam git commit: Cache dofn.proces method.

2016-07-21 Thread robertwb
cdef object window_fn cdef object context # TODO(robertwb): Make this a DoFnContext cdef object tagged_receivers http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7c9d77ac/sdks/python/apache_beam/runners/common.py -

[2/2] incubator-beam git commit: Closes #705

2016-07-21 Thread robertwb
Closes #705 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e5006991 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e5006991 Diff:

[1/2] incubator-beam git commit: Minor cdef value changes.

2016-07-21 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk d78b38d1d -> e5006991d Minor cdef value changes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/19082adb Tree:

[GitHub] incubator-beam pull request #708: Revert "Restore (faster) logging context."

2016-07-21 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/708 Revert "Restore (faster) logging context." This fixes the jenkins build that relies on not-yet-pushed dataflow changes. --- This reve

[1/2] incubator-beam git commit: Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe.

2016-07-28 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 351c3831d -> c155ef0eb Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #752

2016-07-28 Thread robertwb
Closes #752 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c155ef0e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c155ef0e Diff:

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

2016-07-27 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/748 Optimize Map and Flatmap when there are no side inputs. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] incubator-beam pull request #770: Implement add_input for all CombineFns.

2016-08-02 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/770 Implement add_input for all CombineFns. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/3] incubator-beam git commit: Implement add_input for all CombineFns.

2016-08-03 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk e834fa82b -> 65152cab8 Implement add_input for all CombineFns. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ebf28c6 Tree:

[2/3] incubator-beam git commit: Document TupleCombineFns

2016-08-03 Thread robertwb
Document TupleCombineFns Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4a2239d3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4a2239d3 Diff:

[2/3] incubator-beam git commit: Address Robert's comments.

2016-07-20 Thread robertwb
Address Robert's comments. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a3b6fe4b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a3b6fe4b Diff:

[GitHub] incubator-beam pull request #702: Add tests for WindowedValue.

2016-07-20 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/702 Add tests for WindowedValue. I just realized I forgot to add this file when doing #691 . You can merge this pull request into a Git repository by running: $ git pull https

[2/2] incubator-beam git commit: Closes #694

2016-07-20 Thread robertwb
Closes #694 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/22126285 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/22126285 Diff:

[1/2] incubator-beam git commit: Remove "beamhead-02" container workaround

2016-07-20 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 71214b37a -> 221262858 Remove "beamhead-02" container workaround Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7baa6621 Tree:

[4/5] incubator-beam git commit: Clarifying comments.

2016-07-20 Thread robertwb
mon.py index 059359c..ef28c63 100644 --- a/sdks/python/apache_beam/runners/common.py +++ b/sdks/python/apache_beam/runners/common.py @@ -190,6 +190,7 @@ class DoFnState(object): self.step_name, aggregator) +# TODO(robertwb): Replace core.DoFnContext with this. class DoFnContext(objec

[3/5] incubator-beam git commit: Cythonize WindowedValue class.

2016-07-20 Thread robertwb
type(left) is not type(right): + return cmp(type(left), type(right)) +else: + # Don't bother paying the cost of a total ordering. + # TODO(robertwb): Avoid the type checks? + return not WindowedValue._typed_eq(left, right) + + @staticmethod + def _typed_eq(left, right): +

[5/5] incubator-beam git commit: Add Cython DoFnContext and Receiver stubs.

2016-07-20 Thread robertwb
dValue +from apache_beam.utils.windowed_value cimport WindowedValue + +cdef type SideOutputValue, TimestampedValue + cdef class DoFnRunner(object): cdef object dofn cdef object window_fn - cdef object context + cdef object context # TODO(robertwb): Make this a DoFnContext cdef object tagg

[2/5] incubator-beam git commit: Reduce the number of elements in the pvalue caching test.

2016-07-20 Thread robertwb
Reduce the number of elements in the pvalue caching test. It seems this test is causing travis-ci to time out, as the non-compiled version got slightly slower. 100,000 elements should be sufficient to see the effects of not caching. Project:

[1/5] incubator-beam git commit: Closes #691

2016-07-20 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk a643e2009 -> 37f426faf Closes #691 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/37f426fa Tree:

[3/3] incubator-beam git commit: Add size-estimation support to Python SDK Coders

2016-07-20 Thread robertwb
Add size-estimation support to Python SDK Coders Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/48a19eb2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/48a19eb2 Diff:

[1/3] incubator-beam git commit: Closes #678

2016-07-20 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 221262858 -> a643e2009 Closes #678 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a643e200 Tree:

[GitHub] incubator-beam pull request #676: Remove pipeline.apply(pvalue, callable)

2016-07-18 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/676 Remove pipeline.apply(pvalue, callable) Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] incubator-beam pull request #679: Add >> operator for labeling PTransforms.

2016-07-18 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/679 Add >> operator for labeling PTransforms. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[2/2] incubator-beam git commit: Closes #683

2016-07-19 Thread robertwb
Closes #683 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/062af66f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/062af66f Diff:

[1/2] incubator-beam git commit: Fix comment.

2016-07-19 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 561090580 -> 062af66f8 Fix comment. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fdab1f94 Tree:

[2/2] incubator-beam git commit: Remove pipeline.apply(pvalue, callable)

2016-07-18 Thread robertwb
Remove pipeline.apply(pvalue, callable) Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e68eb05e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e68eb05e Diff:

[1/2] incubator-beam git commit: Add >> operator for labeling PTransforms.

2016-07-18 Thread robertwb
alue.py index 6fc3041..063d0b5 100644 --- a/sdks/python/apache_beam/pvalue.py +++ b/sdks/python/apache_beam/pvalue.py @@ -79,10 +79,8 @@ class PValue(object): optional first label and a transform/callable object. It will call the pipeline.apply() method with this modified argument list.

[GitHub] incubator-beam pull request #679: Add >> operator for labeling PTransforms.

2016-07-18 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/679 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #676: Remove pipeline.apply(pvalue, callable)

2016-07-18 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/676 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: Closes #676

2016-07-18 Thread robertwb
Repository: incubator-beam Updated Branches: refs/heads/python-sdk 5daab7fb3 -> ad7c216f4 Closes #676 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ad7c216f Tree:

[2/2] incubator-beam git commit: Fixing broken example tests

2016-07-18 Thread robertwb
Fixing broken example tests Nose does not pick up decorated tests unless the decorator name starts with test_. This resulted in some test being inadvertently disabled. Also OptionsContext (https://github.com/aaltay/incubator-beam/blob/python-sdk/sdks/python/apache_beam/utils/options.py#L457) is

  1   2   3   4   5   >