[1/2] incubator-beam git commit: Closes #965

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 4b7fe2dc5 -> 29b55e956 Closes #965 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/29b55e95 Tree:

[2/2] incubator-beam git commit: Closes #920

Closes #920 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4b7fe2dc Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4b7fe2dc Diff:

[1/2] incubator-beam git commit: Adds a text source to Python SDK.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk ebae225ed -> 4b7fe2dc5 Adds a text source to Python SDK. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2d1e7ff6 Tree:

[3/3] incubator-beam git commit: Closes #964

Closes #964 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ebae225e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ebae225e Diff:

[1/3] incubator-beam git commit: Allow pickling of UnwindowedValues instances

Repository: incubator-beam Updated Branches: refs/heads/python-sdk a60b58a94 -> ebae225ed Allow pickling of UnwindowedValues instances These are trivial wrappers produced when "windowing" by the global window, but may need to be materialized in some cases. Project:

[2/3] incubator-beam git commit: Add unit test for unwindowed iterator picking.

Add unit test for unwindowed iterator picking. Also lifted this out to a top-level class rather than defining it on every element now that it's not longer a simple generator statement. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #964: Allow pickling of UnwindowedValues instanc...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/964 Allow pickling of UnwindowedValues instances Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] incubator-beam git commit: Closes #958

Closes #958 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a60b58a9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a60b58a9 Diff:

[1/2] incubator-beam git commit: Insert a shuffle before write finalization

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 4b584ca26 -> a60b58a94 Insert a shuffle before write finalization Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4602c954 Tree:

[GitHub] incubator-beam pull request #955: Implement dynamic work rebalancing for con...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/955 Implement dynamic work rebalancing for concat sources. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[1/3] incubator-beam git commit: Implement add_input for all CombineFns.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk e834fa82b -> 65152cab8 Implement add_input for all CombineFns. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ebf28c6 Tree:

[2/3] incubator-beam git commit: Document TupleCombineFns

Document TupleCombineFns Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4a2239d3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4a2239d3 Diff:

[GitHub] incubator-beam pull request #770: Implement add_input for all CombineFns.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/770 Implement add_input for all CombineFns. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] incubator-beam git commit: Closes #752

Closes #752 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c155ef0e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c155ef0e Diff:

[1/2] incubator-beam git commit: Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 351c3831d -> c155ef0eb Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #748

Closes #748 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/351c3831 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/351c3831 Diff:

[GitHub] incubator-beam pull request #741: Top improvements

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: Optimize Map and Flatmap when there are no side inputs.

elf.windowing, True) - state = InMemoryUnmergedState() + self.driver = create_trigger_driver(self.windowing, True) + self.state_type = InMemoryUnmergedState + +def process(self, context): + k, vs = context.element + state = self.state_type() # TODO(robertwb): Condit

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/748 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/3] incubator-beam git commit: Better top implementation.

ing). - compare_by_id = {} - - def __init__(self, n, compare, _compare_id=None): # pylint: disable=invalid-name + # TODO(robertwb): Allow taking a key rather than a compare. + def __init__(self, n, compare): self._n = n +self._buffer_size = min(2 * n, n + 1000) self._com

[3/3] incubator-beam git commit: Closes #741

Closes #741 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b4716d9d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b4716d9d Diff:

[2/2] incubator-beam git commit: Closes #726

Closes #726 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53ab635c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53ab635c Diff:

[1/2] incubator-beam git commit: Make TextFileReader observable

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 26ff65795 -> 53ab635c7 Make TextFileReader observable This allows future implementation of size tracking for elements in side input sources. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/748 Optimize Map and Flatmap when there are no side inputs. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] incubator-beam pull request #741: Top improvements

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/741 Top improvements Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM

[2/3] incubator-beam git commit: Update docstring.

Update docstring. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/76f3864b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/76f3864b Diff:

[3/3] incubator-beam git commit: Closes #730

Closes #730 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/26ff6579 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/26ff6579 Diff:

[1/3] incubator-beam git commit: Fix SDK name and version sent to the Cloud Dataflow service

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 153916fe9 -> 26ff65795 Fix SDK name and version sent to the Cloud Dataflow service Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #721: Make DoFnRunner a Receiver.

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/721 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[4/4] incubator-beam git commit: Closes #721

Closes #721 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/153916fe Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/153916fe Diff:

[3/4] incubator-beam git commit: Allow passing logging context directly.

Allow passing logging context directly. This is better than passing a logger module with a specific class. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/de35f282 Tree:

[GitHub] incubator-beam pull request #718: Change all examples and tests to use the n...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/718 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[07/12] incubator-beam git commit: Cleanup and fix combiners_test.

Cleanup and fix combiners_test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/01830510 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/01830510 Diff:

[05/12] incubator-beam git commit: fixup: failing tests expecting name

fixup: failing tests expecting name Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c5b5b14d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c5b5b14d Diff:

[12/12] incubator-beam git commit: Closes #718

Closes #718 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/38d9dea2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/38d9dea2 Diff:

[02/12] incubator-beam git commit: Move names out of transform constructors.

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/031c4cce/sdks/python/apache_beam/transforms/combiners_test.py -- diff --git a/sdks/python/apache_beam/transforms/combiners_test.py

[08/12] incubator-beam git commit: Fixes examples

Fixes examples Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2ff5630b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2ff5630b Diff:

[10/12] incubator-beam git commit: Lint fixes.

Lint fixes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b15d35ca Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b15d35ca Diff:

[03/12] incubator-beam git commit: Move names out of transform constructors.

Move names out of transform constructors. sed -i -r 's/[|] (\S+)[(](["'"'"'][^"'"'"']+.)(, +|([)]))/| \2 >> \1(\4/g' Small number of tests will need to be fixed by hand. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[04/12] incubator-beam git commit: Fix multi-input named PTransforms.

Fix multi-input named PTransforms. Now delegate the __ror__ logic entirely for the naming wrapper. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/937cf69e Tree:

[11/12] incubator-beam git commit: Final cleanup pass.

Final cleanup pass. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e3c078fe Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e3c078fe Diff:

[09/12] incubator-beam git commit: Fix label-sensitive test.

Fix label-sensitive test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2a59a121 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2a59a121 Diff:

[1/4] incubator-beam git commit: Refactor setup.py to separate strings/versions

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 47042ce89 -> 9fe102a5c Refactor setup.py to separate strings/versions Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/af6d3804 Tree:

[3/4] incubator-beam git commit: Refactor examples to use save_main_session

Refactor examples to use save_main_session Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3d64a8c5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3d64a8c5 Diff:

[2/4] incubator-beam git commit: Make save_main_session optional

Make save_main_session optional Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2e36602c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2e36602c Diff:

[4/4] incubator-beam git commit: Closes #723

Closes #723 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9fe102a5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9fe102a5 Diff:

[GitHub] incubator-beam pull request #705: Minor cdef value changes.

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/705 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #702: Add tests for WindowedValue.

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/702 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #721: Make DoFnRunner a Receiver.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/721 Make DoFnRunner a Receiver. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #708: Revert "Restore (faster) logging context."

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/708 Revert "Restore (faster) logging context." This fixes the jenkins build that relies on not-yet-pushed dataflow changes. --- This reve

[1/2] incubator-beam git commit: Closes #702

Repository: incubator-beam Updated Branches: refs/heads/python-sdk e5006991d -> 09b4daf23 Closes #702 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/09b4daf2 Tree:

[2/2] incubator-beam git commit: Add tests for WindowedValue.

Add tests for WindowedValue. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/441fad31 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/441fad31 Diff:

[1/2] incubator-beam git commit: Minor cdef value changes.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk d78b38d1d -> e5006991d Minor cdef value changes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/19082adb Tree:

[2/2] incubator-beam git commit: Closes #705

Closes #705 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e5006991 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e5006991 Diff:

[1/4] incubator-beam git commit: Remove expensive per-element-step logging context.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 37f426faf -> d78b38d1d Remove expensive per-element-step logging context. This is 3-4% of the total runtime. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[4/4] incubator-beam git commit: Restore (faster) logging context.

/apache_beam/runners/common.pxd index f01a362..7191659 100644 --- a/sdks/python/apache_beam/runners/common.pxd +++ b/sdks/python/apache_beam/runners/common.pxd @@ -27,7 +27,7 @@ cdef class DoFnRunner(object): cdef object window_fn cdef object context # TODO(robertwb): Make this a DoFnContext

[3/4] incubator-beam git commit: Closes #706

Closes #706 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d78b38d1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d78b38d1 Diff:

[2/4] incubator-beam git commit: Cache dofn.proces method.

cdef object window_fn cdef object context # TODO(robertwb): Make this a DoFnContext cdef object tagged_receivers http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7c9d77ac/sdks/python/apache_beam/runners/common.py -

[GitHub] incubator-beam pull request #706: Simple DoFnRunner optimizatons.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/706 Simple DoFnRunner optimizatons. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #705: Minor cdef value changes.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/705 Minor cdef value changes. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #691: Add low-level data structures to be shared...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/691 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #702: Add tests for WindowedValue.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/702 Add tests for WindowedValue. I just realized I forgot to add this file when doing #691 . You can merge this pull request into a Git repository by running: $ git pull https

[4/5] incubator-beam git commit: Clarifying comments.

mon.py index 059359c..ef28c63 100644 --- a/sdks/python/apache_beam/runners/common.py +++ b/sdks/python/apache_beam/runners/common.py @@ -190,6 +190,7 @@ class DoFnState(object): self.step_name, aggregator) +# TODO(robertwb): Replace core.DoFnContext with this. class DoFnContext(objec

[3/5] incubator-beam git commit: Cythonize WindowedValue class.

type(left) is not type(right): + return cmp(type(left), type(right)) +else: + # Don't bother paying the cost of a total ordering. + # TODO(robertwb): Avoid the type checks? + return not WindowedValue._typed_eq(left, right) + + @staticmethod + def _typed_eq(left, right): +

[5/5] incubator-beam git commit: Add Cython DoFnContext and Receiver stubs.

dValue +from apache_beam.utils.windowed_value cimport WindowedValue + +cdef type SideOutputValue, TimestampedValue + cdef class DoFnRunner(object): cdef object dofn cdef object window_fn - cdef object context + cdef object context # TODO(robertwb): Make this a DoFnContext cdef object tagg

[2/5] incubator-beam git commit: Reduce the number of elements in the pvalue caching test.

Reduce the number of elements in the pvalue caching test. It seems this test is causing travis-ci to time out, as the non-compiled version got slightly slower. 100,000 elements should be sufficient to see the effects of not caching. Project:

[1/5] incubator-beam git commit: Closes #691

Repository: incubator-beam Updated Branches: refs/heads/python-sdk a643e2009 -> 37f426faf Closes #691 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/37f426fa Tree:

[2/2] incubator-beam git commit: Closes #694

Closes #694 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/22126285 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/22126285 Diff:

[GitHub] incubator-beam pull request #696: Cythonize WindowedValue class.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/696 Cythonize WindowedValue class. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #691: Add low-level data structures to be shared...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/691 Add low-level data structures to be shared with the worker. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[2/2] incubator-beam git commit: Closes #683

Closes #683 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/062af66f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/062af66f Diff:

[1/2] incubator-beam git commit: Fix comment.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 561090580 -> 062af66f8 Fix comment. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fdab1f94 Tree:

[GitHub] incubator-beam pull request #676: Remove pipeline.apply(pvalue, callable)

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/676 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #679: Add >> operator for labeling PTransforms.

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/679 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: Add >> operator for labeling PTransforms.

alue.py index 6fc3041..063d0b5 100644 --- a/sdks/python/apache_beam/pvalue.py +++ b/sdks/python/apache_beam/pvalue.py @@ -79,10 +79,8 @@ class PValue(object): optional first label and a transform/callable object. It will call the pipeline.apply() method with this modified argument list.

[2/2] incubator-beam git commit: Closes #679

Closes #679 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/56109058 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/56109058 Diff:

[2/4] incubator-beam git commit: Used fast primitives coder as fallback coder.

Used fast primitives coder as fallback coder. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/64457d04 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/64457d04 Diff:

[4/4] incubator-beam git commit: Implement coder optimized for coding primitives.

y. + def is_kv_coder(self): +return True + + def key_coder(self): +return self + + def value_coder(self): +return self + + class Base64PickleCoder(Coder): """Coder of objects by Python pickle, then base64 encoding.""" # TODO(robertwb): Do base64 en

[1/4] incubator-beam git commit: Closes #683

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 69f895a2e -> b00f915ee Closes #683 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b00f915e Tree:

[1/2] incubator-beam git commit: Closes #685

Repository: incubator-beam Updated Branches: refs/heads/python-sdk ad7c216f4 -> 69f895a2e Closes #685 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/69f895a2 Tree:

[2/2] incubator-beam git commit: Fixing broken example tests

Fixing broken example tests Nose does not pick up decorated tests unless the decorator name starts with test_. This resulted in some test being inadvertently disabled. Also OptionsContext (https://github.com/aaltay/incubator-beam/blob/python-sdk/sdks/python/apache_beam/utils/options.py#L457) is

[1/2] incubator-beam git commit: Closes #676

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 5daab7fb3 -> ad7c216f4 Closes #676 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ad7c216f Tree:

[2/2] incubator-beam git commit: Remove pipeline.apply(pvalue, callable)

Remove pipeline.apply(pvalue, callable) Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e68eb05e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e68eb05e Diff:

[GitHub] incubator-beam pull request #679: Add >> operator for labeling PTransforms.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/679 Add >> operator for labeling PTransforms. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[GitHub] incubator-beam pull request #676: Remove pipeline.apply(pvalue, callable)

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/676 Remove pipeline.apply(pvalue, callable) Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] incubator-beam git commit: Fixes several issues related to 'filebasedsource'.

Fixes several issues related to 'filebasedsource'. Adds a method 'fileio.ChannelFactory.size_in_bytes()' that can be used to determine the size of a single file. Implements this method for 'ChannelFactory' implementations for GCS and local files. Updates 'filebasedsource' to use this method

[1/2] incubator-beam git commit: Closes #599

Repository: incubator-beam Updated Branches: refs/heads/python-sdk d898d56ae -> 1305c108a Closes #599 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1305c108 Tree:

[1/2] incubator-beam git commit: Update some of the example tests to use assert_that

;""Test for the autocomplete example.""" -import collections import unittest - import apache_beam as beam from apache_beam.examples.complete import autocomplete -from apache_beam.pvalue import AsIter - -# TODO(robertwb): Move to testing utilities. - - -def ass

[1/2] incubator-beam git commit: Closes #653

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 3b6950689 -> c8cef2cba Closes #653 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c8cef2cb Tree:

[2/2] incubator-beam git commit: Fix min and max timestamp on 32-bit machines

Fix min and max timestamp on 32-bit machines Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/afc68bc3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/afc68bc3 Diff:

[GitHub] incubator-beam pull request #657: Accept runners by fully qualified name.

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/657 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[2/3] incubator-beam git commit: Cleanup known runners code.

Cleanup known runners code. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c055e845 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c055e845 Diff:

[1/3] incubator-beam git commit: Accept runners by fully qualified name.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 762a2930a -> 3b6950689 Accept runners by fully qualified name. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/84fe8954 Tree:

[3/3] incubator-beam git commit: Closes #657

Closes #657 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3b695068 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3b695068 Diff:

[1/2] incubator-beam git commit: DoOutputsTuple cleanup

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 7c0c27ac0 -> 762a2930a DoOutputsTuple cleanup This avoids the same (logical) PCollection from being re-created with a different producer. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: Closes #658

Closes #658 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/762a2930 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/762a2930 Diff:

[GitHub] incubator-beam pull request #657: Accept runners by fully qualified name.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/657 Accept runners by fully qualified name. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] incubator-beam pull request #658: DoOutputsTuple cleanup

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/658 DoOutputsTuple cleanup Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #641: Add type hints to bigshuffle to avoid pick...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/641 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: Added cy_combiners.py to the list of Cythonized Python SDK files.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 52c0d893e -> 2b9d81fcc Added cy_combiners.py to the list of Cythonized Python SDK files. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

<    1   2   3   4   5   >