[GitHub] incubator-beam pull request: Optimize the Count CombineFn

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/88 Optimize the Count CombineFn Previously the accumulator was stored as a Long. This uses a singleton long[] to avoid the boxing and unboxing on every increment. This

[1/2] incubator-beam git commit: Fix the java doc for Combine.perKey and ApproximateQuantiles

Repository: incubator-beam Updated Branches: refs/heads/master 3ebdf4886 -> 69ec223e5 Fix the java doc for Combine.perKey and ApproximateQuantiles Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/df1d4862

[2/2] incubator-beam git commit: Fix the java doc for Combine.perKey and ApproximateQuantiles

Fix the java doc for Combine.perKey and ApproximateQuantiles This closes #272 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/69ec223e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/69ec223

incubator-beam-site git commit: Remove duplicate paragraph.

Repository: incubator-beam-site Updated Branches: refs/heads/asf-site 87d9a07fe -> edd0705b8 Remove duplicate paragraph. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/edd0705b Tree: http://gi

incubator-beam-site git commit: Fix blog post typo.

Repository: incubator-beam-site Updated Branches: refs/heads/asf-site edd0705b8 -> 0c5c6471b Fix blog post typo. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam-site/commit/0c5c6471 Tree: http://git-wip-us

[GitHub] incubator-beam pull request #436: Raise error rather than overwrite data whe...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/436 Raise error rather than overwrite data when sharding is not respected This is particularly relevant when withoutSharding() is used due to [BEAM-159]. You can merge this pull request

[1/2] incubator-beam git commit: Make TextFileReader observable

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 26ff65795 -> 53ab635c7 Make TextFileReader observable This allows future implementation of size tracking for elements in side input sources. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://

[2/2] incubator-beam git commit: Closes #726

Closes #726 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53ab635c Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53ab635c Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[2/3] incubator-beam git commit: Allow Top operations to take key argument rather than compare.

tor to rank the elements. + + Args: +compare: (optional) an implementation of "a < b" taking at least two +arguments (a and b). Additional arguments and side inputs specified +in the apply call become additional arguments to the comparator. +key: (optional) a ma

[3/3] incubator-beam git commit: Closes #741

Closes #741 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b4716d9d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b4716d9d Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/3] incubator-beam git commit: Better top implementation.

n, compare, _compare_id=None): # pylint: disable=invalid-name + # TODO(robertwb): Allow taking a key rather than a compare. + def __init__(self, n, compare): self._n = n +self._buffer_size = min(2 * n, n + 1000) self._compare = compare -self._compare_id = _

[GitHub] incubator-beam pull request #748: Optimize Map and Flatmap when there are no...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/748 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #741: Top improvements

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[1/2] incubator-beam git commit: Optimize Map and Flatmap when there are no side inputs.

gger_driver(self.windowing, True) - state = InMemoryUnmergedState() + self.driver = create_trigger_driver(self.windowing, True) + self.state_type = InMemoryUnmergedState + +def process(self, context): + k, vs = context.element + state = self.state_type() # TODO(rober

[2/2] incubator-beam git commit: Closes #748

Closes #748 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/351c3831 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/351c3831 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 351c3831d -> c155ef0eb Increased the GCS buffer size from 1MB to 8MB and introduced a 128kB buffer for the pipe. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/a

[2/2] incubator-beam git commit: Closes #752

Closes #752 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c155ef0e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c155ef0e Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[GitHub] incubator-beam pull request #770: Implement add_input for all CombineFns.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/770 Implement add_input for all CombineFns. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[2/3] incubator-beam git commit: Document TupleCombineFns

Document TupleCombineFns Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4a2239d3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4a2239d3 Diff: http://git-wip-us.apache.org/repos/asf/incubat

[1/3] incubator-beam git commit: Implement add_input for all CombineFns.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk e834fa82b -> 65152cab8 Implement add_input for all CombineFns. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ebf28c6 Tree: http://git-

[3/3] incubator-beam git commit: Closes #770

Closes #770 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/65152cab Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/65152cab Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[GitHub] incubator-beam pull request #785: Use the cythonized DoFnContext everywhere.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/785 Use the cythonized DoFnContext everywhere. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[GitHub] incubator-beam pull request #955: Implement dynamic work rebalancing for con...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/955 Implement dynamic work rebalancing for concat sources. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[1/2] incubator-beam git commit: Insert a shuffle before write finalization

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 4b584ca26 -> a60b58a94 Insert a shuffle before write finalization Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4602c954 Tree: http://g

[2/2] incubator-beam git commit: Closes #958

Closes #958 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a60b58a9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a60b58a9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[GitHub] incubator-beam pull request #964: Allow pickling of UnwindowedValues instanc...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/964 Allow pickling of UnwindowedValues instances Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[1/3] incubator-beam git commit: Allow pickling of UnwindowedValues instances

Repository: incubator-beam Updated Branches: refs/heads/python-sdk a60b58a94 -> ebae225ed Allow pickling of UnwindowedValues instances These are trivial wrappers produced when "windowing" by the global window, but may need to be materialized in some cases. Project: http://git-wip-us.apache.o

[2/3] incubator-beam git commit: Add unit test for unwindowed iterator picking.

Add unit test for unwindowed iterator picking. Also lifted this out to a top-level class rather than defining it on every element now that it's not longer a simple generator statement. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/

[3/3] incubator-beam git commit: Closes #964

Closes #964 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ebae225e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ebae225e Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Adds a text source to Python SDK.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk ebae225ed -> 4b7fe2dc5 Adds a text source to Python SDK. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2d1e7ff6 Tree: http://git-wip-us

[2/2] incubator-beam git commit: Closes #920

Closes #920 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4b7fe2dc Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4b7fe2dc Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[2/2] incubator-beam git commit: Removed unnecessary throttling of rename parallelism.

Removed unnecessary throttling of rename parallelism. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/24bb8f19 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/24bb8f19 Diff: http://git-wip-us

[1/2] incubator-beam git commit: Closes #965

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 4b7fe2dc5 -> 29b55e956 Closes #965 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/29b55e95 Tree: http://git-wip-us.apache.org/repos/asf/

[2/2] incubator-beam git commit: Closes #962

Closes #962 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2f09003e Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2f09003e Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Use sys.executable and "-m pip" to ensure we use the same Python and pip as the currently running one.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 29b55e956 -> 2f09003e3 Use sys.executable and "-m pip" to ensure we use the same Python and pip as the currently running one. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apach

[1/2] incubator-beam git commit: Changed ToStringCoder to BytesCoder in test

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 2f09003e3 -> adda16320 Changed ToStringCoder to BytesCoder in test The former can't be used with sources, as it can only encode. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.ap

[2/2] incubator-beam git commit: Closes #976

Closes #976 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/adda1632 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/adda1632 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Closes #979

Repository: incubator-beam Updated Branches: refs/heads/python-sdk adda16320 -> b6c7478ff Closes #979 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b6c7478f Tree: http://git-wip-us.apache.org/repos/asf/

[2/2] incubator-beam git commit: Updates lint configurations to ignore generated files.

Updates lint configurations to ignore generated files. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/49c03593 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/49c03593 Diff: http://git-wip-u

[3/3] incubator-beam git commit: Closes #978

Closes #978 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c1964bdd Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c1964bdd Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/3] incubator-beam git commit: Updates Dataflow API client.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk b6c7478ff -> c1964bdd6 http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/2e3384e6/sdks/python/apache_beam/internal/clients/dataflow/dataflow_v1b3_messages.py -

[2/3] incubator-beam git commit: Updates Dataflow API client.

Updates Dataflow API client. These files were generated using following command but I updated the licence and package name manually to match the current version. gen_client --discovery_url=dataflow.v1b3 --overwrite --outdir=out pip_package Project: http://git-wip-us.apache.org/repos/asf/incubat

[1/2] incubator-beam git commit: Closes #975

Repository: incubator-beam Updated Branches: refs/heads/python-sdk c1964bdd6 -> acd8d7952 Closes #975 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/acd8d795 Tree: http://git-wip-us.apache.org/repos/asf/

[2/2] incubator-beam git commit: Adds support for specifying a custom service account.

Adds support for specifying a custom service account. Updates Dataflow API client to latest version. Adds ability to skip generated files during lint checks. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/

[GitHub] incubator-beam pull request #981: Insert global windowing before write resul...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/981 Insert global windowing before write results GBK Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/2] incubator-beam git commit: Insert global windowing before write results GBK

Repository: incubator-beam Updated Branches: refs/heads/python-sdk acd8d7952 -> 701aff074 Insert global windowing before write results GBK Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/57a0b6af Tree: htt

[2/2] incubator-beam git commit: Closes #981

Closes #981 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/701aff07 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/701aff07 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Adds __all__ tags to source modules.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 345fc6985 -> 9565b2c27 Adds __all__ tags to source modules. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fbda5eef Tree: http://git-wip

[2/2] incubator-beam git commit: Closes #987

Closes #987 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9565b2c2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9565b2c2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/3] incubator-beam git commit: Using strings instead of integers for identifying CompressionTypes.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 9565b2c27 -> 0fa9c4be6 Using strings instead of integers for identifying CompressionTypes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commi

[3/3] incubator-beam git commit: Closes #989

Closes #989 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0fa9c4be Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0fa9c4be Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[2/3] incubator-beam git commit: Better documentation for CompressionTypes.

Better documentation for CompressionTypes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/04d84c99 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/04d84c99 Diff: http://git-wip-us.apache.org

[5/6] incubator-beam git commit: Move ConcatSource to iobase.

elevant_weights = weights[start[0]:end[0] + 1] -# TODO(robertwb): Implement fraction-at-position to properly scale -# partial start and end sources. -total = sum(relevant_weights) -running_total = [0] -for w in relevant_weights: - running_tota

[4/6] incubator-beam git commit: Implement liquid sharding for concat source.

if weights is None: +n = max(1, end[0] - start[0]) +self._cumulative_weights = [ + max(0, min(1, float(k) / n)) + for k in range(-start[0], len(self._sources) - start[0] + 1)] + else: +assert len(sources) == len(weights) +relevant_weights = weights[

[3/6] incubator-beam git commit: Minor cleanups in docstrings and error messages.

Minor cleanups in docstrings and error messages. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c1f42e62 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c1f42e62 Diff: http://git-wip-us.apac

[1/6] incubator-beam git commit: Move ConcatSource into its own module.

_bundles +self._lock = threading.RLock() +# Lazily-initialized list of RangeTrackers corresponding to each source. +self._range_trackers = [None] * len(source_bundles) +# The currently-being-iterated-over (and latest claimed) source. +self._claimed_source_ix = self._start[0] +

[6/6] incubator-beam git commit: Closes #955

Closes #955 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/753cc9c2 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/753cc9c2 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[2/6] incubator-beam git commit: Allow ConcatSource to take SourceBundles rather than raw Sources

= weights[start[0]:end[0] + 1] -# TODO(robertwb): Implement fraction-at-position to properly scale -# partial start and end sources. -total = sum(relevant_weights) -running_total = [0] -for w in relevant_weights: - running_total.append(max(1,

[GitHub] incubator-beam pull request #955: Implement dynamic work rebalancing for con...

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/955 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #998: Avro sink

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/998 Avro sink Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM

[GitHub] incubator-beam pull request #1000: Avro sinks

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/1000 Avro sinks Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM

[2/2] incubator-beam git commit: Closes #1006

Closes #1006 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bb649061 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bb649061 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff

[1/2] incubator-beam git commit: Add BEAM_PYTHON environment override to set the python executable

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 753cc9c2e -> bb649061f Add BEAM_PYTHON environment override to set the python executable Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/

[2/2] incubator-beam git commit: Closes #1004

Closes #1004 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7d988e3b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7d988e3b Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff

[1/2] incubator-beam git commit: Ignore virtualenv environment in git

Repository: incubator-beam Updated Branches: refs/heads/python-sdk bb649061f -> 7d988e3bb Ignore virtualenv environment in git Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7542904a Tree: http://git-wip

[4/4] incubator-beam git commit: Closes #1000

Closes #1000 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7e744e44 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7e744e44 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff

[3/4] incubator-beam git commit: Cleanup temporary files in textio and avroio tests.

Cleanup temporary files in textio and avroio tests. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f9c565b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f9c565b6 Diff: http://git-wip-us.a

[2/4] incubator-beam git commit: Fix and add test for ReadFromAvro transform.

Fix and add test for ReadFromAvro transform. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8bc965b6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8bc965b6 Diff: http://git-wip-us.apache.o

[1/4] incubator-beam git commit: Implement avro sink.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 7d988e3bb -> 7e744e445 Implement avro sink. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1090ca39 Tree: http://git-wip-us.apache.org/r

[GitHub] incubator-beam pull request #1000: Avro sinks

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/1000 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #1008: Fix python bin test.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/1008 Fix python bin test. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[1/2] incubator-beam git commit: Fix python bin test.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 7e744e445 -> 8a333a661 Fix python bin test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4b4ad732 Tree: http://git-wip-us.apache.org/r

[2/2] incubator-beam git commit: Closes #1008

Closes #1008 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8a333a66 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8a333a66 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff

[1/2] incubator-beam git commit: Allow .whl files to be staged with --extra_package

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 8a333a661 -> 19e3eff91 Allow .whl files to be staged with --extra_package Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/71c474a5 Tree:

[2/2] incubator-beam git commit: Closes #991

Closes #991 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/19e3eff9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/19e3eff9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[GitHub] incubator-beam pull request #1010: Compress serialized function data.

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/1010 Compress serialized function data. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[2/2] incubator-beam git commit: Closes #1010

Closes #1010 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dc92438f Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dc92438f Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff

[1/2] incubator-beam git commit: Compress serialized function data.

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 19e3eff91 -> dc92438fa Compress serialized function data. Pickled data is often quite compressible, but this is particularly useful for concat sources generated for large expansions of filepatterns. Project: http://git-wip-us

[1/3] incubator-beam git commit: Remove internal test config file

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 0545eb593 -> 5434e599b Remove internal test config file Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/436724e5 Tree: http://git-wip-us.

[2/3] incubator-beam git commit: Fix gcsio.exists call

Fix gcsio.exists call Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/cead6457 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/cead6457 Diff: http://git-wip-us.apache.org/repos/asf/incubator-

[3/3] incubator-beam git commit: Closes #558

Closes #558 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5434e599 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5434e599 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[3/4] incubator-beam git commit: Replace call() with check_call()

Replace call() with check_call() Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5fe6d7bf Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5fe6d7bf Diff: http://git-wip-us.apache.org/repos/asf

[4/4] incubator-beam git commit: Get current SDK package from PyPI instead of GitHub

Get current SDK package from PyPI instead of GitHub Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0bda677d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0bda677d Diff: http://git-wip-us.a

[1/4] incubator-beam git commit: Closes #569

Repository: incubator-beam Updated Branches: refs/heads/python-sdk bff980178 -> fbe44ee83 Closes #569 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/fbe44ee8 Tree: http://git-wip-us.apache.org/repos/asf/

[2/4] incubator-beam git commit: Define GOOGLE_PACKAGE_NAME and use it everywhere

Define GOOGLE_PACKAGE_NAME and use it everywhere Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/33720ec4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/33720ec4 Diff: http://git-wip-us.apac

[2/2] incubator-beam git commit: Closes #575

Closes #575 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/25349765 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/25349765 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Use the beamhead label for containers

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 5ab5567ea -> 253497655 Use the beamhead label for containers Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/05f45c8b Tree: http://git-wi

[GitHub] incubator-beam pull request #595: Callable ptransform cleanup

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/595 Callable ptransform cleanup Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] incubator-beam pull request #597: pipeline.options should never be None

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/597 pipeline.options should never be None Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[1/2] incubator-beam git commit: Closes #597

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 6b06e3e22 -> 342d2d798 Closes #597 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/342d2d79 Tree: http://git-wip-us.apache.org/repos/asf/

[GitHub] incubator-beam pull request #597: pipeline.options should never be None

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/597 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: pipeline.options should never be None

pipeline.options should never be None Also fix a typehints error this exposed. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/87961e4b Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/87961e

[4/4] incubator-beam git commit: Closes #595

Closes #595 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9bc04b75 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9bc04b75 Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/4] incubator-beam git commit: Remove unneeded label argument in ptransform_fn

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 342d2d798 -> 9bc04b750 Remove unneeded label argument in ptransform_fn The label is already in the fully qualified name due to nesting. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wi

[2/4] incubator-beam git commit: Cleanup dataflow_test.

Cleanup dataflow_test. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e0643774 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e0643774 Diff: http://git-wip-us.apache.org/repos/asf/incubator

[3/4] incubator-beam git commit: Better error message for poor use of callable apply

Better error message for poor use of callable apply Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c34f332a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c34f332a Diff: http://git-wip-us.a

[GitHub] incubator-beam pull request #595: Callable ptransform cleanup

Github user robertwb closed the pull request at: https://github.com/apache/incubator-beam/pull/595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-beam pull request #604: Add support for ZLIB and DEFLATE compressi...

GitHub user robertwb opened a pull request: https://github.com/apache/incubator-beam/pull/604 Add support for ZLIB and DEFLATE compression Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is

[2/2] incubator-beam git commit: Closes #604

Closes #604 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a580b31a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a580b31a Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/

[1/2] incubator-beam git commit: Add support for ZLIB and DEFLATE compression

Repository: incubator-beam Updated Branches: refs/heads/python-sdk 9bc04b750 -> a580b31ac Add support for ZLIB and DEFLATE compression Code originally contributed by Slaven Bilac. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/a

  1   2   3   4   5   6   >