Repository: incubator-beam
Updated Branches:
refs/heads/master 58000b397 -> a834fb0eb
BEAM-830 Support launch on YARN cluster.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b6b2e202
Tree:
This closes #1517
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a834fb0e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a834fb0e
Diff:
Add JUnit category for stateful ParDo tests
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f8b6bb7f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f8b6bb7f
Diff:
Closes #1410
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6ec45f7e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6ec45f7e
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master e7d7aa938 -> 6ec45f7e7
Reject stateful DoFn in ApexRunner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e85cea78
Tree:
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner c08ebbe79 -> 99001575d
Fix findbugs issues.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/99001575
Tree:
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5553c603/runners/apex/src/main/java/org/apache/beam/runners/apex/translators/utils/SerializablePipelineOptions.java
--
diff --git
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5553c603/runners/apex/src/main/java/org/apache/beam/runners/apex/translators/TranslationContext.java
--
diff --git
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5553c603/runners/apex/src/main/java/org/apache/beam/runners/apex/translation/operators/ApexParDoOperator.java
--
diff --git
Closes #1213
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c08ebbe7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c08ebbe7
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner 9197d1e05 -> c08ebbe79
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5553c603/runners/apex/src/test/java/org/apache/beam/runners/apex/translators/ParDoBoundTranslatorTest.java
BEAM-261 Make translators package private.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5553c603
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5553c603
Diff:
[BEAM-790] Validate PipelineOptions Default annotation
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f605b02a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f605b02a
Diff:
Switch DoFnSignature, etc, from TypeToken to TypeDescriptor
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8336b24c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8336b24c
Diff:
Format Regex according to style guidelines
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/79b04551
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/79b04551
Diff:
Added Regex Transform and test.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bc28799d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bc28799d
Diff:
Rename RegexTransform to just Regex
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6954abe4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6954abe4
Diff:
This closes #1296
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/11024552
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/11024552
Diff:
Adjust for merge from master.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9197d1e0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9197d1e0
Diff:
[BEAM-854] Removes ReifyTimestampsAndWindows
It was used effectively in 2 places:
- GatherAllPanes (used in PAssert):
replaced with a similar custom DoFn.
- A couple of GBK-related transforms:
folded into GroupByKeyOnly.
Also makes GBKO produce globally-windowed KWIs.
Project:
Closes #1183
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c6c41eac
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c6c41eac
Diff:
Implement GroupAlsoByWindowsEvaluator Directly
Invoke the ReduceFnRunner methods directly rather than by constructing a
GroupAlsoByWindowDoFn and a ParDoEvaluator. Instead, construct a
ReduceFnRunner, ReduceFn, and invoke the methods directly.
Project:
Upgrade google-api-services-dataflow to v1b3-rev43-1.22.0
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8c2b99e9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8c2b99e9
Diff:
This closes #1259
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d75d8b2b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d75d8b2b
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner 15287b838 -> 9197d1e05
This closes #1170
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/529f266a
Tree:
This closes #506
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f6a9733f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f6a9733f
Diff:
Refactor and reuse parameter analysis in DoFnSignatures
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8bf6d92c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8bf6d92c
Diff:
This closes #1270
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6a05cf4a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6a05cf4a
Diff:
Closes #1285
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f06deac8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f06deac8
Diff:
[BEAM-790] Validate PipelineOptions Default annotation.
This closes #1159
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bdd71c4b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bdd71c4b
This closes #1154
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/eac68cb2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/eac68cb2
Diff:
gcp-io: add ApiSurfaceTest
Copied from SDK, but I manually recreated the whitelist.
Surely there are refactoring improvements to be had so that we can more easily
reuse ApiSurface across modules.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Merge branch 'master' into apex-runner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ccd5fb3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ccd5fb3
Diff:
Closes #1263
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4dec38ce
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4dec38ce
Diff:
Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in
ROS."
This reverts commit ff5409f15a9f741f437c489b7de763cfa3c68278.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f233d858
This closes #1265
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2a1fdee5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2a1fdee5
Diff:
Makes FileBasedSink use a temporary directory
When writing to /path/to/foo, temporary files would be
written to /path/too/foo-temp-$uid (or something like that),
i.e. as siblings of the final output. That could lead
to issues like http://stackoverflow.com/q/39822859/278042
Now, temporary files
Add Tests for PAssert.IterableAssert#empty()
These tests demonstrate that an empty PCollection has no elements, and
that the empty() assertion affirmatively fails when there are elements
within the PCollection.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1258
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3419fef7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3419fef7
Diff:
This closes #1251
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b93ac70c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b93ac70c
Diff:
This closes #1252
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/faf55c78
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/faf55c78
Diff:
Closes #1284
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6e1e57b0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6e1e57b0
Diff:
Move watermark updates to the end of handleResult
This ensures that any state modifications are visible before watermarks
advancement permits additional progress.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
checkstyle: move from individual modules to root poms
Checkstyle config is no longer needed in individual modules;
instead, move it only to sdks/ , runners/, and examples/ poms.
Individual modules like sdks/java/core may still reference checkstyle
to configure their own options.
The only sketchy
Closes #1050
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/55d95198
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/55d95198
Diff:
Add TypeDescriptor#getTypes
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d936ed89
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d936ed89
Diff:
Fix javadoc throughout Beam
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/717b431c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/717b431c
Diff:
DoFnSignature: Make TypeDescriptor-returning methods public
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/71fa7cdf
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/71fa7cdf
Diff:
[BEAM-889] Let Spark handle the user-provided checkpointDir, but warn if not a
reliable fs.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/90a75d1f
Tree:
This closes #1225
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/52246933
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/52246933
Diff:
Flatten FiredTimers and ExtractFiredTimers
Pass a single collection of fired timers, and have those objects contain
the associated transform and key that they fired for. Timers already
contain the domain they are in.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Enable javadoc for all modules by default
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/aae65db6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/aae65db6
Diff:
move source jar creation into release profile
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/68f26386
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/68f26386
Diff:
This closes #1272
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/14e093a0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/14e093a0
Diff:
Remove @Default from IT options
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1eccd29b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1eccd29b
Diff:
This closes #1276
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/99062d10
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/99062d10
Diff:
Closes #1290
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5864a38b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5864a38b
Diff:
[BEAM-898] Fix Jenkins BigQueryTornadoes IT Failure
This closes #1279
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/46fbfe06
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/46fbfe06
Diff:
Closes #1269
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/15287b83
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/15287b83
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner 51af7e592 -> 15287b838
BEAM-844 Add README, wordcount test, fix View.AsIterable, initial GBK watermark.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Closes #1227
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/51af7e59
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/51af7e59
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner 968eb32b8 -> 51af7e592
BEAM-858 Enable ApexRunner integration test in examples.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Revert "Fix invalid snapshot repo in Travis CI's maven settings.xml"
This reverts commit fbc23ed9234088c9d88bf88b19a3a4d6d5e52446.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3c71e4da
Tree:
This closes #1095
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f603d43e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f603d43e
Diff:
This closes #1173
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/25102f79
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/25102f79
Diff:
Remove checked exceptions from PipelineResult.waitUntilFinish interface.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1559a7f4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1559a7f4
Closes #1160
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/48fdd067
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/48fdd067
Diff:
Use a int Sequence instead of a Random UUID for Aggregator IDs
Aggregator IDs are used to ensure that an Aggregator's identity is
consistent across synchronization barriers. This is only relevant when
constructing the map of Step -> Aggregator to enable querying, as the
DoFns represented within
Merge branch 'master' into apex-runner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6fc47ed1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6fc47ed1
Diff:
Remove unused GroupAlsoByWindowsDoFn.createDefault
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ce3aac2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ce3aac2
Diff:
Display data: minor cleanups
* wrong @Nullable
* Remove unused function
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d2553500
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d2553500
This closes #1196
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4cb1d10d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4cb1d10d
Diff:
Closes #1210
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/215980ad
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/215980ad
Diff:
This closes #1178
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/13af296d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/13af296d
Diff:
Remove use of OldDoFn in ValueWithRecordId
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/40513578
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/40513578
Diff:
[BEAM-443] Update Beam batch examples to call waitUntilFinish()
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e5afbe56
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e5afbe56
Diff:
Remove OldDoFn to DoFn in DirectRunner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/28720191
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/28720191
Diff:
Streamline Travis CI build
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5c558e6f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5c558e6f
Diff:
Remove use of OldDoFn from some DirectRunner tests
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3d086857
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3d086857
Diff:
Add default version for beam-sdks-java-io-kafka
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c8895338
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c8895338
Diff:
Use project-wise version in SparkRunner dep on KafkaIO
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9d6598d3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9d6598d3
Diff:
This closes #1158
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0ec956d3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0ec956d3
Diff:
Make DoFnInfo carry OldDoFn or DoFn
This will allow consumers to prepare to accept
DoFn while still accepting existing jobs that
use OldDoFn. It is a move towards treating
the Fn itself as just a serialized blob.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1162
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53fe3ee4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53fe3ee4
Diff:
Closes #1181
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/09a80ecb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/09a80ecb
Diff:
Use a Constant TupleTag in GroupAlsoByWindowEvaluator
This avoids the cost of generating a random ID, which is moderately
expensive.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/34a236ce
Tree:
This closes #1176
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/064f18a8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/064f18a8
Diff:
[BEAM-809] Create a KryoRegistrator for the SparkRunner.
Use Class#getName() instead of canonicalName().
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/13b83858
Tree:
Closes #1097
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ee6ad2fe
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ee6ad2fe
Diff:
This closes #1205
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f2ec8245
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f2ec8245
Diff:
Use a NavigableSet Instead of a PriorityQueue in WatermarkManager
This removes an O(n) call to remove, replacing it with an O(log(n))
call. This significantly improves scaling behavior of the DirectRunner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1195
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f9e07cfb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f9e07cfb
Diff:
Add experimental ServiceAccount option
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/60907b21
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/60907b21
Diff:
This closes #1202
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3fd3951d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3fd3951d
Diff:
Update Dataflow worker image pointer
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/95bf7a83
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/95bf7a83
Diff:
This closes #1171
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/78e2c038
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/78e2c038
Diff:
Closes #1180
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7091a012
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7091a012
Diff:
Add utility to handle JSON option manipulation
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/35244a68
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/35244a68
Diff:
[BEAM-658] Support Read.Unbounded primitive.
Changed mapSourceFunction to use scala's native Option.
Upgrade to kryo-serializers 0.39 that provides support for ReverseList (used by
Top).
Better logging.
Allow a longer read time-frame for read in tests.
Assert initial parallelism is gerater
Repository: incubator-beam
Updated Branches:
refs/heads/apex-runner fa3a6aa8d -> 968eb32b8
Encode bundle elements in the DirectRunner
This ensures that any changes that are caused when an element is encoded
and decoded is caught within the pipeline.
Project:
1 - 100 of 168 matches
Mail list logo