[GitHub] incubator-beam pull request #1654: [BEAM-1177] Input DStream "bundles" shoul...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1654 [BEAM-1177] Input DStream "bundles" should be in serialized form and include relevant metadata. Be sure to do all of the following to help us incorporate your contribution

[GitHub] incubator-beam pull request #1500: [BEAM-1074] Set default-partitioner in So...

Github user amitsela closed the pull request at: https://github.com/apache/incubator-beam/pull/1500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: [BEAM-362] Port runners to runners-core AggregatoryFactory

Repository: incubator-beam Updated Branches: refs/heads/master d624d3b6b -> 5ebbd500c [BEAM-362] Port runners to runners-core AggregatoryFactory Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/55f04955

[2/2] incubator-beam git commit: This closes #1644

This closes #1644 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ebbd500 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ebbd500 Diff:

[1/2] incubator-beam git commit: [BEAM-932] Enable findbugs validation (and fix existing issues)

Repository: incubator-beam Updated Branches: refs/heads/master 1ad638e51 -> 4323247a3 [BEAM-932] Enable findbugs validation (and fix existing issues) Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1463

This closes #1463 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4323247a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4323247a Diff:

[GitHub] incubator-beam pull request #1614: [BEAM-853] Force streaming execution on b...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1614 [BEAM-853] Force streaming execution on batch pipelines for testing. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam pull request #1579: [BEAM-1130, BEAM-1133] Allow users to def...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1579 [BEAM-1130, BEAM-1133] Allow users to define max records per batch, update tests accordingly. Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] incubator-beam pull request #1578: [BEAM-757, BEAM-807]

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1578 [BEAM-757, BEAM-807] Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[1/2] incubator-beam git commit: [BEAM-921] spark-runner: register sources and coders to serialize with java serializer

Repository: incubator-beam Updated Branches: refs/heads/master e841b1a21 -> bf8a3cb3a [BEAM-921] spark-runner: register sources and coders to serialize with java serializer Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1552

This closes #1552 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bf8a3cb3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bf8a3cb3 Diff:

[GitHub] incubator-beam-site pull request #106: [BEAM-900] Spark quickstart instructi...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam-site/pull/106 [BEAM-900] Spark quickstart instructions. R: @davorbonaci You can merge this pull request into a Git repository by running: $ git pull https://github.com/amitsela/incubator-beam

[2/2] incubator-beam git commit: This closes #1561

This closes #1561 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d9a63117 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d9a63117 Diff:

[1/2] incubator-beam git commit: Remove misc uses of OldDoFn

Repository: incubator-beam Updated Branches: refs/heads/master 63d197cd0 -> d9a631173 Remove misc uses of OldDoFn Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e48b0e6b Tree:

[GitHub] incubator-beam pull request #1553: [BEAM-1111] Reject timers for ParDo in Sp...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1553 [BEAM-] Reject timers for ParDo in SparkRunner streaming evaluators. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam-site pull request #103: [BEAM-507] Fill in the documentation/...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam-site/pull/103 [BEAM-507] Fill in the documentation/runners/spark portion of the web… …site. You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] incubator-beam pull request #1532: [BEAM-329] Spark runner README should hav...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1532 [BEAM-329] Spark runner README should have a proper batch example. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make

[2/2] incubator-beam git commit: This closes #1531

This closes #1531 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9ccf6dbe Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9ccf6dbe Diff:

[1/2] incubator-beam git commit: [BEAM-1094] Set test scope for Kafka IO and junit

Repository: incubator-beam Updated Branches: refs/heads/master b41a46e86 -> 9ccf6dbea [BEAM-1094] Set test scope for Kafka IO and junit Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/baf5e6bd Tree:

[3/3] incubator-beam git commit: This closes #1466

This closes #1466 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6893a727 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6893a727 Diff:

[1/3] incubator-beam git commit: [BEAM-1000, BEAM-1050] Fixed PipelineResult.State Failed for streaming, support non-blocking cancel/waituntilfinish in batch. Added a SparkPipelineResult class to addr

Repository: incubator-beam Updated Branches: refs/heads/master ef9871c36 -> 6893a7270 [BEAM-1000, BEAM-1050] Fixed PipelineResult.State Failed for streaming, support non-blocking cancel/waituntilfinish in batch. Added a SparkPipelineResult class to address PipelineResult#waitUntilFinish()

[2/3] incubator-beam git commit: Redistributed some responsibilities in order to remove getAggregatorValues() form EvaluationContext.

Redistributed some responsibilities in order to remove getAggregatorValues() form EvaluationContext. Inferred excepted exception handling according to existing codebase and tests. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #1500: [BEAM-1074] Set default-partitioner in So...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1500 [BEAM-1074] Set default-partitioner in SourceRDD.Unbounded. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[2/2] incubator-beam git commit: This closes #1450

This closes #1450 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3a8b9b52 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3a8b9b52 Diff:

[1/2] incubator-beam git commit: [BEAM-1052] Add InputDStream id to MicrobatchSource hashcode.

Repository: incubator-beam Updated Branches: refs/heads/master 3f16f2660 -> 3a8b9b521 [BEAM-1052] Add InputDStream id to MicrobatchSource hashcode. Done to avoid collisions between splits of different sources. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[1/2] incubator-beam git commit: [BEAM-851] Determine if the pipeline must be translated into streaming mode (if not set)

Repository: incubator-beam Updated Branches: refs/heads/master 3ad767750 -> 8cc43aa70 [BEAM-851] Determine if the pipeline must be translated into streaming mode (if not set) Now an Evaluator (visitor) detects if there are Unbonded.Read transforms. This approach is based on Flink's

[2/2] incubator-beam git commit: This closes #1436

This closes #1436 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8cc43aa7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8cc43aa7 Diff:

[1/3] incubator-beam git commit: Remove unused WindowingInternals.writePCollectionViewData

Repository: incubator-beam Updated Branches: refs/heads/master 632576b5b -> 3ad767750 Remove unused WindowingInternals.writePCollectionViewData Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/803bbe2a

[3/3] incubator-beam git commit: [BEAM-498] Remove obsolete WindowingInternals#writePCollectionViewData This closes #1430

[BEAM-498] Remove obsolete WindowingInternals#writePCollectionViewData This closes #1430 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ad76775 Tree:

[2/3] incubator-beam git commit: Remove unused body of StreamingPCollectionViewWriterFn

Remove unused body of StreamingPCollectionViewWriterFn Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/07544ef3 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/07544ef3 Diff:

[2/2] incubator-beam git commit: This closes #1393

This closes #1393 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/875631f0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/875631f0 Diff:

[1/2] incubator-beam git commit: [Beam-1001] Add non blocking cancel() and waitUntilFinish() for streaming applications. remove timeout parameer in spark pipeline option.

Repository: incubator-beam Updated Branches: refs/heads/master d93e9a88b -> 875631f07 [Beam-1001] Add non blocking cancel() and waitUntilFinish() for streaming applications. remove timeout parameer in spark pipeline option. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo

[1/2] incubator-beam git commit: [BEAM-983] Fix a bunch of precommit errors from #1332

Repository: incubator-beam Updated Branches: refs/heads/master 201110222 -> dbbd5e448 [BEAM-983] Fix a bunch of precommit errors from #1332 Renames TestPipelineOptions to SparkTestPipelineOptions To avoid confusion with sdk.testing.TestPipelineOptions. Also, a couple of other minor fixes.

[2/2] incubator-beam git commit: This closes #1364

This closes #1364 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dbbd5e44 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dbbd5e44 Diff:

[GitHub] incubator-beam pull request #1362: [BEAM-983] runners/spark/translation/stre...

Github user amitsela closed the pull request at: https://github.com/apache/incubator-beam/pull/1362 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1362: [BEAM-983] add missing license.

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1362 [BEAM-983] add missing license. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[1/2] incubator-beam git commit: [BEAM-891] fix build occasionally fails on IndexOutOfBoundsException.

Repository: incubator-beam Updated Branches: refs/heads/master 47646d641 -> 503f26f44 [BEAM-891] fix build occasionally fails on IndexOutOfBoundsException. Moved "TestPipelineOptions#withTmpCheckpointDir" to TestPipelineOptionsForStreaming. Removed an unused member in

[2/2] incubator-beam git commit: This closes #1332

This closes #1332 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/503f26f4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/503f26f4 Diff:

[2/2] incubator-beam git commit: This closes #1291

This closes #1291 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2bc66f90 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2bc66f90 Diff:

[1/2] incubator-beam git commit: [BEAM-762] Unify spark-runner EvaluationContext and StreamingEvaluationContext

Repository: incubator-beam Updated Branches: refs/heads/master 9c300cde8 -> 2bc66f903 [BEAM-762] Unify spark-runner EvaluationContext and StreamingEvaluationContext PR 1291 review changes. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[1/2] incubator-beam git commit: [BEAM-944] Spark runner causes an exception when creating pipeline options. Create a SparkContextOptions for context-ware options.

Repository: incubator-beam Updated Branches: refs/heads/master cd3f61cf8 -> e43a38355 [BEAM-944] Spark runner causes an exception when creating pipeline options. Create a SparkContextOptions for context-ware options. Move UsesProvidedSparkContext property to SparkPipelineOptions so it's

[2/2] incubator-beam git commit: This closes #1316

This closes #1316 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e43a3835 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e43a3835 Diff:

[GitHub] incubator-beam pull request #1316: [BEAM-944] Spark runner causes an excepti...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1316 [BEAM-944] Spark runner causes an exception when creating pipeline op… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam pull request #1257: [BEAM-880] Avoid emitting default empty i...

Github user amitsela closed the pull request at: https://github.com/apache/incubator-beam/pull/1257 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[1/2] incubator-beam git commit: [BEAM-889] Let Spark handle the user-provided checkpointDir, but warn if not a reliable fs.

Repository: incubator-beam Updated Branches: refs/heads/master 46fbfe06b -> 14e093a0a [BEAM-889] Let Spark handle the user-provided checkpointDir, but warn if not a reliable fs. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1272

This closes #1272 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/14e093a0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/14e093a0 Diff:

[GitHub] incubator-beam-site pull request #65: [BEAM-890] Update compatibility matrix...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam-site/pull/65 [BEAM-890] Update compatibility matrix for Spark. You can merge this pull request into a Git repository by running: $ git pull https://github.com/amitsela/incubator-beam-site

[GitHub] incubator-beam pull request #1271: [BEAM-889] CheckpointDir option does not ...

Github user amitsela closed the pull request at: https://github.com/apache/incubator-beam/pull/1271 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] incubator-beam pull request #1272: [BEAM-889] CheckpointDir option does not ...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1272 [BEAM-889] CheckpointDir option does not accept relative path and req… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam pull request #1271: [BEAM-889] CheckpointDir option does not ...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1271 [BEAM-889] CheckpointDir option does not accept relative path and req… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[1/2] incubator-beam git commit: Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS."

Repository: incubator-beam Updated Branches: refs/heads/master d75d8b2bb -> 6a05cf4a9 Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS." This reverts commit ff5409f15a9f741f437c489b7de763cfa3c68278. Project:

[2/2] incubator-beam git commit: This closes #1270

This closes #1270 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6a05cf4a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6a05cf4a Diff:

[GitHub] incubator-beam pull request #1270: Revert "[BEAM-808] Increase "spark.port.m...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1270 Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindExce… Be sure to do all of the following to help us incorporate your contribution qui

[1/2] incubator-beam git commit: [BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS.

Repository: incubator-beam Updated Branches: refs/heads/master 2c0d0f476 -> 529f266ae [BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1170

This closes #1170 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/529f266a Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/529f266a Diff:

[GitHub] incubator-beam pull request #1257: [BEAM-880] Avoid emitting default empty i...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1257 [BEAM-880] Avoid emitting default empty iterable in PAssert.GroupedGl… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[1/2] incubator-beam git commit: Directly implement ReifyTimestampsAndWindows in SparkRunner

Repository: incubator-beam Updated Branches: refs/heads/master 215980ad3 -> 9c3e3e7a3 Directly implement ReifyTimestampsAndWindows in SparkRunner Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/597e3955

[1/2] incubator-beam git commit: [BEAM-809] Create a KryoRegistrator for the SparkRunner.

Repository: incubator-beam Updated Branches: refs/heads/master 53fe3ee42 -> 78e2c0387 [BEAM-809] Create a KryoRegistrator for the SparkRunner. Use Class#getName() instead of canonicalName(). Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1162

This closes #1162 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53fe3ee4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53fe3ee4 Diff:

[1/2] incubator-beam git commit: [BEAM-799] Support GroupByKey directly.

Repository: incubator-beam Updated Branches: refs/heads/master f2fe1ae46 -> 53fe3ee42 [BEAM-799] Support GroupByKey directly. Remove runner override for GroupByKey. Avoid NPE if no sideInputs are available in reader. Handle CombineFn with or without context. Project:

[GitHub] incubator-beam pull request #1171: [BEAM-809] Create a KryoRegistrator for t...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1171 [BEAM-809] Create a KryoRegistrator for the SparkRunner. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] incubator-beam pull request #1170: [BEAM-808] Increase "spark.port.maxRetrie...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1170 [BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in… Be sure to do all of the following to help us incorporate your contribution quickly

[GitHub] incubator-beam pull request #1162: [BEAM-799] Support GroupByKey directly

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1162 [BEAM-799] Support GroupByKey directly Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] incubator-beam pull request #1161: [BEAM-769] Spark streaming tests fail on ...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1161 [BEAM-769] Spark streaming tests fail on "nothing processed" if runti… Be sure to do all of the following to help us incorporate your contribution quickly

[2/2] incubator-beam git commit: This closes #1153

This closes #1153 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a9a41eb9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a9a41eb9 Diff:

[1/2] incubator-beam git commit: [BEAM-794] Differ combining in case of merging windows with sideInputs.

Repository: incubator-beam Updated Branches: refs/heads/master 4c9058236 -> a9a41eb94 [BEAM-794] Differ combining in case of merging windows with sideInputs. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #1153: [BEAM-794] Differ combining in case of me...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1153 [BEAM-794] Differ combining in case of merging windows with sideInputs. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [x

[GitHub] incubator-beam pull request #1144: [BEAM-781] Remove Spark's batch unit test...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1144 [BEAM-781] Remove Spark's batch unit tests and rely on ROS tests inst… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam pull request #1143: [BEAM-658] Support Read.Unbounded primiti...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1143 [BEAM-658] Support Read.Unbounded primitive. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/3] incubator-beam git commit: [BEAM-259] Execute selected RunnableOnService tests with Spark runner.

[BEAM-259] Execute selected RunnableOnService tests with Spark runner. Handle empty Flatten for bounded. Spark will bubble out a SparkException for user code failure, so this won't catch. Asserting on the error message should be good enough. outputWithTimestamp should handle start/finishBundle

[1/3] incubator-beam git commit: [BEAM-259] Execute selected RunnableOnService tests with Spark runner.

Repository: incubator-beam Updated Branches: refs/heads/master b0cb2e87b -> c472e1227 http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7eecd7ee/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkProcessContext.java

[3/3] incubator-beam git commit: This closes #1055

This closes #1055 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c472e122 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c472e122 Diff:

[2/2] incubator-beam git commit: This closes #1125

This closes #1125 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b0cb2e87 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b0cb2e87 Diff:

[1/2] incubator-beam git commit: [BEAM-744] UnboundedKafkaReader should return as soon as it can.

Repository: incubator-beam Updated Branches: refs/heads/master ea04e618e -> b0cb2e87b [BEAM-744] UnboundedKafkaReader should return as soon as it can. Use timeout directly in nextBatch() Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[2/2] incubator-beam git commit: This closes #1133

This closes #1133 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ea04e618 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ea04e618 Diff:

[1/2] incubator-beam git commit: [BEAM-777] KafkaIO Test should handle reader.start() better.

Repository: incubator-beam Updated Branches: refs/heads/master dde8e35ca -> ea04e618e [BEAM-777] KafkaIO Test should handle reader.start() better. KafkaIOTest : start() can return false Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit:

[GitHub] incubator-beam pull request #1125: [BEAM-744] A runner should be able to ove...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1125 [BEAM-744] A runner should be able to override KafkaIO max wait prope… Be sure to do all of the following to help us incorporate your contribution quickly and easily

[2/2] incubator-beam git commit: This closes #1073

This closes #1073 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/49f94443 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/49f94443 Diff:

[2/2] incubator-beam git commit: This closes #1072

This closes #1072 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d790dfe1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d790dfe1 Diff:

[GitHub] incubator-beam pull request #1073: [BEAM-735] PAssertStreaming should make s...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1073 [BEAM-735] PAssertStreaming should make sure the assertion happened. Be sure to do all of the following to help us incorporate your contribution quickly and easily

[GitHub] incubator-beam pull request #1072: [BEAM-734] Add StreamingListeners via Spa...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1072 [BEAM-734] Add StreamingListeners via SparkPipelineOptions. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] incubator-beam pull request #1055: [BEAM-259] Enable RunnableOnService for b...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1055 [BEAM-259] Enable RunnableOnService for batch. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] incubator-beam pull request #1040: [BEAM-703] SingletonViewFn might exhaust ...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/1040 [BEAM-703] SingletonViewFn might exhaust defaultValue if it's serialized after being used. Be sure to do all of the following to help us incorporate your contribution quickly

[1/2] incubator-beam git commit: [BEAM-657] Support Read.Bounded primitive.

Repository: incubator-beam Updated Branches: refs/heads/master 4872bde8f -> a00d2f810 [BEAM-657] Support Read.Bounded primitive. Support Read.Bounded primitive. Avro requires this for snappy. Create is supported by Read.Bounded now. Read.Bounded support should solve gs issues now. remove

[2/2] incubator-beam git commit: This closes #983

This closes #983 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a00d2f81 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a00d2f81 Diff:

[2/2] incubator-beam git commit: This closes #982

This closes #982 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6082ebcc Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6082ebcc Diff:

[1/2] incubator-beam git commit: [BEAM-613] Revised SimpleStreamingWordCountTest to better test fixed windows.

Repository: incubator-beam Updated Branches: refs/heads/master 843275210 -> 6082ebcce [BEAM-613] Revised SimpleStreamingWordCountTest to better test fixed windows. Revised the test to test multiple batches Set the timeout to 1 ms since it essentially plays no role here. Removed blank lines

[GitHub] incubator-beam pull request #983: [BEAM-657] Support Read.Bounded primitive

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/983 [BEAM-657] Support Read.Bounded primitive Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/TransformTranslator.java -- diff --git

[1/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

Repository: incubator-beam Updated Branches: refs/heads/master 5c23f4954 -> 1ceb12aeb http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/test/java/org/apache/beam/runners/spark/ClearAggregatorsRule.java

[4/4] incubator-beam git commit: This closes #909

This closes #909 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1ceb12ae Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1ceb12ae Diff:

[3/4] incubator-beam git commit: [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming.

[BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming. Refactor translation mechanism to support checkpointing of DStream. Support basic functionality with GroupByKey and ParDo. Added support for grouping operations. Added checkpointDir option, using it

[2/2] incubator-beam git commit: [BEAM-628] Fixed the Graphite metrics sink configuration for spark-submit on yarn.

[BEAM-628] Fixed the Graphite metrics sink configuration for spark-submit on yarn. This closes #945 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/643cf63d Tree:

[1/2] incubator-beam git commit: Fixed the Graphite metrics sink configuration so it actually works when submitting using spark-submit on yarn.

Repository: incubator-beam Updated Branches: refs/heads/master f81b9a041 -> 643cf63d5 Fixed the Graphite metrics sink configuration so it actually works when submitting using spark-submit on yarn. Made the metrics configuration info a bit clearer. Project:

[2/2] incubator-beam git commit: [BEAM-627] Set Spark master only if not set.

[BEAM-627] Set Spark master only if not set. This closes #944 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f81b9a04 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f81b9a04 Diff:

[1/2] incubator-beam git commit: Set master if not already set.

Repository: incubator-beam Updated Branches: refs/heads/master 49208cadd -> f81b9a041 Set master if not already set. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c9610623 Tree:

[GitHub] incubator-beam pull request #944: [BEAM-627] Set master if not already set.

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/944 [BEAM-627] Set master if not already set. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/2] incubator-beam git commit: Support Verifiers in TestSparkRunner

Repository: incubator-beam Updated Branches: refs/heads/master 82ebfd487 -> 49208cadd Support Verifiers in TestSparkRunner Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0f4ef88b Tree:

[GitHub] incubator-beam pull request #909: [BEAM-610] Enable spark's checkpointing me...

GitHub user amitsela opened a pull request: https://github.com/apache/incubator-beam/pull/909 [BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery in streaming Be sure to do all of the following to help us incorporate your contribution quickly

[2/2] incubator-beam git commit: [BEAM-592] Fix SparkRunner Dependency Problem in WordCount This closes #892

[BEAM-592] Fix SparkRunner Dependency Problem in WordCount This closes #892 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/baf5e416 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/baf5e416

  1   2   >