GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1654
[BEAM-1177] Input DStream "bundles" should be in serialized form and
include relevant metadata.
Be sure to do all of the following to help us incorporate your contribution
Github user amitsela closed the pull request at:
https://github.com/apache/incubator-beam/pull/1500
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Repository: incubator-beam
Updated Branches:
refs/heads/master d624d3b6b -> 5ebbd500c
[BEAM-362] Port runners to runners-core AggregatoryFactory
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/55f04955
This closes #1644
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5ebbd500
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5ebbd500
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 1ad638e51 -> 4323247a3
[BEAM-932] Enable findbugs validation (and fix existing issues)
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1463
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4323247a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4323247a
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1614
[BEAM-853] Force streaming execution on batch pipelines for testing.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1579
[BEAM-1130, BEAM-1133] Allow users to define max records per batch, update
tests accordingly.
Be sure to do all of the following to help us incorporate your contribution
quickly
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1578
[BEAM-757, BEAM-807]
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like
Repository: incubator-beam
Updated Branches:
refs/heads/master e841b1a21 -> bf8a3cb3a
[BEAM-921] spark-runner: register sources and coders to serialize with java
serializer
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1552
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bf8a3cb3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bf8a3cb3
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam-site/pull/106
[BEAM-900] Spark quickstart instructions.
R: @davorbonaci
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amitsela/incubator-beam
This closes #1561
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d9a63117
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d9a63117
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 63d197cd0 -> d9a631173
Remove misc uses of OldDoFn
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e48b0e6b
Tree:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1553
[BEAM-] Reject timers for ParDo in SparkRunner streaming evaluators.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam-site/pull/103
[BEAM-507] Fill in the documentation/runners/spark portion of the webâ¦
â¦site.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1532
[BEAM-329] Spark runner README should have a proper batch example.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make
This closes #1531
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/9ccf6dbe
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/9ccf6dbe
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master b41a46e86 -> 9ccf6dbea
[BEAM-1094] Set test scope for Kafka IO and junit
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/baf5e6bd
Tree:
This closes #1466
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6893a727
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6893a727
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master ef9871c36 -> 6893a7270
[BEAM-1000, BEAM-1050] Fixed PipelineResult.State Failed for streaming, support
non-blocking
cancel/waituntilfinish in batch.
Added a SparkPipelineResult class to address PipelineResult#waitUntilFinish()
Redistributed some responsibilities in order to remove getAggregatorValues()
form EvaluationContext.
Inferred excepted exception handling according to existing codebase and tests.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1500
[BEAM-1074] Set default-partitioner in SourceRDD.Unbounded.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure
This closes #1450
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3a8b9b52
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3a8b9b52
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 3f16f2660 -> 3a8b9b521
[BEAM-1052] Add InputDStream id to MicrobatchSource hashcode.
Done to avoid collisions between splits of different sources.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Repository: incubator-beam
Updated Branches:
refs/heads/master 3ad767750 -> 8cc43aa70
[BEAM-851] Determine if the pipeline must be translated into streaming mode (if
not set)
Now an Evaluator (visitor) detects if there are Unbonded.Read transforms.
This approach is based on Flink's
This closes #1436
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8cc43aa7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8cc43aa7
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 632576b5b -> 3ad767750
Remove unused WindowingInternals.writePCollectionViewData
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/803bbe2a
[BEAM-498] Remove obsolete WindowingInternals#writePCollectionViewData
This closes #1430
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3ad76775
Tree:
Remove unused body of StreamingPCollectionViewWriterFn
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/07544ef3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/07544ef3
Diff:
This closes #1393
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/875631f0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/875631f0
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master d93e9a88b -> 875631f07
[Beam-1001] Add non blocking cancel() and waitUntilFinish() for streaming
applications.
remove timeout parameer in spark pipeline option.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Repository: incubator-beam
Updated Branches:
refs/heads/master 201110222 -> dbbd5e448
[BEAM-983] Fix a bunch of precommit errors from #1332
Renames TestPipelineOptions to SparkTestPipelineOptions
To avoid confusion with sdk.testing.TestPipelineOptions.
Also, a couple of other minor fixes.
This closes #1364
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/dbbd5e44
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/dbbd5e44
Diff:
Github user amitsela closed the pull request at:
https://github.com/apache/incubator-beam/pull/1362
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1362
[BEAM-983] add missing license.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like
Repository: incubator-beam
Updated Branches:
refs/heads/master 47646d641 -> 503f26f44
[BEAM-891] fix build occasionally fails on IndexOutOfBoundsException.
Moved "TestPipelineOptions#withTmpCheckpointDir" to
TestPipelineOptionsForStreaming.
Removed an unused member in
This closes #1332
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/503f26f4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/503f26f4
Diff:
This closes #1291
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2bc66f90
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2bc66f90
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 9c300cde8 -> 2bc66f903
[BEAM-762] Unify spark-runner EvaluationContext and StreamingEvaluationContext
PR 1291 review changes.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
Repository: incubator-beam
Updated Branches:
refs/heads/master cd3f61cf8 -> e43a38355
[BEAM-944] Spark runner causes an exception when creating pipeline options.
Create a SparkContextOptions for context-ware options.
Move UsesProvidedSparkContext property to SparkPipelineOptions so it's
This closes #1316
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/e43a3835
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/e43a3835
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1316
[BEAM-944] Spark runner causes an exception when creating pipeline opâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
Github user amitsela closed the pull request at:
https://github.com/apache/incubator-beam/pull/1257
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Repository: incubator-beam
Updated Branches:
refs/heads/master 46fbfe06b -> 14e093a0a
[BEAM-889] Let Spark handle the user-provided checkpointDir, but warn if not a
reliable fs.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1272
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/14e093a0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/14e093a0
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam-site/pull/65
[BEAM-890] Update compatibility matrix for Spark.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amitsela/incubator-beam-site
Github user amitsela closed the pull request at:
https://github.com/apache/incubator-beam/pull/1271
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1272
[BEAM-889] CheckpointDir option does not accept relative path and reqâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1271
[BEAM-889] CheckpointDir option does not accept relative path and reqâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
Repository: incubator-beam
Updated Branches:
refs/heads/master d75d8b2bb -> 6a05cf4a9
Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in
ROS."
This reverts commit ff5409f15a9f741f437c489b7de763cfa3c68278.
Project:
This closes #1270
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6a05cf4a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6a05cf4a
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1270
Revert "[BEAM-808] Increase "spark.port.maxRetries" to avoid BindExceâ¦
Be sure to do all of the following to help us incorporate your contribution
qui
Repository: incubator-beam
Updated Branches:
refs/heads/master 2c0d0f476 -> 529f266ae
[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException in ROS.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1170
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/529f266a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/529f266a
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1257
[BEAM-880] Avoid emitting default empty iterable in PAssert.GroupedGlâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
Repository: incubator-beam
Updated Branches:
refs/heads/master 215980ad3 -> 9c3e3e7a3
Directly implement ReifyTimestampsAndWindows in SparkRunner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/597e3955
Repository: incubator-beam
Updated Branches:
refs/heads/master 53fe3ee42 -> 78e2c0387
[BEAM-809] Create a KryoRegistrator for the SparkRunner.
Use Class#getName() instead of canonicalName().
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1162
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/53fe3ee4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/53fe3ee4
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master f2fe1ae46 -> 53fe3ee42
[BEAM-799] Support GroupByKey directly.
Remove runner override for GroupByKey.
Avoid NPE if no sideInputs are available in reader.
Handle CombineFn with or without context.
Project:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1171
[BEAM-809] Create a KryoRegistrator for the SparkRunner.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1170
[BEAM-808] Increase "spark.port.maxRetries" to avoid BindException inâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1162
[BEAM-799] Support GroupByKey directly
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1161
[BEAM-769] Spark streaming tests fail on "nothing processed" if runtiâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly
This closes #1153
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a9a41eb9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a9a41eb9
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 4c9058236 -> a9a41eb94
[BEAM-794] Differ combining in case of merging windows with sideInputs.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1153
[BEAM-794] Differ combining in case of merging windows with sideInputs.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1144
[BEAM-781] Remove Spark's batch unit tests and rely on ROS tests instâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1143
[BEAM-658] Support Read.Unbounded primitive.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
[BEAM-259] Execute selected RunnableOnService tests with Spark runner.
Handle empty Flatten for bounded.
Spark will bubble out a SparkException for user code failure, so this won't
catch. Asserting on the
error message should be good enough.
outputWithTimestamp should handle start/finishBundle
Repository: incubator-beam
Updated Branches:
refs/heads/master b0cb2e87b -> c472e1227
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/7eecd7ee/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkProcessContext.java
This closes #1055
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c472e122
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c472e122
Diff:
This closes #1125
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b0cb2e87
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b0cb2e87
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master ea04e618e -> b0cb2e87b
[BEAM-744] UnboundedKafkaReader should return as soon as it can.
Use timeout directly in nextBatch()
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
This closes #1133
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/ea04e618
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/ea04e618
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master dde8e35ca -> ea04e618e
[BEAM-777] KafkaIO Test should handle reader.start() better.
KafkaIOTest : start() can return false
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1125
[BEAM-744] A runner should be able to override KafkaIO max wait propeâ¦
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
This closes #1073
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/49f94443
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/49f94443
Diff:
This closes #1072
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d790dfe1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d790dfe1
Diff:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1073
[BEAM-735] PAssertStreaming should make sure the assertion happened.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1072
[BEAM-734] Add StreamingListeners via SparkPipelineOptions.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1055
[BEAM-259] Enable RunnableOnService for batch.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/1040
[BEAM-703] SingletonViewFn might exhaust defaultValue if it's serialized
after being used.
Be sure to do all of the following to help us incorporate your contribution
quickly
Repository: incubator-beam
Updated Branches:
refs/heads/master 4872bde8f -> a00d2f810
[BEAM-657] Support Read.Bounded primitive.
Support Read.Bounded primitive.
Avro requires this for snappy.
Create is supported by Read.Bounded now.
Read.Bounded support should solve gs issues now.
remove
This closes #983
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/a00d2f81
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/a00d2f81
Diff:
This closes #982
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/6082ebcc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/6082ebcc
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 843275210 -> 6082ebcce
[BEAM-613] Revised SimpleStreamingWordCountTest to better test fixed windows.
Revised the test to test multiple batches
Set the timeout to 1 ms since it essentially plays no role here.
Removed blank lines
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/983
[BEAM-657] Support Read.Bounded primitive
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/TransformTranslator.java
--
diff --git
Repository: incubator-beam
Updated Branches:
refs/heads/master 5c23f4954 -> 1ceb12aeb
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/0feb6499/runners/spark/src/test/java/org/apache/beam/runners/spark/ClearAggregatorsRule.java
This closes #909
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1ceb12ae
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/1ceb12ae
Diff:
[BEAM-610] Enable spark's checkpointing mechanism for driver-failure recovery
in streaming.
Refactor translation mechanism to support checkpointing of DStream.
Support basic functionality with GroupByKey and ParDo.
Added support for grouping operations.
Added checkpointDir option, using it
[BEAM-628] Fixed the Graphite metrics sink configuration for spark-submit on
yarn.
This closes #945
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/643cf63d
Tree:
Repository: incubator-beam
Updated Branches:
refs/heads/master f81b9a041 -> 643cf63d5
Fixed the Graphite metrics sink configuration so it actually works when
submitting using spark-submit on yarn.
Made the metrics configuration info a bit clearer.
Project:
[BEAM-627] Set Spark master only if not set.
This closes #944
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f81b9a04
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f81b9a04
Diff:
Repository: incubator-beam
Updated Branches:
refs/heads/master 49208cadd -> f81b9a041
Set master if not already set.
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c9610623
Tree:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/944
[BEAM-627] Set master if not already set.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title
Repository: incubator-beam
Updated Branches:
refs/heads/master 82ebfd487 -> 49208cadd
Support Verifiers in TestSparkRunner
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0f4ef88b
Tree:
GitHub user amitsela opened a pull request:
https://github.com/apache/incubator-beam/pull/909
[BEAM-610] Enable spark's checkpointing mechanism for driver-failure
recovery in streaming
Be sure to do all of the following to help us incorporate your contribution
quickly
[BEAM-592] Fix SparkRunner Dependency Problem in WordCount
This closes #892
Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/baf5e416
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/baf5e416
1 - 100 of 177 matches
Mail list logo