[
https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397251#comment-15397251
]
ASF GitHub Bot commented on BEAM-79:
------------------------------------
GitHub user manuzhang opened a pull request:
https://github.com/apache/incubator-beam/pull/750
[BEAM-79] Merge branch 'master' into gearpump_runner
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manuzhang/incubator-beam gearpump_runner_sync
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/750.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #750
----
commit 34467f92d5a31b47f95b734c737fde0a8277311b
Author: Thomas Groh <[email protected]>
Date: 2016-07-15T17:51:24Z
Make TransformEvaluatorFactory reuse Explicit
Transform Evaluator Factories must be reused for the entire execution of
a Pipeline and must not be reused across pipelines.
Remove EvaluatorKey, and key explicitly by the transform application.
commit ad6ee01825740ee47f25ca036fa5f352375bbe6e
Author: Dan Halperin <[email protected]>
Date: 2016-07-19T06:40:13Z
Checkstyle: enforce package-info.java presence
Ignores tests and examples
commit c4ad11832235eef0b73d299cd02d1a224c130ece
Author: Dan Halperin <[email protected]>
Date: 2016-07-20T16:16:28Z
Closes #692
commit 6d7efe3df8cde590722d94441eca3922a3a67734
Author: Kenneth Knowles <[email protected]>
Date: 2016-07-20T17:30:55Z
This closes #666
commit 5f19e4caf207ff02d226fec2933be2f37ca66b4d
Author: Thomas Groh <[email protected]>
Date: 2016-06-30T17:06:52Z
Add withAllowedLateness with Closing Behavior to Window
This makes the static constructors for withAllowedLateness symmetric to
the PTransform builder methods. It also allows references to
Window#withAllowedLateness(Duration, ClosingBehavior).
commit f547f70e1c9535ff663d124f67c72c7ec2c55e9e
Author: Kenneth Knowles <[email protected]>
Date: 2016-07-20T17:40:13Z
This closes #567
commit 00195d2543eb347cc3669a4ac89e98da0bc4dca4
Author: Thomas Groh <[email protected]>
Date: 2016-06-28T22:44:49Z
Use the ParDo Application to Cache DoFns
A DoFn application is the scope of reuse.
Factor CloningThreadLocal as the top-level class instead of
SerializableCloningThreadLocalCacheLoader, and extract the Fn from the
AppliedPTransform when loading an absent element.
commit 436e4a34ebb222545cb03cb6d39ea4ca2d905254
Author: Kenneth Knowles <[email protected]>
Date: 2016-07-20T17:55:53Z
This closes #554
commit b240525affb205a83054577233f3a4a508fe1c72
Author: Dan Halperin <[email protected]>
Date: 2016-07-18T19:05:02Z
BigQueryIO: move to google-cloud-platform module
* Move package from io to io.gcp.bigquery
* Move from SDK core into GCP-IO module
* Fixup references and import orders
* Separate AvroUtils into generic AvroUtils and BigQueryAvroUtils
* Rewrite a unit test in sdk core to not depend on BigQueryIO
* Fixup Javadoc in SDK core that need not depend on BigQueryIO
* Make utility classes package-private
commit 7ec8781a2e18548a23c882329f0b50f7254202ec
Author: Dan Halperin <[email protected]>
Date: 2016-07-20T20:02:43Z
Closes #681
commit bdb65278873a5010a625dc6a569ba25b17374c06
Author: Kenneth Knowles <[email protected]>
Date: 2016-07-20T04:36:10Z
Add os-maven-plugin to Spark runner
commit 84332ee9716233af928e85c14c534714ab828531
Author: Chandni Singh <[email protected]>
Date: 2016-07-20T00:30:16Z
BEAM-372 verfify if a nested coder consumes bytes equal to encoded bytes
commit 6d5e8186a2da532eb1c29097bc1259a19d9f72c9
Author: Luke Cwik <[email protected]>
Date: 2016-07-21T13:34:52Z
[BEAM-372] added a test that verifies if a coder consumes bytes equal to
encoded bytes
This closes #695
commit ae2144196c351cc2ee544e030d793929d8607696
Author: Dan Halperin <[email protected]>
Date: 2016-07-20T17:26:02Z
BigtableIO: upgrade to 0.9.1
* Use the uber jar
* Remove OS classifier mumbo jumbo
* Move common dependency versioning to root pom
commit 0eb81fdc15bc3dce28e24e73427bd389e0f6d60d
Author: Luke Cwik <[email protected]>
Date: 2016-07-21T15:26:41Z
[BEAM-445] BigtableIO: upgrade to 0.9.1
This closes #701
commit 232c0e9610a5d5559598a2059577b699299a0b76
Author: Thomas Groh <[email protected]>
Date: 2016-07-20T16:49:47Z
Rename DirectRegistrar Registrars
This removes the duplication of "DirectRunner" and "DirectOptions"
classes.
commit 9cdaed0d8a401caf5a9efb626fa629fd6f4f4627
Author: Dan Halperin <[email protected]>
Date: 2016-07-21T17:24:54Z
Closes #700
commit 7337ecf147ca69262978db116a5a1f739aca28ae
Author: Jean-Baptiste Onofré <[email protected]>
Date: 2016-05-31T16:04:52Z
[BEAM-316] Add file scheme support in TextIO
commit 38061d3531a1ca13ef825a6ee50872e420ab1494
Author: Dan Halperin <[email protected]>
Date: 2016-07-18T22:35:06Z
[BEAM-316] Add file URI handling
* Register FileIOChannelFactory for file scheme
* Modify FileIOChannelFactory to dynamically remove the file:// scheme
string.
commit adb472083460bc175a0685133eadb6a336f1a4a2
Author: Dan Halperin <[email protected]>
Date: 2016-07-22T03:17:54Z
Closes #684 and #402
commit 060bc802f0bff8ca1a076e9f93a623058e1d651b
Author: Kenneth Knowles <[email protected]>
Date: 2016-07-22T04:01:36Z
Execute RunnableOnService tests only when runner options provided
Previously, the situation was this:
- All runners inherit a RunnableOnService integration-test
execution referencing runnableOnServicePipelineOptions
whether or not the variable was set. Basically an unbound
variable reference.
- The Dataflow runner had a profile disabling it if
runnableOnServicePipelineOptions was not set.
- Before they got configured, Flink and Spark had to
do extra work to explicitly prevent the invalid
configuration from being used.
After this change:
- All runners inherit the same integration-test execution
but only if the variable it requires is present.
- Dataflow doesn't have any special profile.
- Flink and Spark are unchanged, since they do set
up the variable themselves. When they move to running
only as postcommit, like Dataflow does, the hardcoding
is expected to either move to a profile or move to
the Jenkins invocation.
commit 07a313f1af78349040afd9ff6b01e60cdcd567a3
Author: Dan Halperin <[email protected]>
Date: 2016-07-22T07:39:59Z
Closes #709
commit 75df931a55dbd3f93d9143574cdad7291d3457aa
Author: Jesse Anderson <[email protected]>
Date: 2016-07-22T20:54:26Z
Added BigInteger to TypeDescriptors class.
commit 12b60ffa5d18812674299a97e84af31d063e170c
Author: Dan Halperin <[email protected]>
Date: 2016-07-22T23:51:37Z
Closes #716
commit bf3af5de9e6f8126ffd2ccd3f7a68a84e55e90ff
Author: Pei He <[email protected]>
Date: 2016-07-22T22:03:38Z
[BEAM-480] Move insertAll() from BigQueryTableInserter to BigQueryServices
commit 122cd04663529f4fc44530ed9e144e0dfd68360f
Author: Dan Halperin <[email protected]>
Date: 2016-07-22T23:52:50Z
Closes #717
commit 12fb77811d7bf8c72a3c0a48f391bab3c815fbef
Author: Sela <[email protected]>
Date: 2016-06-27T17:59:07Z
Remove dependency on beam-examples-java.
Duplicate WordCount into spark examlpes package.
Duplicate parts of TfIdf from beam examlpes.
Better reuse of WordCount and its parts.
Remove dependency on beam-examples-java
commit d3dc368666d5b700c5026fe08455e1210692ce3e
Author: Sela <[email protected]>
Date: 2016-07-04T12:33:57Z
Remove JavaDoc links to beam-examples.
commit cf1464465e1e65be2e4896cc6f693f8bbe82b844
Author: Dan Halperin <[email protected]>
Date: 2016-07-23T02:51:50Z
Closes #539
commit 2fe387707d1e115b578f5ee643bb99c0e4667ee0
Author: Maximilian Michels <[email protected]>
Date: 2016-07-20T14:06:06Z
[flink] improve example section in README
- updates the README
- repairs broken exec configuration
----
> Gearpump runner
> ---------------
>
> Key: BEAM-79
> URL: https://issues.apache.org/jira/browse/BEAM-79
> Project: Beam
> Issue Type: New Feature
> Components: runner-ideas
> Reporter: Tyler Akidau
> Assignee: Manu Zhang
>
> Intel is submitting Gearpump (http://www.gearpump.io) to ASF
> (https://wiki.apache.org/incubator/GearpumpProposal). Appears to be a mix of
> low-level primitives a la MillWheel, with some higher level primitives like
> non-merging windowing mixed in. Seems like it would make a nice Beam runner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
