GitHub user kennknowles opened a pull request:
https://github.com/apache/incubator-beam/pull/356
Exact copy of #343 trying to work around Jenkins issue
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [x] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
From Dan's comments on
[INFRA-11878](https://issues.apache.org/jira/browse/INFRA-11878) sometimes
rebasing works. It doesn't seem to be working for #343, but I am not clear on
the parameters of the issue. Trying this in case a fresh PR gets a fresh
workspace from Jenkins, or some such.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kennknowles/incubator-beam copy-343
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/356.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #356
----
commit fa8bc7cd3bea3dac0484b0af139248e1f39d0317
Author: Kenneth Knowles <[email protected]>
Date: 2016-05-02T20:11:12Z
Add TestFlinkPipelineRunner to FlinkRunnerRegistrar
This makes the runner available for selection by integration tests.
commit 3e76a85ec23da83c5ffc734c80eee882be1c3f38
Author: Kenneth Knowles <[email protected]>
Date: 2016-05-02T21:04:20Z
Configure RunnableOnService tests for Flink in batch mode
Today Flink batch supports only global windows. This is a situation we
intend our build to allow, eventually via JUnit category filtering.
For now all the test classes that use non-global windows are excluded
entirely via maven configuration. In the future, it should be on a
per-test-method basis.
commit 84e10a4b0cbc78949f073737948de00bdb8179f3
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-06T06:26:50Z
Fix Dangling Flink DataSets
commit f0899a682c19231f59f539e9699087ad443734cc
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-06T07:38:55Z
Add hamcrest dependency to Flink Runner
Without it the RunnableOnService tests seem to not work
commit 60f75655b5788b400a81abda32410eaa784f0aad
Author: Kenneth Knowles <[email protected]>
Date: 2016-05-06T17:54:41Z
Remove unused threadCount from integration tests
commit 8ec74ac46c10c6f1fd37d5a6049d9cb03008c432
Author: Kenneth Knowles <[email protected]>
Date: 2016-05-06T17:55:16Z
Disable Flink streaming integration tests for now
commit edef0ce1dcf1405e037c23f997cb1c7980e584f0
Author: Kenneth Knowles <[email protected]>
Date: 2016-05-06T19:49:55Z
Special casing job exec AssertionError in TestFlinkPipelineRunner
commit eadabd58f0ffca76535bbc681f051bf4292f69d9
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-10T11:53:03Z
[BEAM-270] Support Timestamps/Windows in Flink Batch
With this change we always use WindowedValue<T> for the underlying Flink
DataSets instead of just T. This allows us to support windowing as well.
This changes also a lot of other stuff enabled by the above:
- Use WindowedValue throughout
- Add proper translation for Window.into()
- Make side inputs window aware
- Make GroupByKey and Combine transformations window aware, this
includes support for merging windows. GroupByKey is implemented as a
Combine with a concatenating CombineFn, for simplicity
This removes Flink specific transformations for things that are handled
by builtin sources/sinks, among other things this:
- Removes special translation for AvroIO.Read/Write and
TextIO.Read/Write
- Removes special support for Write.Bound, this was not working properly
and is now handled by the Beam machinery that uses DoFns for this
- Removes special translation for binary Co-Group, the code was still
in there but was never used
With this change all RunnableOnService tests run on Flink Batch.
commit 19d4e564dba1a595c164393dc525496485632e28
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-13T12:17:50Z
Fix faulty Flink Flatten when PCollectionList is empty
commit 74daea464d1b4ea08660f82fdbec3c041f4e728c
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-13T12:41:20Z
Remove superfluous Flink Tests, Fix those that stay in
All of the stuff in the removed ITCases is covered (in more detail) by
the RunnableOnService tests.
commit 991b2a4a75e8d5b729be9b012dc7f4ec782a9854
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-14T09:48:47Z
Fix last last outstanding test
commit b5c6b1c8f7408c7f7e174df77e71ca062811800c
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-18T15:43:36Z
Remove Flink-Specific ConsoleIO
this can be done using a Sysout in a ParDo
commit aa179ad92bb04bf7732a6c4675b65533417ecab4
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-18T16:33:07Z
Fix checkstyle warning (to force Jenkins to rerun)
commit 6c4af76fcc70a8f36acf3645ef7e0fd85d04fefe
Author: Aljoscha Krettek <[email protected]>
Date: 2016-05-19T12:05:05Z
Add surefire plugin to java 8 example tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
