[GitHub] incubator-beam pull request #389: Test case for package-private PipelineOpti...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/389 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #439: [BEAM-330] Disable exec-maven-plugin clean...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/439 [BEAM-330] Disable exec-maven-plugin cleanupDaemonThreads Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- exec-maven-plugin has a bad interaction with DataflowPipelineRunner when waiting on daemon threads. When running example pipelines, Maven will emit a warning saying background threads are alive after being aborted. This looks to users like an error while there's actually no harm to it. Disabling cleanupDaemonThreads suppresses this warning. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam exec-cleanupDaemonThreads Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/439.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #439 commit 10154c6f0978cf6f33ae7153e85a02607b6f65ed Author: Scott Wegner <sweg...@google.com> Date: 2016-06-09T15:46:37Z Disable exec-maven-plugin cleanupDaemonThreads exec-maven-plugin has a bad interaction with DataflowPipelineRunner when waiting on daemon threads. When running example pipelines, Maven will emit a warning saying background threads are alive after being aborted. This looks to users like an error while there's actually no harm to it. Disabling cleanupDaemonThreads suppresses this warning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #445: Update AutoComplete example to use WriteDi...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/445 Update AutoComplete example to use WriteDisposition.APPEND Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam autocomplete-bigqueryio Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/445.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #445 commit 1e67644169249e3bed034bb09086d53dadf54d02 Author: Scott Wegner <sweg...@google.com> Date: 2016-06-10T20:52:59Z Update AutoComplete example to use WriteDisposition.APPEND --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #444: [BEAM-330] Port cleanupDaemonThreads fix t...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/444 [BEAM-330] Port cleanupDaemonThreads fix to archetype module Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam mvnexec-archetypes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/444.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #444 commit 5928be8d243cbc6ca77dc2454aa37dc986b71103 Author: Scott Wegner <sweg...@google.com> Date: 2016-06-10T19:52:43Z Port cleanupDaemonThreads fix to archetype module --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #441: [BEAM-331] Make AddTimestampFn range deter...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/441 [BEAM-331] Make AddTimestampFn range deterministic Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- The timestamps added in the WindowedWordCount example are currently based on when the bundles are executed, which makes the min/max bounds non-deterministic. This change makes the range based on the construction time. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam wwc-addtimestampfn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/441.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #441 commit 6b1d3a7d5a4e2bb9ecb010102a7d0506cda15986 Author: Scott Wegner <sweg...@google.com> Date: 2016-06-09T18:31:23Z Make AddTimestampFn range deterministic The timestamps added in the WindowedWordCount example are currently based on when the bundles are executed, which makes the min/max bounds non-deterministic. This change makes the range based on the construction time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Test case for package-private Pipelin...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/389 Test case for package-private PipelineOptions Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam pipelineoptions-interfaces Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/389.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #389 commit ebce2b71652a3ccff060abedfbf691b4c11a7836 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-25T16:55:51Z Test case for package-private PipelineOptions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-308] Mark test PipelineOptions ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/390 [BEAM-308] Mark test PipelineOptions interface public Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam pipelineoptions-public Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/390.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #390 commit 7abedfae13319c8e85bfc6a7f3fef4c7edeae9dd Author: Scott Wegner <sweg...@google.com> Date: 2016-05-25T18:40:45Z Mark test PipelineOptions interface public --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #451: Improve BigQueryIO validation for streamin...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/451 Improve BigQueryIO validation for streaming WriteDisposition Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam bigqueryio-streaming Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/451.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #451 commit 0a3ba4eccc45f8db89cb1973fe74f0f93da68a3e Author: Scott Wegner <sweg...@google.com> Date: 2016-06-13T18:05:52Z Improve BigQueryIO validation for streaming WriteDisposition --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #470: Package javadoc for org.apache.beam.sdk.tr...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/470 Package javadoc for org.apache.beam.sdk.transforms.display Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-pkginfo Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/470.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #470 commit 2e0836e730c52c0ef7137f67495f43bb4ec2b06a Author: Scott Wegner <sweg...@google.com> Date: 2016-06-15T17:17:01Z Package javadoc for org.apache.beam.sdk.transforms.display --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-81] --update PipelineOption is ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/5 [BEAM-81] --update PipelineOption is in the wrong interface You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam b23122757 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/5.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5 commit f7889e1f9c0270442812e50f6f4e638d9977cd99 Author: Scott Wegner <sweg...@google.com> Date: 2016-02-29T23:24:49Z Move 'update' pipeline option to correct interface --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: DisplayData tweaks based on transform...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/83 DisplayData tweaks based on transform usage [BEAM-121] Publish DisplayData from common PTransforms * Add boolean-valued display data. * Implement equality for DislpayData.Item * Add ability to override namespace for included subcomponents. * Additional Matchers for testing display data * Update DisplayData inner class privacy You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-tweaks Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/83.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #83 commit a52c1b6cdc033f64f526cff9cf0a2c4d8f23aa14 Author: Scott Wegner <sweg...@google.com> Date: 2016-03-28T20:38:35Z DisplayData tweaks based on transform usage. * Add boolean-valued display data. * Implement equality for DislpayData.Item * Add ability to override namespace for included subcomponents. * Additional Matchers for testing display data * Update DisplayData inner class privacy --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-121] Add display data to window...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/124 [BEAM-121] Add display data to windowing transforms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace "" in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-window Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/124.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #124 commit 49c87478efa7bcaa153a61547bfd6d2b923c8f7e Author: Scott Wegner <sweg...@google.com> Date: 2016-03-28T20:32:27Z Add display data to windowing transforms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-121] Add display data to ParDo ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/125 [BEAM-121] Add display data to ParDo transforms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace "" in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-pardo Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/125.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #125 commit 2f8a2358a17705fc2d197b576423266d63517f1c Author: Scott Wegner <sweg...@google.com> Date: 2016-03-28T20:30:24Z Add display data to ParDo transforms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-173] Publish DisplayData for Pi...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/127 [BEAM-173] Publish DisplayData for PipelineOptions Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace "" in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-pipeline Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/127.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #127 commit 807319596e9a22d244bbbece9cbe220ffa1019b2 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-05T18:42:30Z Move additional matchers into DisplayDataMatchers commit 1c05ccdf081ec8f6a5dac214326c4038c15d8335 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-05T18:46:23Z Support overriding namespace when adding display data commit 1729a1c74bd97922466e5c9325be8b4be8ba1159 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-05T18:48:36Z Support inferring display data type during registration commit a22ec5fa280becc2e6f12c8a81f30f794173b7bf Author: Scott Wegner <sweg...@google.com> Date: 2016-04-05T18:49:26Z Refactor PipelineOptionsFactory property reflection logic into PipelineOptionsReflector. commit a6f065062f33866270d18bded4b0ef87c7cdb1e4 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-05T18:50:13Z Automatically collect display data from PipelineOptions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-119] Integrate DisplayData into...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/65 [BEAM-119] Integrate DisplayData into DataflowPipelineRunner Translate registered display data into the job steps used by DataflowPipelineRunner. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/65.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #65 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Fix malformatted @links in javadoc co...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/21 Fix malformatted @links in javadoc comments You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam fix-javadoc-links Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/21.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21 commit 8634281470314842a6431e0c3499aaef973ed9ec Author: Scott Wegner <sweg...@google.com> Date: 2016-03-04T17:45:59Z Fix malformatted @links in javadoc comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Fix logical error in Sample transform...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/80 Fix logical error in Sample transform javadoc If the `limit` passed into the `Sample` transform is *greater than or equal to* the size of the input, then all the input's elements will be selected. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam sample-docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/80.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #80 commit 4d86379a0ff02de10eda8272710c08d73fcc647d Author: Scott Wegner <sweg...@google.com> Date: 2016-03-28T16:49:54Z Fix logical error in Sample transform javadoc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-121] Add DisplayData for compos...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/236 [BEAM-121] Add DisplayData for composite IO transforms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-io-composites Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/236.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #236 commit 82cfc55be9f8ea11c9107f7b5b88ebccd70e945b Author: Scott Wegner <sweg...@google.com> Date: 2016-04-21T20:36:42Z Register DisplayData from IO primatives commit b43fa0c714e96c8e132779377c03e9518e0555de Author: Scott Wegner <sweg...@google.com> Date: 2016-04-22T21:45:40Z fixup! Register DisplayData from IO primatives commit 31c61ddb580b636e93d8a276758a89bf20daaff7 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-21T21:04:36Z Register DisplayData from composite IO transforms --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Move ClassForDisplayJava8Test to corr...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/238 Move ClassForDisplayJava8Test to correct package Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam java8test-pkg Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/238.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #238 commit 0de96df8d9069f06729ca28fe284660d0fc9b117 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-25T18:59:25Z Move ClassForDisplayJava8Test to correct package --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-199] Update DisplayData APIs to...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/247 [BEAM-199] Update DisplayData APIs to make overrides more understandable Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-api Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/247.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #247 commit 6a63fc6dd536be49971dc4987cc0d0ab3f887829 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-25T23:04:38Z Change API to register DisplayData items to make overrides more understandable --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Minor fixes
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/244 Minor fixes Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam minor-fixes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/244.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #244 commit dccada7b6d3a948518339af71e3e8fc4819584a8 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-26T21:07:10Z Reference correct Lists import commit 236bce7c536692580d93bcc6e45a6dc37f4f9b0b Author: Scott Wegner <sweg...@google.com> Date: 2016-04-26T21:07:23Z Fix multi-line string format --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-231] Remove ClassForDisplay hel...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/259 [BEAM-231] Remove ClassForDisplay helper type Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam classfordisplay Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/259.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #259 commit d1513d512d7a82208c6cfb361996a21235348662 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-27T20:08:49Z Refactor Combine display data to not use ClassForDisplay commit 39c8e5fd3aea8f03884b38301520f3b5dba1f72b Author: Scott Wegner <sweg...@google.com> Date: 2016-04-27T20:15:33Z Refactor DoFnReflector.SimpleDoFnAdapter to not use display data namespace override commit d6f2a453803f25a8670d76ef46bfb39956f3a82b Author: Scott Wegner <sweg...@google.com> Date: 2016-04-27T20:25:12Z Remove ClassForDisplay helper type commit 16e8d5a2b1901f0342e0bcc1b4d8736a0c8e337b Author: Scott Wegner <sweg...@google.com> Date: 2016-04-27T22:00:03Z Add test case for display data multi-level namespace overrides --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-117] Minor updates for DisplayD...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/241 [BEAM-117] Minor updates for DisplayData usage Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- This PR addresses the following JIRA issues: * [[BEAM-195] Rename DisplayDataMatchers#includes](https://issues.apache.org/jira/browse/BEAM-195) * [[BEAM-200] Standardize terminology to "display data" in documentation](https://issues.apache.org/jira/browse/BEAM-200) * [[BEAM-219] Use super.populateDisplayData consistently](https://issues.apache.org/jira/browse/BEAM-219) [[BEAM-199] Improve fluent interface for manipulating DisplayData.Items](https://issues.apache.org/jira/browse/BEAM-199) will be in the next PR as to not bloat this one too much. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-tweaks Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/241.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #241 commit 08b475f1971b3f74c9b321941cdbe2818c25c939 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-25T16:31:13Z Rename DisplayDataMatchers.includes to includesDisplayData to clarify usage commit 5a1d93ce01000df8577acd66d6e9c6ebf5d6948f Author: Scott Wegner <sweg...@google.com> Date: 2016-04-25T16:34:58Z Normalize comments to refer to 'display data' rather than 'display metadata' commit 954e734fbde4a055ee97d84b9f1c91d598fc5de9 Author: Scott Wegner <sweg...@google.com> Date: 2016-04-25T16:41:07Z Add super.populateDisplayData() to standard implementations. The current super implementation is a no-op, but this is the recommended way to implement the pattern --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-176] Add FindBugs analysis to t...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/357 [BEAM-176] Add FindBugs analysis to the build Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- FindBugs is a static analysis tool which looks for common coding issues in Java code. See: http://findbugs.sourceforge.net/ This PR integrates FindBugs into the build process using findbugs-maven-plugin, and subscribes java-sdk-all for analysis. The codebase currently generates many Findbugs issues, which have been added in the findbugs-filter.xml file as a baseline. These issues should be triaged and fixed over time. There will be some cases where we choose to ignore issues reported by FindBugs. In such a case, the issue should be suppressed in code via the `@SuppressFBWarnings` annotation with a comment describing why it is suppressed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam findbugs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #357 commit 10bd439e0650c183bb667366399444ef0998bd04 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-16T22:44:45Z Add FindBugs analysis to the build FindBugs is a static analysis tool which looks for common coding issues in Java code. See: http://findbugs.sourceforge.net/ This commit integrates FindBugs into the build process using findbugs-maven-plugin, and subscribes java-sdk-all for analysis. The codebase currently generates many Findbugs issues, which have been added in the findbugs-filter.xml file as a baseline. These issues should be triaged and fixed over time. There will be some cases where we choose to ignore issues reported by FindBugs. In such a case, the issue should be suppressed in code via the @SuppressFBWarnings annotation with a comment describing why it is suppressed. commit f8706b536788ebbe415648a7658a9bd599ada7ad Author: Scott Wegner <sweg...@google.com> Date: 2016-05-16T23:39:39Z Disable not useful DM_STRING_CTOR warnings These seem to pop up for normal string concatenation. commit 195bd8e435eeade822e5d9a0798df633c23a2149 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-19T18:47:02Z Fix AvroCoder log message null reference. commit feaaac1492977f06bcc2dc5980ca7a3f63dacffe Author: Scott Wegner <sweg...@google.com> Date: 2016-05-19T19:53:23Z Suppress FindBugs warning for checked null dereference commit 94b79eb8a11e107ed3a78873bc050a9d33351913 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-19T19:58:34Z Disable not nuseful SE_NO_SERIALVERSIONID warnings --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-117] Fix bug in PipelineOptions...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/355 [BEAM-117] Fix bug in PipelineOptions DisplayData serialization Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- In the code for populating display data for `PipelineOptions` in the serialized JSON, we had a bug in `ProxyInvocationHandler.Serializer` where it was not checking for null values from previously-deserialized JSON values. As a result, a null value in the JSON would cause us to throw and block pipelines from running. This PR fixes a number of issues: * Properly handle null JSON values in `ProxyInvocatinHandler.Serializer` * Establish a common pattern for handling errors during display data collection, via `DisplayData.from(Throwable)` * Implement resiliency pattern in `ProxyInvocationHandler.Serializer` so bugs in `PipelineOptions` serialization do not prevent running pipelines. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-resiliency Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/355.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #355 commit 8ffeb3da894ee5e25a54f7faa8faeff8fe1ae98f Author: Scott Wegner <sweg...@google.com> Date: 2016-05-19T16:17:37Z Fix bug in PipelineOptions DisplayData serialization --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-117] Runners should be resilien...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/315 [BEAM-117] Runners should be resilient to DisplayData failure Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Display data is collected from PTransforms at Pipeline construction time. Collecting display data runs user code from provided transforms and fn's. These components should be designed not to throw during pipeline construction, however we also shouldn't fail a pipeline if this code does fail. This PR adds resiliency to the DataflowPipelineTranslator, where we collect display data for the Dataflow runner, and also a RunnableOnService test to verify that all runners are resilient to display data failures. Other runners are not yet using display data, but will get this validation for free when they do. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-safe Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/315.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #315 commit d6c5025937cbd0016d0a83d08e53902ae4d4519b Author: Scott Wegner <sweg...@google.com> Date: 2016-05-10T18:19:14Z Runners should be resilient to DisplayData failure Display data is collected from PTransforms at Pipeline construction time. Collecting display data runs user code from provided transforms and fn's. These components should be designed not to throw during pipeline construction, however we also shouldn't fail a pipeline if this code does fail. This PR adds resiliency to the DataflowPipelineTranslator, where we collect display data for the Dataflow runner, and also a RunnableOnService test to verify that all runners are resilient to display data failures. Other runners are not yet using display data, but will get this validation for free when they do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-117] Window display data should...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/330 [BEAM-117] Window display data should be safe for null windowFn Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-windowfn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/330.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #330 commit 37d06b9108ba589f7f7ad51abd223b16c3ddf64b Author: Scott Wegner <sweg...@google.com> Date: 2016-05-12T18:32:30Z Window display data should be safe for null windowFn --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Add labels for SDK display data
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/338 Add labels for SDK display data Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-labels Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/338.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #338 commit 998ba7ba369a34654b9e93f662f7b1d9b7d83a58 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-11T21:17:10Z Add labels for SDK display data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: [BEAM-240] Display data links for inp...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/300 [BEAM-240] Display data links for input and output files Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- With display data, SDK authors have the ability to annotate display items with a link URLs. This adds browse URLs for GCS and local files, and attach them to well-known source/sink display data. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-urls Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/300.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #300 commit c95a61a0df01f93ebeeb897249535f160afdf9eb Author: Scott Wegner <sweg...@google.com> Date: 2016-05-03T23:18:27Z Add browse URL to GcsPath commit 6ff5bae0e5bc1cdc2d10ef9e65b44039c8363407 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-03T23:32:21Z Add browse URL to File IO commit ca40e25c87ddde4693b17169e40c00c0b62be170 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-04T16:38:18Z Add DisplayDataMatcher for linkUrl commit 28739a73ec2510fc9b78fc878346f121e5bd7636 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-04T16:39:04Z Add linkUrl to AvroIO DisplayData commit fefc3ef5b2a3b7c0178a216c7fdaaa4885893144 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-04T16:51:20Z Add linkUrl to FileBasedSource DisplayData commit c28c4470c93b77096ea182ddc29813f7fe69db03 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-04T17:01:39Z Add linkUrl to FileBasedSink DisplayData commit f369a3324a260346bd45c786bccfa8a2a1c15e64 Author: Scott Wegner <sweg...@google.com> Date: 2016-05-04T17:19:15Z Add linkUrl to TextIO DisplayData --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request: Fix AutoValue coverage exclusion for ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/223 Fix AutoValue coverage exclusion for coveralls #109 introduces AutoValue but breaks our coverage tooling. This should fix it, but let's wait for a green jenkins build before merge. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam coveralls-fix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #223 commit d2524ce6cadcc73dd0d2b166bcd7086952d716bd Author: Scott Wegner <sweg...@google.com> Date: 2016-04-20T23:37:24Z Fix AutoValue coverage exclusion for coveralls --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #654: [BEAM-339] Archetype project version shoul...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/654 [BEAM-339] Archetype project version shouldn't be coupled to Beam version Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Follow-up from [a previous archetype PR](https://github.com/apache/incubator-beam/pull/444/files/ba55042275bd9b525ee8716e4e1007b7924a647f#r66819150), the maven artifact version for the generated project should not be tied to the version of Beam. The generated module is a new user project, so the version should represent "initial version", i.e. 0.1. This PR drops the -SNAPSHOT suffix from the version and fixes the version to 0.1 in our tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam archetype-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/654.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #654 commit 5b317b852b1fac9f6320e1a074238e6e7da77b79 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-13T22:59:16Z Archetype generated projects shouldn't have SNAPSHOT version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #660: Bump required Maven version to 3.3
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/660 Bump required Maven version to 3.3 Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam maven-version Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/660.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #660 commit 4a70283eeaa8f3fb8010d396f3dcfbe832049ecd Author: Scott Wegner <sweg...@google.com> Date: 2016-07-14T22:50:12Z Bump required Maven version to 3.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #753: [BEAM-436] Re-enable RunnableOnService tes...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/753 [BEAM-436] Re-enable RunnableOnService tests which need tempRoot Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- @peihe's recent work to improve tempRoot handling means that these tests are now working properly. They can be re-enabled to validate runner display data for i/o transforms. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam io-runnableonservice Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/753.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #753 commit e741f32184838f598d351117e024995ff4f5d3fe Author: Scott Wegner <sweg...@google.com> Date: 2016-07-28T20:35:55Z Re-enable RunnableOnService tests which need tempRoot --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #648: [BEAM-445] Switch to netty-tcnative uber-j...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/648 [BEAM-445] Switch to netty-tcnative uber-jar dependency Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Currently, we are using the os-maven-plugin extension to compute the platform-specific version of the netty-tcnative dependency to use in gcp-io. Unfortunately, netty-tcnative is a transitive dependency and all pom's which depend on gcp-io (directly or transitively) also need to declare the os-maven-plugin dependency. This PR removes os-maven-plugin and it's dynamic resolution. We instead depend on the "uber jar" version of netty-tcnative, which includes all platform-native versions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam os-maven-plugin Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/648.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #648 commit 55ed28eb4d8305ca1968d9b012b22779814d0a44 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-12T23:20:10Z Remove netty-tcnative dynamic os dependency --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #675: Clean up Aggregator javadoc
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/675 Clean up Aggregator javadoc Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- The [current javadoc for `Aggregator`](http://beam.incubator.apache.org/javadoc/0.1.0-incubating/org/apache/beam/sdk/transforms/Aggregator.html) is difficult to read: > Aggregators are created by calling `DoFn.createAggregator(java.lang.String, org.apache.beam.sdk.transforms.Combine.CombineFn)`, typically from the `DoFn` constructor. Elements can be added to the Aggregator by calling `addValue(InputT)`. This simply takes the function parameters out of `createAggregator`: > Aggregators are created by calling `DoFn.createAggregator()`, typically from the `DoFn` constructor. Elements can be added to the Aggregator by calling `addValue(InputT)`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam aggregator-javadoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/675.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #675 commit bfd43ca0ddc99aa81c0f4cb3ad7e757da63fa217 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-18T17:01:29Z Clean up Aggregator javadoc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #640: Prototype simplified all-dynamic display d...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/640 Prototype simplified all-dynamic display data usage Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam dyndisplaydata-prototype4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/640.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #640 commit 1998e896bf30a078d28e0ff120d0b10d0e594639 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-01T21:40:02Z Prototype all-dynamic display data usage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #602: Fix BigtableIO display data label
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/602 Fix BigtableIO display data label Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam bigtable-displaydata Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/602.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #602 commit 064e4c00ad410b041098a918ceb3983d668f1c7a Author: Scott Wegner <sweg...@google.com> Date: 2016-07-07T15:38:14Z Fix BigtableIO display data label --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #614: Add jacoco coverage
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/614 Add jacoco coverage Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam coveralls-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #614 commit 5537f71bd27971c85319884fe06dfc63994409f4 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-08T21:02:10Z Add jacoco coverage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #614: Add jacoco coverage
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/614 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #607: Mark primitive display data tests Runnable...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/607 Mark primitive display data tests RunnableOnService Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-primitives Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/607.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #607 commit 0d30860483a2c8f3d3efde9a45a162e5b8ece476 Author: Scott Wegner <sweg...@google.com> Date: 2016-07-07T20:49:30Z Mark primitive display data tests RunnableOnService --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #640: Prototype simplified all-dynamic display d...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/640 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #577: Job Messages Prototypes
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/577 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #574: Fix, suppress, or file bugs for FindBugs i...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/574 Fix, suppress, or file bugs for FindBugs issues Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- When adding FindBugs integration, many existing issues were uncovered. Of those issues, some are trivial to fix, some are more complicated, and some are unimportant or false positive. This PR goes through and triages the existing baseline issues. For each: * Unimportant / false positive: Move to inline suppression with justification ([example](https://github.com/apache/incubator-beam/compare/master...swegner:findbugs-stale?expand=1#diff-c1df753c9b67cc72f7beed9eaa90R64)) * Trivial fix: Fixed inline ([example](https://github.com/apache/incubator-beam/compare/master...swegner:findbugs-stale?expand=1#diff-d6c918ab6169a2f61f4902c6018778f3R351)) * Non-trivial fix needed: Opened bug and tagged baseline suppression ([example](https://github.com/apache/incubator-beam/compare/master...swegner:findbugs-stale?expand=1#diff-6bffc9239f6c94cf1540559353cd868dR36)) You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam findbugs-stale Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/574.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #574 commit c542117c48ce7983b363d676220392cd12b88ed7 Author: Scott Wegner <sweg...@google.com> Date: 2016-06-30T23:16:14Z Fix and file bugs for FindBugs issues --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #577: Job Messages Prototypes
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/577 Job Messages Prototypes [not to be submitted, just for prototyping feedback] You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam dyndispdata-prototypes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/577.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #577 commit 4394ecbab2abfb061681c0484bdff05338085b0d Author: Scott Wegner <sweg...@google.com> Date: 2016-07-01T21:40:02Z Initial prototype example usage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #842: [BEAM-557] Fix repackaging exclude pattern...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/842 [BEAM-557] Fix repackaging exclude pattern for guava-testlib Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- The previous fix was incorrect in that the exclude pattern requires `.*` at the end. Tested this iteration using `javap` to inspect generated `.class` files. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam guava-testlib Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/842.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #842 commit 2a8ceed623e7909e0d67b71263e255e10a1c7693 Author: Scott Wegner <sweg...@google.com> Date: 2016-08-17T16:28:44Z Fix repackaging exclude pattern for guava-testlib --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #838: Add example pipeline using runtime display...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/838 Add example pipeline using runtime display data aggregators Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam aggregator-example Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/838.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #838 commit c67ae86fd6a38fa5b4616450dba365bcd61fae68 Author: Scott Wegner <sweg...@google.com> Date: 2016-08-16T22:27:23Z Add example pipeline using runtime display data aggregators --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #856: [BEAM-409] Add tests for ApproximateQuanti...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/856 [BEAM-409] Add tests for ApproximateQuantiles calculated buffers Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Note: these tests currently fail due to [[BEAM-409]](https://issues.apache.org/jira/browse/BEAM-409). #853 fixes them, but I wanted to have tests that would verify the fix to go in with it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam approx-quantiles Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #844: Update DoFn javadocs to remove references ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/844 Update DoFn javadocs to remove references to OldDoFn and Dataflow Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Various components had references to `OldDoFn` where it would be more appropriate to describe the "new" `DoFn`. Along the way, updated a few Dataflow references to Beam. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam dofn-javadoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/844.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #844 commit dcab8c2a270f5b3448c3f8d5b8a28aeafb8352ac Author: Scott Wegner <sweg...@google.com> Date: 2016-08-17T21:38:36Z Update DoFn javadocs to remove references to OldDoFn and Dataflow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #906: [BEAM-548] Fixed Combine display data
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/906 [BEAM-548] Fixed Combine display data Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Continuation of #847 You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam CombinePerKeyWithHotKeyFanout Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/906.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #906 commit e8c708ad196645ddf5f01694e89f8d1ca651d315 Author: Ian Zhou <ianz...@google.com> Date: 2016-08-18T20:50:52Z Fixed Combine display data commit fcabbfc843be1d5eef7c34362ec697bfe5456daf Author: Ian Zhou <ianz...@google.com> Date: 2016-08-19T18:26:38Z Added DisplayData test commit 3fc6e45425e9bcfa095803add7555710a8cef2cc Author: Scott Wegner <sweg...@google.com> Date: 2016-08-19T20:32:45Z Delegate populateDipslayData to wrapped combineFn's commit 7ffdf23e5e0f01f6aa0a9e810b2cbd4182f8158f Author: Ian Zhou <ianz...@google.com> Date: 2016-08-19T22:02:28Z Minor changes commit 5079a9487f6653522dc001adbba7b4f86b824345 Author: Scott Wegner <sweg...@google.com> Date: 2016-08-29T22:42:23Z fixup! make KeyedCombineFn non-transient to it is available for display data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #933: Update 404 link to setuptools docs
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/933 Update 404 link to setuptools docs Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam setuptools_link Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/933.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #933 commit 6d0507a33e211cdfeba8a216afdd336f016df8fc Author: Scott Wegner <sweg...@google.com> Date: 2016-09-08T16:32:14Z Update 404 link to setuptools docs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #753: [BEAM-436] Re-enable RunnableOnService tes...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/753 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #933: Update 404 link to setuptools docs
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/933 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1005: Clear requirements cache directory before...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1005 Clear requirements cache directory before using it in test Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- `test_with_requirements_file` was failing locally because I had stale files in my /tmp/dataflow-requirements-cache from a previous package install. This change cleans the cache directory before validating its contents. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam clear-requirements-cache Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1005.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1005 commit ea1d367aacd69ae49d9ac486cbd3c07c269017af Author: Scott Wegner <sweg...@google.com> Date: 2016-09-26T16:42:47Z Clear requirements cache directory before using it in test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1006: [BEAM-678] Add BEAM_PYTHON environment ov...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/1006 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1029: [BEAM-691] Correctly implement display da...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1029 [BEAM-691] Correctly implement display data for DataflowRunner Read transforms Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-streaming Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1029.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1029 commit cda01dbc4038a3a07b808d9d868afee666b219cb Author: Scott Wegner <sweg...@google.com> Date: 2016-09-29T21:43:14Z Set transform names in DisplayDataEvaluator If a transform name is unset, a runner may infer the name from the class simple name, which fails with anonymous classes. In order to be more generically useful, we set transform names within the DisplayDataEvaluator. commit b0ce3f9d26eb7fb1d5b427a987939d555667c408 Author: Scott Wegner <sweg...@google.com> Date: 2016-09-29T21:45:57Z Handle anonymous classes in DataflowUnboundedReadFromBoundedSource.getKind() commit 50d8d34d6f6b7b5d907906431844b41f2d9c7b24 Author: Scott Wegner <sweg...@google.com> Date: 2016-09-29T21:46:33Z Correctly implement display data for DataflowRunner Read transforms The DataflowRunner will replace usages of the Read transform with interally-wrapped implementations for streaming and bounded cases. We were not properly passing display data between these usages. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1005: Clear requirements cache directory before...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/1005 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1056: Static import Assert.assertEquals in Data...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1056 Static import Assert.assertEquals in DataflowUnboundedReadFromBoundedSourceTest Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam assert-staticimport Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1056.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1056 commit 099fd9c740bac1bed5d3b7227c2f5b895808e99a Author: Scott Wegner <sweg...@google.com> Date: 2016-10-05T20:58:40Z Static import Assert.assertEquals --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1034: [BEAM-695] PipelineOptions display data n...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1034 [BEAM-695] PipelineOptions display data needs to handle array types Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- PipelineOptions generates display data for arbitrary option types using #toString(). For array types, this gives an message like [Ljava.lang.String;@fc25934b]. Instead, we detect array types and use Arrays.deepString to build a string based on array values. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-array-string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1034.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1034 commit ca12648f5ce0b744c30cfa6bbcd486519ae1bf95 Author: Scott Wegner <sweg...@google.com> Date: 2016-09-30T17:05:00Z PipelineOptions display data needs to handle array types PipelineOptions generates display data for arbitrary option types using #toString(). For array types, this gives an message like [Ljava.lang.String;@fc25934b]. Instead, we detect array types and use Arrays.deepString to build a string based on array values. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #838: Add example pipeline using runtime display...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/838 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1004: Ignore virtualenv environment in git
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/1004 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1208: (WIP) [BEAM-767] Convert DisplayData incl...
GitHub user swegner opened a pull request:
https://github.com/apache/incubator-beam/pull/1208
(WIP) [BEAM-767] Convert DisplayData include(..) API to nested(..)
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
In the Java SDK, components can register display data directly, as well as
include display data from sub-components via the `include(..)` operation. In
#1008, we add 'paths' to display data metadata and also include a
`delegate(..)` action.
During the change for paths, we noticed an emerging pattern where
components will typically add the `.getClass()` of a subcomponent before
including it. We can support this convention better by embedding it in a single
configurable operation, `.nested(..)`
So this:
```java
@Override
public void populateDisplayData(DisplayData.Builder builder) {
builder
.add(DisplayData.item("windowFn", windowFn.getClass()))
.include("windowFn", windowFn)
.add(DisplayData.item("otherProperty", otherProp));
}
```
becomes this:
```java
@Override
public void populateDisplayData(DisplayData.Builder builder) {
builder
.add(DisplayData.nested("windowFn", windowFn.getClass()))
.add(DisplayData.item("otherProperty", otherProp));
}
```
See https://github.com/apache/incubator-beam/pull/1088#discussion_r83332515
for more context.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/swegner/incubator-beam displaydata-nested
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/1208.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1208
commit 5c2125f583a74ba8aa00df79641905063e22ac24
Author: Scott Wegner <sweg...@google.com>
Date: 2016-10-27T15:20:56Z
Convert DisplayData include(..) API to nested(..)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-beam pull request #1208: (WIP) [BEAM-767] Convert DisplayData incl...
Github user swegner closed the pull request at: https://github.com/apache/incubator-beam/pull/1208 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1218: Remove unnecessary configuration properti...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1218 Remove unnecessary configuration properties from checkstyle.xml Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Is there any reason we need these properties? The default values are not overridden anywhere in our project and would lead to inconsistent style enforcement. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam checkstyle-intellij Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1218.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1218 commit 7a0a8207878e8316f416ab1a1429c051d2ac7470 Author: Scott Wegner <sweg...@google.com> Date: 2016-10-28T16:17:46Z Remove unnecessary configuration properties from checkstyle.xml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1088: Support display data for transforms with ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1088 Support display data for transforms with nested components of the same instance Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam displaydata-nestedclasses Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1088.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1088 commit d1298954b680c40a26c43d204e478561dfc4902f Author: Scott Wegner <sweg...@google.com> Date: 2016-10-12T21:49:41Z Test case for failing display data for nested component structure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1155: [BEAM-795] Use DisplayData delegate in Da...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1155 [BEAM-795] Use DisplayData delegate in Dataflow Read transform override Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- This addresses the failing Dataflow RunnableOnService tests: https://builds.apache.org/job/beam_PostCommit_RunnableOnService_GoogleCloudDataflow/ You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam runnable-on-service Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1155.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1155 commit 4869ef19fa70769831dacc2ac332ec0a11b3f9d8 Author: Scott Wegner <sweg...@google.com> Date: 2016-10-21T18:41:41Z Use DisplayData delegate in Dataflow Read transform override --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1401: Remove dependency on findbugs annotations...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1401 Remove dependency on findbugs annotations artifact Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- The findbugs annotations artifact has an incompatible license and cannot be included within Beam. We were previously referencing it for the @SuppressFBWarning annotation for inline FindBugs suppression. This change moves inline suppressions out to the existing findbugs-filter.xml file. While not ideal as it removes the suppressions from the offending context, it allows us to drop our dependency on the incompatible artifact. We are also referencing the @Nullable attribute from findbugs. This is now source from findbugs jsr305, which does have a compatible license. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam findbugs-convert-suppressions Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1401.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1401 commit 850fa2c0010324447e8ebf9c31ebfec4a1974db5 Author: Scott Wegner <sweg...@google.com> Date: 2016-11-21T21:07:46Z Remove dependency on findbugs annotations artifact The findbugs annotations artifact has an incompatible license and cannot be included within Beam. We were previously referencing it for the @SuppressFBWarning annotation for inline FindBugs suppression. This change moves inline suppressions out to the existing findbugs-filter.xml file. While not ideal as it removes the suppressions from the offending context, it allows us to drop our dependency on the incompatible artifact. We are also referencing the @Nullable attribute from findbugs. This is now source from findbugs jsr305, which does have a compatible license. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1406: [BEAM-1031] Update StarterPipeline to use...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1406 [BEAM-1031] Update StarterPipeline to use the new DoFn Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam starter-dofn Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1406.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1406 commit 9f2111024e3537910a4ca2543444c415215e028c Author: Scott Wegner <sweg...@google.com> Date: 2016-11-22T00:33:07Z Update starter pipeline to use the new DoFn --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1414: [BEAM-1032] Add direct runner dependency ...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1414 [BEAM-1032] Add direct runner dependency to starter archetype Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam starter-runner Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1414.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1414 commit 2a1f829477b1bcabddd28ef27dc1bd88bc72b82e Author: Scott Wegner <sweg...@google.com> Date: 2016-11-22T16:56:17Z Add direct runner dependency to starter archetype --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam-site pull request #55: [BEAM-857] Add IntelliJ Checkstyle doc...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam-site/pull/55 [BEAM-857] Add IntelliJ Checkstyle documentation You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam-site intellij Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam-site/pull/55.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #55 commit db18665a9bfbb3ca2e492c9afb9f8519c9a61596 Author: Scott Wegner <sweg...@google.com> Date: 2016-10-28T17:44:13Z Add IntelliJ Checkstyle documentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1627: [BEAM-1160] Add option to disable failure...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1627 [BEAM-1160] Add option to disable failures if filePattern resolves to empty Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Most PTransforms which take a filePattern have construction-time validation which checks-- among other things-- that the specified filePattern matches at least one file. This is particularly useful for catching typos when specifying input files. Most PTransforms also have an option to disable their construction-time validation. This is generally used when validation cannot be performed at construction time: for example because the proper credentials aren't available or the input specification is late-bound in a template. To allow for these scenarios and still guard against typos, FileBasedSource also validates that the filePattern matches at least one file at runtime. This change adds the ability FileBasedSource to disable this runtime validation, for cases uses case where empty filePatterns should be allowed. FileBasedSource gains a new constructor parameter, and PTransforms which use FileBasedSource have the option exposed in their respective builder APIs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam read-allow-empty-glob Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1627 commit 43195ba27b4303c852e81fa1de493ec4fec641bb Author: Scott Wegner <sweg...@google.com> Date: 2016-12-15T18:12:25Z Add option to disable failures if filePattern resolves to empty Most PTransforms which take a filePattern have construction-time validation which checks-- among other things-- that the specified filePattern matches at least one file. This is particularly useful for catching typos when specifying input files. Most PTransforms also have an option to disable their construction-time validation. This is generally used when validation cannot be performed at construction time: for example because the proper credentials aren't available or the input specification is late-bound in a template. To allow for these scenarios and still guard against typos, FileBasedSource also validates that the filePattern matches at least one file at runtime. This change adds the ability FileBasedSource to disable this runtime validation, for cases uses case where empty filePatterns should be allowed. FileBasedSource gains a new constructor parameter, and PTransforms which use FileBasedSource have the option exposed in their respective builder APIs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1522: [BEAM-1097] Provide a better error messag...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1522 [BEAM-1097] Provide a better error message for non-existing gcpTempLocation Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- gcpTempLocation will default to using the value for tmpLocation, as long as the value is a valid GCP path. Non-valid GCP paths are silently discarded. This change removes existence validation from the default value logic such that downstream validation can provide a better error message. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam gcp-temp-location-error Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1522 commit 9d768df4323a246baa705fd5fb75d08c78abb7f0 Author: Scott Wegner <sweg...@google.com> Date: 2016-12-06T22:19:12Z Provide a better error message for non-existing gcpTempLocation gcpTempLocation will default to using the value for tmpLocation, as long as the value is a valid GCP path. Non-valid GCP paths are silently discarded. This change removes existence validation from the default value logic such that downstream validation can provide a better error message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1621: Fail to split in FileBasedSource if fileP...
GitHub user swegner opened a pull request: https://github.com/apache/incubator-beam/pull/1621 Fail to split in FileBasedSource if filePattern expands to empty Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- Typically, input file patterns are validated during Pipeline construction, but standard Read transforms include an option to disable validation. This is generally useful but can lead to cases where a Pipeline executes successfully with empty inputs. This changes the behavior to fail execution on empty file-based inputs even when validation is disabled. You can merge this pull request into a Git repository by running: $ git pull https://github.com/swegner/incubator-beam empty-glob Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1621.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1621 commit 9fc9d66212bf26087622ad6e042db982d5232c55 Author: Scott Wegner <sweg...@google.com> Date: 2016-12-14T22:52:34Z Fail to split in FileBasedSource if filePattern expands to empty. Typically, input file patterns are validated during Pipeline construction, but standard Read transforms include an option to disable validation. This is generally useful but can lead to cases where a Pipeline executes successfully with empty inputs. This changes the behavior to fail execution on empty file-based inputs even when validation is disabled. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
