[GitHub] beam pull request #4115: [BEAM-3170] support topicPartition in BeamKafkaTabl...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/4115 [BEAM-3170] support topicPartition in BeamKafkaTable Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-3170 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4115.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4115 commit 1c5d33abc7f8ddca0ec284bd47dbc76de559 Author: mingmxu <ming...@ebay.com> Date: 2017-11-11T03:51:37Z support topicPartition in BeamKafkaTable ---
[GitHub] beam pull request #4101: [BEAM-3161] Cannot output with timestamp XXXX
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/4101 [BEAM-3161] Cannot output with timestamp R: @xumingming CC: @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-3161 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/4101.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4101 commit b4e8494708d79d0a2a64e86619bd5b99fcf5799d Author: mingmxu <ming...@ebay.com> Date: 2017-11-08T21:44:18Z change `withAllowedTimestampSkew` ---
[GitHub] beam pull request #3848: [BEAM-2954] update shade configurations in extensio...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3848 [BEAM-2954] update shade configurations in extension/sql Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2954 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3848.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3848 commit 522e6c188809327f098b84dbd7ee07e1dad4a376 Author: mingmxu <ming...@ebay.com> Date: 2017-09-13T17:31:19Z update shade settings to handle Calcite dependencies ---
[GitHub] beam pull request #3794: [TEST PR] to verify [BEAM-301]
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3794 ---
[GitHub] beam pull request #3794: [TEST PR] to verify [BEAM-301]
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3794 [TEST PR] to verify [BEAM-301] Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam VERIFY_MERGE Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3794.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3794 commit 95e14214cbf49c8358872c9d34edc6713bc02e0d Author: Kenneth Knowles <k...@google.com> Date: 2017-05-26T23:07:45Z Port DirectRunner WriteFiles override to SDK-agnostic APIs commit 4a350608100ad3f5dd3bf2798fc9976dcec95467 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-08T20:39:32Z Port DirectRunner TestStream override to SDK-agnostic APIs commit fd74b5ca83a32afc67a318f1f791726c6326c6b4 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-05T21:20:27Z [BEAM-2412] Update HBaseIO to use HBase client 1.2.6 commit 43b8e5f7b2e33c10d823037fa3af3e04e19c373d Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:34:25Z Use CoderTypeSerializer and remove unuse code in FlinkStateInternals commit 1d6c466018aa85a7ef41a0535a1db744fd18e87e Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T06:40:30Z [BEAM-1483] Support SetState in Flink runner and fix MapState to be consistent with InMemoryStateInternals. commit cfbeee2e5bf2ef2d0d0ca839ed2c915e302b0f14 Author: JingsongLi <lzljs3620...@aliyun.com> Date: 2017-06-07T17:31:34Z [BEAM-2423] Abstract StateInternalsTest for the different state internals commit 4f406f50fee333e871459bb5b5d54c7b1719a030 Author: chamik...@google.com <chamik...@google.com> Date: 2017-06-08T21:56:24Z Adds ability to dynamically replace PTransforms during runtime. To this end, adds two interfaces, PTransformMatcher and PTransformOverride. Currently only supports replacements where input and output types are an exact match (we have to address complexities due to type hints before supporting replacements with different types). This will be used by SplittableDoFn where matching ParDo transforms will be dynamically replaced by SplittableParDo. commit 75710a6fd9fa9849680349def917c1c9ddcdd56e Author: Robert Bradshaw <rober...@google.com> Date: 2017-06-09T23:44:55Z Actually test the fn_api_runner. The test suite was not being run due to a typo. Fix breakage due to changes in the code in the meantime. commit cf33f5ee4db646ab5a53f30fb6de084859e3a86a Author: Maria Garcia Herrero <mari...@google.com> Date: 2017-06-10T06:34:59Z Make unique test names for value-provider arguments commit d9887e572902beaf97cf956d8062f61cf9396fb7 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T18:07:47Z Cleanup Combine Tests with Context Split out the "shared" bit of all the accumulators, so they show up as an explicit component of the final result string. Update timestamped creation logic. commit 17d2466d5ed6dd8ac9b9fe23f9ae5f9b2583bc23 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-12T18:51:39Z Improves message when transitively serializing PipelineOptions commit 7816f9844fa871bfc0e05e87c7876e9a5eb1e0e2 Author: Charles Chen <c...@google.com> Date: 2017-06-12T21:17:50Z Reverse removal of NativeWrite evaluator in Python DirectRunner commit 705db38a964525330ae1c13e270d2e793eda59d0 Author: Thomas Groh <tg...@google.com> Date: 2017-06-12T23:55:59Z Check for Deferral on Non-additional inputs Because Side Inputs are represented within the expanded inputs, the check that the transform is a Combine with Side Inputs would never
[GitHub] beam pull request #3767: [BEAM-2809] update JavaDoc of BeamSortRel
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3767 [BEAM-2809] update JavaDoc of BeamSortRel R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2809 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3767.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3767 commit 97ad2f8609230861f6804ae2cba6cada42f4c865 Author: mingmxu <ming...@ebay.com> Date: 2017-08-25T23:45:33Z update JavaDoc of BeamSortRel --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3752: [BEAM-2798] rename simpleQuery to query and query t...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3752 [BEAM-2798] rename simpleQuery to query and query to queryMulti R: @takidau @xumingming You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2798 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3752 commit c580b52198d475dd56272bdf4d751991c0c51a15 Author: mingmxu <ming...@ebay.com> Date: 2017-08-24T00:39:18Z rename simpleQuery to query and query to queryMulti --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3606: [BEAM-301] Merge DSL_SQL to Master
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3606 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3648: sync-up from master branch
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3648 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3716: [BEAM-2747] accept CombineFn as UDAF
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3716 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3716: [BEAM-2747] accept CombineFn as UDAF
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3716 [BEAM-2747] accept CombineFn as UDAF Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2747 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3716.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3716 commit 44107569e689e190df7d9cc4a58ba546d507a2d0 Author: mingmxu <ming...@ebay.com> Date: 2017-08-11T00:42:29Z take CombineFn as UDAF. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3710: [BEAM-2741] update JavaDoc for BeamRecord
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3710 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3707: [BEAM-2748] accept SerializableFunction as UDF
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3707 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3710: [BEAM-2741] update JavaDoc for BeamRecord
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3710 [BEAM-2741] update JavaDoc for BeamRecord R: @takidau @xumingming Ps, the change of `BeamSqlUdfExpression` is to address the question in #3707 You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2741 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3710.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3710 commit c5ea15a2ec85c02aff755cc82d3e550b8fbafc54 Author: mingmxu <ming...@ebay.com> Date: 2017-08-09T20:30:16Z update JavaDoc for BeamRecord, BeamRecordType, quickfix for BeamSqlUdfExpression --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3706: [BEAM-2749] update BeamSqlExample
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3706 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3707: [BEAM-2748] accept SerializableFunction as UDF
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3707 [BEAM-2748] accept SerializableFunction as UDF R: @takidau @xumingming You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2748 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3707.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3707 commit 22fb659a182b2143252edf0fb675331a447da943 Author: mingmxu <ming...@ebay.com> Date: 2017-08-09T06:02:32Z take SerializableFunction as UDF. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3706: [BEAM-2749] update BeamSqlExample
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3706 [BEAM-2749] update BeamSqlExample R: @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2749 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3706.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3706 commit f03cc869db3801b7913a87f7950e16241b877a5a Author: mingmxu <ming...@ebay.com> Date: 2017-08-09T04:52:54Z update example. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3692: [BEAM-2730] make BeamRecord an immutable type
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3692 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3692: [BEAM-2730] make BeamRecord an immutable type
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3692 [BEAM-2730] make BeamRecord an immutable type @robertwb Ps, some lines are expected to delete with #3691 and #3689 You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2730 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3692.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3692 commit ca0663d48fe82c20600966dd2d2852e27d6ca5ed Author: mingmxu <ming...@ebay.com> Date: 2017-08-04T22:29:10Z make BeamRecord an immutable type --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3691: [BEAM-2725] remove nullFields in BeamRecord
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3691 [BEAM-2725] remove nullFields in BeamRecord R: @robertwb You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2725 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3691.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3691 commit a9eb603ff630fe91de61696d553dcdd871291737 Author: mingmxu <ming...@ebay.com> Date: 2017-08-04T20:46:10Z remove nullFields in BeamRecord --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3689: [BEAM-2722] Remove redundant windowing information ...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3689 [BEAM-2722] Remove redundant windowing information from the BeamRecord itself R: @robertwb You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2722 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3689 commit c0b1fed1c7687a1e44e8ee6997be40aa1785ea51 Author: mingmxu <ming...@ebay.com> Date: 2017-08-04T18:12:45Z Remove redundant windowing information from the BeamRecord itself element window information `BoundedWindow` is added in `BeamSqlExpression`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3675: [BEAM-2676] move BeamSqlRow and BeamSqlRowType to s...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3675 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3675: [BEAM-2676] move BeamSqlRow and BeamSqlRowType to s...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3675 [BEAM-2676] move BeamSqlRow and BeamSqlRowType to sdk/java/core create a new PR to get rid of the huge rebase work after #3666. Following the discussion in [BEAM-2676](https://issues.apache.org/jira/browse/BEAM-2676), the changes are outlined as: 1. `BeamRecord` and `BeamRecordTypeProvider` are marked as `@Experimental`; 2. `BeamRecord` is moved to sdk/core, which includes a default `BeamRecordTypeProvider` to define type information; Also a `BeamRecordCoder` is provided as Coder; 3. In _extension/sql_, `BeamSqlRecord` extends `BeamRecord`, `BeamSqlRecordTypeProvider` extends `BeamRecordTypeProvider` to support SQL types; a new Coder `BeamSqlRecordCoder` is provided to align better; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2676_2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3675.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3675 commit 09a7148e603f8d8521f916008f92b0e95ae7c14f Author: mingmxu <ming...@ebay.com> Date: 2017-08-02T08:20:50Z move BeamRecord to sdk/core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3633: [BEAM-2676] move BeamSqlRow and BeamSqlRowType to s...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3633 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3648: sync-up from master branch
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3648 sync-up from master branch BEAM-2672 requires a new function added in #3628 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3648.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3648 commit 4192ac6c72765e8c55480eec66c499ef0798ecf3 Author: Stephen Sisk <s...@google.com> Date: 2017-06-14T16:57:35Z Add maven support for invoking perfkit benchmarker to run IO ITs commit 2f9cbec803791fd4e17fb28ec5590649aabb9622 Author: Stephen Sisk <s...@google.com> Date: 2017-07-20T18:38:40Z io-it-suite-local independent of io-it-suite, k8s properties -> root pom commit 7e63d2cf6c0ea0da4ce68b72536522f68b4a5272 Author: Thomas Groh <tg...@google.com> Date: 2017-07-20T20:34:49Z This closes #3588 commit c52a908cba7765e120a94909ab02c548d1a124ad Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-07-18T20:40:52Z Minor changes to AvroSource in preparation for refactoring commit d4026da1ad1fa0864052b85a66c4af5975327e9f Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-07-18T21:09:03Z Gets rid of opening Avro files in createForSubrangeOfFile codepath commit 1d9160fa3337704b9fbdc423796925be78b0087e Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-07-20T23:59:27Z This closes #3590: [BEAM-2628] Makes AvroSource not open files while splitting commit cfadecb56c64aa155c8b5fd0d8a6654ceb918eba Author: Kenneth Knowles <k...@google.com> Date: 2017-07-18T05:03:09Z Uniquify application nodes in TextIOReadTest and only run pipeline once - These nodes have never had stable unique names, but we have always mutated the pipeline before running it so they got lucky. - A pipeline can also be mutated when run() so it should be considered dead after run() is called. This fixes both issues by uniquifying the names and running a given pipeline only once. commit c2110c97d530b8e90387bf99b3ac7d36201d85c7 Author: Thomas Groh <tg...@google.com> Date: 2017-07-19T17:55:33Z Register a PTransformTranslator for Combine Include the Combine Payload in the Runner API Graph. Add getCombineFn(AppliedPTransform) to extract the CombineFn from an arbitrary transform. Update Pipeline Translation tests to include accumulator coders. commit b6f126dc6000ceba410d5074fd65af141b7ab3c6 Author: Thomas Groh <tg...@google.com> Date: 2017-07-21T17:03:30Z This closes #3597 commit 01492e69d1b1eedb99931f7071a9dae9d60d25fe Author: Kenneth Knowles <k...@google.com> Date: 2017-07-20T03:58:36Z Use RehydratedComponents for memoized rehydration commit f870bf5164b00a1379f1e33a5ef27b3701a87c08 Author: Kenneth Knowles <k...@google.com> Date: 2017-07-21T18:06:02Z This closes #3601: Use RehydratedComponents for memoized rehydration commit e38dc5fbe1f45fe282b4cd96b858631db73130e9 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-07T20:58:11Z Use dehydration-insensitive APIs in WindowEvaluatorFactory commit 8bd64759616a2f262f60ce2998b98f0dd1dd2000 Author: Kenneth Knowles <k...@google.com> Date: 2017-07-21T18:36:46Z This closes #3609: Use dehydration-insensitive APIs in WindowEvaluatorFactory commit fa3a5abbc94db629feae8d7d73a31e7dda06bf76 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-07T21:35:09Z Use dehydration-insensitive APIs in ParDoEvaluatorFactory commit 4b355844a4920bc9faba75f7cd61008bedebaf29 Author: Kenneth Knowles <k...@google.com> Date: 2017-07-21T18:50:48Z This closes #3610: Use dehydration-insensitive APIs in ParDoEvaluatorFactory commit 81c2e90f3a9d109121b686c25b642cc9f16ced43 Author: Kenneth Knowles <k...@google.com> Date: 2017-07-21T19:21:05Z This closes #3608: Uniquify application nodes in TextIOReadTest and only run pipeline once commit 4f7d0b4ed70b40443aede1ce28d0939f08b1d04f Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-07-22T01:01:14Z Unbreaks AvroSource compatibility with Dataflow worker commit f39874860027832fdeff957afdc5cc8326de6d30 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-07-22T01:23:48Z This closes #3616: Unbreaks AvroSource compatibility with Dataflow worker commit a7c18495f91c9d7537271c9595b3ac4d24003ece Author: Luke Cwik <lc...@google.com> Date: 2017-07-23T21:00:17Z [BEAM-2662] Fix Java quickstart for Spark to include Spark version commit b1adbfbcf0a949a92a022c86b061ba3b64b69c9b Author: Pei He <p...@apache.org> Date: 2017-07-19T12:38:49Z ReshuffleTest: replace
[GitHub] beam pull request #3633: [BEAM-2676] move BeamSqlRow and BeamSqlRowType to s...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3633 [BEAM-2676] move BeamSqlRow and BeamSqlRowType to sdk/java/core It contains two parts: 1. remove SQL word in the name, `BeamSqlRow` --> `BeamRow` `BeamSqlRowType` --> `BeamRowType` 2. move from package `org.apache.beam.dsls.sql.schema` to `org.apache.beam.sdk.sd` (**sd** stands for structure data), in module `beam-sdks-java-core` *Hint*: The 4 files are changed to remove dependencies on Calcite/BeamSql, others are updated automate by IDE due to class name change: * BeamRow.java * BeamRowCoder.java * BeamRowType.java * BeamSqlRowCoderTest.java You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2676 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3633.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3633 commit 8e9382862cfac475db286b35d5cab379de83615d Author: mingmxu <ming...@ebay.com> Date: 2017-07-24T23:03:35Z move BeamSqlRow and BeamSqlRowType to sdk/java/core --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3614: [BEAM-2652] cleanup pom.xml
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3614 [BEAM-2652] cleanup pom.xml R: @kennknowles @rangadi @rangadi I also update pom.xml in kafka-io as now kafka-client is shared in multiple modules. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2652 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3614 commit 5dca20f88c1d78ebb65b75c9e777e7af5bcee587 Author: mingmxu <ming...@ebay.com> Date: 2017-07-21T17:40:43Z update pom.xml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3606: Merge DSL_SQL to Master
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3606 Merge DSL_SQL to Master Merge DSL_SQL feature branch back to master branch, with Beam SQL DSL function is ready to use. Resources: 1. thread in apache-beam-dev mail-list https://www.mail-archive.com/dev@beam.apache.org/msg02064.html 2. JIRA tasks https://issues.apache.org/jira/browse/BEAM-2621?jql=labels%20%3D%20dsl_sql_merge 3. Task burndown doc https://docs.google.com/document/d/1EHZgSu4Jd75iplYpYT_K_JwSZxL2DWG8kv_EmQzNXFc/edit?usp=sharing You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam DSL_SQL Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3606.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3606 commit bd99528af89450b44d94abc42a8b884e00cbc26e Author: Tyler Akidau <taki...@apache.org> Date: 2017-06-27T00:36:58Z [BEAM-2503] This closes #3427 commit 32fbc9cec1d0d86e04e3f453b0d75f2ff0e61b56 Author: Ismaël MejÃa <ieme...@gmail.com> Date: 2017-06-26T14:37:51Z Small fixes to make the example run in a runner agnostic way: - Add direct runner default profile - Add findbugs validation and fix existing findbugs issues - Validate division by zero on arithmetic expression + other minor fixes - Update Calcite version to 1.13 commit ab4b118869070e94a4205744d6d60525d3fa2882 Author: Tyler Akidau <taki...@apache.org> Date: 2017-06-28T01:25:56Z This closes #3439 commit 928cec597175c363d444331b35ac8793297a242b Author: James Xu <xumingmi...@gmail.com> Date: 2017-05-29T03:11:34Z [BEAM-2193] Implement FULL, INNER, and OUTER JOIN: - FULL and INNER supported on all variations of unbounded/bounded joins. - OUTER JOIN supported when outer side is unbounded. - Unbounded/bounded joins implemented via side inputs. commit 2096da25e85d97ab52850453c2130ff706d7bcdf Author: Tyler Akidau <taki...@apache.org> Date: 2017-06-29T23:34:45Z [BEAM-2193] This closes #3277 commit a13fce98f61867fcb5adb52c80f1cfd3eecfc436 Author: mingmxu <ming...@ebay.com> Date: 2017-06-26T23:03:51Z UDAF support: - Adds an abstract class BeamSqlUdaf for defining Calcite SQL UDAFs. - Updates built-in COUNT/SUM/AVG/MAX/MIN accumulators to use this new class. commit 7ba77dd435eefd697fd0a452b07e0154dd1cd1b2 Author: Tyler Akidau <taki...@apache.org> Date: 2017-06-30T22:40:05Z [BEAM-2287] This closes #3447 commit 21497194db3ddce37a4747b3de2714b02684c57e Author: James Xu <xumingmi...@gmail.com> Date: 2017-06-27T02:42:40Z BeamSql: refactor the MockedBeamSqlTable and related tests commit bc66698e6880c7788bcea78006c67bfca66b17ce Author: James Xu <xumingmi...@gmail.com> Date: 2017-06-30T06:54:26Z MockedBeamSqlTable -> MockedBoundedTable commit ca2bc723dc00f0a5bf3e6157f8cd79ef4297093b Author: Luke Cwik <lc...@google.com> Date: 2017-07-05T16:34:17Z [BEAM-2515] BeamSql: refactor the MockedBeamSqlTable and related tests This closes #3478 commit 794f1901dcf5fb520a849adce7ce436e8b2f8535 Author: mingmxu <ming...@ebay.com> Date: 2017-07-10T05:26:29Z Test unsupported/invalid cases in DSL tests. commit b8fa0addc86d2249cd2fbbce3d8ad98d27e04604 Author: Tyler Akidau <taki...@apache.org> Date: 2017-07-12T04:05:15Z [BEAM-2574] This closes #3530 commit e9dc5ea81cbbde39bf11ee183e5403b869d21f50 Author: James Xu <xumingmi...@gmail.com> Date: 2017-07-06T03:29:41Z [BEAM-2550] add UnitTest for JOIN in DSL commit aa265e62a6c807ee941b1fc9379411f736754bf5 Author: Tyler Akidau <taki...@apache.org> Date: 2017-07-12T06:47:49Z [BEAM-2550] This closes #3506 commit eb589fb943a45388e4118bb221e52009eb92d1c1 Author: mingmxu <ming...@ebay.com> Date: 2017-07-09T07:52:23Z support TUMBLE/HOP/SESSION _START function commit a96c3a01f18826b4e43ad7e8f26e94c02903acd9 Author: Tyler Akidau <taki...@apache.org> Date: 2017-07-12T16:49:59Z [BEAM-2204] This closes #3527 commit 0580e8b639ef77c7a6534b7b91ecad493950a3aa Author: mingmxu <ming...@ebay.com> Date: 2017-07-12T07:08:35Z Test queries on unbounded PCollections with BeamSql DSL API. Also add getTYPE(fieldName) overrides to BeamSqlRow. commit 8defe6f21524b4ca00dd984176260c1bd0739774 Author: Tyler Akidau <taki...@apache.org> Date: 2017-07-12T17:03:28Z [BEAM-2527] This closes #3477 commit 3a170744861dec12a33a19358f9ce89dbd64491c Author: James Xu <xumingmi...@gmail.com> Date: 2017-07-10T11:58:21Z [BEAM-2564] add integration test for string functions commit 53d27e6c45ab7bbd6877c6674868e8a2e3f9a971 Author: Tyler Akidau <taki...@apache.org> Date: 2017-07-12T17:07:39Z [BEAM-25
[GitHub] beam pull request #3605: [BEAM-2555] add README page in dsl/sql
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3605 [BEAM-2555] add README page in dsl/sql 1. remove README.md to keep align with other BEAM modules; 2. update usages in BeamSqlExample; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2555 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3605.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3605 commit f1aa390d1989543f2848dae6b26596ffd1a5d8db Author: mingmxu <ming...@ebay.com> Date: 2017-07-20T21:32:42Z remove README.md and update usages in BeamSqlExample --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3603: redo PR 3553
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3603 redo PR 3553 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3603.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3603 commit 8cab15338f811f880c6cfb820051cf355f92986b Author: Robert Bradshaw <rober...@google.com> Date: 2017-06-22T01:09:48Z Java Dataflow runner harness compatibility. commit 2a5520018fb25f96b57e52231a8cc0b191abfce0 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-22T19:34:27Z Closes #3418 commit 7471e2736cc22336500f6252ab8448889a2d04d3 Author: Charles Chen <c...@google.com> Date: 2017-06-22T18:29:54Z Clean up test_stream_test and remove stray print statement commit 3dc454a9bab4683efea080a1cd5484f0838561d3 Author: Ahmet Altay <al...@google.com> Date: 2017-06-22T20:17:50Z This closes #3423 commit 0292a24f9c88796542bff55031d84c11f0ab6b16 Author: Colin Phipps <fi...@google.com> Date: 2017-05-15T14:18:16Z [BEAM-2439] Dynamic sizing of Datastore write RPCs This stops the Datastore connector from always sending 500 entities per RPC. Instead, it starts at a lower number which is more likely to complete within the deadline even in adverse conditions, and then increases or reduces the batch size in response to measured latency of past requests. commit 9ed0af8f2c5dabf3af5d664c473a49166b96d7d2 Author: Luke Cwik <lc...@google.com> Date: 2017-06-22T20:34:12Z [BEAM-2439] Dynamic sizing of Datastore write RPCs This closes #3390 commit c05764454e73ab93d0602f34b8b7622d46e1d892 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T03:58:35Z DataflowRunner: Reject SetState and MapState commit 5506be87dc3b8f4c4425a259cf255a168fd7b127 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T20:51:46Z This closes #3420: DataflowRunner: Reject SetState and MapState commit 497cfabea7d6dcee0c5d327022678c571c3ec487 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T18:31:28Z Add window matcher for pane info commit d4e5db51a025a831ddf4e3bc0e003caebabf647b Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T18:56:53Z Tidy LateDataDroppingDoFnRunner commit 1c1f239501349f5120b0d619c4eea9c435500b78 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T19:52:42Z ReduceFnTester can advance clocks without firing timers commit 795760d370bcbe28e1f0ca373ad4c8c841e6e6b5 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T19:53:15Z ReduceFnTester assertion for windows that have data buffered commit 412fd7eab9e58a4d412f4dff5ffec023610b4f22 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T19:56:14Z Drop late data in ReduceFnTester commit 50c43d96adb8c2523cf38c09f32e241eacc47823 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T19:56:34Z Do not GC windows based on processing time timer! commit fda589c00c8920e76cfc9aaa87cecfa94077599d Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T20:04:23Z Add test reproducing BEAM-2505, ignored commit d2b384a20dbb0213d0f63e74713a06d63bad8d39 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T20:05:42Z Add tests for corner cases of processing time timers commit 7645c44b9bae9a77192fbe6744918cfb95ca1a3a Author: Kenneth Knowles <k...@google.com> Date: 2017-06-22T22:41:02Z This closes #3425: Fix processing time timer handling in ReduceFnRunner Add tests for corner cases of processing time timer
[GitHub] beam pull request #3602: Revert "[BEAM-2610] This closes #3553"
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3602 Revert "[BEAM-2610] This closes #3553" This reverts commit ec494f675aa73fbdc7929f9592f33951941962b0, reversing changes made to d89d1ee1a3085269cdf44ec50e29a95c8f43757b. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam revert_3553 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3602.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3602 commit c1b2b96a438b86a8b0023f6943dcf0a4f238ba39 Author: mingmxu <ming...@ebay.com> Date: 2017-07-19T21:26:11Z Revert "[BEAM-2610] This closes #3553" This reverts commit ec494f675aa73fbdc7929f9592f33951941962b0, reversing changes made to d89d1ee1a3085269cdf44ec50e29a95c8f43757b. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3569: [BEAM-2563] Add integration test for math operators
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3569 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3552: [BEAM-2585] add math function RAND_INTEGER and RAND
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3552 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3560: [BEAM-2562] Add integration test for logical operat...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3560 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3569: [BEAM-2563] Add integration test for math operators
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3569 [BEAM-2563] Add integration test for math operators Add integration test for built-in MATH functions, Misc: 1. no SQRT in Calcite, converted to POWER; 2. add DECIMAL in BeamSqlDslBase; 3. fix error in BeamSqlRoundExpression; 4. fix error in BeamSqlSignExpression; 5. wire on COS with BeamSqlCosExpression; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2563 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3569.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3569 commit d40a324e93c6f151dd4a1846d9f7ce022c9a45a4 Author: mingmxu <ming...@ebay.com> Date: 2017-07-16T22:09:09Z integration test for built-in MATH functions, Misc: 1. no SQRT in Calcite, converted to POWER; 2. add DECIMAL in BeamSqlDslBase; 3. fix error in BeamSqlRoundExpression; 4. fix error in BeamSqlSignExpression; --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3561: [BEAM-2609] refine UDF interface
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3561 [BEAM-2609] refine UDF interface A proposal solution for UDF interface. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2609 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3561.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3561 commit a49e47830e5689b9b392d23813626b1cf9636ca6 Author: mingmxu <ming...@ebay.com> Date: 2017-07-14T06:22:14Z proposal for new UDF --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3553: [BEAM-2610] upgrade to version 2.2.0
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3553 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3553: [BEAM-2610] upgrade to version 2.2.0
GitHub user XuMingmin reopened a pull request: https://github.com/apache/beam/pull/3553 [BEAM-2610] upgrade to version 2.2.0 As described in task [BEAM-2610](https://issues.apache.org/jira/browse/BEAM-2610), this is the first PR to do the job. Feel free to merge it if no mis-operation during creating the PR as any issues are supposed to be fixed in the second PR. Btw, the failure is expected as dsl/sql is still using `2.1.0-SNAPSHOT`, and be fixed in PR 2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3553.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3553 commit eae0d05bd7c088accd927dcfe3e511efbb11c9fd Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-20T09:49:25Z This closes #3391 commit 6e4357225477d6beb4cb9735255d1759f4fab168 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-19T18:56:29Z Retries http code 0 (usually network error) commit c1a2226c90bed7b7bf68a4cd240c849dc46e55ac Author: Luke Cwik <lc...@google.com> Date: 2017-06-20T15:53:59Z Retries http code 0 (usually network error) This closes #3394 commit 5e12e9d75ab78f210b3b024a77c52aaec033218c Author: jasonkuster <jasonkus...@google.com> Date: 2017-06-20T19:05:22Z Remove notifications from JDK versions test. commit 0eb4004a8b91760a66585fb486226513686af002 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-20T19:39:34Z This closes #3403: Remove notifications from JDK versions test. commit b7ff103f6ee10b07c50ddbd5a49a6a8ce6686087 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-16T21:27:51Z Increases backoff in GcsUtil commit 59598d8f41e65f9a068d7446457395e112dc3bc7 Author: Luke Cwik <lc...@google.com> Date: 2017-06-20T20:11:06Z Increases backoff in GcsUtil This closes #3381 commit a0523b2dab617d6aee59708a8d8959f42049fce9 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-06-19T18:24:14Z Fix dataflow runner test to call pipeline.run instead of runner.run commit f51fdd960cbfbb9ab2b2870606bd0e221d4beceb Author: chamik...@google.com <chamik...@google.com> Date: 2017-06-20T20:32:49Z This closes #3393 commit 08ec0d4dbff330ecd48c806cd764ab5a96835bd9 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-20T18:01:03Z Port fn_api_runner to be able to use runner protos. commit e4ef23e16859e31e09e5fe6cf861d6f3db816b22 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-20T20:47:31Z Closes #3361 commit f69e3b53fafa4b79b21095d4b65edbe7cfeb7d2a Author: Pei He <p...@apache.org> Date: 2017-06-19T22:55:48Z FlinkRunner: remove the unused ReflectiveOneToOneOverrideFactory. commit 52794096aa8b4d614423fd787835f5b89b1ea1ac Author: Pei He <p...@apache.org> Date: 2017-06-19T23:10:02Z Flink runner: refactor the translator into two phases: rewriting and translating. commit 608a9c4590ebd94e53ee1ec7f3ad60bfb4905c11 Author: Pei He <p...@apache.org> Date: 2017-06-20T21:12:55Z This closes #3275 commit 42a2de91adf1387bb8eaf9aa515a24f6f276bf40 Author: Mairbek Khadikov <mair...@google.com> Date: 2017-06-14T20:03:36Z Support ValueProviders in SpannerIO.Write commit 10e47646dd5f20d4049d670249cae56c51768ae0 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-20T21:25:56Z This closes #3358: [BEAM-1542] Support ValueProviders in SpannerIO commit 69b01a6118702277348d2f625af669225c9ed99e Author: Reuven Lax <re...@google.com> Date: 2017-05-13T19:53:08Z Add spilling code to WriteFiles. commit 698b89e2b5b88403a5c762b039d3ec8c48b25b26 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-20T21:28:39Z This closes #3161: [BEAM-2302] Add spilling code to WriteFiles. commit a06c8bfae6fb9e35deeb4adfdd7761889b12be89 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-02-02T01:26:55Z [BEAM-1377] Splittable DoFn in Dataflow streaming runner Transform expansion and translation for the involved primitive transforms. Of course, the current PR will only work after the respective Dataflow worker and backend changes are released. commit 4f6032c9c1774a9797e3ff25cc2a05fe56453f21 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-19T15:34:31Z Bump Dataflow worker to 20170619 commit fd40d4b29d3e46dfe25bd7cea65eb7b51dde135f Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-20T23:27:26Z This closes #1898: [BEAM-1377] Splittable DoFn in Dataflow streaming runner commit ef19024d2e9dc046c6699aeee1edc483beb9a009 Author: Ahmet Alta
[GitHub] beam pull request #3560: [BEAM-2562] Add integration test for logical operat...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3560 [BEAM-2562] Add integration test for logical operators R: @xumingming @takidau @jbonofre You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2562 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3560.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3560 commit 79b3bda24f5f0907de22188586cdf2e44ca52f24 Author: mingmxu <ming...@ebay.com> Date: 2017-07-13T16:44:27Z support Types.BOOLEAN, add integration test for logical operations --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3553: [BEAM-2610] upgrade to version 2.2.0
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3553 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3554: [BEAM-2610] upgrade to version 2.2.0
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3554 [BEAM-2610] upgrade to version 2.2.0 placeholder for 2nd PR, may need update after #3553 You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2610 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3554.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3554 commit 26b2b7f047e85d0ae8a9b6e1f921164a39b871b8 Author: mingmxu <ming...@ebay.com> Date: 2017-07-13T02:04:08Z upgrade pom to 2.2.0-SNAPSHOT --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3553: [BEAM-2610] upgrade to version 2.2.0
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3553 [BEAM-2610] upgrade to version 2.2.0 As described in task [BEAM-2610](https://issues.apache.org/jira/browse/BEAM-2610), this is the first PR to do the job. Feel free to merge it if no mis-operation during creating the PR as any issues are supposed to be fixed in the second PR. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3553.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3553 commit 87be64e9817da5e5c86a243471021268d6281b33 Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-05-12T13:21:49Z [BEAM-975] Improve default connection options, javadoc and style in MongoDbIO commit b400f4a6f06e5d826d35045658c4a383cdd5953b Author: Jean-Baptiste Onofré <jbono...@apache.org> Date: 2017-06-19T19:24:08Z [BEAM-975] This closes #3118 commit 6d27282562911179ea3ff19fd7eae54e8b89425d Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-19T14:43:28Z Make HBaseIO tests faster by only using the core daemons needed by HBase commit c6c8d9ee6b60fcc5d019c41ddab697477704f6d8 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-19T21:23:01Z This closes #3392 commit 595ca1ec84134328be5be3c8ae21a5a43a5a7166 Author: Luke Cwik <lc...@google.com> Date: 2017-06-19T22:48:06Z [BEAM-1348] Fix type error introduced into Python SDK because of PR/3268 commit 2304972c5c3b944be0047253687d9cd0004ef21f Author: Luke Cwik <lc...@google.com> Date: 2017-06-19T23:05:25Z [BEAM-1348] Fix type error introduced into Python SDK because of PR/3268 This closes #3397 commit 1ec59a08a3fab5ac0918d7f1a33b82427957b630 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-05T21:48:38Z [BEAM-2411] Make the write transform of HBaseIO simpler commit d42f6333141e85964d009110d8bea85ad4763632 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-20T08:00:15Z [BEAM-2411] Add HBaseCoderProviderRegistrar for better coder inference commit eae0d05bd7c088accd927dcfe3e511efbb11c9fd Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-06-20T09:49:25Z This closes #3391 commit 6e4357225477d6beb4cb9735255d1759f4fab168 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-19T18:56:29Z Retries http code 0 (usually network error) commit c1a2226c90bed7b7bf68a4cd240c849dc46e55ac Author: Luke Cwik <lc...@google.com> Date: 2017-06-20T15:53:59Z Retries http code 0 (usually network error) This closes #3394 commit 5e12e9d75ab78f210b3b024a77c52aaec033218c Author: jasonkuster <jasonkus...@google.com> Date: 2017-06-20T19:05:22Z Remove notifications from JDK versions test. commit 0eb4004a8b91760a66585fb486226513686af002 Author: Kenneth Knowles <k...@google.com> Date: 2017-06-20T19:39:34Z This closes #3403: Remove notifications from JDK versions test. commit b7ff103f6ee10b07c50ddbd5a49a6a8ce6686087 Author: Eugene Kirpichov <kirpic...@google.com> Date: 2017-06-16T21:27:51Z Increases backoff in GcsUtil commit 59598d8f41e65f9a068d7446457395e112dc3bc7 Author: Luke Cwik <lc...@google.com> Date: 2017-06-20T20:11:06Z Increases backoff in GcsUtil This closes #3381 commit a0523b2dab617d6aee59708a8d8959f42049fce9 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-06-19T18:24:14Z Fix dataflow runner test to call pipeline.run instead of runner.run commit f51fdd960cbfbb9ab2b2870606bd0e221d4beceb Author: chamik...@google.com <chamik...@google.com> Date: 2017-06-20T20:32:49Z This closes #3393 commit 08ec0d4dbff330ecd48c806cd764ab5a96835bd9 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-20T18:01:03Z Port fn_api_runner to be able to use runner protos. commit e4ef23e16859e31e09e5fe6cf861d6f3db816b22 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-06-20T20:47:31Z Closes #3361 commit f69e3b53fafa4b79b21095d4b65edbe7cfeb7d2a Author: Pei He <p...@apache.org> Date: 2017-06-19T22:55:48Z FlinkRunner: remove the unused ReflectiveOneToOneOverrideFactory. commit 52794096aa8b4d614423fd787835f5b89b1ea1ac Author: Pei He <p...@apache.org> Date: 2017-06-19T23:10:02Z Flink runner: refactor the translator into two phases: rewriting and translating. commit 608a9c4590ebd94e53ee1ec7f3ad60bfb4905c11 Author: Pei He <p...@apache.org> Date: 2017-06-20T21:12:55Z This closes #3275 commit 42a2de91adf1387bb8eaf9aa515a24f6f276bf40 Author: Mairbek Khadikov <mair...@google.com> Date: 2017-06-14T20:03:36Z Support ValueProviders in Spa
[GitHub] beam pull request #3491: [BEAM-2520] add UDF/UDAF in BeamSql.query/simpleQue...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3491 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3552: [BEAM-2585] add math function RAND_INTEGER and RAND
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3552 [BEAM-2585] add math function RAND_INTEGER and RAND R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2585 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3552.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3552 commit 4f5a87c75eea94f86fc9ac0cea84f48653c16f9a Author: mingmxu <ming...@ebay.com> Date: 2017-07-12T20:36:05Z add RAND/RAND_INTEGER --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3527: [BEAM-2204] support TUMBLE/HOP/SESSION_START
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3527 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3477: [BEAM-2527] Test queries on unbounded PCollections ...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3477 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3530: [BEAM-2574] test unsupported/invalid cases in DSL
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3530 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3530: [BEAM-2574] test unsupported/invalid cases in DSL
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3530 [BEAM-2574] test unsupported/invalid cases in DSL R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2574 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3530.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3530 commit 7fcfaa9770f33c7b07fb1cfcea907cfa4aad43c1 Author: mingmxu <ming...@ebay.com> Date: 2017-07-10T05:26:29Z update unit test to cover exception cases --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3527: [BEAM-2204] support TUMBLE/HOP/SESSION_START
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3527 [BEAM-2204] support TUMBLE/HOP/SESSION_START *_START function is supported for TUMBLE/HOP/SESSION window, however *_END is not available due to some issue with Calcite relational expressions, need a follow up with Calcite team. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2204 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3527.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3527 commit 1d33e7ba1a0efe27d5b38ded15f7e3f4b85b0869 Author: mingmxu <ming...@ebay.com> Date: 2017-07-09T07:52:23Z support TUMBLE/HOP/SESSION _START function --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3491: [BEAM-2520] add UDF/UDAF in BeamSql.query/simpleQue...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3491 [BEAM-2520] add UDF/UDAF in BeamSql.query/simpleQuery You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2520 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3491.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3491 commit f52c45270fa04e938df9c816bbc3e8abcf9d4509 Author: mingmxu <ming...@ebay.com> Date: 2017-07-03T04:24:07Z support UDF/UDAF in BeamSql --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3447: [BEAM-2287] UDAF support
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3447 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3477: [BEAM-2527] Test queries on unbounded PCollections ...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3477 [BEAM-2527] Test queries on unbounded PCollections with BeamSql dsl API R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2527 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3477 commit 57c7e777cdc47975630c24b77ccf2c79381556c0 Author: mingmxu <ming...@ebay.com> Date: 2017-06-29T03:54:02Z add tests with unbounded pcollections. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3447: [BEAM-2287] UDAF support
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3447 [BEAM-2287] UDAF support R: @xumingming @takidau add an abstract class `BeamSqlUdaf` following the UDAF definition in Calcite, also COUNT/SUM/AVG/MAX/MIN/ are rewritten with this new format. Note that the unit test is ignored after rebase BEAM-2446. Will re-open it in BEAM-2520. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2287 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3447.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3447 commit eab43c0830655e4497693a9e24f6a560ba742858 Author: mingmxu <ming...@ebay.com> Date: 2017-06-26T23:03:51Z support of UDAF + rebase --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3427: [BEAM-2503] use static table name in BeamSql.simple...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3427 [BEAM-2503] use static table name in BeamSql.simpleQuery R: @takidau As discussed in #3372, here's the part to use fixed table name `PCOLLECTION` in `BeamSql.simpleQuery`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2503 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3427.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3427 commit 1f612049b83a67070d13aae790d61e0f71d79ca7 Author: mingmxu <ming...@ebay.com> Date: 2017-06-22T23:50:58Z use static table name PCOLLECTION in BeamSql.simpleQuery. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3372: [BEAM-2446] restrict scope of BeamSqlEnv in dsl que...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3372 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3371: [BEAM-2452] Update filter/project/aggregation tests...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3371 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3372: [BEAM-2446] restrict scope of BeamSqlEnv in dsl que...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3372 [BEAM-2446] restrict scope of BeamSqlEnv in dsl query R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2446 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3372.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3372 commit eb5852bd258212f3b50d760ea2b63bbcbd55c481 Author: mingmxu <ming...@ebay.com> Date: 2017-06-14T18:50:31Z restrict the scope of BeamSqlEnv --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3371: [BEAM-2452] Update filter/project/aggregation tests...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3371 [BEAM-2452] Update filter/project/aggregation tests with BeamSql R: @xumingming @takidau You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2435 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3371.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3371 commit 5db678f348be501f2b6d422dba2750b44f8137d8 Author: mingmxu <ming...@ebay.com> Date: 2017-06-15T21:48:42Z update unit test with BeamSql --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3342: [BEAM-2436] table is not regeisted in BeamSql.query
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3342 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3340: [BEAM-2433] unify SQL/Sql words in class name
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3340: [BEAM-2433] unify SQL/Sql words in class name
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3340 [BEAM-2433] unify SQL/Sql words in class name Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2433 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3340.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3340 commit 7b6acc96ad1655816514502670b6322e475beb50 Author: mingmxu <ming...@ebay.com> Date: 2017-06-10T06:07:00Z rename SQL to Sql in class name --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3323: upgrade to 2.1.0-SNAPSHOT
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3323 upgrade to 2.1.0-SNAPSHOT upgrade to version 2.1.0-SNAPSHOT in branch `DSL_SQL`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam upgrade_to_v2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3323 commit 03a913a95c99474841a175b727925ba7c1eed4c9 Author: mingmxu <ming...@ebay.com> Date: 2017-06-08T02:27:32Z upgrade to version 2.1.0-SNAPSHOT --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3314: sync-up DSL_SQL from Master
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3314 sync-up DSL_SQL from Master pull latest code from `master` branch to `DSL_SQL` branch. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3314.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3314 commit 465f4385b9e0004c468507b1f403c5a2af4bdbb0 Author: Thomas Groh <tg...@google.com> Date: 2017-05-23T01:12:11Z This closes #3188 commit bb3db2acc07a61844a551ea9f793a0fcf4001878 Author: Thomas Groh <tg...@google.com> Date: 2017-05-17T22:08:50Z Add WindowIntoTranslator This translates Window.Assign into a WindowIntoPayload. commit 3c10c0bc89783fb6aebb0890ce2fdca2f1b4fb75 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T01:38:27Z This closes #3191: Add WindowIntoTranslator commit a32bef96de6dbb4c413176ed5c0b34e4f1a8f221 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T01:46:30Z This closes #3201: Add ReadTranslator commit 9da46fd053ef3e7538733d45cacad435c312e51b Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T02:02:56Z Fix compile error from bad merge commit 3c0f599d64a7f57608f1c18b05f2ab036a8b02fc Author: Colin Phipps <fi...@google.com> Date: 2017-05-10T09:50:56Z Comply with byte limit for Datastore Commit. commit 597b07e0d9fc3bbce68e06cd10b13ea204f2236f Author: Ahmet Altay <al...@google.com> Date: 2017-05-23T15:39:26Z This closes #3043 commit 6ed15278650c326cf58ec1f2aa37855dbfc204eb Author: Jason Kuster <jasonkus...@google.com> Date: 2017-05-23T00:46:49Z Re-rename fat jar so that install doesn't install the bundled jar as the default jar. Signed-off-by: Jason Kuster <jasonkus...@google.com> commit 16ad7e9466544a5d8d67bd2aa35f4538f6e6a82c Author: Dan Halperin <dhalp...@google.com> Date: 2017-05-23T16:21:53Z This closes #3204 commit 94e619d4cbe90ee5d29af346c978d46ab567c951 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T03:27:52Z Revert "Include Additional PTransform inputs in Transform Nodes" This reverts commit 9336230d2a5c18bae89908bcd60db8ea96b7906d. It appears to break the BigQueryTornadoesIT. commit d0601b30cc46796b887b4290d06b79dbef32c786 Author: Thomas Groh <tg...@google.com> Date: 2017-05-23T16:30:39Z This closes #3206 commit 3a62b4f7b20bda2b3c4ca648f90988d387cfe20d Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-19T00:22:25Z Automatically convert examples to use with syntax. commit 474345f5987e47a22d063c7bfcb3638c85a57e64 Author: Robert Bradshaw <rober...@gmail.com> Date: 2017-05-23T17:19:22Z Closes #3180 commit 98a75551064c742d108d8c5ec8fc0783db7761d2 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:28:44Z Move StepContext to top level commit bed1c53fb47e2d623d6671ce69b82579992df642 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:30:33Z Remove StepContext.noteOutput commit 8b7a1f6dfe0ac33814a0b0c67f37f47ab449ec4b Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:34:37Z Rename BaseExecutionContext.StepContext to BaseStepContext commit 59322d51e80e7480710a296f51a4cb65303f5e06 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:35:46Z Move BaseStepContext to the top level commit 248c808a6603dc2c28a0b55296e0d596b8903a08 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:36:41Z Remove extraneous ExecutionContext parameter to BaseStepContext commit 5ac24e0a89b95feafccbe381bdde9c11fdf82a88 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T22:44:17Z Implement StepContext directly in the DirectRunner commit 32c6cb160f42e401f3e170cc8ed18d76c627d3e4 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T23:26:00Z Remove writePCollectionViewData from the Beam codebase commit 0be3cf3462c19f0b007b2329c95ea4865d22cad5 Author: Kenneth Knowles <k...@google.com> Date: 2017-05-22T23:50:41Z Inline and delete BaseExecutionContext commit 97c230af62151fdbe06ac622282d69c74db30b2f Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T00:33:38Z Delete unused ExecutionContext commit acce24ce1388b7953fbb9d87da5bb2271286c58a Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T00:42:10Z Remove unused StepContext name methods commit dc585510e6b4def3a0442114d77e96f2b5d4880f Author: Kenneth Knowles <k...@google.com> Date: 2017-05-23T00:46:12Z Delete unused remnants in DirectExecutionContext c
[GitHub] beam pull request #3250: [BEAM-2010] expose programming interface
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3250 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3250: [Beam-2010] expose programming interface
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3250 [Beam-2010] expose programming interface It's the DSL implementation described in [Apache Beam: design of DSL SQL interface](https://docs.google.com/document/d/1uWXL_yF3UUO5GfCxbL6kWsmC8xCWfICU3RwiQKsk7Mk/edit?usp=sharing). It provides two methods `BeamSql.simpleQuery(...)` and `BeamSql.query(...)` to integrate SQL in Beam pipeline, by converting it into a `PTransform`, refer to `org.apache.beam.dsls.sql.example.BeamSqlExample` as a quick example. Along with this DSL methods, some functions in the back-end are changed: 1. `BeamSqlRowCoder` is updated to have a input parameter `BeamSQLRecordType`, to declare row format, and it cannot be registered during initialization any more; 2. Interface in `BeamRelNode` is changed, to align with the goal, converting into PTransform; The methods for CLI client are marked as `Experimental` in `BeamSql`, the difference is handled in `BeamIOSourceRel`, that whether an upstream PCollection is there, or need to generate from `BaseBeamTable`; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2010 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3250.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3250 commit 3f0c2c356a83163f165684e07b24bbcc9f759ab7 Author: mingmxu <ming...@ebay.com> Date: 2017-05-27T05:07:03Z DSL interface for Beam SQL --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3137: [BEAM-2288] Refine DSL interface
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3137 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3138: [BEAM-2292] PCollection as a Table
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3138 [BEAM-2292] PCollection as a Table add BeamPCollectionTable to create table from PCollection Note that it changes the interface of `BaseBeamTable`, as a `BeamPCollectionTable` doesn't start from `PBegin`, return `PCollection` is more clear: ``` - public abstract PTransform> buildIOReader(); + public abstract PCollection buildIOReader(Pipeline pipeline); ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2292 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3138.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3138 commit 148e0b30cf1a7e6be3ddb1fd75f4e4265090d708 Author: mingmxu <ming...@ebay.com> Date: 2017-05-14T20:22:30Z add BeamPCollectionTable to create table from PCollection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3137: [BEAM-2288] Refine DSL interface
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3137 [BEAM-2288] Refine DSL interface Refine DSL interface as design doc of BEAM-2010: 1. rename BeamSqlRunner to BeamSQLEnvironment; 2. Move pipeline as external parameter, to integrate with none-DSL components; Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2288 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3137.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3137 commit 4d7f78679008edd04da2006cdaba3336de799a43 Author: mingmxu <ming...@ebay.com> Date: 2017-05-14T18:02:37Z Refine DSL interface as design doc of BEAM-2010: 1. rename BeamSqlRunner to BeamSQLEnvironment; 2. Move pipeline as external parameter, to integrate with none-DSL components; --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3135: [BEAM-2234] expose PCollection of each RelNode
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/3135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3136: [BEAM-2196] support UDF
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3136 [BEAM-2196] support UDF Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2196 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3136.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3136 commit 4cf09e181487783d45fb341c7a6f1f5cc30fb7e0 Author: mingmxu <ming...@ebay.com> Date: 2017-05-14T04:42:05Z support UDF --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3135: [BEAM-2234] expose PCollection of each RelNode
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3135 [BEAM-2234] expose PCollection of each RelNode Change the return of `buildBeamPipeline` to `PCollection`, this helps to: 1). donot need to manage the upstream-downstream in `BeamPipelineCreator` as execution tree already have it; 2). easy for unit test with `PAssert`; Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2234 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3135.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3135 commit ebd263c9e29515038b1ee00dc423da44bef65362 Author: mingmxu <ming...@ebay.com> Date: 2017-05-13T04:14:29Z change return type of buildBeamPipeline to `PCollection` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3067: [BEAM-2008] aggregation functions support
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3067 [BEAM-2008] aggregation functions support Support common-used aggregation functions in SQL, including: `COUNT`, `SUM`, `AVG`, `MAX`, `MIN` Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2008 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3067.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3067 commit 9d918e0e0802e2b6618afdf7d162f911249050d4 Author: mingmxu <ming...@ebay.com> Date: 2017-05-10T23:38:13Z Support common-used aggregation functions in SQL, including: COUNT,SUM,AVG,MAX,MIN --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2903: [BEAM-2006] window support
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2903 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #3039: [BEAM-2006] window support
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3039 [BEAM-2006] window support Add support for aggregation: _global_, _HOP_, _TUMBLE_, _SESSION_, only aggregation function `COUNT` You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2006-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/3039.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3039 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2903: [BEAM-2006] window support
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2903 [BEAM-2006] window support Add support for aggregation: global, `HOP`, `TUMBLE`, `SESSION`, only aggregation function `COUNT` Some other changes: 1. upgrade to Apache Calcite 1.12.0; 2. support data type TIMESTAMP, TIME; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2006 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2903.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2903 commit 7f487cac6ef1991bbfca6f6d00d536d09321056e Author: mingmxu <ming...@ebay.com> Date: 2017-05-04T21:43:36Z Add support for aggregation: global, `HOP`, `TUMBLE`, `SESSION`, only aggregation function `COUNT` Some other changes: 1. upgrade to Apache Calcite 1.12.0; 2. support data type TIMESTAMP, TIME; minor fix of `accept` validation. fix window info in project --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2656: [BEAM-2018] refine expression of Calcite method/fun...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2656 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2878: Forward integrate from `master` to `DSL_SQL`
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2878 Forward integrate from `master` to `DSL_SQL` To fix the unit test failure `org.apache.beam.examples.WordCountIT.testE2EWordCount` You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/beam master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2878.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2878 commit ce88c88b14e963ac17fac83dd19495c835c1f6cb Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T01:13:43Z [BEAM-2005] Move getScheme from FileSystemRegistrar to FileSystem Note that I needed to update FileSystems to instantiate the FileSystem(s) upfront instead of remembering the mapping from scheme to registrar. commit ddabab3faded59e782455486fedf74b2ff60b60c Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T16:00:26Z [BEAM-2005] Move getScheme from FileSystemRegistrar to FileSystem This closes #2757 commit fa422b9a81c303616786e8198d22c70df983d28e Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-04-28T15:25:30Z Remove missing references to the flink examples package after refactor commit d08b72892bf900995667a37239e610bc2cd960d0 Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-04-28T16:26:08Z This closes #2760 commit 594705d717dd6e5b81c08dcf4c56f8f6e7422737 Author: Peter Gergo Barna <pba...@hortonworks.com> Date: 2017-04-27T12:37:26Z [BEAM-2094] WordCount examples produce garbage for non-English input text commit bff84e27ce9a479d0ad0ea51d2cd08bcc970af8e Author: Ahmet Altay <al...@google.com> Date: 2017-04-28T16:33:49Z This closes #2740 commit 34054ab27a53546571b4b687629a0997875479dc Author: Ismaël MejÃa <ieme...@apache.org> Date: 2017-04-26T15:43:53Z Fix Javadoc warnings commit ccfff5f82a56d9027541b17ed38a7590afae8134 Author: Davor Bonaci <da...@google.com> Date: 2017-04-28T16:34:35Z This closes #2705 commit c1ceab7c1957f2c3144307aa324cd4ec1bc49524 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-27T22:58:26Z Add getFilename method to ResourceId commit a8a04a2905e1ff639f403be7d9132acb77365e84 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T17:03:15Z This closes #2754 commit 3423bfd48a3f44e0b58eaef870b6adae45db3af6 Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T17:55:52Z [BEAM-2119] Fix FileSystems to have the local filesystem installed by default. commit 1a265e34f31a32300515c454c619a26d61ba2482 Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T18:01:50Z [BEAM-2119] Fix FileSystems to have the local filesystem installed by default. This closes #2765 commit 4041519c6a4bc232e0956190880fdad7c80ba0f9 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T16:13:47Z ResourceId: Add #isDirectory commit 6b97fb1c224cd996d47fcfb6f6c5ae6e05b0caf0 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T18:44:48Z This closes #2761 commit 83872f467be2043813fd69ef73c0375d343b4b13 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T18:20:00Z TextIOTest: shrink size of test data We have assertions that the files are still big enough, so this is safe. Speeds up TextIOTest 2x. commit f9406c76657e7dc6cd094b1e7281242e3f5fc9e9 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T18:46:12Z This closes #2766 commit 2bd668fdb3048043610cba44b880373666d8a222 Author: Kenneth Knowles <k...@google.com> Date: 2017-04-28T19:35:24Z This closes #2734: Fix ApexTimerInternals serialization error. commit 4e6dbf8e1adb81147202212e3220e763075b1e26 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-28T19:12:43Z Remove IOChannelUtils from a few ITs commit 0c269cc21674b6f1d863e3ddb8ab233303086212 Author: Vikas Kedigehalli <vika...@google.com> Date: 2017-04-28T19:14:24Z Use getFileName in NumberedShardedFile commit dbd44faf32601be6f04c351a0aa3828c2e4f07c9 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T21:13:32Z This closes #2768 commit b16e29cbb7edbc2e06dee1ec8512625731980584 Author: Devon Meunier <devon.meun...@shopify.com> Date: 2017-04-28T19:24:36Z [BEAM-2114] Wrap inferred KafkaIO Coders with NullableCoder commit a3e7383ccae77705259f9830613e78c449fed092 Author: Dan Halperin <dhalp...@google.com> Date: 2017-04-28T21:14:07Z This closes #2767 commit 864e2ad26e3748b414d10c53c28e5e18fb4a53e7 Author: Luke Cwik <lc...@google.com> Date: 2017-04-28T17:30:24Z [BEAM-2031] Add support to pass through zero or more Hadoop configurations through as PipelineOptions comm
[GitHub] beam pull request #2339: [BEAM-1805] a new option `asInnerJoin` for CoGroupB...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2339 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2837: [BEAM-2134] expose AUTO_COMMIT to KafkaIO.read()
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2837 [BEAM-2134] expose AUTO_COMMIT to KafkaIO.read() â¦e visible. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2134 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2837.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2837 commit e2a8d1c5abb130f8bc736caaf7b12df2f58659cc Author: mingmxu <ming...@ebay.com> Date: 2017-05-02T19:01:38Z Update JavaDoc of checkpoint, and expose `enable.auto.commit` for more visible. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #226: [BEAM-2078] add BeamSQL feature branch in site
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam-site/pull/226 [BEAM-2078] add BeamSQL feature branch in site create a new PR as previous #224 edited the wrong file. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam-site BEAM-2078-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/226.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #226 commit ee1f7d63919941bdaa3043d46d824d9c21489f03 Author: mingmxu <ming...@ebay.com> Date: 2017-04-30T02:11:45Z add SQL_DSL to page 'work-in-progress' --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2784: [BEAM-2125] update JavaDoc of BoundedWindow
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2784 [BEAM-2125] update JavaDoc of BoundedWindow Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2125 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2784.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2784 commit cd1e159e4b34c2b3f219dec35388c7f56dccbd41 Author: mingmxu <ming...@ebay.com> Date: 2017-04-30T01:51:21Z update JavaDoc for BoundedWindow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #224: [BEAM-2078] add BeamSQL feature branch in site
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam-site/pull/224 [BEAM-2078] add BeamSQL feature branch in site You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam-site BEAM-2078 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/224.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #224 commit 676f67533790a25f8c5734b28614bd8b73e51c50 Author: mingmxu <ming...@ebay.com> Date: 2017-04-25T21:25:00Z add Beam SQL DSL to page 'work-in-progress' --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2656: [BEAM-2018] refine expression of Calcite method/fun...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2656 [BEAM-2018] refine expression of Calcite method/function In this PR, `BeamSQLFnExecutor` is introduced to replace `BeamSQLSpELExecutor`, as the executor of `SqlOperator` expressions. Several common operators are added here as reference, more are needed to cover all operators defined in `org.apache.calcite.sql.fun.SqlStdOperatorTable`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2018 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2656.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2656 commit 3ce5f7bae2b5a7e13c9dd2abf9d4111739540175 Author: mingmxu <ming...@ebay.com> Date: 2017-04-23T05:20:25Z redesign BeamSqlExpression to execute Calcite SQL expression. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2622: [BEAM-1917] add 'equal' function to Filter
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2622 [BEAM-1917] add 'equal' function to Filter a minor change to add `equal` in [Filter.java](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Filter.java) You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-1917 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2622.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2622 commit 6fd1d57b187915d92e07e051da0550c63fb2be66 Author: mingmxu <ming...@ebay.com> Date: 2017-04-20T21:47:10Z Add `equal` method to Filter transform to trigger travis-ci --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2603: [BEAM-2019] Count.globally() requires default value...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2603 [BEAM-2019] Count.globally() requires default values for non-GlobalWindows fix below error of Count.globally() when not using `GlobalWindows`: ``` Exception in thread "main" java.lang.IllegalStateException: Default values are not supported in Combine.globally() \ if the output PCollection is not windowed by GlobalWindows. Instead, use Combine.globally().withoutDefaults() \ to output an empty PCollection if the input PCollection is empty, or Combine.globally().asSingletonView() \ to get the default output of the CombineFn if the input PCollection is empty. ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-2019 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2603.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2603 commit 9e49ccc67bd7b0e36f8986d896087cc9250dbe13 Author: mingmxu <ming...@ebay.com> Date: 2017-04-19T23:11:03Z Use Combine.globally().withoutDefaults() in Count.globally() --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2458: [Beam-1705]: add method in Pipeline to visualize th...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2458 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2530: [BEAM-301] Add a Beam SQL DSL
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2530 [BEAM-301] Add a Beam SQL DSL 1. add `checkstyle` to `dsls` module, and update as indicated; 2. correct the package name from `org.beam.dsls.sql` to `org.apache.beam.dsls.sql`; You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam rename_package Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2530.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2530 commit 529bc9d9375ae162d70de274975c547a878053a3 Author: mingmxu <ming...@ebay.com> Date: 2017-04-13T21:46:28Z checkstyle and rename package correct package from org.beam.dsls.sql to org.apache.beam.dsls.sql update with checkstyle --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2479: [BEAM-301] Add a Beam SQL DSL
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2479 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2479: [BEAM-301] Add a Beam SQL DSL
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2479 [BEAM-301] Add a Beam SQL DSL The initial PR to setup skeleton of Beam SQL. Pass `mvn verify` locally, a Jenkins job is required later for pre-commit check. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam sql_init_pr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2479.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2479 commit 07904d383bb2b14badb031be2bf5222e2d2c8d03 Author: mingmxu <ming...@ebay.com> Date: 2017-04-10T02:49:08Z initial PR to setup Beam SQL skeleton initial PR to setup Beam SQL skeleton format fix in pom.xml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2478: Dsl sql init
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2478 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2478: Dsl sql init
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2478 Dsl sql init Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam dsl_sql_init Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2478.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2478 commit 57fddbd8e3705254a52e5e478a0dcf9493142203 Author: mingmxu <ming...@ebay.com> Date: 2017-03-09T05:09:03Z save work: BaseBeamTable, BeamRelNode, BeamRule commit 673c25871904124010fe59eed0bcb0c9a9161100 Author: mingmxu <ming...@ebay.com> Date: 2017-03-09T19:27:50Z save work: BeamSQLSpELExecutor as an executor for relational operators; commit 2bce67f5727ecad9fe9973fa4af6022f4f4cd6f5 Author: mingmxu <ming...@ebay.com> Date: 2017-03-09T19:49:57Z save work, update buildBeamPipeline() commit 64f8e89e169151fd323cf1bb45432281a19cedef Author: mingmxu <ming...@ebay.com> Date: 2017-03-09T22:30:10Z complete first query: SELECT FROM WHERE commit 616484c58a265fee31c64d66360689cd3ba71355 Author: mingmxu <ming...@ebay.com> Date: 2017-03-10T21:48:49Z save work: support INSERT commit 1272621c37ba76fd7a1997ba43f19d281e8030aa Author: mingmxu <ming...@ebay.com> Date: 2017-03-10T22:22:04Z update license commit e11dfcf9e1b0aae0eb26ebd2246a914957b8e712 Author: mingmxu <mingm...@gmail.com> Date: 2017-03-11T01:45:46Z save work: add BeamSqlRunner as the tier1 interface; commit 1d80bb5b7813949ca3bb05fd92bdd706d4e43084 Author: mingmxu <ming...@ebay.com> Date: 2017-03-11T07:22:25Z remove test code; commit b51ce8d51c2df1be822e7d28400c4a80145fa3ac Author: mingmxu <ming...@ebay.com> Date: 2017-03-11T19:15:49Z fix checkstyle 1 commit 700ab00abf79e31b9a14683f8d708ace0bda6923 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-11T19:17:29Z Merge pull request #1 from apache/master sync up latest code commit f1afcfe7fe81ac1427a18d367c0536933cf85e63 Author: mingmxu <ming...@ebay.com> Date: 2017-03-12T07:30:27Z refine BaseBeamTable, and add one example BeamSqlExample. commit 3c53db8aa941c3be8ec8171c98cb4405febfac40 Author: mingmxu <ming...@ebay.com> Date: 2017-03-13T02:44:53Z add e2e test cases for planner. commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit d5fe32755b626f79abfd950a71e6b46550fa0d2b Author: mingmxu <ming...@ebay.com> Date: 2017-03-30T18:48:50Z save work commit 83ea09ea195315afac94116a248b4995231e2d72 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-03T22:28:03Z Merge pull request #6 from apache/master rebase commit 8b8347268035ddb37bfc5865306368c5dc2cad67 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-05T21:50:06Z Merge pull request #7 from apache/master rebase commit 38f4b6168f5928d01c01bcffc17a7bee6de596cb Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-07T06:21:44Z Merge pull request #8 from apache/master rebase commit d038df411aff0a68df7c73d6a4f67757923b1ae4 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-07T18:28:21Z Merge pull request #9 from XuMingmin/BEAM-301 rebase from BEAM-301 commit c96fa7bd1da706031cb99f81911c3f01eee86028 Author: mingmxu <ming...@ebay.com> Date: 2017-04-09T04:10:31Z refine BeamSQLRow; commit dd266b5d5b297149b22c9bff4b9621c9075e9e93 Author: mingmxu <ming...@ebay.com> Date: 2017-04-10T00:14:03Z Merge remote-tracking branch 'apache/DSL_SQL' into sql-rowrecord Conflicts: dsls/pom.xml commit 4c2fc05dabfad47779ea1f164828f2228abe9994 Author: mingmxu <ming...@ebay.com> Date: 2017-04-10T00:24:02Z revert files that're not impacted commit 1f55b6d1c6171c3ced46148ef6015a9ae2f85edf Author: mingmxu <ming...@ebay.com> Date: 2017-04-10T00:35:23Z code cleanup: reformat i
[GitHub] beam pull request #2477: [BEAM-1921] expose connectionProperties in JdbcIO
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2477 [BEAM-1921] expose connectionProperties in JdbcIO Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam JdbcIO_Para Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2477 commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit 83ea09ea195315afac94116a248b4995231e2d72 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-03T22:28:03Z Merge pull request #6 from apache/master rebase commit 8b8347268035ddb37bfc5865306368c5dc2cad67 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-05T21:50:06Z Merge pull request #7 from apache/master rebase commit 38f4b6168f5928d01c01bcffc17a7bee6de596cb Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-07T06:21:44Z Merge pull request #8 from apache/master rebase commit 49da33daf0e87c823f04a8b4bcfb03511026461c Author: mingmxu <ming...@ebay.com> Date: 2017-04-09T18:34:26Z add `connectionProperties` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2458: [Beam-1705]: add method in Pipeline to visualize th...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2458 [Beam-1705]: add method in Pipeline to visualize the DAG Add a method `visualizePipeline` in `Pipeline` to visualize data flow. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-1705-2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2458.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2458 commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit 83ea09ea195315afac94116a248b4995231e2d72 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-03T22:28:03Z Merge pull request #6 from apache/master rebase commit 8b8347268035ddb37bfc5865306368c5dc2cad67 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-05T21:50:06Z Merge pull request #7 from apache/master rebase commit 58dc625bd92e0da9c153308800c2eb2fbd39e8bf Author: mingmxu <ming...@ebay.com> Date: 2017-04-07T05:17:52Z add function visualizePipeline --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2411: Quickfix: JavaDoc for function `PaneInfo.getTiming(...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2411 Quickfix: JavaDoc for function `PaneInfo.getTiming()` Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam quickfix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2411.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2411 commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit 69f16e43eaec3bd8ba69103176fda4d160ebc6fb Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-04-03T19:27:11Z JavaDoc for function `PaneInfo.getTiming()` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2374: Quickfix: JavaDoc for function `PaneInfo.getTiming(...
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2374 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2374: Quickfix: JavaDoc for function `PaneInfo.getTiming(...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2374 Quickfix: JavaDoc for function `PaneInfo.getTiming()` Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam quickfix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2374.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2374 commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit 588544abce22ee3bb3cbaaaeb0714f449074feb8 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-30T18:56:07Z Fix JavaDoc for function `getTiming()` commit 0663994ec10b787a44cf6eb81dcf16debf332341 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-30T19:00:16Z Update PaneInfo.java commit 9041213b82a203add756abca502487e84f38305c Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-30T19:02:11Z Update PaneInfo.java --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #195: [BEAM-1797] add CoGroupByKey to chapter 'Using ...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam-site/pull/195 [BEAM-1797] add CoGroupByKey to chapter 'Using GroupByKey' `CoGroupByKey` is the main component to create a JOIN operation in Beam pipeline, add the usage to chapter `Using GroupByKey`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam-site asf-site Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/195.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #195 commit 9ae9977702147e8e791166a9a5ab2b724a3aa14c Author: mingmxu <ming...@ebay.com> Date: 2017-03-18T04:08:18Z add code snippet to page design-your-pipeline commit e95f83e069b1433ffed910721fe61dcee891dcff Author: mingmxu <ming...@ebay.com> Date: 2017-03-21T17:35:51Z consistent words commit 116d9bf8854bf7cde252b3a9ecb1e5e52f8aabcb Author: mingmxu <ming...@ebay.com> Date: 2017-03-21T18:26:28Z restate pipeline multiple-in multiple-out commit f31778f698c0f1a14694a7503583d229144ae982 Author: mingmxu <ming...@ebay.com> Date: 2017-03-21T20:43:12Z correct spelling commit 1f8dcbf013081f5bd1473e31c472b8f9a60177cb Author: mingmxu <ming...@ebay.com> Date: 2017-03-27T22:28:26Z add the usage of CoGroupByKey commit dbd3bd8791d5d1780f2b27aa3bae00a9739eea2b Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-27T22:31:17Z Merge pull request #1 from apache/asf-site sync-up code from main repository --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2339: [BEAM-1805] a new option `asInnerJoin` for CoGroupB...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/2339 [BEAM-1805] a new option `asInnerJoin` for CoGroupByKey *INNER_JOIN* is available with [Join](https://github.com/apache/beam/blob/master/sdks/java/extensions/join-library/src/main/java/org/apache/beam/sdk/extensions/joinlibrary/Join.java). However it's limited to two `PCollection`s, it's helpful to add it to `CoGroupByKey` for the case with multiple `PCollection` joined. You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam BEAM-1805 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam/pull/2339.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2339 commit 09df9b5c15dd5b409d9da937225caa073a663255 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-15T21:45:17Z Merge pull request #3 from apache/master sync-up code commit 93929873179a04fb0038400d57b55b3779d7b085 Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-24T23:17:00Z Merge pull request #4 from apache/master sync up code commit 62ff6bd83725414aa2ca11d30a5bd7776afe99fa Author: XuMingmin <xuming...@users.noreply.github.com> Date: 2017-03-27T18:30:44Z Merge pull request #5 from apache/master sync-up code commit 6f087a085094bd2520094ca0c5c540ae74b31542 Author: mingmxu <ming...@ebay.com> Date: 2017-03-27T20:38:45Z initial commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam-site pull request #188: [BEAM-1744] add code snippet in page Design_You...
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam-site/pull/188 [BEAM-1744] add code snippet in page Design_Your_Pipeline You can merge this pull request into a Git repository by running: $ git pull https://github.com/XuMingmin/beam-site asf-site Alternatively you can review and apply these changes as the patch at: https://github.com/apache/beam-site/pull/188.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #188 commit 9ae9977702147e8e791166a9a5ab2b724a3aa14c Author: mingmxu <ming...@ebay.com> Date: 2017-03-18T04:08:18Z add code snippet to page design-your-pipeline --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] beam pull request #2135: BEAM-1514: change default timestamp in KafkaIO
Github user XuMingmin closed the pull request at: https://github.com/apache/beam/pull/2135 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
