[jira] [Commented] (BEAM-2441) DSL SQL maven module location and name

2017-07-24 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098523#comment-16098523 ] James Xu commented on BEAM-2441: Since there is already many extensions in `sdks/java/extensions/sql`, how

[GitHub] beam pull request #3286: [BEAM-2314] Add ValidatesRunner test for merging cu...

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3286 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] beam pull request #3626: [BEAM-1274] Add SSL mutual authentication support

2017-07-24 Thread jbonofre
GitHub user jbonofre opened a pull request: https://github.com/apache/beam/pull/3626 [BEAM-1274] Add SSL mutual authentication support Follow this checklist to help us incorporate your contribution quickly and easily: - [X] Make sure there is a [JIRA

[jira] [Closed] (BEAM-2314) Add ValidatesRunner test for custom merging windows

2017-07-24 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek closed BEAM-2314. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add ValidatesRunner test for custom merging

[jira] [Assigned] (BEAM-2445) DSL SQL to use service locator pattern to automatically register UDFs

2017-07-24 Thread James Xu (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Xu reassigned BEAM-2445: -- Assignee: James Xu > DSL SQL to use service locator pattern to automatically register UDFs >

[2/2] beam git commit: This closes #3286

2017-07-24 Thread aviemzur
This closes #3286 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/0064fb37 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/0064fb37 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/0064fb37 Branch:

[jira] [Commented] (BEAM-2587) Build fails due to python sdk

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098571#comment-16098571 ] Jean-Baptiste Onofré commented on BEAM-2587: Let's try to fix that for 2.1.0. > Build fails

[1/2] beam git commit: [BEAM-2314] Add ValidatesRunner test for merging custom windows

2017-07-24 Thread aviemzur
Repository: beam Updated Branches: refs/heads/master f54072a1b -> 0064fb37a [BEAM-2314] Add ValidatesRunner test for merging custom windows Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/dfa983ce Tree:

[jira] [Updated] (BEAM-2587) Build fails due to python sdk

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated BEAM-2587: --- Fix Version/s: 2.2.0 > Build fails due to python sdk > - >

[jira] [Updated] (BEAM-2587) Build fails due to python sdk

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated BEAM-2587: --- Fix Version/s: (was: 2.2.0) 2.1.0 > Build fails due to python

[jira] [Commented] (BEAM-2636) user_score on DataflowRunner is broken

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098626#comment-16098626 ] ASF GitHub Bot commented on BEAM-2636: -- GitHub user sb2nov opened a pull request:

[GitHub] beam pull request #3627: [BEAM-2636] Make sure we only override the correct ...

2017-07-24 Thread sb2nov
GitHub user sb2nov opened a pull request: https://github.com/apache/beam/pull/3627 [BEAM-2636] Make sure we only override the correct class Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-1274) Add SSL/TLS support to ElasticSearch IO

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098606#comment-16098606 ] ASF GitHub Bot commented on BEAM-1274: -- GitHub user jbonofre opened a pull request:

[3/4] beam git commit: [BEAM-2571] Respect watermark contract in Flink DoFnOperator

2017-07-24 Thread jbonofre
[BEAM-2571] Respect watermark contract in Flink DoFnOperator In Flink, a watermark T specifies that there will be no elements with a timestamp <= T in the future. In Beam, a watermark T specifies that there will not be element with a timestamp < T in the future. This leads to problems when the

[1/4] beam git commit: [BEAM-2571] Clarify pushedback variable name in DoFnOperator

2017-07-24 Thread jbonofre
Repository: beam Updated Branches: refs/heads/release-2.1.0 608de07e4 -> 1cf560b4b [BEAM-2571] Clarify pushedback variable name in DoFnOperator Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/5c4a95aa Tree:

[4/4] beam git commit: This closes #3625

2017-07-24 Thread jbonofre
This closes #3625 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1cf560b4 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/1cf560b4 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/1cf560b4 Branch:

Build failed in Jenkins: beam_PostCommit_Python_Verify #2792

2017-07-24 Thread Apache Jenkins Server
See Changes: [aviemzur] [BEAM-2314] Add ValidatesRunner test for merging custom windows -- [...truncated 575.96 KB...] File was already downloaded

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2699

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Created] (BEAM-2673) BigQuery Sink should use the Load API

2017-07-24 Thread Sourabh Bajaj (JIRA)
Sourabh Bajaj created BEAM-2673: --- Summary: BigQuery Sink should use the Load API Key: BEAM-2673 URL: https://issues.apache.org/jira/browse/BEAM-2673 Project: Beam Issue Type: Bug

[jira] [Commented] (BEAM-2314) Add ValidatesRunner test for custom merging windows

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098573#comment-16098573 ] ASF GitHub Bot commented on BEAM-2314: -- Github user asfgit closed the pull request at:

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2700

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Created] (BEAM-2672) test merge in BeamSqlUdaf

2017-07-24 Thread Xu Mingmin (JIRA)
Xu Mingmin created BEAM-2672: Summary: test merge in BeamSqlUdaf Key: BEAM-2672 URL: https://issues.apache.org/jira/browse/BEAM-2672 Project: Beam Issue Type: Test Components: dsl-sql

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3644

2017-07-24 Thread Apache Jenkins Server
See

[GitHub] beam pull request #3625: [BEAM-2571] Fix FlinkRunner Watermark issues on 2.1...

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3625 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (BEAM-2571) Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré resolved BEAM-2571. Resolution: Fixed > Flink ValidatesRunner failing

[jira] [Commented] (BEAM-2571) Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098448#comment-16098448 ] ASF GitHub Bot commented on BEAM-2571: -- Github user asfgit closed the pull request at:

[2/4] beam git commit: [BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark

2017-07-24 Thread jbonofre
[BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark This is in line with what Flink does and what BoundedSourceWrapper and UnboundedSourceWrapper do. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1686805d

[jira] [Updated] (BEAM-2445) DSL SQL to use service locator pattern to automatically register UDFs

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik updated BEAM-2445: Labels: (was: dsl_sql_merge) > DSL SQL to use service locator pattern to automatically register UDFs >

[GitHub] beam pull request #3630: We shouldn't write to re-created tables for 2 mins

2017-07-24 Thread sb2nov
GitHub user sb2nov opened a pull request: https://github.com/apache/beam/pull/3630 We shouldn't write to re-created tables for 2 mins Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-2673) BigQuery Sink should use the Load API

2017-07-24 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099069#comment-16099069 ] Ahmet Altay commented on BEAM-2673: --- [~chamikara]] Given the added delay in

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3645

2017-07-24 Thread Apache Jenkins Server
See

Build failed in Jenkins: beam_PerformanceTests_Python #137

2017-07-24 Thread Apache Jenkins Server
See Changes: [lcwik] [BEAM-2662] Fix Java quickstart for Spark to include Spark version [pei] ReshuffleTest: replace Iterable equal tests with matchers. --

[jira] [Closed] (BEAM-2652) cleanup pom.xml

2017-07-24 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin closed BEAM-2652. Resolution: Fixed Fix Version/s: Not applicable > cleanup pom.xml > --- > >

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Flink #3474

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Updated] (BEAM-2652) cleanup pom.xml

2017-07-24 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin updated BEAM-2652: - Labels: dsl_sql_merge dsl_sql_review (was: dsl_sql_review) > cleanup pom.xml > --- > >

Jenkins build became unstable: beam_PostCommit_Java_MavenInstall #4442

2017-07-24 Thread Apache Jenkins Server
See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2695

2017-07-24 Thread Apache Jenkins Server
See

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3641

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2673) BigQuery Sink should use the Load API

2017-07-24 Thread Ahmet Altay (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099098#comment-16099098 ] Ahmet Altay commented on BEAM-2673: --- Is not it also hit when python streaming is used with DataflowRunner

[jira] [Created] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

2017-07-24 Thread Xu Mingmin (JIRA)
Xu Mingmin created BEAM-2676: Summary: move BeamSqlRow and BeamSqlRowType to sdk/java/core Key: BEAM-2676 URL: https://issues.apache.org/jira/browse/BEAM-2676 Project: Beam Issue Type: Test

[jira] [Commented] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

2017-07-24 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099155#comment-16099155 ] Xu Mingmin commented on BEAM-2676: -- [~robertwb] [~lcwik] for any comments > move BeamSqlRow and

[jira] [Resolved] (BEAM-2440) DSL SQL: Reduce visibility to simplify backwards compatibility

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2440. - Resolution: Fixed Fix Version/s: 2.2.0 > DSL SQL: Reduce visibility to simplify backwards

[jira] [Created] (BEAM-2677) AvroIO.read without specifying a schema

2017-07-24 Thread Eugene Kirpichov (JIRA)
Eugene Kirpichov created BEAM-2677: -- Summary: AvroIO.read without specifying a schema Key: BEAM-2677 URL: https://issues.apache.org/jira/browse/BEAM-2677 Project: Beam Issue Type: Bug

[GitHub] beam pull request #3632: [BEAM-2677] AvroIO.parseGenericRecords - schemaless...

2017-07-24 Thread jkff
GitHub user jkff opened a pull request: https://github.com/apache/beam/pull/3632 [BEAM-2677] AvroIO.parseGenericRecords - schemaless AvroIO.read To be done properly, this PR needs https://github.com/apache/beam/pull/3549. R: @mairbek CC: @reuvenlax (please take a look at

[jira] [Created] (BEAM-2674) Runner API translators should own their rehydration

2017-07-24 Thread Kenneth Knowles (JIRA)
Kenneth Knowles created BEAM-2674: - Summary: Runner API translators should own their rehydration Key: BEAM-2674 URL: https://issues.apache.org/jira/browse/BEAM-2674 Project: Beam Issue Type:

[jira] [Commented] (BEAM-2445) DSL SQL to use service locator pattern to automatically register UDFs

2017-07-24 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099079#comment-16099079 ] Xu Mingmin commented on BEAM-2445: -- Talked with Luke, we plan to enable service loader for UDFs in both

[jira] [Commented] (BEAM-2673) BigQuery Sink should use the Load API

2017-07-24 Thread Chamikara Jayalath (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099077#comment-16099077 ] Chamikara Jayalath commented on BEAM-2673: -- I think the fix here is to add a new BQ sink (which

[jira] [Commented] (BEAM-2673) BigQuery Sink should use the Load API

2017-07-24 Thread Sourabh Bajaj (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099103#comment-16099103 ] Sourabh Bajaj commented on BEAM-2673: - No as there is no Truncate mode in streaming, you can only

[jira] [Created] (BEAM-2675) Consolidate ValidatesRunner Groovy Definitions

2017-07-24 Thread Jason Kuster (JIRA)
Jason Kuster created BEAM-2675: -- Summary: Consolidate ValidatesRunner Groovy Definitions Key: BEAM-2675 URL: https://issues.apache.org/jira/browse/BEAM-2675 Project: Beam Issue Type:

[GitHub] beam pull request #3631: [BEAM-2675] Consolidate ValidatesRunner suite build...

2017-07-24 Thread jasonkuster
GitHub user jasonkuster opened a pull request: https://github.com/apache/beam/pull/3631 [BEAM-2675] Consolidate ValidatesRunner suite building into one file. Signed-off-by: Jason Kuster Follow this checklist to help us incorporate your contribution

[jira] [Commented] (BEAM-2675) Consolidate ValidatesRunner Groovy Definitions

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099142#comment-16099142 ] ASF GitHub Bot commented on BEAM-2675: -- GitHub user jasonkuster opened a pull request:

[jira] [Updated] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

2017-07-24 Thread Xu Mingmin (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xu Mingmin updated BEAM-2676: - Description: BeamSqlRow/BeamSqlRowType is the fundamental of structured data processing in Beam, like

[jira] [Resolved] (BEAM-2442) DSL SQL: Public classes/methods should not expose/use calcite types

2017-07-24 Thread Luke Cwik (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Cwik resolved BEAM-2442. - Resolution: Fixed Fix Version/s: 2.2.0 > DSL SQL: Public classes/methods should not expose/use

[jira] [Commented] (BEAM-2677) AvroIO.read without specifying a schema

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099182#comment-16099182 ] ASF GitHub Bot commented on BEAM-2677: -- GitHub user jkff opened a pull request:

[jira] [Resolved] (BEAM-1928) Populate Runner API Components from the Java SDK

2017-07-24 Thread Thomas Groh (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Groh resolved BEAM-1928. --- Resolution: Fixed Fix Version/s: 2.2.0 Components are populated. Future work to use the

[GitHub] beam pull request #3628: [BEAM-2558] Migrate checkCombineFn in TestUtils to ...

2017-07-24 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/3628 [BEAM-2558] Migrate checkCombineFn in TestUtils to CombineFnTester Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-2558) Add a CombineFnTester

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098899#comment-16098899 ] ASF GitHub Bot commented on BEAM-2558: -- GitHub user tgroh opened a pull request:

[jira] [Commented] (BEAM-2575) ApexRunner doesn't emit watermarks for additional outputs

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098906#comment-16098906 ] ASF GitHub Bot commented on BEAM-2575: -- GitHub user tgroh opened a pull request:

[GitHub] beam pull request #3629: Cherry-pick pull #3529 into Release 2.1.0

2017-07-24 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/beam/pull/3629 Cherry-pick pull #3529 into Release 2.1.0 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Build failed in Jenkins: beam_PerformanceTests_Python #139

2017-07-24 Thread Apache Jenkins Server
See Changes: [aviemzur] [BEAM-2314] Add ValidatesRunner test for merging custom windows -- Started by timer [EnvInject] - Loading node environment variables.

[jira] [Commented] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099249#comment-16099249 ] ASF GitHub Bot commented on BEAM-2676: -- GitHub user XuMingmin opened a pull request:

[jira] [Commented] (BEAM-1820) Source.getDefaultOutputCoder() should be @Nullable

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099290#comment-16099290 ] ASF GitHub Bot commented on BEAM-1820: -- GitHub user jkff opened a pull request:

[GitHub] beam pull request #3634: [BEAM-1820] Source.getDefaultOutputCoder() throws C...

2017-07-24 Thread jkff
GitHub user jkff opened a pull request: https://github.com/apache/beam/pull/3634 [BEAM-1820] Source.getDefaultOutputCoder() throws CannotProvideCoderException This is based on https://github.com/apache/beam/pull/3549 by @lgajowy, but it turned out that a lot more needs to be done

[2/5] beam git commit: Add stub DisplayDataTranslation

2017-07-24 Thread kenn
Add stub DisplayDataTranslation Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/efe2dc17 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/efe2dc17 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/efe2dc17 Branch:

[1/5] beam git commit: Fix tests that passed invalid input to DynamicDestinations

2017-07-24 Thread kenn
Repository: beam Updated Branches: refs/heads/master 0064fb37a -> 01408c864 Fix tests that passed invalid input to DynamicDestinations Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/12c277f3 Tree:

[3/5] beam git commit: Add Pipeline rehydration from proto

2017-07-24 Thread kenn
Add Pipeline rehydration from proto Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/43481595 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/43481595 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/43481595

[jira] [Commented] (BEAM-2333) Rehydrate Pipeline from Runner API proto

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099405#comment-16099405 ] ASF GitHub Bot commented on BEAM-2333: -- Github user asfgit closed the pull request at:

[GitHub] beam pull request #3334: [BEAM-2333] Go to proto and back before running a p...

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3334 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[5/5] beam git commit: This closes #3334: [BEAM-2333] Go to proto and back before running a pipeline in Java DirectRunner

2017-07-24 Thread kenn
This closes #3334: [BEAM-2333] Go to proto and back before running a pipeline in Java DirectRunner Dehydrate then rehydrate Pipeline before DirectRunner.run() Add Pipeline rehydration from proto Fix tests that passed invalid input to DynamicDestinations Add stub DisplayDataTranslation

[4/5] beam git commit: Dehydrate then rehydrate Pipeline before DirectRunner.run()

2017-07-24 Thread kenn
Dehydrate then rehydrate Pipeline before DirectRunner.run() Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8ca45915 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/8ca45915 Diff:

[GitHub] beam pull request #3633: [BEAM-2676] move BeamSqlRow and BeamSqlRowType to s...

2017-07-24 Thread XuMingmin
GitHub user XuMingmin opened a pull request: https://github.com/apache/beam/pull/3633 [BEAM-2676] move BeamSqlRow and BeamSqlRowType to sdk/java/core It contains two parts: 1. remove SQL word in the name, `BeamSqlRow` --> `BeamRow` `BeamSqlRowType` --> `BeamRowType`

[GitHub] beam pull request #3635: Runner api read good

2017-07-24 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/3635 Runner api read good Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2701

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-79) Gearpump runner

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099471#comment-16099471 ] ASF GitHub Bot commented on BEAM-79: GitHub user manuzhang opened a pull request:

[GitHub] beam pull request #3637: [BEAM-79] Prepare for merging gearpump-runner

2017-07-24 Thread manuzhang
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3637 [BEAM-79] Prepare for merging gearpump-runner Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

[jira] [Commented] (BEAM-79) Gearpump runner

2017-07-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-79?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099465#comment-16099465 ] ASF GitHub Bot commented on BEAM-79: GitHub user manuzhang opened a pull request:

[GitHub] beam pull request #3636: [BEAM-79] merge gearpump-runner into master

2017-07-24 Thread manuzhang
GitHub user manuzhang opened a pull request: https://github.com/apache/beam/pull/3636 [BEAM-79] merge gearpump-runner into master Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA

Build failed in Jenkins: beam_PerformanceTests_Python #140

2017-07-24 Thread Apache Jenkins Server
See Changes: [klk] Add stub DisplayDataTranslation [klk] Fix tests that passed invalid input to DynamicDestinations [klk] Add Pipeline rehydration from proto [klk] Dehydrate then rehydrate Pipeline

[jira] [Updated] (BEAM-2575) ApexRunner doesn't emit watermarks for additional outputs

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré updated BEAM-2575: --- Fix Version/s: 2.1.0 > ApexRunner doesn't emit watermarks for additional outputs >

[GitHub] beam pull request #3550: [BEAM-2571] CombineTest.testSlidingWindowsCombineWi...

2017-07-24 Thread aljoscha
Github user aljoscha closed the pull request at: https://github.com/apache/beam/pull/3550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] beam pull request #3600: [BEAM-2571] CombineTest.testSlidingWindowsCombineWi...

2017-07-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/beam/pull/3600 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Assigned] (BEAM-2657) Create Solr IO

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía reassigned BEAM-2657: -- Assignee: Cao Manh Dat (was: Davor Bonaci) > Create Solr IO > -- > >

[4/4] beam git commit: This closes #3600

2017-07-24 Thread aljoscha
This closes #3600 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/f54072a1 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/f54072a1 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/f54072a1 Branch:

[jira] [Commented] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098070#comment-16098070 ] Jean-Baptiste Onofré commented on BEAM-2669: Thanks for the details ! > Kryo serialization

[jira] [Assigned] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Baptiste Onofré reassigned BEAM-2671: -- Assignee: Jean-Baptiste Onofré (was: Amit Sela) >

[jira] [Commented] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098110#comment-16098110 ] Jean-Baptiste Onofré commented on BEAM-2671: We saw that and IMHO, it should be fix for 2.1.0

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2697

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Updated] (BEAM-2657) Add Solr IO

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated BEAM-2657: --- Summary: Add Solr IO (was: Create Solr IO) > Add Solr IO > --- > > Key:

[jira] [Created] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread Aviem Zur (JIRA)
Aviem Zur created BEAM-2669: --- Summary: Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached Key: BEAM-2669 URL: https://issues.apache.org/jira/browse/BEAM-2669

[jira] [Updated] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread Aviem Zur (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aviem Zur updated BEAM-2669: Description: Today, when we detect re-use of a dataset in a pipeline in Spark runner we eagerly cache it to

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Spark #2696

2017-07-24 Thread Apache Jenkins Server
See

Jenkins build is still unstable: beam_PostCommit_Java_MavenInstall #4444

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Created] (BEAM-2670) ParDoTest.testPipelineOptionsParameter* validatesRunner tests fail on Spark runner

2017-07-24 Thread Etienne Chauchot (JIRA)
Etienne Chauchot created BEAM-2670: -- Summary: ParDoTest.testPipelineOptionsParameter* validatesRunner tests fail on Spark runner Key: BEAM-2670 URL: https://issues.apache.org/jira/browse/BEAM-2670

[jira] [Created] (BEAM-2671) CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner

2017-07-24 Thread Etienne Chauchot (JIRA)
Etienne Chauchot created BEAM-2671: -- Summary: CreateStreamTest.testFirstElementLate validatesRunner test fails on Spark runner Key: BEAM-2671 URL: https://issues.apache.org/jira/browse/BEAM-2671

[3/4] beam git commit: [BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark

2017-07-24 Thread aljoscha
[BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark This is in line with what Flink does and what BoundedSourceWrapper and UnboundedSourceWrapper do. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/84499317

[1/4] beam git commit: [BEAM-2571] Respect watermark contract in Flink DoFnOperator

2017-07-24 Thread aljoscha
Repository: beam Updated Branches: refs/heads/master 5f1d1365b -> f54072a1b [BEAM-2571] Respect watermark contract in Flink DoFnOperator In Flink, a watermark T specifies that there will be no elements with a timestamp <= T in the future. In Beam, a watermark T specifies that there will not

Jenkins build is still unstable: beam_PostCommit_Java_ValidatesRunner_Dataflow #3642

2017-07-24 Thread Apache Jenkins Server
See

Jenkins build is back to stable : beam_PostCommit_Java_ValidatesRunner_Flink #3475

2017-07-24 Thread Apache Jenkins Server
See

[jira] [Commented] (BEAM-2669) Kryo serialization exception when DStreams containing non-Kryo-serializable data are cached

2017-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/BEAM-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098072#comment-16098072 ] Jean-Baptiste Onofré commented on BEAM-2669: I would be in favor of 2, I don't remember the

Build failed in Jenkins: beam_PerformanceTests_Python #138

2017-07-24 Thread Apache Jenkins Server
See Changes: [aljoscha.krettek] [BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark [aljoscha.krettek] [BEAM-2571] Clarify pushedback variable name in DoFnOperator

  1   2   >