[jira] [Updated] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.2

[Sumit Chawla (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Chawla updated BEAM-635:
--
Summary: Release 0.2.0-incubating - Support Flink Release Version 1.1.2  
(was: Release 0.2.0-incubating - Support Flink Release Version 1.1.0)

> Release 0.2.0-incubating - Support Flink Release Version 1.1.2
> --
>
> Key: BEAM-635
> URL: https://issues.apache.org/jira/browse/BEAM-635
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Sumit Chawla
>
> Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
> release
> 0.2.0-incubating has just been released.  It will be great if we can add 
> support for latest Flink 1.1.0 release.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.0

[Sumit Chawla (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Chawla updated BEAM-635:
--
Description: 
Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
release

0.2.0-incubating has just been released.  It will be great if we can add 
support for latest Flink 1.1.0 release.  


  was:
Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
release

0.2.0-incubating has just been release.  It will be great if we can add support 
for latest Flink 1.1.0 release.  



> Release 0.2.0-incubating - Support Flink Release Version 1.1.0
> --
>
> Key: BEAM-635
> URL: https://issues.apache.org/jira/browse/BEAM-635
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Sumit Chawla
>
> Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
> release
> 0.2.0-incubating has just been released.  It will be great if we can add 
> support for latest Flink 1.1.0 release.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1155

[Apache Jenkins Server]

See 

[GitHub] incubator-beam pull request #964: Allow pickling of UnwindowedValues instanc...

[robertwb]

GitHub user robertwb opened a pull request:

https://github.com/apache/incubator-beam/pull/964

Allow pickling of UnwindowedValues instances

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

These are trivial wrappers produced when "windowing" by the global window, 
but may need to be materialized in some cases.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertwb/incubator-beam patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/964.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #964


commit 63904e01f52be498ba3c8c9e1989bcffb02dfcd6
Author: Robert Bradshaw 
Date:   2016-09-16T00:00:42Z

Allow pickling of UnwindowedValues instances

These are trivial wrappers produced when "windowing" by the global window, 
but may need to be materialized in some cases.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.0

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494721#comment-15494721
 ] 

ASF GitHub Bot commented on BEAM-635:
-

GitHub user sumitchawla opened a pull request:

https://github.com/apache/incubator-beam/pull/963

BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Done

Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
release
0.2.0-incubating has just been release. It will be great if we can add 
support for latest Flink 1.1.0 release.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sumitchawla/incubator-beam 
release-0.2.0-incubating

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/963.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #963


commit 76431312a80f62bf93a096be0c846f7a163ae482
Author: Sumit Chawla 
Date:   2016-09-15T22:30:05Z

BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0




> Release 0.2.0-incubating - Support Flink Release Version 1.1.0
> --
>
> Key: BEAM-635
> URL: https://issues.apache.org/jira/browse/BEAM-635
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Sumit Chawla
>
> Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
> release
> 0.2.0-incubating has just been release.  It will be great if we can add 
> support for latest Flink 1.1.0 release.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #963: BEAM-635- Release 0.2.0-incubating - Suppo...

[sumitchawla]

GitHub user sumitchawla opened a pull request:

https://github.com/apache/incubator-beam/pull/963

BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Done

Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
release
0.2.0-incubating has just been release. It will be great if we can add 
support for latest Flink 1.1.0 release.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sumitchawla/incubator-beam 
release-0.2.0-incubating

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/963.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #963


commit 76431312a80f62bf93a096be0c846f7a163ae482
Author: Sumit Chawla 
Date:   2016-09-15T22:30:05Z

BEAM-635- Release 0.2.0-incubating - Support Flink Release Version 1.1.0




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.0

[Sumit Chawla (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Chawla updated BEAM-635:
--
Description: 
Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
release

0.2.0-incubating has just been release.  It will be great if we can add support 
for latest Flink 1.1.0 release.  


  was:
Flink has added support for CodeHale Metrics 
(https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html)

These metrics are more advanced then the current Accumulators. 

Adding support for these to Beam level should be a good addition.

https://github.com/apache/flink/pull/1947#issuecomment-233029166


> Release 0.2.0-incubating - Support Flink Release Version 1.1.0
> --
>
> Key: BEAM-635
> URL: https://issues.apache.org/jira/browse/BEAM-635
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Sumit Chawla
>
> Support Latest Flink Release of version of 1.1.0 in BEAM 0.2.0-incubating 
> release
> 0.2.0-incubating has just been release.  It will be great if we can add 
> support for latest Flink 1.1.0 release.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-635) Release 0.2.0-incubating - Support Flink Release Version 1.1.0

[Sumit Chawla (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sumit Chawla updated BEAM-635:
--
Summary: Release 0.2.0-incubating - Support Flink Release Version 1.1.0  
(was: Support for Flink Release Version 1.1.0)

> Release 0.2.0-incubating - Support Flink Release Version 1.1.0
> --
>
> Key: BEAM-635
> URL: https://issues.apache.org/jira/browse/BEAM-635
> Project: Beam
>  Issue Type: New Feature
>  Components: beam-model
>Reporter: Sumit Chawla
>
> Flink has added support for CodeHale Metrics 
> (https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html)
> These metrics are more advanced then the current Accumulators. 
> Adding support for these to Beam level should be a good addition.
> https://github.com/apache/flink/pull/1947#issuecomment-233029166



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-635) Support for Flink Release Version 1.1.0

[Sumit Chawla (JIRA)]

Sumit Chawla created BEAM-635:
-

 Summary: Support for Flink Release Version 1.1.0
 Key: BEAM-635
 URL: https://issues.apache.org/jira/browse/BEAM-635
 Project: Beam
  Issue Type: New Feature
  Components: beam-model
Reporter: Sumit Chawla


Flink has added support for CodeHale Metrics 
(https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html)

These metrics are more advanced then the current Accumulators. 

Adding support for these to Beam level should be a good addition.

https://github.com/apache/flink/pull/1947#issuecomment-233029166



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #962: [BEAM-] Use sys.executable a...

[chrish42]

GitHub user chrish42 opened a pull request:

https://github.com/apache/incubator-beam/pull/962

[BEAM-] Use sys.executable and "-m pip" to ensure we use the 
same Python and …

The current Python codebase runs subprocesses to call Python and pip. This 
is fine when not running in a virtualenv, or in a virtualenv that was activated 
through modifying the PATH. But it fails if running in a virtualenv either via 
an IDE (I use IntelliJ) or simply when invoking the Python from the virtualenv 
directly (which is a supported way of using a virtualenv).

Instead of relying on the PATH to invoke a Python and hope it's the same 
python executable as the one currently running, Python provides a 
`sys.executable` variable (which gives the full path of the currently running 
python) just for that purpose. Using that, the code will work in all the cases 
described above.

Also, the recommended way of calling `pip` is now `python -m pip`, which 
makes explicit which Python is being used. Switching over to that at the same 
time.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chrish42/incubator-beam use_sys.executable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/962.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #962


commit 557265b1decd1611d7134ac0d73f0c70da2fe9e3
Author: Christian Hudon 
Date:   2016-09-15T19:10:57Z

Use sys.executable and "-m pip" to ensure we use the same Python and pip as 
the currently running one.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1154

[Apache Jenkins Server]

See 

[jira] [Commented] (BEAM-260) Know the getSideInputWindow upper bound so can gc side input state

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494237#comment-15494237
 ] 

Kenneth Knowles commented on BEAM-260:
--

This came up again on the dev list in [this 
thread|http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201609.mbox/%3CCA%2B5xAo0JXSufSTSpa1cQ-yUU%3DgKnTK84HjBqWWL8T3NpiRpbOA%40mail.gmail.com%3E]
 by [~thw]. I have a simple proposed solution at 
https://s.apache.org/beam-windowmappingfn-1-pager. It is small, but may just be 
large enough to merit a \[PROPOSAL\] thread on the mailing list or a "BIP" if 
we ever keep a separate catalog of those.

> Know the getSideInputWindow upper bound so can gc side input state
> --
>
> Key: BEAM-260
> URL: https://issues.apache.org/jira/browse/BEAM-260
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Mark Shields
>Assignee: Kenneth Knowles
>
> We currently have no static knowledge about the getSideInputWindow function, 
> and runners are thus forced to hold on to all side input state / elements in 
> case a future element reaches back into an earlier side input element.
> Maybe we need an upper bound on lag from current to result of 
> getSideInputWindow so we can have a progressing gc horizon as we do for  GKB 
> window state. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1153

[Apache Jenkins Server]

See 

[jira] [Resolved] (BEAM-634) Be able to import Beam codebase in Eclipse and support m2e

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré resolved BEAM-634.
---
Resolution: Fixed

> Be able to import Beam codebase in Eclipse and support m2e
> --
>
> Key: BEAM-634
> URL: https://issues.apache.org/jira/browse/BEAM-634
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.3.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-634) Be able to import Beam codebase in Eclipse and support m2e

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jean-Baptiste Onofré updated BEAM-634:
--
Fix Version/s: 0.3.0-incubating

> Be able to import Beam codebase in Eclipse and support m2e
> --
>
> Key: BEAM-634
> URL: https://issues.apache.org/jira/browse/BEAM-634
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
> Fix For: 0.3.0-incubating
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #949: Update code to import into eclipse using m...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/949


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[2/2] incubator-beam git commit: [BEAM-634] This closes #949

[jbonofre]

[BEAM-634] This closes #949


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/c4036753
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/c4036753
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/c4036753

Branch: refs/heads/master
Commit: c4036753fe95708cfd14bd360c60bdfd7a4ec953
Parents: d71d828 a0ae04b
Author: Jean-Baptiste Onofré 
Authored: Thu Sep 15 16:59:07 2016 +0200
Committer: Jean-Baptiste Onofré 
Committed: Thu Sep 15 16:59:07 2016 +0200

--
 pom.xml | 35 ++--
 runners/direct-java/pom.xml |  8 -
 .../direct/BoundedReadEvaluatorFactory.java |  9 ++---
 .../direct/ParDoMultiEvaluatorFactory.java  | 12 +++
 .../direct/ParDoSingleEvaluatorFactory.java |  9 ++---
 .../direct/UnboundedReadEvaluatorFactory.java   |  9 ++---
 .../apache/beam/runners/flink/package-info.java | 22 
 .../src/main/resources/beam/checkstyle.xml  |  3 ++
 .../src/main/resources/beam/suppressions.xml|  5 +++
 .../beam/sdk/coders/IterableLikeCoder.java  | 22 ++--
 .../beam/sdk/coders/protobuf/ProtoCoder.java|  4 ++-
 .../beam/sdk/util/MergingActiveWindowSet.java   | 12 ---
 .../org/apache/beam/sdk/util/PubsubClient.java  |  2 +-
 .../beam/sdk/util/PubsubJsonClientTest.java | 16 -
 .../apache/beam/sdk/io/kinesis/KinesisIO.java   |  4 ++-
 .../beam/sdk/io/kinesis/KinesisUploader.java|  3 +-
 .../beam/sdk/io/kinesis/package-info.java   | 22 
 .../beam/sdk/io/mongodb/package-info.java   | 22 
 18 files changed, 102 insertions(+), 117 deletions(-)
--

[jira] [Created] (BEAM-634) Be able to import Beam codebase in Eclipse and support m2e

[JIRA]

Jean-Baptiste Onofré created BEAM-634:
-

 Summary: Be able to import Beam codebase in Eclipse and support m2e
 Key: BEAM-634
 URL: https://issues.apache.org/jira/browse/BEAM-634
 Project: Beam
  Issue Type: Improvement
  Components: build-system
Reporter: Jean-Baptiste Onofré
Assignee: Jean-Baptiste Onofré






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-633) Be able to import Beam codebase in Eclipse and support m2e

[JIRA]

Jean-Baptiste Onofré created BEAM-633:
-

 Summary: Be able to import Beam codebase in Eclipse and support m2e
 Key: BEAM-633
 URL: https://issues.apache.org/jira/browse/BEAM-633
 Project: Beam
  Issue Type: Improvement
  Components: build-system
Reporter: Jean-Baptiste Onofré
Assignee: Jean-Baptiste Onofré






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1152

[Apache Jenkins Server]

See 

Re: Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1151

[Dan Halperin]

(filed https://issues.apache.org/jira/browse/BEAM-632)

On Thu, Sep 15, 2016 at 12:14 AM, Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See  GoogleCloudDataflow/changes>
>
>

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1151

[Apache Jenkins Server]

See