[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-05-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304308#comment-15304308
 ] 

ASF GitHub Bot commented on BEAM-103:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam-site/pull/19


> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Aljoscha Krettek
> Fix For: 0.1.0-incubating
>
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-05-24 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298188#comment-15298188
 ] 

ASF GitHub Bot commented on BEAM-103:
-

GitHub user mxm opened a pull request:

https://github.com/apache/incubator-beam-site/pull/19

[BEAM-103] update capability matrix

This reflects the changes of BEAM-103 in the Capability Matrix.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mxm/incubator-beam-site asf-site

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam-site/pull/19.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19


commit 0ee1aa0f08a941abd21a27ed3ab87d0bde1f6f14
Author: Maximilian Michels 
Date:   2016-05-24T13:41:21Z

[BEAM-103] update capability matrix

This reflects the changes of BEAM-103 in the Capability Matrix.

commit 07f63a1c88b94e9395faa8744ba171b7f5bcfc38
Author: Maximilian Michels 
Date:   2016-05-24T13:43:26Z

rebuild Beam web site




> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Aljoscha Krettek
> Fix For: 0.1.0-incubating
>
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-05-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278810#comment-15278810
 ] 

ASF GitHub Bot commented on BEAM-103:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/274


> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Aljoscha Krettek
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-05-03 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268736#comment-15268736
 ] 

ASF GitHub Bot commented on BEAM-103:
-

GitHub user aljoscha opened a pull request:

https://github.com/apache/incubator-beam/pull/274

[BEAM-103][BEAM-130] Make Flink Source Parallel and Checkpointed

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ x] Replace `` in the title with the actual Jira issue
   number, if there is one
---

CC: @mxm for the Flink parts
CC: @tgroh for the tests, I thought you could have something to say about 
that since you started the doc about better testing

I created a new `TestCountingSource` in the `runner-flink` package because 
I didn't want to depend on the data flow runner. Maybe we should move this to a 
common package.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aljoscha/incubator-beam 
flink-parallel-unbounded-source

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/274.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #274


commit 698f50bd7759e3a036e35ca214b1d727f9db4e51
Author: Aljoscha Krettek 
Date:   2016-05-03T11:35:35Z

[BEAM-103][BEAM-130] Make Flink Source Parallel and Checkpointed




> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Aljoscha Krettek
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-04-29 Thread Aljoscha Krettek (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264040#comment-15264040
 ] 

Aljoscha Krettek commented on BEAM-103:
---

@mxm I don't now, thought so. :-)

> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Kostas Kloudas
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-04-29 Thread Kostas Kloudas (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264019#comment-15264019
 ] 

Kostas Kloudas commented on BEAM-103:
-

I agree that this is pressing. 
I will be able to work on this shortly. 

> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Kostas Kloudas
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-103) Make UnboundedSourceWrapper parallel

2016-04-29 Thread Maximilian Michels (JIRA) 

[ 
https://issues.apache.org/jira/browse/BEAM-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264013#comment-15264013
 ] 

Maximilian Michels commented on BEAM-103:
-

Curious to know where I have mentioned that :) I believe [~kkl0u] wanted to 
work on this and he is currently assigned to the issue. Perhaps too many things 
in the pipeline..

+1 I think this is pressing. That's why I initially opened the issue. Do you 
think you'll have time to work on this soon [~kkl0u]?

> Make UnboundedSourceWrapper parallel
> 
>
> Key: BEAM-103
> URL: https://issues.apache.org/jira/browse/BEAM-103
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Maximilian Michels
>Assignee: Kostas Kloudas
>
> As of now {{UnboundedSource}} s are executed with a parallelism of 1 
> regardless of the splits which the source returns. The corresponding 
> {{UnboundedSourceWrapper}} should implement {{RichParallelSourceFunction}} 
> and deal with splits correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)