[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186731#comment-16186731
 ] 

ASF GitHub Bot commented on BEAM-2455:
--

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/3551


> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>Assignee: Paweł Kaczmarczyk
> Fix For: 2.2.0
>
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084132#comment-16084132
 ] 

ASF GitHub Bot commented on BEAM-2455:
--

GitHub user pawel-kaczmarczyk opened a pull request:

https://github.com/apache/beam/pull/3551

[BEAM-2455] Backlog size retrieval for Kinesis source

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`.
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ocadotechnology/incubator-beam 
kinesis_backlog_size

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/3551.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3551


commit 84974b96f49a4a8b3b430185be75a761c135f913
Author: Pawel Kaczmarczyk 
Date:   2017-07-12T15:16:33Z

[BEAM-2455] Backlog size retrieval for Kinesis source




> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>Assignee: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053943#comment-16053943
 ] 

Paweł Kaczmarczyk commented on BEAM-2455:
-

[~davor] [~jbonofre] Here's a pull request for Kinesis reformatting: 
https://github.com/apache/beam/pull/3389. I'll create a new one with actual 
changes once this is merged.

I also have 3 other improvements for this Kinesis connector reported under 
[BEAM-2467], [BEAM-2468] and [BEAM-2469] so please review the proposals and if 
you agree then please assign me to them.

> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>Assignee: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053078#comment-16053078
 ] 

Jean-Baptiste Onofré commented on BEAM-2455:


It sounds reasonable to me ! Thanks !

> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>Assignee: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[Davor Bonaci (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052952#comment-16052952
 ] 

Davor Bonaci commented on BEAM-2455:


You are welcome to do so! I'd just ask you to do it in separate pull requests 
(as opposed to one giant pull request). Thanks so much!

CC: [~jkff], [~sisk], [~jbonofre].

> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052951#comment-16052951
 ] 

Paweł Kaczmarczyk commented on BEAM-2455:
-

[~davor] yes, that was my intention and I already have the implementation. 
Should I reformat the whole Kinesis source code first to make it compliant with 
Beam code style?

> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-2455) Backlog size retrieval for Kinesis source

[Davor Bonaci (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052931#comment-16052931
 ] 

Davor Bonaci commented on BEAM-2455:


[~pawel.kaczmarczyk], would you perhaps like to contribute this?

> Backlog size retrieval for Kinesis source
> -
>
> Key: BEAM-2455
> URL: https://issues.apache.org/jira/browse/BEAM-2455
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-java-extensions
>Reporter: Paweł Kaczmarczyk
>
> Implement backlog size retrieval for Kinesis source with the use of Amazon 
> CloudWatch. This will allow the runners to scale the amount of resources 
> allocated to the pipeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)