[jira] [Commented] (KAFKA-4819) Expose states of active tasks to public API

2017-02-28 Thread Matthias J. Sax (JIRA)

[ 
https://issues.apache.org/jira/browse/KAFKA-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889006#comment-15889006
 ] 

Matthias J. Sax commented on KAFKA-4819:


[~fhussonnois] Thanks for the JIRA an PR. All public API changes require a KIP: 
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals

> Expose states of active tasks to public API
> ---
>
> Key: KAFKA-4819
> URL: https://issues.apache.org/jira/browse/KAFKA-4819
> Project: Kafka
>  Issue Type: New Feature
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Florian Hussonnois
>Priority: Minor
>  Labels: needs-kip
>
> In Kafka 0.10.1.0 the toString method of KafkaStreams class has been 
> implemented mainly to ease topologies debugging. Also,  the streams Metrics 
> has been exposed to public API.
> But currently theres is no way to monitor kstreams tasks states, assignments 
> or consumed offsets.
> I propose to expose the states of active tasks to the public API KafkaStreams.
> For instance, an application can expose a REST API to get the global state of 
> a kstreams topology.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


[jira] [Commented] (KAFKA-4819) Expose states of active tasks to public API

2017-02-28 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/KAFKA-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888916#comment-15888916
 ] 

ASF GitHub Bot commented on KAFKA-4819:
---

GitHub user fhussonnois opened a pull request:

https://github.com/apache/kafka/pull/2612

KAFKA-4819: Expose states for active tasks to public API

Simple implementation of the feature : 
[KAFKA-4819](https://issues.apache.org/jira/browse/KAFKA-4819)
 KAFKA-4819

This PR adds a new method `threadStates` to public API of `KafkaStreams` 
which returns all currently states of running threads and active tasks.

Below is a example for a simple topology consuming from topics; test-p2 and 
test-p4.

[{"name":"StreamThread-1","state":"RUNNING","activeTasks":[{"id":"0_0", 
"assignments":["test-p4-0","test-p2-0"], 
"consumedOffsetsByPartition":[{"topicPartition":"test-p2-0","offset":"test-p2-0"}]},
 {"id":"0_2", "assignments":["test-p4-2"], "consumedOffsetsByPartition":[]}]}, 
{"name":"StreamThread-2","state":"RUNNING","activeTasks":[{"id":"0_1", 
"assignments":["test-p4-1","test-p2-1"], 
"consumedOffsetsByPartition":[{"topicPartition":"test-p2-1","offset":"test-p2-1"}]},
 {"id":"0_3", "assignments":["test-p4-3"], "consumedOffsetsByPartition":[]}]}]

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/fhussonnois/kafka KAFKA-4819

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/2612.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2612


commit 0f8b8123cabdbfcfb44fe59b9be20e13ac253c95
Author: Florian Hussonnois 
Date:   2017-02-23T22:08:01Z

KAFKA-4819: Expose states for active tasks to public API




> Expose states of active tasks to public API
> ---
>
> Key: KAFKA-4819
> URL: https://issues.apache.org/jira/browse/KAFKA-4819
> Project: Kafka
>  Issue Type: New Feature
>  Components: streams
>Affects Versions: 0.10.2.0
>Reporter: Florian Hussonnois
>Priority: Minor
>
> In Kafka 0.10.1.0 the toString method of KafkaStreams class has been 
> implemented mainly to ease topologies debugging. Also,  the streams Metrics 
> has been exposed to public API.
> But currently theres is no way to monitor kstreams tasks states, assignments 
> or consumed offsets.
> I propose to expose the states of active tasks to the public API KafkaStreams.
> For instance, an application can expose a REST API to get the global state of 
> a kstreams topology.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)