[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151985#comment-16151985
 ] 

ASF GitHub Bot commented on FLINK-7409:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4527


> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151946#comment-16151946
 ] 

ASF GitHub Bot commented on FLINK-7409:
---

Github user tillrohrmann commented on the issue:

https://github.com/apache/flink/pull/4527
  
Failing test cases are unrelated. Merging this PR.


> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130112#comment-16130112
 ] 

ASF GitHub Bot commented on FLINK-7409:
---

Github user tillrohrmann commented on the issue:

https://github.com/apache/flink/pull/4527
  
Done @greghogan.


> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126587#comment-16126587
 ] 

ASF GitHub Bot commented on FLINK-7409:
---

Github user greghogan commented on the issue:

https://github.com/apache/flink/pull/4527
  
@tillrohrmann please rebase now that #4492 has been merged


> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[Ufuk Celebi (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123101#comment-16123101
 ] 

Ufuk Celebi commented on FLINK-7409:


Thanks! I think it's really nice that you fixed this in your PR :-)

> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[Till Rohrmann (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123061#comment-16123061
 ] 

Till Rohrmann commented on FLINK-7409:
--

Thanks for the pointer [~uce]. Will link the issue.

> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123039#comment-16123039
 ] 

ASF GitHub Bot commented on FLINK-7409:
---

GitHub user tillrohrmann opened a pull request:

https://github.com/apache/flink/pull/4527

[FLINK-7409] [web] Make WebRuntimeMonitor reactive

## What is the purpose of the change

This commit changes the behaviour of the WebRuntimeMonitor to not longer 
block serving
threads by waiting on the result of futures. Instead the RequestHandler now 
returns a
CompletableFuture which is written out to the Netty 
channel upon
completion. This will improve the performance of our WebRuntimeMonitor.

This PR is based #4492.

## Brief change log

- Change signature of `RequestHandler#handleRequest`
- Adapt existing handlers to return async result
- Adapt test cases

## Verifying this change

This change is already covered by existing tests, such as the tests 
contained in `flink-runtime-web`.

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (no)
  - If yes, how is the feature documented? (not applicable)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/tillrohrmann/flink reactiveMonitor

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4527.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4527


commit 734087f37c7cdf2129e87dda5f43dc4dd0c257ba
Author: Till Rohrmann 
Date:   2017-08-02T16:43:00Z

[FLINK-7381] [web] Decouple WebRuntimeMonitor from ActorGateway

This PR decouples the WebRuntimeMonitor from the ActorGateway by introducing
the JobManagerGateway interface which can have multiple implementations. 
This
is a preliminary step for the integration of the existing WebRuntimeMonitor
with the Flip-6 JobMaster.

Add time unit for web.timeout

This closes #4492.

commit e9ea806b09d2f1396de9bd926cced44bd011
Author: Till Rohrmann 
Date:   2017-08-10T08:56:12Z

[FLINK-7409] [web] Make WebRuntimeMonitor reactive

This commit changes the behaviour of the WebRuntimeMonitor to not longer 
block serving
threads by waiting on the result of futures. Instead the RequestHandler now 
returns a
CompletableFuture which is written out to the Netty 
channel upon
completion. This will improve the performance of our WebRuntimeMonitor.




> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads

[Ufuk Celebi (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123037#comment-16123037
 ] 

Ufuk Celebi commented on FLINK-7409:


I brought this issue up a while back too (in FLINK-3182). If we decide to fix 
this now, I would close/link that issue here.

> WebRuntimeMonitor blocks serving threads
> 
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
>  Issue Type: Improvement
>  Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it 
> retrieves a result from the {{JobManager}} and then waits on the future to 
> obtain the result. This is not a good design since we are blocking server 
> threads with that. Instead I propose to follow a more reactive approach where 
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}} 
> which is in the completion handler written out to the channel.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)