[jira] [Commented] (FLINK-6787) Job-/StoppableException should extend FlinkException
[ https://issues.apache.org/jira/browse/FLINK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152131#comment-16152131 ] ASF GitHub Bot commented on FLINK-6787: --- Github user zjureel closed the pull request at: https://github.com/apache/flink/pull/4500 > Job-/StoppableException should extend FlinkException > > > Key: FLINK-6787 > URL: https://issues.apache.org/jira/browse/FLINK-6787 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: 1.4.0 >Reporter: Chesnay Schepler >Assignee: Fang Yong >Priority: Trivial > Fix For: 1.4.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-6787) Job-/StoppableException should extend FlinkException
[ https://issues.apache.org/jira/browse/FLINK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152130#comment-16152130 ] ASF GitHub Bot commented on FLINK-6787: --- Github user zjureel commented on the issue: https://github.com/apache/flink/pull/4500 Thank you for your merge @aljoscha > Job-/StoppableException should extend FlinkException > > > Key: FLINK-6787 > URL: https://issues.apache.org/jira/browse/FLINK-6787 > Project: Flink > Issue Type: Improvement > Components: Local Runtime >Affects Versions: 1.4.0 >Reporter: Chesnay Schepler >Assignee: Fang Yong >Priority: Trivial > Fix For: 1.4.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] flink pull request #4500: [FLINK-6787] Fix Job-/StoppingException extend Fli...
Github user zjureel closed the pull request at: https://github.com/apache/flink/pull/4500 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4500: [FLINK-6787] Fix Job-/StoppingException extend FlinkExcep...
Github user zjureel commented on the issue: https://github.com/apache/flink/pull/4500 Thank you for your merge @aljoscha --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-3862) Restructure community website
[ https://issues.apache.org/jira/browse/FLINK-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152077#comment-16152077 ] Hai Zhou commented on FLINK-3862: - Hi [~fhue...@gmail.com], Flink IRC channel looks not active, maybe we can consider using Slack instead of IRC? > Restructure community website > - > > Key: FLINK-3862 > URL: https://issues.apache.org/jira/browse/FLINK-3862 > Project: Flink > Issue Type: Task > Components: Project Website >Affects Versions: 1.1.0 >Reporter: Till Rohrmann >Priority: Minor > > The community website contains a large section of third party packages. It > might make sense to create a dedicated third party packages site to declutter > the community site. Furthermore, we should move the IRC communication channel > a bit further down in order to encourage people to rather use other > communication channels. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-7572) Improve TableSchema ValidationException message.
sunjincheng created FLINK-7572:
--
Summary: Improve TableSchema ValidationException message.
Key: FLINK-7572
URL: https://issues.apache.org/jira/browse/FLINK-7572
Project: Flink
Issue Type: Improvement
Reporter: sunjincheng
Assignee: sunjincheng
When sql is very complicated(e.g.: more than 100+ tables), the following
exception information is very confusing:
{code}
org.apache.flink.table.api.TableException: Table column names must be unique.
at org.apache.flink.table.api.TableSchema.(TableSchema.scala:36)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (FLINK-7444) Make external calls non-blocking
[
https://issues.apache.org/jira/browse/FLINK-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7444.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via a3df5a2ca52880b6681446d95f1d916a01f55681
> Make external calls non-blocking
>
>
> Key: FLINK-7444
> URL: https://issues.apache.org/jira/browse/FLINK-7444
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
> Fix For: 1.4.0
>
>
> All external calls from a {{RpcEndpoint}} can be potentially blocking, e.g.
> calls to the {{FatalErrorHandler}}. Therefore, I propose to make all these
> calls coming from the {{RpcEndpoint's}} main thread non-blocking by running
> them in an {{Executor}}. That way the main thread will never be blocked.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7444) Make external calls non-blocking
[
https://issues.apache.org/jira/browse/FLINK-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151994#comment-16151994
]
ASF GitHub Bot commented on FLINK-7444:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4540
> Make external calls non-blocking
>
>
> Key: FLINK-7444
> URL: https://issues.apache.org/jira/browse/FLINK-7444
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
> Fix For: 1.4.0
>
>
> All external calls from a {{RpcEndpoint}} can be potentially blocking, e.g.
> calls to the {{FatalErrorHandler}}. Therefore, I propose to make all these
> calls coming from the {{RpcEndpoint's}} main thread non-blocking by running
> them in an {{Executor}}. That way the main thread will never be blocked.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4540: [FLINK-7444] [rpc] Make external calls non-blockin...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4540 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (FLINK-3182) TaskManagersHandler can block web frontend
[
https://issues.apache.org/jira/browse/FLINK-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann resolved FLINK-3182.
--
Resolution: Duplicate
Fix Version/s: 1.4.0
> TaskManagersHandler can block web frontend
> --
>
> Key: FLINK-3182
> URL: https://issues.apache.org/jira/browse/FLINK-3182
> Project: Flink
> Issue Type: Bug
> Components: Webfrontend
>Affects Versions: 0.10.1
>Reporter: Ufuk Celebi
>Priority: Minor
> Fix For: 1.4.0
>
>
> In {{TaskManagersHandler}}:
> {code}
> Future future = jobManager.ask(
> new JobManagerMessages.RequestTaskManagerInstance(instanceID),
> timeout);
> TaskManagerInstance instance = (TaskManagerInstance) Await.result(
> future,
> timeout);
> {code}
> The default timeout is 10 seconds. Depending on the number of event loop
> threads and requests this can block the web server.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads
[
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151985#comment-16151985
]
ASF GitHub Bot commented on FLINK-7409:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4527
> WebRuntimeMonitor blocks serving threads
>
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
> Issue Type: Improvement
> Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it
> retrieves a result from the {{JobManager}} and then waits on the future to
> obtain the result. This is not a good design since we are blocking server
> threads with that. Instead I propose to follow a more reactive approach where
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}}
> which is in the completion handler written out to the channel.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4527: [FLINK-7409] [web] Make WebRuntimeMonitor reactive
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4527 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (FLINK-7409) WebRuntimeMonitor blocks serving threads
[
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7409.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via ab1fbfdfe6c1f2b6885710f98b9480cb90396ac0
> WebRuntimeMonitor blocks serving threads
>
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
> Issue Type: Improvement
> Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Fix For: 1.4.0
>
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it
> retrieves a result from the {{JobManager}} and then waits on the future to
> obtain the result. This is not a good design since we are blocking server
> threads with that. Instead I propose to follow a more reactive approach where
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}}
> which is in the completion handler written out to the channel.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7078) Support fencing tokens to filter out outdated messages
[ https://issues.apache.org/jira/browse/FLINK-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151979#comment-16151979 ] ASF GitHub Bot commented on FLINK-7078: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4578 > Support fencing tokens to filter out outdated messages > -- > > Key: FLINK-7078 > URL: https://issues.apache.org/jira/browse/FLINK-7078 > Project: Flink > Issue Type: Sub-task > Components: Distributed Coordination >Reporter: Till Rohrmann >Assignee: Till Rohrmann > Labels: flip-6 > Fix For: 1.4.0 > > > In order to guard against split brain situations, it is important that RPC > calls are guarded with a fencing token. The sender attaches his fencing token > to a RPC message which is then used on the receiver side to compare against > the expected fencing token. An example is the leader session ID which we > attach to all critical RPC messages. > So far, in the Flip-6 code base we send fencing tokens explicitly. This is > not only cumbersome but also error-prone because you have to do it for all > RPCs. Therefore, it would be better if we could automatically compare fencing > tokens for a given RPC from a given source. This should ideally happen on the > level of the RPC server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (FLINK-7078) Support fencing tokens to filter out outdated messages
[ https://issues.apache.org/jira/browse/FLINK-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Rohrmann closed FLINK-7078. Resolution: Done Fix Version/s: 1.4.0 Added via 1804aa33d0996810ad35bfa4dd526c86a1e12828 > Support fencing tokens to filter out outdated messages > -- > > Key: FLINK-7078 > URL: https://issues.apache.org/jira/browse/FLINK-7078 > Project: Flink > Issue Type: Sub-task > Components: Distributed Coordination >Reporter: Till Rohrmann >Assignee: Till Rohrmann > Labels: flip-6 > Fix For: 1.4.0 > > > In order to guard against split brain situations, it is important that RPC > calls are guarded with a fencing token. The sender attaches his fencing token > to a RPC message which is then used on the receiver side to compare against > the expected fencing token. An example is the leader session ID which we > attach to all critical RPC messages. > So far, in the Flip-6 code base we send fencing tokens explicitly. This is > not only cumbersome but also error-prone because you have to do it for all > RPCs. Therefore, it would be better if we could automatically compare fencing > tokens for a given RPC from a given source. This should ideally happen on the > level of the RPC server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] flink pull request #4578: [FLINK-7078] [rpc] Introduce FencedRpcEndpoint
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4578 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4578: [FLINK-7078] [rpc] Introduce FencedRpcEndpoint
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4578 Travis passed. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7078) Support fencing tokens to filter out outdated messages
[ https://issues.apache.org/jira/browse/FLINK-7078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151976#comment-16151976 ] ASF GitHub Bot commented on FLINK-7078: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4578 Travis passed. Merging this PR. > Support fencing tokens to filter out outdated messages > -- > > Key: FLINK-7078 > URL: https://issues.apache.org/jira/browse/FLINK-7078 > Project: Flink > Issue Type: Sub-task > Components: Distributed Coordination >Reporter: Till Rohrmann >Assignee: Till Rohrmann > Labels: flip-6 > > In order to guard against split brain situations, it is important that RPC > calls are guarded with a fencing token. The sender attaches his fencing token > to a RPC message which is then used on the receiver side to compare against > the expected fencing token. An example is the leader session ID which we > attach to all critical RPC messages. > So far, in the Flip-6 code base we send fencing tokens explicitly. This is > not only cumbersome but also error-prone because you have to do it for all > RPCs. Therefore, it would be better if we could automatically compare fencing > tokens for a given RPC from a given source. This should ideally happen on the > level of the RPC server. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-7523) test instability in LocalStreamEnvironmentITCase#testRunIsolatedJob
[
https://issues.apache.org/jira/browse/FLINK-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151975#comment-16151975
]
ASF GitHub Bot commented on FLINK-7523:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4596
Rebased onto the latest master.
> test instability in LocalStreamEnvironmentITCase#testRunIsolatedJob
> ---
>
> Key: FLINK-7523
> URL: https://issues.apache.org/jira/browse/FLINK-7523
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Nico Kruber
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: flip-6, test-stability
>
> In a branch with unrelated changes that was created based on the current
> master, {{LocalStreamEnvironmentITCase#testRunIsolatedJob}} seems to fail
> sporadically with
> {code}
> Running
> org.apache.flink.streaming.api.environment.LocalStreamEnvironmentITCase
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.662 sec <<<
> FAILURE! - in
> org.apache.flink.streaming.api.environment.LocalStreamEnvironmentITCase
> testRunIsolatedJob(org.apache.flink.streaming.api.environment.LocalStreamEnvironmentITCase)
> Time elapsed: 1.797 sec <<< ERROR!
> java.util.ConcurrentModificationException: null
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
> at java.util.HashMap$ValueIterator.next(HashMap.java:1466)
> at
> org.apache.flink.runtime.resourcemanager.JobLeaderIdService.clear(JobLeaderIdService.java:120)
> at
> org.apache.flink.runtime.resourcemanager.JobLeaderIdService.stop(JobLeaderIdService.java:98)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManagerRuntimeServices.shutDown(ResourceManagerRuntimeServices.java:51)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManagerRunner.shutDownInternally(ResourceManagerRunner.java:107)
> at
> org.apache.flink.runtime.resourcemanager.ResourceManagerRunner.shutDown(ResourceManagerRunner.java:94)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.shutdownInternally(MiniCluster.java:341)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.shutdown(MiniCluster.java:309)
> at
> org.apache.flink.streaming.api.environment.Flip6LocalStreamEnvironment.execute(Flip6LocalStreamEnvironment.java:124)
> at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1501)
> at
> org.apache.flink.streaming.api.environment.LocalStreamEnvironmentITCase.testRunIsolatedJob(LocalStreamEnvironmentITCase.java:45)
> {code}
> https://travis-ci.org/NicoK/flink/jobs/268340083
> https://transfer.sh/10CvKr/449.1.tar.gz
> relevant parts of the log:
> {code}
>
> Test
> testRunIsolatedJob(org.apache.flink.streaming.api.environment.LocalStreamEnvironmentITCase)
> is running.
>
> 12:38:58,611 INFO
> org.apache.flink.streaming.api.environment.Flip6LocalStreamEnvironment -
> Running job on local embedded Flink mini cluster
> 12:38:58,614 INFO org.apache.flink.runtime.minicluster.MiniCluster
>- Starting Flink Mini Cluster
> 12:38:58,814 INFO org.apache.flink.runtime.minicluster.MiniCluster
>- Starting Metrics Registry
> 12:38:58,822 INFO org.apache.flink.runtime.metrics.MetricRegistry
>- No metrics reporter configured, no metrics will be exposed/reported.
> 12:38:58,823 INFO org.apache.flink.runtime.minicluster.MiniCluster
>- Starting RPC Service(s)
> 12:38:59,191 INFO akka.event.slf4j.Slf4jLogger
>- Slf4jLogger started
> 12:38:59,211 INFO org.apache.flink.runtime.minicluster.MiniCluster
>- Starting high-availability services
> 12:38:59,227 INFO org.apache.flink.runtime.blob.BlobServer
>- Created BLOB server storage directory
> /tmp/blobStore-afe2e280-9b06-4f47-ad6f-a8aa60054470
> 12:38:59,235 INFO org.apache.flink.runtime.blob.BlobServer
>- Started BLOB server at 0.0.0.0:40558 - max concurrent requests: 50 - max
> backlog: 1000
> 12:38:59,236 INFO org.apache.flink.runtime.minicluster.MiniCluster
>- Starting 1 ResourceManger(s)
> 12:38:59,251 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
>- Starting RPC endpoint for
> org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at
> akka://flink/user/resourcemanager_0 .
> 12:38:59,266 INFO
> org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService
> - Proposing
[GitHub] flink issue #4596: [FLINK-7523] Add proper resource shutdown to ResourceMana...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4596 Rebased onto the latest master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7526) TaskExecutor should filter out duplicate JobManager gained leadership messages
[
https://issues.apache.org/jira/browse/FLINK-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151972#comment-16151972
]
ASF GitHub Bot commented on FLINK-7526:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4595
Rebased onto the latest master.
> TaskExecutor should filter out duplicate JobManager gained leadership messages
> --
>
> Key: FLINK-7526
> URL: https://issues.apache.org/jira/browse/FLINK-7526
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> Currently, the {{TaskExecutor}} does not filter out duplicate JobManager
> gained leadership messages. This causes that multiple
> {{JobManagerConnections}} are opened which are not properly closed. In order
> to solve the problem, we should filter out duplicate messages wrt the leader
> session id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4595: [FLINK-7526] [TaskExecutor] Filter out duplicate JobManag...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4595 Rebased onto the latest master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7519) Enrich ClientRestException with HttpResponseStatus
[
https://issues.apache.org/jira/browse/FLINK-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151971#comment-16151971
]
ASF GitHub Bot commented on FLINK-7519:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4588
> Enrich ClientRestException with HttpResponseStatus
> --
>
> Key: FLINK-7519
> URL: https://issues.apache.org/jira/browse/FLINK-7519
> Project: Flink
> Issue Type: Sub-task
> Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{RestClient}} should include the {{HttpResponseStatus}} in its thrown
> {{RestClientException}}. This allows better exception handling on the client
> side (similar to the {{RestHandlerException}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4588: [FLINK-7519] Add HttpResponseStatus to RestClientE...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4588 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (FLINK-7519) Enrich ClientRestException with HttpResponseStatus
[
https://issues.apache.org/jira/browse/FLINK-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7519.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via d7cea586ec60f89ed06baf7ab95ebcf54f42a537
> Enrich ClientRestException with HttpResponseStatus
> --
>
> Key: FLINK-7519
> URL: https://issues.apache.org/jira/browse/FLINK-7519
> Project: Flink
> Issue Type: Sub-task
> Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{RestClient}} should include the {{HttpResponseStatus}} in its thrown
> {{RestClientException}}. This allows better exception handling on the client
> side (similar to the {{RestHandlerException}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7519) Enrich ClientRestException with HttpResponseStatus
[
https://issues.apache.org/jira/browse/FLINK-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151969#comment-16151969
]
ASF GitHub Bot commented on FLINK-7519:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4588
Travis passed. Merging this PR.
> Enrich ClientRestException with HttpResponseStatus
> --
>
> Key: FLINK-7519
> URL: https://issues.apache.org/jira/browse/FLINK-7519
> Project: Flink
> Issue Type: Sub-task
> Components: REST
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> The {{RestClient}} should include the {{HttpResponseStatus}} in its thrown
> {{RestClientException}}. This allows better exception handling on the client
> side (similar to the {{RestHandlerException}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (FLINK-6347) Migrate from Java serialization for MessageAcknowledgingSourceBase's state
[
https://issues.apache.org/jira/browse/FLINK-6347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hai Zhou reassigned FLINK-6347:
---
Assignee: Hai Zhou
> Migrate from Java serialization for MessageAcknowledgingSourceBase's state
> --
>
> Key: FLINK-6347
> URL: https://issues.apache.org/jira/browse/FLINK-6347
> Project: Flink
> Issue Type: Sub-task
> Components: Streaming Connectors
>Reporter: Tzu-Li (Gordon) Tai
>Assignee: Hai Zhou
>
> See umbrella JIRA FLINK-6343 for details. This subtask tracks the migration
> for {{MessageAcknowledgingSourceBase}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (FLINK-6344) Migrate from Java serialization for BucketingSink's state
[ https://issues.apache.org/jira/browse/FLINK-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hai Zhou reassigned FLINK-6344: --- Assignee: Hai Zhou > Migrate from Java serialization for BucketingSink's state > - > > Key: FLINK-6344 > URL: https://issues.apache.org/jira/browse/FLINK-6344 > Project: Flink > Issue Type: Sub-task > Components: Streaming Connectors >Reporter: Tzu-Li (Gordon) Tai >Assignee: Hai Zhou > > See umbrella JIRA FLINK-6343 for details. This subtask tracks the migration > for `BucketingSink`. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] flink issue #4588: [FLINK-7519] Add HttpResponseStatus to RestClientExceptio...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4588 Travis passed. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7522) Add termination future to ClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151968#comment-16151968
]
ASF GitHub Bot commented on FLINK-7522:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4589
> Add termination future to ClusterEntrypoint
> ---
>
> Key: FLINK-7522
> URL: https://issues.apache.org/jira/browse/FLINK-7522
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> In order to wait for the termination of {{ClusterEntrypoint}}, e.g. when
> testing it, a termination future would be helpful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (FLINK-7522) Add termination future to ClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7522.
Resolution: Done
Fix Version/s: 1.4.0
Added via 0cf7f7666a00dbe0d19d5078ef9f14cdb00ed17c
> Add termination future to ClusterEntrypoint
> ---
>
> Key: FLINK-7522
> URL: https://issues.apache.org/jira/browse/FLINK-7522
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> In order to wait for the termination of {{ClusterEntrypoint}}, e.g. when
> testing it, a termination future would be helpful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4589: [FLINK-7522] Add termination future to ClusterEntr...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4589 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7522) Add termination future to ClusterEntrypoint
[
https://issues.apache.org/jira/browse/FLINK-7522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151964#comment-16151964
]
ASF GitHub Bot commented on FLINK-7522:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4589
Travis passed. Merging this PR.
> Add termination future to ClusterEntrypoint
> ---
>
> Key: FLINK-7522
> URL: https://issues.apache.org/jira/browse/FLINK-7522
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> In order to wait for the termination of {{ClusterEntrypoint}}, e.g. when
> testing it, a termination future would be helpful.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7501) Generalize leader id of RegisteredRpcConnection
[
https://issues.apache.org/jira/browse/FLINK-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151963#comment-16151963
]
ASF GitHub Bot commented on FLINK-7501:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4580
> Generalize leader id of RegisteredRpcConnection
> ---
>
> Key: FLINK-7501
> URL: https://issues.apache.org/jira/browse/FLINK-7501
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{RegisteredRpcConnection}} only accepts {{UUIDs}} as leader ids. In
> order to introduce type safe ids for the different Flink components
> ({{JobMaster}}, {{TaskExecutor}}, {{ResourceManager}} and {{Dispatcher}}), we
> should generalize the {{RegisteredRpcConnection}} such that it supports all
> different types of leader ids/fencing tokens.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4589: [FLINK-7522] Add termination future to ClusterEntrypoint
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4589 Travis passed. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request #4580: [FLINK-7501] Generalize RegisteredRpcConnection to...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4580 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Closed] (FLINK-7501) Generalize leader id of RegisteredRpcConnection
[
https://issues.apache.org/jira/browse/FLINK-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7501.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via 0f3de89af4ef4f570e125b0d50110bfa5d0ce80b
> Generalize leader id of RegisteredRpcConnection
> ---
>
> Key: FLINK-7501
> URL: https://issues.apache.org/jira/browse/FLINK-7501
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{RegisteredRpcConnection}} only accepts {{UUIDs}} as leader ids. In
> order to introduce type safe ids for the different Flink components
> ({{JobMaster}}, {{TaskExecutor}}, {{ResourceManager}} and {{Dispatcher}}), we
> should generalize the {{RegisteredRpcConnection}} such that it supports all
> different types of leader ids/fencing tokens.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7501) Generalize leader id of RegisteredRpcConnection
[
https://issues.apache.org/jira/browse/FLINK-7501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151961#comment-16151961
]
ASF GitHub Bot commented on FLINK-7501:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4580
Travis passed. Merging this PR.
> Generalize leader id of RegisteredRpcConnection
> ---
>
> Key: FLINK-7501
> URL: https://issues.apache.org/jira/browse/FLINK-7501
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> The {{RegisteredRpcConnection}} only accepts {{UUIDs}} as leader ids. In
> order to introduce type safe ids for the different Flink components
> ({{JobMaster}}, {{TaskExecutor}}, {{ResourceManager}} and {{Dispatcher}}), we
> should generalize the {{RegisteredRpcConnection}} such that it supports all
> different types of leader ids/fencing tokens.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4580: [FLINK-7501] Generalize RegisteredRpcConnection to suppor...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4580 Travis passed. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7489) Remove job lifecycle methods from public JobMasterGateway interface
[
https://issues.apache.org/jira/browse/FLINK-7489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151959#comment-16151959
]
ASF GitHub Bot commented on FLINK-7489:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4573
> Remove job lifecycle methods from public JobMasterGateway interface
> ---
>
> Key: FLINK-7489
> URL: https://issues.apache.org/jira/browse/FLINK-7489
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{JobMaster}} exposes its job lifecycle methods {{startJobExecution}} and
> {{suspendExecution}} via its {{JobMasterGateway}} to the public. I think
> these methods should not be exposed because it allows other components to
> affect the job execution without proper reason. Only the {{JobManagerRunner}}
> should be responsible for calling these methods and has direct access to the
> {{JobMaster}} instance. Therefore, these methods can directly be implemented
> by the {{JobMaster}} without exposing them via the {{JobMasterGateway}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (FLINK-7489) Remove job lifecycle methods from public JobMasterGateway interface
[
https://issues.apache.org/jira/browse/FLINK-7489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7489.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via 1fc4a609761445ecf77c374ae9daec8a1ada2618
> Remove job lifecycle methods from public JobMasterGateway interface
> ---
>
> Key: FLINK-7489
> URL: https://issues.apache.org/jira/browse/FLINK-7489
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The {{JobMaster}} exposes its job lifecycle methods {{startJobExecution}} and
> {{suspendExecution}} via its {{JobMasterGateway}} to the public. I think
> these methods should not be exposed because it allows other components to
> affect the job execution without proper reason. Only the {{JobManagerRunner}}
> should be responsible for calling these methods and has direct access to the
> {{JobMaster}} instance. Therefore, these methods can directly be implemented
> by the {{JobMaster}} without exposing them via the {{JobMasterGateway}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4579: [FLINK-7500] Set JobMaster leader session id in main thre...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4579 Rebasing this PR onto the latest master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7500) Set JobMaster leader session id in main thread
[
https://issues.apache.org/jira/browse/FLINK-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151960#comment-16151960
]
ASF GitHub Bot commented on FLINK-7500:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4579
Rebasing this PR onto the latest master.
> Set JobMaster leader session id in main thread
> --
>
> Key: FLINK-7500
> URL: https://issues.apache.org/jira/browse/FLINK-7500
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> Currently, the {{JobMaster}} leader id is set via an
> {{AtomicReferenceUpdater}}. In order to make it work with the new
> {{FencedRpcEndpoint}} it should be set in the main thread, because the former
> only allows to modify the leader session id from the main thread.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4573: [FLINK-7489] Remove startJobExecution and suspendE...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4573 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4573: [FLINK-7489] Remove startJobExecution and suspendExecutio...
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4573 Failing test cases are unrelated. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7489) Remove job lifecycle methods from public JobMasterGateway interface
[
https://issues.apache.org/jira/browse/FLINK-7489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151957#comment-16151957
]
ASF GitHub Bot commented on FLINK-7489:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4573
Failing test cases are unrelated. Merging this PR.
> Remove job lifecycle methods from public JobMasterGateway interface
> ---
>
> Key: FLINK-7489
> URL: https://issues.apache.org/jira/browse/FLINK-7489
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: flip-6
>
> The {{JobMaster}} exposes its job lifecycle methods {{startJobExecution}} and
> {{suspendExecution}} via its {{JobMasterGateway}} to the public. I think
> these methods should not be exposed because it allows other components to
> affect the job execution without proper reason. Only the {{JobManagerRunner}}
> should be responsible for calling these methods and has direct access to the
> {{JobMaster}} instance. Therefore, these methods can directly be implemented
> by the {{JobMaster}} without exposing them via the {{JobMasterGateway}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Closed] (FLINK-7457) Make dispatcher highly available
[
https://issues.apache.org/jira/browse/FLINK-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann closed FLINK-7457.
Resolution: Fixed
Fix Version/s: 1.4.0
Fixed via fb3bd1fceba6da362966491e55c3bf27566ede53
> Make dispatcher highly available
>
>
> Key: FLINK-7457
> URL: https://issues.apache.org/jira/browse/FLINK-7457
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The dispatcher component should be made highly available similar to the
> {{ResourceManager}} and the {{JobMasters}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7457) Make dispatcher highly available
[
https://issues.apache.org/jira/browse/FLINK-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151956#comment-16151956
]
ASF GitHub Bot commented on FLINK-7457:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4548
> Make dispatcher highly available
>
>
> Key: FLINK-7457
> URL: https://issues.apache.org/jira/browse/FLINK-7457
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
> Fix For: 1.4.0
>
>
> The dispatcher component should be made highly available similar to the
> {{ResourceManager}} and the {{JobMasters}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4548: [FLINK-7457] Make Dispatcher highly available
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4548 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7457) Make dispatcher highly available
[
https://issues.apache.org/jira/browse/FLINK-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151949#comment-16151949
]
ASF GitHub Bot commented on FLINK-7457:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4548
Travis passed. Merging this PR.
> Make dispatcher highly available
>
>
> Key: FLINK-7457
> URL: https://issues.apache.org/jira/browse/FLINK-7457
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
>
> The dispatcher component should be made highly available similar to the
> {{ResourceManager}} and the {{JobMasters}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4548: [FLINK-7457] Make Dispatcher highly available
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4548 Travis passed. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7444) Make external calls non-blocking
[
https://issues.apache.org/jira/browse/FLINK-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151948#comment-16151948
]
ASF GitHub Bot commented on FLINK-7444:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4540
I've made the `FatalErrorHandler` calls non asynchronous. Rebasing and if
Travis gives green light, I'll merge this PR.
> Make external calls non-blocking
>
>
> Key: FLINK-7444
> URL: https://issues.apache.org/jira/browse/FLINK-7444
> Project: Flink
> Issue Type: Bug
> Components: Distributed Coordination
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
> Labels: flip-6
>
> All external calls from a {{RpcEndpoint}} can be potentially blocking, e.g.
> calls to the {{FatalErrorHandler}}. Therefore, I propose to make all these
> calls coming from the {{RpcEndpoint's}} main thread non-blocking by running
> them in an {{Executor}}. That way the main thread will never be blocked.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4540: [FLINK-7444] [rpc] Make external calls non-blocking
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4540 I've made the `FatalErrorHandler` calls non asynchronous. Rebasing and if Travis gives green light, I'll merge this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter
[ https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151947#comment-16151947 ] Hai Zhou commented on FLINK-4831: - Hi [~Zentol], I checked your code, it's very good. but, I do not know what else need to be modified? I just found via *extends AbstractReporter implements Scheduled*, the implementation would be easier. > Implement a slf4j metric reporter > - > > Key: FLINK-4831 > URL: https://issues.apache.org/jira/browse/FLINK-4831 > Project: Flink > Issue Type: Improvement > Components: Metrics >Affects Versions: 1.1.2 >Reporter: Chesnay Schepler >Priority: Minor > Labels: easyfix, starter > > For debugging purpose it would be very useful to have a log4j metric > reporter. If you don't want to setup a metric backend you currently have to > rely on JMX, which a) works a bit differently than other reporters (for > example it doesn't extend AbstractReporter) and b) makes it a bit tricky to > analyze results as metrics are cleaned up once a job finishes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-7409) WebRuntimeMonitor blocks serving threads
[
https://issues.apache.org/jira/browse/FLINK-7409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151946#comment-16151946
]
ASF GitHub Bot commented on FLINK-7409:
---
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4527
Failing test cases are unrelated. Merging this PR.
> WebRuntimeMonitor blocks serving threads
>
>
> Key: FLINK-7409
> URL: https://issues.apache.org/jira/browse/FLINK-7409
> Project: Flink
> Issue Type: Improvement
> Components: Webfrontend
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Minor
>
> The {{WebRuntimeMonitor}} contains a lot of blocking operations where it
> retrieves a result from the {{JobManager}} and then waits on the future to
> obtain the result. This is not a good design since we are blocking server
> threads with that. Instead I propose to follow a more reactive approach where
> the {{RequestHandler}} returns a {{CompletableFuture}} of {{FullHttpResonse}}
> which is in the completion handler written out to the channel.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4527: [FLINK-7409] [web] Make WebRuntimeMonitor reactive
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/4527 Failing test cases are unrelated. Merging this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7284) Verify compile time and runtime version of Hadoop
[
https://issues.apache.org/jira/browse/FLINK-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151928#comment-16151928
]
Hai Zhou commented on FLINK-7284:
-
similar issue: https://issues.apache.org/jira/browse/FLINK-2581
> Verify compile time and runtime version of Hadoop
> -
>
> Key: FLINK-7284
> URL: https://issues.apache.org/jira/browse/FLINK-7284
> Project: Flink
> Issue Type: Improvement
> Components: Cluster Management
>Affects Versions: 1.4.0
>Reporter: Till Rohrmann
>
> In order to detect a potential version conflict when running a Flink cluster,
> built with Hadoop {{x}}, in an environment which provides Hadoop {{y}}, we
> should automatically check if {{x == y}}. If {{x != y}}, we should terminate
> with an appropriate error message. This behaviour should also be
> disengageable if one wants to run Flink explicitly in a different Hadoop
> environment.
> The check could be done at cluster start up using Hadoops {{VersionInfo}} and
> the build time Hadoop version info. The latter has to be included in the
> Flink binaries.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (FLINK-7568) Bring window documentation up-to-date with latest changes and improve
[
https://issues.apache.org/jira/browse/FLINK-7568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151853#comment-16151853
]
ASF GitHub Bot commented on FLINK-7568:
---
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/4634
R: @alpinegizmo It would probably be very good if you could also have a
look.
> Bring window documentation up-to-date with latest changes and improve
> -
>
> Key: FLINK-7568
> URL: https://issues.apache.org/jira/browse/FLINK-7568
> Project: Flink
> Issue Type: Bug
> Components: Documentation
>Reporter: Aljoscha Krettek
>Assignee: Aljoscha Krettek
>
> Off the top of my head:
> - Make {{ProcessWindowFunction}} the primary window function, threat
> {{WindowFunction}} as legacy
> - Document more specifically how windowing behaves: window boundaries,
> chaining of several windowed operations, timestamps of emitted data, what is
> "key" in a {{ProcessWindowFunction}}...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink issue #4634: [FLINK-7568] Improve Windowing Documentation
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/4634 R: @alpinegizmo It would probably be very good if you could also have a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-7552) Extend SinkFunction interface with SinkContext
[
https://issues.apache.org/jira/browse/FLINK-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151820#comment-16151820
]
ASF GitHub Bot commented on FLINK-7552:
---
Github user pnowojski commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r136717253
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39 @@
*
* @param value The input record.
* @throws Exception
+* @deprecated Use {@link #invoke(SinkContext, Object)}.
*/
- void invoke(IN value) throws Exception;
+ @Deprecated
+ default void invoke(IN value) throws Exception {
+ }
+
+ /**
+* Writes the given value to the sink. This function is called for
every record.
+*
+* @param context Additional context about the input record.
+* @param value The input record.
+* @throws Exception
+*/
+ default void invoke(SinkContext context, IN value) throws Exception {
+ invoke(value);
+ }
+
+ /**
+* Context that {@link SinkFunction SinkFunctions } can use for getting
additional data about
+* an input record.
+*
+* The context is only valid for the duration of a
+* {@link SinkFunction#invoke(SinkContext, Object)} call. Do not store
the context and use
+* afterwards!
+*
+* @param The type of elements accepted by the sink.
+*/
+ @Public // Interface might be extended in the future with additional
methods.
+ interface SinkContext {
+
+ /**
+* Returns the timestamp of the current input record.
+*/
+ long timestamp();
--- End diff --
I would be against unnecessary boxing whenever it is possible. If calling
`timestamp()` doesn't make sense for processing time, then it would be better
to just throw an exception.
> Extend SinkFunction interface with SinkContext
> --
>
> Key: FLINK-7552
> URL: https://issues.apache.org/jira/browse/FLINK-7552
> Project: Flink
> Issue Type: Bug
> Components: DataStream API
>Reporter: Aljoscha Krettek
>Assignee: Aljoscha Krettek
> Fix For: 1.4.0
>
>
> Now that we require Java 8 we can extend the {{SinkFunction}} interface
> without breaking backwards compatibility. I'm proposing this:
> {code}
> /**
> * Interface for implementing user defined sink functionality.
> *
> * @param Input type parameter.
> */
> @Public
> public interface SinkFunction extends Function, Serializable {
> /**
>* Function for standard sink behaviour. This function is called for
> every record.
>*
>* @param value The input record.
>* @throws Exception
>* @deprecated Use {@link #invoke(SinkContext, Object)}.
>*/
> @Deprecated
> default void invoke(IN value) throws Exception {
> }
> /**
>* Writes the given value to the sink. This function is called for
> every record.
>*
>* @param context Additional context about the input record.
>* @param value The input record.
>* @throws Exception
>*/
> default void invoke(SinkContext context, IN value) throws Exception {
> invoke(value);
> }
> /**
>* Context that {@link SinkFunction SinkFunctions } can use for getting
> additional data about
>* an input record.
>*
>* @param The type of elements accepted by the sink.
>*/
> @Public // Interface might be extended in the future with additional
> methods.
> interface SinkContext {
> /**
>* Returns the timestamp of the current input record.
>*/
> long timestamp();
> }
> }
> {code}
> For now, this only allows access to the element timestamp. This would allow
> us to fix the abomination that is {{FlinkKafkaProducer010}}, which is a
> hybrid {{SinkFunction}}/{{StreamOperator}} only because it needs access to
> timestamps.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] flink pull request #4616: [FLINK-7552] [FLINK-7553] Enhance SinkInterface / ...
Github user pnowojski commented on a diff in the pull request:
https://github.com/apache/flink/pull/4616#discussion_r136717253
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/sink/SinkFunction.java
---
@@ -35,6 +35,39 @@
*
* @param value The input record.
* @throws Exception
+* @deprecated Use {@link #invoke(SinkContext, Object)}.
*/
- void invoke(IN value) throws Exception;
+ @Deprecated
+ default void invoke(IN value) throws Exception {
+ }
+
+ /**
+* Writes the given value to the sink. This function is called for
every record.
+*
+* @param context Additional context about the input record.
+* @param value The input record.
+* @throws Exception
+*/
+ default void invoke(SinkContext context, IN value) throws Exception {
+ invoke(value);
+ }
+
+ /**
+* Context that {@link SinkFunction SinkFunctions } can use for getting
additional data about
+* an input record.
+*
+* The context is only valid for the duration of a
+* {@link SinkFunction#invoke(SinkContext, Object)} call. Do not store
the context and use
+* afterwards!
+*
+* @param The type of elements accepted by the sink.
+*/
+ @Public // Interface might be extended in the future with additional
methods.
+ interface SinkContext {
+
+ /**
+* Returns the timestamp of the current input record.
+*/
+ long timestamp();
--- End diff --
I would be against unnecessary boxing whenever it is possible. If calling
`timestamp()` doesn't make sense for processing time, then it would be better
to just throw an exception.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
