Re: Issue with Flink UI for Flink 1.14.0
That issues tracked under https://issues.apache.org/jira/browse/FLINK-25904.
We don't yet know the cause.
On 18/03/2022 13:37, Peter Westermann wrote:
Just started testing Flink 1.14.4 since that fixes FLINK-25732 … and I
am now running into another UI issue. On the jobmanager that is not
currently the active jobmanager, the checkpoints tab is not working.
It just displays “No Data“, when I use the corresponding API
/jobs/{id}/checkpoints, I get the following error:
{"errors":["Internal server error.","side:\norg.apache.commons.math3.exception.NullArgumentException: input
array\n\tat
org.apache.commons.math3.util.MathArrays.verifyValues(MathArrays.java:1650)\n\tat
org.apache.commons.math3.stat.descriptive.AbstractUnivariateStatistic.test(AbstractUnivariateStatistic.java:158)\n\tat
org.apache.commons.math3.stat.descriptive.rank.Percentile.evaluate(Percentile.java:272)\n\tat
org.apache.commons.math3.stat.descriptive.rank.Percentile.evaluate(Percentile.java:241)\n\tat
org.apache.flink.runtime.metrics.DescriptiveStatisticsHistogramStatistics$CommonMetricsSnapshot.getPercentile(DescriptiveStatisticsHistogramStatistics.java:158)\n\tat
org.apache.flink.runtime.metrics.DescriptiveStatisticsHistogramStatistics.getQuantile(DescriptiveStatisticsHistogramStatistics.java:52)\n\tat
org.apache.flink.runtime.checkpoint.StatsSummarySnapshot.getQuantile(StatsSummarySnapshot.java:108)\n\tat
org.apache.flink.runtime.rest.messages.checkpoints.StatsSummaryDto.valueOf(StatsSummaryDto.java:81)\n\tat
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler.createCheckpointingStatistics(CheckpointingStatisticsHandler.java:129)\n\tat
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler.handleRequest(CheckpointingStatisticsHandler.java:84)\n\tat
org.apache.flink.runtime.rest.handler.job.checkpoints.CheckpointingStatisticsHandler.handleRequest(CheckpointingStatisticsHandler.java:58)\n\tat
org.apache.flink.runtime.rest.handler.job.AbstractAccessExecutionGraphHandler.handleRequest(AbstractAccessExecutionGraphHandler.java:68)\n\tat
org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:87)\n\tat
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)\n\tat
java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)\n\tat
util.TokenAwareRunnable.run(TokenAwareRunnable.java:28)\n\tat
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat
java.base/java.lang.Thread.run(Thread.java:834)\n\nEnd of exception on
server side>"]}
Peter Westermann
Analytics Software Architect
cidimage001.jpg@01D78D4C.C00AC080
peter.westerm...@genesys.com <mailto:peter.westerm...@genesys.com>
cidimage001.jpg@01D78D4C.C00AC080
cidimage002.jpg@01D78D4C.C00AC080 <http://www.genesys.com/>
*From: *Chesnay Schepler
*Date: *Friday, January 21, 2022 at 3:28 AM
*To: *Peter Westermann , Dawid Wysakowicz
, user@flink.apache.org
*Subject: *Re: Issue with Flink UI for Flink 1.14.0
While FLINK-24550 was indeed fixed unfortunately a similar bug was
also introduced (https://issues.apache.org/jira/browse/FLINK-25732).
On 20/01/2022 21:18, Peter Westermann wrote:
Just tried this again with Flink 1.14.3 since
https://issues.apache.org/jira/browse/FLINK-24550 is listed as
fixed. I am running into similar errors when calling the
/v1/jobs/overview endpoint (without any running jobs):
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
peter.westerm...@genesys.com <mailto:peter.westerm...@genesys.com>
<http://www.genesys.com/>
*From: *Dawid Wysakowicz
<mailto:dwysakow...@apache.org>
*Date: *Thursday, October 14, 2021 at 10:00 AM
*To: *Peter Westermann
<mailto:no.westerm...@genesys.com>, user@flink.apache.org
<mailto:user@flink.apache.org>
*Subject: *Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for it
FLINK-24550[1]. I believe we should pick it up soonish. Thanks for
reporting the issue!
Best,
Dawid
[1] https://issues.apache.org/jira/browse/FLINK-24550
On 13/10/2021 20:32, Peter Westermann wrote:
Hello,
I just started testing Flink 1.14.0 and noticed some weird
behavior. This i
Re: Issue with Flink UI for Flink 1.14.0
Just started testing Flink 1.14.4 since that fixes FLINK-25732 … and I am now
running into another UI issue. On the jobmanager that is not currently the
active jobmanager, the checkpoints tab is not working. It just displays “No
Data“, when I use the corresponding API /jobs/{id}/checkpoints, I get the
following error:
{"errors":["Internal server error.",""]}
Peter Westermann
Analytics Software Architect
[cidimage001.jpg@01D78D4C.C00AC080]
peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com>
[cidimage001.jpg@01D78D4C.C00AC080]
[cidimage002.jpg@01D78D4C.C00AC080]<http://www.genesys.com/>
From: Chesnay Schepler
Date: Friday, January 21, 2022 at 3:28 AM
To: Peter Westermann , Dawid Wysakowicz
, user@flink.apache.org
Subject: Re: Issue with Flink UI for Flink 1.14.0
While FLINK-24550 was indeed fixed unfortunately a similar bug was also
introduced
(https://issues.apache.org/jira/browse/FLINK-25732<https://issues.apache.org/jira/browse/FLINK-25732>).
On 20/01/2022 21:18, Peter Westermann wrote:
Just tried this again with Flink 1.14.3 since
https://issues.apache.org/jira/browse/FLINK-24550<https://issues.apache.org/jira/browse/FLINK-24550>
is listed as fixed. I am running into similar errors when calling the
/v1/jobs/overview endpoint (without any running jobs):
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
[cid:part1.0DsTQX9K.Qdi1tLpR@apache.org]
peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com>
[cid:part1.0DsTQX9K.Qdi1tLpR@apache.org]
[cid:part2.iLee7F9y.tfm6J3nj@apache.org]<http://www.genesys.com/>
From: Dawid Wysakowicz <mailto:dwysakow...@apache.org>
Date: Thursday, October 14, 2021 at 10:00 AM
To: Peter Westermann
<mailto:no.westerm...@genesys.com>,
user@flink.apache.org<mailto:user@flink.apache.org>
<mailto:user@flink.apache.org>
Subject: Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for it
FLINK-24550[1]. I believe we should pick it up soonish. Thanks for reporting
the issue!
Best,
Dawid
[1]
https://issues.apache.org/jira/browse/FLINK-24550<https://issues.apache.org/jira/browse/FLINK-24550>
On 13/10/2021 20:32, Peter Westermann wrote:
Hello,
I just started testing Flink 1.14.0 and noticed some weird behavior. This is
for a Flink cluster with zookeeper for HA and two job managers (one leader, one
backup). The UI on the leader works fine. The UI on the other job manager does
not load any job-specific data. Same applies to the REST interface. If I
requests job data from /v1/jobs/{jobId}, I get the expected response on the
leader but on the other job manager, I only get an exception stack trace:
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
[cidimage001.jpg@01D78D4C.C00AC080]
peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com>
[cidimage001.jpg@01D78D4C.C00AC080]
[cidimage002.jpg@01D78D4C.C00AC080]<http://www.genesys.com/>
Re: Issue with Flink UI for Flink 1.14.0
Hi,
This issue is impacting all deployments with 2 JobManagers or more (HA
mode), because in this case serialization is used (well, depending on
the JobManager who is responding, the Leader or a Follower).
It prevents:
* usage of Flink UI
* usage of Flink command "flink.sh list"
* usage of Flink REST API "/jobs/overview"
There are some workaround for all impacts, but that is additional work,
so impact is important.
Should it be possible to release sooner than "planned" ?
Thanks !
--
Guillaume
On 10/02/2022 11.35, Roman Khachatryan wrote:
Hi,
AFAIK there are no plans currently to release 1.14.4.
The previous one (1.14.3) was released on Jan 20, so I'd
1.14.4 preparation to start in the next several weeks.
Regards,
Roman
On Tue, Feb 8, 2022 at 7:31 PM Sweta Kalakuntla
wrote:
I am facing the same issue, do we know when 1.14.4 will be released?
Thanks.
On Fri, Jan 21, 2022 at 3:28 AM Chesnay Schepler
wrote:
While FLINK-24550 was indeed fixed unfortunately a similar bug
was also introduced
(https://issues.apache.org/jira/browse/FLINK-25732).
On 20/01/2022 21:18, Peter Westermann wrote:
Just tried this again with Flink 1.14.3 since
https://issues.apache.org/jira/browse/FLINK-24550 is listed
as fixed. I am running into similar errors when calling the
/v1/jobs/overview endpoint (without any running jobs):
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
cidimage001.jpg@01D78D4C.C00AC080
peter.westerm...@genesys.com
<mailto:peter.westerm...@genesys.com>
cidimage001.jpg@01D78D4C.C00AC080
cidimage002.jpg@01D78D4C.C00AC080 <http://www.genesys.com/>
*From: *Dawid Wysakowicz
<mailto:dwysakow...@apache.org>
*Date: *Thursday, October 14, 2021 at 10:00 AM
*To: *Peter Westermann
<mailto:no.westerm...@genesys.com>, user@flink.apache.org
<mailto:user@flink.apache.org>
*Subject: *Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for
it FLINK-24550[1]. I believe we should pick it up soonish.
Thanks for reporting the issue!
Best,
Dawid
[1] https://issues.apache.org/jira/browse/FLINK-24550
On 13/10/2021 20:32, Peter Westermann wrote:
Hello,
I just started testing Flink 1.14.0 and noticed some
weird behavior. This is for a Flink cluster with
zookeeper for HA and two job managers (one leader, one
backup). The UI on the leader works fine. The UI on the
other job manager does not load any job-specific data.
Same applies to the REST interface. If I requests job
data from /v1/jobs/{jobId}, I get the expected response
on the leader but on the other job manager, I only get an
exception stack trace:
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
cidimage001.jpg@01D78D4C.C00AC080
peter.westerm...@genesys.com
<mailto:peter.westerm...@genesys.com>
cidimage001.jpg@01D78D4C.C00AC080
cidimage002.jpg@01D78D4C.C00AC080 <http://www.genesys.com/>
Re: Issue with Flink UI for Flink 1.14.0
Westermann
>>
>> Team Lead – Realtime Analytics
>>
>> [image: cidimage001.jpg@01D78D4C.C00AC080]
>>
>> peter.westerm...@genesys.com
>>
>> [image: cidimage001.jpg@01D78D4C.C00AC080]
>>
>> [image: cidimage002.jpg@01D78D4C.C00AC080] <http://www.genesys.com/>
>>
>>
>>
>>
>>
>> *From: *Dawid Wysakowicz
>>
>> *Date: *Thursday, October 14, 2021 at 10:00 AM
>> *To: *Peter Westermann
>> , user@flink.apache.org
>>
>> *Subject: *Re: Issue with Flink UI for Flink 1.14.0
>>
>> I am afraid it is a bug in flink 1.14. I created a ticket for it
>> FLINK-24550[1]. I believe we should pick it up soonish. Thanks for
>> reporting the issue!
>>
>> Best,
>>
>> Dawid
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-24550
>>
>> On 13/10/2021 20:32, Peter Westermann wrote:
>>
>> Hello,
>>
>>
>>
>> I just started testing Flink 1.14.0 and noticed some weird behavior. This
>> is for a Flink cluster with zookeeper for HA and two job managers (one
>> leader, one backup). The UI on the leader works fine. The UI on the other
>> job manager does not load any job-specific data. Same applies to the REST
>> interface. If I requests job data from /v1/jobs/{jobId}, I get the expected
>> response on the leader but on the other job manager, I only get an
>> exception stack trace:
>>
>>
>>
>> {"errors":["Internal server error.","> side:\njava.util.concurrent.CancellationException\n\tat
>> java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276)\n\tat
>> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)\n\tat
>> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)\n\tat
>> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)\n\tat
>> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)\n\tat
>> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)\n\tat
>> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)\n\tat
>> java.util.Optional.ifPresent(Optional.java:159)\n\tat
>> org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45)\n\tat
>> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)\n\tat
>> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
>> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)\n\tat
>> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)\n\tat
>> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
>> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
>> org.apache.flink
Re: Issue with Flink UI for Flink 1.14.0
I am facing the same issue, do we know when 1.14.4 will be released?
Thanks.
On Fri, Jan 21, 2022 at 3:28 AM Chesnay Schepler wrote:
> While FLINK-24550 was indeed fixed unfortunately a similar bug was also
> introduced (https://issues.apache.org/jira/browse/FLINK-25732).
> On 20/01/2022 21:18, Peter Westermann wrote:
>
> Just tried this again with Flink 1.14.3 since
> https://issues.apache.org/jira/browse/FLINK-24550 is listed as fixed. I
> am running into similar errors when calling the /v1/jobs/overview endpoint
> (without any running jobs):
>
> {"errors":["Internal server error."," side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
> Failed to serialize the result for RPC call :
> requestMultipleJobDetails.\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat
> java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)\n\tat
> java.base/java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:946)\n\tat
> java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2266)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
> akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat
> akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat
> akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat
> akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat
> akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat
> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)\n\tat
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)\n\tat
> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)\n\tat
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)\n\tat
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)\nCaused
> by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat
> java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)\n\tat
> java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)\n\tat
> java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)\n\tat
> java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)\n\tat
> java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)\n\tat
> java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)\n\tat
> org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t...
> 30 more\n\nEnd of exception on server side>"]}
>
>
>
>
>
>
>
> Peter Westermann
>
> Team Lead – Realtime Analytics
>
> [image: cidimage001.jpg@01D78D4C.C00AC080]
>
> peter.westerm...@genesys.com
>
> [image: cidimage001.jpg@01D78D4C.C00AC080]
>
> [image: cidimage002.jpg@01D78D4C.C00AC080] <http://www.genesys.com/>
>
>
>
>
>
> *From: *Dawid Wysakowicz
> *Date: *Thursday, October 14, 2021 at 10:00 AM
> *To: *Peter Westermann
> , user@flink.apache.org
>
> *Subject: *Re: Issue with Flink UI for Flink 1.14.0
>
> I am afrai
Re: Issue with Flink UI for Flink 1.14.0
While FLINK-24550 was indeed fixed unfortunately a similar bug was also
introduced (https://issues.apache.org/jira/browse/FLINK-25732).
On 20/01/2022 21:18, Peter Westermann wrote:
Just tried this again with Flink 1.14.3 since
https://issues.apache.org/jira/browse/FLINK-24550 is listed as fixed.
I am running into similar errors when calling the /v1/jobs/overview
endpoint (without any running jobs):
{"errors":["Internal server error.","side:\norg.apache.flink.runtime.rpc.akka.exceptions.AkkaRpcException:
Failed to serialize the result for RPC call :
requestMultipleJobDetails.\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:417)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$sendAsyncResponse$2(AkkaRpcActor.java:373)\n\tat
java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:930)\n\tat
java.base/java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:946)\n\tat
java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2266)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.sendAsyncResponse(AkkaRpcActor.java:365)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:332)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:217)\n\tat
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)\n\tat
akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)\n\tat
akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)\n\tat
scala.PartialFunction.applyOrElse(PartialFunction.scala:123)\n\tat
scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)\n\tat
akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)\n\tat
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)\n\tat
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)\n\tat
akka.actor.Actor.aroundReceive(Actor.scala:537)\n\tat
akka.actor.Actor.aroundReceive$(Actor.scala:535)\n\tat
akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)\n\tat
akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)\n\tat
akka.actor.ActorCell.invoke(ActorCell.scala:548)\n\tat
akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)\n\tat
akka.dispatch.Mailbox.run(Mailbox.scala:231)\n\tat
akka.dispatch.Mailbox.exec(Mailbox.scala:243)\n\tat
java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)\n\tat
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)\n\tat
java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)\n\tat
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)\n\tat
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)\nCaused
by: java.io.NotSerializableException: java.util.HashMap$Values\n\tat
java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185)\n\tat
java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553)\n\tat
java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510)\n\tat
java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433)\n\tat
java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)\n\tat
java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349)\n\tat
org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:632)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcSerializedValue.valueOf(AkkaRpcSerializedValue.java:66)\n\tat
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.serializeRemoteResultAndVerifySize(AkkaRpcActor.java:400)\n\t...
30 more\n\nEnd of exception on server side>"]}
Peter Westermann
Team Lead – Realtime Analytics
cidimage001.jpg@01D78D4C.C00AC080
peter.westerm...@genesys.com <mailto:peter.westerm...@genesys.com>
cidimage001.jpg@01D78D4C.C00AC080
cidimage002.jpg@01D78D4C.C00AC080 <http://www.genesys.com/>
*From: *Dawid Wysakowicz
*Date: *Thursday, October 14, 2021 at 10:00 AM
*To: *Peter Westermann ,
user@flink.apache.org
*Subject: *Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for it
FLINK-24550[1]. I believe we should pick it up soonish. Thanks for
reporting the issue!
Best,
Dawid
[1] https://issues.apache.org/jira/browse/FLINK-24550
On 13/10/2021 20:32, Peter Westermann wrote:
Hello,
I just started testing Flink 1.14.0 and noticed some weird
behavior. This is for a Flink cluster with zookeeper for HA and
two job managers (one leader, one backup). The UI on the leader
wor
Re: Issue with Flink UI for Flink 1.14.0
Just tried this again with Flink 1.14.3 since
https://issues.apache.org/jira/browse/FLINK-24550 is listed as fixed. I am
running into similar errors when calling the /v1/jobs/overview endpoint
(without any running jobs):
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
[cidimage001.jpg@01D78D4C.C00AC080]
peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com>
[cidimage001.jpg@01D78D4C.C00AC080]
[cidimage002.jpg@01D78D4C.C00AC080]<http://www.genesys.com/>
From: Dawid Wysakowicz
Date: Thursday, October 14, 2021 at 10:00 AM
To: Peter Westermann , user@flink.apache.org
Subject: Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for it
FLINK-24550[1]. I believe we should pick it up soonish. Thanks for reporting
the issue!
Best,
Dawid
[1] https://issues.apache.org/jira/browse/FLINK-24550
On 13/10/2021 20:32, Peter Westermann wrote:
Hello,
I just started testing Flink 1.14.0 and noticed some weird behavior. This is
for a Flink cluster with zookeeper for HA and two job managers (one leader, one
backup). The UI on the leader works fine. The UI on the other job manager does
not load any job-specific data. Same applies to the REST interface. If I
requests job data from /v1/jobs/{jobId}, I get the expected response on the
leader but on the other job manager, I only get an exception stack trace:
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
[cidimage001.jpg@01D78D4C.C00AC080]
peter.westerm...@genesys.com<mailto:peter.westerm...@genesys.com>
[cidimage001.jpg@01D78D4C.C00AC080]
[cidimage002.jpg@01D78D4C.C00AC080]<http://www.genesys.com/>
Re: Issue with Flink UI for Flink 1.14.0
I am afraid it is a bug in flink 1.14. I created a ticket for it
FLINK-24550[1]. I believe we should pick it up soonish. Thanks for
reporting the issue!
Best,
Dawid
[1] https://issues.apache.org/jira/browse/FLINK-24550
On 13/10/2021 20:32, Peter Westermann wrote:
>
> Hello,
>
>
>
> I just started testing Flink 1.14.0 and noticed some weird behavior.
> This is for a Flink cluster with zookeeper for HA and two job managers
> (one leader, one backup). The UI on the leader works fine. The UI on
> the other job manager does not load any job-specific data. Same
> applies to the REST interface. If I requests job data from
> /v1/jobs/{jobId}, I get the expected response on the leader but on the
> other job manager, I only get an exception stack trace:
>
>
>
> {"errors":["Internal server error."," side:\njava.util.concurrent.CancellationException\n\tat
> java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276)\n\tat
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInternal(DefaultExecutionGraphCache.java:98)\n\tat
> org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache.getExecutionGraphInfo(DefaultExecutionGraphCache.java:67)\n\tat
> org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.handleRequest(AbstractExecutionGraphHandler.java:81)\n\tat
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)\n\tat
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)\n\tat
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)\n\tat
> java.util.Optional.ifPresent(Optional.java:159)\n\tat
> org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45)\n\tat
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)\n\tat
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)\n\tat
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)\n\tat
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
> org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:238)\n\tat
> org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:71)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)\n\tat
> org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)\n\tat
>
Issue with Flink UI for Flink 1.14.0
Hello,
I just started testing Flink 1.14.0 and noticed some weird behavior. This is
for a Flink cluster with zookeeper for HA and two job managers (one leader, one
backup). The UI on the leader works fine. The UI on the other job manager does
not load any job-specific data. Same applies to the REST interface. If I
requests job data from /v1/jobs/{jobId}, I get the expected response on the
leader but on the other job manager, I only get an exception stack trace:
{"errors":["Internal server error.",""]}
Peter Westermann
Team Lead – Realtime Analytics
[cidimage001.jpg@01D78D4C.C00AC080]
peter.westerm...@genesys.com
[cidimage001.jpg@01D78D4C.C00AC080]
[cidimage002.jpg@01D78D4C.C00AC080]
