[jira] [Commented] (FLINK-34489) New File Sink end-to-end test timed out

2024-02-27 Thread Matthias Pohl (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-34489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821248#comment-17821248
 ] 

Matthias Pohl commented on FLINK-34489:
---

Same with the 2nd case - The TaskManager failed fatally:
{code}
[...]
2024-02-21 05:34:44,967 INFO  
org.apache.flink.runtime.taskexecutor.TaskExecutor   [] - Could not 
resolve ResourceManager address 
pekko.ssl.tcp://flink@localhost:6123/user/rpc/resourcemanager_*, retrying in 
1 ms: Could not connect to rpc endpoint under address 
pekko.ssl.tcp://flink@localhost:6123/user/rpc/resourcemanager_*.
2024-02-21 05:34:54,407 ERROR 
org.apache.flink.runtime.taskexecutor.TaskExecutor   [] - Fatal error 
occurred in TaskExecutor 
pekko.ssl.tcp://flink@localhost:45603/user/rpc/taskmanager_0.
org.apache.flink.runtime.taskexecutor.exceptions.RegistrationTimeoutException: 
Could not register at the ResourceManager within the specified maximum 
registration duration PT5M. This indicates a problem with this instance. 
Terminating now.
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.registrationTimeout(TaskExecutor.java:1614)
 ~[flink-dist-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$startRegistrationTimeout$18(TaskExecutor.java:1599)
 ~[flink-dist-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRunAsync$4(PekkoRpcActor.java:451)
 ~[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68)
 ~[flink-dist-1.19-SNAPSHOT.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRunAsync(PekkoRpcActor.java:451)
 ~[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:218)
 ~[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168)
 ~[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at 
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) 
[flink-rpc-akkafc9c0aa2-35db-418c-8ff3-f8872cca24d7.jar:1.19-SNAPSHOT]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) [?:?]
at 
java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
 [?:?]
at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) [?:?]
at

[jira] [Commented] (FLINK-34489) New File Sink end-to-end test timed out

2024-02-27 Thread Matthias Pohl (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-34489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821201#comment-17821201
 ] 

Matthias Pohl commented on FLINK-34489:
---

The first case is caused by a TaskManager not coming up on time leaving the job 
with an expected parallelism of 4 hanging on 3 TaskManagers/Slots. The failed 
TM reveals the following error logs:
{code}
[...]
2024-02-21 07:30:28,915 INFO  
org.apache.flink.runtime.taskexecutor.TaskExecutor   [] - Could not 
resolve ResourceManager address 
pekko.ssl.tcp://flink@localhost:6123/user/rpc/resourcemanager_*, retrying in 
1 ms: Could not connect to rpc endpoint under address pekko.ssl
.tcp://flink@localhost:6123/user/rpc/resourcemanager_*.
2024-02-21 07:30:38,939 WARN  org.apache.pekko.remote.EndpointReader
   [] - Discarding inbound message to 
[Actor[pekko://flink/temp/_user_rpc_resourcemanager_*$o]] in read-only 
association to [pekko.ssl.tcp://flink@localhost:6123]. If this happens often 
you may 
consider using pekko.remote.use-passive-connections=off or use Artery TCP.
2024-02-21 07:30:48,954 INFO  
org.apache.flink.runtime.taskexecutor.TaskExecutor   [] - Could not 
resolve ResourceManager address 
pekko.ssl.tcp://flink@localhost:6123/user/rpc/resourcemanager_*, retrying in 
1 ms: Could not connect to rpc endpoint under address pekko.ssl
.tcp://flink@localhost:6123/user/rpc/resourcemanager_*.
2024-02-21 07:30:58,353 ERROR 
org.apache.flink.runtime.taskexecutor.TaskExecutor   [] - Fatal error 
occurred in TaskExecutor 
pekko.ssl.tcp://flink@localhost:33341/user/rpc/taskmanager_0.
org.apache.flink.runtime.taskexecutor.exceptions.RegistrationTimeoutException: 
Could not register at the ResourceManager within the specified maximum 
registration duration PT5M. This indicates a problem with this instance. 
Terminating now.
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.registrationTimeout(TaskExecutor.java:1614)
 ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.taskexecutor.TaskExecutor.lambda$startRegistrationTimeout$18(TaskExecutor.java:1599)
 ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.lambda$handleRunAsync$4(PekkoRpcActor.java:451)
 ~[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.concurrent.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68)
 ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRunAsync(PekkoRpcActor.java:451)
 ~[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleRpcMessage(PekkoRpcActor.java:218)
 ~[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.flink.runtime.rpc.pekko.PekkoRpcActor.handleMessage(PekkoRpcActor.java:168)
 ~[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:127) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:126) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:175) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:176) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at 
org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) 
[flink-rpc-akka1f9d7113-8709-4799-8cb2-0d9f15b6f908.jar:1.20-SNAPSHOT]
at

[jira] [Commented] (FLINK-34489) New File Sink end-to-end test timed out

2024-02-21 Thread Matthias Pohl (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-34489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819313#comment-17819313
 ] 

Matthias Pohl commented on FLINK-34489:
---

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57702=logs=dc1bf4ed-4646-531a-f094-e103042be549=fb3d654d-52f8-5b98-fe9d-b18dd2e2b790=3313

> New File Sink end-to-end test timed out
> ---
>
> Key: FLINK-34489
> URL: https://issues.apache.org/jira/browse/FLINK-34489
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.19.0, 1.20.0
>Reporter: Matthias Pohl
>Priority: Critical
>  Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57707=logs=af184cdd-c6d8-5084-0b69-7e9c67b35f7a=0f3adb59-eefa-51c6-2858-3654d9e0749d=3726
> {code}
> eb 21 07:26:03 Number of produced values 10770/6
> Feb 21 07:39:50 Test (pid: 151375) did not finish after 900 seconds.
> Feb 21 07:39:50 Printing Flink logs and killing it:
> [...]
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)