[jira] [Created] (FLINK-26274) Test local recovery works across TaskManager process restarts
Till Rohrmann created FLINK-26274: - Summary: Test local recovery works across TaskManager process restarts Key: FLINK-26274 URL: https://issues.apache.org/jira/browse/FLINK-26274 Project: Flink Issue Type: Technical Debt Components: Runtime / Coordination Reporter: Till Rohrmann This ticket is a testing task for [FLIP-201|https://cwiki.apache.org/confluence/x/wJuqCw]. When enabling local recovery and configuring a working directory that can be re-read after a process failure, Flink should now be able to recover locally. We should test whether this is the case. Please take a look at the documentation [1, 2] to see how to configure Flink to make use of it. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/working_directory/ [2] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/kubernetes/#enabling-local-recovery-across-pod-restarts -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26271) TaskTest.testCancelTaskExceptionAfterTaskMarkedFailed
Till Rohrmann created FLINK-26271: - Summary: TaskTest.testCancelTaskExceptionAfterTaskMarkedFailed Key: FLINK-26271 URL: https://issues.apache.org/jira/browse/FLINK-26271 Project: Flink Issue Type: Improvement Components: Runtime / Task Affects Versions: 1.13.6 Reporter: Till Rohrmann The test {{TaskTest.testCancelTaskExceptionAfterTaskMarkedFailed}} failed on AZP with {code} Feb 21 03:27:40 [ERROR] testCancelTaskExceptionAfterTaskMarkedFailed(org.apache.flink.runtime.taskmanager.TaskTest) Time elapsed: 0.043 s <<< FAILURE! Feb 21 03:27:40 java.lang.AssertionError: expected: but was: Feb 21 03:27:40 at org.junit.Assert.fail(Assert.java:88) Feb 21 03:27:40 at org.junit.Assert.failNotEquals(Assert.java:834) Feb 21 03:27:40 at org.junit.Assert.assertEquals(Assert.java:118) Feb 21 03:27:40 at org.junit.Assert.assertEquals(Assert.java:144) Feb 21 03:27:40 at org.apache.flink.runtime.taskmanager.TaskTest.testCancelTaskExceptionAfterTaskMarkedFailed(TaskTest.java:598) Feb 21 03:27:40 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Feb 21 03:27:40 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Feb 21 03:27:40 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Feb 21 03:27:40 at java.lang.reflect.Method.invoke(Method.java:498) Feb 21 03:27:40 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) Feb 21 03:27:40 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Feb 21 03:27:40 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) Feb 21 03:27:40 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Feb 21 03:27:40 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Feb 21 03:27:40 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Feb 21 03:27:40 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Feb 21 03:27:40 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) Feb 21 03:27:40 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Feb 21 03:27:40 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) Feb 21 03:27:40 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) Feb 21 03:27:40 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) Feb 21 03:27:40 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) Feb 21 03:27:40 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) Feb 21 03:27:40 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) Feb 21 03:27:40 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) Feb 21 03:27:40 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) Feb 21 03:27:40 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) Feb 21 03:27:40 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Feb 21 03:27:40 at org.junit.runners.ParentRunner.run(ParentRunner.java:363) Feb 21 03:27:40 at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) Feb 21 03:27:40 at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) Feb 21 03:27:40 at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) Feb 21 03:27:40 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) Feb 21 03:27:40 at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) Feb 21 03:27:40 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) Feb 21 03:27:40 at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) Feb 21 03:27:40 at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=31903&view=logs&j=4d4a0d10-fca2-5507-8eed-c07f0bdf4887&t=c2734c79-73b6-521c-e85a-67c7ecae9107&l=5762 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26197) Allow playground egress to keep connection open and push messages
Till Rohrmann created FLINK-26197: - Summary: Allow playground egress to keep connection open and push messages Key: FLINK-26197 URL: https://issues.apache.org/jira/browse/FLINK-26197 Project: Flink Issue Type: Improvement Components: Stateful Functions Affects Versions: statefun-3.2.0, statefun-3.3.0 Reporter: Till Rohrmann In order to improve the getting started experience it would be nice if the playground egress can keep connections to clients open and push new messages eagerly. Moreover, we could keep all messages stored in memory to allow serving old messages. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26158) Update statefun-playground examples to use playground ingress/egress
Till Rohrmann created FLINK-26158: - Summary: Update statefun-playground examples to use playground ingress/egress Key: FLINK-26158 URL: https://issues.apache.org/jira/browse/FLINK-26158 Project: Flink Issue Type: Improvement Components: Stateful Functions Reporter: Till Rohrmann Assignee: Till Rohrmann With FLINK-26153, the statefun-playground has a new ingress/egress that allows to interact with via curl. I propose to update the existing examples to make use of it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26155) Add playground ingress/egress for cURL interaction
Till Rohrmann created FLINK-26155: - Summary: Add playground ingress/egress for cURL interaction Key: FLINK-26155 URL: https://issues.apache.org/jira/browse/FLINK-26155 Project: Flink Issue Type: Improvement Components: Stateful Functions Reporter: Till Rohrmann Assignee: Till Rohrmann In order to give a better playground experience we could add a special playground ingress/egress that supports easy to use cURL interaction (ingesting new messages and consuming produced messages). Ingesting new message could look like: {code} curl -X PUT -H "Content-Type: application/vnd." -d '' localhost:8090/// {code} Consuming could look like: {code} curl -X GET localhost:8091/ {code} The ingress/egress are not meant to be production ready and only support the use cases of the playground. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26153) Add script to update README playground links for Statefun
Till Rohrmann created FLINK-26153: - Summary: Add script to update README playground links for Statefun Key: FLINK-26153 URL: https://issues.apache.org/jira/browse/FLINK-26153 Project: Flink Issue Type: Improvement Components: Build System / Stateful Functions Reporter: Till Rohrmann Assignee: Till Rohrmann In order to keep the playground links in the various README's up to date, it would be helpful to have a script that updates the links. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26098) TableAPI does not forward idleness configuration from DataStream
Till Rohrmann created FLINK-26098: - Summary: TableAPI does not forward idleness configuration from DataStream Key: FLINK-26098 URL: https://issues.apache.org/jira/browse/FLINK-26098 Project: Flink Issue Type: Bug Components: Table SQL / API Affects Versions: 1.14.3, 1.15.0 Reporter: Till Rohrmann The TableAPI does not forward the idleness configuration from a DataStream source. That can lead to the halt of processing if all sources are idle because {{WatermarkAssignerOperator}} [1] will never set a channel to active again. The only way to mitigate the problem is to explicitly configure the idleness for table sources via {{table.exec.source.idle-timeout}}. Configuring this value is actually not easy because creating a {{StreamExecutionEnvironment}} via {{create(StreamExecutionEnvironment, TableConfig)}} is deprecated. [1] https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/wmassigners/WatermarkAssignerOperator.java#L103 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26014) Document how to use the working directory for faster local recoveries
Till Rohrmann created FLINK-26014: - Summary: Document how to use the working directory for faster local recoveries Key: FLINK-26014 URL: https://issues.apache.org/jira/browse/FLINK-26014 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 After having implemented FLIP-198 and FLIP-201, users can now use faster TaskManager failover when using local recovery with persisted volumes. I suggest to add documentation for explaining how to configure Flink to make use of it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26005) TableEnvironment.createTemporarySystemFunction cause NPE when using leftOuterLateralJoin
Till Rohrmann created FLINK-26005: - Summary: TableEnvironment.createTemporarySystemFunction cause NPE when using leftOuterLateralJoin Key: FLINK-26005 URL: https://issues.apache.org/jira/browse/FLINK-26005 Project: Flink Issue Type: Bug Components: Table SQL / Runtime Affects Versions: 1.14.3, 1.15.0 Reporter: Till Rohrmann When trying out the {{Table.leftOuterLateralJoin}} with a table function that was registered via {{TableEnvironment.createTemporarySystemFunction}} the system failed with {code} Exception in thread "main" java.lang.NullPointerException at org.apache.calcite.tools.RelBuilder$Frame.(RelBuilder.java:3332) at org.apache.calcite.tools.RelBuilder$Frame.(RelBuilder.java:3317) at org.apache.calcite.tools.RelBuilder.push(RelBuilder.java:282) at org.apache.calcite.tools.RelBuilder.functionScan(RelBuilder.java:1197) at org.apache.flink.table.planner.plan.QueryOperationConverter$SingleRelVisitor.visit(QueryOperationConverter.java:309) at org.apache.flink.table.planner.plan.QueryOperationConverter$SingleRelVisitor.visit(QueryOperationConverter.java:154) at org.apache.flink.table.operations.CalculatedQueryOperation.accept(CalculatedQueryOperation.java:94) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:151) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:133) at org.apache.flink.table.operations.utils.QueryOperationDefaultVisitor.visit(QueryOperationDefaultVisitor.java:87) at org.apache.flink.table.operations.CalculatedQueryOperation.accept(CalculatedQueryOperation.java:94) at org.apache.flink.table.planner.plan.QueryOperationConverter.lambda$defaultMethod$0(QueryOperationConverter.java:150) at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4390) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:150) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:133) at org.apache.flink.table.operations.utils.QueryOperationDefaultVisitor.visit(QueryOperationDefaultVisitor.java:62) at org.apache.flink.table.operations.JoinQueryOperation.accept(JoinQueryOperation.java:115) at org.apache.flink.table.planner.plan.QueryOperationConverter.lambda$defaultMethod$0(QueryOperationConverter.java:150) at java.base/java.util.Collections$SingletonList.forEach(Collections.java:4854) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:150) at org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:133) at org.apache.flink.table.operations.utils.QueryOperationDefaultVisitor.visit(QueryOperationDefaultVisitor.java:47) at org.apache.flink.table.operations.ProjectQueryOperation.accept(ProjectQueryOperation.java:76) at org.apache.flink.table.planner.calcite.FlinkRelBuilder.queryOperation(FlinkRelBuilder.scala:184) at org.apache.flink.table.planner.delegation.PlannerBase.translateToRel(PlannerBase.scala:214) at org.apache.flink.table.planner.delegation.PlannerBase.$anonfun$translate$1(PlannerBase.scala:182) at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233) at scala.collection.Iterator.foreach(Iterator.scala:937) at scala.collection.Iterator.foreach$(Iterator.scala:937) at scala.collection.AbstractIterator.foreach(Iterator.scala:1425) at scala.collection.IterableLike.foreach(IterableLike.scala:70) at scala.collection.IterableLike.foreach$(IterableLike.scala:69) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike.map(TraversableLike.scala:233) at scala.collection.TraversableLike.map$(TraversableLike.scala:226) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:182) at org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1665) at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeQueryOperation(TableEnvironmentImpl.java:805) at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:1274) at org.apache.flink.table.api.internal.TableImpl.execute(TableImpl.java:601) {code} Interestingly, when using the deprecated {{TableEnvironment.registerFunction}} it worked. Timo mentioned that this could be caused by a missing integration into the new type syst
[jira] [Created] (FLINK-26003) Use Jackson serialization for persisting TaskExecutor state to working directory
Till Rohrmann created FLINK-26003: - Summary: Use Jackson serialization for persisting TaskExecutor state to working directory Key: FLINK-26003 URL: https://issues.apache.org/jira/browse/FLINK-26003 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 In order to avoid Java serialization, we should use a different serialization format for persisting {{TaskExecutor}} state in the working directory. One idea could be to use Jackson for the serialization. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25941) KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint fails on AZP
Till Rohrmann created FLINK-25941: - Summary: KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint fails on AZP Key: FLINK-25941 URL: https://issues.apache.org/jira/browse/FLINK-25941 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint}} fails on AZP with {code} 2022-02-02T17:22:29.5131631Z Feb 02 17:22:29 [ERROR] org.apache.flink.connector.kafka.sink.KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint Time elapsed: 2.186 s <<< FAILURE! 2022-02-02T17:22:29.5146972Z Feb 02 17:22:29 java.lang.AssertionError 2022-02-02T17:22:29.5148918Z Feb 02 17:22:29at org.junit.Assert.fail(Assert.java:87) 2022-02-02T17:22:29.5149843Z Feb 02 17:22:29at org.junit.Assert.assertTrue(Assert.java:42) 2022-02-02T17:22:29.5150644Z Feb 02 17:22:29at org.junit.Assert.assertTrue(Assert.java:53) 2022-02-02T17:22:29.5151730Z Feb 02 17:22:29at org.apache.flink.connector.kafka.sink.KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint(KafkaSinkITCase.java:267) 2022-02-02T17:22:29.5152858Z Feb 02 17:22:29at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-02-02T17:22:29.5153757Z Feb 02 17:22:29at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-02-02T17:22:29.5155002Z Feb 02 17:22:29at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-02-02T17:22:29.5156464Z Feb 02 17:22:29at java.lang.reflect.Method.invoke(Method.java:498) 2022-02-02T17:22:29.5157384Z Feb 02 17:22:29at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2022-02-02T17:22:29.5158445Z Feb 02 17:22:29at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2022-02-02T17:22:29.5159478Z Feb 02 17:22:29at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2022-02-02T17:22:29.5160524Z Feb 02 17:22:29at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2022-02-02T17:22:29.5161758Z Feb 02 17:22:29at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-02-02T17:22:29.5162775Z Feb 02 17:22:29at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2022-02-02T17:22:29.5163744Z Feb 02 17:22:29at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2022-02-02T17:22:29.5164913Z Feb 02 17:22:29at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2022-02-02T17:22:29.5166101Z Feb 02 17:22:29at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2022-02-02T17:22:29.5167030Z Feb 02 17:22:29at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2022-02-02T17:22:29.5167953Z Feb 02 17:22:29at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-02-02T17:22:29.5168956Z Feb 02 17:22:29at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2022-02-02T17:22:29.5169936Z Feb 02 17:22:29at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2022-02-02T17:22:29.5170903Z Feb 02 17:22:29at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2022-02-02T17:22:29.5171953Z Feb 02 17:22:29at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2022-02-02T17:22:29.5172919Z Feb 02 17:22:29at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2022-02-02T17:22:29.5173811Z Feb 02 17:22:29at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2022-02-02T17:22:29.5174874Z Feb 02 17:22:29at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2022-02-02T17:22:29.5175917Z Feb 02 17:22:29at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2022-02-02T17:22:29.5176851Z Feb 02 17:22:29at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2022-02-02T17:22:29.5177816Z Feb 02 17:22:29at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-02-02T17:22:29.5178816Z Feb 02 17:22:29at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2022-02-02T17:22:29.5179929Z Feb 02 17:22:29at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30) 2022-02-02T17:22:29.5180960Z Feb 02 17:22:29at org.junit.rules.RunRules.evaluate(RunRules.java:20) 2022-02-02T17:22:29.5181827Z Feb 02 17:22:29at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-02-02T17:22:29.5182733Z Feb 02 17:22:29at org.junit.runners
[jira] [Created] (FLINK-25940) pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state failed on AZP
Till Rohrmann created FLINK-25940: - Summary: pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state failed on AZP Key: FLINK-25940 URL: https://issues.apache.org/jira/browse/FLINK-25940 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state}} fails on AZP: {code} 2022-02-02T17:44:12.1898582Z Feb 02 17:44:12 === FAILURES === 2022-02-02T17:44:12.1899860Z Feb 02 17:44:12 _ StreamingModeDataStreamTests.test_keyed_process_function_with_state __ 2022-02-02T17:44:12.1900493Z Feb 02 17:44:12 2022-02-02T17:44:12.1901218Z Feb 02 17:44:12 self = 2022-02-02T17:44:12.1901948Z Feb 02 17:44:12 2022-02-02T17:44:12.1902745Z Feb 02 17:44:12 def test_keyed_process_function_with_state(self): 2022-02-02T17:44:12.1903722Z Feb 02 17:44:12 self.env.get_config().set_auto_watermark_interval(2000) 2022-02-02T17:44:12.1904473Z Feb 02 17:44:12 self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime) 2022-02-02T17:44:12.1906780Z Feb 02 17:44:12 data_stream = self.env.from_collection([(1, 'hi', '1603708211000'), 2022-02-02T17:44:12.1908034Z Feb 02 17:44:12 (2, 'hello', '1603708224000'), 2022-02-02T17:44:12.1909166Z Feb 02 17:44:12 (3, 'hi', '1603708226000'), 2022-02-02T17:44:12.1910122Z Feb 02 17:44:12 (4, 'hello', '1603708289000'), 2022-02-02T17:44:12.1911099Z Feb 02 17:44:12 (5, 'hi', '1603708291000'), 2022-02-02T17:44:12.1912451Z Feb 02 17:44:12 (6, 'hello', '1603708293000')], 2022-02-02T17:44:12.1913456Z Feb 02 17:44:12 type_info=Types.ROW([Types.INT(), Types.STRING(), 2022-02-02T17:44:12.1914338Z Feb 02 17:44:12 Types.STRING()])) 2022-02-02T17:44:12.1914811Z Feb 02 17:44:12 2022-02-02T17:44:12.1915317Z Feb 02 17:44:12 class MyTimestampAssigner(TimestampAssigner): 2022-02-02T17:44:12.1915724Z Feb 02 17:44:12 2022-02-02T17:44:12.1916782Z Feb 02 17:44:12 def extract_timestamp(self, value, record_timestamp) -> int: 2022-02-02T17:44:12.1917621Z Feb 02 17:44:12 return int(value[2]) 2022-02-02T17:44:12.1918262Z Feb 02 17:44:12 2022-02-02T17:44:12.1918855Z Feb 02 17:44:12 class MyProcessFunction(KeyedProcessFunction): 2022-02-02T17:44:12.1919363Z Feb 02 17:44:12 2022-02-02T17:44:12.1919744Z Feb 02 17:44:12 def __init__(self): 2022-02-02T17:44:12.1920143Z Feb 02 17:44:12 self.value_state = None 2022-02-02T17:44:12.1920648Z Feb 02 17:44:12 self.list_state = None 2022-02-02T17:44:12.1921298Z Feb 02 17:44:12 self.map_state = None 2022-02-02T17:44:12.1921864Z Feb 02 17:44:12 2022-02-02T17:44:12.1922479Z Feb 02 17:44:12 def open(self, runtime_context: RuntimeContext): 2022-02-02T17:44:12.1923907Z Feb 02 17:44:12 value_state_descriptor = ValueStateDescriptor('value_state', Types.INT()) 2022-02-02T17:44:12.1924922Z Feb 02 17:44:12 self.value_state = runtime_context.get_state(value_state_descriptor) 2022-02-02T17:44:12.1925741Z Feb 02 17:44:12 list_state_descriptor = ListStateDescriptor('list_state', Types.INT()) 2022-02-02T17:44:12.1926482Z Feb 02 17:44:12 self.list_state = runtime_context.get_list_state(list_state_descriptor) 2022-02-02T17:44:12.1927465Z Feb 02 17:44:12 map_state_descriptor = MapStateDescriptor('map_state', Types.INT(), Types.STRING()) 2022-02-02T17:44:12.1927998Z Feb 02 17:44:12 state_ttl_config = StateTtlConfig \ 2022-02-02T17:44:12.1928444Z Feb 02 17:44:12 .new_builder(Time.seconds(1)) \ 2022-02-02T17:44:12.1928943Z Feb 02 17:44:12 .set_update_type(StateTtlConfig.UpdateType.OnReadAndWrite) \ 2022-02-02T17:44:12.1929462Z Feb 02 17:44:12 .set_state_visibility( 2022-02-02T17:44:12.1929939Z Feb 02 17:44:12 StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp) \ 2022-02-02T17:44:12.1930601Z Feb 02 17:44:12 .disable_cleanup_in_background() \ 2022-02-02T17:44:12.1931032Z Feb 02 17:44:12 .build() 2022-02-02T17:44:12.1931480Z Feb 02 17:44:12 map_state_descriptor.enable_time_to_live(state_ttl_co
[jira] [Created] (FLINK-25939) PyFlink YARN per-job on Docker test fails on AZP because it could not acquire all required slots
Till Rohrmann created FLINK-25939: - Summary: PyFlink YARN per-job on Docker test fails on AZP because it could not acquire all required slots Key: FLINK-25939 URL: https://issues.apache.org/jira/browse/FLINK-25939 Project: Flink Issue Type: Bug Components: API / Python, Deployment / YARN Affects Versions: 1.14.3 Reporter: Till Rohrmann The test {{PyFlink YARN per-job on Docker test}} fails on AZP with: {code} 2022-02-03T03:32:41.0643749Z Feb 03 03:32:40 2022-02-03 03:31:12,281 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Values(tuples=[[{ 1 }, { 2 }, { 3 }]]) -> Calc(select=[f0]) -> PythonCalc(select=[add_one(f0) AS a]) -> Sink: Collect table sink (1/1) (1c3df1a51de35e44664305a57f4047a7) switched from SCHEDULED to FAILED on [unassigned resource]. 2022-02-03T03:32:41.0645152Z Feb 03 03:32:40 org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout 2022-02-03T03:32:41.0647529Z Feb 03 03:32:40at org.apache.flink.runtime.jobmaster.slotpool.PhysicalSlotRequestBulkCheckerImpl.lambda$schedulePendingRequestBulkWithTimestampCheck$0(PhysicalSlotRequestBulkCheckerImpl.java:86) ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0648830Z Feb 03 03:32:40at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_312] 2022-02-03T03:32:41.0649605Z Feb 03 03:32:40at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_312] 2022-02-03T03:32:41.0651215Z Feb 03 03:32:40at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:455) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0652935Z Feb 03 03:32:40at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0654715Z Feb 03 03:32:40at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:455) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0656399Z Feb 03 03:32:40at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0657987Z Feb 03 03:32:40at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:78) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0659528Z Feb 03 03:32:40at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163) ~[flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0661186Z Feb 03 03:32:40at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0662745Z Feb 03 03:32:40at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0664203Z Feb 03 03:32:40at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0665591Z Feb 03 03:32:40at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0667179Z Feb 03 03:32:40at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0668708Z Feb 03 03:32:40at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0670130Z Feb 03 03:32:40at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0671634Z Feb 03 03:32:40at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0672985Z Feb 03 03:32:40at akka.actor.Actor.aroundReceive(Actor.scala:537) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0674418Z Feb 03 03:32:40at akka.actor.Actor.aroundReceive$(Actor.scala:535) [flink-rpc-akka_92d95797-5383-46fa-abb4-628d6de63216.jar:1.14-SNAPSHOT] 2022-02-03T03:32:41.0675750Z Feb 03 03:32:40at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala
[jira] [Created] (FLINK-25938) SynchronousCheckpointITCase.taskDispatcherThreadPoolAllowsForSynchronousCheckpoints fails on AZP
Till Rohrmann created FLINK-25938: - Summary: SynchronousCheckpointITCase.taskDispatcherThreadPoolAllowsForSynchronousCheckpoints fails on AZP Key: FLINK-25938 URL: https://issues.apache.org/jira/browse/FLINK-25938 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.14.3 Reporter: Till Rohrmann The test {{SynchronousCheckpointITCase.taskDispatcherThreadPoolAllowsForSynchronousCheckpoints}} fails on AZP with {code} 2022-02-03T02:56:19.1950622Z Feb 03 02:56:19 [ERROR] taskDispatcherThreadPoolAllowsForSynchronousCheckpoints Time elapsed: 10.234 s <<< ERROR! 2022-02-03T02:56:19.1953312Z Feb 03 02:56:19 org.junit.runners.model.TestTimedOutException: test timed out after 10 seconds 2022-02-03T02:56:19.1954442Z Feb 03 02:56:19at java.util.zip.ZipFile.read(Native Method) 2022-02-03T02:56:19.1955186Z Feb 03 02:56:19at java.util.zip.ZipFile.access$1400(ZipFile.java:60) 2022-02-03T02:56:19.1956156Z Feb 03 02:56:19at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:734) 2022-02-03T02:56:19.1957004Z Feb 03 02:56:19at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:434) 2022-02-03T02:56:19.1957886Z Feb 03 02:56:19at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) 2022-02-03T02:56:19.1958732Z Feb 03 02:56:19at sun.misc.Resource.getBytes(Resource.java:124) 2022-02-03T02:56:19.1959541Z Feb 03 02:56:19at java.net.URLClassLoader.defineClass(URLClassLoader.java:463) 2022-02-03T02:56:19.1960342Z Feb 03 02:56:19at java.net.URLClassLoader.access$100(URLClassLoader.java:74) 2022-02-03T02:56:19.1961173Z Feb 03 02:56:19at java.net.URLClassLoader$1.run(URLClassLoader.java:369) 2022-02-03T02:56:19.1962349Z Feb 03 02:56:19at java.net.URLClassLoader$1.run(URLClassLoader.java:363) 2022-02-03T02:56:19.1963187Z Feb 03 02:56:19at java.security.AccessController.doPrivileged(Native Method) 2022-02-03T02:56:19.1964157Z Feb 03 02:56:19at java.net.URLClassLoader.findClass(URLClassLoader.java:362) 2022-02-03T02:56:19.1964917Z Feb 03 02:56:19at java.lang.ClassLoader.loadClass(ClassLoader.java:418) 2022-02-03T02:56:19.1965815Z Feb 03 02:56:19at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) 2022-02-03T02:56:19.1966620Z Feb 03 02:56:19at java.lang.ClassLoader.loadClass(ClassLoader.java:351) 2022-02-03T02:56:19.1967373Z Feb 03 02:56:19at scala.collection.AbstractSeq.(Seq.scala:41) 2022-02-03T02:56:19.1968142Z Feb 03 02:56:19at scala.collection.mutable.AbstractSeq.(Seq.scala:48) 2022-02-03T02:56:19.1969039Z Feb 03 02:56:19at scala.collection.mutable.StringBuilder.(StringBuilder.scala:32) 2022-02-03T02:56:19.1980285Z Feb 03 02:56:19at scala.collection.mutable.StringBuilder.(StringBuilder.scala:49) 2022-02-03T02:56:19.1981000Z Feb 03 02:56:19at scala.collection.mutable.StringBuilder.(StringBuilder.scala:54) 2022-02-03T02:56:19.1981572Z Feb 03 02:56:19at scala.util.PropertiesTrait$class.$init$(Properties.scala:31) 2022-02-03T02:56:19.1982087Z Feb 03 02:56:19at scala.util.Properties$.(Properties.scala:17) 2022-02-03T02:56:19.1982561Z Feb 03 02:56:19at scala.util.Properties$.(Properties.scala) 2022-02-03T02:56:19.1983060Z Feb 03 02:56:19at scala.tools.nsc.interpreter.IMain$Factory.(IMain.scala:1300) 2022-02-03T02:56:19.1983833Z Feb 03 02:56:19at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 2022-02-03T02:56:19.1984904Z Feb 03 02:56:19at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 2022-02-03T02:56:19.1986146Z Feb 03 02:56:19at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 2022-02-03T02:56:19.1987162Z Feb 03 02:56:19at java.lang.reflect.Constructor.newInstance(Constructor.java:423) 2022-02-03T02:56:19.1988035Z Feb 03 02:56:19at java.lang.Class.newInstance(Class.java:442) 2022-02-03T02:56:19.194Z Feb 03 02:56:19at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) 2022-02-03T02:56:19.1989794Z Feb 03 02:56:19at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) 2022-02-03T02:56:19.1990590Z Feb 03 02:56:19at java.util.ServiceLoader$1.next(ServiceLoader.java:480) 2022-02-03T02:56:19.1991363Z Feb 03 02:56:19at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:122) 2022-02-03T02:56:19.1992201Z Feb 03 02:56:19at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:84) 2022-02-03T02:56:19.1993066Z Feb 03 02:56:19at javax.script.ScriptEngineManager.(ScriptEngineManager.java:61) 2022-02-03T02:56:19.1993870Z Feb 03 02:56:19at org.apache.logging.log4j.core.script.ScriptManager.(ScriptManager.java:70) 2022-02-03T02:56:19.1994633Z Feb 03 02:56:19at org.a
[jira] [Created] (FLINK-25937) SQL Client end-to-end test e2e fails on AZP
Till Rohrmann created FLINK-25937: - Summary: SQL Client end-to-end test e2e fails on AZP Key: FLINK-25937 URL: https://issues.apache.org/jira/browse/FLINK-25937 Project: Flink Issue Type: Bug Components: Runtime / Coordination, Table SQL / API Affects Versions: 1.15.0 Reporter: Till Rohrmann The {{SQL Client end-to-end test}} e2e tests fails on AZP when using the {{AdaptiveScheduler}} because the scheduler expects that the parallelism is set for all vertices: {code} Feb 03 03:45:13 org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster. Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609) Feb 03 03:45:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Feb 03 03:45:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Feb 03 03:45:13 at java.lang.Thread.run(Thread.java:748) Feb 03 03:45:13 Caused by: java.util.concurrent.CompletionException: java.lang.IllegalStateException: The adaptive scheduler expects the parallelism being set for each JobVertex (violated JobVertex: f74b775b58627a33e46b8c155b320255). Feb 03 03:45:13 at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606) Feb 03 03:45:13 ... 3 more Feb 03 03:45:13 Caused by: java.lang.IllegalStateException: The adaptive scheduler expects the parallelism being set for each JobVertex (violated JobVertex: f74b775b58627a33e46b8c155b320255). Feb 03 03:45:13 at org.apache.flink.util.Preconditions.checkState(Preconditions.java:215) Feb 03 03:45:13 at org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.assertPreconditions(AdaptiveScheduler.java:296) Feb 03 03:45:13 at org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.(AdaptiveScheduler.java:230) Feb 03 03:45:13 at org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerFactory.createInstance(AdaptiveSchedulerFactory.java:122) Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:115) Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345) Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:322) Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:106) Feb 03 03:45:13 at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:94) Feb 03 03:45:13 at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) Feb 03 03:45:13 at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) Feb 03 03:45:13 ... 3 more {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30662&view=logs&j=fb37c667-81b7-5c22-dd91-846535e99a97&t=39a035c3-c65e-573c-fb66-104c66c28912&l=5782 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25925) JobMasterTest.testJobMasterAcceptsExcessSlotsWhenJobIsRestarting
Till Rohrmann created FLINK-25925: - Summary: JobMasterTest.testJobMasterAcceptsExcessSlotsWhenJobIsRestarting Key: FLINK-25925 URL: https://issues.apache.org/jira/browse/FLINK-25925 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann The test {{JobMasterTest.testJobMasterAcceptsExcessSlotsWhenJobIsRestarting}} fails on AZP with {code} Feb 02 02:49:46 [ERROR] JobMasterTest.testJobMasterAcceptsExcessSlotsWhenJobIsRestarting:1944 Feb 02 02:49:46 Expected: is Feb 02 02:49:46 but: was {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30598&view=logs&j=0e7be18f-84f2-53f0-a32d-4a5e4a174679&t=7c1d86e3-35bd-5fd5-3b7c-30c126a78702&l=9114 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25922) KinesisFirehoseSinkITCase hangs on AZP
Till Rohrmann created FLINK-25922: - Summary: KinesisFirehoseSinkITCase hangs on AZP Key: FLINK-25922 URL: https://issues.apache.org/jira/browse/FLINK-25922 Project: Flink Issue Type: Bug Components: Connectors / Kinesis Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{KinesisFirehoseSinkITCase}} hangs on AZP. {code} 2022-02-02T05:37:07.5806959Z "main" #1 prio=5 os_prio=0 tid=0x7f06ec00b800 nid=0x6145 waiting on condition [0x7f06f4cc5000] 2022-02-02T05:37:07.5807433Zjava.lang.Thread.State: WAITING (parking) 2022-02-02T05:37:07.5807815Zat sun.misc.Unsafe.park(Native Method) 2022-02-02T05:37:07.5808450Z- parking to wait for <0x83f34570> (a java.util.concurrent.CompletableFuture$Signaller) 2022-02-02T05:37:07.5808955Zat java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) 2022-02-02T05:37:07.5809493Zat java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707) 2022-02-02T05:37:07.5810034Zat java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323) 2022-02-02T05:37:07.5810568Zat java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742) 2022-02-02T05:37:07.5811102Zat java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) 2022-02-02T05:37:07.5811812Zat org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1989) 2022-02-02T05:37:07.5812487Zat org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:69) 2022-02-02T05:37:07.5813165Zat org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1969) 2022-02-02T05:37:07.5813832Zat org.apache.flink.connector.firehose.sink.KinesisFirehoseSinkITCase.test(KinesisFirehoseSinkITCase.java:119) 2022-02-02T05:37:07.5814508Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-02-02T05:37:07.5815011Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-02-02T05:37:07.5815584Zat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-02-02T05:37:07.5816176Zat java.lang.reflect.Method.invoke(Method.java:498) 2022-02-02T05:37:07.5816687Zat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2022-02-02T05:37:07.5817256Zat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2022-02-02T05:37:07.5817806Zat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2022-02-02T05:37:07.5818367Zat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2022-02-02T05:37:07.5819005Zat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-02-02T05:37:07.5819542Zat org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2022-02-02T05:37:07.5820056Zat org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-02-02T05:37:07.5820597Zat org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2022-02-02T05:37:07.5821116Zat org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2022-02-02T05:37:07.5821625Zat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2022-02-02T05:37:07.5822198Zat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2022-02-02T05:37:07.5822711Zat org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2022-02-02T05:37:07.5823190Zat org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2022-02-02T05:37:07.5823696Zat org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2022-02-02T05:37:07.5824242Zat org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2022-02-02T05:37:07.5824738Zat org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2022-02-02T05:37:07.5825316Zat org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30) 2022-02-02T05:37:07.5825962Zat org.junit.rules.RunRules.evaluate(RunRules.java:20) 2022-02-02T05:37:07.5826440Zat org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-02-02T05:37:07.5826927Zat org.junit.runners.ParentRunner.run(ParentRunner.java:413) 2022-02-02T05:37:07.5827380Zat org.junit.runner.JUnitCore.run(JUnitCore.java:137) 2022-02-02T05:37:07.5827821Zat org.junit.runner.JUnitCore.run(JUnitCore.java:115) 2022-02-02T05:37:07.5828306Zat org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) 2022-02-02T05:37:07.5828883Zat org.junit.vintage.engine.VintageTestEngine.
[jira] [Created] (FLINK-25899) Add Java connected components example to flink-statefun-playground
Till Rohrmann created FLINK-25899: - Summary: Add Java connected components example to flink-statefun-playground Key: FLINK-25899 URL: https://issues.apache.org/jira/browse/FLINK-25899 Project: Flink Issue Type: Improvement Components: Build System / Stateful Functions Affects Versions: statefun-3.1.1 Reporter: Till Rohrmann In order to show what we can do with Stateful Functions I suggest to add an example for how to calculate the connected components from a stream of vertices. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25898) Add README.md to flink-statefun/statefun-sdk-js
Till Rohrmann created FLINK-25898: - Summary: Add README.md to flink-statefun/statefun-sdk-js Key: FLINK-25898 URL: https://issues.apache.org/jira/browse/FLINK-25898 Project: Flink Issue Type: Improvement Components: Stateful Functions Affects Versions: statefun-3.2.0 Reporter: Till Rohrmann We should add a {{README.md}} to {{flink-statefun/statefun-sdk-js}}. This would then also be displayed on [npmjs.com|https://www.npmjs.com/package/apache-flink-statefun]. Unfortunately, in order to publish the {{README.md}} we have to release a new version of the npm package. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25893) ResourceManagerServiceImpl's lifecycle can lead to exceptions
Till Rohrmann created FLINK-25893: - Summary: ResourceManagerServiceImpl's lifecycle can lead to exceptions Key: FLINK-25893 URL: https://issues.apache.org/jira/browse/FLINK-25893 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.14.3, 1.15.0 Reporter: Till Rohrmann The {{ResourceManagerServiceImpl}} lifecycle can lead to exceptions when calling {{ResourceManagerServiceImpl.deregisterApplication}}. The problem arises when the {{DispatcherResourceManagerComponent}} is shutdown before the {{ResourceManagerServiceImpl}} gains leadership or while it is starting the {{ResourceManager}}. One problem is that {{deregisterApplication}} returns an exceptionally completed future if there is no leading {{ResourceManager}}. Another problem is that if there is a leading {{ResourceManager}}, then it can still be the case that it has not been started yet. If this is the case, then [ResourceManagerGateway.deregisterApplication|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerServiceImpl.java#L143] will be discarded. The reason for this behaviour is that we create a {{ResourceManager}} in one {{Runnable}} and only start it in another. Due to this there can be the {{deregisterApplication}} call that gets the {{lock}} in between. I'd suggest to correct the lifecycle and contract of the {{ResourceManagerServiceImpl.deregisterApplication}}. Please note that due to this problem, the error reporting of this method has been suppressed. See FLINK-25885 for more details. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25855) DefaultDeclarativeSlotPool rejects offered slots when the job is restarting
Till Rohrmann created FLINK-25855: - Summary: DefaultDeclarativeSlotPool rejects offered slots when the job is restarting Key: FLINK-25855 URL: https://issues.apache.org/jira/browse/FLINK-25855 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.14.3, 1.15.0 Reporter: Till Rohrmann The {{DefaultDeclarativeSlotPool}} rejects offered slots if the job is currently restarting. The problem is that in case of a job restart, the scheduler sets the required resources to zero. Hence, all offered slots will be rejected. This is a problem for local recovery because rejected slots will be freed by the {{TaskExecutor}} and thereby all local state will be deleted. Hence, in order to properly support local recovery, we need to handle this situation somehow. I do see different options here: h3. Accept excess slots Accepting excess slots means that the {{DefaultDeclarativeSlotPool}} accepts slots which exceed the currently required set of slots. Advantages: * Easy to implement Disadvantages: * Offered slots that are not really needed will only be freed after the idle slot timeout. This means that some resources might be left unused for some time. h3. Let DefaultDeclarativeSlotPool accept excess slots when job is restarting Here the idea is to only accept excess slots when the job is currently restarting. This will required that the scheduler tells the {{DefaultDeclarativeSlotPool}} about the restarting state. Advantages: * We would only accept excess slots for the time of restarting Disadvantages: * We are complicating the semantics of the {{DefaultDeclarativeSlotPool}}. Moreover, we are introducing additional signals that communicate the restarting state to the pool. h3. Don't immediately free slots on the TaskExecutor when they are rejected Instead of freeing the slot immediately on the {{TaskExecutor}} after it is rejected. We could also retry for some time and only free the slot after some timeout. Advantages: * No changes on the JobMaster side needed. Disadvantages: * Complication of the slot lifecycle on the {{TaskExecutor}} * Unneeded slots are not made available for other jobs as fast as possible -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25849) Differentiate TaskManager sessions
Till Rohrmann created FLINK-25849: - Summary: Differentiate TaskManager sessions Key: FLINK-25849 URL: https://issues.apache.org/jira/browse/FLINK-25849 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann With the introduction of configurable {{ResourceID}} for {{TaskManager}} processes, it can happen that a restarted {{TaskManager}} process will be restarted with the same {{ResourceID}}. When it now tries to register at the {{JobMaster}}, the {{JobMaster}} won't recognize it as a new instance because it only compares the {{ResourceID}}. As a consequence, the {{JobMaster}} things that this is a duplicate registration and ignores it. It would be better if the {{TaskManager}} would send a session id with the registration that could then be used to decide whether a new instance tries to register at the {{JobMaster}} and, therefore, the old one needs to be disconnected or whether the registration attempt is a duplicate. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25831) ExecutionVertex.getLatestPriorAllocation fails if there is an unsuccessful restart attempt
Till Rohrmann created FLINK-25831: - Summary: ExecutionVertex.getLatestPriorAllocation fails if there is an unsuccessful restart attempt Key: FLINK-25831 URL: https://issues.apache.org/jira/browse/FLINK-25831 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.14.3, 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann The {{ExecutionVertex.getLatestPriorAllocation}} does not return the latest prior allocation if there was an unsuccessful restart attempt in between. The problem is that we only look at the last {{Execution}}. Due to this, {{ExecutionVertex.getLatestPriorAllocation}} sometimes returns {{null}} even though there is a prior {{AllocationID}}. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25829) Do not deploy empty statefun-sdk-js jar
Till Rohrmann created FLINK-25829: - Summary: Do not deploy empty statefun-sdk-js jar Key: FLINK-25829 URL: https://issues.apache.org/jira/browse/FLINK-25829 Project: Flink Issue Type: Improvement Components: Stateful Functions Affects Versions: statefun-3.2.0, statefun-3.3.0 Reporter: Till Rohrmann Assignee: Till Rohrmann The {{statefun-sdk-js}} module does not produce a meaningful jar as output. Therefore, we can prevent the deployment of it. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25828) build_stateful_functions.sh can fail if target contains source or javadoc jar
Till Rohrmann created FLINK-25828: - Summary: build_stateful_functions.sh can fail if target contains source or javadoc jar Key: FLINK-25828 URL: https://issues.apache.org/jira/browse/FLINK-25828 Project: Flink Issue Type: New Feature Components: Stateful Functions Affects Versions: statefun-3.2.0, statefun-3.3.0 Reporter: Till Rohrmann {{build_stateful_functions.sh}} can fail if target contains source or javadoc jar. I suggest to exclude these files from the search. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25819) NetworkBufferPoolTest.testIsAvailableOrNotAfterRequestAndRecycleMultiSegments fails on AZP
Till Rohrmann created FLINK-25819: - Summary: NetworkBufferPoolTest.testIsAvailableOrNotAfterRequestAndRecycleMultiSegments fails on AZP Key: FLINK-25819 URL: https://issues.apache.org/jira/browse/FLINK-25819 Project: Flink Issue Type: New Feature Components: Runtime / Network Affects Versions: 1.14.3 Reporter: Till Rohrmann The {{NetworkBufferPoolTest.testIsAvailableOrNotAfterRequestAndRecycleMultiSegments}} fails on AZP with: {code} Jan 26 07:57:03 [ERROR] testIsAvailableOrNotAfterRequestAndRecycleMultiSegments Time elapsed: 10.028 s <<< ERROR! Jan 26 07:57:03 org.junit.runners.model.TestTimedOutException: test timed out after 10 seconds Jan 26 07:57:03 at sun.misc.Unsafe.park(Native Method) Jan 26 07:57:03 at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) Jan 26 07:57:03 at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:426) Jan 26 07:57:03 at java.util.concurrent.FutureTask.get(FutureTask.java:204) Jan 26 07:57:03 at org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:167) Jan 26 07:57:03 at org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:128) Jan 26 07:57:03 at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:258) Jan 26 07:57:03 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Jan 26 07:57:03 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) Jan 26 07:57:03 at java.util.concurrent.FutureTask.run(FutureTask.java:266) Jan 26 07:57:03 at java.lang.Thread.run(Thread.java:748) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30187&view=logs&j=4d4a0d10-fca2-5507-8eed-c07f0bdf4887&t=7b25afdf-cc6c-566f-5459-359dc2585798&l=7350 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25817) FLIP-201: Persist local state in working directory
Till Rohrmann created FLINK-25817: - Summary: FLIP-201: Persist local state in working directory Key: FLINK-25817 URL: https://issues.apache.org/jira/browse/FLINK-25817 Project: Flink Issue Type: New Feature Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann This issue is the umbrella ticket for [FLIP-201|https://cwiki.apache.org/confluence/x/wJuqCw] which aims at adding support for persisting local state in Flink's working directory. This would enable Flink in certain scenarios to recover locally even in case of process failures. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25806) Remove legacy high availability services
Till Rohrmann created FLINK-25806: - Summary: Remove legacy high availability services Key: FLINK-25806 URL: https://issues.apache.org/jira/browse/FLINK-25806 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.16.0 Reporter: Till Rohrmann Fix For: 1.16.0 After FLINK-24038, we should consider removing the legacy high availability services {{ZooKeeperHaServices}} and {{KubernetesHaServices}} since they are now subsumed by the multiple component leader election service that only uses a single leader election per component. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25775) Document JavaScript SDK
Till Rohrmann created FLINK-25775: - Summary: Document JavaScript SDK Key: FLINK-25775 URL: https://issues.apache.org/jira/browse/FLINK-25775 Project: Flink Issue Type: Improvement Components: Stateful Functions Affects Versions: statefun-3.2.0 Reporter: Till Rohrmann Fix For: statefun-3.2.0 The newly introduced JavaScript SDK needs to be documented similarly to the other SDKs. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25772) GCS filessystem fails license checker
Till Rohrmann created FLINK-25772: - Summary: GCS filessystem fails license checker Key: FLINK-25772 URL: https://issues.apache.org/jira/browse/FLINK-25772 Project: Flink Issue Type: Bug Components: Connectors / FileSystem Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 FLINK-11838 made the {{LicenseChecker}} fail with the following exception when compiling using Java 11. {code} 00:30:51,995 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - File '/javax/annotation/security/package.html' in jar '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar' contains match with forbidden regex 'gnu ?\R?[\s/#]*general ?\R?[\s/#]*public ?\R?[\s/#]*license'. 00:30:51,997 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - File '/javax/annotation/package.html' in jar '/tmp/flink-validation-deployment/org/apache/flink/flink-gs-fs-hadoop/1.15-SNAPSHOT/flink-gs-fs-hadoop-1.15-20220121.001624-1.jar' contains match with forbidden regex 'gnu ?\R?[\s/#]*general ?\R?[\s/#]*public ?\R?[\s/#]*license'. 00:32:17,194 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker [] - Found a total of 3 severe license issues {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29841&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25771) CassandraConnectorITCase.testRetrialAndDropTables fails on AZP
Till Rohrmann created FLINK-25771: - Summary: CassandraConnectorITCase.testRetrialAndDropTables fails on AZP Key: FLINK-25771 URL: https://issues.apache.org/jira/browse/FLINK-25771 Project: Flink Issue Type: Bug Components: Connectors / Cassandra Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{CassandraConnectorITCase.testRetrialAndDropTables}} fails on AZP with {code} Jan 23 01:02:52 com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /172.17.0.1:59220 (com.datastax.driver.core.exceptions.OperationTimedOutException: [/172.17.0.1] Timed out waiting for server response)) Jan 23 01:02:52 at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84) Jan 23 01:02:52 at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37) Jan 23 01:02:52 at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) Jan 23 01:02:52 at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) Jan 23 01:02:52 at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63) Jan 23 01:02:52 at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39) Jan 23 01:02:52 at org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testRetrialAndDropTables(CassandraConnectorITCase.java:554) Jan 23 01:02:52 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Jan 23 01:02:52 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Jan 23 01:02:52 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 23 01:02:52 at java.lang.reflect.Method.invoke(Method.java:498) Jan 23 01:02:52 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) Jan 23 01:02:52 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Jan 23 01:02:52 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) Jan 23 01:02:52 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Jan 23 01:02:52 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Jan 23 01:02:52 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Jan 23 01:02:52 at org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:196) Jan 23 01:02:52 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Jan 23 01:02:52 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Jan 23 01:02:52 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) Jan 23 01:02:52 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) Jan 23 01:02:52 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) Jan 23 01:02:52 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) Jan 23 01:02:52 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) Jan 23 01:02:52 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) Jan 23 01:02:52 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) Jan 23 01:02:52 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) Jan 23 01:02:52 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Jan 23 01:02:52 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Jan 23 01:02:52 at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30) Jan 23 01:02:52 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Jan 23 01:02:52 at org.junit.runners.ParentRunner.run(ParentRunner.java:363) Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:128) Jan 23 01:02:52 at org.junit.runners.Suite.runChild(Suite.java:27) Jan 23 01:02:52 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) Jan 23 01:02:52 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) Jan 23 01:02:52 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) Jan 23 01:02:52 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) Jan 23 01:02:52 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) Jan 23 01:02:52
[jira] [Created] (FLINK-25753) KafkaTableITCase.testStartFromGroupOffsetsLatest fails on AZP
Till Rohrmann created FLINK-25753: - Summary: KafkaTableITCase.testStartFromGroupOffsetsLatest fails on AZP Key: FLINK-25753 URL: https://issues.apache.org/jira/browse/FLINK-25753 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{KafkaTableITCase.testStartFromGroupOffsetsLatest}} fails on AZP with {code} 2022-01-21T08:48:26.7044016Z Jan 21 08:48:26 [ERROR] KafkaTableITCase.testStartFromGroupOffsetsLatest Time elapsed: 5.308 s <<< ERROR! 2022-01-21T08:48:26.7044630Z Jan 21 08:48:26 java.util.concurrent.TimeoutException: Can not get the expected result. 2022-01-21T08:48:26.7047268Z Jan 21 08:48:26at org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:214) 2022-01-21T08:48:26.7048108Z Jan 21 08:48:26at org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:230) 2022-01-21T08:48:26.7048836Z Jan 21 08:48:26at org.apache.flink.streaming.connectors.kafka.table.KafkaTableTestUtils.waitingExpectedResults(KafkaTableTestUtils.java:82) 2022-01-21T08:48:26.7049632Z Jan 21 08:48:26at org.apache.flink.streaming.connectors.kafka.table.KafkaTableITCase.testStartFromGroupOffsets(KafkaTableITCase.java:942) 2022-01-21T08:48:26.7050427Z Jan 21 08:48:26at org.apache.flink.streaming.connectors.kafka.table.KafkaTableITCase.testStartFromGroupOffsetsLatest(KafkaTableITCase.java:831) 2022-01-21T08:48:26.7051077Z Jan 21 08:48:26at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-01-21T08:48:26.7051638Z Jan 21 08:48:26at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-01-21T08:48:26.7052284Z Jan 21 08:48:26at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-01-21T08:48:26.7053008Z Jan 21 08:48:26at java.lang.reflect.Method.invoke(Method.java:498) 2022-01-21T08:48:26.7053681Z Jan 21 08:48:26at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2022-01-21T08:48:26.7054328Z Jan 21 08:48:26at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2022-01-21T08:48:26.7054976Z Jan 21 08:48:26at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2022-01-21T08:48:26.7055621Z Jan 21 08:48:26at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2022-01-21T08:48:26.7056251Z Jan 21 08:48:26at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-01-21T08:48:26.7056904Z Jan 21 08:48:26at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2022-01-21T08:48:26.7057520Z Jan 21 08:48:26at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2022-01-21T08:48:26.7058445Z Jan 21 08:48:26at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2022-01-21T08:48:26.7059155Z Jan 21 08:48:26at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-01-21T08:48:26.7059779Z Jan 21 08:48:26at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2022-01-21T08:48:26.7060382Z Jan 21 08:48:26at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2022-01-21T08:48:26.7061036Z Jan 21 08:48:26at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2022-01-21T08:48:26.7061670Z Jan 21 08:48:26at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2022-01-21T08:48:26.7062369Z Jan 21 08:48:26at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2022-01-21T08:48:26.7063456Z Jan 21 08:48:26at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2022-01-21T08:48:26.7064218Z Jan 21 08:48:26at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2022-01-21T08:48:26.7064805Z Jan 21 08:48:26at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2022-01-21T08:48:26.7065375Z Jan 21 08:48:26at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2022-01-21T08:48:26.7065919Z Jan 21 08:48:26at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 2022-01-21T08:48:26.7066437Z Jan 21 08:48:26at org.junit.runners.Suite.runChild(Suite.java:128) 2022-01-21T08:48:26.7066934Z Jan 21 08:48:26at org.junit.runners.Suite.runChild(Suite.java:27) 2022-01-21T08:48:26.7067448Z Jan 21 08:48:26at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2022-01-21T08:48:26.7067979Z Jan 21 08:48:26at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2022-01-21T08:48:26.7068601Z Jan 21 08:48:26at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2022-01-21T08:48:26.7069167Z Jan 21 08:
[jira] [Created] (FLINK-25749) YARNSessionFIFOSecuredITCase.testDetachedMode fails on AZP
Till Rohrmann created FLINK-25749: - Summary: YARNSessionFIFOSecuredITCase.testDetachedMode fails on AZP Key: FLINK-25749 URL: https://issues.apache.org/jira/browse/FLINK-25749 Project: Flink Issue Type: Bug Components: Deployment / YARN Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{YARNSessionFIFOSecuredITCase.testDetachedMode}} fails on AZP: {code} 2022-01-21T03:28:18.3712993Z Jan 21 03:28:18 java.lang.AssertionError: 2022-01-21T03:28:18.3715115Z Jan 21 03:28:18 Found a file /__w/2/s/flink-yarn-tests/target/flink-yarn-tests-fifo-secured/flink-yarn-tests-fifo-secured-logDir-nm-0_0/application_1642735639007_0002/container_1642735639007_0002_01_01/jobmanager.log with a prohibited string (one of [Exception, Started SelectChannelConnector@0.0.0.0:8081]). Excerpts: 2022-01-21T03:28:18.3716389Z Jan 21 03:28:18 [ 2022-01-21T03:28:18.3717531Z Jan 21 03:28:18 2022-01-21 03:27:56,921 INFO org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl [] - Resource manager service is not running. Ignore revoking leadership. 2022-01-21T03:28:18.3720496Z Jan 21 03:28:18 2022-01-21 03:27:56,922 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped dispatcher akka.tcp://flink@11c5f741db81:37697/user/rpc/dispatcher_0. 2022-01-21T03:28:18.3722401Z Jan 21 03:28:18 2022-01-21 03:27:56,922 INFO org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl [] - Interrupted while waiting for queue 2022-01-21T03:28:18.3723661Z Jan 21 03:28:18 java.lang.InterruptedException: null 2022-01-21T03:28:18.3724529Z Jan 21 03:28:18at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_292] 2022-01-21T03:28:18.3725450Z Jan 21 03:28:18at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_292] 2022-01-21T03:28:18.3726239Z Jan 21 03:28:18at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_292] 2022-01-21T03:28:18.3727618Z Jan 21 03:28:18at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:323) [hadoop-yarn-client-2.8.5.jar:?] 2022-01-21T03:28:18.3729147Z Jan 21 03:28:18 2022-01-21 03:27:56,927 WARN org.apache.hadoop.ipc.Client [] - Failed to connect to server: 11c5f741db81/172.25.0.2:39121: retries get failed due to exceeded maximum allowed retries number: 0 2022-01-21T03:28:18.3730293Z Jan 21 03:28:18 java.nio.channels.ClosedByInterruptException: null 2022-01-21T03:28:18.3730834Z Jan 21 03:28:18 java.nio.channels.ClosedByInterruptException: null 2022-01-21T03:28:18.3731499Z Jan 21 03:28:18at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) ~[?:1.8.0_292] 2022-01-21T03:28:18.3732203Z Jan 21 03:28:18at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:658) ~[?:1.8.0_292] 2022-01-21T03:28:18.3733478Z Jan 21 03:28:18at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) ~[hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3734470Z Jan 21 03:28:18at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) ~[hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3735432Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:685) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3736414Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:788) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3737734Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client$Connection.access$3500(Client.java:410) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3738853Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client.getConnection(Client.java:1550) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3739752Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client.call(Client.java:1381) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3740638Z Jan 21 03:28:18at org.apache.hadoop.ipc.Client.call(Client.java:1345) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3741589Z Jan 21 03:28:18at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3742621Z Jan 21 03:28:18at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116) [hadoop-common-2.8.5.jar:?] 2022-01-21T03:28:18.3743549Z Jan 21 03:28:18at com.sun.proxy.$Proxy51.stopContainers(Unknown Source) [?:?] 2022-01-21T03:28:18.3744684Z Jan 21 03:28:18at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.stopContainers(ContainerManagementProtocolPBClientImpl.java:120) [hadoop-yarn-com
[jira] [Created] (FLINK-25747) UdfStreamOperatorCheckpointingITCase hangs on AZP
Till Rohrmann created FLINK-25747: - Summary: UdfStreamOperatorCheckpointingITCase hangs on AZP Key: FLINK-25747 URL: https://issues.apache.org/jira/browse/FLINK-25747 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{UdfStreamOperatorCheckpointingITCase}} hangs on AZP. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29840&view=logs&j=b0a398c0-685b-599c-eb57-c8c2a771138e&t=d13f554f-d4b9-50f8-30ee-d49c6fb0b3cc&l=15424 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25709) GlueSchemaRegistryAvroKinesisITCase fails on AZP
Till Rohrmann created FLINK-25709: - Summary: GlueSchemaRegistryAvroKinesisITCase fails on AZP Key: FLINK-25709 URL: https://issues.apache.org/jira/browse/FLINK-25709 Project: Flink Issue Type: Bug Components: Connectors / Kinesis Affects Versions: 1.15.0 Reporter: Till Rohrmann The {{GlueSchemaRegistryAvroKinesisITCase}} fails on AZP with {code} Jan 19 18:04:11 java.lang.IllegalStateException: Check failed: Docker environment should have more than 2GB free disk space Jan 19 18:04:11 at org.testcontainers.DockerClientFactory.check(DockerClientFactory.java:312) Jan 19 18:04:11 at org.testcontainers.DockerClientFactory.checkDiskSpace(DockerClientFactory.java:301) Jan 19 18:04:11 at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:238) Jan 19 18:04:11 at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:101) Jan 19 18:04:11 at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:107) Jan 19 18:04:11 at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:316) Jan 19 18:04:11 at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1066) Jan 19 18:04:11 at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29) Jan 19 18:04:11 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Jan 19 18:04:11 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) Jan 19 18:04:11 at java.util.concurrent.FutureTask.run(FutureTask.java:266) Jan 19 18:04:11 at java.lang.Thread.run(Thread.java:748) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29722&view=logs&j=af184cdd-c6d8-5084-0b69-7e9c67b35f7a&t=160c9ae5-96fd-516e-1c91-deb81f59292a&l=15816 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25708) Upgrade Flink dependency to 1.14.3
Till Rohrmann created FLINK-25708: - Summary: Upgrade Flink dependency to 1.14.3 Key: FLINK-25708 URL: https://issues.apache.org/jira/browse/FLINK-25708 Project: Flink Issue Type: Improvement Components: Build System / Stateful Functions Affects Versions: statefun-3.2.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: statefun-3.2.0 For the next Statefun 3.2.0 release we should bump our Flink dependency to {{1.14.3}}. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25698) Elasticsearch7DynamicSinkITCase.testWritingDocuments fails on AZP
Till Rohrmann created FLINK-25698: - Summary: Elasticsearch7DynamicSinkITCase.testWritingDocuments fails on AZP Key: FLINK-25698 URL: https://issues.apache.org/jira/browse/FLINK-25698 Project: Flink Issue Type: Bug Components: Connectors / ElasticSearch Affects Versions: 1.14.3 Reporter: Till Rohrmann The test {{Elasticsearch7DynamicSinkITCase.testWritingDocuments}} fails on AZP with {code} 2022-01-19T01:36:13.5231872Z Jan 19 01:36:13 [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 60.838 s <<< FAILURE! - in org.apache.flink.streaming.connectors.elasticsearch.table.Elasticsearch7DynamicSinkITCase 2022-01-19T01:36:13.5233438Z Jan 19 01:36:13 [ERROR] testWritingDocuments Time elapsed: 32.146 s <<< ERROR! 2022-01-19T01:36:13.5234330Z Jan 19 01:36:13 org.apache.flink.runtime.client.JobExecutionException: Job execution failed. 2022-01-19T01:36:13.5235274Z Jan 19 01:36:13at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) 2022-01-19T01:36:13.5238310Z Jan 19 01:36:13at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137) 2022-01-19T01:36:13.5239309Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) 2022-01-19T01:36:13.5239953Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) 2022-01-19T01:36:13.5240822Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-19T01:36:13.5241441Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-19T01:36:13.5242318Z Jan 19 01:36:13at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$1(AkkaInvocationHandler.java:258) 2022-01-19T01:36:13.5243144Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2022-01-19T01:36:13.5244370Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2022-01-19T01:36:13.5245319Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-19T01:36:13.5246074Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-19T01:36:13.5246970Z Jan 19 01:36:13at org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389) 2022-01-19T01:36:13.5247832Z Jan 19 01:36:13at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$null$1(ClassLoadingUtils.java:93) 2022-01-19T01:36:13.5248788Z Jan 19 01:36:13at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) 2022-01-19T01:36:13.5249775Z Jan 19 01:36:13at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$guardCompletionWithContextClassLoader$2(ClassLoadingUtils.java:92) 2022-01-19T01:36:13.5250826Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2022-01-19T01:36:13.5251625Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2022-01-19T01:36:13.5252531Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-19T01:36:13.5253441Z Jan 19 01:36:13at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-19T01:36:13.5254118Z Jan 19 01:36:13at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$1.onComplete(AkkaFutureUtils.java:47) 2022-01-19T01:36:13.5254753Z Jan 19 01:36:13at akka.dispatch.OnComplete.internal(Future.scala:300) 2022-01-19T01:36:13.5255381Z Jan 19 01:36:13at akka.dispatch.OnComplete.internal(Future.scala:297) 2022-01-19T01:36:13.5256202Z Jan 19 01:36:13at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) 2022-01-19T01:36:13.5256842Z Jan 19 01:36:13at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) 2022-01-19T01:36:13.5257400Z Jan 19 01:36:13at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) 2022-01-19T01:36:13.5258296Z Jan 19 01:36:13at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$DirectExecutionContext.execute(AkkaFutureUtils.java:65) 2022-01-19T01:36:13.5259150Z Jan 19 01:36:13at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68) 2022-01-19T01:36:13.5259935Z Jan 19 01:36:13at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284) 2022-01-19T01:36:13.5260764Z Jan 19 01:36:13at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284) 2022-01-19T01:36:13.5261518Z
[jira] [Created] (FLINK-25691) ElasticsearchSinkITCase.testElasticsearchSink fails on AZP
Till Rohrmann created FLINK-25691: - Summary: ElasticsearchSinkITCase.testElasticsearchSink fails on AZP Key: FLINK-25691 URL: https://issues.apache.org/jira/browse/FLINK-25691 Project: Flink Issue Type: Bug Components: Connectors / ElasticSearch Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{ElasticsearchSinkITCase.testElasticsearchSink}} fails on AZP with {code} 2022-01-18T08:10:11.9777311Z Jan 18 08:10:11 [ERROR] org.apache.flink.streaming.connectors.elasticsearch7.ElasticsearchSinkITCase.testElasticsearchSink Time elapsed: 31.816 s <<< ERROR! 2022-01-18T08:10:11.9778438Z Jan 18 08:10:11 org.apache.flink.runtime.client.JobExecutionException: Job execution failed. 2022-01-18T08:10:11.9779184Z Jan 18 08:10:11at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) 2022-01-18T08:10:11.9779993Z Jan 18 08:10:11at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137) 2022-01-18T08:10:11.9780892Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) 2022-01-18T08:10:11.9781726Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) 2022-01-18T08:10:11.9782380Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-18T08:10:11.9783097Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-18T08:10:11.9783866Z Jan 18 08:10:11at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$1(AkkaInvocationHandler.java:258) 2022-01-18T08:10:11.9784615Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2022-01-18T08:10:11.9791362Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2022-01-18T08:10:11.9792139Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-18T08:10:11.9793011Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-18T08:10:11.9793620Z Jan 18 08:10:11at org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389) 2022-01-18T08:10:11.9794267Z Jan 18 08:10:11at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$null$1(ClassLoadingUtils.java:93) 2022-01-18T08:10:11.9795177Z Jan 18 08:10:11at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) 2022-01-18T08:10:11.9796451Z Jan 18 08:10:11at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$guardCompletionWithContextClassLoader$2(ClassLoadingUtils.java:92) 2022-01-18T08:10:11.9797325Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2022-01-18T08:10:11.9798108Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2022-01-18T08:10:11.9798749Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2022-01-18T08:10:11.9799364Z Jan 18 08:10:11at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2022-01-18T08:10:11.970Z Jan 18 08:10:11at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$1.onComplete(AkkaFutureUtils.java:47) 2022-01-18T08:10:11.9800561Z Jan 18 08:10:11at akka.dispatch.OnComplete.internal(Future.scala:300) 2022-01-18T08:10:11.9801061Z Jan 18 08:10:11at akka.dispatch.OnComplete.internal(Future.scala:297) 2022-01-18T08:10:11.9801661Z Jan 18 08:10:11at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) 2022-01-18T08:10:11.9802186Z Jan 18 08:10:11at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) 2022-01-18T08:10:11.9802713Z Jan 18 08:10:11at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) 2022-01-18T08:10:11.9803348Z Jan 18 08:10:11at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$DirectExecutionContext.execute(AkkaFutureUtils.java:65) 2022-01-18T08:10:11.9804008Z Jan 18 08:10:11at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68) 2022-01-18T08:10:11.9804600Z Jan 18 08:10:11at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284) 2022-01-18T08:10:11.9805253Z Jan 18 08:10:11at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284) 2022-01-18T08:10:11.9805882Z Jan 18 08:10:11at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284) 2022-01-18T08:10:11.9806617Z Jan 18 08:10:11at akka.pattern.PromiseAc
[jira] [Created] (FLINK-25675) UnalignedCheckpointCompatibilityITCase.test hangs on AZP
Till Rohrmann created FLINK-25675: - Summary: UnalignedCheckpointCompatibilityITCase.test hangs on AZP Key: FLINK-25675 URL: https://issues.apache.org/jira/browse/FLINK-25675 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.13.5 Reporter: Till Rohrmann The test {{UnalignedCheckpointCompatibilityITCase.test}} hangs on AZP. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29511&view=logs&j=39d5b1d5-3b41-54dc-6458-1e2ddd1cdcf3&t=a99e99c7-21cd-5a1f-7274-585e62b72f56&l=6048 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25674) CassandraConnectorITCase.testCassandraPojoNoAnnotatedKeyspaceAtLeastOnceSink fails on AZP
Till Rohrmann created FLINK-25674: - Summary: CassandraConnectorITCase.testCassandraPojoNoAnnotatedKeyspaceAtLeastOnceSink fails on AZP Key: FLINK-25674 URL: https://issues.apache.org/jira/browse/FLINK-25674 Project: Flink Issue Type: Bug Components: Connectors / Cassandra Affects Versions: 1.15.0 Reporter: Till Rohrmann The test {{CassandraConnectorITCase.testCassandraPojoNoAnnotatedKeyspaceAtLeastOnceSink}} fails on AZP with {code} 2022-01-17T02:20:49.5493218Z Jan 17 02:20:49 [ERROR] testCassandraPojoNoAnnotatedKeyspaceAtLeastOnceSink Time elapsed: 15.145 s <<< ERROR! 2022-01-17T02:20:49.5494292Z Jan 17 02:20:49 com.datastax.driver.core.exceptions.AlreadyExistsException: Table flink.testpojonoannotatedkeyspace already exists 2022-01-17T02:20:49.5495503Z Jan 17 02:20:49at com.datastax.driver.core.exceptions.AlreadyExistsException.copy(AlreadyExistsException.java:111) 2022-01-17T02:20:49.5496540Z Jan 17 02:20:49at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) 2022-01-17T02:20:49.5497594Z Jan 17 02:20:49at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) 2022-01-17T02:20:49.5498647Z Jan 17 02:20:49at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63) 2022-01-17T02:20:49.5499594Z Jan 17 02:20:49at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39) 2022-01-17T02:20:49.5501059Z Jan 17 02:20:49at org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testCassandraPojoNoAnnotatedKeyspaceAtLeastOnceSink(CassandraConnectorITCase.java:449) 2022-01-17T02:20:49.5502208Z Jan 17 02:20:49at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-01-17T02:20:49.5503180Z Jan 17 02:20:49at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-01-17T02:20:49.5504178Z Jan 17 02:20:49at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-01-17T02:20:49.5604696Z Jan 17 02:20:49at java.lang.reflect.Method.invoke(Method.java:498) 2022-01-17T02:20:49.5605959Z Jan 17 02:20:49at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2022-01-17T02:20:49.5606983Z Jan 17 02:20:49at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2022-01-17T02:20:49.5608008Z Jan 17 02:20:49at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2022-01-17T02:20:49.5608991Z Jan 17 02:20:49at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2022-01-17T02:20:49.5609957Z Jan 17 02:20:49at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-01-17T02:20:49.5610970Z Jan 17 02:20:49at org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:192) 2022-01-17T02:20:49.5612021Z Jan 17 02:20:49at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2022-01-17T02:20:49.5613033Z Jan 17 02:20:49at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2022-01-17T02:20:49.5613888Z Jan 17 02:20:49at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2022-01-17T02:20:49.5614902Z Jan 17 02:20:49at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2022-01-17T02:20:49.5615847Z Jan 17 02:20:49at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2022-01-17T02:20:49.5616769Z Jan 17 02:20:49at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2022-01-17T02:20:49.5617759Z Jan 17 02:20:49at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2022-01-17T02:20:49.5618667Z Jan 17 02:20:49at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2022-01-17T02:20:49.5619532Z Jan 17 02:20:49at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2022-01-17T02:20:49.5620398Z Jan 17 02:20:49at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2022-01-17T02:20:49.5621274Z Jan 17 02:20:49at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2022-01-17T02:20:49.5622475Z Jan 17 02:20:49at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2022-01-17T02:20:49.5623473Z Jan 17 02:20:49at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-01-17T02:20:49.5624427Z Jan 17 02:20:49at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2022-01-17T02:20:49.5625607Z Jan 17 02:20:49at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:30) 2022-01-17T02:20:49.5626610Z Ja
[jira] [Created] (FLINK-25673) AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond hangs on AZP
Till Rohrmann created FLINK-25673: - Summary: AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond hangs on AZP Key: FLINK-25673 URL: https://issues.apache.org/jira/browse/FLINK-25673 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond}} hangs on AZP: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=29512&view=logs&j=a57e0635-3fad-5b08-57c7-a4142d7d6fa9&t=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7&l=18044 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25633) CPUResourceTest does not work with German Locale
Till Rohrmann created FLINK-25633: - Summary: CPUResourceTest does not work with German Locale Key: FLINK-25633 URL: https://issues.apache.org/jira/browse/FLINK-25633 Project: Flink Issue Type: Bug Components: Tests Affects Versions: 1.14.2, 1.13.5 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0, 1.13.6, 1.14.3 The {{CPUResourceTest}} does not work with a German Locale because it expects decimals to be formatted with a dot (e.g. {{0.00}} instead of {{0,00}}). I propose to fix the Locale in the {{pom.xml}} to US in order to fix this problem. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25564) TaskManagerProcessFailureStreamingRecoveryITCase>AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure fails on AZP
Till Rohrmann created FLINK-25564: - Summary: TaskManagerProcessFailureStreamingRecoveryITCase>AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure fails on AZP Key: FLINK-25564 URL: https://issues.apache.org/jira/browse/FLINK-25564 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{TaskManagerProcessFailureStreamingRecoveryITCase>AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure}} fails on AZP with {code} Jan 07 05:07:22 [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 31.057 s <<< FAILURE! - in org.apache.flink.test.recovery.TaskManagerProcessFailureStreamingRecoveryITCase Jan 07 05:07:22 [ERROR] org.apache.flink.test.recovery.TaskManagerProcessFailureStreamingRecoveryITCase.testTaskManagerProcessFailure Time elapsed: 31.012 s <<< FAILURE! Jan 07 05:07:22 java.lang.AssertionError: The program encountered a IOExceptionList : /tmp/junit2133275241637829858/junit7793757951823298127 Jan 07 05:07:22 at org.junit.Assert.fail(Assert.java:89) Jan 07 05:07:22 at org.apache.flink.test.recovery.AbstractTaskManagerProcessFailureRecoveryTest.testTaskManagerProcessFailure(AbstractTaskManagerProcessFailureRecoveryTest.java:205) Jan 07 05:07:22 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Jan 07 05:07:22 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Jan 07 05:07:22 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 07 05:07:22 at java.lang.reflect.Method.invoke(Method.java:498) Jan 07 05:07:22 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Jan 07 05:07:22 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Jan 07 05:07:22 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Jan 07 05:07:22 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Jan 07 05:07:22 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Jan 07 05:07:22 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Jan 07 05:07:22 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Jan 07 05:07:22 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Jan 07 05:07:22 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Jan 07 05:07:22 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) Jan 07 05:07:22 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Jan 07 05:07:22 at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) Jan 07 05:07:22 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) Jan 07 05:07:22 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) Jan 07 05:07:22 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) Jan 07 05:07:22 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) Jan 07 05:07:22 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) Jan 07 05:07:22 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) Jan 07 05:07:22 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) Jan 07 05:07:22 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) Jan 07 05:07:22 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Jan 07 05:07:22 at org.junit.runners.ParentRunner.run(ParentRunner.java:413) Jan 07 05:07:22 at org.junit.runner.JUnitCore.run(JUnitCore.java:137) Jan 07 05:07:22 at org.junit.runner.JUnitCore.run(JUnitCore.java:115) Jan 07 05:07:22 at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) Jan 07 05:07:22 at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) Jan 07 05:07:22 at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) Jan 07 05:07:22 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) Jan 07 05:07:22 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) Jan 07 05:07:22 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) Jan 07 05:07:22 at org.junit.platform.launcher.c
[jira] [Created] (FLINK-25554) DistributedCacheDfsTest.testSubmittingJobViaRestClusterClient fails on AZP
Till Rohrmann created FLINK-25554: - Summary: DistributedCacheDfsTest.testSubmittingJobViaRestClusterClient fails on AZP Key: FLINK-25554 URL: https://issues.apache.org/jira/browse/FLINK-25554 Project: Flink Issue Type: Bug Components: Client / Job Submission Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{DistributedCacheDfsTest.testSubmittingJobViaRestClusterClient}} failed on AZP with {code} Jan 06 03:46:14 [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 49.939 s <<< FAILURE! - in org.apache.flink.hdfstests.DistributedCacheDfsTest Jan 06 03:46:14 [ERROR] org.apache.flink.hdfstests.DistributedCacheDfsTest.testSubmittingJobViaRestClusterClient Time elapsed: 30.009 s <<< ERROR! Jan 06 03:46:14 org.junit.runners.model.TestTimedOutException: test timed out after 3 milliseconds Jan 06 03:46:14 at sun.misc.Unsafe.park(Native Method) Jan 06 03:46:14 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) Jan 06 03:46:14 at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707) Jan 06 03:46:14 at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323) Jan 06 03:46:14 at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742) Jan 06 03:46:14 at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) Jan 06 03:46:14 at org.apache.flink.hdfstests.DistributedCacheDfsTest.testSubmittingJobViaRestClusterClient(DistributedCacheDfsTest.java:155) Jan 06 03:46:14 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Jan 06 03:46:14 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Jan 06 03:46:14 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 06 03:46:14 at java.lang.reflect.Method.invoke(Method.java:498) Jan 06 03:46:14 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Jan 06 03:46:14 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Jan 06 03:46:14 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Jan 06 03:46:14 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Jan 06 03:46:14 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Jan 06 03:46:14 at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) Jan 06 03:46:14 at java.util.concurrent.FutureTask.run(FutureTask.java:266) Jan 06 03:46:14 at java.lang.Thread.run(Thread.java:748) Jan 06 03:46:14 Jan 06 03:46:17 Formatting using clusterid: testClusterID Jan 06 03:46:20 Formatting using clusterid: testClusterID Jan 06 03:46:25 Formatting using clusterid: testClusterID Jan 06 03:46:32 Formatting using clusterid: testClusterID Jan 06 03:46:47 Formatting using clusterid: testClusterID Jan 06 03:46:50 Formatting using clusterid: testClusterID Jan 06 03:46:52 Formatting using clusterid: testClusterID {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28994&view=logs&j=c91190b6-40ae-57b2-5999-31b869b0a7c1&t=41463ccd-0694-5d4d-220d-8f771e7d098b&l=15920 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25533) Preferred AllocationIDs are not respected when fulfilling pending slot requests
Till Rohrmann created FLINK-25533: - Summary: Preferred AllocationIDs are not respected when fulfilling pending slot requests Key: FLINK-25533 URL: https://issues.apache.org/jira/browse/FLINK-25533 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.14.2, 1.13.5, 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 In order to make best use of local recovery, we have to forward the set of preferred allocations to the {{DeclarativeSlotPoolBridge}} where new slots are matched with pending slot requests. At the moment this is not the case and this means that whenever we try to recover locally while not having all slots available, we might do wrong scheduling decisions. In order to improve the situation, I propose to forward the set of preferred allocations to the {{DeclarativeSlotPoolBridge}}. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25523) KafkaSourceITCase$KafkaSpecificTests.testTimestamp fails on AZP
Till Rohrmann created FLINK-25523: - Summary: KafkaSourceITCase$KafkaSpecificTests.testTimestamp fails on AZP Key: FLINK-25523 URL: https://issues.apache.org/jira/browse/FLINK-25523 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{KafkaSourceITCase$KafkaSpecificTests.testTimestamp}} fails on AZP with {code} 2022-01-05T03:08:57.1647316Z java.util.concurrent.TimeoutException: The topic metadata failed to propagate to Kafka broker. 2022-01-05T03:08:57.1660635Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:214) 2022-01-05T03:08:57.1667856Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:230) 2022-01-05T03:08:57.1668778Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:216) 2022-01-05T03:08:57.1670072Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:98) 2022-01-05T03:08:57.1671078Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:216) 2022-01-05T03:08:57.1671942Zat org.apache.flink.connector.kafka.source.KafkaSourceITCase$KafkaSpecificTests.testTimestamp(KafkaSourceITCase.java:104) 2022-01-05T03:08:57.1672619Zat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-01-05T03:08:57.1673715Zat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-01-05T03:08:57.1675000Zat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-01-05T03:08:57.1675907Zat java.base/java.lang.reflect.Method.invoke(Method.java:566) 2022-01-05T03:08:57.1676587Zat org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688) 2022-01-05T03:08:57.1677316Zat org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) 2022-01-05T03:08:57.1678380Zat org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) 2022-01-05T03:08:57.1679264Zat org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) 2022-01-05T03:08:57.1680002Zat org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140) 2022-01-05T03:08:57.1680776Zat org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92) 2022-01-05T03:08:57.1681682Zat org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) 2022-01-05T03:08:57.1682442Zat org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) 2022-01-05T03:08:57.1683450Zat org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) 2022-01-05T03:08:57.1685362Zat org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) 2022-01-05T03:08:57.1686284Zat org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) 2022-01-05T03:08:57.1687152Zat org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) 2022-01-05T03:08:57.1687818Zat org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) 2022-01-05T03:08:57.1688479Zat org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) 2022-01-05T03:08:57.1689376Zat org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210) 2022-01-05T03:08:57.1690108Zat org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) 2022-01-05T03:08:57.1690825Zat org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206) 2022-01-05T03:08:57.1691470Zat org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131) 2022-01-05T03:08:57.1692151Zat org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65) 2022-01-05T03:08:57.1693014Zat org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) 2022-01-05T03:08:57.1693762Zat org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) 2022-01-05T03:08:57.1694461Zat org.junit
[jira] [Created] (FLINK-25522) KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime
Till Rohrmann created FLINK-25522: - Summary: KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime Key: FLINK-25522 URL: https://issues.apache.org/jira/browse/FLINK-25522 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime}} failed on AZP with: {code} 2022-01-05T04:31:25.7208273Z java.util.concurrent.TimeoutException: The topic metadata failed to propagate to Kafka broker. 2022-01-05T04:31:25.7210543Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:214) 2022-01-05T04:31:25.7211289Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:230) 2022-01-05T04:31:25.7212025Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:216) 2022-01-05T04:31:25.7212944Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:98) 2022-01-05T04:31:25.7213794Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:216) 2022-01-05T04:31:25.7214854Zat org.apache.flink.streaming.connectors.kafka.shuffle.KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecovery(KafkaShuffleExactlyOnceITCase.java:158) 2022-01-05T04:31:25.7215823Zat org.apache.flink.streaming.connectors.kafka.shuffle.KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime(KafkaShuffleExactlyOnceITCase.java:81) 2022-01-05T04:31:25.7216532Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-01-05T04:31:25.7217307Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-01-05T04:31:25.7217917Zat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2022-01-05T04:31:25.7218437Zat java.lang.reflect.Method.invoke(Method.java:498) 2022-01-05T04:31:25.7218969Zat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2022-01-05T04:31:25.7219572Zat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2022-01-05T04:31:25.7220183Zat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2022-01-05T04:31:25.7220770Zat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2022-01-05T04:31:25.7221346Zat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2022-01-05T04:31:25.7221959Zat org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) 2022-01-05T04:31:25.7222603Zat org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) 2022-01-05T04:31:25.7223413Zat java.util.concurrent.FutureTask.run(FutureTask.java:266) 2022-01-05T04:31:25.7223871Zat java.lang.Thread.run(Thread.java:748) 2022-01-05T04:31:25.7321823Z java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicExistsException: Topic 'partition_failure_recovery_ProcessingTime' already exists. 2022-01-05T04:31:25.7323411Zat org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) 2022-01-05T04:31:25.7324069Zat org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) 2022-01-05T04:31:25.7324696Zat org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) 2022-01-05T04:31:25.7325309Zat org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260) 2022-01-05T04:31:25.7326077Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:214) 2022-01-05T04:31:25.7326999Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:98) 2022-01-05T04:31:25.7327659Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:216) 2022-01-05T04:31:25.7328418Zat org.apache.flink.streaming.connectors.kafka.shuffle.KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecovery(KafkaShuffleExactlyOnceITCase.java:158) 2022-01-05T04:31:25.7329328Zat org.apache.flink.streaming.connectors.kafka.shuffle.KafkaShuffleExactlyOnceITCase.testAssignedToPartitionFailureRecoveryProcessingTime(KafkaShuffleExactlyOnceITCase.java:81) 2022-01-05T04:31:25.7330013Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2022-01-05T04:31:25.7330507Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2022-01
[jira] [Created] (FLINK-25469) FlinkKafkaProducerITCase.testScaleUpAfterScalingDown fails on AZP
Till Rohrmann created FLINK-25469: - Summary: FlinkKafkaProducerITCase.testScaleUpAfterScalingDown fails on AZP Key: FLINK-25469 URL: https://issues.apache.org/jira/browse/FLINK-25469 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{FlinkKafkaProducerITCase.testScaleUpAfterScalingDown}} fails with {code} Dec 28 15:21:42 [ERROR] org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testScaleUpAfterScalingDown Time elapsed: 138.55 s <<< FAILURE! Dec 28 15:21:42 java.lang.AssertionError: Expected elements: <[0, 1, 2, 3, 4, 5, 6, 7, 8]>, but was: elements: <[0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8]> Dec 28 15:21:42 at org.junit.Assert.fail(Assert.java:89) Dec 28 15:21:42 at org.apache.flink.streaming.connectors.kafka.KafkaTestBase.assertExactlyOnceForTopic(KafkaTestBase.java:331) Dec 28 15:21:42 at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testScaleUpAfterScalingDown(FlinkKafkaProducerITCase.java:460) Dec 28 15:21:42 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Dec 28 15:21:42 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Dec 28 15:21:42 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Dec 28 15:21:42 at java.lang.reflect.Method.invoke(Method.java:498) Dec 28 15:21:42 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Dec 28 15:21:42 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Dec 28 15:21:42 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Dec 28 15:21:42 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Dec 28 15:21:42 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Dec 28 15:21:42 at org.apache.flink.testutils.junit.RetryRule$RetryOnFailureStatement.evaluate(RetryRule.java:135) Dec 28 15:21:42 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Dec 28 15:21:42 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) Dec 28 15:21:42 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Dec 28 15:21:42 at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) Dec 28 15:21:42 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) Dec 28 15:21:42 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) Dec 28 15:21:42 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) Dec 28 15:21:42 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) Dec 28 15:21:42 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) Dec 28 15:21:42 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) Dec 28 15:21:42 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) Dec 28 15:21:42 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) Dec 28 15:21:42 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Dec 28 15:21:42 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Dec 28 15:21:42 at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) Dec 28 15:21:42 at org.junit.rules.RunRules.evaluate(RunRules.java:20) Dec 28 15:21:42 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Dec 28 15:21:42 at org.junit.runners.ParentRunner.run(ParentRunner.java:413) Dec 28 15:21:42 at org.junit.runner.JUnitCore.run(JUnitCore.java:137) Dec 28 15:21:42 at org.junit.runner.JUnitCore.run(JUnitCore.java:115) Dec 28 15:21:42 at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) Dec 28 15:21:42 at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) Dec 28 15:21:42 at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) Dec 28 15:21:42 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) Dec 28 15:21:42 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) Dec 28 15:21:42 at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) Dec 28 15:21:42
[jira] [Created] (FLINK-25468) Local recovery fails if local state storage and RocksDB working directory are not on the same volume
Till Rohrmann created FLINK-25468: - Summary: Local recovery fails if local state storage and RocksDB working directory are not on the same volume Key: FLINK-25468 URL: https://issues.apache.org/jira/browse/FLINK-25468 Project: Flink Issue Type: Bug Components: Runtime / State Backends Affects Versions: 1.14.2, 1.13.5, 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0, 1.13.6, 1.14.3 Local recovery with RocksDB fails if the state storage directory is not on the same volume as RocksDB's working directory. The reason is that the {{RocksDBHandle}} only tries to hard link the RocksDB files when calling {{restoreInstanceDirectoryFromPath}}. If hard linking is not supported, then the operation fails. In order to harden this behaviour, I suggest to fall back to copying the files over if hard linking fails. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25457) Read timeouts when downloading from alicloud-mvn-mirror
Till Rohrmann created FLINK-25457: - Summary: Read timeouts when downloading from alicloud-mvn-mirror Key: FLINK-25457 URL: https://issues.apache.org/jira/browse/FLINK-25457 Project: Flink Issue Type: Bug Components: Build System / Azure Pipelines Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 Some of our builds fail with {code} [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project org.apache.flink:flink-parent:1.15-SNAPSHOT (/__w/3/s/pom.xml) has 7 errors [ERROR] Unresolveable build extension: Plugin org.apache.felix:maven-bundle-plugin:3.0.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.felix:maven-bundle-plugin:jar:3.0.1: Could not transfer artifact org.apache.felix:maven-bundle-plugin:pom:3.0.1 from/to alicloud-mvn-mirror (http://172.17.0.1:/repository/maven-central/): transfer failed for http://172.17.0.1:/repository/maven-central/org/apache/felix/maven-bundle-plugin/3.0.1/maven-bundle-plugin-3.0.1.pom: Read timed out -> [Help 2] [ERROR] Non-resolvable import POM: Could not transfer artifact com.fasterxml.jackson:jackson-bom:pom:2.13.0 from/to alicloud-mvn-mirror (http://172.17.0.1:***@ line 545, column 16: Read timed out -> [Help 3] [ERROR] Non-resolvable import POM: Could not transfer artifact org.junit:junit-bom:pom:5.8.1 from/to alicloud-mvn-mirror (http://172.17.0.1:***@ line 583, column 16: Read timed out -> [Help 3] [ERROR] Non-resolvable import POM: Could not transfer artifact org.testcontainers:testcontainers-bom:pom:1.16.2 from/to alicloud-mvn-mirror (http://172.17.0.1:***@ line 801, column 16: Read timed out -> [Help 3] [ERROR] 'dependencies.dependency.version' for org.junit.jupiter:junit-jupiter:jar is missing. @ line 193, column 15 [ERROR] 'dependencies.dependency.version' for org.junit.vintage:junit-vintage-engine:jar is missing. @ line 199, column 15 [ERROR] 'dependencies.dependency.version' for org.testcontainers:junit-jupiter:jar is missing. @ line 249, column 15 [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException [ERROR] [Help 3] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28603&view=logs&j=3e60b793-4158-5027-ac6d-4cdc51dffe1e&t=d5ed4970-7667-5f7e-2ece-62e410f74748&l=32 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25456) FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint
Till Rohrmann created FLINK-25456: - Summary: FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint Key: FLINK-25456 URL: https://issues.apache.org/jira/browse/FLINK-25456 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.14.2 Reporter: Till Rohrmann Fix For: 1.14.3 The test {{FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint}} fails with {code} 2021-12-27T02:54:54.8464375Z Dec 27 02:54:54 [ERROR] Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 285.279 s <<< FAILURE! - in org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase 2021-12-27T02:54:54.8465354Z Dec 27 02:54:54 [ERROR] testScaleDownBeforeFirstCheckpoint Time elapsed: 85.514 s <<< FAILURE! 2021-12-27T02:54:54.8468827Z Dec 27 02:54:54 java.lang.AssertionError: Detected producer leak. Thread name: kafka-producer-network-thread | producer-MockTask-002a002c-18 2021-12-27T02:54:54.8469779Z Dec 27 02:54:54at org.junit.Assert.fail(Assert.java:89) 2021-12-27T02:54:54.8470485Z Dec 27 02:54:54at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.checkProducerLeak(FlinkKafkaProducerITCase.java:847) 2021-12-27T02:54:54.8471842Z Dec 27 02:54:54at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testScaleDownBeforeFirstCheckpoint(FlinkKafkaProducerITCase.java:381) 2021-12-27T02:54:54.8472724Z Dec 27 02:54:54at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-27T02:54:54.8473509Z Dec 27 02:54:54at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-27T02:54:54.8474704Z Dec 27 02:54:54at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-27T02:54:54.8475523Z Dec 27 02:54:54at java.base/java.lang.reflect.Method.invoke(Method.java:566) 2021-12-27T02:54:54.8476258Z Dec 27 02:54:54at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-27T02:54:54.8476949Z Dec 27 02:54:54at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-27T02:54:54.8477632Z Dec 27 02:54:54at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-12-27T02:54:54.8478451Z Dec 27 02:54:54at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-27T02:54:54.8479282Z Dec 27 02:54:54at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-27T02:54:54.8479976Z Dec 27 02:54:54at org.apache.flink.testutils.junit.RetryRule$RetryOnFailureStatement.evaluate(RetryRule.java:135) 2021-12-27T02:54:54.8480696Z Dec 27 02:54:54at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-12-27T02:54:54.8481410Z Dec 27 02:54:54at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-12-27T02:54:54.8482009Z Dec 27 02:54:54at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-27T02:54:54.8482636Z Dec 27 02:54:54at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-12-27T02:54:54.8483267Z Dec 27 02:54:54at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-12-27T02:54:54.8483900Z Dec 27 02:54:54at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-12-27T02:54:54.8484574Z Dec 27 02:54:54at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-12-27T02:54:54.8485214Z Dec 27 02:54:54at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-12-27T02:54:54.8485838Z Dec 27 02:54:54at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-12-27T02:54:54.8486441Z Dec 27 02:54:54at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-12-27T02:54:54.8487037Z Dec 27 02:54:54at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-12-27T02:54:54.8487620Z Dec 27 02:54:54at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-12-27T02:54:54.8488391Z Dec 27 02:54:54at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-27T02:54:54.8489050Z Dec 27 02:54:54at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2021-12-27T02:54:54.8489685Z Dec 27 02:54:54at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-12-27T02:54:54.8490269Z Dec 27 02:54:54at org.junit.rules.RunRules.evaluate(RunRules.java:20) 2021-12-27T02:54:54.8490844Z Dec 27 02:54:54at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-27T02:54:54.8491498Z Dec
[jira] [Created] (FLINK-25455) FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator fails on AZP
Till Rohrmann created FLINK-25455: - Summary: FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator fails on AZP Key: FLINK-25455 URL: https://issues.apache.org/jira/browse/FLINK-25455 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.14.2, 1.13.5, 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0, 1.13.6, 1.14.3 The test {{FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator}} fails on AZP with {code} 2021-12-27T02:56:56.3618475Z java.util.concurrent.TimeoutException: The topic metadata failed to propagate to Kafka broker. 2021-12-27T02:56:56.3619339Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:214) 2021-12-27T02:56:56.3619958Zat org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:230) 2021-12-27T02:56:56.3620643Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:216) 2021-12-27T02:56:56.3621504Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:98) 2021-12-27T02:56:56.3625442Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:216) 2021-12-27T02:56:56.3626958Zat org.apache.flink.streaming.connectors.kafka.FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator(FlinkKafkaInternalProducerITCase.java:206) 2021-12-27T02:56:56.3627771Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-27T02:56:56.3628571Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-27T02:56:56.3629167Zat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-27T02:56:56.3629689Zat java.lang.reflect.Method.invoke(Method.java:498) 2021-12-27T02:56:56.3630183Zat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-27T02:56:56.3630764Zat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-27T02:56:56.3631484Zat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-12-27T02:56:56.3632056Zat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-27T02:56:56.3632655Zat org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) 2021-12-27T02:56:56.3633576Zat org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) 2021-12-27T02:56:56.3634123Zat java.util.concurrent.FutureTask.run(FutureTask.java:266) 2021-12-27T02:56:56.3634579Zat java.lang.Thread.run(Thread.java:748) 2021-12-27T02:56:56.3785761Z java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicExistsException: Topic 'flink-kafka-producer-txn-coordinator-changed' already exists. 2021-12-27T02:56:56.3787263Zat org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) 2021-12-27T02:56:56.3787944Zat org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) 2021-12-27T02:56:56.3788739Zat org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) 2021-12-27T02:56:56.3789355Zat org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260) 2021-12-27T02:56:56.3790029Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:214) 2021-12-27T02:56:56.3791029Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:98) 2021-12-27T02:56:56.3791831Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:216) 2021-12-27T02:56:56.3792748Zat org.apache.flink.streaming.connectors.kafka.FlinkKafkaInternalProducerITCase.testProducerWhenCommitEmptyPartitionsToOutdatedTxnCoordinator(FlinkKafkaInternalProducerITCase.java:206) 2021-12-27T02:56:56.3793447Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-27T02:56:56.3793931Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-27T02:56:56.3794501Zat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-27T02:56:56.3795020Zat java.lang.reflect.Method.invoke(Method.java:498) 2021-12-27T02:56:56.3795538Zat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-27T02:56:56.3796121Zat org.junit.internal.runners.model.ReflectiveCallable.run(Reflecti
[jira] [Created] (FLINK-25436) Allow BlobServer/BlobCache to clean up unused blobs after recovering from working directory
Till Rohrmann created FLINK-25436: - Summary: Allow BlobServer/BlobCache to clean up unused blobs after recovering from working directory Key: FLINK-25436 URL: https://issues.apache.org/jira/browse/FLINK-25436 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 In order to let the {{BlobServer}} and the {{BlobCache}} properly clean up unused blobs that are recovered from the working directory, we have to register them for clean up and offer hooks to delete irrelevant job artifacts. I propose to scan the blobStorage directory at startup and to register for transient blobs the expiry timeouts. Moreover, for the {{BlobServer}} we need to add a {{retainJobs}} method that deletes all jobs that are not in the given list of {{JobIDs}}. Last but not least we also need to register the permanent blobs in the {{PermanentBlobCacheService}} so that they get timed out if not used anymore. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25427) SavepointITCase.testTriggerSavepointAndResumeWithNoClaim fails on AZP
Till Rohrmann created FLINK-25427: - Summary: SavepointITCase.testTriggerSavepointAndResumeWithNoClaim fails on AZP Key: FLINK-25427 URL: https://issues.apache.org/jira/browse/FLINK-25427 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{SavepointITCase.testTriggerSavepointAndResumeWithNoClaim}} fails on AZP with {code} 2021-12-23T03:10:26.4240179Z Dec 23 03:10:26 [ERROR] org.apache.flink.test.checkpointing.SavepointITCase.testTriggerSavepointAndResumeWithNoClaim Time elapsed: 62.289 s <<< ERROR! 2021-12-23T03:10:26.4240998Z Dec 23 03:10:26 java.util.concurrent.TimeoutException: Condition was not met in given timeout. 2021-12-23T03:10:26.4241716Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:166) 2021-12-23T03:10:26.4242643Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:144) 2021-12-23T03:10:26.4243295Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitUntilCondition(CommonTestUtils.java:136) 2021-12-23T03:10:26.4244433Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitForAllTaskRunning(CommonTestUtils.java:210) 2021-12-23T03:10:26.4245166Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitForAllTaskRunning(CommonTestUtils.java:184) 2021-12-23T03:10:26.4245830Z Dec 23 03:10:26at org.apache.flink.runtime.testutils.CommonTestUtils.waitForAllTaskRunning(CommonTestUtils.java:172) 2021-12-23T03:10:26.4246870Z Dec 23 03:10:26at org.apache.flink.test.checkpointing.SavepointITCase.testTriggerSavepointAndResumeWithNoClaim(SavepointITCase.java:446) 2021-12-23T03:10:26.4247813Z Dec 23 03:10:26at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-23T03:10:26.4248808Z Dec 23 03:10:26at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-23T03:10:26.4249426Z Dec 23 03:10:26at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-23T03:10:26.4250192Z Dec 23 03:10:26at java.lang.reflect.Method.invoke(Method.java:498) 2021-12-23T03:10:26.4251196Z Dec 23 03:10:26at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-23T03:10:26.4252160Z Dec 23 03:10:26at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-23T03:10:26.4252888Z Dec 23 03:10:26at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-12-23T03:10:26.4253547Z Dec 23 03:10:26at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-23T03:10:26.4254142Z Dec 23 03:10:26at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-23T03:10:26.4254932Z Dec 23 03:10:26at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-12-23T03:10:26.4255513Z Dec 23 03:10:26at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-12-23T03:10:26.4256091Z Dec 23 03:10:26at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-12-23T03:10:26.4256636Z Dec 23 03:10:26at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-12-23T03:10:26.4257165Z Dec 23 03:10:26at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-23T03:10:26.4257744Z Dec 23 03:10:26at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-12-23T03:10:26.4258312Z Dec 23 03:10:26at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-12-23T03:10:26.4258884Z Dec 23 03:10:26at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-12-23T03:10:26.4259488Z Dec 23 03:10:26at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-12-23T03:10:26.4260049Z Dec 23 03:10:26at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-12-23T03:10:26.4260579Z Dec 23 03:10:26at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-12-23T03:10:26.4261108Z Dec 23 03:10:26at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-12-23T03:10:26.4261648Z Dec 23 03:10:26at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-12-23T03:10:26.4262183Z Dec 23 03:10:26at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-12-23T03:10:26.4262794Z Dec 23 03:10:26at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-23T03:10:26.4263312Z Dec 23 03:10:26at org.junit.runners.ParentRunner.run
[jira] [Created] (FLINK-25426) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP because it cannot allocate enough network buffers
Till Rohrmann created FLINK-25426: - Summary: UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP because it cannot allocate enough network buffers Key: FLINK-25426 URL: https://issues.apache.org/jira/browse/FLINK-25426 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint}} fails with {code} 2021-12-23T02:54:46.2862342Z Dec 23 02:54:46 [ERROR] UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint Time elapsed: 2.992 s <<< ERROR! 2021-12-23T02:54:46.2865774Z Dec 23 02:54:46 java.lang.OutOfMemoryError: Could not allocate enough memory segments for NetworkBufferPool (required (Mb): 64, allocated (Mb): 14, missing (Mb): 50). Cause: Direct buffer memory. The direct out-of-memory error has occurred. This can mean two things: either job(s) require(s) a larger size of JVM direct memory or there is a direct memory leak. The direct memory can be allocated by user code or some of its dependencies. In this case 'taskmanager.memory.task.off-heap.size' configuration option should be increased. Flink framework and its dependencies also consume the direct memory, mostly for network communication. The most of network memory is managed by Flink and should not result in out-of-memory error. In certain special cases, in particular for jobs with high parallelism, the framework may require more direct memory which is not managed by Flink. In this case 'taskmanager.memory.framework.off-heap.size' configuration option should be increased. If the error persists then there is probably a direct memory leak in user code or some of its dependencies which has to be investigated and fixed. The task executor has to be shutdown... 2021-12-23T02:54:46.2868239Z Dec 23 02:54:46at org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.(NetworkBufferPool.java:138) 2021-12-23T02:54:46.2868975Z Dec 23 02:54:46at org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createNettyShuffleEnvironment(NettyShuffleServiceFactory.java:140) 2021-12-23T02:54:46.2869771Z Dec 23 02:54:46at org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createNettyShuffleEnvironment(NettyShuffleServiceFactory.java:94) 2021-12-23T02:54:46.2870550Z Dec 23 02:54:46at org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createShuffleEnvironment(NettyShuffleServiceFactory.java:79) 2021-12-23T02:54:46.2871312Z Dec 23 02:54:46at org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createShuffleEnvironment(NettyShuffleServiceFactory.java:58) 2021-12-23T02:54:46.2872062Z Dec 23 02:54:46at org.apache.flink.runtime.taskexecutor.TaskManagerServices.createShuffleEnvironment(TaskManagerServices.java:414) 2021-12-23T02:54:46.2872767Z Dec 23 02:54:46at org.apache.flink.runtime.taskexecutor.TaskManagerServices.fromConfiguration(TaskManagerServices.java:282) 2021-12-23T02:54:46.2873436Z Dec 23 02:54:46at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.startTaskManager(TaskManagerRunner.java:523) 2021-12-23T02:54:46.2877615Z Dec 23 02:54:46at org.apache.flink.runtime.minicluster.MiniCluster.startTaskManager(MiniCluster.java:645) 2021-12-23T02:54:46.2878247Z Dec 23 02:54:46at org.apache.flink.runtime.minicluster.MiniCluster.startTaskManagers(MiniCluster.java:626) 2021-12-23T02:54:46.2878856Z Dec 23 02:54:46at org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:379) 2021-12-23T02:54:46.2879487Z Dec 23 02:54:46at org.apache.flink.runtime.testutils.MiniClusterResource.startMiniCluster(MiniClusterResource.java:209) 2021-12-23T02:54:46.2880152Z Dec 23 02:54:46at org.apache.flink.runtime.testutils.MiniClusterResource.before(MiniClusterResource.java:95) 2021-12-23T02:54:46.2880821Z Dec 23 02:54:46at org.apache.flink.test.util.MiniClusterWithClientResource.before(MiniClusterWithClientResource.java:64) 2021-12-23T02:54:46.2881519Z Dec 23 02:54:46at org.apache.flink.test.checkpointing.UnalignedCheckpointTestBase.execute(UnalignedCheckpointTestBase.java:151) 2021-12-23T02:54:46.2882310Z Dec 23 02:54:46at org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint(UnalignedCheckpointRescaleITCase.java:534) 2021-12-23T02:54:46.2882978Z Dec 23 02:54:46at jdk.internal.reflect.GeneratedMethodAccessor123.invoke(Unknown Source) 2021-12-23T02:54:46.2883574Z Dec 23 02:54:46at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-23T02:54:46.2884171Z Dec 23 02:54:46at java.base/java.lang.reflect.Method.invoke(Method.java:566) 2021-12-23T02:54:46.2884732Z Dec 23 02:54:46at org.junit.
[jira] [Created] (FLINK-25406) Make sure that FileSystemBlobStore flushes all writes to disk to persist stored blobs
Till Rohrmann created FLINK-25406: - Summary: Make sure that FileSystemBlobStore flushes all writes to disk to persist stored blobs Key: FLINK-25406 URL: https://issues.apache.org/jira/browse/FLINK-25406 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 In order to avoid that the {{FileSystemBlobStore}} loses blobs, we have to make sure that all written results are flushed from the OS page cache to disk (fsynced). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25405) Let BlobServer/BlobCache detect & delete corrupted blobs
Till Rohrmann created FLINK-25405: - Summary: Let BlobServer/BlobCache detect & delete corrupted blobs Key: FLINK-25405 URL: https://issues.apache.org/jira/browse/FLINK-25405 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25404) Store blobs in /blobs
Till Rohrmann created FLINK-25404: - Summary: Store blobs in /blobs Key: FLINK-25404 URL: https://issues.apache.org/jira/browse/FLINK-25404 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 With FLINK-25402, we now have a working directory at our disposal. I suggest to store the blobs in this directory to make the recoverable in case of a process restart with the same working directory. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25403) Make BlobServer/BlobCache compatible with local working directories
Till Rohrmann created FLINK-25403: - Summary: Make BlobServer/BlobCache compatible with local working directories Key: FLINK-25403 URL: https://issues.apache.org/jira/browse/FLINK-25403 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 With [FLIP-198|https://cwiki.apache.org/confluence/x/ZZiqCw] Flink processes can now have a working directory that they use to store "semi-persistent" information. The stored blobs by the {{BlobServer}} and the {{BlobCache}} can be candidates for such a storage because in case of a recovery we might not have to re-download them. In order to make the {{BlobServer}} and {{BlobCache}} work together with the local working directories, they have be able to detect corrupted blobs (e.g. due to process crashes when downloading blobs) and delete them. Therefore, I propose to let the {{BlobServer}} and {{BlobCache}} remember which blobs they have verified (by recomputing their checksums) and to delete those that did not pass. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25402) FLIP-198: Working directory for Flink processes
Till Rohrmann created FLINK-25402: - Summary: FLIP-198: Working directory for Flink processes Key: FLINK-25402 URL: https://issues.apache.org/jira/browse/FLINK-25402 Project: Flink Issue Type: New Feature Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Assignee: Till Rohrmann Fix For: 1.15.0 This issue implements [FLIP-198|https://cwiki.apache.org/confluence/x/ZZiqCw]. The idea is to introduce a working directory that can be used by Flink processes to store data that can be recovered if the process is restarted with access to the same working directory (e.g. via persistent volumes or restarting the process on the same node). The working directory will be default created in the temporary directory of the executing node. Moreover, the path will include the resource id of the Flink process. This will ensure that by default (random resource id), a newly started Flink process will see an empty working directory. This is an optional feature that might not be usable for all Flink deployments. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25399) AZP fails with exit code 137 when running checkpointing test cases
Till Rohrmann created FLINK-25399: - Summary: AZP fails with exit code 137 when running checkpointing test cases Key: FLINK-25399 URL: https://issues.apache.org/jira/browse/FLINK-25399 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The AZP build for fine grained resource management failed with exit code 137, when running an extensive list of checkpointing tests: {code} 2021-12-21T06:06:08.8728404Z Dec 21 06:06:08 [INFO] Running org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase 2021-12-21T06:06:37.6584668Z Dec 21 06:06:37 Starting org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 0]. 2021-12-21T06:06:37.6585685Z Dec 21 06:06:37 Starting org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 0]. 2021-12-21T06:06:37.6593448Z Dec 21 06:06:37 Finished org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 0]. 2021-12-21T06:06:41.3044200Z Dec 21 06:06:41 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:06:41.3045146Z Dec 21 06:06:41 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:06:49.7482529Z Dec 21 06:06:49 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindowWithKVStateMinMaxParallelism[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:06:49.7483922Z Dec 21 06:06:49 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindowWithKVStateMinMaxParallelism[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:06:56.7462828Z Dec 21 06:06:56 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:06:56.7463831Z Dec 21 06:06:56 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:06.7225398Z Dec 21 06:07:06 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindowWithKVStateMaxMaxParallelism[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:06.7226580Z Dec 21 06:07:06 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testTumblingTimeWindowWithKVStateMaxMaxParallelism[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:12.1987555Z Dec 21 06:07:12 Starting org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 1]. 2021-12-21T06:07:12.1992168Z Dec 21 06:07:12 Starting org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 1]. 2021-12-21T06:07:12.1993591Z Dec 21 06:07:12 Finished org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase#shouldRescaleUnalignedCheckpoint[upscale union from 3 to 7, buffersPerChannel = 1]. 2021-12-21T06:07:16.3825669Z Dec 21 06:07:15 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testPreAggregatedTumblingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:16.3826827Z Dec 21 06:07:15 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testPreAggregatedTumblingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:23.4489701Z Dec 21 06:07:23 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testPreAggregatedSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:23.4495250Z Dec 21 06:07:23 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testPreAggregatedSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 0]. 2021-12-21T06:07:29.8407484Z Dec 21 06:07:29 Starting org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 2]. 2021-12-21T06:07:29.8417385Z Dec 21 06:07:29 Finished org.apache.flink.test.checkpointing.EventTimeWindowCheckpointingITCase#testSlidingTimeWindow[statebackend type =MEM, buffersPerChannel = 2]. 2021-12-21T06:07:31.2376012Z Dec 21 06:07:31 Starting org.apache.flink.tes
[jira] [Created] (FLINK-25381) JdbcExactlyOnceSinkE2eTest.testInsert fails on AZP
Till Rohrmann created FLINK-25381: - Summary: JdbcExactlyOnceSinkE2eTest.testInsert fails on AZP Key: FLINK-25381 URL: https://issues.apache.org/jira/browse/FLINK-25381 Project: Flink Issue Type: Bug Components: Connectors / JDBC Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{JdbcExactlyOnceSinkE2eTest.testInsert}} fails on AZP with {code} 2021-12-20T03:13:06.5415073Z Dec 20 03:13:06 org.apache.flink.runtime.client.JobExecutionException: Job execution failed. 2021-12-20T03:13:06.5415676Z Dec 20 03:13:06at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) 2021-12-20T03:13:06.5416393Z Dec 20 03:13:06at org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137) 2021-12-20T03:13:06.5417081Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616) 2021-12-20T03:13:06.5417686Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) 2021-12-20T03:13:06.5418283Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2021-12-20T03:13:06.5418892Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2021-12-20T03:13:06.5424332Z Dec 20 03:13:06at org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$1(AkkaInvocationHandler.java:258) 2021-12-20T03:13:06.5425255Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2021-12-20T03:13:06.5425994Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2021-12-20T03:13:06.5426730Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2021-12-20T03:13:06.5427397Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2021-12-20T03:13:06.5428007Z Dec 20 03:13:06at org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389) 2021-12-20T03:13:06.5428649Z Dec 20 03:13:06at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$null$1(ClassLoadingUtils.java:93) 2021-12-20T03:13:06.5429604Z Dec 20 03:13:06at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68) 2021-12-20T03:13:06.5430674Z Dec 20 03:13:06at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.lambda$guardCompletionWithContextClassLoader$2(ClassLoadingUtils.java:92) 2021-12-20T03:13:06.5431696Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) 2021-12-20T03:13:06.5432744Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) 2021-12-20T03:13:06.5433645Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2021-12-20T03:13:06.5434697Z Dec 20 03:13:06at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) 2021-12-20T03:13:06.5435356Z Dec 20 03:13:06at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$1.onComplete(AkkaFutureUtils.java:47) 2021-12-20T03:13:06.5436145Z Dec 20 03:13:06at akka.dispatch.OnComplete.internal(Future.scala:300) 2021-12-20T03:13:06.5436855Z Dec 20 03:13:06at akka.dispatch.OnComplete.internal(Future.scala:297) 2021-12-20T03:13:06.5437690Z Dec 20 03:13:06at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) 2021-12-20T03:13:06.5438367Z Dec 20 03:13:06at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) 2021-12-20T03:13:06.5439147Z Dec 20 03:13:06at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) 2021-12-20T03:13:06.5440092Z Dec 20 03:13:06at org.apache.flink.runtime.concurrent.akka.AkkaFutureUtils$DirectExecutionContext.execute(AkkaFutureUtils.java:65) 2021-12-20T03:13:06.5441108Z Dec 20 03:13:06at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68) 2021-12-20T03:13:06.5442086Z Dec 20 03:13:06at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284) 2021-12-20T03:13:06.5443269Z Dec 20 03:13:06at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284) 2021-12-20T03:13:06.5444506Z Dec 20 03:13:06at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284) 2021-12-20T03:13:06.5445078Z Dec 20 03:13:06at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:621) 2021-12-20T03:13:06.5445682Z Dec 20 03:13:06at akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:24) 202
[jira] [Created] (FLINK-25354) Document API graduation process
Till Rohrmann created FLINK-25354: - Summary: Document API graduation process Key: FLINK-25354 URL: https://issues.apache.org/jira/browse/FLINK-25354 Project: Flink Issue Type: Sub-task Components: Documentation Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 Document how the API graduation process works in the developer wiki and also add a paragraph to Flink's documentation so that users understand how fast they can APIs expect to stabilize. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25353) Add tests to ensure API graduation process is honoured
Till Rohrmann created FLINK-25353: - Summary: Add tests to ensure API graduation process is honoured Key: FLINK-25353 URL: https://issues.apache.org/jira/browse/FLINK-25353 Project: Flink Issue Type: Sub-task Components: Tests Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 We need a test that ensures that the API graduation process is honoured. Concretely, this means that the test should fail if it sees an annotated API that has its API stability since more than 2 minor release cycles w/o having a reason for the missed graduation, then the test should fail. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25352) Update stability annotations to include the since and missedGraduations fields
Till Rohrmann created FLINK-25352: - Summary: Update stability annotations to include the since and missedGraduations fields Key: FLINK-25352 URL: https://issues.apache.org/jira/browse/FLINK-25352 Project: Flink Issue Type: Sub-task Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 In order to implement the graduation process outlined in FLIP-197, we need to extend our stability annotations to include a {{since}} and {{missedGraduations}} fields. The idea of {{since}} is that it tells since when something has this stability guarantee. The idea of the {{missedGraduations}} field is to record reasons why an API has not been graduated. {code} @Target(ElementType.TYPE) public @interface PublicEvolving { FlinkVersion since(); GraduationMiss[] missedGraduations(); } public @interface GraduationMiss { FlinkVersion graduation(); String reason(); } // Usage @PublicEvolving( since = FlinkVersion.V1_11_0, missedGraduations = { @GraduationMiss(graduation = FlinkVersion.V1_13_0, reason = "foobar"), @GraduationMiss(graduation = FlinkVersion.V1_14_0, reason = "barfoo") }) public class Foobar {} {code} -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25351) Add FlinkVersion
Till Rohrmann created FLINK-25351: - Summary: Add FlinkVersion Key: FLINK-25351 URL: https://issues.apache.org/jira/browse/FLINK-25351 Project: Flink Issue Type: Sub-task Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 In order to check when a method needs to graduate we need a {{FlinkVersion}} enum that can represent the different versions. Moreover, we should add it to the release guide that this enum needs to be extended for every Flink version. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25350) Verify stability guarantees of annotated classes
Till Rohrmann created FLINK-25350: - Summary: Verify stability guarantees of annotated classes Key: FLINK-25350 URL: https://issues.apache.org/jira/browse/FLINK-25350 Project: Flink Issue Type: Sub-task Components: Tests Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 In order to give API stability guarantees we should add a test which ensures that an API class can at most give a stability guarantee of the weakest annotated method a user needs to implement for this class. This will ensure that we can extend API classes with methods that have weaker guarantees but that must have default implementations. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25349) Add test that ensures that parameter/return types have the same or stronger stability guarantee than method
Till Rohrmann created FLINK-25349: - Summary: Add test that ensures that parameter/return types have the same or stronger stability guarantee than method Key: FLINK-25349 URL: https://issues.apache.org/jira/browse/FLINK-25349 Project: Flink Issue Type: Sub-task Components: Build System Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 For FLIP-196, we need a test that ensures that the transitive closure of methods wrt stability guarantees is respected. This means that return and parameter types of a method that has a stability guarantee must be at least as strong as the guarantee that is given for the method. We might already have an architecture test for this case (we should check for it). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25348) Update release guide to reset japicmp exceptions for every release
Till Rohrmann created FLINK-25348: - Summary: Update release guide to reset japicmp exceptions for every release Key: FLINK-25348 URL: https://issues.apache.org/jira/browse/FLINK-25348 Project: Flink Issue Type: Sub-task Components: Build System Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 I propose to clean up the japicmp maven plugin exclusion for every minor release for @Public and the exclusions for @PublicEvolving with every patch release. Currently, we don’t do this and that’s why we have accumulated quite some list of exclusions that a) might shadow other problems and b) nobody really knows why they are still relevant. I would propose to make this part of the release guide. The result should be that we minimize our set of exclusions. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25346) FLIP-197: API stability graduation process
Till Rohrmann created FLINK-25346: - Summary: FLIP-197: API stability graduation process Key: FLINK-25346 URL: https://issues.apache.org/jira/browse/FLINK-25346 Project: Flink Issue Type: Improvement Components: API / DataStream Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 This ticket is an umbrella ticket for the work on [FLIP-197|https://cwiki.apache.org/confluence/x/J5eqCw]. It will involve the introduction of an API graduation process that is guarded by automated tests. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25345) FLIP-196: Source API stability guarantees
Till Rohrmann created FLINK-25345: - Summary: FLIP-196: Source API stability guarantees Key: FLINK-25345 URL: https://issues.apache.org/jira/browse/FLINK-25345 Project: Flink Issue Type: Improvement Components: API / DataStream, Documentation Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 This ticket is an umbrella ticket for the work on [FLIP-196|https://cwiki.apache.org/confluence/x/IJeqCw] which will properly document and guard the agreed upon source API stability guarantees. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25259) NO_CALLS_TO_VISIBLE_FOR_TESTING_METHODS should take the package into consideration
Till Rohrmann created FLINK-25259: - Summary: NO_CALLS_TO_VISIBLE_FOR_TESTING_METHODS should take the package into consideration Key: FLINK-25259 URL: https://issues.apache.org/jira/browse/FLINK-25259 Project: Flink Issue Type: Improvement Components: Tests Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The new architecture test {{NO_CALLS_TO_VISIBLE_FOR_TESTING_METHODS}} checks whether production code calls a method that is annotated with {{@VisibleForTesting}}. This can lead to false positives if this method is called from production code in the same package. Therefore, I suggest to take also the package into consideration when checking for this condition. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25257) Make the TaskExecutor not depend on concrete BlobServiceCache implementation
Till Rohrmann created FLINK-25257: - Summary: Make the TaskExecutor not depend on concrete BlobServiceCache implementation Key: FLINK-25257 URL: https://issues.apache.org/jira/browse/FLINK-25257 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 In order to better evolve the {{TaskExecutor}} component, I suggest to not make it depend on the concrete {{BlobServiceCache}} implementation. Instead I suggest to introduce a {{TaskExecutorBlobCacheService}} interface that can be implemented by the {{BlobServiceCache}}. Additionally, should make the {{TaskExecutor}} independent of the {{PermanentBlobCache}} and {{TransientBlobCache}} implementations. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25243) Fabric8FlinkKubeClientITCase.testCheckAndUpdateConfigMapConcurrently
Till Rohrmann created FLINK-25243: - Summary: Fabric8FlinkKubeClientITCase.testCheckAndUpdateConfigMapConcurrently Key: FLINK-25243 URL: https://issues.apache.org/jira/browse/FLINK-25243 Project: Flink Issue Type: Bug Components: Deployment / Kubernetes Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{Fabric8FlinkKubeClientITCase.testCheckAndUpdateConfigMapConcurrently}} fails on AZP with {code} 2021-12-10T01:11:11.4908955Z Dec 10 01:11:11 [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.186 s <<< FAILURE! - in org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClientITCase 2021-12-10T01:11:11.4920172Z Dec 10 01:11:11 [ERROR] testCheckAndUpdateConfigMapConcurrently(org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClientITCase) Time elapsed: 0.637 s <<< ERROR! 2021-12-10T01:11:11.4921592Z Dec 10 01:11:11 java.util.concurrent.ExecutionException: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted. 2021-12-10T01:11:11.4925669Z Dec 10 01:11:11at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) 2021-12-10T01:11:11.4926769Z Dec 10 01:11:11at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928) 2021-12-10T01:11:11.4927867Z Dec 10 01:11:11at org.apache.flink.kubernetes.kubeclient.Fabric8FlinkKubeClientITCase.testCheckAndUpdateConfigMapConcurrently(Fabric8FlinkKubeClientITCase.java:144) 2021-12-10T01:11:11.4928904Z Dec 10 01:11:11at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-10T01:11:11.4929769Z Dec 10 01:11:11at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-10T01:11:11.4930729Z Dec 10 01:11:11at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-10T01:11:11.4931607Z Dec 10 01:11:11at java.lang.reflect.Method.invoke(Method.java:498) 2021-12-10T01:11:11.4932485Z Dec 10 01:11:11at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) 2021-12-10T01:11:11.4933449Z Dec 10 01:11:11at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-10T01:11:11.4934395Z Dec 10 01:11:11at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) 2021-12-10T01:11:11.4935330Z Dec 10 01:11:11at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-10T01:11:11.4937971Z Dec 10 01:11:11at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-10T01:11:11.4939315Z Dec 10 01:11:11at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2021-12-10T01:11:11.4940050Z Dec 10 01:11:11at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-12-10T01:11:11.4940711Z Dec 10 01:11:11at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) 2021-12-10T01:11:11.4941320Z Dec 10 01:11:11at org.junit.rules.RunRules.evaluate(RunRules.java:20) 2021-12-10T01:11:11.4941928Z Dec 10 01:11:11at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) 2021-12-10T01:11:11.4942797Z Dec 10 01:11:11at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) 2021-12-10T01:11:11.4943519Z Dec 10 01:11:11at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) 2021-12-10T01:11:11.4944180Z Dec 10 01:11:11at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) 2021-12-10T01:11:11.4944815Z Dec 10 01:11:11at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) 2021-12-10T01:11:11.4945461Z Dec 10 01:11:11at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) 2021-12-10T01:11:11.4946107Z Dec 10 01:11:11at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) 2021-12-10T01:11:11.4946749Z Dec 10 01:11:11at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) 2021-12-10T01:11:11.4947402Z Dec 10 01:11:11at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) 2021-12-10T01:11:11.4948030Z Dec 10 01:11:11at org.junit.rules.RunRules.evaluate(RunRules.java:20) 2021-12-10T01:11:11.4948625Z Dec 10 01:11:11at org.junit.runners.ParentRunner.run(ParentRunner.java:363) 2021-12-10T01:11:11.4949291Z Dec 10 01:11:11at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) 2021-12-10T01:11:11.4951248Z Dec 10 01:11:11at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) 2021-12-10T01:11:11.4951968Z Dec 10 01:11:11at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:
[jira] [Created] (FLINK-25233) UpsertKafkaTableITCase.testAggregate fails on AZP
Till Rohrmann created FLINK-25233: - Summary: UpsertKafkaTableITCase.testAggregate fails on AZP Key: FLINK-25233 URL: https://issues.apache.org/jira/browse/FLINK-25233 Project: Flink Issue Type: Bug Components: Table SQL / Ecosystem Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 {{UpsertKafkaTableITCase.testAggregate}} fails on AZP with {code} 2021-12-09T01:41:49.8038402Z Dec 09 01:41:49 [ERROR] UpsertKafkaTableITCase.testAggregate Time elapsed: 90.624 s <<< ERROR! 2021-12-09T01:41:49.8039372Z Dec 09 01:41:49 java.util.concurrent.ExecutionException: org.apache.flink.table.api.TableException: Failed to wait job finish 2021-12-09T01:41:49.8040303Z Dec 09 01:41:49at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) 2021-12-09T01:41:49.8040956Z Dec 09 01:41:49at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) 2021-12-09T01:41:49.8041862Z Dec 09 01:41:49at org.apache.flink.table.api.internal.TableResultImpl.awaitInternal(TableResultImpl.java:118) 2021-12-09T01:41:49.8042939Z Dec 09 01:41:49at org.apache.flink.table.api.internal.TableResultImpl.await(TableResultImpl.java:81) 2021-12-09T01:41:49.8044130Z Dec 09 01:41:49at org.apache.flink.streaming.connectors.kafka.table.UpsertKafkaTableITCase.wordCountToUpsertKafka(UpsertKafkaTableITCase.java:436) 2021-12-09T01:41:49.8045308Z Dec 09 01:41:49at org.apache.flink.streaming.connectors.kafka.table.UpsertKafkaTableITCase.testAggregate(UpsertKafkaTableITCase.java:79) 2021-12-09T01:41:49.8045940Z Dec 09 01:41:49at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-09T01:41:49.8052892Z Dec 09 01:41:49at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-09T01:41:49.8053812Z Dec 09 01:41:49at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-09T01:41:49.8054458Z Dec 09 01:41:49at java.lang.reflect.Method.invoke(Method.java:498) 2021-12-09T01:41:49.8055027Z Dec 09 01:41:49at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-09T01:41:49.8055649Z Dec 09 01:41:49at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-09T01:41:49.8056644Z Dec 09 01:41:49at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-12-09T01:41:49.8057911Z Dec 09 01:41:49at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-09T01:41:49.8058858Z Dec 09 01:41:49at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-09T01:41:49.8059907Z Dec 09 01:41:49at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2021-12-09T01:41:49.8060871Z Dec 09 01:41:49at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-12-09T01:41:49.8061847Z Dec 09 01:41:49at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-12-09T01:41:49.8062898Z Dec 09 01:41:49at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-12-09T01:41:49.8063804Z Dec 09 01:41:49at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-09T01:41:49.8064963Z Dec 09 01:41:49at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-12-09T01:41:49.8065992Z Dec 09 01:41:49at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-12-09T01:41:49.8066940Z Dec 09 01:41:49at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-12-09T01:41:49.8067939Z Dec 09 01:41:49at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-12-09T01:41:49.8068904Z Dec 09 01:41:49at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-12-09T01:41:49.8069837Z Dec 09 01:41:49at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-12-09T01:41:49.8070715Z Dec 09 01:41:49at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-12-09T01:41:49.8071587Z Dec 09 01:41:49at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-12-09T01:41:49.8072582Z Dec 09 01:41:49at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-12-09T01:41:49.8073540Z Dec 09 01:41:49at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 2021-12-09T01:41:49.8074407Z Dec 09 01:41:49at org.junit.runners.Suite.runChild(Suite.java:128) 2021-12-09T01:41:49.8075215Z Dec 09 01:41:49at org.junit.runners.Suite.runChild(Suite.java:27) 2021-12-09T01:41:49.8080759Z Dec 09 01:41:49at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-12-09T01:41:49.8081754Z Dec 09
[jira] [Created] (FLINK-25223) ElasticsearchWriterITCase fails on AZP
Till Rohrmann created FLINK-25223: - Summary: ElasticsearchWriterITCase fails on AZP Key: FLINK-25223 URL: https://issues.apache.org/jira/browse/FLINK-25223 Project: Flink Issue Type: Bug Components: Connectors / ElasticSearch Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The {{ElasticsearchWriterITCase}} fails on AZP because {code} 2021-12-08T13:56:59.5449851Z Dec 08 13:56:59 [ERROR] org.apache.flink.connector.elasticsearch.sink.ElasticsearchWriterITCase Time elapsed: 171.046 s <<< ERROR! 2021-12-08T13:56:59.5450680Z Dec 08 13:56:59 org.testcontainers.containers.ContainerLaunchException: Container startup failed 2021-12-08T13:56:59.5451652Z Dec 08 13:56:59at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336) 2021-12-08T13:56:59.5452677Z Dec 08 13:56:59at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317) 2021-12-08T13:56:59.5453637Z Dec 08 13:56:59at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.start(TestcontainersExtension.java:242) 2021-12-08T13:56:59.5454757Z Dec 08 13:56:59at org.testcontainers.junit.jupiter.TestcontainersExtension$StoreAdapter.access$200(TestcontainersExtension.java:229) 2021-12-08T13:56:59.5455946Z Dec 08 13:56:59at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$null$1(TestcontainersExtension.java:59) 2021-12-08T13:56:59.5457322Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore.lambda$getOrComputeIfAbsent$4(ExtensionValuesStore.java:86) 2021-12-08T13:56:59.5458571Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.computeValue(ExtensionValuesStore.java:223) 2021-12-08T13:56:59.5459771Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore$MemoizingSupplier.get(ExtensionValuesStore.java:211) 2021-12-08T13:56:59.5460693Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.evaluate(ExtensionValuesStore.java:191) 2021-12-08T13:56:59.5461437Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore$StoredValue.access$100(ExtensionValuesStore.java:171) 2021-12-08T13:56:59.5462198Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.ExtensionValuesStore.getOrComputeIfAbsent(ExtensionValuesStore.java:89) 2021-12-08T13:56:59.5467999Z Dec 08 13:56:59at org.junit.jupiter.engine.execution.NamespaceAwareStore.getOrComputeIfAbsent(NamespaceAwareStore.java:53) 2021-12-08T13:56:59.5468791Z Dec 08 13:56:59at org.testcontainers.junit.jupiter.TestcontainersExtension.lambda$beforeAll$2(TestcontainersExtension.java:59) 2021-12-08T13:56:59.5469436Z Dec 08 13:56:59at java.util.ArrayList.forEach(ArrayList.java:1259) 2021-12-08T13:56:59.5470058Z Dec 08 13:56:59at org.testcontainers.junit.jupiter.TestcontainersExtension.beforeAll(TestcontainersExtension.java:59) 2021-12-08T13:56:59.5470846Z Dec 08 13:56:59at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$10(ClassBasedTestDescriptor.java:381) 2021-12-08T13:56:59.5471641Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) 2021-12-08T13:56:59.5472403Z Dec 08 13:56:59at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:381) 2021-12-08T13:56:59.5473190Z Dec 08 13:56:59at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:205) 2021-12-08T13:56:59.5474001Z Dec 08 13:56:59at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:80) 2021-12-08T13:56:59.5474759Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:148) 2021-12-08T13:56:59.5475833Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) 2021-12-08T13:56:59.5476739Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) 2021-12-08T13:56:59.5477520Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) 2021-12-08T13:56:59.5478227Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) 2021-12-08T13:56:59.5479190Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) 2021-12-08T13:56:59.5479936Z Dec 08 13:56:59at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) 2021-12
[jira] [Created] (FLINK-25214) FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on AZP with incorrect result
Till Rohrmann created FLINK-25214: - Summary: FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce fails on AZP with incorrect result Key: FLINK-25214 URL: https://issues.apache.org/jira/browse/FLINK-25214 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The {{FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce}} fails on AZP with: {code} 2021-12-06T09:25:04.6680108Z Dec 06 09:25:04 [INFO] Running org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase 2021-12-06T09:29:12.9836055Z java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=deleteTopics, deadlineMs=1638783005846, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s) 2021-12-06T09:29:12.9837836Zat org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) 2021-12-06T09:29:12.9838954Zat org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) 2021-12-06T09:29:12.9840187Zat org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104) 2021-12-06T09:29:12.9841265Zat org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272) 2021-12-06T09:29:12.9842616Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.tryDelete(KafkaTestEnvironmentImpl.java:184) 2021-12-06T09:29:12.9844259Zat org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.deleteTestTopic(KafkaTestEnvironmentImpl.java:169) 2021-12-06T09:29:12.9845414Zat org.apache.flink.streaming.connectors.kafka.KafkaTestBase.deleteTestTopic(KafkaTestBase.java:226) 2021-12-06T09:29:12.9846724Zat org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase.testMigrateFromAtLeastOnceToExactlyOnce(FlinkKafkaProducerITCase.java:593) 2021-12-06T09:29:12.9847782Zat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-12-06T09:29:12.9848744Zat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-12-06T09:29:12.9849994Zat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-12-06T09:29:12.9850849Zat java.lang.reflect.Method.invoke(Method.java:498) 2021-12-06T09:29:12.9851743Zat org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-12-06T09:29:12.9852821Zat org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-12-06T09:29:12.9856589Zat org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-12-06T09:29:12.9857480Zat org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-12-06T09:29:12.9858129Zat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-06T09:29:12.9859027Zat org.apache.flink.testutils.junit.RetryRule$RetryOnFailureStatement.evaluate(RetryRule.java:135) 2021-12-06T09:29:12.9860062Zat org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-12-06T09:29:12.9860880Zat org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-12-06T09:29:12.9861657Zat org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-12-06T09:29:12.9862561Zat org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-12-06T09:29:12.9863345Zat org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-12-06T09:29:12.9864115Zat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-12-06T09:29:12.9865058Zat org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-12-06T09:29:12.9865970Zat org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-12-06T09:29:12.9866852Zat org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-12-06T09:29:12.9867634Zat org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-12-06T09:29:12.9868237Zat org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-12-06T09:29:12.9869005Zat org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-12-06T09:29:12.9870173Zat org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-12-06T09:29:12.9871080Zat org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2021-12-06T09:29:12.9871904Zat org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-12-06T09:29:12.9872769Zat org.junit.rules.RunRules.evaluate(RunRules.java:20) 2021-12-06T09:29:12.9873467Zat org.junit.runners.Paren
[jira] [Created] (FLINK-25186) KafkaDynamicTableFactoryTest and UpsertKafkaDynamicTableFactoryTest fails on AZP
Till Rohrmann created FLINK-25186: - Summary: KafkaDynamicTableFactoryTest and UpsertKafkaDynamicTableFactoryTest fails on AZP Key: FLINK-25186 URL: https://issues.apache.org/jira/browse/FLINK-25186 Project: Flink Issue Type: Bug Components: Connectors / Kafka, Table SQL / Ecosystem Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 A lot of {{KafkaDynamicTableFactoryTest}} and {{UpsertKafkaDynamicTableFactoryTest}} tests fail on AZP. {code} Dec 06 03:00:28 [ERROR] UpsertKafkaDynamicTableFactoryTest.testInvalidSinkBufferFlush Dec 06 03:00:28 Expected: (an instance of org.apache.flink.table.api.ValidationException and Expected failure cause is ) Dec 06 03:00:28 but: Expected failure cause is The throwable https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27569&view=logs&j=ce8f3cc3-c1ea-5281-f5eb-df9ebd24947f&t=918e890f-5ed9-5212-a25e-962628fb4bc5&l=10186 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25185) StreamFaultToleranceTestBase hangs on AZP
Till Rohrmann created FLINK-25185: - Summary: StreamFaultToleranceTestBase hangs on AZP Key: FLINK-25185 URL: https://issues.apache.org/jira/browse/FLINK-25185 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing, Test Infrastructure Affects Versions: 1.13.3 Reporter: Till Rohrmann The {{StreamFaultToleranceTestBase}} hangs on AZP. {code} 2021-12-06T04:24:48.1676089Z == 2021-12-06T04:24:48.1678883Z === WARNING: This task took already 95% of the available time budget of 237 minutes === 2021-12-06T04:24:48.1679596Z == 2021-12-06T04:24:48.1680326Z == 2021-12-06T04:24:48.1680877Z The following Java processes are running (JPS) 2021-12-06T04:24:48.1681467Z == 2021-12-06T04:24:48.6514536Z 13701 surefirebooter17740627448580534543.jar 2021-12-06T04:24:48.6515353Z 1622 Jps 2021-12-06T04:24:48.6515795Z 780 Launcher 2021-12-06T04:24:48.6825889Z == 2021-12-06T04:24:48.6826565Z Printing stack trace of Java process 13701 2021-12-06T04:24:48.6827012Z == 2021-12-06T04:24:49.1876086Z 2021-12-06 04:24:49 2021-12-06T04:24:49.1877098Z Full thread dump OpenJDK 64-Bit Server VM (11.0.10+9 mixed mode): 2021-12-06T04:24:49.1877362Z 2021-12-06T04:24:49.1877672Z Threads class SMR info: 2021-12-06T04:24:49.1878049Z _java_thread_list=0x7f254c007630, length=365, elements={ 2021-12-06T04:24:49.1878504Z 0x7f2598028000, 0x7f2598280800, 0x7f2598284800, 0x7f2598299000, 2021-12-06T04:24:49.1878973Z 0x7f259829b000, 0x7f259829d800, 0x7f259829f800, 0x7f25982a1800, 2021-12-06T04:24:49.1879680Z 0x7f2598337800, 0x7f25983e3000, 0x7f2598431000, 0x7f2528016000, 2021-12-06T04:24:49.1896613Z 0x7f2599003000, 0x7f259972e000, 0x7f2599833800, 0x7f259984c000, 2021-12-06T04:24:49.1897558Z 0x7f259984f000, 0x7f2599851000, 0x7f2599892000, 0x7f2599894800, 2021-12-06T04:24:49.1898075Z 0x7f2499a16000, 0x7f2485acd800, 0x7f2485ace000, 0x7f24876bb800, 2021-12-06T04:24:49.1898562Z 0x7f2461e59000, 0x7f2499a0e800, 0x7f2461e5e800, 0x7f2461e81000, 2021-12-06T04:24:49.1899037Z 0x7f24dc015000, 0x7f2461e86800, 0x7f2448002000, 0x7f24dc01c000, 2021-12-06T04:24:49.1899522Z 0x7f2438001000, 0x7f2438003000, 0x7f2438005000, 0x7f2438006800, 2021-12-06T04:24:49.1899982Z 0x7f2438008800, 0x7f2434017800, 0x7f243401a800, 0x7f2414008800, 2021-12-06T04:24:49.1900495Z 0x7f24e8089800, 0x7f24e809, 0x7f23e4005800, 0x7f24e8092800, 2021-12-06T04:24:49.1901163Z 0x7f24e8099000, 0x7f2414015800, 0x7f24dc04c000, 0x7f2414018800, 2021-12-06T04:24:49.1901680Z 0x7f241402, 0x7f24dc058000, 0x7f24dc05b000, 0x7f2414022000, 2021-12-06T04:24:49.1902283Z 0x7f24d400f000, 0x7f241402e800, 0x7f2414031800, 0x7f2414033800, 2021-12-06T04:24:49.1902880Z 0x7f2414035000, 0x7f2414037000, 0x7f2414038800, 0x7f241403a800, 2021-12-06T04:24:49.1903354Z 0x7f241403c000, 0x7f241403e000, 0x7f241403f800, 0x7f2414041800, 2021-12-06T04:24:49.1903812Z 0x7f2414043000, 0x7f2414045000, 0x7f24dc064800, 0x7f2414047000, 2021-12-06T04:24:49.1904284Z 0x7f2414048800, 0x7f241404a800, 0x7f241404c800, 0x7f241404e000, 2021-12-06T04:24:49.1904800Z 0x7f241405, 0x7f2414051800, 0x7f2414053800, 0x7f2414055000, 2021-12-06T04:24:49.1905455Z 0x7f2414057000, 0x7f2414059000, 0x7f241405a800, 0x7f241405c800, 2021-12-06T04:24:49.1906098Z 0x7f241405e000, 0x7f241406, 0x7f2414062000, 0x7f2414063800, 2021-12-06T04:24:49.1906728Z 0x7f22e400c800, 0x7f2328008000, 0x7f2284007000, 0x7f22cc019800, 2021-12-06T04:24:49.1907396Z 0x7f21f8004000, 0x7f2304012800, 0x7f230001b000, 0x7f223c011000, 2021-12-06T04:24:49.1908080Z 0x7f24e40c1800, 0x7f2454001000, 0x7f24e40c3000, 0x7f2454003000, 2021-12-06T04:24:49.1908794Z 0x7f24e40c5000, 0x7f2454004800, 0x7f2444002000, 0x7f2444002800, 2021-12-06T04:24:49.1909522Z 0x7f245808b800, 0x7f24b8032800, 0x7f24ac021000, 0x7f24b8034800, 2021-12-06T04:24:49.1910280Z 0x7f24b8036800, 0x7f24ac032800, 0x7f24b8052000, 0x7f24ac033800, 2021-12-06T04:24:49.1911023Z 0x7f24ac035000, 0x7f24b8067000, 0x7f24ac036000, 0x7f241407d000, 2021-12-06T
[jira] [Created] (FLINK-25181) KafkaSourceITCase.testValueOnlyDeserializer fails on AZP
Till Rohrmann created FLINK-25181: - Summary: KafkaSourceITCase.testValueOnlyDeserializer fails on AZP Key: FLINK-25181 URL: https://issues.apache.org/jira/browse/FLINK-25181 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.13.3 Reporter: Till Rohrmann Fix For: 1.13.4 The test case {{KafkaSourceITCase.testValueOnlyDeserializer}} fails on AZP with {code} Dec 06 00:53:02 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 62.314 s <<< FAILURE! - in org.apache.flink.connector.kafka.source.KafkaSourceITCase Dec 06 00:53:02 [ERROR] testValueOnlyDeserializer(org.apache.flink.connector.kafka.source.KafkaSourceITCase) Time elapsed: 3.018 s <<< FAILURE! Dec 06 00:53:02 java.lang.AssertionError: expected:<660> but was:<456> Dec 06 00:53:02 at org.junit.Assert.fail(Assert.java:88) Dec 06 00:53:02 at org.junit.Assert.failNotEquals(Assert.java:834) Dec 06 00:53:02 at org.junit.Assert.assertEquals(Assert.java:645) Dec 06 00:53:02 at org.junit.Assert.assertEquals(Assert.java:631) Dec 06 00:53:02 at org.apache.flink.connector.kafka.source.KafkaSourceITCase.testValueOnlyDeserializer(KafkaSourceITCase.java:173) Dec 06 00:53:02 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Dec 06 00:53:02 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Dec 06 00:53:02 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Dec 06 00:53:02 at java.lang.reflect.Method.invoke(Method.java:498) Dec 06 00:53:02 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) Dec 06 00:53:02 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Dec 06 00:53:02 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) Dec 06 00:53:02 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Dec 06 00:53:02 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) Dec 06 00:53:02 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) Dec 06 00:53:02 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) Dec 06 00:53:02 at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) Dec 06 00:53:02 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) Dec 06 00:53:02 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) Dec 06 00:53:02 at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) Dec 06 00:53:02 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) Dec 06 00:53:02 at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) Dec 06 00:53:02 at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) Dec 06 00:53:02 at org.junit.runners.ParentRunner.run(ParentRunner.java:363) Dec 06 00:53:02 at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) Dec 06 00:53:02 at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) Dec 06 00:53:02 at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) Dec 06 00:53:02 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) Dec 06 00:53:02 at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) Dec 06 00:53:02 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) Dec 06 00:53:02 at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) Dec 06 00:53:02 at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27568&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5&l=6558 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25180) Jepsen test fails while setting up libzip4
Till Rohrmann created FLINK-25180: - Summary: Jepsen test fails while setting up libzip4 Key: FLINK-25180 URL: https://issues.apache.org/jira/browse/FLINK-25180 Project: Flink Issue Type: Bug Components: Test Infrastructure Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The Jepsen tests fail from time to time while trying to set up libzip4. {code} java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: throw+: {:type :jepsen.control/nonzero-exit, :cmd "sudo -S -u root bash -c \"cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes libzip4\"", :exit -1, :out "Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: libzip4 0 upgraded, 1 newly installed, 0 to remove and 120 not upgraded. Need to get 40.6 kB of archives. After this operation, 103 kB of additional disk space will be used. Get:1 http://cdn-aws.deb.debian.org/debian stretch/main amd64 libzip4 amd64 1.1.2-1.1+b1 [40.6 kB] Fetched 40.6 kB in 0s (0 B/s) Selecting previously unselected package libzip4:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 49065 files and directories currently installed.) Preparing to unpack .../libzip4_1.1.2-1.1+b1_amd64.deb ... Unpacking libzip4:amd64 (1.1.2-1.1+b1) ... Setting up libzip4:amd64 (1.1.2-1.1+b1) ... ", :err "", :host "172.31.4.8", :action {:cmd "sudo -S -u root bash -c \"cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes libzip4\"", :in "root "}} {:type :jepsen.control/nonzero-exit, :cmd "sudo -S -u root bash -c \"cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes libzip4\"", :exit -1, :out "Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: libzip4 0 upgraded, 1 newly installed, 0 to remove and 120 not upgraded. Need to get 40.6 kB of archives. After this operation, 103 kB of additional disk space will be used. Get:1 http://cdn-aws.deb.debian.org/debian stretch/main amd64 libzip4 amd64 1.1.2-1.1+b1 [40.6 kB] Fetched 40.6 kB in 0s (0 B/s) Selecting previously unselected package libzip4:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 49065 files and directories currently installed.) Preparing to unpack .../libzip4_1.1.2-1.1+b1_amd64.deb ... Unpacking libzip4:amd64 (1.1.2-1.1+b1) ... Setting up libzip4:amd64 (1.1.2-1.1+b1) ... ", :err "", :host "172.31.4.8", :action {:cmd "sudo -S -u root bash -c \"cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes libzip4\"", :in "root"}} {code} https://app.travis-ci.com/github/dataArtisans/flink-jepsen-ci/jobs/550915650#L1300 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25069) YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP
Till Rohrmann created FLINK-25069: - Summary: YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP Key: FLINK-25069 URL: https://issues.apache.org/jira/browse/FLINK-25069 Project: Flink Issue Type: Bug Components: Deployment / YARN Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager}} fails on AZP with: {code} 2021-11-25T18:28:27.9848753Z Nov 25 18:28:27 [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 3,676.541 s <<< FAILURE! - in org.apache.flink.yarn.YARNHighAvailabilityITCase 2021-11-25T18:28:27.9849967Z Nov 25 18:28:27 [ERROR] org.apache.flink.yarn.YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager Time elapsed: 70.846 s <<< ERROR! 2021-11-25T18:28:27.9850929Z Nov 25 18:28:27 java.util.concurrent.ExecutionException: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph. 2021-11-25T18:28:27.9854591Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) 2021-11-25T18:28:27.9855441Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) 2021-11-25T18:28:27.9856301Z Nov 25 18:28:27at org.apache.flink.yarn.YARNHighAvailabilityITCase.submitJob(YARNHighAvailabilityITCase.java:378) 2021-11-25T18:28:27.9857202Z Nov 25 18:28:27at org.apache.flink.yarn.YARNHighAvailabilityITCase.lambda$testJobRecoversAfterKillingTaskManager$1(YARNHighAvailabilityITCase.java:204) 2021-11-25T18:28:27.9858300Z Nov 25 18:28:27at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:288) 2021-11-25T18:28:27.9859245Z Nov 25 18:28:27at org.apache.flink.yarn.YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager(YARNHighAvailabilityITCase.java:197) 2021-11-25T18:28:27.9860026Z Nov 25 18:28:27at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-11-25T18:28:27.9860705Z Nov 25 18:28:27at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-11-25T18:28:27.9861466Z Nov 25 18:28:27at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-25T18:28:27.9862158Z Nov 25 18:28:27at java.lang.reflect.Method.invoke(Method.java:498) 2021-11-25T18:28:27.9863016Z Nov 25 18:28:27at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-11-25T18:28:27.9863959Z Nov 25 18:28:27at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-11-25T18:28:27.9864829Z Nov 25 18:28:27at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-11-25T18:28:27.9865604Z Nov 25 18:28:27at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-11-25T18:28:27.9866300Z Nov 25 18:28:27at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) 2021-11-25T18:28:27.9867044Z Nov 25 18:28:27at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) 2021-11-25T18:28:27.9867692Z Nov 25 18:28:27at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2021-11-25T18:28:27.9868220Z Nov 25 18:28:27at java.lang.Thread.run(Thread.java:748) 2021-11-25T18:28:27.9869072Z Nov 25 18:28:27Suppressed: java.lang.AssertionError: There is at least one application on the cluster that is not finished.[App application_1637861234319_0001 is in state RUNNING.] 2021-11-25T18:28:27.9870263Z Nov 25 18:28:27at org.junit.Assert.fail(Assert.java:89) 2021-11-25T18:28:27.9870862Z Nov 25 18:28:27at org.apache.flink.yarn.YarnTestBase$CleanupYarnApplication.close(YarnTestBase.java:325) 2021-11-25T18:28:27.9871516Z Nov 25 18:28:27at org.apache.flink.yarn.YarnTestBase.runTest(YarnTestBase.java:289) 2021-11-25T18:28:27.9871986Z Nov 25 18:28:27... 13 more 2021-11-25T18:28:27.9872665Z Nov 25 18:28:27 Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph. 2021-11-25T18:28:27.9873393Z Nov 25 18:28:27at org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$11(RestClusterClient.java:433) 2021-11-25T18:28:27.9874102Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:884) 2021-11-25T18:28:27.9874774Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:866) 2021-11-25T18:28:27.9875454Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) 2021-11-25T18:28:27.9876123Z Nov 25 18:28:27at java.util.concurrent.CompletableFuture.completeEx
[jira] [Created] (FLINK-25053) Document how to use the usrlib to load code in the user code class loader
Till Rohrmann created FLINK-25053: - Summary: Document how to use the usrlib to load code in the user code class loader Key: FLINK-25053 URL: https://issues.apache.org/jira/browse/FLINK-25053 Project: Flink Issue Type: Bug Components: Documentation, Runtime / Coordination Affects Versions: 1.13.3, 1.12.5, 1.14.0, 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0, 1.14.1, 1.13.4 With FLINK-13993 we introduced the {{usrlib}} directory that can be used to load code in the user code class loader. This functionality has not been properly documented so that it is very hard to use. I would suggest to change this so that our users can benefit from this cool feature. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25040) FlinkKafkaInternalProducerITCase.testInitTransactionId failed on AZP
Till Rohrmann created FLINK-25040: - Summary: FlinkKafkaInternalProducerITCase.testInitTransactionId failed on AZP Key: FLINK-25040 URL: https://issues.apache.org/jira/browse/FLINK-25040 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{FlinkKafkaInternalProducerITCase.testInitTransactionId}} failed on AZP with: {code} Nov 24 09:25:41 [ERROR] org.apache.flink.connector.kafka.sink.FlinkKafkaInternalProducerITCase.testInitTransactionId Time elapsed: 82.766 s <<< ERROR! Nov 24 09:25:41 org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6 milliseconds while awaiting InitProducerId Nov 24 09:25:41 {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26987&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=15a22db7-8faa-5b34-3920-d33c9f0ca23c&l=6726 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25039) AZP fails with license check
Till Rohrmann created FLINK-25039: - Summary: AZP fails with license check Key: FLINK-25039 URL: https://issues.apache.org/jira/browse/FLINK-25039 Project: Flink Issue Type: Bug Components: Build System / Azure Pipelines Affects Versions: 1.15.0 Reporter: Till Rohrmann The AZP build fails with a license check: {code} 21:26:40,233 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - Missing META-INF/LICENSE in /tmp/flink-validation-deployment/org/apache/flink/flink-sql-parquet_2.12/1.15-SNAPSHOT/flink-sql-parquet_2.12-1.15-20211123.212027-1-tests.jar 21:26:40,738 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - The notice file in /tmp/flink-validation-deployment/org/apache/flink/flink-connector-cassandra_2.12/1.15-SNAPSHOT/flink-connector-cassandra_2.12-1.15-20211123.211736-1-tests.jar does not contain the expected entries. 21:26:40,739 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - Missing META-INF/LICENSE in /tmp/flink-validation-deployment/org/apache/flink/flink-connector-cassandra_2.12/1.15-SNAPSHOT/flink-connector-cassandra_2.12-1.15-20211123.211736-1-tests.jar 21:26:41,673 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - The notice file in /tmp/flink-validation-deployment/org/apache/flink/flink-kubernetes/1.15-SNAPSHOT/flink-kubernetes-1.15-20211123.212114-1-tests.jar does not contain the expected entries. 21:26:41,675 ERROR org.apache.flink.tools.ci.licensecheck.JarFileChecker [] - Missing META-INF/LICENSE in /tmp/flink-validation-deployment/org/apache/flink/flink-kubernetes/1.15-SNAPSHOT/flink-kubernetes-1.15-20211123.212114-1-tests.jar 21:28:00,582 WARN org.apache.flink.tools.ci.licensecheck.LicenseChecker [] - Found a total of 5 severe license issues == License Check failed. See previous output for details. == ##[error]Bash exited with code '1'. {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26967&view=logs&j=946871de-358d-5815-3994-8175615bc253&t=e0240c62-4570-5d1c-51af-dd63d2093da1&l=30668 https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26967&view=logs&j=e9d3d34f-3d15-59f4-0e3e-35067d100dfe&t=a7382ec4-87d2-5a9d-7c53-a2f93e317458&l=31863 https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26967&view=logs&j=6e8542d7-de38-5a33-4aca-458d6c87066d&t=dffc2faa-5b48-5b4e-0797-dec1b1f74872&l=31863 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25037) Compilation of compile_cron_python_wheels failed on AZP
Till Rohrmann created FLINK-25037: - Summary: Compilation of compile_cron_python_wheels failed on AZP Key: FLINK-25037 URL: https://issues.apache.org/jira/browse/FLINK-25037 Project: Flink Issue Type: Bug Components: API / Python, Build System / Azure Pipelines Affects Versions: 1.12.5 Reporter: Till Rohrmann Fix For: 1.12.6 The compilation of {{compile_cron_python_wheels}} failed on AZP with {code} == Compiling Flink == Invoking mvn with 'mvn -Dmaven.wagon.http.pool=false --settings /__w/1/s/tools/ci/google-mirror-settings.xml -Dorg.slf4j.simpleLogger.showDateTime=true -Dorg.slf4j.simpleLogger.dateTimeFormat=HH:mm:ss.SSS -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn --no-snapshot-updates -B -Dhadoop.version=2.8.3 -Dinclude_hadoop_aws -Dscala-2.11 clean deploy -DaltDeploymentRepository=validation_repository::default::file:/tmp/flink-validation-deployment -Dmaven.repo.local=/home/vsts/work/1/.m2/repository -Dflink.convergence.phase=install -Pcheck-convergence -Dflink.forkCount=2 -Dflink.forkCountTestPackage=2 -Dmaven.javadoc.skip=true -U -DskipTests' [ERROR] Could not create local repository at /home/vsts/work/1/.m2/repository -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LocalRepositoryNotAccessibleException == Compiling Flink failed. == {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26968&view=logs&j=a29bcfe1-064d-50b9-354f-07802213a3c0&t=47ff6576-c9dc-5eab-9db8-183dcca3bede -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25026) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP
Till Rohrmann created FLINK-25026: - Summary: UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP Key: FLINK-25026 URL: https://issues.apache.org/jira/browse/FLINK-25026 Project: Flink Issue Type: Bug Components: Runtime / Checkpointing Reporter: Till Rohrmann {{UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint}} fails on AZP with {code} 2021-11-23T00:58:03.8286352Z Nov 23 00:58:03 [ERROR] Tests run: 72, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 716.362 s <<< FAILURE! - in org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase 2021-11-23T00:58:03.8288790Z Nov 23 00:58:03 [ERROR] shouldRescaleUnalignedCheckpoint[downscale union from 3 to 2, buffersPerChannel = 1] Time elapsed: 4.051 s <<< ERROR! 2021-11-23T00:58:03.8289953Z Nov 23 00:58:03 org.apache.flink.runtime.client.JobExecutionException: Job execution failed. 2021-11-23T00:58:03.8291473Z Nov 23 00:58:03at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) 2021-11-23T00:58:03.8292776Z Nov 23 00:58:03at org.apache.flink.test.checkpointing.UnalignedCheckpointTestBase.execute(UnalignedCheckpointTestBase.java:168) 2021-11-23T00:58:03.8294520Z Nov 23 00:58:03at org.apache.flink.test.checkpointing.UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint(UnalignedCheckpointRescaleITCase.java:534) 2021-11-23T00:58:03.8295909Z Nov 23 00:58:03at jdk.internal.reflect.GeneratedMethodAccessor123.invoke(Unknown Source) 2021-11-23T00:58:03.8297310Z Nov 23 00:58:03at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-23T00:58:03.8298922Z Nov 23 00:58:03at java.base/java.lang.reflect.Method.invoke(Method.java:566) 2021-11-23T00:58:03.8300298Z Nov 23 00:58:03at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-11-23T00:58:03.8301741Z Nov 23 00:58:03at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-11-23T00:58:03.8303233Z Nov 23 00:58:03at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-11-23T00:58:03.8304514Z Nov 23 00:58:03at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-11-23T00:58:03.8305736Z Nov 23 00:58:03at org.junit.rules.Verifier$1.evaluate(Verifier.java:35) 2021-11-23T00:58:03.8306856Z Nov 23 00:58:03at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-11-23T00:58:03.8308218Z Nov 23 00:58:03at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-11-23T00:58:03.8309532Z Nov 23 00:58:03at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-11-23T00:58:03.8310780Z Nov 23 00:58:03at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-11-23T00:58:03.8312026Z Nov 23 00:58:03at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-11-23T00:58:03.8313515Z Nov 23 00:58:03at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-11-23T00:58:03.8314842Z Nov 23 00:58:03at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-11-23T00:58:03.8316116Z Nov 23 00:58:03at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-11-23T00:58:03.8317538Z Nov 23 00:58:03at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-11-23T00:58:03.8320044Z Nov 23 00:58:03at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-11-23T00:58:03.8321044Z Nov 23 00:58:03at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-11-23T00:58:03.8321978Z Nov 23 00:58:03at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-11-23T00:58:03.8322915Z Nov 23 00:58:03at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-11-23T00:58:03.8323848Z Nov 23 00:58:03at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-11-23T00:58:03.8325330Z Nov 23 00:58:03at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 2021-11-23T00:58:03.8337747Z Nov 23 00:58:03at org.junit.runners.Suite.runChild(Suite.java:128) 2021-11-23T00:58:03.8339178Z Nov 23 00:58:03at org.junit.runners.Suite.runChild(Suite.java:27) 2021-11-23T00:58:03.8340038Z Nov 23 00:58:03at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-11-23T00:58:03.8340967Z Nov 23 00:58:03at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-11-23T00:58:03.8341889Z Nov 23 00:58:03at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-11-23T00:58:03.8342923Z Nov 23 00:58:03at org.junit.runners.ParentRunner.a
[jira] [Created] (FLINK-25025) BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount failed on AZP
Till Rohrmann created FLINK-25025: - Summary: BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount failed on AZP Key: FLINK-25025 URL: https://issues.apache.org/jira/browse/FLINK-25025 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.14.0 Reporter: Till Rohrmann Fix For: 1.14.1 The test {{BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount}} fails on AZP with: {code} Nov 23 04:20:13 [ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.167 s <<< FAILURE! - in org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonGroupWindowAggregateFunctionOperatorTest Nov 23 04:20:13 [ERROR] testFinishBundleTriggeredByCount Time elapsed: 0.068 s <<< FAILURE! Nov 23 04:20:13 java.lang.AssertionError: expected:<6> but was:<4> Nov 23 04:20:13 at org.junit.Assert.fail(Assert.java:89) Nov 23 04:20:13 at org.junit.Assert.failNotEquals(Assert.java:835) Nov 23 04:20:13 at org.junit.Assert.assertEquals(Assert.java:647) Nov 23 04:20:13 at org.junit.Assert.assertEquals(Assert.java:633) Nov 23 04:20:13 at org.apache.flink.table.runtime.util.RowDataHarnessAssertor.assertOutputEquals(RowDataHarnessAssertor.java:80) Nov 23 04:20:13 at org.apache.flink.table.runtime.util.RowDataHarnessAssertor.assertOutputEquals(RowDataHarnessAssertor.java:60) Nov 23 04:20:13 at org.apache.flink.table.runtime.operators.python.aggregate.arrow.ArrowPythonAggregateFunctionOperatorTestBase.assertOutputEquals(ArrowPythonAggregateFunctionOperatorTestBase.java:62) Nov 23 04:20:13 at org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount(BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.java:206) Nov 23 04:20:13 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Nov 23 04:20:13 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Nov 23 04:20:13 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Nov 23 04:20:13 at java.lang.reflect.Method.invoke(Method.java:498) Nov 23 04:20:13 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Nov 23 04:20:13 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Nov 23 04:20:13 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Nov 23 04:20:13 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Nov 23 04:20:13 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Nov 23 04:20:13 at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) Nov 23 04:20:13 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) Nov 23 04:20:13 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) Nov 23 04:20:13 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) Nov 23 04:20:13 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) Nov 23 04:20:13 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) Nov 23 04:20:13 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) Nov 23 04:20:13 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) Nov 23 04:20:13 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) Nov 23 04:20:13 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Nov 23 04:20:13 at org.junit.runners.ParentRunner.run(ParentRunner.java:413) Nov 23 04:20:13 at org.junit.runner.JUnitCore.run(JUnitCore.java:137) Nov 23 04:20:13 at org.junit.runner.JUnitCore.run(JUnitCore.java:115) Nov 23 04:20:13 at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43) Nov 23 04:20:13 at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) Nov 23 04:20:13 at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) Nov 23 04:20:13 at java.util.Iterator.forEachRemaining(Iterator.java:116) Nov 23 04:20:13 at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) Nov 23 04:20:13 at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) Nov 23 04:20:13 at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) Nov 23 04:20:13 at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForE
[jira] [Created] (FLINK-24872) PulsarSourceITCase.testMultipleSplits failed on AZP
Till Rohrmann created FLINK-24872: - Summary: PulsarSourceITCase.testMultipleSplits failed on AZP Key: FLINK-24872 URL: https://issues.apache.org/jira/browse/FLINK-24872 Project: Flink Issue Type: Bug Components: Connectors / Pulsar Affects Versions: 1.14.0, 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0, 1.14.1 The test {{PulsarSourceITCase.testMultipleSplits}} and {{PulsarSourceITCase. testTaskManagerFailure}} failed on AZP with {code} 021-11-07T00:02:30.5034103Z Nov 07 00:02:30 [ERROR] testMultipleSplits{TestEnvironment, ExternalContext}[1] Time elapsed: 20.006 s <<< ERROR! 2021-11-07T00:02:30.5035364Z Nov 07 00:02:30 java.lang.RuntimeException: Failed to fetch next result 2021-11-07T00:02:30.5036469Z Nov 07 00:02:30at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109) 2021-11-07T00:02:30.5038261Z Nov 07 00:02:30at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80) 2021-11-07T00:02:30.5039047Z Nov 07 00:02:30at org.apache.flink.connectors.test.common.utils.TestDataMatchers$MultipleSplitDataMatcher.matchesSafely(TestDataMatchers.java:158) 2021-11-07T00:02:30.5039771Z Nov 07 00:02:30at org.apache.flink.connectors.test.common.utils.TestDataMatchers$MultipleSplitDataMatcher.matchesSafely(TestDataMatchers.java:138) 2021-11-07T00:02:30.5044060Z Nov 07 00:02:30at org.hamcrest.TypeSafeDiagnosingMatcher.matches(TypeSafeDiagnosingMatcher.java:55) 2021-11-07T00:02:30.5044780Z Nov 07 00:02:30at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:12) 2021-11-07T00:02:30.5045427Z Nov 07 00:02:30at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8) 2021-11-07T00:02:30.5046363Z Nov 07 00:02:30at org.apache.flink.connectors.test.common.testsuites.SourceTestSuiteBase.testMultipleSplits(SourceTestSuiteBase.java:160) 2021-11-07T00:02:30.5047398Z Nov 07 00:02:30at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-11-07T00:02:30.5048018Z Nov 07 00:02:30at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-11-07T00:02:30.5048567Z Nov 07 00:02:30at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-07T00:02:30.5049051Z Nov 07 00:02:30at java.lang.reflect.Method.invoke(Method.java:498) 2021-11-07T00:02:30.5049530Z Nov 07 00:02:30at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688) 2021-11-07T00:02:30.5050081Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) 2021-11-07T00:02:30.5050689Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) 2021-11-07T00:02:30.5051331Z Nov 07 00:02:30at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) 2021-11-07T00:02:30.5062122Z Nov 07 00:02:30at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140) 2021-11-07T00:02:30.5063450Z Nov 07 00:02:30at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:92) 2021-11-07T00:02:30.5064534Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) 2021-11-07T00:02:30.5065318Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) 2021-11-07T00:02:30.5065977Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) 2021-11-07T00:02:30.5066633Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) 2021-11-07T00:02:30.5067678Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) 2021-11-07T00:02:30.5068618Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) 2021-11-07T00:02:30.5069469Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) 2021-11-07T00:02:30.5070358Z Nov 07 00:02:30at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) 2021-11-07T00:02:30.5071553Z Nov 07 00:02:30at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210) 2021-11-07T00:02:30.5072667Z Nov 07 00:02:30at org.junit.platform.engine.support.hiera
[jira] [Created] (FLINK-24867) E2e tests take longer than the maximum 310 minutes on AZP
Till Rohrmann created FLINK-24867: - Summary: E2e tests take longer than the maximum 310 minutes on AZP Key: FLINK-24867 URL: https://issues.apache.org/jira/browse/FLINK-24867 Project: Flink Issue Type: Bug Components: Build System / Azure Pipelines, Tests Affects Versions: 1.13.3 Reporter: Till Rohrmann The e2e tests took longer than the maximum 310 minutes in one AZP run. This made the build step fail. {code} ##[error]The job running on agent Azure Pipelines 9 ran longer than the maximum time of 310 minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134 Agent: Azure Pipelines 9 Started: Today at 09:25 Duration: 5h 10m 11s {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26257&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-24866) AZP crashed in Post-job: Cache Maven local repo
Till Rohrmann created FLINK-24866: - Summary: AZP crashed in Post-job: Cache Maven local repo Key: FLINK-24866 URL: https://issues.apache.org/jira/browse/FLINK-24866 Project: Flink Issue Type: Bug Components: Build System / Azure Pipelines Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 An AZP build failed while running the post-job: Cache Maven local repo step with exit code 2: {code} Resolved to: maven|Linux|kI+vc4kUoz33JEfRluJAo4vEVFz7aQdIKJJbq3fbuGw= ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 2919f31f-021b-468b-851e-f92f99f5681f Getting a pipeline cache artifact with one of the following fingerprints: Fingerprint: `maven|Linux|kI+vc4kUoz33JEfRluJAo4vEVFz7aQdIKJJbq3fbuGw=` There is a cache miss. tar: f202add2a23c497f93e0ceff83df8823_archive.tar: Wrote only 6144 of 10240 bytes tar: Error is not recoverable: exiting now ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 2919f31f-021b-468b-851e-f92f99f5681f ##[error]Process returned non-zero exit code: 2 {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26271&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=85df99a3-dd32-4a6c-8fa0-7c375f4cbc3a&l=212 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-24844) CassandraConnectorITCase.testCassandraBatchPojoFormat fails on AZP
Till Rohrmann created FLINK-24844: - Summary: CassandraConnectorITCase.testCassandraBatchPojoFormat fails on AZP Key: FLINK-24844 URL: https://issues.apache.org/jira/browse/FLINK-24844 Project: Flink Issue Type: Bug Components: Connectors / Cassandra Affects Versions: 1.14.0 Reporter: Till Rohrmann Fix For: 1.14.1 The test {{CassandraConnectorITCase.testCassandraBatchPojoFormat}} fails on AZP with {code} 2021-11-09T00:32:42.1369473Z Nov 09 00:32:42 [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 152.962 s <<< FAILURE! - in org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase 2021-11-09T00:32:42.1371638Z Nov 09 00:32:42 [ERROR] testCassandraBatchPojoFormat Time elapsed: 20.378 s <<< ERROR! 2021-11-09T00:32:42.1372881Z Nov 09 00:32:42 com.datastax.driver.core.exceptions.AlreadyExistsException: Table flink.batches already exists 2021-11-09T00:32:42.1373913Z Nov 09 00:32:42at com.datastax.driver.core.exceptions.AlreadyExistsException.copy(AlreadyExistsException.java:111) 2021-11-09T00:32:42.1374921Z Nov 09 00:32:42at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37) 2021-11-09T00:32:42.1379615Z Nov 09 00:32:42at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245) 2021-11-09T00:32:42.1380668Z Nov 09 00:32:42at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:63) 2021-11-09T00:32:42.1381523Z Nov 09 00:32:42at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:39) 2021-11-09T00:32:42.1382552Z Nov 09 00:32:42at org.apache.flink.streaming.connectors.cassandra.CassandraConnectorITCase.testCassandraBatchPojoFormat(CassandraConnectorITCase.java:543) 2021-11-09T00:32:42.1383487Z Nov 09 00:32:42at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-11-09T00:32:42.1384433Z Nov 09 00:32:42at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-11-09T00:32:42.1385336Z Nov 09 00:32:42at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-09T00:32:42.1386119Z Nov 09 00:32:42at java.lang.reflect.Method.invoke(Method.java:498) 2021-11-09T00:32:42.1387204Z Nov 09 00:32:42at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-11-09T00:32:42.1388225Z Nov 09 00:32:42at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-11-09T00:32:42.1389101Z Nov 09 00:32:42at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-11-09T00:32:42.1400913Z Nov 09 00:32:42at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-11-09T00:32:42.1401588Z Nov 09 00:32:42at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-11-09T00:32:42.1402487Z Nov 09 00:32:42at org.apache.flink.testutils.junit.RetryRule$RetryOnExceptionStatement.evaluate(RetryRule.java:192) 2021-11-09T00:32:42.1403055Z Nov 09 00:32:42at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-11-09T00:32:42.1403556Z Nov 09 00:32:42at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-11-09T00:32:42.1404008Z Nov 09 00:32:42at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-11-09T00:32:42.1404650Z Nov 09 00:32:42at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-11-09T00:32:42.1405151Z Nov 09 00:32:42at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-11-09T00:32:42.1405632Z Nov 09 00:32:42at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-11-09T00:32:42.1406166Z Nov 09 00:32:42at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-11-09T00:32:42.1406670Z Nov 09 00:32:42at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-11-09T00:32:42.1407125Z Nov 09 00:32:42at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-11-09T00:32:42.1407599Z Nov 09 00:32:42at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-11-09T00:32:42.1408258Z Nov 09 00:32:42at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-11-09T00:32:42.1408715Z Nov 09 00:32:42at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-11-09T00:32:42.1409216Z Nov 09 00:32:42at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) 2021-11-09T00:32:42.1409717Z Nov 09 00:32:42at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) 2021-11-09T00:32:42.1410251Z Nov 09 00:32:42at org.junit.ru
[jira] [Created] (FLINK-24841) ChainOrderTest.testMigrationAndRestore fails on AZP
Till Rohrmann created FLINK-24841: - Summary: ChainOrderTest.testMigrationAndRestore fails on AZP Key: FLINK-24841 URL: https://issues.apache.org/jira/browse/FLINK-24841 Project: Flink Issue Type: Bug Components: API / DataStream, Runtime / State Backends Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{ChainOrderTest.testMigrationAndRestore}} fails on AZP with {code} 2021-11-09T00:07:04.9969557Z Nov 09 00:07:04 [ERROR] Tests run: 12, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 97.296 s <<< FAILURE! - in org.apache.flink.test.state.operator.restore.unkeyed.ChainOrderTest 2021-11-09T00:07:04.9970349Z Nov 09 00:07:04 [ERROR] testMigrationAndRestore[Migrate Savepoint: 1.5] Time elapsed: 73.544 s <<< ERROR! 2021-11-09T00:07:04.9984239Z Nov 09 00:07:04 java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Job 2f4599006c2734303b1d6033f69844e9 is not a streaming job. 2021-11-09T00:07:04.9985462Z Nov 09 00:07:04at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) 2021-11-09T00:07:04.9986224Z Nov 09 00:07:04at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) 2021-11-09T00:07:04.9986910Z Nov 09 00:07:04at org.apache.flink.test.state.operator.restore.AbstractOperatorRestoreTestBase.migrateJob(AbstractOperatorRestoreTestBase.java:160) 2021-11-09T00:07:04.9987650Z Nov 09 00:07:04at org.apache.flink.test.state.operator.restore.AbstractOperatorRestoreTestBase.testMigrationAndRestore(AbstractOperatorRestoreTestBase.java:114) 2021-11-09T00:07:04.9988579Z Nov 09 00:07:04at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-11-09T00:07:04.9992663Z Nov 09 00:07:04at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-11-09T00:07:04.9993717Z Nov 09 00:07:04at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-09T00:07:04.9994648Z Nov 09 00:07:04at java.base/java.lang.reflect.Method.invoke(Method.java:566) 2021-11-09T00:07:04.9995423Z Nov 09 00:07:04at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) 2021-11-09T00:07:04.9996143Z Nov 09 00:07:04at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) 2021-11-09T00:07:04.9996674Z Nov 09 00:07:04at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) 2021-11-09T00:07:04.9997227Z Nov 09 00:07:04at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) 2021-11-09T00:07:04.9998428Z Nov 09 00:07:04at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-11-09T00:07:04.9998938Z Nov 09 00:07:04at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) 2021-11-09T00:07:04.462Z Nov 09 00:07:04at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) 2021-11-09T00:07:04.946Z Nov 09 00:07:04at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) 2021-11-09T00:07:05.391Z Nov 09 00:07:04at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) 2021-11-09T00:07:05.900Z Nov 09 00:07:04at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) 2021-11-09T00:07:05.0001404Z Nov 09 00:07:04at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) 2021-11-09T00:07:05.0001897Z Nov 09 00:07:04at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) 2021-11-09T00:07:05.0002437Z Nov 09 00:07:04at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) 2021-11-09T00:07:05.0002914Z Nov 09 00:07:04at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-11-09T00:07:05.0003377Z Nov 09 00:07:04at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) 2021-11-09T00:07:05.0003846Z Nov 09 00:07:04at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) 2021-11-09T00:07:05.0004393Z Nov 09 00:07:04at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) 2021-11-09T00:07:05.0004855Z Nov 09 00:07:04at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) 2021-11-09T00:07:05.0005310Z Nov 09 00:07:04at org.junit.runners.ParentRunner.run(ParentRunner.java:413) 2021-11-09T00:07:05.0005724Z Nov 09 00:07:04at org.junit.runners.Suite.runChild(Suite.java:128) 2021-11-09T00:07:05.0006372Z Nov 09 00:07:04at org.junit.runners.Suite.runChild(Suite.java:27) 2021-11-09T00:07:05.0006796Z Nov 09 00:07:04at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) 2021-11-09T00:07:05.0007236Z Nov 09 00:07:04at o
[jira] [Created] (FLINK-24840) JobManagerHAProcessFailureRecoveryITCase crashed on AZP
Till Rohrmann created FLINK-24840: - Summary: JobManagerHAProcessFailureRecoveryITCase crashed on AZP Key: FLINK-24840 URL: https://issues.apache.org/jira/browse/FLINK-24840 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{JobManagerHAProcessFailureRecoveryITCase}} crashes on AZP. {code} 2021-11-08T23:27:32.9623564Z Nov 08 23:27:32 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (integration-tests) on project flink-tests: There are test failures. 2021-11-08T23:27:32.9624265Z Nov 08 23:27:32 [ERROR] 2021-11-08T23:27:32.9624981Z Nov 08 23:27:32 [ERROR] Please refer to /__w/2/s/flink-tests/target/surefire-reports for the individual test results. 2021-11-08T23:27:32.9625718Z Nov 08 23:27:32 [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. 2021-11-08T23:27:32.9626539Z Nov 08 23:27:32 [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-11-08T23:27:32.9627945Z Nov 08 23:27:32 [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-tests/target && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=2 -XX:+UseG1GC -jar /__w/2/s/flink-tests/target/surefire/surefirebooter6973421711355178650.jar /__w/2/s/flink-tests/target/surefire 2021-11-08T22-42-26_652-jvmRun2 surefire3220623274989834473tmp surefire_1316080825708637501141tmp 2021-11-08T23:27:32.9628850Z Nov 08 23:27:32 [ERROR] Error occurred in starting fork, check output in log 2021-11-08T23:27:32.9629327Z Nov 08 23:27:32 [ERROR] Process Exit Code: 239 2021-11-08T23:27:32.9629645Z Nov 08 23:27:32 [ERROR] Crashed tests: 2021-11-08T23:27:32.9630025Z Nov 08 23:27:32 [ERROR] org.apache.flink.test.recovery.JobManagerHAProcessFailureRecoveryITCase 2021-11-08T23:27:32.9630643Z Nov 08 23:27:32 [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-11-08T23:27:32.9632522Z Nov 08 23:27:32 [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-tests/target && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=2 -XX:+UseG1GC -jar /__w/2/s/flink-tests/target/surefire/surefirebooter6973421711355178650.jar /__w/2/s/flink-tests/target/surefire 2021-11-08T22-42-26_652-jvmRun2 surefire3220623274989834473tmp surefire_1316080825708637501141tmp 2021-11-08T23:27:32.9633418Z Nov 08 23:27:32 [ERROR] Error occurred in starting fork, check output in log 2021-11-08T23:27:32.9635503Z Nov 08 23:27:32 [ERROR] Process Exit Code: 239 2021-11-08T23:27:32.9636287Z Nov 08 23:27:32 [ERROR] Crashed tests: 2021-11-08T23:27:32.9636863Z Nov 08 23:27:32 [ERROR] org.apache.flink.test.recovery.JobManagerHAProcessFailureRecoveryITCase 2021-11-08T23:27:32.9637609Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510) 2021-11-08T23:27:32.9638331Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:457) 2021-11-08T23:27:32.9638951Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:298) 2021-11-08T23:27:32.9639505Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246) 2021-11-08T23:27:32.9640108Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183) 2021-11-08T23:27:32.9640916Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011) 2021-11-08T23:27:32.9641819Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857) 2021-11-08T23:27:32.9642447Z Nov 08 23:27:32 [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132) 2021-11-08T23:27:32.9643039Z Nov 08 23:27:32 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) 2021-11-08T23:27:32.9643582Z Nov 08 23:27:32 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) 2021-11-08T23:27:32.9644142Z Nov 08 23:27:32 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) 2021-11-08T23:27:32.9644753Z Nov 08 23:27:32 [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) 2021-11-08T23:27:32.9645396Z Nov 08 23:27:32 [ERROR] at org.apache.maven.lifecycle.internal.Li
[jira] [Created] (FLINK-24839) CsvFilesystemStreamSinkITCase.testPart times out on AZP
Till Rohrmann created FLINK-24839: - Summary: CsvFilesystemStreamSinkITCase.testPart times out on AZP Key: FLINK-24839 URL: https://issues.apache.org/jira/browse/FLINK-24839 Project: Flink Issue Type: Bug Reporter: Till Rohrmann The test {{CsvFilesystemStreamSinkITCase.testPart}} times out on AZP. {code} 2021-11-08T16:36:28.6542078Z Nov 08 16:36:28 org.junit.runners.model.TestTimedOutException: test timed out after 20 seconds 2021-11-08T16:36:28.6561998Z Nov 08 16:36:28at java.io.ObjectOutputStream.putFields(ObjectOutputStream.java:463) 2021-11-08T16:36:28.6581789Z Nov 08 16:36:28at java.util.Locale.writeObject(Locale.java:2156) 2021-11-08T16:36:28.6601916Z Nov 08 16:36:28at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 2021-11-08T16:36:28.6621871Z Nov 08 16:36:28at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 2021-11-08T16:36:28.663Z Nov 08 16:36:28at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 2021-11-08T16:36:28.6633082Z Nov 08 16:36:28at java.lang.reflect.Method.invoke(Method.java:498) 2021-11-08T16:36:28.6633845Z Nov 08 16:36:28at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1154) 2021-11-08T16:36:28.6634442Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) 2021-11-08T16:36:28.6634968Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6637691Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6640766Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6641958Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6642763Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6643563Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6644365Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6645138Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6647747Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6648657Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6649439Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6650189Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6650958Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6651975Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6652632Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6653314Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6664918Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6665679Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.409Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6667211Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6667907Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6668585Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6669301Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6669991Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) 2021-11-08T16:36:28.6670706Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) 2021-11-08T16:36:28.6671353Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) 2021-11-08T16:36:28.6672227Z Nov 08 16:36:28at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 2021-11-08T16:36:28.6672878Z Nov 08 16:36:28at java.io.ObjectOutputStream.writeSerialData(
[jira] [Created] (FLINK-24821) FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails on AZP
Till Rohrmann created FLINK-24821: - Summary: FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails on AZP Key: FLINK-24821 URL: https://issues.apache.org/jira/browse/FLINK-24821 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.14.0, 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0, 1.14.1 The test {{FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer}} fails on AZP with {code} Nov 07 23:02:14 [ERROR] testRestoreProducer[Migration Savepoint: 1.8] Time elapsed: 2.008 s <<< ERROR! Nov 07 23:02:14 java.net.BindException: Address already in use Nov 07 23:02:14 at sun.nio.ch.Net.bind0(Native Method) Nov 07 23:02:14 at sun.nio.ch.Net.bind(Net.java:461) Nov 07 23:02:14 at sun.nio.ch.Net.bind(Net.java:453) Nov 07 23:02:14 at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222) Nov 07 23:02:14 at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85) Nov 07 23:02:14 at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78) Nov 07 23:02:14 at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90) Nov 07 23:02:14 at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:120) Nov 07 23:02:14 at org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:93) Nov 07 23:02:14 at org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:148) Nov 07 23:02:14 at java.lang.Thread.run(Thread.java:748) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=26096&view=logs&j=72d4811f-9f0d-5fd0-014a-0bc26b72b642&t=e424005a-b16e-540f-196d-da062cc19bdf&l=7302 It looks that there is a race condition for avaiable ports. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-24795) FlinkKafkaProducerITCase.testRunOutOfProducersInThePool times out on AZP
Till Rohrmann created FLINK-24795: - Summary: FlinkKafkaProducerITCase.testRunOutOfProducersInThePool times out on AZP Key: FLINK-24795 URL: https://issues.apache.org/jira/browse/FLINK-24795 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The test {{FlinkKafkaProducerITCase.testRunOutOfProducersInThePool}} times out on AZP. {code} Nov 05 04:21:47 [ERROR] Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 283.667 s <<< FAILURE! - in org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase Nov 05 04:21:47 [ERROR] testRunOutOfProducersInThePool Time elapsed: 61.299 s <<< ERROR! Nov 05 04:21:47 org.apache.kafka.common.errors.TimeoutException: org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId Nov 05 04:21:47 Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=25993&view=logs&j=c5f0071e-1851-543e-9a45-9ac140befc32&t=15a22db7-8faa-5b34-3920-d33c9f0ca23c&l=6712 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-24794) pyflink/datastream/tests/test_data_stream.py fails on AZP
Till Rohrmann created FLINK-24794: - Summary: pyflink/datastream/tests/test_data_stream.py fails on AZP Key: FLINK-24794 URL: https://issues.apache.org/jira/browse/FLINK-24794 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.14.0 Reporter: Till Rohrmann Fix For: 1.14.1 The test {{pyflink/datastream/tests/test_data_stream.py}} fails on AZP with {code} ##[error]Exit code 137 returned from process: file name '/bin/docker', arguments 'exec -i -u 1004 c51e2746884724c100e2ea1ff39c2552ed90ad0edaf87b3c460f70c7780efd0c /__a/externals/node/bin/node /__w/_temp/containerHandlerInvoker.js'. {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=25995&view=logs&j=9cada3cb-c1d3-5621-16da-0f718fb86602&t=c67e71ed-6451-5d26-8920-5a8cf9651901&l=21843 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-24793) DefaultSchedulerLocalRecoveryITCase fails on AZP
Till Rohrmann created FLINK-24793: - Summary: DefaultSchedulerLocalRecoveryITCase fails on AZP Key: FLINK-24793 URL: https://issues.apache.org/jira/browse/FLINK-24793 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 {{DefaultSchedulerLocalRecoveryITCase.testLocalRecoveryFull}} and {{DefaultSchedulerLocalRecoveryITCase. testLocalRecoveryRegion}} fails on AZP with: {code} Nov 04 23:01:32 java.lang.IllegalArgumentException: attempt does not exist Nov 04 23:01:32 at org.apache.flink.runtime.executiongraph.ArchivedExecutionVertex.getPriorExecutionAttempt(ArchivedExecutionVertex.java:109) Nov 04 23:01:32 at org.apache.flink.test.runtime.DefaultSchedulerLocalRecoveryITCase.assertNonLocalRecoveredTasksEquals(DefaultSchedulerLocalRecoveryITCase.java:92) Nov 04 23:01:32 at org.apache.flink.test.runtime.DefaultSchedulerLocalRecoveryITCase.testLocalRecoveryInternal(DefaultSchedulerLocalRecoveryITCase.java:80) Nov 04 23:01:32 at org.apache.flink.test.runtime.DefaultSchedulerLocalRecoveryITCase.testLocalRecoveryFull(DefaultSchedulerLocalRecoveryITCase.java:65) Nov 04 23:01:32 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Nov 04 23:01:32 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Nov 04 23:01:32 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Nov 04 23:01:32 at java.lang.reflect.Method.invoke(Method.java:498) Nov 04 23:01:32 at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) Nov 04 23:01:32 at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) Nov 04 23:01:32 at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) Nov 04 23:01:32 at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) Nov 04 23:01:32 at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45) Nov 04 23:01:32 at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) Nov 04 23:01:32 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Nov 04 23:01:32 at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) Nov 04 23:01:32 at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) Nov 04 23:01:32 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) Nov 04 23:01:32 at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) Nov 04 23:01:32 at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) Nov 04 23:01:32 at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) Nov 04 23:01:32 at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) Nov 04 23:01:32 at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) Nov 04 23:01:32 at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) Nov 04 23:01:32 at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) Nov 04 23:01:32 at org.junit.runners.ParentRunner.run(ParentRunner.java:413) Nov 04 23:01:32 at org.junit.runner.JUnitCore.run(JUnitCore.java:137) Nov 04 23:01:32 at org.junit.runner.JUnitCore.run(JUnitCore.java:115) Nov 04 23:01:32 at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43) Nov 04 23:01:32 at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) Nov 04 23:01:32 at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) Nov 04 23:01:32 at java.util.Iterator.forEachRemaining(Iterator.java:116) Nov 04 23:01:32 at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) Nov 04 23:01:32 at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) Nov 04 23:01:32 at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) Nov 04 23:01:32 at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) Nov 04 23:01:32 at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) Nov 04 23:01:32 at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) Nov 04 23:01:32 at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) {code} https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=25983&view=logs&j=8fd9202e-fd17-5b26-353c-ac1ff76c8f28&t=ea7cf968-e585-52cb-e0fc-f48de023a7ca&l=4451 -- This message was sent
[jira] [Created] (FLINK-24792) OperatorCoordinatorSchedulerTest crashed JVM on AZP
Till Rohrmann created FLINK-24792: - Summary: OperatorCoordinatorSchedulerTest crashed JVM on AZP Key: FLINK-24792 URL: https://issues.apache.org/jira/browse/FLINK-24792 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.15.0 Reporter: Till Rohrmann Fix For: 1.15.0 The {{OperatorCoordinatorSchedulerTest}} crashed the JVM on AZP: {code} 2021-11-04T23:04:42.9147131Z Nov 04 23:04:42 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project flink-runtime: There are test failures. 2021-11-04T23:04:42.9148019Z Nov 04 23:04:42 [ERROR] 2021-11-04T23:04:42.9149110Z Nov 04 23:04:42 [ERROR] Please refer to /__w/2/s/flink-runtime/target/surefire-reports for the individual test results. 2021-11-04T23:04:42.9149931Z Nov 04 23:04:42 [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. 2021-11-04T23:04:42.9150465Z Nov 04 23:04:42 [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-11-04T23:04:42.9152168Z Nov 04 23:04:42 [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-runtime && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /__w/2/s/flink-runtime/target/surefire/surefirebooter6689593993396607766.jar /__w/2/s/flink-runtime/target/surefire 2021-11-04T22-41-25_783-jvmRun1 surefire1850634831631421222tmp surefire_4465457992371306280265tmp 2021-11-04T23:04:42.9153058Z Nov 04 23:04:42 [ERROR] Error occurred in starting fork, check output in log 2021-11-04T23:04:42.9153527Z Nov 04 23:04:42 [ERROR] Process Exit Code: 239 2021-11-04T23:04:42.9153989Z Nov 04 23:04:42 [ERROR] Crashed tests: 2021-11-04T23:04:42.9154899Z Nov 04 23:04:42 [ERROR] org.apache.flink.runtime.operators.coordination.OperatorCoordinatorSchedulerTest 2021-11-04T23:04:42.9155833Z Nov 04 23:04:42 [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? 2021-11-04T23:04:42.9164535Z Nov 04 23:04:42 [ERROR] Command was /bin/sh -c cd /__w/2/s/flink-runtime && /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /__w/2/s/flink-runtime/target/surefire/surefirebooter6689593993396607766.jar /__w/2/s/flink-runtime/target/surefire 2021-11-04T22-41-25_783-jvmRun1 surefire1850634831631421222tmp surefire_4465457992371306280265tmp 2021-11-04T23:04:42.9165558Z Nov 04 23:04:42 [ERROR] Error occurred in starting fork, check output in log 2021-11-04T23:04:42.9165928Z Nov 04 23:04:42 [ERROR] Process Exit Code: 239 2021-11-04T23:04:42.9166270Z Nov 04 23:04:42 [ERROR] Crashed tests: 2021-11-04T23:04:42.9166760Z Nov 04 23:04:42 [ERROR] org.apache.flink.runtime.operators.coordination.OperatorCoordinatorSchedulerTest 2021-11-04T23:04:42.9167319Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510) 2021-11-04T23:04:42.9167954Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:457) 2021-11-04T23:04:42.9168865Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:298) 2021-11-04T23:04:42.9169721Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246) 2021-11-04T23:04:42.9170555Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183) 2021-11-04T23:04:42.9171215Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011) 2021-11-04T23:04:42.9172268Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857) 2021-11-04T23:04:42.9172889Z Nov 04 23:04:42 [ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132) 2021-11-04T23:04:42.9173460Z Nov 04 23:04:42 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) 2021-11-04T23:04:42.9174087Z Nov 04 23:04:42 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) 2021-11-04T23:04:42.9174961Z Nov 04 23:04:42 [ERROR] at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) 2021-11-04T23:04:42.9175904Z Nov 04 23:04:42 [ERROR] at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) 2021-11-04T23:04:42.9177000Z Nov 04 23:04:42 [ERROR] at org.apache.maven.lifecycle.inter