[jira] [Commented] (FLINK-25937) SQL Client end-to-end test e2e fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487935#comment-17487935
]
Yun Gao commented on FLINK-25937:
-
UpsertKafkaTableITCase:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=c5612577-f1f7-5977-6ff6-7432788526f7=ffa8837a-b445-534e-cdf4-db364cf8235d=36182
> SQL Client end-to-end test e2e fails on AZP
> ---
>
> Key: FLINK-25937
> URL: https://issues.apache.org/jira/browse/FLINK-25937
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The {{SQL Client end-to-end test}} e2e tests fails on AZP when using the
> {{AdaptiveScheduler}} because the scheduler expects that the parallelism is
> set for all vertices:
> {code}
> Feb 03 03:45:13 org.apache.flink.runtime.client.JobInitializationException:
> Could not start the JobMaster.
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609)
> Feb 03 03:45:13 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Feb 03 03:45:13 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Feb 03 03:45:13 at java.lang.Thread.run(Thread.java:748)
> Feb 03 03:45:13 Caused by: java.util.concurrent.CompletionException:
> java.lang.IllegalStateException: The adaptive scheduler expects the
> parallelism being set for each JobVertex (violated JobVertex:
> f74b775b58627a33e46b8c155b320255).
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
> Feb 03 03:45:13 ... 3 more
> Feb 03 03:45:13 Caused by: java.lang.IllegalStateException: The adaptive
> scheduler expects the parallelism being set for each JobVertex (violated
> JobVertex: f74b775b58627a33e46b8c155b320255).
> Feb 03 03:45:13 at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.assertPreconditions(AdaptiveScheduler.java:296)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.(AdaptiveScheduler.java:230)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerFactory.createInstance(AdaptiveSchedulerFactory.java:122)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:115)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:322)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:106)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:94)
> Feb 03 03:45:13 at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> Feb 03 03:45:13 ... 3 more
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30662=logs=fb37c667-81b7-5c22-dd91-846535e99a97=39a035c3-c65e-573c-fb66-104c66c28912=5782
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * 78c2513ffe72a83ba689789fd5c14dbda4d93c99 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30824) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25426) UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on AZP because it cannot allocate enough network buffers
[
https://issues.apache.org/jira/browse/FLINK-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487930#comment-17487930
]
Yun Gao commented on FLINK-25426:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=b78d9d30-509a-5cea-1fef-db7abaa325ae=6602
> UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint fails on
> AZP because it cannot allocate enough network buffers
>
>
> Key: FLINK-25426
> URL: https://issues.apache.org/jira/browse/FLINK-25426
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Assignee: Anton Kalashnikov
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> The test
> {{UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint}} fails
> with
> {code}
> 2021-12-23T02:54:46.2862342Z Dec 23 02:54:46 [ERROR]
> UnalignedCheckpointRescaleITCase.shouldRescaleUnalignedCheckpoint Time
> elapsed: 2.992 s <<< ERROR!
> 2021-12-23T02:54:46.2865774Z Dec 23 02:54:46 java.lang.OutOfMemoryError:
> Could not allocate enough memory segments for NetworkBufferPool (required
> (Mb): 64, allocated (Mb): 14, missing (Mb): 50). Cause: Direct buffer memory.
> The direct out-of-memory error has occurred. This can mean two things: either
> job(s) require(s) a larger size of JVM direct memory or there is a direct
> memory leak. The direct memory can be allocated by user code or some of its
> dependencies. In this case 'taskmanager.memory.task.off-heap.size'
> configuration option should be increased. Flink framework and its
> dependencies also consume the direct memory, mostly for network
> communication. The most of network memory is managed by Flink and should not
> result in out-of-memory error. In certain special cases, in particular for
> jobs with high parallelism, the framework may require more direct memory
> which is not managed by Flink. In this case
> 'taskmanager.memory.framework.off-heap.size' configuration option should be
> increased. If the error persists then there is probably a direct memory leak
> in user code or some of its dependencies which has to be investigated and
> fixed. The task executor has to be shutdown...
> 2021-12-23T02:54:46.2868239Z Dec 23 02:54:46 at
> org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.(NetworkBufferPool.java:138)
> 2021-12-23T02:54:46.2868975Z Dec 23 02:54:46 at
> org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createNettyShuffleEnvironment(NettyShuffleServiceFactory.java:140)
> 2021-12-23T02:54:46.2869771Z Dec 23 02:54:46 at
> org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createNettyShuffleEnvironment(NettyShuffleServiceFactory.java:94)
> 2021-12-23T02:54:46.2870550Z Dec 23 02:54:46 at
> org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createShuffleEnvironment(NettyShuffleServiceFactory.java:79)
> 2021-12-23T02:54:46.2871312Z Dec 23 02:54:46 at
> org.apache.flink.runtime.io.network.NettyShuffleServiceFactory.createShuffleEnvironment(NettyShuffleServiceFactory.java:58)
> 2021-12-23T02:54:46.2872062Z Dec 23 02:54:46 at
> org.apache.flink.runtime.taskexecutor.TaskManagerServices.createShuffleEnvironment(TaskManagerServices.java:414)
> 2021-12-23T02:54:46.2872767Z Dec 23 02:54:46 at
> org.apache.flink.runtime.taskexecutor.TaskManagerServices.fromConfiguration(TaskManagerServices.java:282)
> 2021-12-23T02:54:46.2873436Z Dec 23 02:54:46 at
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner.startTaskManager(TaskManagerRunner.java:523)
> 2021-12-23T02:54:46.2877615Z Dec 23 02:54:46 at
> org.apache.flink.runtime.minicluster.MiniCluster.startTaskManager(MiniCluster.java:645)
> 2021-12-23T02:54:46.2878247Z Dec 23 02:54:46 at
> org.apache.flink.runtime.minicluster.MiniCluster.startTaskManagers(MiniCluster.java:626)
> 2021-12-23T02:54:46.2878856Z Dec 23 02:54:46 at
> org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:379)
> 2021-12-23T02:54:46.2879487Z Dec 23 02:54:46 at
> org.apache.flink.runtime.testutils.MiniClusterResource.startMiniCluster(MiniClusterResource.java:209)
> 2021-12-23T02:54:46.2880152Z Dec 23 02:54:46 at
> org.apache.flink.runtime.testutils.MiniClusterResource.before(MiniClusterResource.java:95)
> 2021-12-23T02:54:46.2880821Z Dec 23 02:54:46 at
> org.apache.flink.test.util.MiniClusterWithClientResource.before(MiniClusterWithClientResource.java:64)
> 2021-12-23T02:54:46.2881519Z Dec 23 02:54:46 at
>
[jira] [Commented] (FLINK-25941) KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487929#comment-17487929
]
Yun Gao commented on FLINK-25941:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=ce8f3cc3-c1ea-5281-f5eb-df9ebd24947f=918e890f-5ed9-5212-a25e-962628fb4bc5=35677
> KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint fails
> on AZP
> ---
>
> Key: FLINK-25941
> URL: https://issues.apache.org/jira/browse/FLINK-25941
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test
> {{KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint}}
> fails on AZP with
> {code}
> 2022-02-02T17:22:29.5131631Z Feb 02 17:22:29 [ERROR]
> org.apache.flink.connector.kafka.sink.KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint
> Time elapsed: 2.186 s <<< FAILURE!
> 2022-02-02T17:22:29.5146972Z Feb 02 17:22:29 java.lang.AssertionError
> 2022-02-02T17:22:29.5148918Z Feb 02 17:22:29 at
> org.junit.Assert.fail(Assert.java:87)
> 2022-02-02T17:22:29.5149843Z Feb 02 17:22:29 at
> org.junit.Assert.assertTrue(Assert.java:42)
> 2022-02-02T17:22:29.5150644Z Feb 02 17:22:29 at
> org.junit.Assert.assertTrue(Assert.java:53)
> 2022-02-02T17:22:29.5151730Z Feb 02 17:22:29 at
> org.apache.flink.connector.kafka.sink.KafkaSinkITCase.testAbortTransactionsAfterScaleInBeforeFirstCheckpoint(KafkaSinkITCase.java:267)
> 2022-02-02T17:22:29.5152858Z Feb 02 17:22:29 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2022-02-02T17:22:29.5153757Z Feb 02 17:22:29 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2022-02-02T17:22:29.5155002Z Feb 02 17:22:29 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2022-02-02T17:22:29.5156464Z Feb 02 17:22:29 at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2022-02-02T17:22:29.5157384Z Feb 02 17:22:29 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> 2022-02-02T17:22:29.5158445Z Feb 02 17:22:29 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2022-02-02T17:22:29.5159478Z Feb 02 17:22:29 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> 2022-02-02T17:22:29.5160524Z Feb 02 17:22:29 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2022-02-02T17:22:29.5161758Z Feb 02 17:22:29 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2022-02-02T17:22:29.5162775Z Feb 02 17:22:29 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2022-02-02T17:22:29.5163744Z Feb 02 17:22:29 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> 2022-02-02T17:22:29.5164913Z Feb 02 17:22:29 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> 2022-02-02T17:22:29.5166101Z Feb 02 17:22:29 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> 2022-02-02T17:22:29.5167030Z Feb 02 17:22:29 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> 2022-02-02T17:22:29.5167953Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 2022-02-02T17:22:29.5168956Z Feb 02 17:22:29 at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> 2022-02-02T17:22:29.5169936Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> 2022-02-02T17:22:29.5170903Z Feb 02 17:22:29 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> 2022-02-02T17:22:29.5171953Z Feb 02 17:22:29 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> 2022-02-02T17:22:29.5172919Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> 2022-02-02T17:22:29.5173811Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> 2022-02-02T17:22:29.5174874Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> 2022-02-02T17:22:29.5175917Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> 2022-02-02T17:22:29.5176851Z Feb 02 17:22:29 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> 2022-02-02T17:22:29.5177816Z Feb 02 17:22:29 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2022-02-02T17:22:29.5178816Z Feb 02 17:22:29
[jira] [Commented] (FLINK-25825) MySqlCatalogITCase fails on azure
[
https://issues.apache.org/jira/browse/FLINK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487928#comment-17487928
]
Yun Gao commented on FLINK-25825:
-
Hi [~RocMarshal] Very thanks for the investigation. Might it because some
elements in the list does not implement the `equals` method properly?
> MySqlCatalogITCase fails on azure
> -
>
> Key: FLINK-25825
> URL: https://issues.apache.org/jira/browse/FLINK-25825
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30189=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=13677
>
> {code}
> 2022-01-26T06:04:42.8019913Z Jan 26 06:04:42 [ERROR]
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath Time
> elapsed: 2.166 *s <<< FAILURE!
> 2022-01-26T06:04:42.8025522Z Jan 26 06:04:42 java.lang.AssertionError:
> expected: java.util.ArrayList<[+I[1, -1, 1, null, true, null, hello, 2021-0
> 8-04, 2021-08-04T01:54:16, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1,
> \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99, 9 9,
> -1.0, 1.0, set_ele1, -1, 1, col_text, 10:32:34, 2021-08-04T01:54:16,
> col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:54:16.463, 09:33:43,
> 2021-08-04T01:54:16.463, null], +I[2, -1, 1, null, true, null, hello,
> 2021-08-04, 2021-08-04T01:53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1,
> -1, 1, \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99,
> 99, -1.0, 1.0, set_ele1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-
> 04T01:53:19, col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:53:19.098,
> 09:33:43, 2021-08-04T01:53:19.098, null]]> but was: java.util.ArrayL
> ist<[+I[1, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:54:16,
> -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_ele1, -1,
> 1, col_text, 10:32:34, 2021-08-04T01:54:16, col_tinytext, -1, 1, null ,
> col_varchar, 2021-08-04T01:54:16.463, 09:33:43, 2021-08-04T01:54:16.463,
> null], +I[2, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:
> 53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_el
> e1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-04T01:53:19, col_tinytext,
> -1, 1, null, col_varchar, 2021-08-04T01:53:19.098, 09:33:43, 2021-08-0
> 4T01:53:19.098, null]]>
> 2022-01-26T06:04:42.8029336Z Jan 26 06:04:42 at
> org.junit.Assert.fail(Assert.java:89)
> 2022-01-26T06:04:42.8029824Z Jan 26 06:04:42 at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2022-01-26T06:04:42.8030319Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2022-01-26T06:04:42.8030815Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2022-01-26T06:04:42.8031419Z Jan 26 06:04:42 at
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath(MySqlCatalogITCase.java*:306)
> {code}
>
> {code}
> 2022-01-26T06:04:43.2899378Z Jan 26 06:04:43 [ERROR] Failures:
> 2022-01-26T06:04:43.2907942Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testFullPath:306 expected: java.util.ArrayList<[+I[1, -1,
> 1, null, true,
> 2022-01-26T06:04:43.2914065Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testGetTable:253 expected:<(
> 2022-01-26T06:04:43.2983567Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testSelectToInsert:323 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.2997373Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalog:291 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.3010450Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalogDB:278 expected:
> java.util.ArrayList<[+I[1, -1, 1, nul
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25825) MySqlCatalogITCase fails on azure
[
https://issues.apache.org/jira/browse/FLINK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487928#comment-17487928
]
Yun Gao edited comment on FLINK-25825 at 2/7/22, 7:48 AM:
--
Hi [~RocMarshal] Very thanks for the investigation! Might it because some
elements in the list does not implement the `equals` method properly?
was (Author: gaoyunhaii):
Hi [~RocMarshal] Very thanks for the investigation. Might it because some
elements in the list does not implement the `equals` method properly?
> MySqlCatalogITCase fails on azure
> -
>
> Key: FLINK-25825
> URL: https://issues.apache.org/jira/browse/FLINK-25825
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30189=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=13677
>
> {code}
> 2022-01-26T06:04:42.8019913Z Jan 26 06:04:42 [ERROR]
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath Time
> elapsed: 2.166 *s <<< FAILURE!
> 2022-01-26T06:04:42.8025522Z Jan 26 06:04:42 java.lang.AssertionError:
> expected: java.util.ArrayList<[+I[1, -1, 1, null, true, null, hello, 2021-0
> 8-04, 2021-08-04T01:54:16, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1,
> \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99, 9 9,
> -1.0, 1.0, set_ele1, -1, 1, col_text, 10:32:34, 2021-08-04T01:54:16,
> col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:54:16.463, 09:33:43,
> 2021-08-04T01:54:16.463, null], +I[2, -1, 1, null, true, null, hello,
> 2021-08-04, 2021-08-04T01:53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1,
> -1, 1, \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99,
> 99, -1.0, 1.0, set_ele1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-
> 04T01:53:19, col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:53:19.098,
> 09:33:43, 2021-08-04T01:53:19.098, null]]> but was: java.util.ArrayL
> ist<[+I[1, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:54:16,
> -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_ele1, -1,
> 1, col_text, 10:32:34, 2021-08-04T01:54:16, col_tinytext, -1, 1, null ,
> col_varchar, 2021-08-04T01:54:16.463, 09:33:43, 2021-08-04T01:54:16.463,
> null], +I[2, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:
> 53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_el
> e1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-04T01:53:19, col_tinytext,
> -1, 1, null, col_varchar, 2021-08-04T01:53:19.098, 09:33:43, 2021-08-0
> 4T01:53:19.098, null]]>
> 2022-01-26T06:04:42.8029336Z Jan 26 06:04:42 at
> org.junit.Assert.fail(Assert.java:89)
> 2022-01-26T06:04:42.8029824Z Jan 26 06:04:42 at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2022-01-26T06:04:42.8030319Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2022-01-26T06:04:42.8030815Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2022-01-26T06:04:42.8031419Z Jan 26 06:04:42 at
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath(MySqlCatalogITCase.java*:306)
> {code}
>
> {code}
> 2022-01-26T06:04:43.2899378Z Jan 26 06:04:43 [ERROR] Failures:
> 2022-01-26T06:04:43.2907942Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testFullPath:306 expected: java.util.ArrayList<[+I[1, -1,
> 1, null, true,
> 2022-01-26T06:04:43.2914065Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testGetTable:253 expected:<(
> 2022-01-26T06:04:43.2983567Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testSelectToInsert:323 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.2997373Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalog:291 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.3010450Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalogDB:278 expected:
> java.util.ArrayList<[+I[1, -1, 1, nul
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25825) MySqlCatalogITCase fails on azure
[
https://issues.apache.org/jira/browse/FLINK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487927#comment-17487927
]
Yun Gao commented on FLINK-25825:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=14489
> MySqlCatalogITCase fails on azure
> -
>
> Key: FLINK-25825
> URL: https://issues.apache.org/jira/browse/FLINK-25825
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30189=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=13677
>
> {code}
> 2022-01-26T06:04:42.8019913Z Jan 26 06:04:42 [ERROR]
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath Time
> elapsed: 2.166 *s <<< FAILURE!
> 2022-01-26T06:04:42.8025522Z Jan 26 06:04:42 java.lang.AssertionError:
> expected: java.util.ArrayList<[+I[1, -1, 1, null, true, null, hello, 2021-0
> 8-04, 2021-08-04T01:54:16, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1,
> \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99, 9 9,
> -1.0, 1.0, set_ele1, -1, 1, col_text, 10:32:34, 2021-08-04T01:54:16,
> col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:54:16.463, 09:33:43,
> 2021-08-04T01:54:16.463, null], +I[2, -1, 1, null, true, null, hello,
> 2021-08-04, 2021-08-04T01:53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1,
> -1, 1, \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99,
> 99, -1.0, 1.0, set_ele1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-
> 04T01:53:19, col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:53:19.098,
> 09:33:43, 2021-08-04T01:53:19.098, null]]> but was: java.util.ArrayL
> ist<[+I[1, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:54:16,
> -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_ele1, -1,
> 1, col_text, 10:32:34, 2021-08-04T01:54:16, col_tinytext, -1, 1, null ,
> col_varchar, 2021-08-04T01:54:16.463, 09:33:43, 2021-08-04T01:54:16.463,
> null], +I[2, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:
> 53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_el
> e1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-04T01:53:19, col_tinytext,
> -1, 1, null, col_varchar, 2021-08-04T01:53:19.098, 09:33:43, 2021-08-0
> 4T01:53:19.098, null]]>
> 2022-01-26T06:04:42.8029336Z Jan 26 06:04:42 at
> org.junit.Assert.fail(Assert.java:89)
> 2022-01-26T06:04:42.8029824Z Jan 26 06:04:42 at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2022-01-26T06:04:42.8030319Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2022-01-26T06:04:42.8030815Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2022-01-26T06:04:42.8031419Z Jan 26 06:04:42 at
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath(MySqlCatalogITCase.java*:306)
> {code}
>
> {code}
> 2022-01-26T06:04:43.2899378Z Jan 26 06:04:43 [ERROR] Failures:
> 2022-01-26T06:04:43.2907942Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testFullPath:306 expected: java.util.ArrayList<[+I[1, -1,
> 1, null, true,
> 2022-01-26T06:04:43.2914065Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testGetTable:253 expected:<(
> 2022-01-26T06:04:43.2983567Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testSelectToInsert:323 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.2997373Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalog:291 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.3010450Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalogDB:278 expected:
> java.util.ArrayList<[+I[1, -1, 1, nul
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-25839) 'Run kubernetes application HA test' failed on azure due to could not get 3 completed checkpoints in 120 sec
[
https://issues.apache.org/jira/browse/FLINK-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487921#comment-17487921
]
Yang Wang edited comment on FLINK-25839 at 2/7/22, 7:47 AM:
I have merged a PR to print the previous logs of failed pod. It is useful to
debug the root cause for this ticket.
Actual, the failed kubernetes-application-ha test is a victim by other unstable
test. Let's wait for another instance.
was (Author: fly_in_gis):
I have merged a PR to print the previous logs of failed pod. It is very useful
to debug the root cause for this ticket.
> 'Run kubernetes application HA test' failed on azure due to could not get 3
> completed checkpoints in 120 sec
>
>
> Key: FLINK-25839
> URL: https://issues.apache.org/jira/browse/FLINK-25839
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Assignee: Yang Wang
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> {code:java}
> Jan 27 02:07:33 deployment.apps/flink-native-k8s-application-ha-1 condition
> met
> Jan 27 02:07:33 Waiting for job
> (flink-native-k8s-application-ha-1-d8dc997d5-v8cpz) to have at least 3
> completed checkpoints ...
> Jan 27 02:09:45 Could not get 3 completed checkpoints in 120 sec
> Jan 27 02:09:45 Stopping job timeout watchdog (with pid=217858)
> Jan 27 02:09:45 Debugging failed Kubernetes test:
> Jan 27 02:09:45 Currently existing Kubernetes resources
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30261=logs=af885ea8-6b05-5dc2-4a37-eab9c0d1ab09=f779a55a-0ffe-5bbc-8824-3a79333d4559=5376
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25924) KDF Integration tests intermittently fails
[
https://issues.apache.org/jira/browse/FLINK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487926#comment-17487926
]
Yun Gao commented on FLINK-25924:
-
The test blocked again on azure:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=ba53eb01-1462-56a3-8e98-0dd97fbcaab5=2e426bf0-b717-56bb-ab62-d63086457354=82464
> KDF Integration tests intermittently fails
> --
>
> Key: FLINK-25924
> URL: https://issues.apache.org/jira/browse/FLINK-25924
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
>Reporter: Zichen Liu
>Assignee: Ahmed Hamdy
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> Intermittent failures introduced as part of merge (PR#18314:
> [FLINK-24228[connectors/firehose] - Unified Async Sink for Kinesis
> Firehose|https://github.com/apache/flink/pull/18314]):
> # Failures are intermittent and affecting c. 1 in 7 of builds- on
> {{flink-ci.flink}} and {{flink-ci.flink-master-mirror}} .
> # The issue looks identical to the KinesaliteContainer startup issue
> (Appendix 1).
> # I have managed to reproduce the issue locally - if I start some parallel
> containers and keep them running - and then run {{KinesisFirehoseSinkITCase}}
> then c. 1 in 6 gives the error.
> # The errors have a slightly different appearance on
> {{flink-ci.flink-master-mirror}} vs {{flink-ci.flink}} which has the same
> appearance as local. I only hope it is a difference in logging/killing
> environment variables. (and that there aren’t 2 distinct issues)
> Appendix 1:
> {code:java}
> org.testcontainers.containers.ContainerLaunchException: Container startup
> failed
> at
> org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336)
> at
> org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
> at
> org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1066)
> at
> ... 11 more
> Caused by: org.testcontainers.containers.ContainerLaunchException: Could not
> create/start container
> at
> org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525)
> at
> org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
> at
> org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
> ... 12 more
> Caused by: org.rnorth.ducttape.TimeoutException: Timeout waiting for result
> with exception
> at
> org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:54)
> at
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Reopened] (FLINK-25924) KDF Integration tests intermittently fails
[
https://issues.apache.org/jira/browse/FLINK-25924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao reopened FLINK-25924:
-
> KDF Integration tests intermittently fails
> --
>
> Key: FLINK-25924
> URL: https://issues.apache.org/jira/browse/FLINK-25924
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
>Reporter: Zichen Liu
>Assignee: Ahmed Hamdy
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> Intermittent failures introduced as part of merge (PR#18314:
> [FLINK-24228[connectors/firehose] - Unified Async Sink for Kinesis
> Firehose|https://github.com/apache/flink/pull/18314]):
> # Failures are intermittent and affecting c. 1 in 7 of builds- on
> {{flink-ci.flink}} and {{flink-ci.flink-master-mirror}} .
> # The issue looks identical to the KinesaliteContainer startup issue
> (Appendix 1).
> # I have managed to reproduce the issue locally - if I start some parallel
> containers and keep them running - and then run {{KinesisFirehoseSinkITCase}}
> then c. 1 in 6 gives the error.
> # The errors have a slightly different appearance on
> {{flink-ci.flink-master-mirror}} vs {{flink-ci.flink}} which has the same
> appearance as local. I only hope it is a difference in logging/killing
> environment variables. (and that there aren’t 2 distinct issues)
> Appendix 1:
> {code:java}
> org.testcontainers.containers.ContainerLaunchException: Container startup
> failed
> at
> org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:336)
> at
> org.testcontainers.containers.GenericContainer.start(GenericContainer.java:317)
> at
> org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1066)
> at
> ... 11 more
> Caused by: org.testcontainers.containers.ContainerLaunchException: Could not
> create/start container
> at
> org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:525)
> at
> org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:331)
> at
> org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
> ... 12 more
> Caused by: org.rnorth.ducttape.TimeoutException: Timeout waiting for result
> with exception
> at
> org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:54)
> at
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * ad74b6cfec708adbca9a115f5692022aedf46915 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30823) * 78c2513ffe72a83ba689789fd5c14dbda4d93c99 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30824) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24433) "No space left on device" in Azure e2e tests
[
https://issues.apache.org/jira/browse/FLINK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487922#comment-17487922
]
Yun Gao commented on FLINK-24433:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=e1276d0f-df12-55ec-86b5-c0ad597d83c9
> "No space left on device" in Azure e2e tests
>
>
> Key: FLINK-24433
> URL: https://issues.apache.org/jira/browse/FLINK-24433
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines
>Affects Versions: 1.15.0
>Reporter: Dawid Wysakowicz
>Priority: Critical
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=24668=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=070ff179-953e-5bda-71fa-d6599415701c=19772
> {code}
> Sep 30 17:08:42 Job has been submitted with JobID
> 5594c18e128a328ede39cfa59cb3cb07
> Sep 30 17:08:56 2021-09-30 17:08:56,809 main ERROR Recovering from
> StringBuilderEncoder.encode('2021-09-30 17:08:56,807 WARN
> org.apache.flink.streaming.api.operators.collect.CollectResultFetcher [] - An
> exception occurred when fetching query results
> Sep 30 17:08:56 java.util.concurrent.ExecutionException:
> org.apache.flink.runtime.rest.util.RestClientException: [Internal server
> error., Sep 30 17:08:56 org.apache.flink.runtime.messages.FlinkJobNotFoundException:
> Could not find Flink job (5594c18e128a328ede39cfa59cb3cb07)
> Sep 30 17:08:56 at
> org.apache.flink.runtime.dispatcher.Dispatcher.getJobMasterGateway(Dispatcher.java:923)
> Sep 30 17:08:56 at
> org.apache.flink.runtime.dispatcher.Dispatcher.performOperationOnJobMasterGateway(Dispatcher.java:937)
> Sep 30 17:08:56 at
> org.apache.flink.runtime.dispatcher.Dispatcher.deliverCoordinationRequestToCoordina2021-09-30T17:08:57.1584224Z
> ##[error]No space left on device
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (FLINK-25839) 'Run kubernetes application HA test' failed on azure due to could not get 3 completed checkpoints in 120 sec
[
https://issues.apache.org/jira/browse/FLINK-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Wang reassigned FLINK-25839:
-
Assignee: Yang Wang
> 'Run kubernetes application HA test' failed on azure due to could not get 3
> completed checkpoints in 120 sec
>
>
> Key: FLINK-25839
> URL: https://issues.apache.org/jira/browse/FLINK-25839
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Assignee: Yang Wang
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> {code:java}
> Jan 27 02:07:33 deployment.apps/flink-native-k8s-application-ha-1 condition
> met
> Jan 27 02:07:33 Waiting for job
> (flink-native-k8s-application-ha-1-d8dc997d5-v8cpz) to have at least 3
> completed checkpoints ...
> Jan 27 02:09:45 Could not get 3 completed checkpoints in 120 sec
> Jan 27 02:09:45 Stopping job timeout watchdog (with pid=217858)
> Jan 27 02:09:45 Debugging failed Kubernetes test:
> Jan 27 02:09:45 Currently existing Kubernetes resources
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30261=logs=af885ea8-6b05-5dc2-4a37-eab9c0d1ab09=f779a55a-0ffe-5bbc-8824-3a79333d4559=5376
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25839) 'Run kubernetes application HA test' failed on azure due to could not get 3 completed checkpoints in 120 sec
[
https://issues.apache.org/jira/browse/FLINK-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487921#comment-17487921
]
Yang Wang commented on FLINK-25839:
---
I have merged a PR to print the previous logs of failed pod. It is very useful
to debug the root cause for this ticket.
> 'Run kubernetes application HA test' failed on azure due to could not get 3
> completed checkpoints in 120 sec
>
>
> Key: FLINK-25839
> URL: https://issues.apache.org/jira/browse/FLINK-25839
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> {code:java}
> Jan 27 02:07:33 deployment.apps/flink-native-k8s-application-ha-1 condition
> met
> Jan 27 02:07:33 Waiting for job
> (flink-native-k8s-application-ha-1-d8dc997d5-v8cpz) to have at least 3
> completed checkpoints ...
> Jan 27 02:09:45 Could not get 3 completed checkpoints in 120 sec
> Jan 27 02:09:45 Stopping job timeout watchdog (with pid=217858)
> Jan 27 02:09:45 Debugging failed Kubernetes test:
> Jan 27 02:09:45 Currently existing Kubernetes resources
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30261=logs=af885ea8-6b05-5dc2-4a37-eab9c0d1ab09=f779a55a-0ffe-5bbc-8824-3a79333d4559=5376
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] wangyang0918 merged pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s e2e tests
wangyang0918 merged pull request #18639: URL: https://github.com/apache/flink/pull/18639 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-25281) StreamingModeDataStreamTests::test_set_stream_env failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487913#comment-17487913
]
Yun Gao edited comment on FLINK-25281 at 2/7/22, 7:34 AM:
--
There is an opened issue specially for test_keyed_process_function_with_state:
https://issues.apache.org/jira/browse/FLINK-25940
was (Author: gaoyunhaii):
I'll open a new issue for test_keyed_process_function_with_state~
> StreamingModeDataStreamTests::test_set_stream_env failed on azure
> -
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
> === FAILURES
> ===
> 2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
> StreamExecutionEnvironmentTests.test_set_stream_env __
> 2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
> testMethod=test_set_stream_env>
> 2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0907483Z Dec 13 02:25:17
> @unittest.skipIf(on_windows(), "Symbolic link is not supported on Windows,
> skipping.")
> 2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def
> test_set_stream_env(self):
> 2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
> 2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
> sys.executable
> 2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
> 2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
> 2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
> os.path.join(tmp_dir, "py_exec")
> 2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
> python_exec_link_path)
> 2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
> env.set_python_executable(python_exec_link_path)
> 2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
> 2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
> 2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
> os.environ["python"] == python_exec_link_path
> 2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
> 2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
> env.from_collection([1, 2, 3, 4, 5])
> 2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
> ds.map(check_python_exec).add_sink(self.test_sink)
> 2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
> python executable")
> 2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
> pyflink/datastream/tests/test_stream_execution_environment.py:546:
> 2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
> pyflink/datastream/stream_execution_environment.py:691: in execute
> 2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
> JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
> 2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
> .tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
> __call__
> 2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
> self.target_id, self.name)
> 2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146:
> in deco
> 2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
> 2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
> 2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
>
> 2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
> 'execute'
> 2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
> gateway_client, target_id=None, name=None):
> 2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
> received from the Java gateway into a Python object.
> 2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
> representation of integers are converted to Python
> 2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
> representation of
[jira] [Commented] (FLINK-25940) pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state failed on AZP
[
https://issues.apache.org/jira/browse/FLINK-25940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487917#comment-17487917
]
Yun Gao commented on FLINK-25940:
-
Perhaps cc [~hxbks2ks]~
> pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state
> failed on AZP
>
>
> Key: FLINK-25940
> URL: https://issues.apache.org/jira/browse/FLINK-25940
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test
> {{pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state}}
> fails on AZP:
> {code}
> 2022-02-02T17:44:12.1898582Z Feb 02 17:44:12
> === FAILURES
> ===
> 2022-02-02T17:44:12.1899860Z Feb 02 17:44:12 _
> StreamingModeDataStreamTests.test_keyed_process_function_with_state __
> 2022-02-02T17:44:12.1900493Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1901218Z Feb 02 17:44:12 self =
> testMethod=test_keyed_process_function_with_state>
> 2022-02-02T17:44:12.1901948Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1902745Z Feb 02 17:44:12 def
> test_keyed_process_function_with_state(self):
> 2022-02-02T17:44:12.1903722Z Feb 02 17:44:12
> self.env.get_config().set_auto_watermark_interval(2000)
> 2022-02-02T17:44:12.1904473Z Feb 02 17:44:12
> self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
> 2022-02-02T17:44:12.1906780Z Feb 02 17:44:12 data_stream =
> self.env.from_collection([(1, 'hi', '1603708211000'),
> 2022-02-02T17:44:12.1908034Z Feb 02 17:44:12
>(2, 'hello', '1603708224000'),
> 2022-02-02T17:44:12.1909166Z Feb 02 17:44:12
>(3, 'hi', '1603708226000'),
> 2022-02-02T17:44:12.1910122Z Feb 02 17:44:12
>(4, 'hello', '1603708289000'),
> 2022-02-02T17:44:12.1911099Z Feb 02 17:44:12
>(5, 'hi', '1603708291000'),
> 2022-02-02T17:44:12.1912451Z Feb 02 17:44:12
>(6, 'hello', '1603708293000')],
> 2022-02-02T17:44:12.1913456Z Feb 02 17:44:12
> type_info=Types.ROW([Types.INT(), Types.STRING(),
> 2022-02-02T17:44:12.1914338Z Feb 02 17:44:12
>Types.STRING()]))
> 2022-02-02T17:44:12.1914811Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1915317Z Feb 02 17:44:12 class
> MyTimestampAssigner(TimestampAssigner):
> 2022-02-02T17:44:12.1915724Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1916782Z Feb 02 17:44:12 def
> extract_timestamp(self, value, record_timestamp) -> int:
> 2022-02-02T17:44:12.1917621Z Feb 02 17:44:12 return
> int(value[2])
> 2022-02-02T17:44:12.1918262Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1918855Z Feb 02 17:44:12 class
> MyProcessFunction(KeyedProcessFunction):
> 2022-02-02T17:44:12.1919363Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1919744Z Feb 02 17:44:12 def __init__(self):
> 2022-02-02T17:44:12.1920143Z Feb 02 17:44:12 self.value_state
> = None
> 2022-02-02T17:44:12.1920648Z Feb 02 17:44:12 self.list_state
> = None
> 2022-02-02T17:44:12.1921298Z Feb 02 17:44:12 self.map_state =
> None
> 2022-02-02T17:44:12.1921864Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1922479Z Feb 02 17:44:12 def open(self,
> runtime_context: RuntimeContext):
> 2022-02-02T17:44:12.1923907Z Feb 02 17:44:12
> value_state_descriptor = ValueStateDescriptor('value_state', Types.INT())
> 2022-02-02T17:44:12.1924922Z Feb 02 17:44:12 self.value_state
> = runtime_context.get_state(value_state_descriptor)
> 2022-02-02T17:44:12.1925741Z Feb 02 17:44:12
> list_state_descriptor = ListStateDescriptor('list_state', Types.INT())
> 2022-02-02T17:44:12.1926482Z Feb 02 17:44:12 self.list_state
> = runtime_context.get_list_state(list_state_descriptor)
> 2022-02-02T17:44:12.1927465Z Feb 02 17:44:12
> map_state_descriptor = MapStateDescriptor('map_state', Types.INT(),
> Types.STRING())
> 2022-02-02T17:44:12.1927998Z Feb 02 17:44:12 state_ttl_config
> = StateTtlConfig \
> 2022-02-02T17:44:12.1928444Z Feb 02 17:44:12
> .new_builder(Time.seconds(1)) \
> 2022-02-02T17:44:12.1928943Z Feb 02 17:44:12
>
[GitHub] [flink] flinkbot edited a comment on pull request #18561: [FLINK-25874] loose PyFlink dependency of 'python-dateutil' to '>=2.8.0,<3'
flinkbot edited a comment on pull request #18561: URL: https://github.com/apache/flink/pull/18561#issuecomment-1025075470 ## CI report: * 2b81551555d3b9bdfb1300bbd6fc2b718294956e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30822) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-25967) StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao closed FLINK-25967.
---
Resolution: Duplicate
> StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on
> azure
> ---
>
> Key: FLINK-25967
> URL: https://issues.apache.org/jira/browse/FLINK-25967
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Major
>
> {code:java}
> 2022-02-07T03:46:14.7065997Z Feb 07 03:46:14
> === FAILURES
> ===
> 2022-02-07T03:46:14.7077277Z Feb 07 03:46:14 _
> StreamingModeDataStreamTests.test_keyed_process_function_with_state __
> 2022-02-07T03:46:14.703Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7078301Z Feb 07 03:46:14 self =
> testMethod=test_keyed_process_function_with_state>
> 2022-02-07T03:46:14.7078856Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7079241Z Feb 07 03:46:14 def
> test_keyed_process_function_with_state(self):
> 2022-02-07T03:46:14.7079730Z Feb 07 03:46:14
> self.env.get_config().set_auto_watermark_interval(2000)
> 2022-02-07T03:46:14.7080460Z Feb 07 03:46:14
> self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
> 2022-02-07T03:46:14.7081786Z Feb 07 03:46:14 data_stream =
> self.env.from_collection([(1, 'hi', '1603708211000'),
> 2022-02-07T03:46:14.7082489Z Feb 07 03:46:14
>(2, 'hello', '1603708224000'),
> 2022-02-07T03:46:14.7083172Z Feb 07 03:46:14
>(3, 'hi', '1603708226000'),
> 2022-02-07T03:46:14.7083808Z Feb 07 03:46:14
>(4, 'hello', '1603708289000'),
> 2022-02-07T03:46:14.7084439Z Feb 07 03:46:14
>(5, 'hi', '1603708291000'),
> 2022-02-07T03:46:14.7085060Z Feb 07 03:46:14
>(6, 'hello', '1603708293000')],
> 2022-02-07T03:46:14.7085549Z Feb 07 03:46:14
> type_info=Types.ROW([Types.INT(), Types.STRING(),
> 2022-02-07T03:46:14.7086019Z Feb 07 03:46:14
>Types.STRING()]))
> 2022-02-07T03:46:14.7086374Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7086749Z Feb 07 03:46:14 class
> MyTimestampAssigner(TimestampAssigner):
> 2022-02-07T03:46:14.7087121Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7087695Z Feb 07 03:46:14 def
> extract_timestamp(self, value, record_timestamp) -> int:
> 2022-02-07T03:46:14.7088140Z Feb 07 03:46:14 return
> int(value[2])
> 2022-02-07T03:46:14.7088469Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7088848Z Feb 07 03:46:14 class
> MyProcessFunction(KeyedProcessFunction):
> 2022-02-07T03:46:14.7089220Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7089537Z Feb 07 03:46:14 def __init__(self):
> 2022-02-07T03:46:14.7089916Z Feb 07 03:46:14 self.value_state
> = None
> 2022-02-07T03:46:14.7090294Z Feb 07 03:46:14 self.list_state
> = None
> 2022-02-07T03:46:14.7090676Z Feb 07 03:46:14 self.map_state =
> None
> 2022-02-07T03:46:14.7091117Z Feb 07 03:46:14
> 2022-02-07T03:46:14.7091482Z Feb 07 03:46:14 def open(self,
> runtime_context: RuntimeContext):
> 2022-02-07T03:46:14.7092223Z Feb 07 03:46:14
> value_state_descriptor = ValueStateDescriptor('value_state', Types.INT())
> 2022-02-07T03:46:14.7092790Z Feb 07 03:46:14 self.value_state
> = runtime_context.get_state(value_state_descriptor)
> 2022-02-07T03:46:14.7093544Z Feb 07 03:46:14
> list_state_descriptor = ListStateDescriptor('list_state', Types.INT())
> 2022-02-07T03:46:14.7094102Z Feb 07 03:46:14 self.list_state
> = runtime_context.get_list_state(list_state_descriptor)
> 2022-02-07T03:46:14.7094883Z Feb 07 03:46:14
> map_state_descriptor = MapStateDescriptor('map_state', Types.INT(),
> Types.STRING())
> 2022-02-07T03:46:14.7095550Z Feb 07 03:46:14 state_ttl_config
> = StateTtlConfig \
> 2022-02-07T03:46:14.7095960Z Feb 07 03:46:14
> .new_builder(Time.seconds(1)) \
> 2022-02-07T03:46:14.7096643Z Feb 07 03:46:14
> .set_update_type(StateTtlConfig.UpdateType.OnReadAndWrite) \
> 2022-02-07T03:46:14.7097104Z Feb 07 03:46:14
> .set_state_visibility(
> 2022-02-07T03:46:14.7097563Z Feb 07 03:46:14
> StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp) \
> 2022-02-07T03:46:14.7098044Z Feb
[jira] [Commented] (FLINK-25940) pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state failed on AZP
[
https://issues.apache.org/jira/browse/FLINK-25940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487916#comment-17487916
]
Yun Gao commented on FLINK-25940:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=821b528f-1eed-5598-a3b4-7f748b13f261=6bb545dd-772d-5d8c-f258-f5085fba3295=23725
> pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state
> failed on AZP
>
>
> Key: FLINK-25940
> URL: https://issues.apache.org/jira/browse/FLINK-25940
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The test
> {{pyflink/datastream/tests/test_data_stream.py::StreamingModeDataStreamTests::test_keyed_process_function_with_state}}
> fails on AZP:
> {code}
> 2022-02-02T17:44:12.1898582Z Feb 02 17:44:12
> === FAILURES
> ===
> 2022-02-02T17:44:12.1899860Z Feb 02 17:44:12 _
> StreamingModeDataStreamTests.test_keyed_process_function_with_state __
> 2022-02-02T17:44:12.1900493Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1901218Z Feb 02 17:44:12 self =
> testMethod=test_keyed_process_function_with_state>
> 2022-02-02T17:44:12.1901948Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1902745Z Feb 02 17:44:12 def
> test_keyed_process_function_with_state(self):
> 2022-02-02T17:44:12.1903722Z Feb 02 17:44:12
> self.env.get_config().set_auto_watermark_interval(2000)
> 2022-02-02T17:44:12.1904473Z Feb 02 17:44:12
> self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
> 2022-02-02T17:44:12.1906780Z Feb 02 17:44:12 data_stream =
> self.env.from_collection([(1, 'hi', '1603708211000'),
> 2022-02-02T17:44:12.1908034Z Feb 02 17:44:12
>(2, 'hello', '1603708224000'),
> 2022-02-02T17:44:12.1909166Z Feb 02 17:44:12
>(3, 'hi', '1603708226000'),
> 2022-02-02T17:44:12.1910122Z Feb 02 17:44:12
>(4, 'hello', '1603708289000'),
> 2022-02-02T17:44:12.1911099Z Feb 02 17:44:12
>(5, 'hi', '1603708291000'),
> 2022-02-02T17:44:12.1912451Z Feb 02 17:44:12
>(6, 'hello', '1603708293000')],
> 2022-02-02T17:44:12.1913456Z Feb 02 17:44:12
> type_info=Types.ROW([Types.INT(), Types.STRING(),
> 2022-02-02T17:44:12.1914338Z Feb 02 17:44:12
>Types.STRING()]))
> 2022-02-02T17:44:12.1914811Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1915317Z Feb 02 17:44:12 class
> MyTimestampAssigner(TimestampAssigner):
> 2022-02-02T17:44:12.1915724Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1916782Z Feb 02 17:44:12 def
> extract_timestamp(self, value, record_timestamp) -> int:
> 2022-02-02T17:44:12.1917621Z Feb 02 17:44:12 return
> int(value[2])
> 2022-02-02T17:44:12.1918262Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1918855Z Feb 02 17:44:12 class
> MyProcessFunction(KeyedProcessFunction):
> 2022-02-02T17:44:12.1919363Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1919744Z Feb 02 17:44:12 def __init__(self):
> 2022-02-02T17:44:12.1920143Z Feb 02 17:44:12 self.value_state
> = None
> 2022-02-02T17:44:12.1920648Z Feb 02 17:44:12 self.list_state
> = None
> 2022-02-02T17:44:12.1921298Z Feb 02 17:44:12 self.map_state =
> None
> 2022-02-02T17:44:12.1921864Z Feb 02 17:44:12
> 2022-02-02T17:44:12.1922479Z Feb 02 17:44:12 def open(self,
> runtime_context: RuntimeContext):
> 2022-02-02T17:44:12.1923907Z Feb 02 17:44:12
> value_state_descriptor = ValueStateDescriptor('value_state', Types.INT())
> 2022-02-02T17:44:12.1924922Z Feb 02 17:44:12 self.value_state
> = runtime_context.get_state(value_state_descriptor)
> 2022-02-02T17:44:12.1925741Z Feb 02 17:44:12
> list_state_descriptor = ListStateDescriptor('list_state', Types.INT())
> 2022-02-02T17:44:12.1926482Z Feb 02 17:44:12 self.list_state
> = runtime_context.get_list_state(list_state_descriptor)
> 2022-02-02T17:44:12.1927465Z Feb 02 17:44:12
> map_state_descriptor = MapStateDescriptor('map_state', Types.INT(),
> Types.STRING())
> 2022-02-02T17:44:12.1927998Z Feb 02 17:44:12 state_ttl_config
> = StateTtlConfig \
> 2022-02-02T17:44:12.1928444Z Feb
[GitHub] [flink] wangyang0918 commented on a change in pull request #18450: [FLINK-25053][docs] Document how to use the usrlib to load code in the user code class loader
wangyang0918 commented on a change in pull request #18450:
URL: https://github.com/apache/flink/pull/18450#discussion_r800374864
##
File path: docs/content.zh/docs/deployment/resource-providers/yarn.md
##
@@ -237,7 +237,14 @@ The configuration parameter for specifying the REST
endpoint port is [rest.bind-
### User jars & Classpath
-By default Flink will include the user jars into the system classpath when
running a single job. This behavior can be controlled with the
[yarn.per-job-cluster.include-user-jar]({{< ref "docs/deployment/config"
>}}#yarn-per-job-cluster-include-user-jar) parameter.
+**Session Mode & Application Mode**
+
+When deploying Flink with Session/Application Mode on Yarn, only the JAR file
specified in startup command will be recognized as user-jars and included into
user classpath.
Review comment:
For YARN application mode, the JAR file specified in startup command
will be added to system classpath by default. It could also be controlled by
`yarn.per-job-cluster.include-user-jar` although the name is misleading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests::test_set_stream_env failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Summary: StreamingModeDataStreamTests::test_set_stream_env failed on azure
(was: StreamingModeDataStreamTests#test_set_stream_env failed on azure)
> StreamingModeDataStreamTests::test_set_stream_env failed on azure
> -
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
> === FAILURES
> ===
> 2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
> StreamExecutionEnvironmentTests.test_set_stream_env __
> 2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
> testMethod=test_set_stream_env>
> 2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0907483Z Dec 13 02:25:17
> @unittest.skipIf(on_windows(), "Symbolic link is not supported on Windows,
> skipping.")
> 2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def
> test_set_stream_env(self):
> 2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
> 2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
> sys.executable
> 2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
> 2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
> 2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
> os.path.join(tmp_dir, "py_exec")
> 2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
> python_exec_link_path)
> 2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
> env.set_python_executable(python_exec_link_path)
> 2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
> 2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
> 2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
> os.environ["python"] == python_exec_link_path
> 2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
> 2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
> env.from_collection([1, 2, 3, 4, 5])
> 2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
> ds.map(check_python_exec).add_sink(self.test_sink)
> 2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
> python executable")
> 2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
> pyflink/datastream/tests/test_stream_execution_environment.py:546:
> 2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
> pyflink/datastream/stream_execution_environment.py:691: in execute
> 2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
> JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
> 2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
> .tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
> __call__
> 2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
> self.target_id, self.name)
> 2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146:
> in deco
> 2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
> 2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
> 2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
>
> 2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
> 'execute'
> 2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
> gateway_client, target_id=None, name=None):
> 2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
> received from the Java gateway into a Python object.
> 2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
> representation of integers are converted to Python
> 2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
> representation of objects are converted to JavaObject
> 2021-12-13T02:25:17.0933779Z Dec 13 02:25:17 instances, etc.
> 2021-12-13T02:25:17.0934191Z Dec 13 02:25:17
>
[jira] [Created] (FLINK-25967) StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on azure
Yun Gao created FLINK-25967:
---
Summary:
StreamingModeDataStreamTests.test_keyed_process_function_with_state failed on
azure
Key: FLINK-25967
URL: https://issues.apache.org/jira/browse/FLINK-25967
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.15.0
Reporter: Yun Gao
{code:java}
2022-02-07T03:46:14.7065997Z Feb 07 03:46:14
=== FAILURES ===
2022-02-07T03:46:14.7077277Z Feb 07 03:46:14 _
StreamingModeDataStreamTests.test_keyed_process_function_with_state __
2022-02-07T03:46:14.703Z Feb 07 03:46:14
2022-02-07T03:46:14.7078301Z Feb 07 03:46:14 self =
2022-02-07T03:46:14.7078856Z Feb 07 03:46:14
2022-02-07T03:46:14.7079241Z Feb 07 03:46:14 def
test_keyed_process_function_with_state(self):
2022-02-07T03:46:14.7079730Z Feb 07 03:46:14
self.env.get_config().set_auto_watermark_interval(2000)
2022-02-07T03:46:14.7080460Z Feb 07 03:46:14
self.env.set_stream_time_characteristic(TimeCharacteristic.EventTime)
2022-02-07T03:46:14.7081786Z Feb 07 03:46:14 data_stream =
self.env.from_collection([(1, 'hi', '1603708211000'),
2022-02-07T03:46:14.7082489Z Feb 07 03:46:14
(2, 'hello', '1603708224000'),
2022-02-07T03:46:14.7083172Z Feb 07 03:46:14
(3, 'hi', '1603708226000'),
2022-02-07T03:46:14.7083808Z Feb 07 03:46:14
(4, 'hello', '1603708289000'),
2022-02-07T03:46:14.7084439Z Feb 07 03:46:14
(5, 'hi', '1603708291000'),
2022-02-07T03:46:14.7085060Z Feb 07 03:46:14
(6, 'hello', '1603708293000')],
2022-02-07T03:46:14.7085549Z Feb 07 03:46:14
type_info=Types.ROW([Types.INT(), Types.STRING(),
2022-02-07T03:46:14.7086019Z Feb 07 03:46:14
Types.STRING()]))
2022-02-07T03:46:14.7086374Z Feb 07 03:46:14
2022-02-07T03:46:14.7086749Z Feb 07 03:46:14 class
MyTimestampAssigner(TimestampAssigner):
2022-02-07T03:46:14.7087121Z Feb 07 03:46:14
2022-02-07T03:46:14.7087695Z Feb 07 03:46:14 def
extract_timestamp(self, value, record_timestamp) -> int:
2022-02-07T03:46:14.7088140Z Feb 07 03:46:14 return
int(value[2])
2022-02-07T03:46:14.7088469Z Feb 07 03:46:14
2022-02-07T03:46:14.7088848Z Feb 07 03:46:14 class
MyProcessFunction(KeyedProcessFunction):
2022-02-07T03:46:14.7089220Z Feb 07 03:46:14
2022-02-07T03:46:14.7089537Z Feb 07 03:46:14 def __init__(self):
2022-02-07T03:46:14.7089916Z Feb 07 03:46:14 self.value_state =
None
2022-02-07T03:46:14.7090294Z Feb 07 03:46:14 self.list_state =
None
2022-02-07T03:46:14.7090676Z Feb 07 03:46:14 self.map_state =
None
2022-02-07T03:46:14.7091117Z Feb 07 03:46:14
2022-02-07T03:46:14.7091482Z Feb 07 03:46:14 def open(self,
runtime_context: RuntimeContext):
2022-02-07T03:46:14.7092223Z Feb 07 03:46:14
value_state_descriptor = ValueStateDescriptor('value_state', Types.INT())
2022-02-07T03:46:14.7092790Z Feb 07 03:46:14 self.value_state =
runtime_context.get_state(value_state_descriptor)
2022-02-07T03:46:14.7093544Z Feb 07 03:46:14
list_state_descriptor = ListStateDescriptor('list_state', Types.INT())
2022-02-07T03:46:14.7094102Z Feb 07 03:46:14 self.list_state =
runtime_context.get_list_state(list_state_descriptor)
2022-02-07T03:46:14.7094883Z Feb 07 03:46:14
map_state_descriptor = MapStateDescriptor('map_state', Types.INT(),
Types.STRING())
2022-02-07T03:46:14.7095550Z Feb 07 03:46:14 state_ttl_config =
StateTtlConfig \
2022-02-07T03:46:14.7095960Z Feb 07 03:46:14
.new_builder(Time.seconds(1)) \
2022-02-07T03:46:14.7096643Z Feb 07 03:46:14
.set_update_type(StateTtlConfig.UpdateType.OnReadAndWrite) \
2022-02-07T03:46:14.7097104Z Feb 07 03:46:14
.set_state_visibility(
2022-02-07T03:46:14.7097563Z Feb 07 03:46:14
StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp) \
2022-02-07T03:46:14.7098044Z Feb 07 03:46:14
.disable_cleanup_in_background() \
2022-02-07T03:46:14.7098425Z Feb 07 03:46:14 .build()
2022-02-07T03:46:14.7098835Z Feb 07 03:46:14
map_state_descriptor.enable_time_to_live(state_ttl_config)
2022-02-07T03:46:14.7099362Z Feb 07 03:46:14 self.map_state =
runtime_context.get_map_state(map_state_descriptor)
2022-02-07T03:46:14.7099787Z Feb 07 03:46:14
[jira] [Commented] (FLINK-25281) StreamingModeDataStreamTests failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487913#comment-17487913
]
Yun Gao commented on FLINK-25281:
-
I'll open a new issue for test_keyed_process_function_with_state~
> StreamingModeDataStreamTests failed on azure
>
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
> === FAILURES
> ===
> 2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
> StreamExecutionEnvironmentTests.test_set_stream_env __
> 2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
> testMethod=test_set_stream_env>
> 2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0907483Z Dec 13 02:25:17
> @unittest.skipIf(on_windows(), "Symbolic link is not supported on Windows,
> skipping.")
> 2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def
> test_set_stream_env(self):
> 2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
> 2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
> sys.executable
> 2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
> 2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
> 2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
> os.path.join(tmp_dir, "py_exec")
> 2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
> python_exec_link_path)
> 2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
> env.set_python_executable(python_exec_link_path)
> 2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
> 2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
> 2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
> os.environ["python"] == python_exec_link_path
> 2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
> 2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
> env.from_collection([1, 2, 3, 4, 5])
> 2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
> ds.map(check_python_exec).add_sink(self.test_sink)
> 2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
> python executable")
> 2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
> pyflink/datastream/tests/test_stream_execution_environment.py:546:
> 2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
> pyflink/datastream/stream_execution_environment.py:691: in execute
> 2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
> JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
> 2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
> .tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
> __call__
> 2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
> self.target_id, self.name)
> 2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146:
> in deco
> 2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
> 2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
> 2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
>
> 2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
> 'execute'
> 2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
> gateway_client, target_id=None, name=None):
> 2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
> received from the Java gateway into a Python object.
> 2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
> representation of integers are converted to Python
> 2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
> representation of objects are converted to JavaObject
> 2021-12-13T02:25:17.0933779Z Dec 13 02:25:17 instances, etc.
> 2021-12-13T02:25:17.0934191Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0934809Z Dec 13 02:25:17 :param answer: the
> string
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Priority: Critical (was: Major)
> StreamingModeDataStreamTests failed on azure
>
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
> === FAILURES
> ===
> 2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
> StreamExecutionEnvironmentTests.test_set_stream_env __
> 2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
> testMethod=test_set_stream_env>
> 2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0907483Z Dec 13 02:25:17
> @unittest.skipIf(on_windows(), "Symbolic link is not supported on Windows,
> skipping.")
> 2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def
> test_set_stream_env(self):
> 2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
> 2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
> sys.executable
> 2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
> 2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
> 2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
> os.path.join(tmp_dir, "py_exec")
> 2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
> python_exec_link_path)
> 2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
> env.set_python_executable(python_exec_link_path)
> 2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
> 2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
> 2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
> os.environ["python"] == python_exec_link_path
> 2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
> 2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
> env.from_collection([1, 2, 3, 4, 5])
> 2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
> ds.map(check_python_exec).add_sink(self.test_sink)
> 2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
> python executable")
> 2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
> pyflink/datastream/tests/test_stream_execution_environment.py:546:
> 2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
> pyflink/datastream/stream_execution_environment.py:691: in execute
> 2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
> JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
> 2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
> .tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
> __call__
> 2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
> self.target_id, self.name)
> 2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146:
> in deco
> 2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
> 2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
> 2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
>
> 2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
> 'execute'
> 2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
> gateway_client, target_id=None, name=None):
> 2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
> received from the Java gateway into a Python object.
> 2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
> representation of integers are converted to Python
> 2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
> representation of objects are converted to JavaObject
> 2021-12-13T02:25:17.0933779Z Dec 13 02:25:17 instances, etc.
> 2021-12-13T02:25:17.0934191Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0934809Z Dec 13 02:25:17 :param answer: the
> string returned by the Java gateway
> 2021-12-13T02:25:17.0935350Z Dec 13 02:25:17
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests#test_set_stream_env failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Summary: StreamingModeDataStreamTests#test_set_stream_env failed on azure
(was: StreamingModeDataStreamTests failed on azure)
> StreamingModeDataStreamTests#test_set_stream_env failed on azure
>
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> 2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
> === FAILURES
> ===
> 2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
> StreamExecutionEnvironmentTests.test_set_stream_env __
> 2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
> testMethod=test_set_stream_env>
> 2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0907483Z Dec 13 02:25:17
> @unittest.skipIf(on_windows(), "Symbolic link is not supported on Windows,
> skipping.")
> 2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def
> test_set_stream_env(self):
> 2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
> 2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
> sys.executable
> 2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
> 2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
> 2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
> os.path.join(tmp_dir, "py_exec")
> 2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
> python_exec_link_path)
> 2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
> env.set_python_executable(python_exec_link_path)
> 2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
> 2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
> 2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
> os.environ["python"] == python_exec_link_path
> 2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
> 2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
> env.from_collection([1, 2, 3, 4, 5])
> 2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
> ds.map(check_python_exec).add_sink(self.test_sink)
> 2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
> python executable")
> 2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
> pyflink/datastream/tests/test_stream_execution_environment.py:546:
> 2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
> pyflink/datastream/stream_execution_environment.py:691: in execute
> 2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
> JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
> 2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
> .tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
> __call__
> 2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
> self.target_id, self.name)
> 2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146:
> in deco
> 2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
> 2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
> 2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
>
> 2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
> 'execute'
> 2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
> gateway_client, target_id=None, name=None):
> 2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
> received from the Java gateway into a Python object.
> 2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
> 2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
> representation of integers are converted to Python
> 2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
> representation of objects are converted to JavaObject
> 2021-12-13T02:25:17.0933779Z Dec 13 02:25:17 instances, etc.
> 2021-12-13T02:25:17.0934191Z Dec 13 02:25:17
>
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Description:
{code:java}
2021-12-13T02:25:17.0905034Z Dec 13 02:25:17
=== FAILURES ===
2021-12-13T02:25:17.0905626Z Dec 13 02:25:17 _
StreamExecutionEnvironmentTests.test_set_stream_env __
2021-12-13T02:25:17.0906084Z Dec 13 02:25:17
2021-12-13T02:25:17.0906569Z Dec 13 02:25:17 self =
2021-12-13T02:25:17.0907047Z Dec 13 02:25:17
2021-12-13T02:25:17.0907483Z Dec 13 02:25:17 @unittest.skipIf(on_windows(),
"Symbolic link is not supported on Windows, skipping.")
2021-12-13T02:25:17.0909942Z Dec 13 02:25:17 def test_set_stream_env(self):
2021-12-13T02:25:17.0910424Z Dec 13 02:25:17 import sys
2021-12-13T02:25:17.0910816Z Dec 13 02:25:17 python_exec =
sys.executable
2021-12-13T02:25:17.0911369Z Dec 13 02:25:17 tmp_dir = self.tempdir
2021-12-13T02:25:17.0911736Z Dec 13 02:25:17 env = self.env
2021-12-13T02:25:17.0912153Z Dec 13 02:25:17 python_exec_link_path =
os.path.join(tmp_dir, "py_exec")
2021-12-13T02:25:17.0912876Z Dec 13 02:25:17 os.symlink(python_exec,
python_exec_link_path)
2021-12-13T02:25:17.0913342Z Dec 13 02:25:17
env.set_python_executable(python_exec_link_path)
2021-12-13T02:25:17.0913799Z Dec 13 02:25:17
2021-12-13T02:25:17.0914365Z Dec 13 02:25:17 def check_python_exec(i):
2021-12-13T02:25:17.0914944Z Dec 13 02:25:17 import os
2021-12-13T02:25:17.0915541Z Dec 13 02:25:17 assert
os.environ["python"] == python_exec_link_path
2021-12-13T02:25:17.0917263Z Dec 13 02:25:17 return i
2021-12-13T02:25:17.0917659Z Dec 13 02:25:17
2021-12-13T02:25:17.0918488Z Dec 13 02:25:17 ds =
env.from_collection([1, 2, 3, 4, 5])
2021-12-13T02:25:17.0919021Z Dec 13 02:25:17
ds.map(check_python_exec).add_sink(self.test_sink)
2021-12-13T02:25:17.0919717Z Dec 13 02:25:17 > env.execute("test set
python executable")
2021-12-13T02:25:17.0920265Z Dec 13 02:25:17
2021-12-13T02:25:17.0920707Z Dec 13 02:25:17
pyflink/datastream/tests/test_stream_execution_environment.py:546:
2021-12-13T02:25:17.0921533Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2021-12-13T02:25:17.0922325Z Dec 13 02:25:17
pyflink/datastream/stream_execution_environment.py:691: in execute
2021-12-13T02:25:17.0922897Z Dec 13 02:25:17 return
JobExecutionResult(self._j_stream_execution_environment.execute(j_stream_graph))
2021-12-13T02:25:17.0924157Z Dec 13 02:25:17
.tox/py37-cython/lib/python3.7/site-packages/py4j/java_gateway.py:1286: in
__call__
2021-12-13T02:25:17.0924680Z Dec 13 02:25:17 answer, self.gateway_client,
self.target_id, self.name)
2021-12-13T02:25:17.0925131Z Dec 13 02:25:17 pyflink/util/exceptions.py:146: in
deco
2021-12-13T02:25:17.0925615Z Dec 13 02:25:17 return f(*a, **kw)
2021-12-13T02:25:17.0926311Z Dec 13 02:25:17 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2021-12-13T02:25:17.0926793Z Dec 13 02:25:17
2021-12-13T02:25:17.0927550Z Dec 13 02:25:17 answer = 'xro13244'
2021-12-13T02:25:17.0928239Z Dec 13 02:25:17 gateway_client =
2021-12-13T02:25:17.0929456Z Dec 13 02:25:17 target_id = 'o13195', name =
'execute'
2021-12-13T02:25:17.0929975Z Dec 13 02:25:17
2021-12-13T02:25:17.0930616Z Dec 13 02:25:17 def get_return_value(answer,
gateway_client, target_id=None, name=None):
2021-12-13T02:25:17.0931506Z Dec 13 02:25:17 """Converts an answer
received from the Java gateway into a Python object.
2021-12-13T02:25:17.0931993Z Dec 13 02:25:17
2021-12-13T02:25:17.0932493Z Dec 13 02:25:17 For example, string
representation of integers are converted to Python
2021-12-13T02:25:17.0933249Z Dec 13 02:25:17 integer, string
representation of objects are converted to JavaObject
2021-12-13T02:25:17.0933779Z Dec 13 02:25:17 instances, etc.
2021-12-13T02:25:17.0934191Z Dec 13 02:25:17
2021-12-13T02:25:17.0934809Z Dec 13 02:25:17 :param answer: the string
returned by the Java gateway
2021-12-13T02:25:17.0935350Z Dec 13 02:25:17 :param gateway_client: the
gateway client used to communicate with the Java
2021-12-13T02:25:17.0935983Z Dec 13 02:25:17 Gateway. Only
necessary if the answer is a reference (e.g., object,
2021-12-13T02:25:17.0936593Z Dec 13 02:25:17 list, map)
2021-12-13T02:25:17.0937254Z Dec 13 02:25:17 :param target_id: the name
of the object from which the answer comes from
2021-12-13T02:25:17.0937783Z Dec 13 02:25:17 (e.g., *object1* in
`object1.hello()`). Optional.
2021-12-13T02:25:17.0938649Z Dec 13 02:25:17 :param name: the name of
the member from which the answer comes from
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Summary: StreamingModeDataStreamTests failed on azure (was:
StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on
azure)
> StreamingModeDataStreamTests failed on azure
>
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 13 03:03:08 pip_test_code.py success!
> Dec 13 03:03:09 ___ summary
>
> Dec 13 03:03:09 py36-cython: commands succeeded
> Dec 13 03:03:09 ERROR: py37-cython: commands failed
> Dec 13 03:03:09 py38-cython: commands succeeded
> Dec 13 03:03:09 tox checks... [FAILED]
> Dec 13 03:03:09 Process exited with EXIT CODE: 1.
> Dec 13 03:03:09 Trying to KILL watchdog (2760).
> /__w/1/s/tools/ci/watchdog.sh: line 100: 2760 Terminated
> watchdog
> Dec 13 03:03:09 Searching for .dump, .dumpstream and related files in
> '/__w/1/s'
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/bin/bash'. This may indicate a child process inherited the STDIO
> streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Test - python
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28010=logs=bdd9ea51-4de2-506a-d4d9-f3930e4d2355=dd50312f-73b5-56b5-c172-4d81d03e2ef1=24236
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Summary:
StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on
azure (was: Azure failed due to python tests "tox check failed")
> StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on
> azure
> ---
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Build System / Azure Pipelines
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 13 03:03:08 pip_test_code.py success!
> Dec 13 03:03:09 ___ summary
>
> Dec 13 03:03:09 py36-cython: commands succeeded
> Dec 13 03:03:09 ERROR: py37-cython: commands failed
> Dec 13 03:03:09 py38-cython: commands succeeded
> Dec 13 03:03:09 tox checks... [FAILED]
> Dec 13 03:03:09 Process exited with EXIT CODE: 1.
> Dec 13 03:03:09 Trying to KILL watchdog (2760).
> /__w/1/s/tools/ci/watchdog.sh: line 100: 2760 Terminated
> watchdog
> Dec 13 03:03:09 Searching for .dump, .dumpstream and related files in
> '/__w/1/s'
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/bin/bash'. This may indicate a child process inherited the STDIO
> streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Test - python
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28010=logs=bdd9ea51-4de2-506a-d4d9-f3930e4d2355=dd50312f-73b5-56b5-c172-4d81d03e2ef1=24236
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25281) StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25281:
Component/s: (was: Build System / Azure Pipelines)
> StreamingModeDataStreamTests#test_keyed_process_function_with_state failed on
> azure
> ---
>
> Key: FLINK-25281
> URL: https://issues.apache.org/jira/browse/FLINK-25281
> Project: Flink
> Issue Type: Bug
> Components: API / Python
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 13 03:03:08 pip_test_code.py success!
> Dec 13 03:03:09 ___ summary
>
> Dec 13 03:03:09 py36-cython: commands succeeded
> Dec 13 03:03:09 ERROR: py37-cython: commands failed
> Dec 13 03:03:09 py38-cython: commands succeeded
> Dec 13 03:03:09 tox checks... [FAILED]
> Dec 13 03:03:09 Process exited with EXIT CODE: 1.
> Dec 13 03:03:09 Trying to KILL watchdog (2760).
> /__w/1/s/tools/ci/watchdog.sh: line 100: 2760 Terminated
> watchdog
> Dec 13 03:03:09 Searching for .dump, .dumpstream and related files in
> '/__w/1/s'
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/bin/bash'. This may indicate a child process inherited the STDIO
> streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Test - python
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28010=logs=bdd9ea51-4de2-506a-d4d9-f3930e4d2355=dd50312f-73b5-56b5-c172-4d81d03e2ef1=24236
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25937) SQL Client end-to-end test e2e fails on AZP
[
https://issues.apache.org/jira/browse/FLINK-25937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487909#comment-17487909
]
Yun Gao commented on FLINK-25937:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30817=logs=fb37c667-81b7-5c22-dd91-846535e99a97=39a035c3-c65e-573c-fb66-104c66c28912=5704
> SQL Client end-to-end test e2e fails on AZP
> ---
>
> Key: FLINK-25937
> URL: https://issues.apache.org/jira/browse/FLINK-25937
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: test-stability
>
> The {{SQL Client end-to-end test}} e2e tests fails on AZP when using the
> {{AdaptiveScheduler}} because the scheduler expects that the parallelism is
> set for all vertices:
> {code}
> Feb 03 03:45:13 org.apache.flink.runtime.client.JobInitializationException:
> Could not start the JobMaster.
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1609)
> Feb 03 03:45:13 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> Feb 03 03:45:13 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> Feb 03 03:45:13 at java.lang.Thread.run(Thread.java:748)
> Feb 03 03:45:13 Caused by: java.util.concurrent.CompletionException:
> java.lang.IllegalStateException: The adaptive scheduler expects the
> parallelism being set for each JobVertex (violated JobVertex:
> f74b775b58627a33e46b8c155b320255).
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
> Feb 03 03:45:13 ... 3 more
> Feb 03 03:45:13 Caused by: java.lang.IllegalStateException: The adaptive
> scheduler expects the parallelism being set for each JobVertex (violated
> JobVertex: f74b775b58627a33e46b8c155b320255).
> Feb 03 03:45:13 at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.assertPreconditions(AdaptiveScheduler.java:296)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.(AdaptiveScheduler.java:230)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerFactory.createInstance(AdaptiveSchedulerFactory.java:122)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:115)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:345)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:322)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:106)
> Feb 03 03:45:13 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:94)
> Feb 03 03:45:13 at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
> Feb 03 03:45:13 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> Feb 03 03:45:13 ... 3 more
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30662=logs=fb37c667-81b7-5c22-dd91-846535e99a97=39a035c3-c65e-573c-fb66-104c66c28912=5782
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18220: [FLINK-25410] Flink CLI should exit when app is accepted with detach …
flinkbot edited a comment on pull request #18220: URL: https://github.com/apache/flink/pull/18220#issuecomment-1001978981 ## CI report: * 8dc3a423728c27a335cd6f2f393ef79fb801e9c7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30820) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18640: [FLINK-24880][python] Fix PeriodicThread to handle properly for negative wait timeout value
flinkbot edited a comment on pull request #18640: URL: https://github.com/apache/flink/pull/18640#issuecomment-1031032425 ## CI report: * ab5350d7c1b61aea750c3157deca0132ff506082 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30821) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s e2e tests
flinkbot edited a comment on pull request #18639: URL: https://github.com/apache/flink/pull/18639#issuecomment-1031013661 ## CI report: * 6273e9a4cfb0b0cf14d95926fb6c7748d82ad815 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30819) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25966) KafkaSourceITCase failed on azure due to Create test topic : topic1 failed
[
https://issues.apache.org/jira/browse/FLINK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25966:
Labels: test-stability (was: )
> KafkaSourceITCase failed on azure due to Create test topic : topic1 failed
> --
>
> Key: FLINK-25966
> URL: https://issues.apache.org/jira/browse/FLINK-25966
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.14.3
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Feb 07 00:50:05 [ERROR]
> org.apache.flink.connector.kafka.source.KafkaSourceITCase Time elapsed:
> 136.316 s <<< FAILURE!
> Feb 07 00:50:05 java.lang.AssertionError: Create test topic : topic1 failed,
> org.apache.kafka.common.errors.TimeoutException: Aborted due to timeout.
> Feb 07 00:50:05 at org.junit.Assert.fail(Assert.java:88)
> Feb 07 00:50:05 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:226)
> Feb 07 00:50:05 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:112)
> Feb 07 00:50:05 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:221)
> Feb 07 00:50:05 at
> org.apache.flink.connector.kafka.source.KafkaSourceTestEnv.createTestTopic(KafkaSourceTestEnv.java:187)
> Feb 07 00:50:05 at
> org.apache.flink.connector.kafka.source.KafkaSourceTestEnv.setupTopic(KafkaSourceTestEnv.java:224)
> Feb 07 00:50:05 at
> org.apache.flink.connector.kafka.source.KafkaSourceITCase.setup(KafkaSourceITCase.java:67)
> Feb 07 00:50:05 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Feb 07 00:50:05 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Feb 07 00:50:05 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Feb 07 00:50:05 at java.lang.reflect.Method.invoke(Method.java:498)
> Feb 07 00:50:05 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Feb 07 00:50:05 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Feb 07 00:50:05 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Feb 07 00:50:05 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> Feb 07 00:50:05 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Feb 07 00:50:05 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> Feb 07 00:50:05 at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30816=logs=4be4ed2b-549a-533d-aa33-09e28e360cc8=0db94045-2aa0-53fa-f444-0130d6933518=7426
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-25966) KafkaSourceITCase failed on azure due to Create test topic : topic1 failed
Yun Gao created FLINK-25966:
---
Summary: KafkaSourceITCase failed on azure due to Create test
topic : topic1 failed
Key: FLINK-25966
URL: https://issues.apache.org/jira/browse/FLINK-25966
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.14.3
Reporter: Yun Gao
{code:java}
Feb 07 00:50:05 [ERROR]
org.apache.flink.connector.kafka.source.KafkaSourceITCase Time elapsed:
136.316 s <<< FAILURE!
Feb 07 00:50:05 java.lang.AssertionError: Create test topic : topic1 failed,
org.apache.kafka.common.errors.TimeoutException: Aborted due to timeout.
Feb 07 00:50:05 at org.junit.Assert.fail(Assert.java:88)
Feb 07 00:50:05 at
org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:226)
Feb 07 00:50:05 at
org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:112)
Feb 07 00:50:05 at
org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:221)
Feb 07 00:50:05 at
org.apache.flink.connector.kafka.source.KafkaSourceTestEnv.createTestTopic(KafkaSourceTestEnv.java:187)
Feb 07 00:50:05 at
org.apache.flink.connector.kafka.source.KafkaSourceTestEnv.setupTopic(KafkaSourceTestEnv.java:224)
Feb 07 00:50:05 at
org.apache.flink.connector.kafka.source.KafkaSourceITCase.setup(KafkaSourceITCase.java:67)
Feb 07 00:50:05 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
Feb 07 00:50:05 at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Feb 07 00:50:05 at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Feb 07 00:50:05 at java.lang.reflect.Method.invoke(Method.java:498)
Feb 07 00:50:05 at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
Feb 07 00:50:05 at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Feb 07 00:50:05 at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
Feb 07 00:50:05 at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
Feb 07 00:50:05 at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
Feb 07 00:50:05 at
org.junit.runners.ParentRunner.run(ParentRunner.java:363)
Feb 07 00:50:05 at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
Feb 07 00:50:05 at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
Feb 07 00:50:05 at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
Feb 07 00:50:05 at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
Feb 07 00:50:05 at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
Feb 07 00:50:05 at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
Feb 07 00:50:05 at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
Feb 07 00:50:05 at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30816=logs=4be4ed2b-549a-533d-aa33-09e28e360cc8=0db94045-2aa0-53fa-f444-0130d6933518=7426
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-20830) Add a type of HEADLESS_CLUSTER_IP for rest service type
[ https://issues.apache.org/jira/browse/FLINK-20830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang updated FLINK-20830: -- Fix Version/s: 1.15.0 > Add a type of HEADLESS_CLUSTER_IP for rest service type > --- > > Key: FLINK-20830 > URL: https://issues.apache.org/jira/browse/FLINK-20830 > Project: Flink > Issue Type: Improvement > Components: Deployment / Kubernetes >Reporter: Aitozi >Assignee: Aitozi >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > pull-request-available > Fix For: 1.15.0 > > > Now we can choose ClusterIP or NodePort or LoadBalancer as rest service type. > But in our internal kubernetes cluster, there is no kube-proxy, and ClusterIP > mode rely on kube-proxy. So I think can we support another type of > HEADLESS_CLUSTER_IP to directly talk to jobmanager pod? cc [~fly_in_gis] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (FLINK-20830) Add a type of HEADLESS_CLUSTER_IP for rest service type
[ https://issues.apache.org/jira/browse/FLINK-20830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Wang closed FLINK-20830. - Resolution: Fixed Fixed via: master(release-1.15): 1ae09ae6942eaaf5a6197c68fc12ee4e9fc1a105 > Add a type of HEADLESS_CLUSTER_IP for rest service type > --- > > Key: FLINK-20830 > URL: https://issues.apache.org/jira/browse/FLINK-20830 > Project: Flink > Issue Type: Improvement > Components: Deployment / Kubernetes >Reporter: Aitozi >Assignee: Aitozi >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > pull-request-available > > Now we can choose ClusterIP or NodePort or LoadBalancer as rest service type. > But in our internal kubernetes cluster, there is no kube-proxy, and ClusterIP > mode rely on kube-proxy. So I think can we support another type of > HEADLESS_CLUSTER_IP to directly talk to jobmanager pod? cc [~fly_in_gis] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25449) KafkaSourceITCase.testRedundantParallelism failed on azure
[
https://issues.apache.org/jira/browse/FLINK-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25449:
Description:
{code:java}
Dec 25 00:51:07 Caused by: java.lang.RuntimeException: One or more fetchers
have encountered exception
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:223)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:154)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:116)
Dec 25 00:51:07 at
org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:305)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:69)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:423)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:204)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:684)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:639)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:650)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:623)
Dec 25 00:51:07 at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779)
Dec 25 00:51:07 at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
Dec 25 00:51:07 at java.lang.Thread.run(Thread.java:748)
Dec 25 00:51:07 Caused by: java.lang.RuntimeException: SplitFetcher thread 0
received unexpected exception while polling the records
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:148)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:103)
Dec 25 00:51:07 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
Dec 25 00:51:07 at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
Dec 25 00:51:07 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Dec 25 00:51:07 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Dec 25 00:51:07 ... 1 more
Dec 25 00:51:07 Caused by: java.lang.IllegalStateException: Consumer is not
subscribed to any topics or assigned any partitions
Dec 25 00:51:07 at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1223)
Dec 25 00:51:07 at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1211)
Dec 25 00:51:07 at
org.apache.flink.connector.kafka.source.reader.KafkaPartitionSplitReader.fetch(KafkaPartitionSplitReader.java:108)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:56)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:140)
Dec 25 00:51:07 ... 6 more
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28589=logs=1fc6e7bf-633c-5081-c32a-9dea24b05730=80a658d1-f7f6-5d93-2758-53ac19fd5b19=6612
was:
{code:java}
Dec 25 00:51:07 at akka.dispatch.Mailbox.run(Mailbox.scala:225)
Dec 25 00:51:07 at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
Dec 25 00:51:07 ... 4 more
Dec 25 00:51:07 Caused by: java.lang.RuntimeException: One or more fetchers
have encountered exception
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:223)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:154)
Dec 25 00:51:07 at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:116)
Dec 25 00:51:07 at
org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:305)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:69)
Dec 25 00:51:07 at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66)
Dec 25 00:51:07 at
[GitHub] [flink] wangyang0918 merged pull request #18461: [FLINK-20830][Kubernetes]Add type of Headless_Cluster_IP for external…
wangyang0918 merged pull request #18461: URL: https://github.com/apache/flink/pull/18461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-23944) PulsarSourceITCase.testTaskManagerFailure is instable
[
https://issues.apache.org/jira/browse/FLINK-23944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487891#comment-17487891
]
Yun Gao edited comment on FLINK-23944 at 2/7/22, 6:28 AM:
--
1.14:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30815=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=494f6362-8ffa-5ff8-9158-c7f00e541279=27847
was (Author: gaoyunhaii):
1.13:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30815=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=494f6362-8ffa-5ff8-9158-c7f00e541279=27847
> PulsarSourceITCase.testTaskManagerFailure is instable
> -
>
> Key: FLINK-23944
> URL: https://issues.apache.org/jira/browse/FLINK-23944
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
>Affects Versions: 1.14.0
>Reporter: Dian Fu
>Assignee: Yufei Zhang
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> [https://dev.azure.com/dianfu/Flink/_build/results?buildId=430=logs=f3dc9b18-b77a-55c1-591e-264c46fe44d1=2d3cd81e-1c37-5c31-0ee4-f5d5cdb9324d]
> It's from my personal azure pipeline, however, I'm pretty sure that I have
> not touched any code related to this.
> {code:java}
> Aug 24 10:44:13 [ERROR] testTaskManagerFailure{TestEnvironment,
> ExternalContext, ClusterControllable}[1] Time elapsed: 258.397 s <<< FAILURE!
> Aug 24 10:44:13 java.lang.AssertionError: Aug 24 10:44:13 Aug 24 10:44:13
> Expected: Records consumed by Flink should be identical to test data and
> preserve the order in split Aug 24 10:44:13 but: Mismatched record at
> position 7: Expected '0W6SzacX7MNL4xLL3BZ8C3ljho4iCydbvxIl' but was
> 'wVi5JaJpNvgkDEOBRC775qHgw0LyRW2HBxwLmfONeEmr' Aug 24 10:44:13 at
> org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) Aug 24 10:44:13
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8) Aug 24
> 10:44:13 at
> org.apache.flink.connectors.test.common.testsuites.SourceTestSuiteBase.testTaskManagerFailure(SourceTestSuiteBase.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-23944) PulsarSourceITCase.testTaskManagerFailure is instable
[
https://issues.apache.org/jira/browse/FLINK-23944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487891#comment-17487891
]
Yun Gao commented on FLINK-23944:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30815=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=494f6362-8ffa-5ff8-9158-c7f00e541279=27847
> PulsarSourceITCase.testTaskManagerFailure is instable
> -
>
> Key: FLINK-23944
> URL: https://issues.apache.org/jira/browse/FLINK-23944
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
>Affects Versions: 1.14.0
>Reporter: Dian Fu
>Assignee: Yufei Zhang
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> [https://dev.azure.com/dianfu/Flink/_build/results?buildId=430=logs=f3dc9b18-b77a-55c1-591e-264c46fe44d1=2d3cd81e-1c37-5c31-0ee4-f5d5cdb9324d]
> It's from my personal azure pipeline, however, I'm pretty sure that I have
> not touched any code related to this.
> {code:java}
> Aug 24 10:44:13 [ERROR] testTaskManagerFailure{TestEnvironment,
> ExternalContext, ClusterControllable}[1] Time elapsed: 258.397 s <<< FAILURE!
> Aug 24 10:44:13 java.lang.AssertionError: Aug 24 10:44:13 Aug 24 10:44:13
> Expected: Records consumed by Flink should be identical to test data and
> preserve the order in split Aug 24 10:44:13 but: Mismatched record at
> position 7: Expected '0W6SzacX7MNL4xLL3BZ8C3ljho4iCydbvxIl' but was
> 'wVi5JaJpNvgkDEOBRC775qHgw0LyRW2HBxwLmfONeEmr' Aug 24 10:44:13 at
> org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) Aug 24 10:44:13
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8) Aug 24
> 10:44:13 at
> org.apache.flink.connectors.test.common.testsuites.SourceTestSuiteBase.testTaskManagerFailure(SourceTestSuiteBase.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-23944) PulsarSourceITCase.testTaskManagerFailure is instable
[
https://issues.apache.org/jira/browse/FLINK-23944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487891#comment-17487891
]
Yun Gao edited comment on FLINK-23944 at 2/7/22, 6:27 AM:
--
1.13:
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30815=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=494f6362-8ffa-5ff8-9158-c7f00e541279=27847
was (Author: gaoyunhaii):
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30815=logs=8fd975ef-f478-511d-4997-6f15fe8a1fd3=494f6362-8ffa-5ff8-9158-c7f00e541279=27847
> PulsarSourceITCase.testTaskManagerFailure is instable
> -
>
> Key: FLINK-23944
> URL: https://issues.apache.org/jira/browse/FLINK-23944
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
>Affects Versions: 1.14.0
>Reporter: Dian Fu
>Assignee: Yufei Zhang
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0
>
>
> [https://dev.azure.com/dianfu/Flink/_build/results?buildId=430=logs=f3dc9b18-b77a-55c1-591e-264c46fe44d1=2d3cd81e-1c37-5c31-0ee4-f5d5cdb9324d]
> It's from my personal azure pipeline, however, I'm pretty sure that I have
> not touched any code related to this.
> {code:java}
> Aug 24 10:44:13 [ERROR] testTaskManagerFailure{TestEnvironment,
> ExternalContext, ClusterControllable}[1] Time elapsed: 258.397 s <<< FAILURE!
> Aug 24 10:44:13 java.lang.AssertionError: Aug 24 10:44:13 Aug 24 10:44:13
> Expected: Records consumed by Flink should be identical to test data and
> preserve the order in split Aug 24 10:44:13 but: Mismatched record at
> position 7: Expected '0W6SzacX7MNL4xLL3BZ8C3ljho4iCydbvxIl' but was
> 'wVi5JaJpNvgkDEOBRC775qHgw0LyRW2HBxwLmfONeEmr' Aug 24 10:44:13 at
> org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) Aug 24 10:44:13
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8) Aug 24
> 10:44:13 at
> org.apache.flink.connectors.test.common.testsuites.SourceTestSuiteBase.testTaskManagerFailure(SourceTestSuiteBase.java:271)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18461: [FLINK-20830][Kubernetes]Add type of Headless_Cluster_IP for external…
flinkbot edited a comment on pull request #18461: URL: https://github.com/apache/flink/pull/18461#issuecomment-1019901761 ## CI report: * 5935b3b166f9fd6ec43f4428e804b7a54a436a8c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30818) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * 4ef0a8253f4104c21a6ec06a8b9a6894526db471 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29773) * ad74b6cfec708adbca9a115f5692022aedf46915 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30823) * 78c2513ffe72a83ba689789fd5c14dbda4d93c99 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30824) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * 4ef0a8253f4104c21a6ec06a8b9a6894526db471 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29773) * ad74b6cfec708adbca9a115f5692022aedf46915 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30823) * 78c2513ffe72a83ba689789fd5c14dbda4d93c99 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-25825) MySqlCatalogITCase fails on azure
[
https://issues.apache.org/jira/browse/FLINK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487871#comment-17487871
]
RocMarshal edited comment on FLINK-25825 at 2/7/22, 5:08 AM:
-
Sorry for not checking the issue in time. [~MartijnVisser]
I looked over some examples of error reporting, I found this phenomenon
accidental.
And in these cases, the output of all test data meets the expectation. Are
there something related with the test framework or test-container?
was (Author: rocmarshal):
Sorry for not checking the issue in time. [~MartijnVisser]
I looked over some examples of error reporting, I found this phenomenon
accidental.
And in these cases, the output of all test data meets the expectation. Are
there have something related with the test framework or test-container?
> MySqlCatalogITCase fails on azure
> -
>
> Key: FLINK-25825
> URL: https://issues.apache.org/jira/browse/FLINK-25825
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30189=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=13677
>
> {code}
> 2022-01-26T06:04:42.8019913Z Jan 26 06:04:42 [ERROR]
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath Time
> elapsed: 2.166 *s <<< FAILURE!
> 2022-01-26T06:04:42.8025522Z Jan 26 06:04:42 java.lang.AssertionError:
> expected: java.util.ArrayList<[+I[1, -1, 1, null, true, null, hello, 2021-0
> 8-04, 2021-08-04T01:54:16, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1,
> \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99, 9 9,
> -1.0, 1.0, set_ele1, -1, 1, col_text, 10:32:34, 2021-08-04T01:54:16,
> col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:54:16.463, 09:33:43,
> 2021-08-04T01:54:16.463, null], +I[2, -1, 1, null, true, null, hello,
> 2021-08-04, 2021-08-04T01:53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1,
> -1, 1, \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99,
> 99, -1.0, 1.0, set_ele1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-
> 04T01:53:19, col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:53:19.098,
> 09:33:43, 2021-08-04T01:53:19.098, null]]> but was: java.util.ArrayL
> ist<[+I[1, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:54:16,
> -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_ele1, -1,
> 1, col_text, 10:32:34, 2021-08-04T01:54:16, col_tinytext, -1, 1, null ,
> col_varchar, 2021-08-04T01:54:16.463, 09:33:43, 2021-08-04T01:54:16.463,
> null], +I[2, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:
> 53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_el
> e1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-04T01:53:19, col_tinytext,
> -1, 1, null, col_varchar, 2021-08-04T01:53:19.098, 09:33:43, 2021-08-0
> 4T01:53:19.098, null]]>
> 2022-01-26T06:04:42.8029336Z Jan 26 06:04:42 at
> org.junit.Assert.fail(Assert.java:89)
> 2022-01-26T06:04:42.8029824Z Jan 26 06:04:42 at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2022-01-26T06:04:42.8030319Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2022-01-26T06:04:42.8030815Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2022-01-26T06:04:42.8031419Z Jan 26 06:04:42 at
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath(MySqlCatalogITCase.java*:306)
> {code}
>
> {code}
> 2022-01-26T06:04:43.2899378Z Jan 26 06:04:43 [ERROR] Failures:
> 2022-01-26T06:04:43.2907942Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testFullPath:306 expected: java.util.ArrayList<[+I[1, -1,
> 1, null, true,
> 2022-01-26T06:04:43.2914065Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testGetTable:253 expected:<(
> 2022-01-26T06:04:43.2983567Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testSelectToInsert:323 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.2997373Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalog:291 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.3010450Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalogDB:278 expected:
> java.util.ArrayList<[+I[1, -1, 1, nul
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25825) MySqlCatalogITCase fails on azure
[
https://issues.apache.org/jira/browse/FLINK-25825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487871#comment-17487871
]
RocMarshal commented on FLINK-25825:
Sorry for not checking the issue in time. [~MartijnVisser]
I looked over some examples of error reporting, I found this phenomenon
accidental.
And in these cases, the output of all test data meets the expectation. Are
there have something related with the test framework or test-container?
> MySqlCatalogITCase fails on azure
> -
>
> Key: FLINK-25825
> URL: https://issues.apache.org/jira/browse/FLINK-25825
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.15.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Labels: test-stability
> Fix For: 1.15.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30189=logs=e9af9cde-9a65-5281-a58e-2c8511d36983=c520d2c3-4d17-51f1-813b-4b0b74a0c307=13677
>
> {code}
> 2022-01-26T06:04:42.8019913Z Jan 26 06:04:42 [ERROR]
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath Time
> elapsed: 2.166 *s <<< FAILURE!
> 2022-01-26T06:04:42.8025522Z Jan 26 06:04:42 java.lang.AssertionError:
> expected: java.util.ArrayList<[+I[1, -1, 1, null, true, null, hello, 2021-0
> 8-04, 2021-08-04T01:54:16, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1,
> \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99, 9 9,
> -1.0, 1.0, set_ele1, -1, 1, col_text, 10:32:34, 2021-08-04T01:54:16,
> col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:54:16.463, 09:33:43,
> 2021-08-04T01:54:16.463, null], +I[2, -1, 1, null, true, null, hello,
> 2021-08-04, 2021-08-04T01:53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1,
> -1, 1, \{"k1": "v1"}, null, col_longtext, null, -1, 1, col_mediumtext, -99,
> 99, -1.0, 1.0, set_ele1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-
> 04T01:53:19, col_tinytext, -1, 1, null, col_varchar, 2021-08-04T01:53:19.098,
> 09:33:43, 2021-08-04T01:53:19.098, null]]> but was: java.util.ArrayL
> ist<[+I[1, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:54:16,
> -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_ele1, -1,
> 1, col_text, 10:32:34, 2021-08-04T01:54:16, col_tinytext, -1, 1, null ,
> col_varchar, 2021-08-04T01:54:16.463, 09:33:43, 2021-08-04T01:54:16.463,
> null], +I[2, -1, 1, null, true, null, hello, 2021-08-04, 2021-08-04T01:
> 53:19, -1, 1, -1.0, 1.0, enum2, -9.1, 9.1, -1, 1, -1, 1, \{"k1": "v1"}, null,
> col_longtext, null, -1, 1, col_mediumtext, -99, 99, -1.0, 1.0, set_el
> e1,set_ele12, -1, 1, col_text, 10:32:34, 2021-08-04T01:53:19, col_tinytext,
> -1, 1, null, col_varchar, 2021-08-04T01:53:19.098, 09:33:43, 2021-08-0
> 4T01:53:19.098, null]]>
> 2022-01-26T06:04:42.8029336Z Jan 26 06:04:42 at
> org.junit.Assert.fail(Assert.java:89)
> 2022-01-26T06:04:42.8029824Z Jan 26 06:04:42 at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2022-01-26T06:04:42.8030319Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2022-01-26T06:04:42.8030815Z Jan 26 06:04:42 at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2022-01-26T06:04:42.8031419Z Jan 26 06:04:42 at
> org.apache.flink.connector.jdbc.catalog.MySqlCatalogITCase.testFullPath(MySqlCatalogITCase.java*:306)
> {code}
>
> {code}
> 2022-01-26T06:04:43.2899378Z Jan 26 06:04:43 [ERROR] Failures:
> 2022-01-26T06:04:43.2907942Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testFullPath:306 expected: java.util.ArrayList<[+I[1, -1,
> 1, null, true,
> 2022-01-26T06:04:43.2914065Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testGetTable:253 expected:<(
> 2022-01-26T06:04:43.2983567Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testSelectToInsert:323 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.2997373Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalog:291 expected:
> java.util.ArrayList<[+I[1, -1, 1, null,
> 2022-01-26T06:04:43.3010450Z Jan 26 06:04:43 [ERROR]
> MySqlCatalogITCase.testWithoutCatalogDB:278 expected:
> java.util.ArrayList<[+I[1, -1, 1, nul
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * 4ef0a8253f4104c21a6ec06a8b9a6894526db471 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29773) * ad74b6cfec708adbca9a115f5692022aedf46915 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30823) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18417: [FLINK-15455][network] Enabled tcp connection reuse across multi jobs.
flinkbot edited a comment on pull request #18417: URL: https://github.com/apache/flink/pull/18417#issuecomment-1017267306 ## CI report: * 4ef0a8253f4104c21a6ec06a8b9a6894526db471 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=29773) * ad74b6cfec708adbca9a115f5692022aedf46915 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18561: [FLINK-25874] loose PyFlink dependency of 'python-dateutil' to '>=2.8.0,<3'
flinkbot edited a comment on pull request #18561: URL: https://github.com/apache/flink/pull/18561#issuecomment-1025075470 ## CI report: * 8fd795009f123e2b799a50735a616231a5bbbc87 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30473) * 2b81551555d3b9bdfb1300bbd6fc2b718294956e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30822) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18561: [FLINK-25874] loose PyFlink dependency of 'python-dateutil' to '>=2.8.0,<3'
flinkbot edited a comment on pull request #18561: URL: https://github.com/apache/flink/pull/18561#issuecomment-1025075470 ## CI report: * 8fd795009f123e2b799a50735a616231a5bbbc87 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30473) * 2b81551555d3b9bdfb1300bbd6fc2b718294956e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] martin-liu commented on pull request #18561: [FLINK-25874] loose PyFlink dependency of 'python-dateutil' to '>=2.8.0,<3'
martin-liu commented on pull request #18561: URL: https://github.com/apache/flink/pull/18561#issuecomment-1031045547 @dianfu Good catch, accepted the suggestion, thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-25874) PyFlink package dependencies conflict
[
https://issues.apache.org/jira/browse/FLINK-25874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu reassigned FLINK-25874:
---
Assignee: Martin Liu
> PyFlink package dependencies conflict
> -
>
> Key: FLINK-25874
> URL: https://issues.apache.org/jira/browse/FLINK-25874
> Project: Flink
> Issue Type: Improvement
>Reporter: Martin Liu
>Assignee: Martin Liu
>Priority: Major
> Labels: pull-request-available
>
> I need to install PyFlink with some other libraries in a project, and I found
> there's dependency conflict with `great-expectations` because PyFlink has
> pined dependency `python-dateutil==2.8.0`.
> {code:java}
> There are incompatible versions in the resolved dependencies:
> python-dateutil==2.8.0 (from apache-flink==1.14.3->-r requirements.in (line
> 4))
> python-dateutil>=2.8.1 (from great-expectations==0.14.4->-r requirements.in
> (line 5)) {code}
> I have to use newer version of great-expectations (>=0.13.1) for some
> features, so this is blocking me.
> I found `python-dateutil` v2.8.0 is released in 2019-02-05, see
> [https://github.com/dateutil/dateutil/releases/tag/2.8.0], is there any way
> to loose the dependency, e.g. >=2.8.0 ?
> (BTW, `cloudpickle==1.2.2` is also old which released in 2019-09-10, see
> [https://github.com/cloudpipe/cloudpickle/releases/tag/v1.2.2] )
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-25874) PyFlink package dependencies conflict
[
https://issues.apache.org/jira/browse/FLINK-25874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu updated FLINK-25874:
Priority: Major (was: Blocker)
> PyFlink package dependencies conflict
> -
>
> Key: FLINK-25874
> URL: https://issues.apache.org/jira/browse/FLINK-25874
> Project: Flink
> Issue Type: Improvement
>Reporter: Martin Liu
>Priority: Major
> Labels: pull-request-available
>
> I need to install PyFlink with some other libraries in a project, and I found
> there's dependency conflict with `great-expectations` because PyFlink has
> pined dependency `python-dateutil==2.8.0`.
> {code:java}
> There are incompatible versions in the resolved dependencies:
> python-dateutil==2.8.0 (from apache-flink==1.14.3->-r requirements.in (line
> 4))
> python-dateutil>=2.8.1 (from great-expectations==0.14.4->-r requirements.in
> (line 5)) {code}
> I have to use newer version of great-expectations (>=0.13.1) for some
> features, so this is blocking me.
> I found `python-dateutil` v2.8.0 is released in 2019-02-05, see
> [https://github.com/dateutil/dateutil/releases/tag/2.8.0], is there any way
> to loose the dependency, e.g. >=2.8.0 ?
> (BTW, `cloudpickle==1.2.2` is also old which released in 2019-09-10, see
> [https://github.com/cloudpipe/cloudpickle/releases/tag/v1.2.2] )
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] dianfu commented on a change in pull request #18561: [FLINK-25874] loose PyFlink dependency of 'python-dateutil' to '>=2.8.0,<3'
dianfu commented on a change in pull request #18561: URL: https://github.com/apache/flink/pull/18561#discussion_r800297620 ## File path: flink-python/dev/dev-requirements.txt ## @@ -17,7 +17,7 @@ wheel apache-beam==2.27.0 cython==0.29.24 py4j==0.10.9.3 -python-dateutil==2.8.0 +python-dateutil==2.8.2 Review comment: Keep it consistent with setup.py? ```suggestion python-dateutil>=2.8.0,<3 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24038) DispatcherResourceManagerComponent fails to deregister application if no leading ResourceManager
[
https://issues.apache.org/jira/browse/FLINK-24038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487854#comment-17487854
]
Yang Wang commented on FLINK-24038:
---
I will keep an eye on the multiple component leader election ha services when
it is used in the production environment.
> DispatcherResourceManagerComponent fails to deregister application if no
> leading ResourceManager
>
>
> Key: FLINK-24038
> URL: https://issues.apache.org/jira/browse/FLINK-24038
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.14.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> With FLINK-21667 we introduced a change that can cause the
> {{DispatcherResourceManagerComponent}} to fail when trying to stop the
> application. The problem is that the {{DispatcherResourceManagerComponent}}
> needs a leading {{ResourceManager}} to successfully execute the
> stop/deregister application call. If this is not the case, then it will fail
> fatally. In the case of multiple standby JobManager processes it can happen
> that the leading {{ResourceManager}} runs somewhere else.
> I do see two possible solutions:
> 1. Run the leader election process for the whole JobManager process
> 2. Move the registration/deregistration of the application out of the
> {{ResourceManager}} so that it can be executed w/o a leader
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (FLINK-25893) ResourceManagerServiceImpl's lifecycle can lead to exceptions
[
https://issues.apache.org/jira/browse/FLINK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song reassigned FLINK-25893:
Assignee: Xintong Song
> ResourceManagerServiceImpl's lifecycle can lead to exceptions
> -
>
> Key: FLINK-25893
> URL: https://issues.apache.org/jira/browse/FLINK-25893
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0, 1.14.3
>Reporter: Till Rohrmann
>Assignee: Xintong Song
>Priority: Critical
> Labels: pull-request-available
>
> The {{ResourceManagerServiceImpl}} lifecycle can lead to exceptions when
> calling {{ResourceManagerServiceImpl.deregisterApplication}}. The problem
> arises when the {{DispatcherResourceManagerComponent}} is shutdown before the
> {{ResourceManagerServiceImpl}} gains leadership or while it is starting the
> {{ResourceManager}}.
> One problem is that {{deregisterApplication}} returns an exceptionally
> completed future if there is no leading {{ResourceManager}}.
> Another problem is that if there is a leading {{ResourceManager}}, then it
> can still be the case that it has not been started yet. If this is the case,
> then
> [ResourceManagerGateway.deregisterApplication|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerServiceImpl.java#L143]
> will be discarded. The reason for this behaviour is that we create a
> {{ResourceManager}} in one {{Runnable}} and only start it in another. Due to
> this there can be the {{deregisterApplication}} call that gets the {{lock}}
> in between.
> I'd suggest to correct the lifecycle and contract of the
> {{ResourceManagerServiceImpl.deregisterApplication}}.
> Please note that due to this problem, the error reporting of this method has
> been suppressed. See FLINK-25885 for more details.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25893) ResourceManagerServiceImpl's lifecycle can lead to exceptions
[
https://issues.apache.org/jira/browse/FLINK-25893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487853#comment-17487853
]
Xintong Song commented on FLINK-25893:
--
I also have the impression of a similar discussion, just couldn't remember whom
I discussed with. :P
I'm leaning towards to option 2).
I'm afraid your option 1) would not work.
- Any component that is responsible for the deregistering would need a leader
election. Otherwise, we may accidentally deregister the application from a
non-leading master process while there is another leading master process. Thus,
we still face the same problem that deregistering is called while no leader is
elected.
- It exposes the Kubernetes/Yarn client to either {{ClusterEntrypoint}} or
{{Dispatcher}}, which complicates the system.
For option 2), changing the contract means the process will exit with code 0
when there's no leading RM. That should not affect the native Kubernetes & Yarn
deployment (neither of them relies on the exit code for process restarting),
but will help the standalone Kubernetes deployment (which performs nothing in
deregistering and relies on the exit code for restarting).
If we have consensus, I can work on this ticket and make the following changes:
- Makes {{ResourceManagerGateway#deregisterApplication}} wait for the leading
RM being fully started.
- Change the contract to not deregister the application if there's no leading
RM in the process.
> ResourceManagerServiceImpl's lifecycle can lead to exceptions
> -
>
> Key: FLINK-25893
> URL: https://issues.apache.org/jira/browse/FLINK-25893
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0, 1.14.3
>Reporter: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available
>
> The {{ResourceManagerServiceImpl}} lifecycle can lead to exceptions when
> calling {{ResourceManagerServiceImpl.deregisterApplication}}. The problem
> arises when the {{DispatcherResourceManagerComponent}} is shutdown before the
> {{ResourceManagerServiceImpl}} gains leadership or while it is starting the
> {{ResourceManager}}.
> One problem is that {{deregisterApplication}} returns an exceptionally
> completed future if there is no leading {{ResourceManager}}.
> Another problem is that if there is a leading {{ResourceManager}}, then it
> can still be the case that it has not been started yet. If this is the case,
> then
> [ResourceManagerGateway.deregisterApplication|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/ResourceManagerServiceImpl.java#L143]
> will be discarded. The reason for this behaviour is that we create a
> {{ResourceManager}} in one {{Runnable}} and only start it in another. Due to
> this there can be the {{deregisterApplication}} call that gets the {{lock}}
> in between.
> I'd suggest to correct the lifecycle and contract of the
> {{ResourceManagerServiceImpl.deregisterApplication}}.
> Please note that due to this problem, the error reporting of this method has
> been suppressed. See FLINK-25885 for more details.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] wangyang0918 commented on pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s e2e tests
wangyang0918 commented on pull request #18639: URL: https://github.com/apache/flink/pull/18639#issuecomment-1031035534 cc @gaoyunhaii Could you give a pass for this simple change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24880) Error messages "OverflowError: timeout value is too large" shown when executing PyFlink jobs
[
https://issues.apache.org/jira/browse/FLINK-24880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487851#comment-17487851
]
Kafka Chris commented on FLINK-24880:
-
[~dianfu] Wow, Thank You. I can confirm the job was successful, it just kept
popping up in the logs. Let me know if I can provide more info.
> Error messages "OverflowError: timeout value is too large" shown when
> executing PyFlink jobs
>
>
> Key: FLINK-24880
> URL: https://issues.apache.org/jira/browse/FLINK-24880
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Affects Versions: 1.14.0
> Environment: Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018,
> 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
> {{python3 -m pip install apache-flink==1.14.0}}
> pycharm
>Reporter: wangdonglin
>Assignee: Dian Fu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0, 1.14.3
>
> Attachments: demo999.log, demo999.py
>
>
> The following exception shown when executing PyFlink jobs according to the
> demo show in
> [https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/datastream/intro_to_datastream_api/]
>
> h2. Common Structure of Python DataStream API Programs
>
> Exception in thread Thread-15:
> Traceback (most recent call last):
> File "D:\soft\py\lib\threading.py", line 932, in _bootstrap_inner
> self.run()
> File
> "C:\Users\wangdonglin\AppData\Roaming\Python\Python38\site-packages\apache_beam\runners\worker\data_plane.py",
> line 218, in run
> while not self._finished.wait(next_call - time.time()):
> File "D:\soft\py\lib\threading.py", line 558, in wait
> signaled = self._cond.wait(timeout)
> File "D:\soft\py\lib\threading.py", line 306, in wait
> gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18640: [FLINK-24880][python] Fix PeriodicThread to handle properly for negative wait timeout value
flinkbot edited a comment on pull request #18640: URL: https://github.com/apache/flink/pull/18640#issuecomment-1031032425 ## CI report: * ab5350d7c1b61aea750c3157deca0132ff506082 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30821) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24880) Error messages "OverflowError: timeout value is too large" shown when executing PyFlink jobs
[
https://issues.apache.org/jira/browse/FLINK-24880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487850#comment-17487850
]
Dian Fu commented on FLINK-24880:
-
[~kafkachris] Oh, thanks for reporting this issue. It seems that it doesn't
handle correctly for a corner case. Have submitted a
[PR|https://github.com/apache/flink/pull/18640] to address it. Besides, I think
it should not affect the execution of PyFlink jobs as it only occurs when
stopping jobs and you could just ignore it for now. Feel free to correct me if
this is not the case for your job.
> Error messages "OverflowError: timeout value is too large" shown when
> executing PyFlink jobs
>
>
> Key: FLINK-24880
> URL: https://issues.apache.org/jira/browse/FLINK-24880
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Affects Versions: 1.14.0
> Environment: Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018,
> 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
> {{python3 -m pip install apache-flink==1.14.0}}
> pycharm
>Reporter: wangdonglin
>Assignee: Dian Fu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0, 1.14.3
>
> Attachments: demo999.log, demo999.py
>
>
> The following exception shown when executing PyFlink jobs according to the
> demo show in
> [https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/datastream/intro_to_datastream_api/]
>
> h2. Common Structure of Python DataStream API Programs
>
> Exception in thread Thread-15:
> Traceback (most recent call last):
> File "D:\soft\py\lib\threading.py", line 932, in _bootstrap_inner
> self.run()
> File
> "C:\Users\wangdonglin\AppData\Roaming\Python\Python38\site-packages\apache_beam\runners\worker\data_plane.py",
> line 218, in run
> while not self._finished.wait(next_call - time.time()):
> File "D:\soft\py\lib\threading.py", line 558, in wait
> signaled = self._cond.wait(timeout)
> File "D:\soft\py\lib\threading.py", line 306, in wait
> gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot commented on pull request #18640: [FLINK-24880][python] Fix PeriodicThread to handle properly for negative wait timeout value
flinkbot commented on pull request #18640: URL: https://github.com/apache/flink/pull/18640#issuecomment-1031032769 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ab5350d7c1b61aea750c3157deca0132ff506082 (Mon Feb 07 03:20:22 UTC 2022) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18640: [FLINK-24880][python] Fix PeriodicThread to handle properly for negative wait timeout value
flinkbot commented on pull request #18640: URL: https://github.com/apache/flink/pull/18640#issuecomment-1031032425 ## CI report: * ab5350d7c1b61aea750c3157deca0132ff506082 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Reopened] (FLINK-24880) Error messages "OverflowError: timeout value is too large" shown when executing PyFlink jobs
[
https://issues.apache.org/jira/browse/FLINK-24880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu reopened FLINK-24880:
-
> Error messages "OverflowError: timeout value is too large" shown when
> executing PyFlink jobs
>
>
> Key: FLINK-24880
> URL: https://issues.apache.org/jira/browse/FLINK-24880
> Project: Flink
> Issue Type: Improvement
> Components: API / Python
>Affects Versions: 1.14.0
> Environment: Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018,
> 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
> {{python3 -m pip install apache-flink==1.14.0}}
> pycharm
>Reporter: wangdonglin
>Assignee: Dian Fu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0, 1.14.3
>
> Attachments: demo999.log, demo999.py
>
>
> The following exception shown when executing PyFlink jobs according to the
> demo show in
> [https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/datastream/intro_to_datastream_api/]
>
> h2. Common Structure of Python DataStream API Programs
>
> Exception in thread Thread-15:
> Traceback (most recent call last):
> File "D:\soft\py\lib\threading.py", line 932, in _bootstrap_inner
> self.run()
> File
> "C:\Users\wangdonglin\AppData\Roaming\Python\Python38\site-packages\apache_beam\runners\worker\data_plane.py",
> line 218, in run
> while not self._finished.wait(next_call - time.time()):
> File "D:\soft\py\lib\threading.py", line 558, in wait
> signaled = self._cond.wait(timeout)
> File "D:\soft\py\lib\threading.py", line 306, in wait
> gotit = waiter.acquire(True, timeout)
> OverflowError: timeout value is too large
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] dianfu opened a new pull request #18640: [FLINK-24880][python] Fix PeriodicThread to handle properly for negative wait timeout value
dianfu opened a new pull request #18640: URL: https://github.com/apache/flink/pull/18640 ## What is the purpose of the change *This pull request fixes PeriodicThread to handle properly for negative wait timeout value* ## Verifying this change This change is a trivial rework without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? ( no) - If yes, how is the feature documented? (not applicable ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25875) "StatsDReporter. FilterCharacters" for special processing of the characters are comprehensive enough
[ https://issues.apache.org/jira/browse/FLINK-25875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487847#comment-17487847 ] 赵富午 commented on FLINK-25875: - We can have some discussions, and I hope we can resolve this as soon as possible. > "StatsDReporter. FilterCharacters" for special processing of the characters > are comprehensive enough > > > Key: FLINK-25875 > URL: https://issues.apache.org/jira/browse/FLINK-25875 > Project: Flink > Issue Type: Improvement > Components: Runtime / Metrics >Affects Versions: 1.14.3 >Reporter: 赵富午 >Priority: Major > Labels: pull-request-available > Attachments: image-2022-01-29-11-55-20-400.png > > > I based on the 「org.Apache.Flink.Metrics.Statsd.StatsDReporter」 metrics > collection, query and display and use 「Graphite」, I found some flink metrics > cannot be queried to, after screening, I found the reason, These indicators > cannot be parsed because they contain space characters. I further track > source code, I found 「StatsDReporter.FilterCharacters」 function for the > processing of special characters is not qualified, only to replace ":" > character, for other special characters and didn't do a good replacement, > such as a space character. > !image-2022-01-29-11-55-38-064.png! > [https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/#garbagecollection] > JVM "Garbagecollection" grouping, indicators have space characters, > indicators cannot be used because indicators cannot be correctly parsed and > stored in the database. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25875) "StatsDReporter. FilterCharacters" for special processing of the characters are comprehensive enough
[ https://issues.apache.org/jira/browse/FLINK-25875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487843#comment-17487843 ] 赵富午 commented on FLINK-25875: - May I ask if I have described the problem clearly? > "StatsDReporter. FilterCharacters" for special processing of the characters > are comprehensive enough > > > Key: FLINK-25875 > URL: https://issues.apache.org/jira/browse/FLINK-25875 > Project: Flink > Issue Type: Improvement > Components: Runtime / Metrics >Affects Versions: 1.14.3 >Reporter: 赵富午 >Priority: Major > Labels: pull-request-available > Attachments: image-2022-01-29-11-55-20-400.png > > > I based on the 「org.Apache.Flink.Metrics.Statsd.StatsDReporter」 metrics > collection, query and display and use 「Graphite」, I found some flink metrics > cannot be queried to, after screening, I found the reason, These indicators > cannot be parsed because they contain space characters. I further track > source code, I found 「StatsDReporter.FilterCharacters」 function for the > processing of special characters is not qualified, only to replace ":" > character, for other special characters and didn't do a good replacement, > such as a space character. > !image-2022-01-29-11-55-38-064.png! > [https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/#garbagecollection] > JVM "Garbagecollection" grouping, indicators have space characters, > indicators cannot be used because indicators cannot be correctly parsed and > stored in the database. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18220: [FLINK-25410] Flink CLI should exit when app is accepted with detach …
flinkbot edited a comment on pull request #18220: URL: https://github.com/apache/flink/pull/18220#issuecomment-1001978981 ## CI report: * e41ae0b97f0af281c0f11cf66806901361d85302 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30810) * 8dc3a423728c27a335cd6f2f393ef79fb801e9c7 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30820) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zuston commented on a change in pull request #18220: [FLINK-25410] Flink CLI should exit when app is accepted with detach …
zuston commented on a change in pull request #18220:
URL: https://github.com/apache/flink/pull/18220#discussion_r800286195
##
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClientWrapper.java
##
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.yarn;
+
+import org.apache.flink.annotation.Internal;
+
+import org.apache.hadoop.service.Service;
+import org.apache.hadoop.yarn.api.records.ApplicationId;
+import org.apache.hadoop.yarn.api.records.ApplicationReport;
+import org.apache.hadoop.yarn.client.api.YarnClient;
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+
+import java.io.IOException;
+import java.util.Objects;
+
+/** Wrapper class for {@link YarnClient}. */
+@Internal
+public class YarnClientWrapper implements AutoCloseable {
+private YarnClient yarnClient;
+private boolean allowToStop;
+
+public YarnClientWrapper(YarnClient yarnClient, boolean allowToStop) {
+this.yarnClient = yarnClient;
+this.allowToStop = allowToStop;
+}
+
+public ApplicationReport getApplicationReport(ApplicationId appId)
+throws YarnException, IOException {
+return yarnClient.getApplicationReport(appId);
+}
+
+@Override
+public void close() throws Exception {
+if (allowToStop) {
+yarnClient.close();
+}
+}
+
+public boolean isClosed() {
+return yarnClient.isInState(Service.STATE.STOPPED);
+}
+
+public static YarnClientWrapper of(YarnClient yarnClient, boolean
allowToStop) {
+return new YarnClientWrapper(yarnClient, allowToStop);
+}
+
+public static YarnClientWrapper of(YarnConfiguration yarnConfiguration,
boolean allowToStop) {
+final YarnClient newlyCreatedYarnClient =
YarnClient.createYarnClient();
+newlyCreatedYarnClient.init(yarnConfiguration);
+newlyCreatedYarnClient.start();
+return of(newlyCreatedYarnClient, allowToStop);
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) {
+return true;
+}
+if (o == null || getClass() != o.getClass()) {
+return false;
+}
+YarnClientWrapper that = (YarnClientWrapper) o;
+return allowToStop == that.allowToStop &&
yarnClient.equals(that.yarnClient);
+}
+
+@Override
+public int hashCode() {
+return Objects.hash(yarnClient, allowToStop);
+}
Review comment:
Using Reference in YarnClientWrapper and then removed the
method of `equals` and `hashcode`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18220: [FLINK-25410] Flink CLI should exit when app is accepted with detach …
flinkbot edited a comment on pull request #18220: URL: https://github.com/apache/flink/pull/18220#issuecomment-1001978981 ## CI report: * e41ae0b97f0af281c0f11cf66806901361d85302 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30810) * 8dc3a423728c27a335cd6f2f393ef79fb801e9c7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s 2e tests
flinkbot edited a comment on pull request #18639: URL: https://github.com/apache/flink/pull/18639#issuecomment-1031013661 ## CI report: * 6273e9a4cfb0b0cf14d95926fb6c7748d82ad815 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30819) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s 2e tests
flinkbot commented on pull request #18639: URL: https://github.com/apache/flink/pull/18639#issuecomment-1031014271 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6273e9a4cfb0b0cf14d95926fb6c7748d82ad815 (Mon Feb 07 02:32:54 UTC 2022) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-25839).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s 2e tests
flinkbot commented on pull request #18639: URL: https://github.com/apache/flink/pull/18639#issuecomment-1031013661 ## CI report: * 6273e9a4cfb0b0cf14d95926fb6c7748d82ad815 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25839) 'Run kubernetes application HA test' failed on azure due to could not get 3 completed checkpoints in 120 sec
[
https://issues.apache.org/jira/browse/FLINK-25839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-25839:
---
Labels: pull-request-available test-stability (was: test-stability)
> 'Run kubernetes application HA test' failed on azure due to could not get 3
> completed checkpoints in 120 sec
>
>
> Key: FLINK-25839
> URL: https://issues.apache.org/jira/browse/FLINK-25839
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: pull-request-available, test-stability
>
> {code:java}
> Jan 27 02:07:33 deployment.apps/flink-native-k8s-application-ha-1 condition
> met
> Jan 27 02:07:33 Waiting for job
> (flink-native-k8s-application-ha-1-d8dc997d5-v8cpz) to have at least 3
> completed checkpoints ...
> Jan 27 02:09:45 Could not get 3 completed checkpoints in 120 sec
> Jan 27 02:09:45 Stopping job timeout watchdog (with pid=217858)
> Jan 27 02:09:45 Debugging failed Kubernetes test:
> Jan 27 02:09:45 Currently existing Kubernetes resources
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=30261=logs=af885ea8-6b05-5dc2-4a37-eab9c0d1ab09=f779a55a-0ffe-5bbc-8824-3a79333d4559=5376
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] wangyang0918 opened a new pull request #18639: [FLINK-25839][e2e] Print the previous logs of restarted pod for k8s 2e tests
wangyang0918 opened a new pull request #18639: URL: https://github.com/apache/flink/pull/18639 This PR is trying to print the previous logs of restarted pod. It is very useful to debug the failed K8s related e2e tests(e.g. FLINK-25839). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-24571) elasticsearch dynamic index support the use of proctime
[
https://issues.apache.org/jira/browse/FLINK-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487828#comment-17487828
]
jinfeng edited comment on FLINK-24571 at 2/7/22, 2:17 AM:
--
I'll update the pr in this week. To make sure proctime dynamc index only works
on append only stream.
was (Author: hackergin):
I'll update the pr in this week. To make sure dynamic of proctime only works
on append only stream.
> elasticsearch dynamic index support the use of proctime
>
>
> Key: FLINK-24571
> URL: https://issues.apache.org/jira/browse/FLINK-24571
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
>Reporter: jinfeng
>Assignee: jinfeng
>Priority: Major
> Labels: pull-request-available
>
> When use eleasticsearch dynamic index, we must special a field. If there is
> no time-attribute field, we can't use dynamic index. so support dynamic
> index using proctime may be a good choice.
> Before implement this feature, we should discuss the proctime index pattern.
>
> The current dynamic index is :
> {code:sql}
> Create table esSink (
> f1 timestamp(3)
> ) with (
> 'index' = 'myusers_{f1|-MM-dd}'
> );
> {code}
> When using proctime as dynamic index, It would be like this.
> {code:sql}
> Create table esSink (
>f1 varchar
> ) with (
> 'index' = 'myusers_{|-MM-dd}'
> );
> {code}
> When field name is not specified, we use proctime to generate the dynamic
> index.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-24571) elasticsearch dynamic index support the use of proctime
[
https://issues.apache.org/jira/browse/FLINK-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487828#comment-17487828
]
jinfeng commented on FLINK-24571:
-
I'll update the pr in this week. To make sure dynamic of proctime only works
on append only stream.
> elasticsearch dynamic index support the use of proctime
>
>
> Key: FLINK-24571
> URL: https://issues.apache.org/jira/browse/FLINK-24571
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
>Reporter: jinfeng
>Assignee: jinfeng
>Priority: Major
> Labels: pull-request-available
>
> When use eleasticsearch dynamic index, we must special a field. If there is
> no time-attribute field, we can't use dynamic index. so support dynamic
> index using proctime may be a good choice.
> Before implement this feature, we should discuss the proctime index pattern.
>
> The current dynamic index is :
> {code:sql}
> Create table esSink (
> f1 timestamp(3)
> ) with (
> 'index' = 'myusers_{f1|-MM-dd}'
> );
> {code}
> When using proctime as dynamic index, It would be like this.
> {code:sql}
> Create table esSink (
>f1 varchar
> ) with (
> 'index' = 'myusers_{|-MM-dd}'
> );
> {code}
> When field name is not specified, we use proctime to generate the dynamic
> index.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18461: [FLINK-20830][Kubernetes]Add type of Headless_Cluster_IP for external…
flinkbot edited a comment on pull request #18461: URL: https://github.com/apache/flink/pull/18461#issuecomment-1019901761 ## CI report: * 182394f479ac46311553e073947a792627794dfd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30359) * 5935b3b166f9fd6ec43f4428e804b7a54a436a8c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30818) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24571) elasticsearch dynamic index support the use of proctime
[
https://issues.apache.org/jira/browse/FLINK-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jinfeng updated FLINK-24571:
Labels: pull-request-available (was: pull-request-available stale-assigned)
> elasticsearch dynamic index support the use of proctime
>
>
> Key: FLINK-24571
> URL: https://issues.apache.org/jira/browse/FLINK-24571
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / ElasticSearch
>Reporter: jinfeng
>Assignee: jinfeng
>Priority: Major
> Labels: pull-request-available
>
> When use eleasticsearch dynamic index, we must special a field. If there is
> no time-attribute field, we can't use dynamic index. so support dynamic
> index using proctime may be a good choice.
> Before implement this feature, we should discuss the proctime index pattern.
>
> The current dynamic index is :
> {code:sql}
> Create table esSink (
> f1 timestamp(3)
> ) with (
> 'index' = 'myusers_{f1|-MM-dd}'
> );
> {code}
> When using proctime as dynamic index, It would be like this.
> {code:sql}
> Create table esSink (
>f1 varchar
> ) with (
> 'index' = 'myusers_{|-MM-dd}'
> );
> {code}
> When field name is not specified, we use proctime to generate the dynamic
> index.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18461: [FLINK-20830][Kubernetes]Add type of Headless_Cluster_IP for external…
flinkbot edited a comment on pull request #18461: URL: https://github.com/apache/flink/pull/18461#issuecomment-1019901761 ## CI report: * 182394f479ac46311553e073947a792627794dfd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30359) * 5935b3b166f9fd6ec43f4428e804b7a54a436a8c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wangyang0918 commented on pull request #18461: [FLINK-20830][Kubernetes]Add type of Headless_Cluster_IP for external…
wangyang0918 commented on pull request #18461: URL: https://github.com/apache/flink/pull/18461#issuecomment-1031004301 Rebase master and force pushed. I will merge this PR once the azure pipeline gives a pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #18434: [FLINK-25742][akka] Remove the serialization of rpc invocation at Fli…
KarmaGYZ commented on pull request #18434: URL: https://github.com/apache/flink/pull/18434#issuecomment-1030999706 @zentol Now, the `RemoteRpcInvocation` are serialized twice, first in Flink and second in Akka. This PR removes the first serialization in Flink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18555: [FLINK-25492][state/changelog] Place metadata at the beginning of the in-memory state changelog
flinkbot edited a comment on pull request #18555: URL: https://github.com/apache/flink/pull/18555#issuecomment-1024279714 ## CI report: * 0eac00f84a730b7000bb07e742140713730b485c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30814) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-4399) Add support for oversized messages
[
https://issues.apache.org/jira/browse/FLINK-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4399:
--
Labels: auto-deprioritized-major auto-deprioritized-minor flip-6 (was:
auto-deprioritized-major flip-6 stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Add support for oversized messages
> --
>
> Key: FLINK-4399
> URL: https://issues.apache.org/jira/browse/FLINK-4399
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Environment: FLIP-6 feature branch
>Reporter: Stephan Ewen
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> flip-6
>
> Currently, messages larger than the maximum Akka Framesize cause an error
> when being transported. We should add a way to pass messages that are larger
> than the Framesize, as may happen for:
> - {{collect()}} calls that collect large data sets (via accumulators)
> - Job submissions and operator deployments where the functions closures are
> large (for example because it contains large pre-loaded data)
> - Function restore in cases where restored state is larger than
> checkpointed state (union state)
> I suggest to use the {{BlobManager}} to transfer large payload.
> - On the sender side, oversized messages are stored under a transient blob
> (which is deleted after first retrieval, or after a certain number of minutes)
> - The sender sends a "pointer to blob message" instead.
> - The receiver grabs the message from the blob upon receiving the pointer
> message
> The RPC Service should be optionally initializable with a "large message
> handler" which is internally the {{BlobManager}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-4502) Cassandra connector documentation has misleading consistency guarantees
[ https://issues.apache.org/jira/browse/FLINK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-4502: -- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Cassandra connector documentation has misleading consistency guarantees > --- > > Key: FLINK-4502 > URL: https://issues.apache.org/jira/browse/FLINK-4502 > Project: Flink > Issue Type: Bug > Components: Connectors / Cassandra, Documentation >Affects Versions: 1.1.0 >Reporter: Elias Levy >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > The Cassandra connector documentation states that "enableWriteAheadLog() is > an optional method, that allows exactly-once processing for non-deterministic > algorithms." This claim appears to be false. > From what I gather, the write ahead log feature of the connector works as > follows: > - The sink is replaced with a stateful operator that writes incoming messages > to the state backend based on checkpoint they belong in. > - When the operator is notified that a Flink checkpoint has been completed > it, for each set of checkpoints older than and including the committed one: > * reads its messages from the state backend > * writes them to Cassandra > * records that it has committed them to Cassandra for the specific > checkpoint and operator instance >* and erases them from the state backend. > This process attempts to avoid resubmitting queries to Cassandra that would > otherwise occur when recovering a job from a checkpoint and having messages > replayed. > Alas, this does not guarantee exactly once semantics at the sink. The writes > to Cassandra that occur when the operator is notified that checkpoint is > completed are not atomic and they are potentially non-idempotent. If the job > dies while writing to Cassandra or before committing the checkpoint via > committer, queries will be replayed when the job recovers. Thus the > documentation appear to be incorrect in stating this provides exactly-once > semantics. > There also seems to be an issue in GenericWriteAheadSink's > notifyOfCompletedCheckpoint which may result in incorrect output. If > sendValues returns false because a write failed, instead of bailing, it > simply moves on to the next checkpoint to commit if there is one, keeping the > previous one around to try again later. But that can result in newer data > being overwritten with older data when the previous checkpoint is retried. > Although given that CassandraCommitter implements isCheckpointCommitted as > checkpointID <= this.lastCommittedCheckpointID, it actually means that when > it goes back to try the uncommitted older checkpoint it will consider it > committed, even though some of its data may not have been written out, and > the data will be discarded. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-4574) Strengthen fetch interval implementation in Kinesis consumer
[
https://issues.apache.org/jira/browse/FLINK-4574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4574:
--
Labels: auto-deprioritized-major auto-deprioritized-minor
pull-request-available (was: auto-deprioritized-major pull-request-available
stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Strengthen fetch interval implementation in Kinesis consumer
>
>
> Key: FLINK-4574
> URL: https://issues.apache.org/jira/browse/FLINK-4574
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
>Affects Versions: 1.1.0
>Reporter: Tzu-Li (Gordon) Tai
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor,
> pull-request-available
>
> As pointed out by [~rmetzger], right now the fetch interval implementation in
> the {{ShardConsumer}} class of the Kinesis consumer can lead to much longer
> interval times than specified by the user, ex. say the specified fetch
> interval is {{f}}, it takes {{x}} to complete a {{getRecords()}} call, and
> {{y}} to complete processing the fetched records for emitting, than the
> actual interval between each fetch is actually {{f+x+y}}.
> The main problem with this is that we can never guarantee how much time has
> past since the last {{getRecords}} call, thus can not guarantee that returned
> shard iterators will not have expired the next time we use them, even if we
> limit the user-given value for {{f}} to not be longer than the iterator
> expire time.
> I propose to improve this by, per {{ShardConsumer}}, use a
> {{ScheduledExecutorService}} / {{Timer}} to do the fixed-interval fetching,
> and a separate blocking queue that collects the fetched records for emitting.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-4558) Add support for synchronizing streams
[ https://issues.apache.org/jira/browse/FLINK-4558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-4558: -- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Add support for synchronizing streams > - > > Key: FLINK-4558 > URL: https://issues.apache.org/jira/browse/FLINK-4558 > Project: Flink > Issue Type: Improvement > Components: API / DataStream >Affects Versions: 1.1.0 >Reporter: Elias Levy >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > As mentioned on the [mailing > list|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/synchronizing-two-streams-td6830.html], > there are use cases that require synchronizing two streams on via their > times and where it is not practical to buffer all messages from one streams > while waiting for the other to synchronize. Flink should add functionality > to enable such use cases. > This could be implemented by modifying TwoInputStreamOperator so that calls > to processElement1 and processElement2 could return a value indicating that > the element can't yet be processed, having the framework then pause > processing for some time, potentially using exponential back off with a hard > maximum, and then allowing the back pressure system to do its work and pause > the stream. > Alternatively, an API could be added to explicitly pause/unpause a stream. > For ease of use either of these mechanism should be used to create a > SynchronizedTwoInputStreamOperator that end users can utilize by passing a > configurable time delta to use as a synchronization threshold. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-4553) Add support for table sampling
[
https://issues.apache.org/jira/browse/FLINK-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4553:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Add support for table sampling
> --
>
> Key: FLINK-4553
> URL: https://issues.apache.org/jira/browse/FLINK-4553
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Reporter: Timo Walther
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Calcite SQL defines 3 sampling functions. What we want to implement and how
> is up for discussion:
> {code}
> SELECT * FROM myTable TABLESAMPLE SUBSTITUTE('medium')
> SELECT * FROM myTable TABLESAMPLE TABLESAMPLE BERNOULLI(percentage)
> SELECT * FROM myTable TABLESAMPLE SYSTEM(percentage)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-4498) Better Cassandra sink documentation
[ https://issues.apache.org/jira/browse/FLINK-4498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-4498: -- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Better Cassandra sink documentation > --- > > Key: FLINK-4498 > URL: https://issues.apache.org/jira/browse/FLINK-4498 > Project: Flink > Issue Type: Improvement > Components: Connectors / Cassandra, Documentation >Affects Versions: 1.1.0 >Reporter: Elias Levy >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > > The Cassandra sink documentation is somewhat muddled and could be improved. > For instance, the fact that is only supports tuples and POJO's that use > DataStax Mapper annotations is only mentioned in passing, and it is not clear > that the reference to tuples only applies to Flink Java tuples and not Scala > tuples. > The documentation also does not mention that setQuery() is only necessary for > tuple streams. > The explanation of the write ahead log could use some cleaning up to clarify > when it is appropriate to use, ideally with an example. Maybe this would be > best as a blog post to expand on the type of non-deterministic streams this > applies to. > It would also be useful to mention that tuple elements will be mapped to > Cassandra columns using the Datastax Java driver's default encoders, which > are somewhat limited (e.g. to write to a blob column the type in the tuple > must be a java.nio.ByteBuffer and not just a byte[]). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-4233) Simplify leader election / leader session ID assignment
[ https://issues.apache.org/jira/browse/FLINK-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-4233: -- Labels: auto-deprioritized-major auto-deprioritized-minor (was: auto-deprioritized-major stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Simplify leader election / leader session ID assignment > --- > > Key: FLINK-4233 > URL: https://issues.apache.org/jira/browse/FLINK-4233 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.0.3 >Reporter: Stephan Ewen >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor > > Currently, there are two separate actions and znodes involved in leader > election and communication of the leader session ID and leader URL. > This leads to some quite elaborate code that tries to make sure that the > leader session ID and leader URL always eventually converge to those of the > leader. > It is simpler to just encode both the ID and the URL into an id-string that > is attached to the leader latch znode. One would have to create a new leader > latch each time a contender re-applies for leadership. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-4620) Automatically creating savepoints
[
https://issues.apache.org/jira/browse/FLINK-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4620:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> Automatically creating savepoints
> -
>
> Key: FLINK-4620
> URL: https://issues.apache.org/jira/browse/FLINK-4620
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / State Backends
>Affects Versions: 1.1.2
>Reporter: Niels Basjes
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> In the current versions of Flink you can run an external command and then a
> savepoint is persisted in a durable location.
> Feature request: Make this a lot more automatic and easy to use.
> _Proposed workflow_
> # In my application I do something like this:
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.setStateBackend(new FsStateBackend("hdfs:///tmp/applicationState"));
> env.enableCheckpointing(5000, CheckpointingMode.EXACTLY_ONCE);
> env.enableAutomaticSavePoints(30);
> env.enableAutomaticSavePointCleaner(10);
> {code}
> # When I start the application for the first time the state backend is
> 'empty'.
> I expect the system to start in a clean state.
> After 10 minutes (30ms) a savepoint is created and stored.
> # When I stop and start the topology again it will automatically restore the
> last available savepoint.
> Things to think about:
> * Note that this feature still means the manual version is useful!!
> * What to do on startup if the state is incompatible with the topology? Fail
> the startup?
> * How many automatic savepoints to we keep? Only the last one?
> * Perhaps the API should allow multiple automatic savepoints at different
> intervals in different locations.
> {code}
> // Make every 10 minutes and keep the last 10
> env.enableAutomaticSavePoints(30, new
> FsStateBackend("hdfs:///tmp/applicationState"), 10);
> // Make every 24 hours and keep the last 30
> // Useful for being able to reproduce a problem a few days later
> env.enableAutomaticSavePoints(8640, new
> FsStateBackend("hdfs:///tmp/applicationDailyStateSnapshot"), 30);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-4578) AggregateOperator incorrectly sets ForwardedField with nested composite types
[
https://issues.apache.org/jira/browse/FLINK-4578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-4578:
--
Labels: auto-deprioritized-major auto-deprioritized-minor (was:
auto-deprioritized-major stale-minor)
Priority: Not a Priority (was: Minor)
This issue was labeled "stale-minor" 7 days ago and has not received any
updates so it is being deprioritized. If this ticket is actually Minor, please
raise the priority and ask a committer to assign you the issue or revive the
public discussion.
> AggregateOperator incorrectly sets ForwardedField with nested composite types
> -
>
> Key: FLINK-4578
> URL: https://issues.apache.org/jira/browse/FLINK-4578
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
>Reporter: Gábor Gévay
>Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> When an aggregation is called on a grouped DataSet,
> {{AggregateOperator.translateToDataFlow}} tries to determine whether the
> field that is being aggregated is the same field that the grouping is based
> on. If this is not the case, then it adds the ForwardedField property for the
> key field.
> However, the mechanism that makes this decision breaks when there are nested
> composite types involved, because it gets the key positions with
> {{getKeys().computeLogicalKeyPositions()}}, which returns the _flat_
> positions, whereas the position of the field to aggregate is counted only on
> the outer type.
> Example code: https://github.com/ggevay/flink/tree/agg-bad-forwarded-fields
> Here, I have changed the WordCount example to have the type
> {{Tuple3, String, Integer>}}, and do {{.groupBy(1).sum(2)}}
> (which groups by the String field and sums the Integer field). If you set a
> breakpoint into {{AggregateOperator.translateToDataFlow}}, you can see that
> {{logicalKeyPositions}} contains 2, and {{fields}} also contains 2, which
> causes {{keyFieldUsedInAgg}} to be erroneously set to true. The problem is
> caused by the Tuple2 being counted as 2 fields in {{logicalKeyPositions}},
> but only 1 field in {{fields}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-4642) Remove unnecessary Guava dependency from flink-streaming-java
[ https://issues.apache.org/jira/browse/FLINK-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flink Jira Bot updated FLINK-4642: -- Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned (was: auto-deprioritized-major auto-unassigned stale-minor) Priority: Not a Priority (was: Minor) This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Remove unnecessary Guava dependency from flink-streaming-java > - > > Key: FLINK-4642 > URL: https://issues.apache.org/jira/browse/FLINK-4642 > Project: Flink > Issue Type: Bug > Components: API / DataStream >Affects Versions: 1.1.2 >Reporter: Stephan Ewen >Priority: Not a Priority > Labels: auto-deprioritized-major, auto-deprioritized-minor, > auto-unassigned > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18555: [FLINK-25492][state/changelog] Place metadata at the beginning of the in-memory state changelog
flinkbot edited a comment on pull request #18555: URL: https://github.com/apache/flink/pull/18555#issuecomment-1024279714 ## CI report: * 4a05b15e68bcece7aba07377a627ead54e2fc3b8 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30457) * 0eac00f84a730b7000bb07e742140713730b485c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30814) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18555: [FLINK-25492][state/changelog] Place metadata at the beginning of the in-memory state changelog
flinkbot edited a comment on pull request #18555: URL: https://github.com/apache/flink/pull/18555#issuecomment-1024279714 ## CI report: * 4a05b15e68bcece7aba07377a627ead54e2fc3b8 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30457) * 0eac00f84a730b7000bb07e742140713730b485c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25965) Add a Couchbase sink
Nir Tsruya created FLINK-25965: -- Summary: Add a Couchbase sink Key: FLINK-25965 URL: https://issues.apache.org/jira/browse/FLINK-25965 Project: Flink Issue Type: New Feature Components: Connectors / Common Reporter: Nir Tsruya h2. Motivation *User stories:* As a Flink user, I’d like to use CouchBase as sink for my data pipeline. *Scope:* * Implement an asynchronous sink for CouchBase inheriting the AsyncSinkBase class. The implementation can for now reside in its own module in flink-connectors. * Implement an asynchornous sink writer for CouchBase by extending the AsyncSinkWriter. The implemented Sink Writer will be used by the Sink class that will be created as part of this story. * Java / code-level docs. * End to end testing -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink-statefun-playground] igalshilman commented on a change in pull request #20: [FLINK-25935] Add LocalEnvironmentEntrypoint
igalshilman commented on a change in pull request #20:
URL:
https://github.com/apache/flink-statefun-playground/pull/20#discussion_r800038143
##
File path: java/greeter/docker-compose.yml
##
@@ -34,36 +34,21 @@ services:
#StateFun runtime
###
- statefun-manager:
-image: apache/flink-statefun:3.2.0-java11
-expose:
- - "6123"
+ statefun:
+image: statefun-playground-entrypoint:latest
+command:
+ --module file:///module.yaml
+ --set parallelism.default=1
ports:
- "8081:8081"
-environment:
Review comment:
I'm such a big fan of red diffs
##
File path: java/greeter/docker-compose.yml
##
@@ -34,36 +34,21 @@ services:
#StateFun runtime
###
- statefun-manager:
-image: apache/flink-statefun:3.2.0-java11
-expose:
- - "6123"
+ statefun:
+image: statefun-playground-entrypoint:latest
+command:
+ --module file:///module.yaml
+ --set parallelism.default=1
ports:
- "8081:8081"
-environment:
- ROLE: master
- MASTER_HOST: statefun-manager
-volumes:
- - ./module.yaml:/opt/statefun/modules/greeter/module.yaml
-
- statefun-worker:
-image: apache/flink-statefun:3.2.0-java11
-expose:
- - "6121"
- - "6122"
depends_on:
- - statefun-manager
- kafka
- greeter-functions
links:
- - "statefun-manager:statefun-manager"
- "kafka:kafka"
- "greeter-functions:greeter-functions"
-environment:
- ROLE: worker
- MASTER_HOST: statefun-manager
volumes:
- - ./module.yaml:/opt/statefun/modules/greeter/module.yaml
+ - ./module.yaml:/module.yaml
Review comment:
I guess that this makes sense for the playground runner.
##
File path: java/greeter/docker-compose.yml
##
@@ -34,36 +34,21 @@ services:
#StateFun runtime
###
- statefun-manager:
-image: apache/flink-statefun:3.2.0-java11
-expose:
- - "6123"
+ statefun:
+image: statefun-playground-entrypoint:latest
Review comment:
why not `apache/statefun-playground` ?
##
File path: playground-internal/statefun-playground-entrypoint/pom.xml
##
@@ -0,0 +1,144 @@
+
+
+http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+org.apache.flink
+statefun-playground-entrypoint
+3.2.0
+jar
+
+
+3.2.0
+1.14.3
+1.7.35
+11
+11
+
+
+
+
+
+org.apache.flink
+statefun-flink-core
+${statefun.version}
+
+
+
+org.apache.flink
+statefun-flink-io-bundle
+${statefun.version}
+
+
+
+
+org.apache.flink
+flink-java
+${flink.version}
+
+
+org.apache.flink
+flink-streaming-java_2.12
+${flink.version}
+
+
+org.apache.flink
+flink-clients_2.12
+${flink.version}
+
+
+org.apache.flink
+flink-runtime-web_2.12
+${flink.version}
+
+
+org.apache.flink
+flink-statebackend-rocksdb_2.12
+${flink.version}
+
+
+
+
+org.slf4j
+slf4j-api
+${slf4j.version}
+
+
+org.slf4j
+slf4j-log4j12
+${slf4j.version}
+
+
+
+
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
Review comment:
nit: you can configure it, to not produce a reduced pom.
##
File path: java/greeter/docker-compose.yml
##
@@ -34,36 +34,21 @@ services:
#StateFun runtime
###
- statefun-manager:
-image: apache/flink-statefun:3.2.0-java11
-expose:
- - "6123"
+ statefun:
+image: statefun-playground-entrypoint:latest
+command:
+ --module file:///module.yaml
Review comment:
What do you think of leaving this command out,
So that folks would know that they can tweak this, but it would work without
specifying any command?
##
File path: playground-internal/statefun-playground-entrypoint/Dockerfile
##
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
[GitHub] [flink] flinkbot edited a comment on pull request #18637: [FLINK-25433][runtime] Adds retry mechanism to DefaultResourceCleaner
flinkbot edited a comment on pull request #18637: URL: https://github.com/apache/flink/pull/18637#issuecomment-1030235273 ## CI report: * 3a47f3be8a7d903ec01d27d67788d427f9a0243c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=30813) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
