[jira] [Commented] (FLINK-25318) Improvement of scheduler and execution for Flink OLAP
[ https://issues.apache.org/jira/browse/FLINK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459716#comment-17459716 ] Xintong Song commented on FLINK-25318: -- Thanks [~zjureel], for creating this umbrella, and more importantly for sharing the experiences and contributing the improvements from ByteDance's internal practices in Flink OLAP. > Improvement of scheduler and execution for Flink OLAP > - > > Key: FLINK-25318 > URL: https://issues.apache.org/jira/browse/FLINK-25318 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination, Runtime / Network >Affects Versions: 1.14.0, 1.12.5, 1.13.3 >Reporter: Shammon >Priority: Major > Labels: Umbrella > Fix For: 1.15.0 > > > We use flink to perform OLAP queries. We launch flink session cluster, submit > batch jobs to the cluster as OLAP queries, and fetch the jobs' results. OLAP > jobs are generally small queries which will finish at the seconds or > milliseconds, and users always submit multiple jobs to the session cluster > concurrently. We found the qps and latency of jobs will be greatly affected > when there're tens jobs are running, even when there's little data in each > query. We will give the result of benchmark for the latest version later. > After discussed with [~xtsong], and thanks for his advice, we create this > issue to trace and manager Flink OLAP related improvements. More users and > developers are welcome and feel free to create Flink OLAP related subtasks > here, thanks -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25307) Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure
[
https://issues.apache.org/jira/browse/FLINK-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459715#comment-17459715
]
Dawid Wysakowicz commented on FLINK-25307:
--
Looking at logs it seems that we do not even reach a point when the Dispatcher
REST endpoint is up, from the point of view of the client. The logs show:
{code}
Dec 14 15:38:55 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:41:06 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:43:17 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:45:28 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:47:39 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:49:50 Waiting for Dispatcher REST endpoint to come up...
Dec 14 15:51:41 Test (pid: 93533) did not finish after 900 seconds.
Dec 14 15:51:41 Printing Flink logs and killing it:
{code}
What is strange is the frequency with which we're querying the Dispatcher. We
query it every ~2minutes, whereas as far as I can tell looking at the code we
should query it every ~30seconds. Could it be some caused by a load on our
machines?
> Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test
> timeout on azure
> ---
>
> Key: FLINK-25307
> URL: https://issues.apache.org/jira/browse/FLINK-25307
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Runtime / Checkpointing
>Affects Versions: 1.13.3, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/common.sh: line 860:
> kill: (93166) - No such process
> Dec 14 10:30:13 Stopping job timeout watchdog (with pid=93166)
> Dec 14 10:30:13 [FAIL] Test script contains errors.
> Dec 14 10:30:13 Checking for errors...
> Dec 14 10:30:14 No errors in log files.
> Dec 14 10:30:14 Checking for exceptions...
> Dec 14 10:30:14 No exceptions in log files.
> Dec 14 10:30:14 Checking for non-empty .out files...
> Dec 14 10:30:14 No non-empty .out files.
> Dec 14 10:30:14
> Dec 14 10:30:14 [FAIL] 'Resuming Savepoint (hashmap, async, no parallelism
> change) end-to-end test' failed after 15 minutes and 0 seconds! Test exited
> with exit code 1
> Dec 14 10:30:14
> 10:30:14 ##[group]Environment Information
> Dec 14 10:30:15 Searching for .dump, .dumpstream and related files in
> '/home/vsts/work/1/s'
> dmesg: read kernel buffer failed: Operation not permitted
> Dec 14 10:30:16 Stopping taskexecutor daemon (pid: 93751) on host fv-az43-70.
> Dec 14 10:30:17 Stopping standalonesession daemon (pid: 93500) on host
> fv-az43-70.
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/usr/bin/bash'. This may indicate a child process inherited the
> STDIO streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Run e2e tests
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28088=logs=bea52777-eaf8-5663-8482-18fbc3630e81=b2642e3a-5b86-574d-4c8a-f7e2842bfb14=79112
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18088: [FLINK-25174][table] Introduce managed table interfaces and callback
flinkbot edited a comment on pull request #18088: URL: https://github.com/apache/flink/pull/18088#issuecomment-992202380 ## CI report: * 2d0504a97f7125e2bc2cb88989fd9de07344ed62 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28143) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28153) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25318) Improvement of scheduler and execution for Flink OLAP
[ https://issues.apache.org/jira/browse/FLINK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-25318: - Fix Version/s: 1.15.0 > Improvement of scheduler and execution for Flink OLAP > - > > Key: FLINK-25318 > URL: https://issues.apache.org/jira/browse/FLINK-25318 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination, Runtime / Network >Affects Versions: 1.14.0, 1.12.5, 1.13.3 >Reporter: Shammon >Priority: Major > Labels: Umbrella > Fix For: 1.15.0 > > > We use flink to perform OLAP queries. We launch flink session cluster, submit > batch jobs to the cluster as OLAP queries, and fetch the jobs' results. OLAP > jobs are generally small queries which will finish at the seconds or > milliseconds, and users always submit multiple jobs to the session cluster > concurrently. We found the qps and latency of jobs will be greatly affected > when there're tens jobs are running, even when there's little data in each > query. We will give the result of benchmark for the latest version later. > After discussed with [~xtsong], and thanks for his advice, we create this > issue to trace and manager Flink OLAP related improvements. More users and > developers are welcome and feel free to create Flink OLAP related subtasks > here, thanks -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-25318) Improvement of scheduler and execution for Flink OLAP
[ https://issues.apache.org/jira/browse/FLINK-25318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-25318: - Labels: Umbrella (was: ) > Improvement of scheduler and execution for Flink OLAP > - > > Key: FLINK-25318 > URL: https://issues.apache.org/jira/browse/FLINK-25318 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination, Runtime / Network >Affects Versions: 1.14.0, 1.12.5, 1.13.3 >Reporter: Shammon >Priority: Major > Labels: Umbrella > > We use flink to perform OLAP queries. We launch flink session cluster, submit > batch jobs to the cluster as OLAP queries, and fetch the jobs' results. OLAP > jobs are generally small queries which will finish at the seconds or > milliseconds, and users always submit multiple jobs to the session cluster > concurrently. We found the qps and latency of jobs will be greatly affected > when there're tens jobs are running, even when there's little data in each > query. We will give the result of benchmark for the latest version later. > After discussed with [~xtsong], and thanks for his advice, we create this > issue to trace and manager Flink OLAP related improvements. More users and > developers are welcome and feel free to create Flink OLAP related subtasks > here, thanks -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18101: [FLINK-25222][tests] Remove NetworkFailureProxy used in Kafka tests
flinkbot edited a comment on pull request #18101: URL: https://github.com/apache/flink/pull/18101#issuecomment-993331950 ## CI report: * 60e9a62a2bc8a3aa67e95dec3690157bcf55bf0f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28090) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25307) Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure
[
https://issues.apache.org/jira/browse/FLINK-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25307:
Affects Version/s: 1.13.3
> Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test
> timeout on azure
> ---
>
> Key: FLINK-25307
> URL: https://issues.apache.org/jira/browse/FLINK-25307
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Runtime / Checkpointing
>Affects Versions: 1.13.3, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/common.sh: line 860:
> kill: (93166) - No such process
> Dec 14 10:30:13 Stopping job timeout watchdog (with pid=93166)
> Dec 14 10:30:13 [FAIL] Test script contains errors.
> Dec 14 10:30:13 Checking for errors...
> Dec 14 10:30:14 No errors in log files.
> Dec 14 10:30:14 Checking for exceptions...
> Dec 14 10:30:14 No exceptions in log files.
> Dec 14 10:30:14 Checking for non-empty .out files...
> Dec 14 10:30:14 No non-empty .out files.
> Dec 14 10:30:14
> Dec 14 10:30:14 [FAIL] 'Resuming Savepoint (hashmap, async, no parallelism
> change) end-to-end test' failed after 15 minutes and 0 seconds! Test exited
> with exit code 1
> Dec 14 10:30:14
> 10:30:14 ##[group]Environment Information
> Dec 14 10:30:15 Searching for .dump, .dumpstream and related files in
> '/home/vsts/work/1/s'
> dmesg: read kernel buffer failed: Operation not permitted
> Dec 14 10:30:16 Stopping taskexecutor daemon (pid: 93751) on host fv-az43-70.
> Dec 14 10:30:17 Stopping standalonesession daemon (pid: 93500) on host
> fv-az43-70.
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/usr/bin/bash'. This may indicate a child process inherited the
> STDIO streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Run e2e tests
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28088=logs=bea52777-eaf8-5663-8482-18fbc3630e81=b2642e3a-5b86-574d-4c8a-f7e2842bfb14=79112
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] RocMarshal commented on a change in pull request #17770: [FLINK-24272][Connectors / FileSystem] FileSystem compaction should throw unsupported exception in at-least-once mode
RocMarshal commented on a change in pull request #17770: URL: https://github.com/apache/flink/pull/17770#discussion_r769336188 ## File path: docs/content/docs/connectors/table/filesystem.md ## @@ -195,6 +195,7 @@ When running file compaction in production, please be aware that: - Only files in a single checkpoint are compacted, that is, at least the same number of files as the number of checkpoints is generated. - The file before merging is invisible, so the visibility of the file may be: checkpoint interval + compaction time. - If the compaction takes too long, it will backpressure the job and extend the time period of checkpoint. +- unsupported at-least-once mode Review comment: ```suggestion - Unsupported at-least-once mode ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24216) org.apache.flink.test.scheduling.AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond blocked on azure
[
https://issues.apache.org/jira/browse/FLINK-24216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459710#comment-17459710
]
Yun Gao commented on FLINK-24216:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28134=logs=a549b384-c55a-52c0-c451-00e0477ab6db=81f2da51-a161-54c7-5b84-6001fed26530=12121
> org.apache.flink.test.scheduling.AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond
> blocked on azure
> --
>
> Key: FLINK-24216
> URL: https://issues.apache.org/jira/browse/FLINK-24216
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0, 1.14.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23735=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7=12914
> {code:java}
> Sep 08 06:46:01 "main" #1 prio=5 os_prio=0 tid=0x7f23c400b800 nid=0x498c
> waiting on condition [0x7f23cc276000]
> Sep 08 06:46:01java.lang.Thread.State: WAITING (parking)
> Sep 08 06:46:01 at sun.misc.Unsafe.park(Native Method)
> Sep 08 06:46:01 - parking to wait for <0x86d021a8> (a
> java.util.concurrent.CountDownLatch$Sync)
> Sep 08 06:46:01 at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> Sep 08 06:46:01 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> Sep 08 06:46:01 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> Sep 08 06:46:01 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> Sep 08 06:46:01 at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> Sep 08 06:46:01 at
> org.apache.flink.test.scheduling.AdaptiveSchedulerITCase$DummySource.awaitRunning(AdaptiveSchedulerITCase.java:272)
> Sep 08 06:46:01 at
> org.apache.flink.test.scheduling.AdaptiveSchedulerITCase$DummySource.access$100(AdaptiveSchedulerITCase.java:255)
> Sep 08 06:46:01 at
> org.apache.flink.test.scheduling.AdaptiveSchedulerITCase.testStopWithSavepointFailOnFirstSavepointSucceedOnSecond(AdaptiveSchedulerITCase.java:219)
> Sep 08 06:46:01 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Sep 08 06:46:01 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Sep 08 06:46:01 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Sep 08 06:46:01 at java.lang.reflect.Method.invoke(Method.java:498)
> Sep 08 06:46:01 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> Sep 08 06:46:01 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Sep 08 06:46:01 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> Sep 08 06:46:01 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Sep 08 06:46:01 at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> Sep 08 06:46:01 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Sep 08 06:46:01 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> Sep 08 06:46:01 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Sep 08 06:46:01 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18101: [FLINK-25222][tests] Remove NetworkFailureProxy used in Kafka tests
flinkbot edited a comment on pull request #18101: URL: https://github.com/apache/flink/pull/18101#issuecomment-993331950 ## CI report: * 60e9a62a2bc8a3aa67e95dec3690157bcf55bf0f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28090) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25317) ApplicationDispatcherBootstrapITCase blocked on azure
[
https://issues.apache.org/jira/browse/FLINK-25317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao updated FLINK-25317:
Labels: test-stability (was: )
> ApplicationDispatcherBootstrapITCase blocked on azure
> -
>
> Key: FLINK-25317
> URL: https://issues.apache.org/jira/browse/FLINK-25317
> Project: Flink
> Issue Type: Bug
> Components: Client / Job Submission
>Affects Versions: 1.14.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 15 05:25:12
> Dec 15 05:25:12 "VM Thread" os_prio=0 tid=0x7f7b641bc000 nid=0x2543b
> runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "Gang worker#0 (Parallel GC Threads)" os_prio=0
> tid=0x7f7b64020800 nid=0x25428 runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "Gang worker#1 (Parallel GC Threads)" os_prio=0
> tid=0x7f7b64022000 nid=0x2542a runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "G1 Main Concurrent Mark GC Thread" os_prio=0
> tid=0x7f7b64046000 nid=0x25431 runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "Gang worker#0 (G1 Parallel Marking Threads)" os_prio=0
> tid=0x7f7b64048000 nid=0x25432 runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "G1 Concurrent Refinement Thread#0" os_prio=0
> tid=0x7f7b64028000 nid=0x25430 runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "G1 Concurrent Refinement Thread#1" os_prio=0
> tid=0x7f7b64026800 nid=0x2542e runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "G1 Concurrent Refinement Thread#2" os_prio=0
> tid=0x7f7b64024800 nid=0x2542b runnable
> Dec 15 05:25:12
> Dec 15 05:25:12 "VM Periodic Task Thread" os_prio=0 tid=0x7f7b64207800
> nid=0x25446 waiting on condition
> Dec 15 05:25:12
> Dec 15 05:25:12 JNI global references: 2194
> Dec 15 05:25:12
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28133=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=7b25afdf-cc6c-566f-5459-359dc2585798=9362
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25055) Support listen and notify mechanism for PartitionRequest
[ https://issues.apache.org/jira/browse/FLINK-25055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shammon updated FLINK-25055: Parent: FLINK-25318 Issue Type: Sub-task (was: Improvement) > Support listen and notify mechanism for PartitionRequest > > > Key: FLINK-25055 > URL: https://issues.apache.org/jira/browse/FLINK-25055 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Affects Versions: 1.14.0, 1.12.5, 1.13.3 >Reporter: Shammon >Assignee: Shammon >Priority: Major > > We submit batch jobs to flink session cluster with eager scheduler for olap. > JM deploys subtasks to TaskManager independently, and the downstream subtasks > may start before the upstream ones are ready. The downstream subtask sends > PartitionRequest to upstream ones, and may receive PartitionNotFoundException > from them. Then it will retry to send PartitionRequest after a few ms until > timeout. > The current approach raises two problems. First, there will be too many retry > PartitionRequest messages. Each downstream subtask will send PartitionRequest > to all its upstream subtasks and the total number of messages will be O(N*N), > where N is the parallelism of subtasks. Secondly, the interval between > polling retries will increase the delay for upstream and downstream tasks to > confirm PartitionRequest. > We want to support listen and notify mechanism for PartitionRequest when the > job needs no failover. Upstream TaskManager will add the PartitionRequest to > a listen list with a timeout checker, and notify the request when the task > register its partition in the TaskManager. > [~nkubicek] I noticed that your scenario of using flink is similar to ours. > What do you think? And hope to hear from you [~trohrmann] THX -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Closed] (FLINK-25279) KafkaSourceLegacyITCase.testKeyValueSupport failed on azure due to topic already exists
[
https://issues.apache.org/jira/browse/FLINK-25279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yun Gao closed FLINK-25279.
---
Resolution: Duplicate
> KafkaSourceLegacyITCase.testKeyValueSupport failed on azure due to topic
> already exists
> ---
>
> Key: FLINK-25279
> URL: https://issues.apache.org/jira/browse/FLINK-25279
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 12 03:00:12 [ERROR] Failures:
> Dec 12 03:00:12 [ERROR]
> KafkaSourceLegacyITCase.testKeyValueSupport:58->KafkaConsumerTestBase.runKeyValueTest:1528->KafkaTestBase.createTestTopic:222
> Create test topic : keyvaluetest failed,
> org.apache.kafka.common.errors.TopicExistsException: Topic 'keyvaluetest'
> already exists.
> Dec 12 03:00:12 [INFO]
> Dec 12 03:00:12 [ERROR] Tests run: 177, Failures: 1, Errors: 0, Skipped: 0
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27995=logs=72d4811f-9f0d-5fd0-014a-0bc26b72b642=e424005a-b16e-540f-196d-da062cc19bdf=35697
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25279) KafkaSourceLegacyITCase.testKeyValueSupport failed on azure due to topic already exists
[
https://issues.apache.org/jira/browse/FLINK-25279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459709#comment-17459709
]
Yun Gao commented on FLINK-25279:
-
Would close due to duplicate with
https://issues.apache.org/jira/browse/FLINK-24119
> KafkaSourceLegacyITCase.testKeyValueSupport failed on azure due to topic
> already exists
> ---
>
> Key: FLINK-25279
> URL: https://issues.apache.org/jira/browse/FLINK-25279
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Major
> Labels: test-stability
>
> {code:java}
> Dec 12 03:00:12 [ERROR] Failures:
> Dec 12 03:00:12 [ERROR]
> KafkaSourceLegacyITCase.testKeyValueSupport:58->KafkaConsumerTestBase.runKeyValueTest:1528->KafkaTestBase.createTestTopic:222
> Create test topic : keyvaluetest failed,
> org.apache.kafka.common.errors.TopicExistsException: Topic 'keyvaluetest'
> already exists.
> Dec 12 03:00:12 [INFO]
> Dec 12 03:00:12 [ERROR] Tests run: 177, Failures: 1, Errors: 0, Skipped: 0
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27995=logs=72d4811f-9f0d-5fd0-014a-0bc26b72b642=e424005a-b16e-540f-196d-da062cc19bdf=35697
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (FLINK-15455) Enable TCP connection reuse across multiple jobs.
[ https://issues.apache.org/jira/browse/FLINK-15455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shammon updated FLINK-15455: Parent: FLINK-25318 Issue Type: Sub-task (was: Improvement) > Enable TCP connection reuse across multiple jobs. > - > > Key: FLINK-15455 > URL: https://issues.apache.org/jira/browse/FLINK-15455 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > > Currently, tcp connections can be only reuse by tasks residing in the same > TaskManager and consumes the same IntermediateResult. And after job finish or > failover, the TCP connections are closed and new connections must be setup > latter. > As an improvement, we can make tcp connections a cluster level resource which > can be reused by multi jobs. The advantages are as follows: > # Reduce the number of TCP connections so we can save some resources. > # Reduce the overhead of connection setup and close so restarted jobs after > failover and latter jobs submitted to the same session cluster can reuse the > previous connections. > We use Flink session cluster as a service for ad-hoc queries and the users > can produce some statistics or create some statements and reports at any > time. Most of the queries finish in 2s and we find tcp connection reuse help > a lot to reduce the average execution time which means more queries can be > processed using the same resource and time with even better user experience. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-24119) KafkaITCase.testTimestamps fails due to "Topic xxx already exist"
[
https://issues.apache.org/jira/browse/FLINK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459708#comment-17459708
]
Yun Gao commented on FLINK-24119:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=27995=logs=72d4811f-9f0d-5fd0-014a-0bc26b72b642=e424005a-b16e-540f-196d-da062cc19bdf=35697
> KafkaITCase.testTimestamps fails due to "Topic xxx already exist"
> -
>
> Key: FLINK-24119
> URL: https://issues.apache.org/jira/browse/FLINK-24119
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.14.0
>Reporter: Xintong Song
>Assignee: Fabian Paul
>Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0, 1.14.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23328=logs=c5f0071e-1851-543e-9a45-9ac140befc32=15a22db7-8faa-5b34-3920-d33c9f0ca23c=7419
> {code}
> Sep 01 15:53:20 [ERROR] Tests run: 23, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 162.65 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.kafka.KafkaITCase
> Sep 01 15:53:20 [ERROR] testTimestamps Time elapsed: 23.237 s <<< FAILURE!
> Sep 01 15:53:20 java.lang.AssertionError: Create test topic : tstopic failed,
> org.apache.kafka.common.errors.TopicExistsException: Topic 'tstopic' already
> exists.
> Sep 01 15:53:20 at org.junit.Assert.fail(Assert.java:89)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:226)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:112)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:212)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaITCase.testTimestamps(KafkaITCase.java:191)
> Sep 01 15:53:20 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Sep 01 15:53:20 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Sep 01 15:53:20 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Sep 01 15:53:20 at java.lang.reflect.Method.invoke(Method.java:498)
> Sep 01 15:53:20 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> Sep 01 15:53:20 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Sep 01 15:53:20 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> Sep 01 15:53:20 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Sep 01 15:53:20 at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fapaul commented on pull request #18101: [FLINK-25222][tests] Remove NetworkFailureProxy used in Kafka tests
fapaul commented on pull request #18101: URL: https://github.com/apache/flink/pull/18101#issuecomment-994423273 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25318) Improvement of scheduler and execution for Flink OLAP
Shammon created FLINK-25318: --- Summary: Improvement of scheduler and execution for Flink OLAP Key: FLINK-25318 URL: https://issues.apache.org/jira/browse/FLINK-25318 Project: Flink Issue Type: Improvement Components: Runtime / Coordination, Runtime / Network Affects Versions: 1.13.3, 1.12.5, 1.14.0 Reporter: Shammon We use flink to perform OLAP queries. We launch flink session cluster, submit batch jobs to the cluster as OLAP queries, and fetch the jobs' results. OLAP jobs are generally small queries which will finish at the seconds or milliseconds, and users always submit multiple jobs to the session cluster concurrently. We found the qps and latency of jobs will be greatly affected when there're tens jobs are running, even when there's little data in each query. We will give the result of benchmark for the latest version later. After discussed with [~xtsong], and thanks for his advice, we create this issue to trace and manager Flink OLAP related improvements. More users and developers are welcome and feel free to create Flink OLAP related subtasks here, thanks -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-24119) KafkaITCase.testTimestamps fails due to "Topic xxx already exist"
[
https://issues.apache.org/jira/browse/FLINK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459707#comment-17459707
]
Yun Gao commented on FLINK-24119:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28133=logs=ce8f3cc3-c1ea-5281-f5eb-df9ebd24947f=918e890f-5ed9-5212-a25e-962628fb4bc5=7374
> KafkaITCase.testTimestamps fails due to "Topic xxx already exist"
> -
>
> Key: FLINK-24119
> URL: https://issues.apache.org/jira/browse/FLINK-24119
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.14.0
>Reporter: Xintong Song
>Assignee: Fabian Paul
>Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0, 1.14.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23328=logs=c5f0071e-1851-543e-9a45-9ac140befc32=15a22db7-8faa-5b34-3920-d33c9f0ca23c=7419
> {code}
> Sep 01 15:53:20 [ERROR] Tests run: 23, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 162.65 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.kafka.KafkaITCase
> Sep 01 15:53:20 [ERROR] testTimestamps Time elapsed: 23.237 s <<< FAILURE!
> Sep 01 15:53:20 java.lang.AssertionError: Create test topic : tstopic failed,
> org.apache.kafka.common.errors.TopicExistsException: Topic 'tstopic' already
> exists.
> Sep 01 15:53:20 at org.junit.Assert.fail(Assert.java:89)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.createTestTopic(KafkaTestEnvironmentImpl.java:226)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironment.createTestTopic(KafkaTestEnvironment.java:112)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.createTestTopic(KafkaTestBase.java:212)
> Sep 01 15:53:20 at
> org.apache.flink.streaming.connectors.kafka.KafkaITCase.testTimestamps(KafkaITCase.java:191)
> Sep 01 15:53:20 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Sep 01 15:53:20 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Sep 01 15:53:20 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Sep 01 15:53:20 at java.lang.reflect.Method.invoke(Method.java:498)
> Sep 01 15:53:20 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> Sep 01 15:53:20 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Sep 01 15:53:20 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> Sep 01 15:53:20 at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> Sep 01 15:53:20 at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> Sep 01 15:53:20 at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18111: [FLINK-25094][Test]Correct the LatencyTrackingMapStateTest#verifyIterator
flinkbot edited a comment on pull request #18111: URL: https://github.com/apache/flink/pull/18111#issuecomment-994285128 ## CI report: * 930b0c12d7dad620b6e15c23289f2e2666ce0ed0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28141) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] becketqin commented on a change in pull request #17991: [FLINK-25132][connector/kafka] Move record deserializing from SplitFetcher to RecordEmitter to support object-reusing deseriali
becketqin commented on a change in pull request #17991:
URL: https://github.com/apache/flink/pull/17991#discussion_r769321825
##
File path:
flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/reader/KafkaPartitionSplitReader.java
##
@@ -105,84 +91,38 @@ public KafkaPartitionSplitReader(
}
@Override
-public RecordsWithSplitIds> fetch() throws
IOException {
-KafkaPartitionSplitRecords> recordsBySplits =
-new KafkaPartitionSplitRecords<>();
+public RecordsWithSplitIds> fetch() throws
IOException {
ConsumerRecords consumerRecords;
try {
consumerRecords = consumer.poll(Duration.ofMillis(POLL_TIMEOUT));
} catch (WakeupException we) {
-recordsBySplits.prepareForRead();
-return recordsBySplits;
+return null;
Review comment:
It seems that rerunning null here still might cause NPE in the
`FetchTask.run()` even though null is only returned when the consumer is
wakenup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25317) ApplicationDispatcherBootstrapITCase blocked on azure
Yun Gao created FLINK-25317:
---
Summary: ApplicationDispatcherBootstrapITCase blocked on azure
Key: FLINK-25317
URL: https://issues.apache.org/jira/browse/FLINK-25317
Project: Flink
Issue Type: Bug
Components: Client / Job Submission
Affects Versions: 1.14.0
Reporter: Yun Gao
{code:java}
Dec 15 05:25:12
Dec 15 05:25:12 "VM Thread" os_prio=0 tid=0x7f7b641bc000 nid=0x2543b
runnable
Dec 15 05:25:12
Dec 15 05:25:12 "Gang worker#0 (Parallel GC Threads)" os_prio=0
tid=0x7f7b64020800 nid=0x25428 runnable
Dec 15 05:25:12
Dec 15 05:25:12 "Gang worker#1 (Parallel GC Threads)" os_prio=0
tid=0x7f7b64022000 nid=0x2542a runnable
Dec 15 05:25:12
Dec 15 05:25:12 "G1 Main Concurrent Mark GC Thread" os_prio=0
tid=0x7f7b64046000 nid=0x25431 runnable
Dec 15 05:25:12
Dec 15 05:25:12 "Gang worker#0 (G1 Parallel Marking Threads)" os_prio=0
tid=0x7f7b64048000 nid=0x25432 runnable
Dec 15 05:25:12
Dec 15 05:25:12 "G1 Concurrent Refinement Thread#0" os_prio=0
tid=0x7f7b64028000 nid=0x25430 runnable
Dec 15 05:25:12
Dec 15 05:25:12 "G1 Concurrent Refinement Thread#1" os_prio=0
tid=0x7f7b64026800 nid=0x2542e runnable
Dec 15 05:25:12
Dec 15 05:25:12 "G1 Concurrent Refinement Thread#2" os_prio=0
tid=0x7f7b64024800 nid=0x2542b runnable
Dec 15 05:25:12
Dec 15 05:25:12 "VM Periodic Task Thread" os_prio=0 tid=0x7f7b64207800
nid=0x25446 waiting on condition
Dec 15 05:25:12
Dec 15 05:25:12 JNI global references: 2194
Dec 15 05:25:12
{code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28133=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=7b25afdf-cc6c-566f-5459-359dc2585798=9362
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-25227) Comparing the equality of the same (boxed) numeric values returns false
[
https://issues.apache.org/jira/browse/FLINK-25227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459704#comment-17459704
]
Francesco Guardiani commented on FLINK-25227:
-
[~TsReaper] but then I suggest we focus on hardening the code generator fixing
these functions that generate boxed types, more than introducing new functions
that generate boxed types that we'll need to fix later. Can you come up with a
list of such functions?
For previous versions I agree to just push the fix you proposed in
https://github.com/apache/flink/pull/18076.
> Comparing the equality of the same (boxed) numeric values returns false
> ---
>
> Key: FLINK-25227
> URL: https://issues.apache.org/jira/browse/FLINK-25227
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Affects Versions: 1.14.0, 1.12.5, 1.13.3
>Reporter: Caizhi Weng
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.15.0, 1.13.6, 1.14.3
>
>
> Add the following test case to {{TableEnvironmentITCase}} to reproduce this
> bug.
> {code:scala}
> @Test
> def myTest(): Unit = {
> val data = Seq(
> Row.of(
> java.lang.Integer.valueOf(1000),
> java.lang.Integer.valueOf(2000),
> java.lang.Integer.valueOf(1000),
> java.lang.Integer.valueOf(2000))
> )
> tEnv.executeSql(
> s"""
>|create table T (
>| a int,
>| b int,
>| c int,
>| d int
>|) with (
>| 'connector' = 'values',
>| 'bounded' = 'true',
>| 'data-id' = '${TestValuesTableFactory.registerData(data)}'
>|)
>|""".stripMargin)
> tEnv.executeSql("select greatest(a, b) = greatest(c, d) from T").print()
> }
> {code}
> The result is false, which is obviously incorrect.
> This is caused by the generated java code:
> {code:java}
> public class StreamExecCalc$8 extends
> org.apache.flink.table.runtime.operators.TableStreamOperator
> implements
> org.apache.flink.streaming.api.operators.OneInputStreamOperator {
> private final Object[] references;
> org.apache.flink.table.data.BoxedWrapperRowData out =
> new org.apache.flink.table.data.BoxedWrapperRowData(1);
> private final
> org.apache.flink.streaming.runtime.streamrecord.StreamRecord outElement =
> new
> org.apache.flink.streaming.runtime.streamrecord.StreamRecord(null);
> public StreamExecCalc$8(
> Object[] references,
> org.apache.flink.streaming.runtime.tasks.StreamTask task,
> org.apache.flink.streaming.api.graph.StreamConfig config,
> org.apache.flink.streaming.api.operators.Output output,
> org.apache.flink.streaming.runtime.tasks.ProcessingTimeService
> processingTimeService)
> throws Exception {
> this.references = references;
> this.setup(task, config, output);
> if (this instanceof
> org.apache.flink.streaming.api.operators.AbstractStreamOperator) {
>
> ((org.apache.flink.streaming.api.operators.AbstractStreamOperator) this)
> .setProcessingTimeService(processingTimeService);
> }
> }
> @Override
> public void open() throws Exception {
> super.open();
> }
> @Override
> public void
> processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord
> element)
> throws Exception {
> org.apache.flink.table.data.RowData in1 =
> (org.apache.flink.table.data.RowData) element.getValue();
> int field$0;
> boolean isNull$0;
> int field$1;
> boolean isNull$1;
> int field$3;
> boolean isNull$3;
> int field$4;
> boolean isNull$4;
> boolean isNull$6;
> boolean result$7;
> isNull$3 = in1.isNullAt(2);
> field$3 = -1;
> if (!isNull$3) {
> field$3 = in1.getInt(2);
> }
> isNull$0 = in1.isNullAt(0);
> field$0 = -1;
> if (!isNull$0) {
> field$0 = in1.getInt(0);
> }
> isNull$1 = in1.isNullAt(1);
> field$1 = -1;
> if (!isNull$1) {
> field$1 = in1.getInt(1);
> }
> isNull$4 = in1.isNullAt(3);
> field$4 = -1;
> if (!isNull$4) {
> field$4 = in1.getInt(3);
> }
> out.setRowKind(in1.getRowKind());
> java.lang.Integer result$2 = field$0;
> boolean nullTerm$2 = false;
> if (!nullTerm$2) {
> java.lang.Integer cur$2 = field$0;
> if (isNull$0) {
> nullTerm$2 = true;
> } else {
> int compareResult = result$2.compareTo(cur$2);
> if ((true &&
[GitHub] [flink] flinkbot edited a comment on pull request #18102: [FLINK-25033][runtime] Let some scheduler components updatable
flinkbot edited a comment on pull request #18102: URL: https://github.com/apache/flink/pull/18102#issuecomment-993332110 ## CI report: * 5125a3e6b02685be25cf24dac93fcdd939880754 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28091) * df17e67d40973994f1b39d403b1a9fe4279bfab4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28151) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18113: [FLINK-25315][tests] add new extensions and utils to help the Junit5 migration
flinkbot edited a comment on pull request #18113: URL: https://github.com/apache/flink/pull/18113#issuecomment-994409071 ## CI report: * 4664b97e3029a9c17abb1cfef7d3b45a85d825ac Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28152) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18088: [FLINK-25174][table] Introduce managed table interfaces and callback
flinkbot edited a comment on pull request #18088: URL: https://github.com/apache/flink/pull/18088#issuecomment-992202380 ## CI report: * 7660ffb0e155a244663d6ac0c93576a893a293aa Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28140) * 2d0504a97f7125e2bc2cb88989fd9de07344ed62 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28143) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18113: [FLINK-25315][tests] add new extensions and utils to help the Junit5 migration
flinkbot commented on pull request #18113: URL: https://github.com/apache/flink/pull/18113#issuecomment-994409071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18102: [FLINK-25033][runtime] Let some scheduler components updatable
flinkbot edited a comment on pull request #18102: URL: https://github.com/apache/flink/pull/18102#issuecomment-993332110 ## CI report: * 5125a3e6b02685be25cf24dac93fcdd939880754 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28091) * df17e67d40973994f1b39d403b1a9fe4279bfab4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18068: [FLINK-25105][checkpoint] Enables final checkpoint by default
flinkbot edited a comment on pull request #18068: URL: https://github.com/apache/flink/pull/18068#issuecomment-989975508 ## CI report: * 8c44c251f1be40e49bee24875b9625e859070010 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27923) * cee493f0c7ff22bb13e7a7a9a6c67ecb10b292d0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28149) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18087: [FLINK-25268] Support task manager node-label in Yarn deployment
flinkbot edited a comment on pull request #18087: URL: https://github.com/apache/flink/pull/18087#issuecomment-992135476 ## CI report: * f34d532366bda2f53429bf6ebc46d43e49b498b5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28020) * aed137998ec369495e0d8ef5d745531a9315e095 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28150) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18042: [FLINK-25076][table-planner] Improve operator name for sql job
flinkbot edited a comment on pull request #18042: URL: https://github.com/apache/flink/pull/18042#issuecomment-987776252 ## CI report: * 040a74cd89b7a6527c3cdac8bdce72972f5ff062 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28136) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16108: [FLINK-22821][core] Stabilize NetUtils#getAvailablePort in order to avoid wrongly allocating any used ports
flinkbot edited a comment on pull request #16108: URL: https://github.com/apache/flink/pull/16108#issuecomment-856641884 ## CI report: * 03e33a13e8826f6cda49070c16679533f1223e94 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28137) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15684: [FLINK-20899][table-planner-blink][hotfix] use cost factory in RelOptCluster to create HepPlanner.
flinkbot edited a comment on pull request #15684: URL: https://github.com/apache/flink/pull/15684#issuecomment-823050966 ## CI report: * d2e4faadb080c3c64fbae293f18cdaec3e0c3d95 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16855) * 7021654aaf32d24780f187924af65e693510473c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28148) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25315) Add some extensions and utils to help the Junit5 migration
[ https://issues.apache.org/jira/browse/FLINK-25315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25315: --- Labels: Tests pull-request-available (was: Tests) > Add some extensions and utils to help the Junit5 migration > -- > > Key: FLINK-25315 > URL: https://issues.apache.org/jira/browse/FLINK-25315 > Project: Flink > Issue Type: Technical Debt > Components: Tests >Reporter: Hang Ruan >Priority: Minor > Labels: Tests, pull-request-available > > Add some extensions and utils to help the Junit5 migration. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] ruanhang1993 opened a new pull request #18113: [FLINK-25315][tests] add new extensions and utils to help the Junit5 migration
ruanhang1993 opened a new pull request #18113: URL: https://github.com/apache/flink/pull/18113 ## What is the purpose of the change This pull request adds new extensions and utils to help the Junit5 migration. ## Brief change log - Add `ParameterizedTestExtension` and two annotations to help to migrate the Junit4 parameterized test - Add `TempDirUtils` to provide some common methods in `org.junit.rules.TemporaryFolder` - Migrate `LongHashTableTest` to Junit5 as an example to show how to use the new extension ## Verifying this change This change is already covered by existing tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25307) Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure
[
https://issues.apache.org/jira/browse/FLINK-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459698#comment-17459698
]
Yun Gao commented on FLINK-25307:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28112=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=79192
> Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test
> timeout on azure
> ---
>
> Key: FLINK-25307
> URL: https://issues.apache.org/jira/browse/FLINK-25307
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Runtime / Checkpointing
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Priority: Critical
> Labels: test-stability
>
> {code:java}
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/common.sh: line 860:
> kill: (93166) - No such process
> Dec 14 10:30:13 Stopping job timeout watchdog (with pid=93166)
> Dec 14 10:30:13 [FAIL] Test script contains errors.
> Dec 14 10:30:13 Checking for errors...
> Dec 14 10:30:14 No errors in log files.
> Dec 14 10:30:14 Checking for exceptions...
> Dec 14 10:30:14 No exceptions in log files.
> Dec 14 10:30:14 Checking for non-empty .out files...
> Dec 14 10:30:14 No non-empty .out files.
> Dec 14 10:30:14
> Dec 14 10:30:14 [FAIL] 'Resuming Savepoint (hashmap, async, no parallelism
> change) end-to-end test' failed after 15 minutes and 0 seconds! Test exited
> with exit code 1
> Dec 14 10:30:14
> 10:30:14 ##[group]Environment Information
> Dec 14 10:30:15 Searching for .dump, .dumpstream and related files in
> '/home/vsts/work/1/s'
> dmesg: read kernel buffer failed: Operation not permitted
> Dec 14 10:30:16 Stopping taskexecutor daemon (pid: 93751) on host fv-az43-70.
> Dec 14 10:30:17 Stopping standalonesession daemon (pid: 93500) on host
> fv-az43-70.
> The STDIO streams did not close within 10 seconds of the exit event from
> process '/usr/bin/bash'. This may indicate a child process inherited the
> STDIO streams and has not yet exited.
> ##[error]Bash exited with code '1'.
> Finishing: Run e2e tests
> {code}
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=28088=logs=bea52777-eaf8-5663-8482-18fbc3630e81=b2642e3a-5b86-574d-4c8a-f7e2842bfb14=79112
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18112: [FLINK-25305][checkpoint] Always wait for input channel state and result partition state get completed in AsyncRunnable
flinkbot edited a comment on pull request #18112: URL: https://github.com/apache/flink/pull/18112#issuecomment-994403719 ## CI report: * 6bb8f5418b1dcd085b16a00f1370a88262d3e4a9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28147) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18087: [FLINK-25268] Support task manager node-label in Yarn deployment
flinkbot edited a comment on pull request #18087: URL: https://github.com/apache/flink/pull/18087#issuecomment-992135476 ## CI report: * f34d532366bda2f53429bf6ebc46d43e49b498b5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28020) * aed137998ec369495e0d8ef5d745531a9315e095 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18068: [FLINK-25105][checkpoint] Enables final checkpoint by default
flinkbot edited a comment on pull request #18068: URL: https://github.com/apache/flink/pull/18068#issuecomment-989975508 ## CI report: * 8c44c251f1be40e49bee24875b9625e859070010 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27923) * cee493f0c7ff22bb13e7a7a9a6c67ecb10b292d0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15684: [FLINK-20899][table-planner-blink][hotfix] use cost factory in RelOptCluster to create HepPlanner.
flinkbot edited a comment on pull request #15684: URL: https://github.com/apache/flink/pull/15684#issuecomment-823050966 ## CI report: * d2e4faadb080c3c64fbae293f18cdaec3e0c3d95 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16855) * 7021654aaf32d24780f187924af65e693510473c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18112: [FLINK-25305][checkpoint] Always wait for input channel state and result partition state get completed in AsyncRunnable
flinkbot commented on pull request #18112: URL: https://github.com/apache/flink/pull/18112#issuecomment-994404371 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6bb8f5418b1dcd085b16a00f1370a88262d3e4a9 (Wed Dec 15 07:08:37 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #17711: [FLINK-24737][runtime-web] Update outdated web dependencies
MartijnVisser commented on pull request #17711: URL: https://github.com/apache/flink/pull/17711#issuecomment-994404275 > > Can you have a look at the failing build and fix it? > > @MartijnVisser Conflicts have been addressed. Thanks @yangjunhan The CI unfortunately still fails and reports: `[INFO] npm ERR! @jest/types not accessible from pretty-format` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24948) Special character in column names breaks JDBC statement parsing
[ https://issues.apache.org/jira/browse/FLINK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459696#comment-17459696 ] Martijn Visser commented on FLINK-24948: CC [~afedulov] [~arvid] > Special character in column names breaks JDBC statement parsing > --- > > Key: FLINK-24948 > URL: https://issues.apache.org/jira/browse/FLINK-24948 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.12.4 >Reporter: Paul Lin >Assignee: Paul Lin >Priority: Major > Labels: pull-request-available > > Currently, JDBC connector assumes columns names respect Java identifier > naming restrictions, but Databases that support JDBC may have different > naming restrictions. For example, MySQL allows dots and colons in column > names. In that case, JDBC connector would have trouble parsing the SQL. > We could fix this by validating field names in `JdbcDmlOptions`. In addition, > it'd be good to clarify the naming restrictions of Flink SQL, so users and > connector developers would know the standard. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18112: [FLINK-25305][checkpoint] Always wait for input channel state and result partition state get completed in AsyncRunnable
flinkbot commented on pull request #18112: URL: https://github.com/apache/flink/pull/18112#issuecomment-994403719 ## CI report: * 6bb8f5418b1dcd085b16a00f1370a88262d3e4a9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gaoyunhaii commented on pull request #18112: [FLINK-25305][checkpoint] Always wait for input channel state and result partition state get completed in AsyncRunnable
gaoyunhaii commented on pull request #18112: URL: https://github.com/apache/flink/pull/18112#issuecomment-994402749 @dawidwys @pnowojski could you help to have a look on this PR~? very thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25305) Always wait for input channel state and result partition state get completed in AsyncRunnable
[
https://issues.apache.org/jira/browse/FLINK-25305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-25305:
---
Labels: pull-request-available (was: )
> Always wait for input channel state and result partition state get completed
> in AsyncRunnable
> -
>
> Key: FLINK-25305
> URL: https://issues.apache.org/jira/browse/FLINK-25305
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
>Affects Versions: 1.15.0
>Reporter: Yun Gao
>Assignee: Yun Gao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
>
> {code:java}
> 29245 [jobmanager-io-thread-12] INFO
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Decline
> checkpoint 16 by task 07fea3eb73acb4898317b4aa2c9fea30 of job
> da6de908107aa847cde5e9e0beb4812b at 064277c9-73dc-4bf2-8729-91ab16bbe8c6 @
> localhost (dataPort=-1).org.apache.flink.util.SerializedThrowable:
> Asynchronous task checkpoint failed.
> at
> org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.handleExecutionException(AsyncCheckpointRunnable.java:321)
> ~[classes/:?]
> at
> org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:158)
> ~[classes/:?]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_271]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_271]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_271]
> Caused by: org.apache.flink.util.SerializedThrowable: Could not materialize
> checkpoint 16 for operator keyed (1/5)#5.
> at
> org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.handleExecutionException(AsyncCheckpointRunnable.java:299)
> ~[classes/:?]
> ... 4 more
> Caused by: org.apache.flink.util.SerializedThrowable
> at org.apache.flink.util.Preconditions.checkState(Preconditions.java:177)
> ~[classes/:?]
> at
> org.apache.flink.util.Preconditions.checkCompletedNormally(Preconditions.java:261)
> ~[classes/:?]
> at
> org.apache.flink.util.concurrent.FutureUtils.checkStateAndGet(FutureUtils.java:1193)
> ~[classes/:?]
> at
> org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder.build(CheckpointMetricsBuilder.java:133)
> ~[classes/:?]
> at
> org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.reportCompletedSnapshotStates(AsyncCheckpointRunnable.java:248)
> ~[classes/:?]
> at
> org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:139)
> ~[classes/:?]
> ... 3 more
> {code}
>
> When both unaligned checkpoint and final checkpoint is enabled, some
> checkpoints would fail due to the above exception at the async phase,
> indicating that the checkpoint metric futures are not fully fulfilled.
> The exception should be caused by when a task is restored with previously
> fully finished, when taking checkpoint, we would skip snapshotting the state
> of the operators. Specially, we would also not includes the
> InputChannelStates and the ResultPartitionState attached to the operator.
> Then with unaligned checkpoint, there would be the following bad case:
> 1. The task received the first barrier.
> 2. With the process of unaligned checkpoint, the task would snapshot the
> state of the operators.
> 3. The checkpoint would start the asynchronous part.
> 4. Normally in the asynchronous part, it would wait till all the state
> futures get done, including the channel states and result partition states.
> With this method, it ensures the asynchronous part would wait till the last
> barrier arrived. But if the task has been fully finished before, these states
> are ignored and the assumption is broken.
> 5. Then the asynchronous part would fail since when it try to build the
> CheckpointMetrics, the alignment for this checkpoint is in fact not done yet.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] gaoyunhaii opened a new pull request #18112: [FLINK-25305][checkpoint] Always wait for input channel state and result partition state get completed in AsyncRunnable
gaoyunhaii opened a new pull request #18112: URL: https://github.com/apache/flink/pull/18112 ## What is the purpose of the change This PR fixes the issue that when a task finished on restore taking unaligned checkpoints, currently the asynchronous part does not wait till the alignment of barriers get finished, thus throws exception due to some metrics are not set yet. ## Brief change log - 6bb8f5418b1dcd085b16a00f1370a88262d3e4a9 also keeps the channel states for operators finished on restore, and wait for these states get completed explicitly in the asynchronous part. ## Verifying this change This change added tests and can be verified by the added unit test. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: **yes** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-25316) BlobServer can get stuck during shutdown
[
https://issues.apache.org/jira/browse/FLINK-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459683#comment-17459683
]
Robert Metzger edited comment on FLINK-25316 at 12/15/21, 7:02 AM:
---
I reverted the commit from FLINK-24156, but I'm still facing the same issue.
{code}
"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #94 daemon
prio=5 os_prio=0 tid=0x004017d57000 nid=0x2f1 in Object.wait()
[0x00402671e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xd6bba0f8> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1252)
- locked <0xd6bba0f8> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:318)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
- locked <0xd5d27630> (a java.lang.Object)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$shutDownAsync$4(ClusterEntrypoint.java:505)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint$$Lambda$1102/855423197.get(Unknown
Source)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$composeAfterwards$20(FutureUtils.java:728)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$1085/270874580.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$null$19(FutureUtils.java:739)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$1094/602149412.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent.lambda$closeAsyncInternal$2(DispatcherResourceManagerComponent.java:198)
at
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent$$Lambda$1129/841631053.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils$CompletionConjunctFuture.completeFuture(FutureUtils.java:1000)
- locked <0xc1e39828> (a java.lang.Object)
at
org.apache.flink.util.concurrent.FutureUtils$CompletionConjunctFuture$$Lambda$528/169049466.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$forwardTo$24(FutureUtils.java:1372)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$575/1712666248.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$forwardTo$24(FutureUtils.java:1372)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$575/1712666248.accept(Unknown
Source)
at
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * c19158e316f30c51317558d2eead210aca728f55 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25273) Some doubts about the FLINK-22848
[ https://issues.apache.org/jira/browse/FLINK-25273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459691#comment-17459691 ] Jianhui Dong commented on FLINK-25273: -- Thank u. I sent an email about this issue to user-zh mailing list before and did not receive a reply, so I open the ticket here. I will send an email to dev mailing list again. > Some doubts about the FLINK-22848 > - > > Key: FLINK-25273 > URL: https://issues.apache.org/jira/browse/FLINK-25273 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Jianhui Dong >Priority: Major > > I have been in contact with Flink and Calcite for a while, and I have some > questions about this issue: https://issues.apache.org/jira/browse/FLINK-22848. > First of all, the discussion about this issue mentioned that since calcite > did not support the syntax analysis of set a=b (without quotes), regular > expressions were used. However, I checked the commit history some days ago, > and I found that calcite should already support SET syntax parsing (see > SqlSetOption) in v1.14 or even earlier. but its problem is that it would > recognize the `true/false/unknown/null` tokens as keywords, causing the > parsing to be worse than expected, but this problem can be solved by > restricting the syntax, like use '' in the issue FLINK-22848. > Then I investigated the earliest version of flink that introduced calcite, > flink should have introduced Calcite 1.16 in 1.5 at the earliest version. At > that time, calcite should already support the syntax of SET a=b (without > quotes), so I would like to find out What exactly does the "not supported" > mentioned in the issue FLINK-22848 means? Maybe you can give a more specific > case. > In addition, I also have some doubts about the results of the discussion of > the issue. I think it is indeed a more elegant solution to use the SQL parser > to analyze it, but When calcite has built-in support for SET grammar, why do > we need to extend the SET grammar to re-support it? Even this change may > cause backward-incompatible. > In my personal opinion of view, is that we can solve this problem by adding > special restrictions on the above tokens on the basis of native Calcite > analysis, such as in the user manual because values such as `unknown` and > `null` are meaningless in the production environment. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink-ml] lindong28 commented on a change in pull request #24: [FLINK-24557] Add Estimator and Transformer for K-nearest neighbor
lindong28 commented on a change in pull request #24:
URL: https://github.com/apache/flink-ml/pull/24#discussion_r769301266
##
File path:
flink-ml-lib/src/main/java/org/apache/flink/ml/classification/knn/KnnModelData.java
##
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.ml.classification.knn;
+
+import org.apache.flink.api.common.serialization.Encoder;
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.connector.file.src.reader.SimpleStreamFormat;
+import org.apache.flink.core.fs.FSDataInputStream;
+import org.apache.flink.core.memory.DataInputView;
+import org.apache.flink.core.memory.DataInputViewStreamWrapper;
+import org.apache.flink.core.memory.DataOutputView;
+import org.apache.flink.core.memory.DataOutputViewStreamWrapper;
+import org.apache.flink.ml.linalg.DenseMatrix;
+import org.apache.flink.ml.linalg.DenseVector;
+import org.apache.flink.ml.linalg.typeinfo.DenseMatrixSerializer;
+import org.apache.flink.ml.linalg.typeinfo.DenseVectorSerializer;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.table.api.Table;
+import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
+import org.apache.flink.table.api.internal.TableImpl;
+
+import java.io.EOFException;
+import java.io.IOException;
+import java.io.OutputStream;
+
+/**
+ * Model data of {@link KnnModel}.
+ *
+ * This class also provides methods to convert model data from Table to a
data stream, and
+ * classes to save/load model data.
+ */
+public class KnnModelData {
+
+public DenseMatrix packedFeatures;
Review comment:
Can we make it `final`? Same for `featureNormSquares` and `labels`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-25272) Flink sql syntax analysis is inconsistent with the sql grammer declared now
[ https://issues.apache.org/jira/browse/FLINK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianhui Dong closed FLINK-25272. Resolution: Invalid > Flink sql syntax analysis is inconsistent with the sql grammer declared now > --- > > Key: FLINK-25272 > URL: https://issues.apache.org/jira/browse/FLINK-25272 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Jianhui Dong >Priority: Minor > > Syntax analysis in the > code(https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/main/codegen/includes/parserImpls.ftl) > is inconsistent with the grammar declared on the > module(https://github.com/apache/flink/edit/master/docs/content/docs/dev/table/sql/create.md). > The document states that all CREATE statement formats are `CREATE TABLE [IF > NOT EXISTS]`, there is no `TEMPORAL` keywords, but in > flink-sql-parser/src/main/codegen/includes/parserImpls.ftl::SqlCreateExtended, > it will recorganize `TEMPORAL` token and set `isTemporary` with true if the > keyword exists. > Besides, I think the field `isTemporary` is outdated too, every table in > Flink is a temporal table as the document wrote. In fact, when they are > stored in the catalog, they are all converted to `CatalogSchemaTable` which > inherits the `TemporalTable` interface. > BTW, the view and the function have the same condition. IMO, I think we > should remove the `TEMPORAL` token in parserImpls.ftl and also remove the > `isTemporary` field which now always is false. > This issue may be a little confused cause my English is not very well. If you > have any questions, you can add comments under the issue or contact me at > lam...@apache.org and I could also provide a pull request for this. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (FLINK-25272) Flink sql syntax analysis is inconsistent with the sql grammer declared now
[ https://issues.apache.org/jira/browse/FLINK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459687#comment-17459687 ] Jianhui Dong commented on FLINK-25272: -- Thank you for your reply, I'm sorry that I misunderstood these two words. > Flink sql syntax analysis is inconsistent with the sql grammer declared now > --- > > Key: FLINK-25272 > URL: https://issues.apache.org/jira/browse/FLINK-25272 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Reporter: Jianhui Dong >Priority: Minor > > Syntax analysis in the > code(https://github.com/apache/flink/blob/master/flink-table/flink-sql-parser/src/main/codegen/includes/parserImpls.ftl) > is inconsistent with the grammar declared on the > module(https://github.com/apache/flink/edit/master/docs/content/docs/dev/table/sql/create.md). > The document states that all CREATE statement formats are `CREATE TABLE [IF > NOT EXISTS]`, there is no `TEMPORAL` keywords, but in > flink-sql-parser/src/main/codegen/includes/parserImpls.ftl::SqlCreateExtended, > it will recorganize `TEMPORAL` token and set `isTemporary` with true if the > keyword exists. > Besides, I think the field `isTemporary` is outdated too, every table in > Flink is a temporal table as the document wrote. In fact, when they are > stored in the catalog, they are all converted to `CatalogSchemaTable` which > inherits the `TemporalTable` interface. > BTW, the view and the function have the same condition. IMO, I think we > should remove the `TEMPORAL` token in parserImpls.ftl and also remove the > `isTemporary` field which now always is false. > This issue may be a little confused cause my English is not very well. If you > have any questions, you can add comments under the issue or contact me at > lam...@apache.org and I could also provide a pull request for this. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-24557) Support Knn in FlinkML
[ https://issues.apache.org/jira/browse/FLINK-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-24557: --- Labels: pull-request-available (was: ) > Support Knn in FlinkML > -- > > Key: FLINK-24557 > URL: https://issues.apache.org/jira/browse/FLINK-24557 > Project: Flink > Issue Type: Bug > Components: Library / Machine Learning >Reporter: weibo zhao >Priority: Major > Labels: pull-request-available > > Support Knn in FlinkML -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink-ml] lindong28 commented on pull request #24: [FLINK-24557] Add Estimator and Transformer for K-nearest neighbor
lindong28 commented on pull request #24: URL: https://github.com/apache/flink-ml/pull/24#issuecomment-994379627 Thanks for the update! The PR LGTM overall. I have only one question as specified in the unresolved comment. Thank you and @zhipeng93 both for all the efforts improving the code readability (e.g. variable names). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-ml] lindong28 edited a comment on pull request #24: [FLINK-24557] Add Estimator and Transformer for K-nearest neighbor
lindong28 edited a comment on pull request #24: URL: https://github.com/apache/flink-ml/pull/24#issuecomment-994379627 Thanks for the update! The PR LGTM overall. I have only one question as specified in the unresolved comment. Thank you and @zhipeng93 both for all the efforts improving the code readability (e.g. variable names). @gaoyunhaii could you help review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) * 773e747be9b0637e143f0c09e6fcb09e50df3cf6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] xiangqiao123 commented on a change in pull request #18023: [FLINK-25032] Allow to create execution vertices and execution edges lazily
xiangqiao123 commented on a change in pull request #18023:
URL: https://github.com/apache/flink/pull/18023#discussion_r769297327
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/ExecutionJobVertex.java
##
@@ -307,13 +323,21 @@ public JobVertexID getJobVertexId() {
@Override
public ExecutionVertex[] getTaskVertices() {
+if (taskVertices == null) {
Review comment:
Why not use `checkState(isInitialized()); ` to check whether
ExecutionJobVertex is initialized?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-25316) BlobServer can get stuck during shutdown
[
https://issues.apache.org/jira/browse/FLINK-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459683#comment-17459683
]
Robert Metzger commented on FLINK-25316:
I reverted the commit from FLINK-24156, but I'm still facing the same issue.
{code}
"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #94 daemon
prio=5 os_prio=0 tid=0x004017d57000 nid=0x2f1 in Object.wait()
[0x00402671e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xd6bba0f8> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1252)
- locked <0xd6bba0f8> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:318)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
- locked <0xd5d27630> (a java.lang.Object)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$shutDownAsync$4(ClusterEntrypoint.java:505)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint$$Lambda$1102/855423197.get(Unknown
Source)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$composeAfterwards$20(FutureUtils.java:728)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$1085/270874580.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$null$19(FutureUtils.java:739)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$1094/602149412.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent.lambda$closeAsyncInternal$2(DispatcherResourceManagerComponent.java:198)
at
org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent$$Lambda$1129/841631053.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils$CompletionConjunctFuture.completeFuture(FutureUtils.java:1000)
- locked <0xc1e39828> (a java.lang.Object)
at
org.apache.flink.util.concurrent.FutureUtils$CompletionConjunctFuture$$Lambda$528/169049466.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$forwardTo$24(FutureUtils.java:1372)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$575/1712666248.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$forwardTo$24(FutureUtils.java:1372)
at
org.apache.flink.util.concurrent.FutureUtils$$Lambda$575/1712666248.accept(Unknown
Source)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
[jira] [Commented] (FLINK-24948) Special character in column names breaks JDBC statement parsing
[ https://issues.apache.org/jira/browse/FLINK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459681#comment-17459681 ] Paul Lin commented on FLINK-24948: -- [~jark] I've prepared a PR, PTAL. Thanks! > Special character in column names breaks JDBC statement parsing > --- > > Key: FLINK-24948 > URL: https://issues.apache.org/jira/browse/FLINK-24948 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.12.4 >Reporter: Paul Lin >Assignee: Paul Lin >Priority: Major > Labels: pull-request-available > > Currently, JDBC connector assumes columns names respect Java identifier > naming restrictions, but Databases that support JDBC may have different > naming restrictions. For example, MySQL allows dots and colons in column > names. In that case, JDBC connector would have trouble parsing the SQL. > We could fix this by validating field names in `JdbcDmlOptions`. In addition, > it'd be good to clarify the naming restrictions of Flink SQL, so users and > connector developers would know the standard. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28146) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18007: [FLINK-25085][runtime] Add a scheduled thread pool in Endpoint and close it when the endpoint is stopped
flinkbot edited a comment on pull request #18007: URL: https://github.com/apache/flink/pull/18007#issuecomment-985453331 ## CI report: * c89e806d14e0030c662ba18f058b3993253ff22d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28086) * efaace940767ea9dd2a1436079828a9ee57db912 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17991: [FLINK-25132][connector/kafka] Move record deserializing from SplitFetcher to RecordEmitter to support object-reusing deserializer
flinkbot edited a comment on pull request #17991: URL: https://github.com/apache/flink/pull/17991#issuecomment-984442302 ## CI report: * d541852268be83515afb4bccb19f456ea732acea Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27942) * 08212a397f81e4024e27e3a3421796b56e394ed3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28145) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17991: [FLINK-25132][connector/kafka] Move record deserializing from SplitFetcher to RecordEmitter to support object-reusing deserializer
flinkbot edited a comment on pull request #17991: URL: https://github.com/apache/flink/pull/17991#issuecomment-984442302 ## CI report: * d541852268be83515afb4bccb19f456ea732acea Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27942) * 08212a397f81e4024e27e3a3421796b56e394ed3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25316) BlobServer can get stuck during shutdown
[
https://issues.apache.org/jira/browse/FLINK-25316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger updated FLINK-25316:
---
Description:
The cluster shutdown can get stuck
{code}
"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #89 daemon
prio=5 os_prio=0 tid=0x004017d7 nid=0x2ec in Object.wait()
[0x00402a9b5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1252)
- locked <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:319)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
- locked <0xd5d27350> (a java.lang.Object)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$shutDownAsync$4(ClusterEntrypoint.java:505
{code}
because the BlobServer.run() method ignores interrupts:
{code}
"BLOB Server listener at 6124" #30 daemon prio=5 os_prio=0
tid=0x00401c929800 nid=0x2b4 runnable [0x0040263f9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:560)
at java.net.ServerSocket.accept(ServerSocket.java:528)
at
org.apache.flink.util.NetUtils.acceptWithoutTimeout(NetUtils.java:143)
at org.apache.flink.runtime.blob.BlobServer.run(BlobServer.java:268)
{code}
This issue was introduced in FLINK-24156 and first mentioned in
https://issues.apache.org/jira/browse/FLINK-24113?focusedCommentId=17459414=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17459414
was:
The cluster shutdown can get stuck
{code}
"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #89 daemon
prio=5 os_prio=0 tid=0x004017d7 nid=0x2ec in Object.wait()
[0x00402a9b5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1252)
- locked <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:319)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
- locked <0xd5d27350> (a java.lang.Object)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$shutDownAsync$4(ClusterEntrypoint.java:505
{code}
because the BlobServer.run() method ignores interrupts:
{code}
"BLOB Server listener at 6124" #30 daemon prio=5 os_prio=0
tid=0x00401c929800 nid=0x2b4 runnable [0x0040263f9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:560)
at java.net.ServerSocket.accept(ServerSocket.java:528)
at
org.apache.flink.util.NetUtils.acceptWithoutTimeout(NetUtils.java:143)
at org.apache.flink.runtime.blob.BlobServer.run(BlobServer.java:268)
{code}
This issue was introduced in FLINK-24156.
> BlobServer can get stuck during shutdown
>
>
> Key: FLINK-25316
> URL: https://issues.apache.org/jira/browse/FLINK-25316
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Robert Metzger
>Priority: Critical
> Fix For: 1.15.0
>
>
> The cluster shutdown can get stuck
> {code}
> "AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #89 daemon
> prio=5 os_prio=0 tid=0x004017d7 nid=0x2ec in Object.wait()
> [0x00402a9b5000]
>java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xd6c48368> (a
> org.apache.flink.runtime.blob.BlobServer)
> at java.lang.Thread.join(Thread.java:1252)
> - locked <0xd6c48368> (a
> org.apache.flink.runtime.blob.BlobServer)
> at java.lang.Thread.join(Thread.java:1326)
> at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:319)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
> -
[jira] [Commented] (FLINK-24113) Introduce option in Application Mode to disable shutdown
[ https://issues.apache.org/jira/browse/FLINK-24113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459674#comment-17459674 ] Robert Metzger commented on FLINK-24113: I filed a separate ticket https://issues.apache.org/jira/browse/FLINK-25316 > Introduce option in Application Mode to disable shutdown > > > Key: FLINK-24113 > URL: https://issues.apache.org/jira/browse/FLINK-24113 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.15.0 >Reporter: Robert Metzger >Assignee: Nicolaus Weidner >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > Attachments: shutdown_issue.log > > > Currently a Flink JobManager started in Application Mode will shut down once > the job has completed. > When doing a "stop with savepoint" operation, we want to keep the JobManager > alive after the job has stopped to retrieve and persist the final savepoint > location. > Currently, Flink waits up to 5 minutes and then shuts down. > I'm proposing to introduce a new configuration flag "application mode > shutdown behavior": "keepalive" (naming things is hard ;) ) which will keep > the JobManager in ApplicationMode running until a REST call confirms that it > can shutdown. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25316) BlobServer can get stuck during shutdown
Robert Metzger created FLINK-25316:
--
Summary: BlobServer can get stuck during shutdown
Key: FLINK-25316
URL: https://issues.apache.org/jira/browse/FLINK-25316
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.15.0
Reporter: Robert Metzger
Fix For: 1.15.0
The cluster shutdown can get stuck
{code}
"AkkaRpcService-Supervisor-Termination-Future-Executor-thread-1" #89 daemon
prio=5 os_prio=0 tid=0x004017d7 nid=0x2ec in Object.wait()
[0x00402a9b5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1252)
- locked <0xd6c48368> (a
org.apache.flink.runtime.blob.BlobServer)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.flink.runtime.blob.BlobServer.close(BlobServer.java:319)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.stopClusterServices(ClusterEntrypoint.java:406)
- locked <0xd5d27350> (a java.lang.Object)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$shutDownAsync$4(ClusterEntrypoint.java:505
{code}
because the BlobServer.run() method ignores interrupts:
{code}
"BLOB Server listener at 6124" #30 daemon prio=5 os_prio=0
tid=0x00401c929800 nid=0x2b4 runnable [0x0040263f9000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:560)
at java.net.ServerSocket.accept(ServerSocket.java:528)
at
org.apache.flink.util.NetUtils.acceptWithoutTimeout(NetUtils.java:143)
at org.apache.flink.runtime.blob.BlobServer.run(BlobServer.java:268)
{code}
This issue was introduced in FLINK-24156.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17711: [FLINK-24737][runtime-web] Update outdated web dependencies
flinkbot edited a comment on pull request #17711: URL: https://github.com/apache/flink/pull/17711#issuecomment-962917757 ## CI report: * edcadd9f732f4717bf5043b4c45c97e461e4e11b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26195) * 15f87a534f7515d99eaed71344c88cddbdf4f1ac Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28144) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] yangjunhan commented on pull request #17711: [FLINK-24737][runtime-web] Update outdated web dependencies
yangjunhan commented on pull request #17711: URL: https://github.com/apache/flink/pull/17711#issuecomment-994350770 > Can you have a look at the failing build and fix it? @MartijnVisser Conflicts have been addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17711: [FLINK-24737][runtime-web] Update outdated web dependencies
flinkbot edited a comment on pull request #17711: URL: https://github.com/apache/flink/pull/17711#issuecomment-962917757 ## CI report: * edcadd9f732f4717bf5043b4c45c97e461e4e11b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26195) * 15f87a534f7515d99eaed71344c88cddbdf4f1ac UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16108: [FLINK-22821][core] Stabilize NetUtils#getAvailablePort in order to avoid wrongly allocating any used ports
flinkbot edited a comment on pull request #16108: URL: https://github.com/apache/flink/pull/16108#issuecomment-856641884 ## CI report: * 6cc9790481b8459c816d5750b08d8c8c4d5bbd26 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=27920) * 03e33a13e8826f6cda49070c16679533f1223e94 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28137) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] paul8263 commented on pull request #16108: [FLINK-22821][core] Stabilize NetUtils#getAvailablePort in order to avoid wrongly allocating any used ports
paul8263 commented on pull request #16108: URL: https://github.com/apache/flink/pull/16108#issuecomment-994342748 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-25315) Add some extensions and utils to help the Junit5 migration
Hang Ruan created FLINK-25315: - Summary: Add some extensions and utils to help the Junit5 migration Key: FLINK-25315 URL: https://issues.apache.org/jira/browse/FLINK-25315 Project: Flink Issue Type: Technical Debt Components: Tests Reporter: Hang Ruan Add some extensions and utils to help the Junit5 migration. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18088: [FLINK-25174][table] Introduce managed table interfaces and callback
flinkbot edited a comment on pull request #18088: URL: https://github.com/apache/flink/pull/18088#issuecomment-992202380 ## CI report: * 5288bcedbae945ad9d8e310850865a0533a002c3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28139) * 7660ffb0e155a244663d6ac0c93576a893a293aa Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28140) * 2d0504a97f7125e2bc2cb88989fd9de07344ed62 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28143) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-25231) Update PyFlink to use the new type system
[ https://issues.apache.org/jira/browse/FLINK-25231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459666#comment-17459666 ] Dian Fu commented on FLINK-25231: - [~csq] Sure. Thanks for taking this ticket. Have assigned it to you~ > Update PyFlink to use the new type system > - > > Key: FLINK-25231 > URL: https://issues.apache.org/jira/browse/FLINK-25231 > Project: Flink > Issue Type: Improvement > Components: API / Python >Reporter: Dian Fu >Assignee: Shuiqiang Chen >Priority: Major > Fix For: 1.15.0 > > > Currently, there are a lot of places in PyFlink Table API still using the > legacy type system. We need to revisit this and migrate them to the new type > system(DataType/LogicalType). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #18088: [FLINK-25174][table] Introduce managed table interfaces and callback
flinkbot edited a comment on pull request #18088: URL: https://github.com/apache/flink/pull/18088#issuecomment-992202380 ## CI report: * 5288bcedbae945ad9d8e310850865a0533a002c3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28139) * 7660ffb0e155a244663d6ac0c93576a893a293aa Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28140) * 2d0504a97f7125e2bc2cb88989fd9de07344ed62 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-25231) Update PyFlink to use the new type system
[ https://issues.apache.org/jira/browse/FLINK-25231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dian Fu reassigned FLINK-25231: --- Assignee: Shuiqiang Chen > Update PyFlink to use the new type system > - > > Key: FLINK-25231 > URL: https://issues.apache.org/jira/browse/FLINK-25231 > Project: Flink > Issue Type: Improvement > Components: API / Python >Reporter: Dian Fu >Assignee: Shuiqiang Chen >Priority: Major > Fix For: 1.15.0 > > > Currently, there are a lot of places in PyFlink Table API still using the > legacy type system. We need to revisit this and migrate them to the new type > system(DataType/LogicalType). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot edited a comment on pull request #17919: [FLINK-24419][table-planner] Trim to precision when casting to BINARY/VARBINARY
flinkbot edited a comment on pull request #17919: URL: https://github.com/apache/flink/pull/17919#issuecomment-979659835 ## CI report: * 0a89d73af633e62f39b9ba728b0551a95a28eaba Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28069) * 7dca4860db2cdd790f30a7f38556a850280a00fb Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28142) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17919: [FLINK-24419][table-planner] Trim to precision when casting to BINARY/VARBINARY
flinkbot edited a comment on pull request #17919: URL: https://github.com/apache/flink/pull/17919#issuecomment-979659835 ## CI report: * 0a89d73af633e62f39b9ba728b0551a95a28eaba Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28069) * 7dca4860db2cdd790f30a7f38556a850280a00fb UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18088: [FLINK-25174][table] Introduce managed table interfaces and callback
flinkbot edited a comment on pull request #18088: URL: https://github.com/apache/flink/pull/18088#issuecomment-992202380 ## CI report: * 5288bcedbae945ad9d8e310850865a0533a002c3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28139) * 7660ffb0e155a244663d6ac0c93576a893a293aa Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28140) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18110: [FLINK-25310][Documentation] Fix the incorrect description for output network buffers in buffer debloat documenation
flinkbot edited a comment on pull request #18110: URL: https://github.com/apache/flink/pull/18110#issuecomment-994244366 ## CI report: * c7ce675ebbb98339bc05e357623ced593df8efa3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28138) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] ljz2051 commented on pull request #18111: [FLINK-25094][Test]Correct the LatencyTrackingMapStateTest#verifyIterator
ljz2051 commented on pull request #18111: URL: https://github.com/apache/flink/pull/18111#issuecomment-994287269 @Myasuka Could you review this change for LatencyTrackingMapStateTest when you are free? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-25159) Streamline E2E surefire configuration setup
[ https://issues.apache.org/jira/browse/FLINK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459641#comment-17459641 ] Yufei Zhang edited comment on FLINK-25159 at 12/15/21, 4:40 AM: [~chesnay] Hi Chesnay, do you know how to configure the pipeline to run only one e2e tests ? I'm searching to run only pulsar e2e tests on pipeline. Thank you ~ EDIT: I currently only include puslar e2e in my testing pipeline in the surefire plugin, but wondering if there is a better way to do it EDIT2: also wondering what is the difference between e2e_1_ci_build and e2e_2_ci_build in the pipeline was (Author: affe): [~chesnay] Hi Chesnay, do you know how to configure the pipeline to run only one e2e tests ? I'm searching to run only pulsar e2e tests on pipeline. Thank you ~ EDIT: I currently only include puslar e2e in my testing pipeline in the surefire plugin, but wondering if there is a better way to do it > Streamline E2E surefire configuration setup > --- > > Key: FLINK-25159 > URL: https://issues.apache.org/jira/browse/FLINK-25159 > Project: Flink > Issue Type: Technical Debt > Components: Build System, Tests >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > Our current surefire setup for the Java E2E tests is exceedingly complicated, > because of when it was introduced we split the e2e tests across separate > profiles. > This is at least currently not necessary. > Additionally, it appears that not many people are aware of this setup, which > has resulted in various tests for pulsar, pubsub, kinesis and kafka (and > maybe others) to not even being run on CI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot commented on pull request #18111: [FLINK-25094][Test]Correct the LatencyTrackingMapStateTest#verifyIterator
flinkbot commented on pull request #18111: URL: https://github.com/apache/flink/pull/18111#issuecomment-994286257 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 930b0c12d7dad620b6e15c23289f2e2666ce0ed0 (Wed Dec 15 04:39:38 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18111: [FLINK-25094][Test]Correct the LatencyTrackingMapStateTest#verifyIterator
flinkbot edited a comment on pull request #18111: URL: https://github.com/apache/flink/pull/18111#issuecomment-994285128 ## CI report: * 930b0c12d7dad620b6e15c23289f2e2666ce0ed0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=28141) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-25159) Streamline E2E surefire configuration setup
[ https://issues.apache.org/jira/browse/FLINK-25159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17459641#comment-17459641 ] Yufei Zhang edited comment on FLINK-25159 at 12/15/21, 4:37 AM: [~chesnay] Hi Chesnay, do you know how to configure the pipeline to run only one e2e tests ? I'm searching to run only pulsar e2e tests on pipeline. Thank you ~ EDIT: I currently only include puslar e2e in my testing pipeline in the surefire plugin, but wondering if there is a better way to do it was (Author: affe): [~chesnay] Hi Chesnay, do you know how to configure the pipeline to run only one e2e tests ? I'm searching to run only pulsar e2e tests on pipeline. Thank you ~ > Streamline E2E surefire configuration setup > --- > > Key: FLINK-25159 > URL: https://issues.apache.org/jira/browse/FLINK-25159 > Project: Flink > Issue Type: Technical Debt > Components: Build System, Tests >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > Our current surefire setup for the Java E2E tests is exceedingly complicated, > because of when it was introduced we split the e2e tests across separate > profiles. > This is at least currently not necessary. > Additionally, it appears that not many people are aware of this setup, which > has resulted in various tests for pulsar, pubsub, kinesis and kafka (and > maybe others) to not even being run on CI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [flink] flinkbot commented on pull request #18111: [FLINK-25094][Test]Correct the LatencyTrackingMapStateTest#verifyIter…
flinkbot commented on pull request #18111: URL: https://github.com/apache/flink/pull/18111#issuecomment-994285128 ## CI report: * 930b0c12d7dad620b6e15c23289f2e2666ce0ed0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-25094) The verify code in LatencyTrackingMapStateTest#verifyIterator is not actually executed
[
https://issues.apache.org/jira/browse/FLINK-25094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-25094:
---
Labels: pull-request-available (was: )
> The verify code in LatencyTrackingMapStateTest#verifyIterator is not actually
> executed
> --
>
> Key: FLINK-25094
> URL: https://issues.apache.org/jira/browse/FLINK-25094
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends, Tests
>Reporter: Jinzhong Li
>Assignee: Jinzhong Li
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> In LatencyTrackingMapStateTest,
> iterator()/entries().iterator()/keys().iterator()/values().iterator() will be
> invoke before verifyIterator method is invoked, this is,
> iterator()/... will be invode before putting the test data into
> latencyTrackingMapState. So the verify code is not actually executed since
> "iterator.hasNext()" is always false.
> {code:java}
> private void verifyIterator(
> LatencyTrackingMapState
> latencyTrackingState,
> LatencyTrackingMapState.MapStateLatencyMetrics
> latencyTrackingStateMetric,
> Iterator iterator,
> boolean removeIterator)
> throws Exception {
> ThreadLocalRandom random = ThreadLocalRandom.current();
> for (int index = 1; index <= SAMPLE_INTERVAL; index++) {
> latencyTrackingState.put((long) index, random.nextDouble());
> }
> int count = 1;
> while (iterator.hasNext()) {
> int expectedResult = count == SAMPLE_INTERVAL ? 0 : count;
> assertEquals(expectedResult,
> latencyTrackingStateMetric.getIteratorHasNextCount());
> iterator.next();
> assertEquals(expectedResult,
> latencyTrackingStateMetric.getIteratorNextCount());
> if (removeIterator) {
> iterator.remove();
> assertEquals(expectedResult,
> latencyTrackingStateMetric.getIteratorRemoveCount());
> }
> count += 1;
> }
> // as we call #hasNext on more time than #next, to avoid complex check,
> just reset hasNext
> // counter in the end.
> latencyTrackingStateMetric.resetIteratorHasNextCount();
> latencyTrackingState.clear();
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
