[jira] [Closed] (FLINK-33052) codespeed and benchmark server is down

2023-10-19 Thread Zakelly Lan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zakelly Lan closed FLINK-33052. --- Resolution: Fixed > codespeed and benchmark server is down > --

[jira] [Comment Edited] (FLINK-33316) Avoid unnecessary heavy getStreamOperatorFactory

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1554#comment-1554 ] Rui Fan edited comment on FLINK-33316 at 10/20/23 3:51 AM: --- Thanks for your

[jira] [Commented] (FLINK-33316) Avoid unnecessary heavy getStreamOperatorFactory

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1554#comment-1554 ] Rui Fan commented on FLINK-33316: - Thanks for your explanation! I will cherry-pick them and don't block

[jira] [Commented] (FLINK-33316) Avoid unnecessary heavy getStreamOperatorFactory

2023-10-19 Thread Dong Lin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1552#comment-1552 ] Dong Lin commented on FLINK-33316: -- [~fanrui] I think it makes sense to cherry-pick this to the next

Re: [PR] [FLINK-21883][scheduler] Implement cooldown period for adaptive scheduler [flink]

2023-10-19 Thread via GitHub
1996fanrui commented on code in PR #22985: URL: https://github.com/apache/flink/pull/22985#discussion_r1366420311 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/Executing.java: ## @@ -67,13 +77,33 @@ class Executing extends StateWithExecutionGraph

Re: [PR] [FLINK-33313][table] fix RexNodeExtractor#extractConjunctiveConditions throws an Exception when handle binary literal [flink]

2023-10-19 Thread via GitHub
libenchao commented on code in PR #23551: URL: https://github.com/apache/flink/pull/23551#discussion_r1365541368 ## flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/RexNodeExtractor.scala: ## @@ -502,6 +503,9 @@ class

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
clarax commented on PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1772027448 Refactored as @gyfora suggusted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-32073][checkpoint] Implement file merging in snapshot [flink]

2023-10-19 Thread via GitHub
fredia commented on code in PR #23514: URL: https://github.com/apache/flink/pull/23514#discussion_r1366385484 ## flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FileMergingCheckpointStateOutputStream.java: ## @@ -0,0 +1,255 @@ +/* + * Licensed to the

Re: [PR] [FLINK-21883][scheduler] Implement cooldown period for adaptive scheduler [flink]

2023-10-19 Thread via GitHub
1996fanrui commented on code in PR #22985: URL: https://github.com/apache/flink/pull/22985#discussion_r1366403842 ## flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptive/ExecutingTest.java: ## @@ -252,28 +259,94 @@ public void

Re: [PR] [FLINK-21883][scheduler] Implement cooldown period for adaptive scheduler [flink]

2023-10-19 Thread via GitHub
1996fanrui commented on code in PR #22985: URL: https://github.com/apache/flink/pull/22985#discussion_r1366402713 ## flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptive/AdaptiveScheduler.java: ## @@ -1162,16 +1167,24 @@ private ExecutionGraph

Re: [PR] [FLINK-33318] Expose aggregated collector metrics and measure timeMsPerSecond [flink]

2023-10-19 Thread via GitHub
1996fanrui commented on code in PR #23554: URL: https://github.com/apache/flink/pull/23554#discussion_r1366386337 ## flink-runtime/src/main/java/org/apache/flink/runtime/metrics/util/MetricUtils.java: ## @@ -222,16 +224,32 @@ private static void

[jira] [Commented] (FLINK-33316) Avoid unnecessary heavy getStreamOperatorFactory

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1547#comment-1547 ] Rui Fan commented on FLINK-33316: - Hi [~lindong] , this extra memory cost is introduced since 1.17, and

[jira] [Updated] (FLINK-33316) Avoid unnecessary heavy getStreamOperatorFactory

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan updated FLINK-33316: Fix Version/s: 1.19.0 > Avoid unnecessary heavy getStreamOperatorFactory >

[jira] [Updated] (FLINK-33322) ProjectWatermarkAssignerTransposeRule support nested nested projection push down in transpose

2023-10-19 Thread lijingwei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lijingwei updated FLINK-33322: -- Description: ProjectWatermarkAssignerTransposeRule support nested projection push down in transpose

[jira] [Assigned] (FLINK-33322) ProjectWatermarkAssignerTransposeRule support nested nested projection push down in transpose

2023-10-19 Thread Benchao Li (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benchao Li reassigned FLINK-33322: -- Assignee: lijingwei > ProjectWatermarkAssignerTransposeRule support nested nested projection

[jira] [Updated] (FLINK-33322) ProjectWatermarkAssignerTransposeRule support nested nested projection push down in transpose

2023-10-19 Thread lijingwei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lijingwei updated FLINK-33322: -- Description: ProjectWatermarkAssignerTransposeRule support nested nested projection push down in

[jira] [Created] (FLINK-33322) ProjectWatermarkAssignerTransposeRule support nested nested projection push down in transpose

2023-10-19 Thread lijingwei (Jira)
lijingwei created FLINK-33322: - Summary: ProjectWatermarkAssignerTransposeRule support nested nested projection push down in transpose Key: FLINK-33322 URL: https://issues.apache.org/jira/browse/FLINK-33322

[jira] [Commented] (FLINK-33173) Support currentFetchEventTimeLag metrics for KafkaSource

2023-10-19 Thread tanjialiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1535#comment-1535 ] tanjialiang commented on FLINK-33173: - [~martijnvisser] It's my pleasure, i will try it. > Support

[jira] [Commented] (FLINK-33320) Support Dynamic Logger Level Adjustment

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1527#comment-1527 ] Rui Fan commented on FLINK-33320: - The FLIP is necessary due to some of new rest apis are introduced. >

Re: [PR] [FLINK-33171][table planner] Consistent implicit type coercion support for equal and non-equal comparisons for codegen [flink]

2023-10-19 Thread via GitHub
fengjiajie commented on PR #23478: URL: https://github.com/apache/flink/pull/23478#issuecomment-1771947970 > Thanks for the update, looks good to me in general, and I just left some minor comments. Thank you for the suggestion. It has been modified. -- This is an automated message

[jira] [Commented] (FLINK-33320) Support Dynamic Logger Level Adjustment

2023-10-19 Thread RocMarshal (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1522#comment-1522 ] RocMarshal commented on FLINK-33320: Hi, [~fanrui] Thank you for the assign~ I'm willing to

[jira] [Commented] (FLINK-32716) Give 'Default'(or maybe 'None') option for 'scheduler-mode'

2023-10-19 Thread Yao Zhang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1520#comment-1520 ] Yao Zhang commented on FLINK-32716: --- Hi [~kination] , The issue ID in your commit message is

[jira] [Commented] (FLINK-32721) agg max/min supports char type

2023-10-19 Thread Yao Zhang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1518#comment-1518 ] Yao Zhang commented on FLINK-32721: --- Hi [~jackylau], Are you still working on this? Based on the

Re: [PR] Draft: FLINK-28229 (CI) [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23558: URL: https://github.com/apache/flink/pull/23558#issuecomment-1771818640 ## CI report: * 0fb5ebc2b1f72507b30c5d83488956b2756d08d1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] Draft: FLINK-28229 (CI) [flink]

2023-10-19 Thread via GitHub
afedulov closed pull request #23557: Draft: FLINK-28229 (CI) URL: https://github.com/apache/flink/pull/23557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] Draft: FLINK-28229 (CI) [flink]

2023-10-19 Thread via GitHub
afedulov opened a new pull request, #23558: URL: https://github.com/apache/flink/pull/23558 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Draft: FLINK-28229 (CI) [flink]

2023-10-19 Thread via GitHub
afedulov commented on PR #23557: URL: https://github.com/apache/flink/pull/23557#issuecomment-1771814807 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Draft: FLINK-28050 (CI) [flink]

2023-10-19 Thread via GitHub
afedulov opened a new pull request, #23557: URL: https://github.com/apache/flink/pull/23557 [FLINK-28050][connectors] Move DataGeneratorSource tests into a separate module -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [FLINK-32812][Connectors/Hbase]Fix HBaseRowDataLookupFunction HTable instantiation of thread safety problems [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #23170: [FLINK-32812][Connectors/Hbase]Fix HBaseRowDataLookupFunction HTable instantiation of thread safety problems URL: https://github.com/apache/flink/pull/23170 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [FLINK-32812][Connectors/Hbase]Fix HBaseRowDataLookupFunction HTable instantiation of thread safety problems [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #23170: URL: https://github.com/apache/flink/pull/23170#issuecomment-1771770285 @SunShine-1204 thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [FLINK-28950][HBase] Disable HBase2 version check in hbase-default.xml [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #20562: [FLINK-28950][HBase] Disable HBase2 version check in hbase-default.xml URL: https://github.com/apache/flink/pull/20562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-28950][HBase] Disable HBase2 version check in hbase-default.xml [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #20562: URL: https://github.com/apache/flink/pull/20562#issuecomment-1771768681 @xicm thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [hotfix] [connectors/hbase] Fix equal method of HBaseWriteOptions [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #19300: URL: https://github.com/apache/flink/pull/19300#issuecomment-1771767955 @LB-Yu thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [hotfix] [connectors/hbase] Fix equal method of HBaseWriteOptions [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #19300: [hotfix] [connectors/hbase] Fix equal method of HBaseWriteOptions URL: https://github.com/apache/flink/pull/19300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] [FLINK-26601][Connector/HBase]Flink HBase HBaseRowDataAsyncLookupFunc… [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #19068: URL: https://github.com/apache/flink/pull/19068#issuecomment-1771766356 @zhisheng17 thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [FLINK-26601][Connector/HBase]Flink HBase HBaseRowDataAsyncLookupFunc… [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #19068: [FLINK-26601][Connector/HBase]Flink HBase HBaseRowDataAsyncLookupFunc… URL: https://github.com/apache/flink/pull/19068 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-23719] [connectors/hbase] Support switch WAL in Flink SQL DDL options for HBase sink [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #16859: [FLINK-23719] [connectors/hbase] Support switch WAL in Flink SQL DDL options for HBase sink URL: https://github.com/apache/flink/pull/16859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] [FLINK-23719] [connectors/hbase] Support switch WAL in Flink SQL DDL options for HBase sink [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #16859: URL: https://github.com/apache/flink/pull/16859#issuecomment-1771764232 @LB-Yu thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [FLINK-19647][Connectors/HBase]Support the limit push down for the hb… [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #15847: [FLINK-19647][Connectors/HBase]Support the limit push down for the hb… URL: https://github.com/apache/flink/pull/15847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [FLINK-19647][Connectors/HBase]Support the limit push down for the hb… [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #15847: URL: https://github.com/apache/flink/pull/15847#issuecomment-1771763214 @shouweikun thanks for your contribution The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [FLINK-17874][Connectors/HBase]Handling the NPE for hbase-connector [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #12289: [FLINK-17874][Connectors/HBase]Handling the NPE for hbase-connector URL: https://github.com/apache/flink/pull/12289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [FLINK-28050][connectors] Migrate StreamExecutionEnvironment#fromElements() implementation to FLIP-27 Source API [flink]

2023-10-19 Thread via GitHub
afedulov commented on PR #23553: URL: https://github.com/apache/flink/pull/23553#issuecomment-1771760579 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-17874][Connectors/HBase]Handling the NPE for hbase-connector [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #12289: URL: https://github.com/apache/flink/pull/12289#issuecomment-1771760570 @cyq89051127 The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

[jira] [Updated] (FLINK-8862) Support HBase snapshot read

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-8862: -- Labels: auto-deprioritized-major auto-deprioritized-minor pull-request-available (was:

Re: [PR] [FLINK-8862] [HBase] Support HBase snapshot read [flink]

2023-10-19 Thread via GitHub
snuyanzin closed pull request #5639: [FLINK-8862] [HBase] Support HBase snapshot read URL: https://github.com/apache/flink/pull/5639 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [FLINK-8862] [HBase] Support HBase snapshot read [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on PR #5639: URL: https://github.com/apache/flink/pull/5639#issuecomment-1771758946 @neoremind The Flink HBase connector resides in it's own repository nowadays, if this code change is still relevant, please open the PR in

Re: [PR] [FLINK-33321] VertexFlameGraphFactoryTest#verifyRecursively fails on Java 21 [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23556: URL: https://github.com/apache/flink/pull/23556#issuecomment-1771751473 ## CI report: * 7482d599e20db3eb60eb7a10608e0329dc489411 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[jira] [Updated] (FLINK-33321) VertexFlameGraphFactoryTest#verifyRecursively fails on Java 21

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33321: --- Labels: pull-request-available (was: ) > VertexFlameGraphFactoryTest#verifyRecursively

[PR] [FLINK-33321] VertexFlameGraphFactoryTest#verifyRecursively fails on Java 21 [flink]

2023-10-19 Thread via GitHub
snuyanzin opened a new pull request, #23556: URL: https://github.com/apache/flink/pull/23556 ## What is the purpose of the change in java 21 it was changed representation of lambda in stack trace which is used in current test The PR makes the test aware of it ## Verifying

Re: [PR] [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec… [flink]

2023-10-19 Thread via GitHub
yigress commented on code in PR #23425: URL: https://github.com/apache/flink/pull/23425#discussion_r1366104698 ## flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java: ## @@ -109,6 +109,20 @@ public class CheckpointingOptions {

Re: [PR] [FLINK-25857] Add committer metrics to track the status of committables [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23555: URL: https://github.com/apache/flink/pull/23555#issuecomment-1771656433 ## CI report: * 294b2c1c8f6d7fe0c725bd80962db2eb0c7818a4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#issuecomment-1771652868 Sorry @clarax I think I gave confusing feedback and didn't express myself clearly. I was thinking something like this: ``` interface AutoScalerEventHandler {

[jira] [Updated] (FLINK-25857) Add committer metrics to track the status of committables

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-25857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-25857: --- Labels: pull-request-available (was: ) > Add committer metrics to track the status of

[PR] [FLINK-25857] Add committer metrics to track the status of committables [flink]

2023-10-19 Thread via GitHub
pvary opened a new pull request, #23555: URL: https://github.com/apache/flink/pull/23555 ## What is the purpose of the change This is the implementation of main part of [FLIP-371: Provide initialization context for Committer creation in

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
clarax commented on code in PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1366039222 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java: ## @@ -33,19 +35,15 @@ @Experimental public

[jira] [Created] (FLINK-33321) VertexFlameGraphFactoryTest#verifyRecursively fails on Java 21

2023-10-19 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33321: --- Summary: VertexFlameGraphFactoryTest#verifyRecursively fails on Java 21 Key: FLINK-33321 URL: https://issues.apache.org/jira/browse/FLINK-33321 Project: Flink

Re: [PR] [FLINK-28050][connectors] Migrate StreamExecutionEnvironment#fromElements() implementation to FLIP-27 Source API [flink]

2023-10-19 Thread via GitHub
afedulov commented on PR #23553: URL: https://github.com/apache/flink/pull/23553#issuecomment-1771495374 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [FLINK-28050][connectors] Migrate StreamExecutionEnvironment#fromElements() implementation to FLIP-27 Source API [flink]

2023-10-19 Thread via GitHub
afedulov commented on PR #23553: URL: https://github.com/apache/flink/pull/23553#issuecomment-1771489496 The addition of the `OutputTypeConfigurable` was required to achieve feature parity and compatibility with the existing fromElements API

Re: [PR] [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec… [flink]

2023-10-19 Thread via GitHub
yigress commented on PR #23425: URL: https://github.com/apache/flink/pull/23425#issuecomment-1771447069 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[jira] [Commented] (FLINK-23411) Expose Flink checkpoint details metrics

2023-10-19 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-23411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1364#comment-1364 ] Piotr Nowojski commented on FLINK-23411: Just FYI. I'm creating at the moment a PoC of using

[jira] [Updated] (FLINK-33194) AWS Connector should directly depend on 3rd-party libs instead of flink-shaded repo

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33194: --- Labels: pull-request-available (was: ) > AWS Connector should directly depend on 3rd-party

Re: [PR] [FLINK-33194][Connectors/Kinesis] Remove dependency on Flink Shaded [flink-connector-aws]

2023-10-19 Thread via GitHub
boring-cyborg[bot] commented on PR #109: URL: https://github.com/apache/flink-connector-aws/pull/109#issuecomment-1771314238 Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html) -- This is

[PR] [FLINK-33194][Connectors/Kinesis] Remove dependency on Flink Shaded [flink-connector-aws]

2023-10-19 Thread via GitHub
Jiabao-Sun opened a new pull request, #109: URL: https://github.com/apache/flink-connector-aws/pull/109 ## Purpose of the change [FLINK-33194] Remove dependency on Flink Shaded ## Verifying this change This change is already covered by existing tests. ##

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365801223 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricEvaluator.java: ## @@ -154,6 +153,56 @@ private Map evaluateMetrics(

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365800397 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricEvaluator.java: ## @@ -154,6 +153,56 @@ private Map evaluateMetrics(

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365800397 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricEvaluator.java: ## @@ -154,6 +153,56 @@ private Map evaluateMetrics(

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#issuecomment-1771297136 I missed that we could scale down again after making the right adjustment using the observed rate. LGTM apart from the comments on the diffs. -- This is an automated

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#issuecomment-1771263621 > If we catch up during the stabilization period, then there is no overestimation of the true processing rate. I thought this change is to prevent scenarios where we are

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#issuecomment-1771258357 If we catch up during the stabilization period, then there is no overestimation of the true processing rate. I thought this change is to prevent scenarios where we are are

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#issuecomment-1771237467 > I don't know why we would start collecting already in the stabilization period. That breaks the contract we have established. We can already collect metrics during the

Re: [PR] [FLINK-33171][table planner] Consistent implicit type coercion support for equal and non-equal comparisons for codegen [flink]

2023-10-19 Thread via GitHub
LadyForest commented on code in PR #23478: URL: https://github.com/apache/flink/pull/23478#discussion_r1365165868 ## flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/codegen/calls/ScalarOperatorGens.scala: ## @@ -345,44 +345,83 @@ object

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on code in PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365742349 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java: ## @@ -33,19 +35,15 @@ @Experimental public

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365734089 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java: ## @@ -151,6 +151,31 @@ private static

[jira] [Commented] (FLINK-33187) Don't record duplicate event if no change

2023-10-19 Thread Clara Xiong (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1327#comment-1327 ] Clara Xiong commented on FLINK-33187: - Anyone have opinions on the default interval? Is 30 min a

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
clarax commented on code in PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365727935 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java: ## @@ -33,19 +35,15 @@ @Experimental public

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365724745 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -138,15 +138,19 @@ public CollectedMetricHistory

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365595263 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -97,32 +99,30 @@ public CollectedMetricHistory

[jira] [Commented] (FLINK-33293) Data loss with Kafka Sink

2023-10-19 Thread Jasmin Redzepovic (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1319#comment-1319 ] Jasmin Redzepovic commented on FLINK-33293: --- I managed to reproduce data loss using 1.16.2 and

[jira] [Commented] (FLINK-30738) Disable verbose mode

2023-10-19 Thread Jiabao Sun (Jira)
[ https://issues.apache.org/jira/browse/FLINK-30738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1313#comment-1313 ] Jiabao Sun commented on FLINK-30738: Hi [~chesnay], could you help review that PR when you have

[jira] [Updated] (FLINK-33293) Data loss with Kafka Sink

2023-10-19 Thread Jasmin Redzepovic (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jasmin Redzepovic updated FLINK-33293: -- Attachment: job_1_16_2_run1.log > Data loss with Kafka Sink >

[jira] [Updated] (FLINK-33293) Data loss with Kafka Sink

2023-10-19 Thread Jasmin Redzepovic (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jasmin Redzepovic updated FLINK-33293: -- Attachment: job_1_17_1_run1.log > Data loss with Kafka Sink >

[jira] [Updated] (FLINK-33293) Data loss with Kafka Sink

2023-10-19 Thread Jasmin Redzepovic (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jasmin Redzepovic updated FLINK-33293: -- Attachment: job_1_16_2_run2.log > Data loss with Kafka Sink >

[jira] [Updated] (FLINK-33293) Data loss with Kafka Sink

2023-10-19 Thread Jasmin Redzepovic (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jasmin Redzepovic updated FLINK-33293: -- Attachment: job_1_17_1_run2.log > Data loss with Kafka Sink >

[jira] [Assigned] (FLINK-33320) Support Dynamic Logger Level Adjustment

2023-10-19 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Fan reassigned FLINK-33320: --- Assignee: RocMarshal > Support Dynamic Logger Level Adjustment >

Re: [PR] [FLINK-33318] Expose aggregated collector metrics and measure timeMsPerSecond [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23554: URL: https://github.com/apache/flink/pull/23554#issuecomment-1771136942 ## CI report: * c49d5a0d848326446fc38b709aac38927cd7f934 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] [FLINK-33318] Expose aggregated collector metrics and measure timeMsPerSecond [flink]

2023-10-19 Thread via GitHub
gyfora opened a new pull request, #23554: URL: https://github.com/apache/flink/pull/23554 ## What is the purpose of the change As discussed in https://cwiki.apache.org/confluence/display/FLINK/FLIP-361%3A+Improve+GC+Metrics. This commit adds the All collector group to measure time +

[jira] [Updated] (FLINK-33318) Create TimeMsPerSec metrics for each collector and add metrics across all collectors

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-33318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-33318: --- Labels: pull-request-available (was: ) > Create TimeMsPerSec metrics for each collector

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on code in PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365657154 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java: ## @@ -33,19 +35,15 @@ @Experimental public

Re: [PR] [FLINK-33187] using hashcode for parallelism map comparison [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
gyfora commented on code in PR #685: URL: https://github.com/apache/flink-kubernetes-operator/pull/685#discussion_r1365655692 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/AutoScalerEventHandler.java: ## @@ -33,19 +35,15 @@ @Experimental public

Re: [PR] [FLINK-33310] Scala before 2.12.18 doesn't compile on Java 21 [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on code in PR #23548: URL: https://github.com/apache/flink/pull/23548#discussion_r1365634770 ## pom.xml: ## @@ -1124,6 +1124,35 @@ under the License. + + java21 +

Re: [PR] [FLINK-28050][connectors] Migrate StreamExecutionEnvironment#fromElements() implementation to FLIP-27 Source API [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23553: URL: https://github.com/apache/flink/pull/23553#issuecomment-1771102556 ## CI report: * 227741d560869d8610b9a80e65b9181ffd25bdb6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

Re: [PR] [FLINK-33310] Scala before 2.12.18 doesn't compile on Java 21 [flink]

2023-10-19 Thread via GitHub
snuyanzin commented on code in PR #23548: URL: https://github.com/apache/flink/pull/23548#discussion_r1365634770 ## pom.xml: ## @@ -1124,6 +1124,35 @@ under the License. + + java21 +

Re: [PR] Srichter local rescaling [flink]

2023-10-19 Thread via GitHub
StefanRRichter commented on PR #23552: URL: https://github.com/apache/flink/pull/23552#issuecomment-1771094601 @flinkbot run azur -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (FLINK-28050) Introduce Source API alternative to SourceExecutionContext#fromElements(*) methods

2023-10-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/FLINK-28050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-28050: --- Labels: pull-request-available (was: ) > Introduce Source API alternative to

[PR] [FLINK-28050][connectors] Migrate StreamExecutionEnvironment#fromElements() implementation to FLIP-27 Source API [flink]

2023-10-19 Thread via GitHub
afedulov opened a new pull request, #23553: URL: https://github.com/apache/flink/pull/23553 ## What is the purpose of the change The purpose is to change the internal implementation of `StreamExecutionEnvironment#fromElements()` that is currently based on SourceFunction API (deprecated,

Re: [PR] Srichter local rescaling [flink]

2023-10-19 Thread via GitHub
flinkbot commented on PR #23552: URL: https://github.com/apache/flink/pull/23552#issuecomment-1771091957 ## CI report: * 6eb1b70e0a0659bffee97ce464aa18373278c47b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure`

[PR] Srichter local rescaling [flink]

2023-10-19 Thread via GitHub
StefanRRichter opened a new pull request, #23552: URL: https://github.com/apache/flink/pull/23552 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring

[jira] [Created] (FLINK-33320) Support Dynamic Logger Level Adjustment

2023-10-19 Thread RocMarshal (Jira)
RocMarshal created FLINK-33320: -- Summary: Support Dynamic Logger Level Adjustment Key: FLINK-33320 URL: https://issues.apache.org/jira/browse/FLINK-33320 Project: Flink Issue Type: Improvement

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365609077 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -97,32 +99,30 @@ public CollectedMetricHistory

Re: [PR] [FLINK-33306] Use observed source throughput as true processing rate [flink-kubernetes-operator]

2023-10-19 Thread via GitHub
mxm commented on code in PR #686: URL: https://github.com/apache/flink-kubernetes-operator/pull/686#discussion_r1365558874 ## flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java: ## @@ -97,32 +99,30 @@ public CollectedMetricHistory

  1   2   >