[jira] [Created] (FLINK-35221) Support SQL 2011 reserved keywords as identifiers in Flink HiveParser

2024-04-24 Thread Wencong Liu (Jira)
Wencong Liu created FLINK-35221: --- Summary: Support SQL 2011 reserved keywords as identifiers in Flink HiveParser Key: FLINK-35221 URL: https://issues.apache.org/jira/browse/FLINK-35221 Project: Flink

Re: [ DISCUSS ] FLIP-XXX : [Plugin] Enhancing Flink Failure Management in Kubernetes with Dynamic Termination Log Integration

2024-04-24 Thread Swathi C
Sure Ahmed and Martijn. Fetching the flink particular job related failure and adding this logic to termination-log is definitely a sub-task of pluggable enricher as we can leverage pluggable enricher to achieve this. But for CRUD level failures, which is mainly used to notify if the job manager

Re: [ DISCUSS ] FLIP-XXX : [Plugin] Enhancing Flink Failure Management in Kubernetes with Dynamic Termination Log Integration

2024-04-24 Thread Ahmed Hamdy
Hi, I agree with the Martijn, We can reformulate the FLIP to introduce termination log as supported pluggable enricher. If you believe the scope of work is a subset (Further implementation) we can just add a Jira ticket for it. IMO this will also help with implementation taking the existing

Re: [DISCUSS] FLIP-XXX Support currentFetchEventTimeLag and processingLag metrics

2024-04-24 Thread jialiang tan
Hi lorenzo, thanks for your feedback! > There can be major discrepancies between the absolute time got by the > TaskManagers (if clocks are not synchronized via ntp for example), and the > results of the metric might be quite distant for different TMs. I don't know much about this. can more

[jira] [Created] (FLINK-35229) join An error occurred when the table was empty

2024-04-24 Thread lixu (Jira)
lixu created FLINK-35229: Summary: join An error occurred when the table was empty Key: FLINK-35229 URL: https://issues.apache.org/jira/browse/FLINK-35229 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35230) Split FlinkSqlParserImplTest to reduce the code lines.

2024-04-24 Thread Feng Jin (Jira)
Feng Jin created FLINK-35230: Summary: Split FlinkSqlParserImplTest to reduce the code lines. Key: FLINK-35230 URL: https://issues.apache.org/jira/browse/FLINK-35230 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread yue ma
hi Yanfei, Thanks for your feedback and reminders I have updated related information. In fact, most of them use the default Configrations. Yanfei Lei 于2024年4月23日周二 12:51写道: > Hi Yue & Roman, > > Thanks for initiating this FLIP and all the efforts for the upgrade. > > 8.10.0 introduces some new

Re: [DISCUSS] FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table

2024-04-24 Thread Feng Jin
Hi Ron Thank you for initiating this FLIP. My current questions are as follows: 1. From my current understanding, the workflow handle should not be bound to the Dynamic Table. Therefore, if the workflow is modified, does it mean that the scheduling information corresponding to the Dynamic Table

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread Congxian Qiu
Thanks for driving this, yue We also observed significant performance improvements in some cases after bumped the Rocksdb version, +1 for this work Best, Congxian yue ma 于2024年4月24日周三 19:16写道: > hi Yanfei, > > Thanks for your feedback and reminders I have updated related information. > In

[jira] [Created] (FLINK-35231) Test relies on listJobsFunction in TestingClusterClient are broken

2024-04-24 Thread Artem Plyusnin (Jira)
Artem Plyusnin created FLINK-35231: -- Summary: Test relies on listJobsFunction in TestingClusterClient are broken Key: FLINK-35231 URL: https://issues.apache.org/jira/browse/FLINK-35231 Project:

[jira] [Created] (FLINK-35232) Support for retry settings on GCS connector

2024-04-24 Thread Vikas M (Jira)
Vikas M created FLINK-35232: --- Summary: Support for retry settings on GCS connector Key: FLINK-35232 URL: https://issues.apache.org/jira/browse/FLINK-35232 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-445: Support dynamic parallelism inference for HiveSource

2024-04-24 Thread Xia Sun
Hi everyone, Thanks for all the feedback! If there are no more comments, I would like to start the vote thread, thanks again! Best, Xia Ahmed Hamdy 于2024年4月18日周四 21:31写道: > Hi Xia, > I have read through the FLIP and discussion and the new version of the FLIP > looks better. > +1 for the

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread Hangxiang Yu
Hi, Yue. Very glad to see that IngestDB will be used to improve the rescaling performance of RocksDB. And +1 for the upgrade. Thanks for the great work! On Thu, Apr 25, 2024 at 5:16 AM Martijn Visser wrote: > +1 > > On Wed, Apr 24, 2024 at 5:31 PM Congxian Qiu > wrote: > > > Thanks for driving

[jira] [Created] (FLINK-35233) HBase lookup result is wrong when lookup cache is enabled

2024-04-24 Thread tanjialiang (Jira)
tanjialiang created FLINK-35233: --- Summary: HBase lookup result is wrong when lookup cache is enabled Key: FLINK-35233 URL: https://issues.apache.org/jira/browse/FLINK-35233 Project: Flink

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread Martijn Visser
+1 On Wed, Apr 24, 2024 at 5:31 PM Congxian Qiu wrote: > Thanks for driving this, yue > > We also observed significant performance improvements in some cases after > bumped the Rocksdb version, +1 for this work > > Best, > Congxian > > > yue ma 于2024年4月24日周三 19:16写道: > > > hi Yanfei, > > > >

Re: [DISCUSS] FLIP-445: Support dynamic parallelism inference for HiveSource

2024-04-24 Thread Venkatakrishnan Sowrirajan
Hi Xia, +1 on introducing dynamic parallelism inference for HiveSource. Orthogonal to this discussion, curious, how commonly HiveSource is used these days in the industry given the popularity of table formats/sources like Iceberg, Hudi and Delta lake? Thanks Venkat On Wed, Apr 24, 2024, 7:41 

Re: [ DISCUSS ] FLIP-XXX : [Plugin] Enhancing Flink Failure Management in Kubernetes with Dynamic Termination Log Integration

2024-04-24 Thread Martijn Visser
I would prefer a separate FLIP On Wed, Apr 24, 2024 at 3:25 PM Swathi C wrote: > Sure Ahmed and Martijn. > Fetching the flink particular job related failure and adding this logic to > termination-log is definitely a sub-task of pluggable enricher as we can > leverage pluggable enricher to

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread yue ma
Hi lorenzo.affetti, Thank you for your feedback. I'm sorry, I didn't fully understand your question. >From the results of the StateBenchmark, it appears that interfaces with regression mainly include (listAdd/listAddAll/mapUpdate/mapRemove/valueAdd/valueUpdate), all of which call Rocksdb.put. So

[jira] [Created] (FLINK-35234) Fix NullPointerException of org.apache.flink.cdc.common.configuration.ConfigurationUtils#convertToString

2024-04-24 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35234: - Summary: Fix NullPointerException of org.apache.flink.cdc.common.configuration.ConfigurationUtils#convertToString Key: FLINK-35234 URL:

[jira] [Created] (FLINK-35228) DynamicKafkaSource does not read re-added topic for the same cluster

2024-04-24 Thread Jira
Ignas Daukšas created FLINK-35228: - Summary: DynamicKafkaSource does not read re-added topic for the same cluster Key: FLINK-35228 URL: https://issues.apache.org/jira/browse/FLINK-35228 Project:

Re: [VOTE] FLIP-436: Introduce Catalog-related Syntax

2024-04-24 Thread Yubin Li
Hi everyone, During the implementation of the "describe catalog" syntax, it was found that the original output style needed to be improved. ``` desc catalog extended cat2; +--+-+ | catalog_description_item |

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Yuepeng Pan
+1(non-binding) Best, Yuepeng Pan At 2024-04-24 16:05:07, "Rui Fan" <1996fan...@gmail.com> wrote: >+1(binding) > >Best, >Rui > >On Wed, Apr 24, 2024 at 4:03 PM Mate Czagany wrote: > >> Hi everyone, >> >> I'd like to start a vote on the FLIP-446: Kubernetes Operator State >> Snapshot CRD [1].

Re: [VOTE] FLIP-436: Introduce Catalog-related Syntax

2024-04-24 Thread Ahmed Hamdy
Hi, +1 (non-binding) Best Regards Ahmed Hamdy On Wed, 24 Apr 2024 at 09:58, Yubin Li wrote: > Hi everyone, > > During the implementation of the "describe catalog" syntax, it was > found that the original output style needed to be improved. > ``` > desc catalog extended cat2; > >

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

2024-04-24 Thread Ahmed Hamdy
Thanks Danny, +1 (non-binding) - Verified Checksums and hashes - Verified Signatures - Reviewed web PR - github tag exists - Build source Best Regards Ahmed Hamdy On Tue, 23 Apr 2024 at 03:47, Muhammet Orazov wrote: > Thanks Danny, +1 (non-binding) > > - Checked 512 hash > - Checked gpg

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

2024-04-24 Thread Ahmed Hamdy
Thanks Danny, +1 (non-binding) - Verified Checksums and hashes - Verified Signatures - Reviewed web PR - github tag exists - Build source Best Regards Ahmed Hamdy On Tue, 23 Apr 2024 at 03:57, Muhammet Orazov wrote: > Thanks! +1 (non-binding) > > - Checked 512 hash > - Checked gpg signature

Re: [DISCUSS] FLIP-XXX Support currentFetchEventTimeLag and processingLag metrics

2024-04-24 Thread lorenzo . affetti
Hello jialiang tan, and thank you for your contribution! Here are my questions: 1 - I don't know if exposing this as metric to the user could create some harm. There can be major discrepancies between the absolute time got by the TaskManagers (if clocks are not synchronized via ntp for

Re: [DISCUSS] FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table

2024-04-24 Thread lorenzo . affetti
Hello Ron Liu! Thank you for your FLIP! Here are my considerations: 1. About the Operations interfaces, how can they be empty? Should not they provide at least a `run` or `execute` method (similar to the command pattern)? In this way, their implementation can wrap all the implementations

[jira] [Created] (FLINK-35222) Adding getJobType for AccessExecutionGraph

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35222: --- Summary: Adding getJobType for AccessExecutionGraph Key: FLINK-35222 URL: https://issues.apache.org/jira/browse/FLINK-35222 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35224) Show the JobType on Flink WebUI

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35224: --- Summary: Show the JobType on Flink WebUI Key: FLINK-35224 URL: https://issues.apache.org/jira/browse/FLINK-35224 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35225) Remove Execution mode in Flink WebUI

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35225: --- Summary: Remove Execution mode in Flink WebUI Key: FLINK-35225 URL: https://issues.apache.org/jira/browse/FLINK-35225 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35226) Deprecate execution-mode in ExecutionConfigInfo related rest api

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35226: --- Summary: Deprecate execution-mode in ExecutionConfigInfo related rest api Key: FLINK-35226 URL: https://issues.apache.org/jira/browse/FLINK-35226 Project: Flink

[VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Mate Czagany
Hi everyone, I'd like to start a vote on the FLIP-446: Kubernetes Operator State Snapshot CRD [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1]

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Ferenc Csaky
+1 (non-binding), looking forward to this! Best, Ferenc On Wednesday, April 24th, 2024 at 10:03, Mate Czagany wrote: > > > Hi everyone, > > I'd like to start a vote on the FLIP-446: Kubernetes Operator State > Snapshot CRD [1]. The discussion thread is here [2]. > > The vote will be

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread lorenzo . affetti
Hello yuema! Thank you for the proposal. In light of what is happening to state backends with FLIP-423 and others, it definitely makes sense to leverage the full power of latest FRocksDB. Small nit and question for you: do you have any idea how to justify the regression in write perf? Thank

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread yue ma
hi Roman, Thank you very much for your feedback and effort, especially for your help in releasing Frocksdb products and testing the performance. Roman Khachatryan 于2024年4月23日周二 12:07写道: > Hi, > > Thanks for writing the proposal and preparing the upgrade. > > FRocksDB definitely needs to be

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread yue ma
Hi , Zakelly Lan Thank you very much for your feedback and effort, especially for the help in State Benchmark testing ! Zakelly Lan 于2024年4月24日周三 10:46写道: > Hi Yue, > > Thanks for this proposal! > > Given the great improvement we could have, the slight regression in write > performance is a

[jira] [Created] (FLINK-35223) Add jobType in JobDetailsInfo related rest api

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35223: --- Summary: Add jobType in JobDetailsInfo related rest api Key: FLINK-35223 URL: https://issues.apache.org/jira/browse/FLINK-35223 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35227) Remove execution-mode in ExecutionConfigInfo

2024-04-24 Thread Rui Fan (Jira)
Rui Fan created FLINK-35227: --- Summary: Remove execution-mode in ExecutionConfigInfo Key: FLINK-35227 URL: https://issues.apache.org/jira/browse/FLINK-35227 Project: Flink Issue Type: Sub-task

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Gyula Fóra
+1 (binding) Gyula On Wed, Apr 24, 2024 at 10:07 AM Ferenc Csaky wrote: > +1 (non-binding), looking forward to this! > > Best, > Ferenc > > > > > On Wednesday, April 24th, 2024 at 10:03, Mate Czagany > wrote: > > > > > > > Hi everyone, > > > > I'd like to start a vote on the FLIP-446:

Re: [VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Rui Fan
+1(binding) Best, Rui On Wed, Apr 24, 2024 at 4:03 PM Mate Czagany wrote: > Hi everyone, > > I'd like to start a vote on the FLIP-446: Kubernetes Operator State > Snapshot CRD [1]. The discussion thread is here [2]. > > The vote will be open for at least 72 hours unless there is an objection