Re: [VOTE] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding

2024-05-08 Thread Timo Walther
+1 (binding) Thanks, Timo On 08.05.24 11:10, Stefan Richter wrote: Hi Alan, Thanks for this proposal, the ability to exclude functions from constant folding makes sense to me. +1 (binding) Best, Stefan On 8. May 2024, at 02:01, Alan Sheinberg wrote: Hi everyone, I'd like to start a

Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding

2024-05-06 Thread Timo Walther
Hi Alan, thanks for the update. From my side, the FLIP seems good for voting. Since it only touches a small API surface, I guess the proposal is not very controversial. Feel free to start a vote thread by tomorrow. If there are no objections? Thanks, Timo On 02.05.24 09:45, Muhammet

Re: [Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding

2024-04-30 Thread Timo Walther
Hi Alan, thanks for starting this small FLIP. A new method in FunctionDefinition makes a lot of sense and the lack of it was an issue in past already. We should clearly separate determinism and constant folding behavior. Since FunctionDefinition is kind of a declaration, it should rather

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-04-09 Thread Timo Walther
r platform has introduced Persistent Derived Table, and there are precedents in the industry; could Derived Table be a candidate? Of course, look forward to your better suggestions. Best, Ron Timo Walther 于2024年3月25日周一 18:49写道: After thinking about this more, this discussion boils down to whethe

[jira] [Created] (FLINK-35001) Avoid scientific notation for DOUBLE to STRING

2024-04-03 Thread Timo Walther (Jira)
Timo Walther created FLINK-35001: Summary: Avoid scientific notation for DOUBLE to STRING Key: FLINK-35001 URL: https://issues.apache.org/jira/browse/FLINK-35001 Project: Flink Issue Type

Re: [VOTE] FLIP-437: Support ML Models in Flink SQL

2024-04-02 Thread Timo Walther
+1 (binding) Thanks, Timo On 29.03.24 17:30, Hao Li wrote: Hi devs, I'd like to start a vote on the FLIP-437: Support ML Models in Flink SQL [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1]

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-28 Thread Timo Walther
.ftl#L814 Thanks, Hao On Tue, Mar 26, 2024 at 4:15 AM Timo Walther wrote: Hi Hao, > `TABLE(my_data)` and `MODEL(my_cat.my_db.classifier_model)` doesn't > work since `TABLE` and `MODEL` are already key words This argument doesn't count. The parser supports introducing key

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-26 Thread Timo Walther
model DDL into 1.20. For predict and evaluate functions, if we can't get into the 1.20 release, we can get them into the 1.21 release for sure. Thanks, Hao On Mon, Mar 25, 2024 at 1:25 AM Timo Walther wrote: Hi Jark and Hao, thanks for the information, Jark! Great that the Calcite communit

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-03-25 Thread Timo Walther
behavior is closer to a MV. If we add a special keyword to MV, the optimizer would know that the data cannot be used for query optimizations. I will ask more people for their opinion. Regards, Timo On 25.03.24 10:45, Timo Walther wrote: Hi Ron and Lincoln, thanks for the quick response

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-03-25 Thread Timo Walther
gether nicely. What do you think about this? [1] https://calcite.apache.org/docs/materialized_views.html [2] https://cloud.google.com/looker/docs/derived-tables#persistent_derived_tables Best, Lincoln Lee Timo Walther 于2024年3月22日周五 17:54写道: Hi Ron, thanks for the detailed answer. Sorry,

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-25 Thread Timo Walther
te community is preparing the 1.37 release, so we can bump the version if needed in Flink 1.19. Best, Jark [1]: https://issues.apache.org/jira/browse/CALCITE-6254 On Fri, 22 Mar 2024 at 21:46, Timo Walther wrote: Hi everyone, this is a very important change to the Flink SQL syntax but we ca

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-22 Thread Timo Walther
Hi everyone, this is a very important change to the Flink SQL syntax but we can't wait until the SQL standard is ready for this. So I'm +1 on introducing the MODEL concept as a first class citizen in Flink. For your information: Over the past months I have already spent a significant amount

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-03-22 Thread Timo Walther
r example, a query including window aggregate based on processing time? or a query including global order by? 2. Whether modify the query of dynamic table is allowed? Or we could only refresh a dynamic table based on initial query? 3. How to use dynamic table? The dynamic table seems to be

[jira] [Created] (FLINK-34902) INSERT INTO column mismatch leads to IndexOutOfBoundsException

2024-03-21 Thread Timo Walther (Jira)
Timo Walther created FLINK-34902: Summary: INSERT INTO column mismatch leads to IndexOutOfBoundsException Key: FLINK-34902 URL: https://issues.apache.org/jira/browse/FLINK-34902 Project: Flink

Re: [DISCUSS] FLIP-435: Introduce a New Dynamic Table for Simplifying Data Pipelines

2024-03-12 Thread Timo Walther
Hi Lincoln & Ron, thanks for proposing this FLIP. I think a design similar to what you propose has been in the heads of many people, however, I'm wondering how this will fit into the bigger picture. I haven't deeply reviewed the FLIP yet, but would like to ask some initial questions:

Re: Default scale and precision SQL data types

2024-03-12 Thread Timo Walther
Hi Sergei, please check with the SQL standard before. Most of these values have been derived from the standard. I don't like the default of TIMESTAMP(6) for timestamps but this is what the standard dictates. Same for not allowing VARCHAR(0) or VARCHAR defaulting to VARCHAR(1). Changes to

[jira] [Created] (FLINK-34476) Window TVFs with named parameters don't support column expansion

2024-02-20 Thread Timo Walther (Jira)
Timo Walther created FLINK-34476: Summary: Window TVFs with named parameters don't support column expansion Key: FLINK-34476 URL: https://issues.apache.org/jira/browse/FLINK-34476 Project: Flink

[jira] [Created] (FLINK-34469) Implement TableDistribution toString

2024-02-19 Thread Timo Walther (Jira)
Timo Walther created FLINK-34469: Summary: Implement TableDistribution toString Key: FLINK-34469 URL: https://issues.apache.org/jira/browse/FLINK-34469 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34316) Reduce instantiation of ScanRuntimeProvider for in streaming mode

2024-01-30 Thread Timo Walther (Jira)
Timo Walther created FLINK-34316: Summary: Reduce instantiation of ScanRuntimeProvider for in streaming mode Key: FLINK-34316 URL: https://issues.apache.org/jira/browse/FLINK-34316 Project: Flink

Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-15 Thread Timo Walther
+1 (binding) Cheers, Timo On 09.01.24 10:58, xiangyu feng wrote: +1 (non-binding) Regards, Xiangyu Feng Danny Cranmer 于2024年1月9日周二 17:50写道: +1 (binding) Thanks, Danny On Tue, Jan 9, 2024 at 9:31 AM Feng Jin wrote: +1 (non-binding) Best, Feng Jin On Tue, Jan 9, 2024 at 5:29 PM

[jira] [Created] (FLINK-34095) Add a restore test for StreamExecAsyncCalc

2024-01-15 Thread Timo Walther (Jira)
Timo Walther created FLINK-34095: Summary: Add a restore test for StreamExecAsyncCalc Key: FLINK-34095 URL: https://issues.apache.org/jira/browse/FLINK-34095 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-34094) Document new AsyncScalarFunction

2024-01-15 Thread Timo Walther (Jira)
Timo Walther created FLINK-34094: Summary: Document new AsyncScalarFunction Key: FLINK-34094 URL: https://issues.apache.org/jira/browse/FLINK-34094 Project: Flink Issue Type: Sub-task

Re: [VOTE] FLIP-387: Support named parameters for functions and call procedures

2024-01-05 Thread Timo Walther
by default and developers should explicitly state if a parameter is optional instead of using our default value. Regarding this part, I have already made modifications in the document. Best, Feng On Fri, Jan 5, 2024 at 3:38 PM Timo Walther wrote: Thanks, for starting the VOTE thread and thanks

Re: [VOTE] FLIP-387: Support named parameters for functions and call procedures

2024-01-04 Thread Timo Walther
Thanks, for starting the VOTE thread and thanks for considering my feedback. One last comment before I'm also happy to give my +1 to this: Why is ArgumentHint's default isOptinal=true? Shouldn't it be false by default? Many function implementers will forget to set this to false and suddenly

Re: [DISCUSS][FLINK-31830] Align the Nullability Handling of ROW between SQL and TableAPI

2024-01-02 Thread Timo Walther
Hi Jane, thanks for the heavy investigation and extensive summaries. I'm sorry that I ignored this discussion for too long but would like to help in shaping a sustainable long-term solution. I fear that changing: - RowType#copy() - RowType's constructor - FieldsDataType#nullable() will not

Re: [VOTE] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-28 Thread Timo Walther
+1 (binding) Cheers, Timo > Am 28.12.2023 um 03:13 schrieb Yuepeng Pan : > > +1 (non-binding). > > Best, > Yuepeng Pan. > > > > > At 2023-12-28 09:19:37, "Lincoln Lee" wrote: >> +1 (binding) >> >> Best, >> Lincoln Lee >> >> >> Martijn Visser 于2023年12月27日周三 23:16写道: >> >>> +1

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-19 Thread Timo Walther
ed in 1.19, we can consider officially deprecating the old group window agg syntax in the release note. WDYT? -- Best! Xuyang At 2023-12-14 17:51:01, "Timo Walther" wrote: Hi Xuyang, I'm not spliting this flip is that all of these subtasks like session window tv

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-19 Thread Timo Walther
wup. Then maybe we consider only INSERT mode graphs and ones where we can solve the watermark constraints. Thanks, Alan On Mon, Dec 18, 2023 at 2:36 AM Timo Walther wrote: Hi Xuyang and Alan, thanks for this productive discussion. > Would it make a difference if it were exposed by the e

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-18 Thread Timo Walther
reasonable approach to me. -Alan [1] https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/metric_reporters/ On Fri, Dec 15, 2023 at 2:56 AM Timo Walther wrote: 1. Override the function `getRequirements` in `AsyncScalarFunction` > If the user overrides `requirements

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-15 Thread Timo Walther
everaging the async table function[1] to improve the throughput before. About the configs, what do you think using hints as mentioned in [1]. [1]: https://cwiki.apache.org/confluence/display/FLINK/FLIP-313%3A+Add+support+of+User+Defined+AsyncTableFunction Thanks, Aitozi. Timo Walther 于2023年12

Re: [DISCUSS] FLIP-387: Support named parameters for functions and call procedures

2023-12-14 Thread Timo Walther
Hi Feng, thank you for proposing this FLIP. This nicely completes FLIP-65 which is great for usability. I read the FLIP and have some feedback: 1) ArgumentNames annotation > Deprecate the ArgumentNames annotation as it is not user-friendly for specifying argument names with optional

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-14 Thread Timo Walther
ppy to help here with the review as well and will reserve some time for it in coming days. On Tue, Dec 12, 2023 at 12:18 PM Timo Walther wrote: Hi Xuyang, thanks for proposing this FLIP. In my opinion the FLIP touches too many topics at the same time and should be split into multiple FLIPs

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-14 Thread Timo Walther
The configuration in Flink is complicated and I fear we won't have enough capacity to substantially fix it. The introduction of ReadableConfig, WritableConfig, and typed ConfigOptions was a right step into making the code more maintainable. From the Flink side, every read access should go

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-14 Thread Timo Walther
Hi Alan, thanks for proposing this FLIP. It's a great addition to Flink and has been requested multiple times. It will be in particular interesting for accessing REST endpoints and other remote services. Great that we can generalize and reuse parts of the Python planner rules and code for

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-12 Thread Timo Walther
Hi Xuyang, thanks for proposing this FLIP. In my opinion the FLIP touches too many topics at the same time and should be split into multiple FLIPs. We should stay focused on what is needed for Flink 2.0. Let me summarizing the topics: 1) SESSION Window TVF Aggregation This has been agreed

[jira] [Created] (FLINK-33784) CatalogStoreFactory can not be configured via StreamExecutionEnvironment

2023-12-08 Thread Timo Walther (Jira)
Timo Walther created FLINK-33784: Summary: CatalogStoreFactory can not be configured via StreamExecutionEnvironment Key: FLINK-33784 URL: https://issues.apache.org/jira/browse/FLINK-33784 Project

Re: [DISCUSS] Release Flink 1.18.1

2023-12-08 Thread Timo Walther
Thanks for taking care of this Jing. +1 to release 1.18.1 for this. Cheers, Timo On 08.12.23 10:00, Benchao Li wrote: I've merged FLINK-33313 to release-1.18 branch. Péter Váry 于2023年12月8日周五 16:56写道: Hi Jing, Thanks for taking care of this! +1 (non-binding) Peter Sergey Nuyanzin ezt

Re: SQL return type change from 1.17 to 1.18

2023-12-07 Thread Timo Walther
Hi Peter, thanks for reaching out to the Flink community. This is indeed a serious issue. As the author of the Flink type system, DataType and many related utilities I strongly vote for reverting FLINK-33523: - It changes the Flink type system without a FLIP. - It breaks backwards

[jira] [Created] (FLINK-33666) MergeTableLikeUtil uses different constraint name than Schema

2023-11-27 Thread Timo Walther (Jira)
Timo Walther created FLINK-33666: Summary: MergeTableLikeUtil uses different constraint name than Schema Key: FLINK-33666 URL: https://issues.apache.org/jira/browse/FLINK-33666 Project: Flink

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Timo Walther
+1 (binding) Thanks for working on this FLIP. It will be a nice continuation of the previous work. Cheers, Timo On 21.11.23 13:19, Gyula Fóra wrote: +1 (binding) Gyula On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: +1 (non-binding) Thanks for driving this. Best, Xiangyu Feng

[RESULT][VOTE] FLIP-376: Add DISTRIBUTED BY clause

2023-11-09 Thread Timo Walther
/cft2d9jc2lr8gv6dyyz7b62188mf07sj Cheers, Timo On 08.11.23 10:14, Leonard Xu wrote: +1(binding) Best, Leonard 2023年11月8日 下午1:05,Sergey Nuyanzin 写道: +1 (binding) On Wed, Nov 8, 2023 at 6:02 AM Zhanghao Chen wrote: +1 (non-binding) Best, Zhanghao Chen From: Timo

[jira] [Created] (FLINK-33497) Update the Kafka connector to support DISTRIBUTED BY clause

2023-11-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-33497: Summary: Update the Kafka connector to support DISTRIBUTED BY clause Key: FLINK-33497 URL: https://issues.apache.org/jira/browse/FLINK-33497 Project: Flink

[jira] [Created] (FLINK-33496) Expose DISTRIBUTED BY clause via parser

2023-11-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-33496: Summary: Expose DISTRIBUTED BY clause via parser Key: FLINK-33496 URL: https://issues.apache.org/jira/browse/FLINK-33496 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33495) Add catalog and connector ability API and validation

2023-11-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-33495: Summary: Add catalog and connector ability API and validation Key: FLINK-33495 URL: https://issues.apache.org/jira/browse/FLINK-33495 Project: Flink Issue

[jira] [Created] (FLINK-33494) FLIP-376: Add DISTRIBUTED BY clause

2023-11-09 Thread Timo Walther (Jira)
Timo Walther created FLINK-33494: Summary: FLIP-376: Add DISTRIBUTED BY clause Key: FLINK-33494 URL: https://issues.apache.org/jira/browse/FLINK-33494 Project: Flink Issue Type: New Feature

Re: FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-07 Thread Timo Walther
Thanks for the FLIP, Piotr. In order to follow the FLIP process[1], please prefix the email subject with "[DISCUSS]". Also, some people might have added filters to their email clients to highlight those discussions. Thanks, Timo [1]

[VOTE] FLIP-376: Add DISTRIBUTED BY clause

2023-11-06 Thread Timo Walther
Hi everyone, I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-11-03 Thread Timo Walther
Best regards, Martijn On Thu, Nov 2, 2023 at 11:00 AM Timo Walther wrote: Hi Jing, I agree this is confusing. THe Spark API calls it bucketBy in the programmatic API. But anyway, we should discuss the SQL semantics here. It's like a "WHERE" is called "filter" in th

[jira] [Created] (FLINK-33447) Avoid CompiledPlan recompilation during loading

2023-11-03 Thread Timo Walther (Jira)
Timo Walther created FLINK-33447: Summary: Avoid CompiledPlan recompilation during loading Key: FLINK-33447 URL: https://issues.apache.org/jira/browse/FLINK-33447 Project: Flink Issue Type

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-11-02 Thread Timo Walther
ot;. Not really used in SQL, but afaiu Spark uses the concept[1]. [1] https://spark.apache.org/docs/3.1.1/api/python/reference/api/pyspark.sql.DataFrameWriter.bucketBy.html Best regards, Jing On Mon, Oct 30, 2023 at 5:25 PM Timo Walther wrote: Hi Jing, > Have you considered using BUCKET BY d

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-31 Thread Timo Walther
have one more question. What does Flink check and throw exceptions for the bucketing? For example, do we check interfaces when executing create/alter DDL and when used as a source? Best, Jark On Tue, 31 Oct 2023 at 00:25, Timo Walther wrote: Hi Jing, > Have you considered using BUC

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-30 Thread Timo Walther
HASH(uid) INTO 6 BUCKETS - For advanced users, the algorithm can be defined explicitly. - Currently, either HASH() or RANGE(). " Did you mean users can use their own algorithm? How to do it? Best regards, Jing On Mon, Oct 30, 2023 at 11:13 AM Timo Walther wrote: Let me reply to the feedback

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-30 Thread Timo Walther
ilesystem` connector. Regards, Timo On 30.10.23 10:44, Timo Walther wrote: Hi Jark, my intention was to avoid too complex syntax in the first version. In the past years, we could enable use cases also without this clause, so we should be careful with overloading it with too functionality in t

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-30 Thread Timo Walther
+1 for this. The design looks good to me! We can add some documentation for connector developers. For example: for sink, If there needs some keyby, please finish the keyby by the connector itself. SupportsBucketing is just a marker interface. Best, Jingsong On Thu, Oct 26, 2023 at 5:00 PM

Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-30 Thread Timo Walther
ace. Best, Jingsong On Thu, Oct 26, 2023 at 5:00 PM Timo Walther wrote: Hi everyone, I would like to start a discussion on FLIP-376: Add DISTRIBUTED BY clause [1]. Many SQL vendors expose the concepts of Partitioning, Bucketing, and Clustering. This FLIP continues the work of previous FLIPs and

[DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-26 Thread Timo Walther
Hi everyone, I would like to start a discussion on FLIP-376: Add DISTRIBUTED BY clause [1]. Many SQL vendors expose the concepts of Partitioning, Bucketing, and Clustering. This FLIP continues the work of previous FLIPs and would like to introduce the concept of "Bucketing" to Flink. This

[jira] [Created] (FLINK-33183) Enable metadata columns in NduAnalyzer with retract if non-virtual

2023-10-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-33183: Summary: Enable metadata columns in NduAnalyzer with retract if non-virtual Key: FLINK-33183 URL: https://issues.apache.org/jira/browse/FLINK-33183 Project: Flink

[jira] [Created] (FLINK-33182) Allow metadata columns in NduAnalyzer with ChangelogNormalize

2023-10-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-33182: Summary: Allow metadata columns in NduAnalyzer with ChangelogNormalize Key: FLINK-33182 URL: https://issues.apache.org/jira/browse/FLINK-33182 Project: Flink

[jira] [Created] (FLINK-33169) Window TVFs don't consider column expansion

2023-09-29 Thread Timo Walther (Jira)
Timo Walther created FLINK-33169: Summary: Window TVFs don't consider column expansion Key: FLINK-33169 URL: https://issues.apache.org/jira/browse/FLINK-33169 Project: Flink Issue Type: Sub

[RESULT][VOTE] FLIP-348: Make expanding behavior of virtual metadata columns configurable

2023-09-05 Thread Timo Walther
(binding) Best, Jark 2023年8月31日 18:54,Jing Ge 写道: +1(binding) On Thu, Aug 31, 2023 at 11:22 AM Sergey Nuyanzin wrote: +1 (binding) On Thu, Aug 31, 2023 at 9:28 AM Benchao Li wrote: +1 (binding) Martijn Visser 于2023年8月31日周四 15:24写道: +1 (binding) On Thu, Aug 31, 2023 at 9:09 AM Timo

[jira] [Created] (FLINK-33028) FLIP-348: Make expanding behavior of virtual metadata columns configurable

2023-09-04 Thread Timo Walther (Jira)
Timo Walther created FLINK-33028: Summary: FLIP-348: Make expanding behavior of virtual metadata columns configurable Key: FLINK-33028 URL: https://issues.apache.org/jira/browse/FLINK-33028 Project

[VOTE] FLIP-348: Make expanding behavior of virtual metadata columns configurable

2023-08-31 Thread Timo Walther
Hi everyone, I'd like to start a vote on FLIP-348: Make expanding behavior of virtual metadata columns configurable [1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-29 Thread Timo Walther
:21 AM Benchao Li wrote: It sounds good to me too, that we avoid introducing the concept of "system columns" for now. Timo Walther 于2023年8月18日周五 22:38写道: Great, I also like my last suggestion as it is even more elegant. I will update the FLIP until Monday. Regards, Timo On 1

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-08-18 Thread Timo Walther
oduce a bunch of configuration, because lots of the streaming SQL syntax are extensions of SQL standard. Best, Jark On Thu, 17 Aug 2023 at 15:43, Timo Walther wrote: +1 for this proposal. Not every data team would like to enable hints. Also because they are an extension to the SQL standard.

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-18 Thread Timo Walther
, but not introducing any concept of system/pseudo columns for now. Best, Jark On Tue, 15 Aug 2023 at 23:25, Timo Walther wrote: Hi everyone, I would like to bump this thread up again. Esp. I would like to hear opinions on my latest suggestions to simply use METADATA VIRTUAL as system columns and only

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-08-17 Thread Timo Walther
+1 for this proposal. Not every data team would like to enable hints. Also because they are an extension to the SQL standard. It might also be the case that custom rules would be overwritten otherwise. Setting hints could also be the exclusive task of a DevOp team. Regards, Timo On

Re: Plans for Schema Evolution in Table API

2023-08-15 Thread Timo Walther
Hi Ashish, sorry for the late reply. There are currently no concrete plans to support schema evolution in Table API. Until recently, Flink version evolution was the biggest topic. In the near future we can rediscuss query and state evolution in more detail. Personally, I think we will need

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-15 Thread Timo Walther
concepts. Looking forward to any kind of feedback. Thanks, Timo On 07.08.23 12:07, Timo Walther wrote: Hi everyone, thanks for the various feedback and lively discussion. Sorry, for the late reply as I was on vacation. Let me answer to some of the topics: 1) Systems columns should not be shown

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-07 Thread Timo Walther
orm providers will use ${variable_name} to define their own configurations and allow them to be embedded into SQL scripts. Will there be any conflict with option 3? Best regards, Jing On Tue, Jul 25, 2023 at 7:00 PM Konstantin Knauf wrote: Hi Timo, this makes sense to me. Option 3 se

[jira] [Created] (FLINK-32689) Insufficient validation for table.local-time-zone

2023-07-26 Thread Timo Walther (Jira)
Timo Walther created FLINK-32689: Summary: Insufficient validation for table.local-time-zone Key: FLINK-32689 URL: https://issues.apache.org/jira/browse/FLINK-32689 Project: Flink Issue Type

[DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-07-25 Thread Timo Walther
Hi everyone, I would like to start a discussion about introducing the concept of "System Columns" in SQL and Table API. The subject sounds bigger than it actually is. Luckily, Flink SQL already exposes the concept of metadata columns. And this proposal is just a slight adjustment for how

[jira] [Created] (FLINK-32657) Revert upgrading ExecNode versions for StateMetadata

2023-07-24 Thread Timo Walther (Jira)
Timo Walther created FLINK-32657: Summary: Revert upgrading ExecNode versions for StateMetadata Key: FLINK-32657 URL: https://issues.apache.org/jira/browse/FLINK-32657 Project: Flink Issue

Re: [DISCUSS][2.0] FLIP-344: Remove parameter in RichFunction#open

2023-07-24 Thread Timo Walther
+1 But instead we should add a OpenContext there to keep the signature stable but still be able to add parameters. Regards, Timo On 21.07.23 12:24, Jing Ge wrote: +1 On Fri, Jul 21, 2023 at 10:22 AM Yuxin Tan wrote: +1 Best, Yuxin Xintong Song 于2023年7月21日周五 12:04写道: +1 Best,

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-24 Thread Timo Walther
+1 Regards, Timo On 24.07.23 04:00, liu ron wrote: +1 Best, Ron Lincoln Lee 于2023年7月21日周五 16:09写道: +1 Best, Lincoln Lee Leonard Xu 于2023年7月21日周五 16:07写道: +1 Best, Leonard On Jul 21, 2023, at 4:02 PM, yuxia wrote: +1(binging) Best regards, Yuxia - 原始邮件 - 发件人: "Jane

[jira] [Created] (FLINK-32613) Disable check for single rowtime attribute for sinks

2023-07-17 Thread Timo Walther (Jira)
Timo Walther created FLINK-32613: Summary: Disable check for single rowtime attribute for sinks Key: FLINK-32613 URL: https://issues.apache.org/jira/browse/FLINK-32613 Project: Flink Issue

[jira] [Created] (FLINK-32470) SqlValidateException should be exposed as ValidationException

2023-06-28 Thread Timo Walther (Jira)
Timo Walther created FLINK-32470: Summary: SqlValidateException should be exposed as ValidationException Key: FLINK-32470 URL: https://issues.apache.org/jira/browse/FLINK-32470 Project: Flink

[jira] [Created] (FLINK-32466) Invalid input strategy for CONCAT allows BINARY strings

2023-06-28 Thread Timo Walther (Jira)
Timo Walther created FLINK-32466: Summary: Invalid input strategy for CONCAT allows BINARY strings Key: FLINK-32466 URL: https://issues.apache.org/jira/browse/FLINK-32466 Project: Flink

Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode

2023-05-25 Thread Timo Walther
les. Regards, Timo On 25.05.23 12:29, Timo Walther wrote: Hi Feng, thanks for proposing this FLIP. It makes a lot of sense to finally support querying tables at a specific point in time or hopefully also ranges soon. Following time-versioned tables. Here is some feedback from my side: 1. Syntax

Re: [DISCUSS] FLIP-308: Support Time Travel In Batch Mode

2023-05-25 Thread Timo Walther
Hi Feng, thanks for proposing this FLIP. It makes a lot of sense to finally support querying tables at a specific point in time or hopefully also ranges soon. Following time-versioned tables. Here is some feedback from my side: 1. Syntax Can you elaborate a bit on the Calcite restrictions?

Re: [External] [DISCUSS] FLIP-292: Support configuring state TTL at operator level for Table API & SQL programs

2023-04-03 Thread Timo Walther
Hi Jane, thanks for proposing this FLIP. More state insights and fine-grained state TTL are a frequently requested feature for Flink SQL. Eventually, we need to address this. I agree with the previous responses that doing this with a hint might cause more confusion than it actually helps.

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-09 Thread Timo Walther
WatermarkStrategy.java#L169 On Wed, Mar 1, 2023 at 3:54 PM Timo Walther wrote: Reg. 2: > event gap emit strategy [...] no matter in DataStream or SQL Jark raised a very good point. I thought we only expose what is contained in DataStream API already. If this strategy is not part of

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-09 Thread Timo Walther
Hi Lincoln, thanks for proposing the FLIP. The general idea to expose the target columns in DynamicTableSink#Context sounds good to me. In the FLIP I found the JavaDoc a bit confusing: ``` The column list will be empty for 'insert into target select ...'. ``` This could mean both optional

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-09 Thread Timo Walther
+1 (binding) Thanks, Timo On 08.03.23 14:59, Sergey Nuyanzin wrote: +1 (binding) i guess based on [1], please correct me if i am wrong [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Actions On Wed, Mar 8, 2023 at 8:04 AM yuxia wrote: +1 (non-binding)

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-01 Thread Timo Walther
CompiledPlan in your FLIP. This change has implications on the JSON format as watermark strategy of ExecNode becomes more complex, see WatermarkPushDownSpec Best Kui Yuan Timo Walther 于2023年2月27日周一 18:05写道: Hi Kui Yuan, thanks for working on this FLIP. Let me also give some comments about the pro

Re: [DISCUSS] FLIP-297: Improve Auxiliary Sql Statements

2023-02-28 Thread Timo Walther
Hi Ran Tao, Thanks for working on this FLIP. The FLIP is in a pretty good shape already and I don't have much to add. Will we also support ILIKE in queries? Or is this a pure DDL expressions? For consistency, we should support it in SELECT and Table API as well. I hope this is not too much

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-02-27 Thread Timo Walther
Hi Kui Yuan, thanks for working on this FLIP. Let me also give some comments about the proposed changes. I support the direction of this FLIP about handling these watermark-specific properties through options and /*+OPTIONS(...) */ hints. Regarding naming, I would like to keep the options

Re: [DISCUSS] Proposal to improve Auxiliary Sql Statements

2023-02-21 Thread Timo Walther
Hi Ran, adding additional filter and selection capabilities makes sense. However, I would recommend to not only check Spark or Hive but also focus on bigger commercial players such as Snowflake, Oracle, SQL Server. In any case, maybe we should also investigate more thoughts on a Flink

[ANNOUNCE] New Apache Flink Committer - Sergey Nuyanzin

2023-02-21 Thread Timo Walther
ecosystem. Currently, he is upgrading Flink's SQL engine to the latest Apache Calcite version [1][2][3] and helps in updating other project-wide dependencies as well. Please join me in congratulating Sergey Nuyanzin for becoming a Flink Committer! Best, Timo Walther (on behalf of the Flink PMC

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-06 Thread Timo Walther
Hi Feng, this is indeed a good proposal. 1) It makes sense to improve the catalog listing for platform providers. 2) Other feedback from the past has shown that users would like to avoid the default in-memory catalog and offer their catalog before a TableEnvironment session starts. 3) Also

Re: [DISCUSS] Hybrid Source Connector

2022-12-12 Thread Timo Walther
Hi Ran, Thanks for proposing a FLIP. Btw according to the process, the subject of this email should be `[DISCUSS] FLIP-278: Hybrid Source Connector` so that people can identify this discussion as a FLIP discussion. Supporting the hybrid source for SQL was a long-standing issue on our

Re: [DISCUSS] FLIP-275: Support Remote SQL Client Based on SQL Gateway

2022-12-12 Thread Timo Walther
Hi everyone, sorry to jump into this discussion so late. > So we decided to revert the RowFormat related changes and let the client to resolve the print format. Could you elaborate a bit on this topic in the FLIP? I still believe that we need 2 types of output formats. Format A: for the

[jira] [Created] (FLINK-30226) Offer a Identity(De)SerializationSchema

2022-11-28 Thread Timo Walther (Jira)
Timo Walther created FLINK-30226: Summary: Offer a Identity(De)SerializationSchema Key: FLINK-30226 URL: https://issues.apache.org/jira/browse/FLINK-30226 Project: Flink Issue Type

Re: [VOTE] Release flink-shaded 16.1, release candidate #1

2022-11-23 Thread Timo Walther
+1 (binding) Thanks, Timo On 23.11.22 15:53, Piotr Nowojski wrote: +1 (binding) czw., 10 lis 2022 o 11:38 Martijn Visser napisał(a): +1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified

Re: [DISCUSS] FLINK-20578 Cannot create empty array using ARRAY[]

2022-10-28 Thread Timo Walther
Actually, the new type inference stack for UDFs is smart enough to solve this issue. It could derive a data type for the array from the surrounding call (expected data type). So this can be supported with the right type inference logic: cast(ARRAY() as int) Unfortunately, ARRAY is fully

Re: [DISCUSS] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

2022-10-21 Thread Timo Walther
Makes sense to me. The serializer stack is pretty complex right now, the more legacy we remove the better. Regards, Timo On 20.10.22 12:49, Chesnay Schepler wrote: +1 Sounds like a good reason to drop these long-deprecated APIs. On 19/10/2022 15:13, Piotr Nowojski wrote: Hi devs, I would

Re: [VOTE] FLIP-260: Expose Finish Method For TableFunction

2022-09-22 Thread Timo Walther
+1 (binding) Please make also sure to update ExpressionReducer during implementation. This is not mentioned in the FLIP yet. Thanks, Timo On 22.09.22 10:41, Piotr Nowojski wrote: +1 (binding) czw., 22 wrz 2022 o 08:21 Lincoln Lee napisał(a): Hi everyone, Thanks for all your feedback

[jira] [Created] (FLINK-29379) Back ExecutionConfig and CheckpointConfig by Configuration

2022-09-21 Thread Timo Walther (Jira)
Timo Walther created FLINK-29379: Summary: Back ExecutionConfig and CheckpointConfig by Configuration Key: FLINK-29379 URL: https://issues.apache.org/jira/browse/FLINK-29379 Project: Flink

[jira] [Created] (FLINK-29309) Relax allow-client-job-configurations for Table API and parameters

2022-09-15 Thread Timo Walther (Jira)
Timo Walther created FLINK-29309: Summary: Relax allow-client-job-configurations for Table API and parameters Key: FLINK-29309 URL: https://issues.apache.org/jira/browse/FLINK-29309 Project: Flink

[jira] [Created] (FLINK-29267) Support external type systems in DDL

2022-09-12 Thread Timo Walther (Jira)
Timo Walther created FLINK-29267: Summary: Support external type systems in DDL Key: FLINK-29267 URL: https://issues.apache.org/jira/browse/FLINK-29267 Project: Flink Issue Type: Improvement

[ANNOUNCE] New Apache Flink PMC Member - Martijn Visser

2022-09-09 Thread Timo Walther
). Congratulations and welcome, Martijn! Cheers, Timo Walther (On behalf of the Apache Flink PMC)

  1   2   3   4   5   6   7   8   9   10   >