[DISCUSS] SQL CTAS Syntax

2021-05-27 Thread Danny Chan
table-datasource.html [3] Hive 3.0: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableCreate/Drop/TruncateTable [4] MySQL 5.6: https://dev.mysql.com/doc/refman/5.6/en/create-table-select.html Best, Danny Chan

Re: [DISCUSS] FLIP-154: SQL Implicit Type Coercion

2021-03-10 Thread Danny Chan
Thanks for the feedback, Jingsong ~ This design mainly follows the behaviors of PostgreSQL and SQL-SERVER, because their rules are more in line with the SQL standard. I have fixed the WIKI and add more details about the diff in it. Best, Danny Chan Jingsong Li 于2021年3月11日周四 下午1:34写道

Re: Upgrade calcite version

2021-03-10 Thread Danny Chan
basic and SQL planning (the relational algebra) that really few people have, so i would not suggest to do that at all for yourself. What is the purpose to must upgrade Calcite, can you share something ~ Best, Danny Chan 盛森林 于2021年2月4日周四 下午10:47写道: > Hi, > I want to upgrade calcite t

[DISCUSS] FLIP-154: SQL Implicit Type Coercion

2021-03-09 Thread Danny Chan
for the type coercion rules (the matrix) for Flink SQL soon. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-154%3A+SQL+Implicit+Type+Coercion Best, Danny Chan

Re: Re: [ANNOUNCE] Welcome Guowei Ma as a new Apache Flink Committer

2021-01-19 Thread Danny Chan
Congratulations Guowei! Best, Danny Jark Wu 于2021年1月20日周三 下午2:47写道: > Congratulations Guowei! > > Cheers, > Jark > > On Wed, 20 Jan 2021 at 14:36, SHI Xiaogang wrote: > > > Congratulations MA! > > > > Regards, > > Xiaogang > > > > Yun Tang 于2021年1月20日周三 下午2:24写道: > > > > > Congratulations

Re: [DISCUSS] Make Temporal Join syntax easier to use

2020-12-22 Thread Danny Chan
Thanks Shuo Cheng for driving this discussion ~ Generally i have the same feeling that our temporal table syntax is a little verbose compared to KSQL or other RDBMS. (they just use the normal join syntax for the temporal join). >I think maybe we can add a pseudo-column `PROCTIME` (or `_PROCTIME`

Re: Re: Re: [VOTE] FLIP-145: Support SQL windowing table-valued function (2nd)

2020-11-11 Thread Danny Chan
+1 Jingsong Li 于2020年11月12日周四 下午2:18写道: > +1 > > On Thu, Nov 12, 2020 at 2:07 PM hailongwang <18868816...@163.com> wrote: > > > > > > > +1(no-binding) > > > > > > > > > > 在 2020-11-11 19:44:40,"刘大龙" 写道: > > > > > >+1 > > > > > >> -原始邮件- > > >> 发件人: "Timo Walther" > > >> 发送时间:

Re: [ANNOUNCE] New Apache Flink Committer - Congxian Qiu

2020-11-01 Thread Danny Chan
Congrats, Doctor Qiu! Well deserved! Congxian Qiu 于2020年10月31日周六 下午9:45写道: > Thanks all for the support. It's a great honor for me. > > Best, > Congxian > > > Paul Lam 于2020年10月30日周五 下午3:18写道: > > > Congrats, Congxian! Well deserved! > > > > Best, > > Paul Lam > > > > > 2020年10月30日

Re: [VOTE]FLIP-149: Introduce the upsert-kafka connector

2020-10-27 Thread Danny Chan
+1 Timo Walther 于2020年10月23日周五 下午5:22写道: > +1 > > Thanks, > Timo > > On 23.10.20 10:21, Jingsong Li wrote: > > +1 > > > > On Fri, Oct 23, 2020 at 3:52 PM Konstantin Knauf > wrote: > > > >> +1 > >> > >> On Fri, Oct 23, 2020 at 9:36 AM Jark Wu wrote: > >> > >>> +1 > >>> > >>> On Fri, 23 Oct

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-19 Thread Danny Chan
Per FLIP-145, we have many runtime operators to implement and bridge it with the planner. Best, Danny Chan 在 2020年10月19日 +0800 PM6:59,Robert Metzger ,写道: > Thank you for your responses so far. > > @Kurt, Jingsong, Danny: Which JIRAs/FLIPs are going to benefit from the > extension

Re: [DISCUSS] Release 1.12 Feature Freeze

2020-10-19 Thread Danny Chan
+1 for Kurt suggestion, there are many features for SQL yet, 2 more days are valuable. Best, Danny Chan 在 2020年10月19日 +0800 PM6:22,Jingsong Li ,写道: > Hi Robert, > > Thanks for your detailed explanation. > > At present, we are preparing or participating in Flink forward, so +1 fo

Re: [DISCUSS] FLIP-149: Introduce the KTable Connector

2020-10-19 Thread Danny Chan
. For the source > Shouldn’t this be an extension of existing Kafka connector instead of a >totally new connector ? How could we achieve that (e.g. set up the parallelism correctly) ? Best, Danny Chan 在 2020年10月19日 +0800 PM5:17,Jingsong Li ,写道: > Thanks Shengkai for your proposa

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Danny Chan
+1, nice job ! Best, Danny Chan 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: > Hi all, > > I would like to start the vote for FLIP-146 [1], which is discussed and > reached consensus in the discussion thread [2]. The vote will be open until > 20th Oct. (72h, exclude we

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-15 Thread Danny Chan
t have meanings. For un-bounded data set we already has the watermark to handle the out-of-orderness data, and for bounded data set, we can use the regular sort here because current table argument allows any query actually. Best, Danny Chan 在 2020年10月15日 +0800 PM5:16,dev@flink.apache.org,写道: >

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-10 Thread Danny Chan
TVF as a normal UDTF, it can be chained with any kind of other non-windowed operators, we may need some time to give clear semantics for them (especially it is good if there are real use cases), as a start, let us focus the windowed operations first. I'm also +1 for voting ~ Best, Danny Ch

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-08 Thread Danny Chan
Congrats, Zhu Zhu ~ Best, Danny Chan 在 2020年10月9日 +0800 PM1:05,dev@flink.apache.org,写道: > > Congrats, Zhu Zhu

Re: Re: [ANNOUNCE] New Apache Flink Committer - Godfrey He

2020-09-22 Thread Danny Chan
Congrats Godfrey! Well deserved! Best, Danny Chan 在 2020年9月21日 +0800 AM10:16,dev@flink.apache.org,写道: > > Congrats Godfrey! Well deserved!

Re: [ANNOUNCE] New Apache Flink Committer - Niels Basjes

2020-09-14 Thread Danny Chan
Congratulations!  Best, Danny Chan 在 2020年9月15日 +0800 AM9:31,dev@flink.apache.org,写道: > > Congratulations! 

Re: [VOTE] FLIP-107: Handling of metadata in SQL connectors

2020-09-12 Thread Danny Chan
+1, non-binding ~ Konstantin Knauf 于2020年9月11日 周五上午2:04写道: > +1 (binding) > > > > On Thu, Sep 10, 2020 at 4:29 PM Dawid Wysakowicz > > wrote: > > > > > +1 (binding) > > > > > > On 10/09/2020 14:03, Aljoscha Krettek wrote: > > > > +1 (binding) > > > > > > > > Aljoscha > > > > > > > > On 10.09.20

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-10 Thread Danny Chan
Thanks for driving this Timo, +1 for voting ~ Best, Danny Chan 在 2020年9月10日 +0800 PM3:54,Timo Walther ,写道: > Thanks everyone for this healthy discussion. I updated the FLIP with the > outcome. I think the result is one of the last core API refactoring and > users will be happ

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-10 Thread Danny Chan
Thanks for driving this Timo, +1 for voting ~ Best, Danny Chan 在 2020年9月10日 +0800 PM3:47,Timo Walther ,写道: > Thanks everyone for this healthy discussion. I updated the FLIP with the > outcome. I think the result is very powerful but also very easy to > declare. Thanks for all the cont

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Danny Chan
en continue to a > vote. > > Regards, > Timo > > > On 09.09.20 10:50, Danny Chan wrote: > > I think it would bring in much confusion by a different API name just > because the DataStream generic type is different. > > If there are ChangelogMode that only works for Row,

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Danny Chan
t; > > If we want to use "offset INT SYSTEM_METADATA("offset")", then I think we > > must further discuss about "PERSISED" or "VIRTUAL" keyword for query-sink > > schema problem. > > Personally, I think we can use a shorter keyword "

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-09 Thread Danny Chan
uently used > > >>>> properties > > >>>>>>>> are not hidden in the MAP anymore: > > >>>>>>>> > > >>>>>>>> debezium-json.ingestion-timestamp > > >>>>>>>> debezium-json.source.timestamp > > >&g

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Danny Chan
to declare the ChangelogMode explicitly, and there are some confusions: • Should DataStream of Row type always use #fromChangelogStream ? • Does fromChangelogStream works for only INSERT ChangelogMode ? Best, Danny Chan 在 2020年9月9日 +0800 PM4:21,Timo Walther ,写道: > I had this in the inital des

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Danny Chan
), the ChangelogMode is INSERT. Best, Danny Chan 在 2020年9月9日 +0800 PM2:53,dev@flink.apache.org,写道: > > But I think the planner needs to > know whether the input is insert-only or not.

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-09 Thread Danny Chan
Thanks for the summary Timo ~ I want to clarify a little bit, what is the conclusion about the fromChangelogStream and toChangelogStream, should we use this name or we use fromDataStream with an optional ChangelogMode flag ? Best, Danny Chan 在 2020年9月8日 +0800 PM8:22,Timo Walther ,写道: >

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-08 Thread Danny Chan
is the behavior if fromInsertStream() accepts a changelog stream ? "This means potentially duplicate definition of fields and their data types etc” I agree, because table already has an underlying schema there. Best, Danny Chan 在 2020年9月3日 +0800 PM8:12,Timo Walther ,写道: > Hi Danny, >

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-09-07 Thread Danny Chan
cal value can be any java object. [1] https://oracle-base.com/articles/11g/virtual-columns-11gr1 [2] https://dev.mysql.com/doc/refman/5.7/en/create-table-generated-columns.html Best, Danny Chan 在 2020年9月4日 +0800 PM4:48,Timo Walther ,写道: > Hi everyone, > > I completely reworked FLIP-107. It

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-03 Thread Danny Chan
rnned internal row, if people did want a POJO there, how about we bind the DataType to the existing schema, like this toDataStream(table, schema.bindTo(DataType)) Best, Danny Chan 在 2020年9月3日 +0800 PM3:18,dev@flink.apache.org,写道: > > It is a more conservative approach to introduce that in a &

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-02 Thread Danny Chan
thing from the API level, although I would not vote  `fromDataStream(DataStream, Expression…)` it is still better than `fromInsertStream(DataStream).select()`. Best, Danny Chan 在 2020年9月2日 +0800 PM4:19,Timo Walther ,写道: > Hi everyone > > thanks for your feedback. It's a lot of cont

Re: [DISCUSS] FLIP-136: Improve interoperability between DataStream and Table API

2020-09-01 Thread Danny Chan
tream API explicitly. In the SQL world, no projection function outputs type of time-attribute, we better still put the time-attributes in the scope of the table metadata. Best, Danny Chan 在 2020年8月19日 +0800 PM4:22,Timo Walther ,写道: > Hi everyone, > > I would like to propose a FLIP that aim

Re: [ANNOUNCE] New Flink Committer: David Anderson

2020-08-12 Thread Danny Chan
Congratulations, David! jincheng sun 于2020年8月13日 周四上午8:54写道: > Congrats and welcome David! > > Best, > Jincheng > > > David Anderson 于2020年8月13日周四 上午2:51写道: > > > Thank you one and all for the warm reception! > > > > I'm honored to join the group of committers, and look forward to > continuing

Re: [ANNOUNCE] New PMC member: Piotr Nowojski

2020-07-08 Thread Danny Chan
Congratulations and nice work Piotr ~ Best, Danny Chan 在 2020年7月7日 +0800 PM10:00,dev@flink.apache.org,写道: > > Congratulations!

Re: [ANNOUNCE] New Flink Committer: Benchao Li

2020-06-10 Thread Danny Chan
Congrats Benchao! Best, Danny Chan 在 2020年6月10日 +0800 AM11:57,dev@flink.apache.org,写道: > > Congrats Benchao!

Re: [ANNOUNCE] New Apache Flink Committer - Xintong Song

2020-06-05 Thread Danny Chan
Congratulations Xintong ! Best, Danny Chan 在 2020年6月5日 +0800 PM2:20,dev@flink.apache.org,写道: > > Congratulations Xintong

Re: [DISCUSS] Releasing "fat" and "slim" Flink distributions

2020-06-05 Thread Danny Chan
+1, at least, we should keep an out of the box SQL-CLI, it’s very poor experience to add such required format jars for SQL users. Best, Danny Chan 在 2020年6月5日 +0800 AM11:14,Jingsong Li ,写道: > Hi all, > > Considering that 1.11 will be released soon, what about my previous > proposa

Re: [DISCUSS] Exact feature freeze date

2020-05-15 Thread Danny Chan
+1 for Monday morning in Europe. Best, Danny Chan 在 2020年5月15日 +0800 PM10:51,Zhijiang ,写道: > +1 for Monday morning in Europe. > > Best, > Zhijiang > > > -- > From:Yun Tang > Send Time:2020年5月15日(星期五) 2

Re: changing the output files names in Streamfilesink from part-00 to something else

2020-05-13 Thread Danny Chan
/filesystem/StreamingFileSink.java#L71 Best, Danny Chan 在 2020年5月14日 +0800 AM2:05,dhurandar S ,写道: > > StreamFileSink

Re: [DISCUSS] Hierarchies in ConfigOption

2020-05-06 Thread Danny Chan
t encounter, that means we force many users to obey rules for cases they would never have. [1] https://docs.ksqldb.io/en/latest/developer-guide/create-a-table/ [2] https://prestodb.io/docs/current/sql/create-table.html Best, Danny Chan 在 2020年5月4日 +0800 PM11:34,Till Rohrmann ,写道: > Hi everyone, >

Re: [DISCUSS] Exact feature freeze date

2020-04-23 Thread Danny Chan
+1 for extending the feature freeze until May 15th. Best, Danny Chan 在 2020年4月24日 +0800 AM9:51,Yangze Guo ,写道: > +1 > > Best, > Yangze Guo > > On Fri, Apr 24, 2020 at 9:49 AM Dian Fu wrote: > > > > +1 > > > > Regards, > > Dian > > >

Re: [ANNOUNCE] New Apache Flink PMC Member - Hequn Chen

2020-04-17 Thread Danny Chan
Congratulations, Hequn, and thanks for your work for community ~ Best, Danny Chan 在 2020年4月17日 +0800 PM2:44,Becket Qin ,写道: > Hi all, > > I am glad to announce that Hequn Chen has joined the Flink PMC. > > Hequn has contributed to Flink for years. He has worked on several > co

[RESULT][VOTE] FLIP-113: Support dynamic table options for Flink SQL

2020-04-12 Thread Danny Chan
everyone for joining the discussion and giving feedback! Best, Danny Chan

[VOTE] FLIP-113: Supports Dynamic Table Options for Flink SQL

2020-04-09 Thread Danny Chan
Hi all, I would like to start the vote for FLIP-113 [1], which is discussed and reached a consensus in the discussion thread [2]. The vote will be open until April 13nd (72h), unless there is an objection or not enough votes. Best, Danny Chan [1] https://cwiki.apache.org/confluence/display

Re: [VOTE] FLIP-71: E2E View support in Flink SQL

2020-04-09 Thread Danny Chan
+1 from my side. Best, Danny Chan 在 2020年4月9日 +0800 PM9:23,Timo Walther ,写道: > +1 (binding) > > Thanks for your efforts. > > Regards, > Timo > > > On 09.04.20 14:46, Zhenghua Gao wrote: > > Hi all, > > > > I'd like to start the vote for FLIP-71[1]

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-04-08 Thread Danny Chan
t; On Wed, 8 Apr 2020 at 18:59, Danny Chan wrote: > > > `table.dynamic-table-options.enabled` seems fine to me, I would make a > new > > `TableConfigOptions` class and put the config option there ~ > > > > What do you think about the new class to put ? > >

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-04-08 Thread Danny Chan
`table.dynamic-table-options.enabled` seems fine to me, I would make a new `TableConfigOptions` class and put the config option there ~ What do you think about the new class to put ? Best, Danny Chan 在 2020年4月8日 +0800 PM5:33,dev@flink.apache.org,写道: > > `table.dynamic-table-options.enabled`

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-04-07 Thread Danny Chan
> > > > disable this by default I would be (reluctantly) OK with the feature. > > > > Companies that create deployments or set up the SQL environment for > > > > users can enable the feature if they want. > > > > > > > > But yes, I also agree that we

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-04-05 Thread Danny Chan
use blacklist makes the code not too verbose to enumerate all the duplicate keys with #supportedKeys .(Not very strong reason, but i think as a connector developer, it makes sense) Best, Danny Chan 在 2020年4月3日 +0800 PM11:28,Timo Walther ,写道: > Hi everyone, > > @Aljoscha: I disagr

Re: [DISCUSS] Change default planner to blink planner in 1.11

2020-04-01 Thread Danny Chan
+1 to use blink-planner as the default. It’s really awful to upgrade Calcite for both planners each time. I believe user would also want a default planner that is more powerful. Best, Danny Chan 在 2020年4月1日 +0800 PM10:26,Leonard Xu ,写道: > +1 to make the Blink planner be the planner in 1

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-04-01 Thread Danny Chan
or keys, and it does not support wildcard yet. Best, Danny Chan 在 2020年4月1日 +0800 PM9:12,Dawid Wysakowicz ,写道: > Hi, > Few comments from my side: > 1. Regarding the motivation: > I think the example with changing the update-mode is not a good one. In the > long term this should be done

Re: [VOTE] FLIP-113: Supports Dynamic Table Options for Flink SQL

2020-04-01 Thread Danny Chan
Thanks, the vote cancels, I would fire a new one if we make agreement in the discussion. Best, Danny Chan 在 2020年4月1日 +0800 AM10:38,Jingsong Li ,写道: > Hi Danny, > > If I haven't missed anything, I don't know how to support the dynamic > options of format. > Connector do

Re: [ANNOUNCE] New Committers and PMC member

2020-04-01 Thread Danny Chan
Congratulations! Best, Danny Chan 在 2020年4月1日 +0800 PM7:36,dev@flink.apache.org,写道: > > Congratulations!

Re: [VOTE] FLIP-110: Support LIKE clause in CREATE TABLE

2020-03-31 Thread Danny Chan
+1 for this feature, although the WITH syntax breaks the SQL standard, but it’s compatible with our CREATE TABLE syntax, seems well from my side. Best, Danny Chan 在 2020年3月31日 +0800 PM5:46,Dawid Wysakowicz ,写道: > Hi, > > Just wanted to notify the voters that after a comment from J

Re: [DISCUSS] FLIP-122: New Connector Property Keys for New Factory

2020-03-30 Thread Danny Chan
, such as “some-key-prefix” = “k1=v1, k2=v2 ..." • Should the ConfigOption support the wildcard ? (If we plan to support the current multi-line style) Best, Danny Chan 在 2020年3月31日 +0800 AM12:37,Jark Wu ,写道: > Hi all, > > Thanks for the feedbacks. > > It seems that we have

[VOTE] FLIP-113: Supports Dynamic Table Options for Flink SQL

2020-03-30 Thread Danny Chan
Hi all, I would like to start the vote for FLIP-113 [1], which is discussed and reached a consensus in the discussion thread [2]. The vote will be open until April 2nd (72h), unless there is an objection or not enough votes. Thanks, Danny Chan [1] https://cwiki.apache.org/confluence/display

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-26 Thread Danny Chan
Sorry, i meant white-list ~ Danny Chan 于2020年3月27日周五 下午12:40写道: > Thanks everyone for the feedback ~ > > - For the global config option belongs to `ExecutionConfigOptions` or > `OptimizerConfigOptions`, i have to strong objections, switch > to `OptimizerConfigOptions` is okey t

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-26 Thread Danny Chan
to > use a whitelist and force people to think about what should be exposed > as a hint. By default, no option should be exposed. It is better to be > conservative here. > > Regards, > Timo > > > On 26.03.20 10:31, Danny Chan wrote: > > Thanks Kurt for the s

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-26 Thread Danny Chan
ble-options.enabled" should belong to ` > > *OptimizerConfigOptions*` > > > > Best, > > Kurt > > > > > > On Thu, Mar 26, 2020 at 4:40 PM Timo Walther wrote: > > > > > Thanks for the update Danny. +1 for this proposal. > > > > &

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-26 Thread Danny Chan
to ` > *OptimizerConfigOptions*` > > Best, > Kurt > > > On Thu, Mar 26, 2020 at 4:40 PM Timo Walther wrote: > > > Thanks for the update Danny. +1 for this proposal. > > > > Regards, > > Timo > > > > On 26.03.20 04:51, Danny Chan wrote: >

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-25 Thread Danny Chan
g/confluence/display/FLINK/FLIP-113%3A+Supports+Dynamic+Table+Options+for+Flink+SQL Best, Danny Chan Jark Wu 于2020年3月18日周三 下午10:38写道: > Hi everyone, > > Sorry, but I'm not sure about the `supportedHintOptions`. I'm afraid it > doesn't solve the problems but increases some de

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-18 Thread Danny Chan
/CsvTableSourceFactoryBase.java#L92 Best, Danny Chan 在 2020年3月18日 +0800 PM9:10,Timo Walther ,写道: > Hi everyone, > > +1 to Kurt's suggestion. Let's just have it in source and sink factories > for now. We can still move this method up in the future. Currently, I > don't see a ne

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-17 Thread Danny Chan
Yes, I think we should move the `supportedHintOptions` from TableFactory to TableSourceFactory, and we also need to add the interface to TableSinkFactory though because sink target table may also have hints attached. Best, Danny Chan 在 2020年3月18日 +0800 AM11:08,Kurt Young ,写道: > Have

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-17 Thread Danny Chan
factories in order to merge these properties (and the logic are almost the same), what do you think about this? [1]  https://docs.databricks.com/spark/latest/spark-sql/language-manual/create-table.html Best, Danny Chan 在 2020年3月17日 +0800 PM10:10,Timo Walther ,写道: > Hi Danny, > > thanks for

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-12 Thread Danny Chan
gt; query would have better user experience. > > > > Both 2 & 3 can make this happen. The cons of #2 is breaking SQL > compliant, > > and for #3, it only breaks some > > unwritten rules, but we can have an explanation on that. And I really > doubt > > whether

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-11 Thread Danny Chan
eta like index column, and it is very convenient to specify table properties. Or shall we not call  /*+ PROPERTIES(offset=123) */ table hint, we can call it table dynamic parameters. Best, Danny Chan 在 2020年3月11日 +0800 PM9:20,Aljoscha Krettek ,写道: > Hi, > > I don't understand this discu

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-11 Thread Danny Chan
the hint syntax without changing the SQL parser, and makes clear that we are at liberty to ignore hints entirely. Julian [1] https://www.amazon.com/s?k=9781565924017  <https://www.amazon.com/s?k=9781565924017> Best, Danny Chan 在 2020年3月11日 +0800 PM4:03,Timo Walther ,写道: >

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-11 Thread Danny Chan
can reference it in the query, I’m confused for that, can you elaborate a little ? Best, Danny Chan 在 2020年3月11日 +0800 PM12:52,Bowen Li ,写道: > Thanks Danny for kicking off the effort > > The root cause of too much manual work is Flink DDL has mixed 3 types of > params together and doesn't han

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-10 Thread Danny Chan
annot > overwrite existing properties? I think this would be a different story... > > Regards, > Timo > > > On 10.03.20 10:34, Danny Chan wrote: > > Thanks Timo ~ > > > > Personally I would say that offset > 0 and start offset = 10 does not > have the sa

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-10 Thread Danny Chan
prune, we can do that if we have an offset column, but most of the case we do not define that, so there is actually no conflict or overlap. Best, Danny Chan 在 2020年3月10日 +0800 PM4:28,Timo Walther ,写道: > Hi Danny, > > shouldn't FLIP-110[1] solve most of the problems we have around defin

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-10 Thread Danny Chan
or a string literal, which means that it’s compatible with our DDL syntax. We support simple identifier because many other hints do not have the component complex keys like the table properties, and we want to unify the parse block. Best, Danny Chan 在 2020年3月10日 +0800 PM3:19,wenlong.lwl ,写道: > Hi Da

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-10 Thread Danny Chan
have removed the query hints part and change the title. [1]  https://docs.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-query?view=sql-server-ver15 Best, Danny Chan 在 2020年3月9日 +0800 PM5:46,Timo Walther ,写道: > Hi Danny, > > thanks for the proposal. I agree with Jark and Jingsong

Re: [DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-09 Thread Danny Chan
Note: we only plan to support table hints in Flink release 1.11, so please focus mainly on the table hints part and just ignore the planner hints, sorry for that mistake ~ Best, Danny Chan 在 2020年3月9日 +0800 PM4:36,Danny Chan ,写道: > Hi, fellows ~ > > I would like to propose the support

[DISCUSS]FLIP-113: Support SQL and planner hints

2020-03-09 Thread Danny Chan
comments. You can access the FLIP here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-113%3A+SQL+and+Planner+Hints Best, Danny Chan

Re: [DISCUSS] FLIP-107: Reading table columns from different parts of source records

2020-03-03 Thread Danny Chan
1270 Best, Danny Chan 在 2020年3月2日 +0800 PM6:16,Dawid Wysakowicz ,写道: > Hi Jark, > Ad. 2 I added a section to discuss relation to FLIP-63 > Ad. 3 Yes, I also tried to somewhat keep hierarchy of properties. Therefore > you have the key.format.type. > I also considered exactly what you

Re: [DISCUSS] Drop connectors for Elasticsearch 2.x and 5.x

2020-02-10 Thread Danny Chan
5.x seems to have a lot of users, is the 6.x completely compatible with 5.x ~ Best, Danny Chan 在 2020年2月10日 +0800 PM9:45,Dawid Wysakowicz ,写道: > Hi all, > > As described in this https://issues.apache.org/jira/browse/FLINK-11720 > ticket our elasticsearch 5.x connector does not work ou

Re: [DISCUSS] Remove registration of TableSource/TableSink in Table Env and ConnectTableDescriptor

2020-02-05 Thread Danny Chan
that cannot be specified though properties, removing them force us to re-think the TableFactory/properties interface. Best, Danny Chan 在 2020年2月5日 +0800 PM3:58,Dawid Wysakowicz ,写道: > Hi Kurt, > > I fully agree with the proposal. Yes it was an omission that we did not > deprecate the Tabl

Re: [VOTE] FLIP-27 - Refactor Source Interface

2020-02-03 Thread Danny Chan
+1 (non-binding), thanks for the great work ! Best, Danny Chan 在 2020年2月4日 +0800 AM11:20,dev@flink.apache.org,写道: > > +1 (non-binding), thanks for driving. > FLIP-27 is the basis of a lot of follow-up work.

Re: [ANNOUNCE] Yu Li became a Flink committer

2020-02-03 Thread Danny Chan
Congratulations Yu! Best, Danny Chan 在 2020年2月4日 +0800 AM1:46,dev@flink.apache.org,写道: > > Congratulations Yu!

Re: [VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2020-01-14 Thread Danny Chan
+1 (non-binding) Best, Danny Chan 在 2019年12月31日 +0800 PM5:09,Forward Xu ,写道: > Hi all, > > I'd like to start the vote of FLIP-90 [1] since that we have reached an > agreement on the design in the discussion thread [2]. > > This vote will be open for at least 72 hours. Unless the

Re: [VOTE] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2020-01-09 Thread Danny Chan
Thanks Forward ~ +1 from my side and would review your Calcite PR this weekend :) Overall it looks good, and I believe we can merge it soon ~ Best, Danny Chan 在 2020年1月10日 +0800 AM11:04,Jark Wu ,写道: > Thanks Forward for driving this, > > The design doc looks very good to me. > +1

Re: [DISCUSS] FLIP-92: JDBC catalog and Postgres catalog

2020-01-09 Thread Danny Chan
fference-between-schema-database-in-mysql Best, Danny Chan 在 2020年1月10日 +0800 AM10:34,Jark Wu ,写道: > Thanks Bowen for the reply, > > A user-facing JDBCCatalog and 'catalog.type' = 'jdbc' sounds good to me. > > I have some other minor comments when I went through the updated > docum

Re: [DISCUSS] FLIP-90: Support SQL 2016-2017 JSON functions in Flink SQL

2020-01-07 Thread Danny Chan
Thanks Forward ~ I would help to view code from the Calcite side. Best, Danny Chan 在 2020年1月7日 +0800 PM2:01,Forward Xu ,写道: > Thanks Jark for checking the doc. hi,Timo, please help to check to see if > there is anything else to add. > > Best, > Forward > > Jark Wu

Re: [ANNOUNCE] Progress of Apache Flink 1.10 #3

2020-01-02 Thread Danny Chan
Thanks for the great work ~ Best, Danny Chan 在 2020年1月2日 +0800 PM8:48,Yu Li ,写道: > Hi community, > > It has been three weeks since we cut the release-1.10 branch and kicked off > the release testing phase [1]. Thanks to all the efforts of the community, > 46 blocker and 11 cri

Re: What is the suggested way to validate SQL?

2019-12-30 Thread Danny Chan
rcion But this would comes for Flink version 1.11 ~ Best, Danny Chan 在 2019年12月27日 +0800 PM5:44,dev@flink.apache.org,写道: > > "INSERT INTO > sinkTable SELECT f1,f2 FROM sourceTable"

Re: [DISCUSS] Correct the terminology of "Time-windowed Join" to "Interval Join" in Table API & SQL

2019-12-22 Thread Danny Chan
[1] https://github.com/apache/flink/blob/cce1cef50d993aba5060ea5ac597174525ae895e/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/datastream/KeyedStream.java#L449 Best, Danny Chan 在 2019年12月23日 +0800 AM11:42,Jark Wu ,写道: > Hi everyone, > > Currently, in the Table API & S

Re: [DISCUSS] have separate Flink distributions with built-in Hive dependencies

2019-12-15 Thread Danny Chan
-hive-version-uber.jar to use as a lib of SQL-CLI or other use cases. [1] https://spark.apache.org/downloads.html [2] https://www.elastic.co/guide/en/elasticsearch/hadoop/current/hive.html Best, Danny Chan 在 2019年12月14日 +0800 AM3:03,dev@flink.apache.org,写道: > > https://ci.apache.org/projects

Re: [ANNOUNCE] Zhu Zhu becomes a Flink committer

2019-12-15 Thread Danny Chan
Congrats Zhu Zhu! Best, Danny Chan 在 2019年12月14日 +0800 AM12:51,dev@flink.apache.org,写道: > > Congrats Zhu Zhu and welcome on board!

Re: [DISCUSS] Overwrite and partition inserting support in 1.10

2019-12-11 Thread Danny Chan
is resolved. Best, Danny Chan 在 2019年12月11日 +0800 PM5:29,Jingsong Li ,写道: > Hi Dev, > > After cutting out the branch of 1.10, I tried the following functions of > SQL-CLI and found that it does not support: > - insert overwrite > - PARTITION (partcol1=val1, partcol2=val2 ...)

Re: [ANNOUNCE] Feature freeze for Apache Flink 1.10.0 release

2019-12-10 Thread Danny Chan
Hi Yu & Gary, Thanks for your nice job ~ Best, Danny Chan 在 2019年11月19日 +0800 PM11:44,dev@flink.apache.org,写道: > > Hi Yu & Gary, > > Thanks a lot for your work and looking forward to the 1.10 release. :)

Re: [VOTE] Release 1.8.3, release candidate #3

2019-12-09 Thread Danny Chan
Hi Hequn, +1 (non-binding) - verified checksums and hashes - built from local with MacOS 10.14 and JDK8 - do some check in the SQL-CLI - run some tests in IDE Best, Danny Chan 在 2019年12月5日 +0800 PM9:39,Hequn Cheng ,写道: > Hi everyone, > > Please review and vote on the release can

Re: FLIP 87: Primary key constraints in Table API

2019-11-24 Thread Danny Chan
Thanks for the work, Dawid, a big + 1 ! Best, Danny Chan 在 2019年11月22日 +0800 PM8:51,Dawid Wysakowicz ,写道: > Hi everyone, > > I would like to start a vote on FLIP-87. > > Please vote for the following design document: > > https://cwiki.apache.org/confluence/display/FLINK/F

Re: [DISCUSS] Releasing Flink 1.8.3

2019-11-10 Thread Danny Chan
Thanks Hequn to be in change of the release ! Best, Danny Chan 在 2019年11月9日 +0800 PM3:59,Hequn Cheng ,写道: > Hi Jincheng, > > Thanks for kicking this discussion off! > > +1 to the 1.8.3 release as it would be nice to have these important fixes > and also two > months have

Re: [ANNOUNCE] Jark Wu is now part of the Flink PMC

2019-11-10 Thread Danny Chan
Congrats Jark! Well deserved ! Best, Danny Chan 在 2019年11月11日 +0800 AM9:59,dev@flink.apache.org,写道: > > Congrats Jark!

Re: [VOTE] FLIP-59: Enable execution configuration from Configuration object

2019-11-08 Thread Danny Chan
Thanks for driving on this, David. +1(non-binding) from my side, we really need this feature ! Best, Danny Chan 在 2019年11月7日 +0800 PM5:04,Dawid Wysakowicz ,写道: > Hello, > > please vote for FLIP-59 > <https://cwiki.apache.org/confluence/display/FLINK/FLIP-59%3A+Enable+executio

Re: [VOTE] FLIP-69: Flink SQL DDL Enhancement

2019-11-08 Thread Danny Chan
+1(non-binding), nice job, Terry ~ Best, Danny Chan 在 2019年11月5日 +0800 PM10:32,dev@flink.apache.org,写道: > > +1 to the long missing feature in Flink SQL.

[RESULT][VOTE] FLIP-70: Flink SQL Computed Column Design

2019-10-31 Thread Danny Chan
) There were no disapproving votes. Thus, FLIP-70 has been accepted. Thanks everyone for joining the discussion and giving feedback! Best, Danny Chan

Re: [DISCUSS] Move flink-orc to flink-formats from flink-connectors

2019-10-29 Thread Danny Chan
+1 to move to flink-format. Best, Danny Chan 在 2019年10月29日 +0800 AM11:10,dev@flink.apache.org,写道: > > +1 to move to flink-format.

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-29 Thread Danny Chan
Congratulations :) Best, Danny Chan 在 2019年10月29日 +0800 PM4:14,dev@flink.apache.org,写道: > > Congratulations :)

[VOTE] FLIP-70: Flink SQL Computed Column Design

2019-10-28 Thread Danny Chan
://cwiki.apache.org/confluence/display/FLINK/FLIP-70%3A+Flink+SQL+Computed+Column+Design [2]  http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-70-Support-Computed-Column-for-Flink-SQL-td33126.html Best, Danny Chan

  1   2   >