+1
Personally, I would declare it as `collect(): ClosableIterator` to
avoid an additional class in the API and reuse existing Flink utils.
Regards,
Timo
On 17.05.20 10:21, Jingsong Li wrote:
+1
Best,
Jingsong Lee
On Sun, May 17, 2020 at 3:42 PM Kurt Young wrote:
+1 from my side.
Best,
+1
Best,
Jingsong Lee
On Sun, May 17, 2020 at 3:42 PM Kurt Young wrote:
> +1 from my side.
>
> Best,
> Kurt
>
>
> On Sun, May 17, 2020 at 12:41 PM godfrey he wrote:
>
> > hi everyone,
> >
> > I would like to bring up another topic about the return value of
> > TableResult#collect() method.
>
+1 from my side.
Best,
Kurt
On Sun, May 17, 2020 at 12:41 PM godfrey he wrote:
> hi everyone,
>
> I would like to bring up another topic about the return value of
> TableResult#collect() method.
> Currently, the return type is `Iterator`, we meet some problems when
> implementing
hi everyone,
I would like to bring up another topic about the return value of
TableResult#collect() method.
Currently, the return type is `Iterator`, we meet some problems when
implementing FLINK-14807[1].
In current design, the sink operator has a buffer pool which buffers the
data from
Thanks for the update Godfrey!
+1 to this approach.
Since there can be only one primary key, I'd also be fine to just use `PRI`
even if it is composite, but `PRI(f0, f5)` might be more convenient for
users.
Thanks, Fabian
Am Do., 7. Mai 2020 um 09:31 Uhr schrieb godfrey he :
> Hi fabian,
>
Hi fabian,
Thanks for you suggestions.
Agree with you that `UNQ(f2, f3)` is more clear.
A table can have only ONE primary key,
this primary key can consist of single or multiple columns. [1]
if primary key consists of single column,
we can simply use `PRI` (or `PRI(xx)`) to represent it.
if
Hi Godfrey,
This looks good to me.
One side note, indicating unique constraints with "UNQ" is probably not
enough.
There might be multiple unique constraints and users would like to know
which field combinations are unique.
So in your example above, "UNQ(f2, f3)" might be a better marker.
Just
Hi @fhue...@gmail.com @Timo Walther @Dawid Wysakowicz
What do you think we limit watermark must be defined on top-level column ?
if we do that, we can add an expression column to represent watermark like
compute column,
An example of all cases:
create table MyTable (
f0 BIGINT NOT NULL,
f1
Hi Fabian,
the broken example is:
create table MyTable (
f0 BIGINT NOT NULL,
f1 ROW,
f2 VARCHAR<256>,
f3 AS f0 + 1,
PRIMARY KEY (f0),
UNIQUE (f3, f2),
WATERMARK f1.q2 AS (`f1.q2` - INTERVAL '3' SECOND)
) with (...)
name
type
key
compute column
watermark
f0
Hi,
I'm in favor of Fabian's proposal.
First, watermark is not a column, but a metadata just like primary key, so
shouldn't stand with columns.
Second, AFAIK, primary key can only be defined on top-level columns.
Third, I think watermark can also follow primary key than only allow to
define on
Hi Godfrey,
The formatting of your example seems to be broken.
Could you send them again please?
Regarding your points
> because watermark express can be a sub-column, just like `f1.q2` in above
example I give.
I would put the watermark information in the row of the top-level field and
indicate
Hi Fabian, Aljoscha
Thanks for the feedback.
Agree with you that we can deal with primary key as you mentioned.
now, the type column has contained the nullability attribute, e.g. BIGINT
NOT NULL.
(I'm also ok that we use two columns to represent type just like mysql)
>Why I treat `watermark` as
+1 I like the general idea of printing the results as a table.
On the specifics I don't know enough but Fabians suggestions seems to
make sense to me.
Aljoscha
On 29.04.20 10:56, Fabian Hueske wrote:
Hi Godfrey,
Thanks for starting this discussion!
In my mind, WATERMARK is a property (or
Hi Godfrey,
Thanks for starting this discussion!
In my mind, WATERMARK is a property (or constraint) of a field, just like
PRIMARY KEY.
Take this example from MySQL:
mysql> CREATE TABLE people (id INT NOT NULL, name VARCHAR(128) NOT NULL,
age INT, PRIMARY KEY (id));
Query OK, 0 rows affected
Hi everyone,
I would like to bring up a discussion about the result type of describe
statement,
which is introduced in FLIP-84[1].
In previous version, we define the result type of `describe` statement is a
single column as following
Statement
Result Schema
Result Value
Result Kind
Examples
Hi everyone,
I would like to bring up a discussion about the result type of describe
statement.
In previous version, we define the result type of describe statement is a
single column as following
Statement
Result Schema
Result Value
Result Kind
Examples
DESCRIBE xx
field name: result
Hi Timo,
Sorry for the late reply, and thanks for your correction.
I missed DQL for job submission scenario.
I'll fix the document right away.
Best,
Godfrey
Timo Walther 于2020年4月3日周五 下午9:53写道：
> Hi Godfrey,
>
> I'm sorry to jump in again but I still need to clarify some things
> around
Hi Godfrey,
I'm sorry to jump in again but I still need to clarify some things
around TableResult.
The FLIP says:
"For DML, this method returns TableResult until the job is submitted.
For other statements, TableResult is returned until the execution is
finished."
I thought we agreed on
Hi Aljoscha, Dawid, Timo,
Thanks so much for the detailed explanation.
Agree with you that the multiline story is not completed now, and we can
keep discussion.
I will add current discussions and conclusions to the FLIP.
Best,
Godfrey
Timo Walther 于2020年4月1日周三 下午11:27写道：
> Hi Godfrey,
>
>
Hi Godfrey,
first of all, I agree with Dawid. The multiline story is not completed
by this FLIP. It just verifies the big picture.
1. "control the execution logic through the proposed method if they know
what the statements are"
This is a good point that also Fabian raised in the linked
When considering the multi-line support I think it is helpful to start
with a use case in mind. In my opinion consumers of this method will be:
1. sql-client
2. third-part sql based platforms
@Godfrey As for the quit/source/... commands. I think those belong to
the responsibility of
One comment to `executeMultilineSql`, I'm afraid sometimes user might
forget to
iterate the returned iterators, e.g. user submits a bunch of DDLs and
expect the
framework will execute them one by one. But it didn't.
Best,
Kurt
On Wed, Apr 1, 2020 at 5:10 PM Aljoscha Krettek wrote:
> Agreed to
Agreed to what Dawid and Timo said.
To answer your question about multi line SQL: no, we don't think we need
this in Flink 1.11, we only wanted to make sure that the interfaces that
we now put in place will potentially allow this in the future.
Best,
Aljoscha
On 01.04.20 09:31, godfrey he
Hi Timo,
Regarding to "`execute` method throws checked exception",
is that mean we should convert the checked exception to unchecked
exception
or we need add ERROR type in ResultKind.
for the second approach, I still think it's not convenient
for the user to check exception when calling
Hi, Timo & Dawid,
Thanks so much for the effort of `multiline statements supporting`,
I have a few questions about this method:
1. users can well control the execution logic through the proposed method
if they know what the statements are (a statement is a DDL, a DML or
others).
but if a
Thank you Timo for the great summary! It covers (almost) all the topics.
Even though in the end we are not suggesting much changes to the current
state of FLIP I think it is important to lay out all possible use cases
so that we do not change the execution model every release.
There is one
Hi Godfrey,
Aljoscha, Dawid, Klou, and I had another discussion around FLIP-84. In
particular, we discussed how the current status of the FLIP and the
future requirements around multiline statements, async/sync, collect()
fit together.
We also updated the FLIP-84 Feedback Summary document
Hi, Timo & Jark
Thanks for your explanation.
Agree with you that async execution should always be async,
and sync execution scenario can be covered by async execution.
It helps provide an unified entry point for batch and streaming.
I think we can also use sync execution for some testing.
So, I
Hi,
I didn't follow the full discussion.
But I share the same concern with Timo that streaming queries should always
be async.
Otherwise, I can image it will cause a lot of confusion and problems if
users don't deeply keep the "sync" in mind (e.g. client hangs).
Besides, the streaming mode is
Hi Godfrey,
maybe I wasn't expressing my biggest concern enough in my last mail.
Even in a singleline and sync execution, I think that streaming queries
should not block the execution. Otherwise it is not possible to call
collect() or print() on them afterwards.
"there are too many things
Hi Timo,
Agree with you that streaming queries is our top priority,
but I think there are too many things need to discuss for multiline
statements:
e.g.
1. what's the behaivor of DDL and DML mixing for async execution:
create table t1 xxx;
create table t2 xxx;
insert into t2 select * from t1
Hi Godfrey,
executing streaming queries must be our top priority because this is
what distinguishes Flink from competitors. If we change the execution
behavior, we should think about the other cases as well to not break the
API a third time.
I fear that just having an async execute method
Hi Timo,
I agree with you that streaming queries mostly need async execution.
In fact, our original plan is only introducing sync methods in this FLIP,
and async methods (like "executeSqlAsync") will be introduced in the future
which is mentioned in the appendix.
Maybe the async methods also
Hi Timo,
Thanks for the updating.
Regarding to "multiline statement support", I'm also fine that
`TableEnvironment.executeSql()` only supports single line statement, and we
can support multiline statement later (needs more discussion about this).
Regarding to "StatementSet.explian()", I don't
Hi Jark,
good question. Actually, there was an ERROR kind that could have been
enabled via a config option. Such that everything ends up in the
TableResult. But @Kurt had some concerns which is why we didn't add this
kind of result yet.
Regards,
Timo
On 25.03.20 12:00, Jark Wu wrote:
Hi
Hi Godfrey,
The changes sounds good to me. +1 to start another voting.
A minor question: does the ResultKind contain an ERROR kind?
Best,
Jark
On Wed, 25 Mar 2020 at 18:51, Timo Walther wrote:
> Hi Godfrey,
>
> thanks for starting the discussion on the mailing list. And sorry again
> for
Hi Godfrey,
thanks for starting the discussion on the mailing list. And sorry again
for the late reply to FLIP-84. I have updated the Google doc one more
time to incorporate the offline discussions.
From Dawid's and my view, it is fine to postpone the multiline support
to a separate method.
37 matches
Mail list logo
