Thanks all,
As your suggestion, I'd like to do:
- FLIP-63: move "CREATE TABLE ... PARTITIONED BY" and "MSCK REPAIR TABLE"
to further discussion chapter.
- Remove hive dialect limitation for supported "INSERT OVERWRITE" and
"INSERT ... PARTITION(...)".
- Limit "CREATE TABLE ... PARTITIONED BY" to h
Thanks all for the healthy discussions. I'd just like to point out a light
difference between standard and standard compatibility. Most of DB vendors
meant the latter when they claim following a sql standard. However, it
doesn't mean they don't have any syntax beyond the standard grammar.
Extensio
Hi Timo,
Thanks for your feedback.
The reason of `The DDL can like this (With hive dialect)` is:
The syntax of creating partition table is controversial, so we think we
should put it aside for the time being to make it invisible to users. Since
we implemented this syntax in 1.9, we decided to put
Hi everyone,
sorry, I was not aware that FLIP-63 already lists a lot of additional
SQL grammar. It was accepted though an official voting process so I
guess we can adopt the listed grammar for Flink SQL.
The only thing that confuses me is the mentioning of `The DDL can like
this (With hive d
Hi Timo,
I understand we need further discussion about syntax/dialect for 1.11. But
as Jark has pointed out, the current implementation violates the accepted
design of FLIP-63, which IMO qualifies as a bug. Given that it's a bug and
has great impact on the usability of our Hive integration, do you
Hi Timo,
I am OK if you think they are not bug and they should not be included in
1.10.
I think they have been accepted in FLIP-63. And there is no objection. It
has been more than three months since the discussion of FLIP-63. It's been
six months since Flink added these two syntaxs.
But I can a
Hi Jingsong,
I will also add my opinion here for future discussions:
We had long discussions around SQL syntax in the past (e.g. for
WATERMARK or the concept of SYSTEM/TEMPORARY catalog objects) but in the
end all parties were happy and we came up with a good long-term solution
that is unlike
Hi Jingsong,
Thanks for the explanation, I think I misunderstood your point at the
begining.
As FLIP-63 proposed, INSERT OVERWRITE and INSERT PARTITION syntax are added
to Flink's
SQL syntax, but CREATE PARTITION TABLE should be limited under Hive
dialect.
However, the current implementation is op
Hi Jark,
Let's recall FLIP-63,
We supported these syntax in hive dialect at 1.9. All of my reasons for
launching FLIP-63 are to bring partition support to Flink itself.
Not only batch, but also we have the need to stream jobs to write partition
files today, which is also one of our very important
Hi jingsong,
Watermark is not a standard syntax, that's why we had a FLIP and long
discussion to add it to
Flink's SQL syntax. I think if we want to add INSERT OVERWRITE and
PARTITION syntax to
Flink's own syntax, we also need a FLIP or a VOTE, and this may can't
happen soon (we should
hear more
Thanks Jingsong for bring up this discussion ~
After reviewing FLIP-63, it seems that we have made a conclusion for the syntax
- INSERT OVERWRITE ...
- INSERT INTO … PARTITION
Which means that they should not have the Hive dialect limitation, so I’m
inclined that the behaviors for SQL-CLI is un
Hi Jark,
> The dialect restriction is introduced on purpose, because OVERWRITE and
PARTITION syntax are not SQL standard.
My understanding is that watermark [1] is also a non-standard grammar. We
can extend SQL standard syntax.
> Even in the discussion of FLIP-63, the community have different op
+1 to fix it in 1.10. If this feature doesn't work via SQL CLI, I guess it
doesn't work for most Hive users.
+0 to remove the dialect check. I don't see much benefit this check can
bring to users, except that it prevents users from accidentally using some
Hive features, which doesn't seem to be ver
Thanks Jingsong,
OVERWRITE and PARTITION are very fundamental features for Hive users.
I'm sorry to hear that it doesn't work in SQL CLI.
> Remove hive dialect limitation for these two grammars?
The dialect restriction is introduced on purpose, because OVERWRITE and
PARTITION syntax
are not SQL s
Hi Jingsong,
Thanks a lot for reporting this issue.
IIRC, we added [INSERT OVERWRITE] and [PARTITION] clauses to support Hive
integration before FLIP-63 was proposed to introduce generic partition
support to Flink. Thus when we added these syntax, we were intentionally
conservative and limited th
Hi Dev,
After cutting out the branch of 1.10, I tried the following functions of
SQL-CLI and found that it does not support:
- insert overwrite
- PARTITION (partcol1=val1, partcol2=val2 ...)
The SQL pattern is:
INSERT { INTO | OVERWRITE } TABLE tablename1 [PARTITION (partcol1=val1,
partcol2=val2 .
16 matches
Mail list logo
