and clean
up the deprecated code. We still need to update the version, so that we can
verify that we are compatible with the plan after the first change, but not
compatible with the plan earlier.
Best,
Wenlong
On Mon, 6 Dec 2021 at 21:27, Timo Walther wrote:
Hi Godfrey,
> design
Timo Walther created FLINK-25199:
Summary: fromValues does not emit final MAX watermark
Key: FLINK-25199
URL: https://issues.apache.org/jira/browse/FLINK-25199
Project: Flink
Issue Type: Bug
+1 (binding)
Thanks,
Timo
On 06.12.21 17:28, David Morávek wrote:
+1 (non-binding)
On Mon, Dec 6, 2021 at 4:55 PM Ingo Bürk wrote:
+1 (non-binding)
Ingo
On Mon, Dec 6, 2021 at 4:44 PM Chesnay Schepler
wrote:
Hello,
after recent discussions on the dev
patible with the state generated by the
given version node) ?
The names also indicate that only one version value can be set.
WDYT?
Best,
Godfrey
Timo Walther 于2021年12月2日周四 下午11:42写道：
Response to Marios's feedback:
> there should be some good logging in place when the upgrade is t
Hi Till,
thanks for starting this discussion. I think this topic should have been
discussed way earlier. I have two questions:
1) It might be an implementation detail but where do you expect this
`FlinkVersion` to be located? This is actually a quite important class
that also needs to be
ithout calling COMPILE should be sufficient. The file can also
manually be deleted if necessary.
What do you think?
Regards,
Timo
On 02.12.21 16:09, Timo Walther wrote:
Hi Till,
Yes, you might have to. But not a new plan from the SQL query but a
migration from the old plan to the ne
Flink 1.11 to 1.12, right?
Cheers,
Till
On Thu, Dec 2, 2021 at 3:39 PM Timo Walther wrote:
Response to Till's feedback:
> compiled plan won't be changed after being written initially
This is not entirely correct. We give guarantees for keeping the query
up and running. We reserve us the ri
SQL statement but from
the old plan. I have added an example in section 10.1.1. In general,
both persisted entities "plan" and "savepoint" can evolve independently
from each other.
Thanks,
Timo
On 02.12.21 15:10, Timo Walther wrote:
Response to Godfrey's feedback:
> "EXP
he code base?
Please check the section 10.1.1 again. I added a more complex example.
Thanks,
Timo
On 01.12.21 16:29, Timo Walther wrote:
Response to Francesco's feedback:
> *Proposed changes #6*: Other than defining this rule of thumb, we
must also make sure that compiling plans wit
plan file. This is already considered
in the COMPILE PLAN ... FROM ... even though this is future work. Also
savepoint migration.
Thanks for all the feedback!
Timo
On 30.11.21 14:28, Timo Walther wrote:
Response to Wenlongs's feedback:
> I would prefer not to provide such a shortcut, l
TABLE IF NOT EXISTS is not executed again. So a mismatch could also
occur there.
Regards,
Timo
On 30.11.21 14:17, Timo Walther wrote:
Hi everyone,
thanks for the feedback so far. Let me answer each email indvidually.
I will start with a response to Ingo's feedback:
> Will the JSON plan
Hi everyone,
thanks for the feedback so far. Let me answer each email indvidually.
I will start with a response to Ingo's feedback:
> Will the JSON plan's schema be considered an API?
No, not in the first version. This is explicitly mentioned in the
`General JSON Plan Assumptions`. I tried
song
On Sat, Nov 13, 2021 at 1:27 AM Timo Walther wrote:
Hi everyone,
even though the DISCUSS thread was open for 2 weeks. I have the feeling
that the VOTE was initiated to quickly. At least a final "I will start
the vote soon. Last call for comments." would have been nic
Hi everyone,
as many of you know, one of the biggest weaknesses of Flink's Table &
SQL API are the difficulties around stateful upgrades between Flink
minor versions (e.g. 1.13->1.14). Currently, we cannot provide any
backwards guarantees in those scenarios and need to force users to
Hi everyone,
thanks for finally have this discussion on the mailing list. As both a
contributor and user, I have experienced a couple issues around
nullability coming out of nowhere in a pipeline. This discussion should
not only cover CAST but failure handling in general.
Let me summarize
Hi everyone,
On behalf of the PMC, I'm very happy to announce Jing Zhang as a new
Flink committer.
Jing has been very active in the Flink community esp. in the Table/SQL
area for quite some time: 81 PRs [1] in total and is also active on
answering questions on the user mailing list. She is
Hi everyone,
even though the DISCUSS thread was open for 2 weeks. I have the feeling
that the VOTE was initiated to quickly. At least a final "I will start
the vote soon. Last call for comments." would have been nice.
I also added some comments in the DISCUSS thread. Let's hope we can
Hi everyone,
sorry for the delay in joining this thread. I went through the FLIP and
have some comments (maybe overlapping with Stephan's comments, which I
haven't read yet):
a. > More importantly, in order to solve the cognitive bar...
It would be great if we can add not only `Receive any
Timo Walther created FLINK-24877:
Summary: Implicitly add time attributes for VALUES clause
Key: FLINK-24877
URL: https://issues.apache.org/jira/browse/FLINK-24877
Project: Flink
Issue Type
+1 (binding) thanks for working on this.
Regards,
Timo
On 05.11.21 10:14, Sergey Nuyanzin wrote:
Also there is a short demo showing some of the features mentioned in this
FLIP.
It is available at https://asciinema.org/a/446247?speed=3.0 (It was also
mentioned in [DISCUSS] thread)
On Wed, Nov
Timo Walther created FLINK-24803:
Summary: Fix cast BINARY/VARBINARY to STRING
Key: FLINK-24803
URL: https://issues.apache.org/jira/browse/FLINK-24803
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-24802:
Summary: Improve cast ROW to STRING
Key: FLINK-24802
URL: https://issues.apache.org/jira/browse/FLINK-24802
Project: Flink
Issue Type: Sub-task
orward:
if a word not inside quoted string, not inside a comment or a hint
and matches anything from
SQL92 (
*org.apache.calcite.sql.parser.SqlAbstractParserImpl#getSql92ReservedWords*)),
then it will be highlighted as a keyword.
On Tue, Nov 2, 2021 at 12:09 PM Timo Walther wrote:
Hi Sergey,
Hi Sergey,
thanks for this nice demo video. It looks very nice and makes the SQL
Client an even more useful tool.
What I miss a bit in the FLIP is the implementation details.
For example, who is responsible for parsing comments? I guess the SQL
Client and not the Flink SQL parser will take
Timo Walther created FLINK-24717:
Summary: Push partitions before filters
Key: FLINK-24717
URL: https://issues.apache.org/jira/browse/FLINK-24717
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-24716:
Summary: Non-equality predicates on partition columns lead to
incorrect plans
Key: FLINK-24716
URL: https://issues.apache.org/jira/browse/FLINK-24716
Project: Flink
Hi,
this is an interesting idea. But as far as I can see, by looking at
other SQL engines like Microsoft SQL Server:
https://docs.microsoft.com/en-us/sql/t-sql/queries/select-over-clause-transact-sql?view=sql-server-ver15
The range is a well-defined set of keywords and CURRENT_TIMESTAMP is
Timo Walther created FLINK-24714:
Summary: Validate partition columns for ResolvedCatalogTable
Key: FLINK-24714
URL: https://issues.apache.org/jira/browse/FLINK-24714
Project: Flink
Issue
Timo Walther created FLINK-24700:
Summary: Clarify semantics of filter, projection, partition, and
metadata pushdown
Key: FLINK-24700
URL: https://issues.apache.org/jira/browse/FLINK-24700
Project
Hi Thomas,
thanks for your feedback.
The error that you are experiencing is definitely a bug in 1.13.3 and
the missing method should be reintroduced in the next patch version to
make code compiled against older patch versions run again.
Regarding the discussion points:
I agree that
Hi Francesco,
thanks for starting this discussion. It is definitely time to clean up
more connectors and formats that were used for the old planner but are
actually not intended for the DataStream API.
+1 for deprecating and dropping the mentioned formats. Users can either
use Table API or
Timo Walther created FLINK-24599:
Summary: Make checking for type root and family less verbose
Key: FLINK-24599
URL: https://issues.apache.org/jira/browse/FLINK-24599
Project: Flink
Issue
Timo Walther created FLINK-24532:
Summary: Invalid deletions for time-versioned join with
ChangelogNormalize
Key: FLINK-24532
URL: https://issues.apache.org/jira/browse/FLINK-24532
Project: Flink
, Martijn Visser wrote:
Hi Timo,
Sounds good to me.
However, I still need a PMC to make to the release. I definitely
volunteer
to help out with keeping track of things, communication etc. Hopefully
there's a PMC who can help.
Best regards,
Martijn
Op ma 27 sep. 2021 om 11:32 schreef Timo Walther
Timo Walther created FLINK-24427:
Summary: Hide Scala in flink-table-planner from API
Key: FLINK-24427
URL: https://issues.apache.org/jira/browse/FLINK-24427
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-24399:
Summary: Make handling of DataType less verbose
Key: FLINK-24399
URL: https://issues.apache.org/jira/browse/FLINK-24399
Project: Flink
Issue Type
for which
we can evolve quicker.
That sounds great ! I'm glad to see that we are making the API more
friendly !
[1]. https://github.com/danny0405
On Tue, Sep 28, 2021 at 3:52 PM Timo Walther wrote:
Hi Zheng,
I'm very sorry for the inconvenience that we have caused with our API
changes.
Timo Walther created FLINK-24396:
Summary: Add @Public annotations to Table & SQL API
Key: FLINK-24396
URL: https://issues.apache.org/jira/browse/FLINK-24396
Project: Flink
Issue Type:
Hi Zheng,
I'm very sorry for the inconvenience that we have caused with our API
changes. We are trying our best to avoid API breaking changes. Thanks
for giving us feedback.
There has been a reason why Table API was marked as @PublicEvolving
instead of @Public. Over the last two years, we
Hi Martijn,
we are currently in the process of backporting a couple of PRs to the
1.13 branch to fix the partially broken primary key support in Flink
SQL. See FLINK-20374 for more information.
I hope we can finalize this in one or two days. It should be completed
in 1.13.3 to avoid
Timo Walther created FLINK-24254:
Summary: Support hints configuration hints
Key: FLINK-24254
URL: https://issues.apache.org/jira/browse/FLINK-24254
Project: Flink
Issue Type: Bug
Timo Walther created FLINK-24186:
Summary: Disable single rowtime column check for collect/print
Key: FLINK-24186
URL: https://issues.apache.org/jira/browse/FLINK-24186
Project: Flink
Issue
Hi Ingo,
thanks for starting this discussion. Having more automation is
definitely desirable. Esp. in the API / SDK areas where we frequently
have to add similar comments to PRs. The more checks the better. We
definitely also need more guidelines (e.g. how to develop a Flink
connector) but
Timo Walther created FLINK-24116:
Summary: PRIMARY KEY declaration on insert-only table is not
validated globally
Key: FLINK-24116
URL: https://issues.apache.org/jira/browse/FLINK-24116
Project
Timo Walther created FLINK-24103:
Summary: Create time-based LAST_VALUE / FIRST_VALUE
Key: FLINK-24103
URL: https://issues.apache.org/jira/browse/FLINK-24103
Project: Flink
Issue Type
Timo Walther created FLINK-24087:
Summary: Kafka connector StartupMode requires a table dependency
Key: FLINK-24087
URL: https://issues.apache.org/jira/browse/FLINK-24087
Project: Flink
Timo Walther created FLINK-24054:
Summary: Let SinkUpsertMaterializer emit +U instead of only +I
Key: FLINK-24054
URL: https://issues.apache.org/jira/browse/FLINK-24054
Project: Flink
Issue
Hi Xingcan,
in theory there should be no hard blocker for supporting this. The
implementation should be flexible enough at most locations. We just
adopted 38 from the Blink code base which adopted it from Hive.
However, this could be a breaking change for existing pipelines and we
would
Timo Walther created FLINK-24033:
Summary: Propagate unique keys for fromChangelogStream
Key: FLINK-24033
URL: https://issues.apache.org/jira/browse/FLINK-24033
Project: Flink
Issue Type
Timo Walther created FLINK-23920:
Summary: SchemaTranslator looses primary key if schema is inferred
Key: FLINK-23920
URL: https://issues.apache.org/jira/browse/FLINK-23920
Project: Flink
Timo Walther created FLINK-23915:
Summary: Propagate unique keys for temporary tables
Key: FLINK-23915
URL: https://issues.apache.org/jira/browse/FLINK-23915
Project: Flink
Issue Type: Bug
Hi everyone,
this sounds definitely like a bug to me. Computing metadata might be
very expensive and a connector might expose a long list of metadata
keys. It was therefore intended to project the metadata if possible. I'm
pretty sure that this worked before (at least when implementing
Timo Walther created FLINK-23895:
Summary: Upsert materializer is not inserted for all sink providers
Key: FLINK-23895
URL: https://issues.apache.org/jira/browse/FLINK-23895
Project: Flink
Timo Walther created FLINK-23834:
Summary: Test StreamTableEnvironment batch mode manually
Key: FLINK-23834
URL: https://issues.apache.org/jira/browse/FLINK-23834
Project: Flink
Issue Type
Timo Walther created FLINK-23832:
Summary: Add documentation for batch mode in StreamTableEnvironment
Key: FLINK-23832
URL: https://issues.apache.org/jira/browse/FLINK-23832
Project: Flink
Timo Walther created FLINK-23768:
Summary: Test StreamTableEnvironment batch mode in Python
Key: FLINK-23768
URL: https://issues.apache.org/jira/browse/FLINK-23768
Project: Flink
Issue Type
Timo Walther created FLINK-23707:
Summary: Use consistent managed memory weights for StreamNode
Key: FLINK-23707
URL: https://issues.apache.org/jira/browse/FLINK-23707
Project: Flink
Issue
Hi everyone,
I'm not deeply involved in the discussion but I quickly checked out the
proposed interfaces because it seems they are using Table API heavily
and would like to leave some feedback here:
I have the feeling that the proposed interfaces are a bit too simplified.
Methods like
Sorry, I meant "will be deprecated in Flink 1.14"
On 09.08.21 19:32, Timo Walther wrote:
Hi Dominik,
`toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated
in Flink 1.13. It uses the old type system and might not match perfectly
with the other reworked type sys
Hi Dominik,
`toAppendStream` is soft deprecated in Flink 1.13 and will be deprecated
in Flink 1.13. It uses the old type system and might not match perfectly
with the other reworked type system in new functions and sources.
For SQL, a lot of Avro classes need to be treated as RAW types. But
Timo Walther created FLINK-23663:
Summary: Reduce state size in ChangelogNormalize through filter
push down
Key: FLINK-23663
URL: https://issues.apache.org/jira/browse/FLINK-23663
Project: Flink
Timo Walther created FLINK-23646:
Summary: Use pipeline name consistently across DataStream API and
Table API
Key: FLINK-23646
URL: https://issues.apache.org/jira/browse/FLINK-23646
Project: Flink
Timo Walther created FLINK-23498:
Summary: Use a layered configuration in Executor and Planner
Key: FLINK-23498
URL: https://issues.apache.org/jira/browse/FLINK-23498
Project: Flink
Issue
Timo Walther created FLINK-23482:
Summary: Simplify BlinkExecutorFactory stack
Key: FLINK-23482
URL: https://issues.apache.org/jira/browse/FLINK-23482
Project: Flink
Issue Type: Sub-task
Timo Walther created FLINK-23470:
Summary: Use blocking shuffles but pipeline within a slot
Key: FLINK-23470
URL: https://issues.apache.org/jira/browse/FLINK-23470
Project: Flink
Issue Type
Timo Walther created FLINK-23450:
Summary: Properties map is not set in DebeziumAvroFormatFactory
Key: FLINK-23450
URL: https://issues.apache.org/jira/browse/FLINK-23450
Project: Flink
Issue
Timo Walther created FLINK-23426:
Summary: Support changelog processing in batch mode
Key: FLINK-23426
URL: https://issues.apache.org/jira/browse/FLINK-23426
Project: Flink
Issue Type: Sub
Hi Srini,
welcome aboard! Great to see more adoption in the SQL space. Looking
forward to collaboration.
Regards,
Timo
On 19.07.21 10:58, Till Rohrmann wrote:
Hi Srini,
Welcome to the Flink community :-) Great to hear what you are planning to
do with Flink at LinkedIn. I think sharing this
+1 (binding)
I went through all commits one more time and could not spot anything
that would block a release.
Thanks Chesnay!
Timo
On 15.07.21 09:02, Chesnay Schepler wrote:
Hi everyone,
Please review and vote on the release candidate #1 for the version 14.0,
as follows:
[ ] +1, Approve
Timo Walther created FLINK-23424:
Summary: Disable key sorting for Transformations created within
table planner
Key: FLINK-23424
URL: https://issues.apache.org/jira/browse/FLINK-23424
Project: Flink
Timo Walther created FLINK-23402:
Summary: Expose a consistent GlobalDataExchangeMode
Key: FLINK-23402
URL: https://issues.apache.org/jira/browse/FLINK-23402
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-23384:
Summary: Use upper case name for all BuiltInFunctionDefinitions
Key: FLINK-23384
URL: https://issues.apache.org/jira/browse/FLINK-23384
Project: Flink
Issue
Timo Walther created FLINK-23373:
Summary: Support object reuse disabled in OperatorChain
Key: FLINK-23373
URL: https://issues.apache.org/jira/browse/FLINK-23373
Project: Flink
Issue Type
Timo Walther created FLINK-23372:
Summary: Disable AllVerticesInSameSlotSharingGroupByDefault in
DataStream batch mode
Key: FLINK-23372
URL: https://issues.apache.org/jira/browse/FLINK-23372
Project
Timo Walther created FLINK-23371:
Summary: Disable AutoWatermarkInterval for bounded legacy sources
Key: FLINK-23371
URL: https://issues.apache.org/jira/browse/FLINK-23371
Project: Flink
Timo Walther created FLINK-23370:
Summary: Propagate Boundedness of SourceFunctionProvider to
Transformation
Key: FLINK-23370
URL: https://issues.apache.org/jira/browse/FLINK-23370
Project: Flink
Timo Walther created FLINK-23330:
Summary: Deprecate toAppendStream and toRetractStream
Key: FLINK-23330
URL: https://issues.apache.org/jira/browse/FLINK-23330
Project: Flink
Issue Type: Sub
Timo Walther created FLINK-23320:
Summary: Support semi/anti temporal joins
Key: FLINK-23320
URL: https://issues.apache.org/jira/browse/FLINK-23320
Project: Flink
Issue Type: New Feature
Timo Walther created FLINK-23319:
Summary: Support semi/anti lookup joins
Key: FLINK-23319
URL: https://issues.apache.org/jira/browse/FLINK-23319
Project: Flink
Issue Type: New Feature
et would not be touched by the
bot
as
long
as
there
is
a
single Sub-Task that has a discussion or an
update.
If
you
experience
something different, this is a bug.
Is there a reason why it is important to
assign
all
Sub-Tasks
to
the
same
person immediately? I am not sure i
Timo Walther created FLINK-23313:
Summary: Reintroduce temporal table function documentation
Key: FLINK-23313
URL: https://issues.apache.org/jira/browse/FLINK-23313
Project: Flink
Issue Type
Timo Walther created FLINK-23306:
Summary: FlinkRelMdUniqueKeys causes exception when used with new
Schema
Key: FLINK-23306
URL: https://issues.apache.org/jira/browse/FLINK-23306
Project: Flink
Timo Walther created FLINK-23305:
Summary: Support semi/anti interval joins
Key: FLINK-23305
URL: https://issues.apache.org/jira/browse/FLINK-23305
Project: Flink
Issue Type: New Feature
iting this up, this also reflects my understanding.
I think a blog post would be nice, ideally with an explicit call for
feedback so we learn about user concerns.
A blog post has a lot more reach than an ML thread.
Best,
Stephan
On Wed, Jun 23, 2021 at 12:23 PM Timo Walther
wrote:
Hi every
Hi everyone,
as discussed previously [1] and tracked in FLINK-14437, we executed the
last step of FLIP-32 and removed all occurences of the term "blink" in
the code base.
This includes renaming the following Maven modules:
flink-table-planner-blink -> flink-table-planner
Hi Jack,
thanks for sharing your proposal with us. I totally understand the
issues that you are trying to solve. Having a more flexible type support
in the connectors is definitely a problem that we would like to address
in the mid term. It is already considered in on our internal roadmap
Hi everyone,
I'm sending this email to make sure everyone is on the same page about
slowly deprecating the DataSet API.
There have been a few thoughts mentioned in presentations, offline
discussions, and JIRA issues. However, I have observed that there are
still some concerns or different
Timo Walther created FLINK-23116:
Summary: Update documentation about TableDescriptors
Key: FLINK-23116
URL: https://issues.apache.org/jira/browse/FLINK-23116
Project: Flink
Issue Type: Sub
+1 (binding)
Thanks for driving this.
Regards,
Timo
On 21.06.21 13:24, Ingo Bürk wrote:
Hi everyone,
thanks for all the feedback so far. Based on the discussion[1] we seem to
have consensus, so I would like to start a vote on FLIP-129 for which the
FLIP has now also been updated[2].
The
Timo Walther created FLINK-23072:
Summary: Add benchmarks for SQL internal and external serializers
Key: FLINK-23072
URL: https://issues.apache.org/jira/browse/FLINK-23072
Project: Flink
Timo Walther created FLINK-23040:
Summary: Consider ConfigOption fallback keys in FactoryUtil
Key: FLINK-23040
URL: https://issues.apache.org/jira/browse/FLINK-23040
Project: Flink
Issue
Timo Walther created FLINK-23019:
Summary: Avoid errors when identifiers use reserved keywords
Key: FLINK-23019
URL: https://issues.apache.org/jira/browse/FLINK-23019
Project: Flink
Issue
Hi Ingo,
thanks for giving FLIP-129 an update before finally implementing it. I
wouldn't start with a voting thread right away but collect more feedback
from the community in a [DISCUSS] thread before.
Also, voting threads should be performed on the updated wiki page and
include the voting
Timo Walther created FLINK-22916:
Summary: Revisit and close JIRA issues around legacy planner
Key: FLINK-22916
URL: https://issues.apache.org/jira/browse/FLINK-22916
Project: Flink
Issue
Timo Walther created FLINK-22880:
Summary: Remove "blink" term in code base
Key: FLINK-22880
URL: https://issues.apache.org/jira/browse/FLINK-22880
Project: Flink
Issue Type
Timo Walther created FLINK-22879:
Summary: Remove "blink" suffix from table modules
Key: FLINK-22879
URL: https://issues.apache.org/jira/browse/FLINK-22879
Project: Flink
Issue
Timo Walther created FLINK-22878:
Summary: Allow placeholder options in format factories
Key: FLINK-22878
URL: https://issues.apache.org/jira/browse/FLINK-22878
Project: Flink
Issue Type
Timo Walther created FLINK-22877:
Summary: Remove BatchTableEnvironment and related API classes
Key: FLINK-22877
URL: https://issues.apache.org/jira/browse/FLINK-22877
Project: Flink
Issue
Timo Walther created FLINK-22872:
Summary: Remove usages of legacy planner test utilities in Python
Key: FLINK-22872
URL: https://issues.apache.org/jira/browse/FLINK-22872
Project: Flink
Timo Walther created FLINK-22864:
Summary: Remove the legacy planner code base
Key: FLINK-22864
URL: https://issues.apache.org/jira/browse/FLINK-22864
Project: Flink
Issue Type: Sub-task
201 - 300 of 1352 matches
Mail list logo
