Thanks for the reply, Andrey.
Regarding building from local dist:
- Yes, I bring this up mostly for development purpose. Since k8s is
popular, I believe more and more developers would like to test their
work on k8s cluster. I'm not sure should all developers write a custom
docker file themselves
Yu,
Thanks for the explanation.
I've no concerns. I was just trying to get some inputs for prioritizing
tasks on my side, and ~1month sounds good to me.
Thank you~
Xintong Song
On Wed, Mar 11, 2020 at 12:15 PM Yu Li wrote:
> bq. what is the time plan for 1.10.1?
>
> According to the
Zhijiang created FLINK-16537:
Summary: Implement ResultPartition state recovery for unaligned
checkpoint
Key: FLINK-16537
URL: https://issues.apache.org/jira/browse/FLINK-16537
Project: Flink
Zhijiang created FLINK-16536:
Summary: Implement InputChannel state recovery for unaligned
checkpoint
Key: FLINK-16536
URL: https://issues.apache.org/jira/browse/FLINK-16536
Project: Flink
godfrey he created FLINK-16535:
--
Summary: BatchTableSink#emitDataSet returns DataSink
Key: FLINK-16535
URL: https://issues.apache.org/jira/browse/FLINK-16535
Project: Flink
Issue Type: Sub-task
Thanks for compiling the list of 1.11 efforts Zhijiang and Piotr! This
helps a lot to better understand what the community is currently working
on. Looking forward to another successful release.
Best Regards,
Yu
On Wed, 11 Mar 2020 at 11:17, Zhijiang
wrote:
> Hi community,
>
>
> Not more than
Kurt Young created FLINK-16534:
--
Summary: Support specify savepoint path when submitting sql job
through sql client
Key: FLINK-16534
URL: https://issues.apache.org/jira/browse/FLINK-16534
Project: Flink
godfrey he created FLINK-16533:
--
Summary: ExecutionEnvironment supports executing plan
Key: FLINK-16533
URL: https://issues.apache.org/jira/browse/FLINK-16533
Project: Flink
Issue Type:
Thanks Danny for kicking off the effort
The root cause of too much manual work is Flink DDL has mixed 3 types of
params together and doesn't handle each of them very well. Below are how I
categorize them and corresponding solutions in my mind:
- type 1: Metadata of external data, like external
bq. what is the time plan for 1.10.1?
According to the history, the first patch release of a major version will
take ~1month from discussion started, depending on the speed of blocker
issue resolving:
* 1.8.1: started discussion on May 28th [1], released on Jul 3rd [2]
* 1.9.1: started discussion
+1 on the overall design and thanks for the efforts!
I totally agree with the plan of implementing the MVP first. However, since
the FLIP is for the whole feature instead of only MVP, how about adding a
*Roadmap* or *Future Work* section to write down plans include (but not
limited to):
* Dynamic
Thanks Yu, for the kick off and volunteering to be the release manager.
+1 for the proposal.
One quick question, what is the time plan for 1.10.1?
Thank you~
Xintong Song
On Wed, Mar 11, 2020 at 11:51 AM Zhijiang
wrote:
> Thanks for driving this release, Yu!
> +1 on my side
>
>
>
>
>
>
Thanks for driving this release, Yu!
+1 on my side
Best,
Zhijiang
--
From:Yu Li
Send Time:2020 Mar. 10 (Tue.) 20:25
To:dev
Subject:Re: [DISCUSS] Releasing Flink 1.10.1
Thanks for the supplement Hequn. Yes will also keep
Rui Li created FLINK-16532:
--
Summary: Cannot insert white space characters as partition value
for Hive table
Key: FLINK-16532
URL: https://issues.apache.org/jira/browse/FLINK-16532
Project: Flink
Jark Wu created FLINK-16531:
---
Summary: Add full integration tests for "GROUPING SETS" for
streaming mode
Key: FLINK-16531
URL: https://issues.apache.org/jira/browse/FLINK-16531
Project: Flink
Jark Wu created FLINK-16530:
---
Summary: Add documentation about "GROUPING SETS" and "CUBE"
support in streaming mode
Key: FLINK-16530
URL: https://issues.apache.org/jira/browse/FLINK-16530
Project: Flink
Hi community,
Not more than one month ago we have released Flink 1.10. We are now heading for
the Flink 1.11 release and we, as release managers, would like to share with
you what are the features that the community is currently working on and we are
hoping that will be part of the Flink 1.11
If a specific connector want to have such parameter and read if out of
configuration, then that's fine.
If we are talking about a configuration for all kinds of sources, I would
be super careful about that.
It's true it can solve maybe 80% cases, but it will also make the left 20%
feels weird.
Jark Wu created FLINK-16529:
---
Summary: Add ignore_parse_errors() method to Json format in python
API
Key: FLINK-16529
URL: https://issues.apache.org/jira/browse/FLINK-16529
Project: Flink
Issue
Hi Kurt,
#3 Regarding to global offset:
I'm not saying to use the global configuration to override connector
properties by the planner.
But the connector should take this configuration and translate into their
client API.
AFAIK, almost all the message queues support eariliest and latest and a
Good to have such lovely discussions. I also want to share some of my
opinions.
#1 Regarding to error handling: I also think ignore invalid hints would be
dangerous, maybe
the simplest solution is just throw an exception.
#2 Regarding to property replacement: I don't think we should constraint
Thanks Timo and Fabian ~
compared to the hints, FLIP-110 is fully compliant to the SQL standard.
I don't think so, here is the syntax of CREATE TABLE in SQL-2011 IWD
9075-2:201?(E) 11.3 :
::=
CREATE [ ] TABLE
[ WITH ]
[ ON COMMIT ROWS ]
::=
|
|
::=
[ {
Jark Wu created FLINK-16528:
---
Summary: Support Limit push down for streaming sources
Key: FLINK-16528
URL: https://issues.apache.org/jira/browse/FLINK-16528
Project: Flink
Issue Type: New Feature
Thanks for the survey, Aljoscha. I'll also close FLINK-15925.
I second Robert's concern. It sounds a bit weird that we keep something in
our project that does not work and is not maintained. Might also cause
confusions for future new comers who may not be aware of our conclusion
today. Maybe we
Thanks for your update Klou!
Best,
tison.
Kostas Kloudas 于2020年3月11日周三 上午2:05写道：
> Hi all,
>
> The FLIP was updated under the section "First Version Deliverables".
>
> Cheers,
> Kostas
>
> On Tue, Mar 10, 2020 at 4:10 PM Kostas Kloudas wrote:
> >
> > Hi all,
> >
> > Yes I will do that. From
Noah created FLINK-16527:
Summary: hadoop.version of flink-shaded-hadoop-2-uber should be
checked
Key: FLINK-16527
URL: https://issues.apache.org/jira/browse/FLINK-16527
Project: Flink
Issue Type:
+1
Thanks for putting this together, looking forward to the experimental
support in the next release.
One clarification: since the MVP won't support rescaling, does it imply
that savepoints will always use aligned checkpointing? If so, this would
still block the user from taking a savepoint and
Thanks for working on improvements to the Flink Docker container images.
This will be important as more and more users are looking to adopt
Kubernetes and other deployment tooling that relies on Docker images.
A generic, dynamic configuration mechanism based on environment variables
is essential
Hi All,
Thanks a lot for the feedback!
*@Yangze Guo*
- Regarding the flink_docker_utils#install_flink function, I think it
> should also support build from local dist and build from a
> user-defined archive.
I suppose you bring this up mostly for development purpose or powerful
users.
Most of
Hi!
We are testing the following 3 way time windowed join to keep the retained
state size small. Using joins for the first time here. It works in unit
tests but we are not able to get expected results in production. We are
still troubleshooting this issue. Can you please help us review this in
Hi all,
The FLIP was updated under the section "First Version Deliverables".
Cheers,
Kostas
On Tue, Mar 10, 2020 at 4:10 PM Kostas Kloudas wrote:
>
> Hi all,
>
> Yes I will do that. From the discussion, I will add that:
> 1) for the cli, we are planning to add a "run-application" command
> 2)
YufeiLiu created FLINK-16526:
Summary: Escape character doesn't work for computed column
Key: FLINK-16526
URL: https://issues.apache.org/jira/browse/FLINK-16526
Project: Flink
Issue Type: Bug
Hi everyone,
I want to jump in the discussion about the "dynamic start offset" problem.
First of all, I share the same concern with Timo and Fabian, that the
"start offset" affects the query semantics, i.e. the query result.
But "hints" is just used for optimization which should affect the
Fabian Paul created FLINK-16525:
---
Summary: TwoPhaseCommitSinkFunction subtask logs misleading name
Key: FLINK-16525
URL: https://issues.apache.org/jira/browse/FLINK-16525
Project: Flink
Issue
+1 on the suggestion to add "What is Stateful Functions" to the left
navigation bar.
That might also mean it would be nice to have a slight rework to the main
image on the website, illustrating the use cases of Flink (this one [1]).
On the image it does mention "Event-Driven Applications", but
Hi all,
Yes I will do that. From the discussion, I will add that:
1) for the cli, we are planning to add a "run-application" command
2) for deployment in Yarn we are planning to use LocalResources to let
Yarn do the jar transfer
3) for Standalone/containers, we assume that dependencies/jars are
Huang Xingbo created FLINK-16524:
Summary: Optimize the result of FlattenRowCoder to generator to
eliminate unnecessary function calls
Key: FLINK-16524
URL: https://issues.apache.org/jira/browse/FLINK-16524
Hi Danny,
compared to the hints, FLIP-110 is fully compliant to the SQL standard.
I don't think that `CREATE TEMPORARY TABLE Temp (LIKE t) WITH (k=v)` is
too verbose or awkward for the power of basically changing the entire
connector. Usually, this statement would just precede the query in a
Hi,
RE error handling:
I don't think it is a good idea to simply log invalid hints. This makes it
very hard for systems that integrate Flink to consume these errors.
I'm not saying we should throw an exception. We could also execute a query
with invalid hints but should have a mechanism to
Thanks for your response.
@Kostas Kloudas Could we update the cli changes and
how to fetch the
user jars to FLIP document? I think other dev or users may have the similar
questions.
Best,
Yang
Aljoscha Krettek 于2020年3月10日周二 下午9:03写道：
> On 10.03.20 03:31, Yang Wang wrote:
> > For the
On 10.03.20 14:35, Robert Metzger wrote:
I'm wondering whether we should file a ticket to remove the *.bat files in
bin/ ?
We can leave them there because they're not doing much harm, and
removing them might actively break some existing setup.
Best,
Aljoscha
I'm wondering whether we should file a ticket to remove the *.bat files in
bin/ ?
On Tue, Mar 10, 2020 at 12:46 PM Aljoscha Krettek
wrote:
> Since there was no-one that said we should keep the windows scripts and
> no-one responded on the user ML thread I'll close the Jira issues/PRs
> about
Hi all,
I would like to start the vote for FLIP-76 [1], which is discussed and
reached a consensus in the discussion thread [2].
The vote will be open until March. 13th (72h), unless there is an objection
or not enough votes.
Thanks,
Arvid
[1]
On 10.03.20 03:31, Yang Wang wrote:
For the "run-job", do you mean to submit a Flink job to an existing session
or
just like the current per-job to start a dedicated Flink cluster? Then will
"flink run" be deprecated?
I was talking about the per-job mode that starts a dedicated Flink
cluster.
+1 on adding a "What is Stateful Functions" link below the "What is Apache
Flink" entry and integrating into the Flink docs gradually (instead of
hiding it behind until fully integrated).
Best Regards,
Yu
On Tue, 10 Mar 2020 at 19:33, Stephan Ewen wrote:
> Hi all!
>
> I think it would be nice
Thanks for the supplement Hequn. Yes will also keep an eye on these
existing blocker issues.
Best Regards,
Yu
On Tue, 10 Mar 2020 at 19:10, Hequn Cheng wrote:
> Hi Yu,
>
> Thanks a lot for raising the discussion and volunteer as the release
> manager!
>
> I found there are some other
Since there was no-one that said we should keep the windows scripts and
no-one responded on the user ML thread I'll close the Jira issues/PRs
about extending the scripts.
Aljoscha
Hi all!
I think it would be nice to mention Stateful Function on the Flink website.
At the moment, Stateful Functions is very hard to discover, and with the
first release of it under Apache Flink, it would be a good time to change
that.
My proposal would be to add a "What is Stateful Functions?"
Hi Yu,
Thanks a lot for raising the discussion and volunteer as the release
manager!
I found there are some other issues[1] which are marked as a blocker:
- FLINK-16454 Update the copyright year in NOTICE files
- FLINK-16262 Class loader problem with
FlinkKafkaProducer.Semantic.EXACTLY_ONCE and
Hi Jincheng,
Yes, your help would be very helpful. Thanks a lot!
Best Regards,
Yu
On Tue, 10 Mar 2020 at 18:24, jincheng sun wrote:
> Thanks for bring up the discussion Yu. I would like to give you a hand at
> the last stage when the RC is finished.(If you need) :)
>
> Best,
> Jincheng
>
>
Thanks for bring up the discussion Yu. I would like to give you a hand at
the last stage when the RC is finished.(If you need) :)
Best,
Jincheng
Yu Li 于2020年3月10日周二 下午5:49写道：
> Hi All,
>
> It has been almost one month since we released Flink 1.10.0. We already
> have more than 40 resolved
Hi All,
It has been almost one month since we released Flink 1.10.0. We already
have more than 40 resolved improvements/bugs in the release-1.10 branch,
and I propose to start the 1.10.1 release cycle.
Most noticeable fixes are:
- FLINK-16241 [legal] Remove the license and notice file in
Thanks Timo ~
Personally I would say that offset > 0 and start offset = 10 does not have the
same semantic, so from the SQL aspect, we can not implement a “starting offset”
hint for query with such a syntax.
And the CREATE TABLE LIKE syntax is a DDL which is just verbose for defining
such
luojiangyu created FLINK-16523:
--
Summary: The improvement for local state of rocksdb backend.
Key: FLINK-16523
URL: https://issues.apache.org/jira/browse/FLINK-16523
Project: Flink
Issue Type:
Huang Xingbo created FLINK-16522:
Summary: Take use of Python3 type hint to rebuild Python Function
signature
Key: FLINK-16522
URL: https://issues.apache.org/jira/browse/FLINK-16522
Project: Flink
Hi Danny,
shouldn't FLIP-110[1] solve most of the problems we have around defining
table properties more dynamically without manual schema work? Also
offset definition is easier with such a syntax. They must not be defined
in catalog but could be temporary tables that extend from the original
Canbin Zheng created FLINK-16521:
Summary: Remove unused FileUtils#isClassFile
Key: FLINK-16521
URL: https://issues.apache.org/jira/browse/FLINK-16521
Project: Flink
Issue Type: Improvement
Hi LakeShen,
you can change the port with
conf.setInteger(RestOptions.PORT, 8082);
or if want to be on the safe side specify a range
conf.setString(RestOptions.BIND_PORT, "8081-8099");
On Mon, Mar 9, 2020 at 10:47 AM LakeShen wrote:
> Hi community,
>now I am moving the flink job to
Tzu-Li (Gordon) Tai created FLINK-16520:
---
Summary: Move Kafka client properties resolution in
KafkaSinkProvider to KafkaEgressBuilder
Key: FLINK-16520
URL: https://issues.apache.org/jira/browse/FLINK-16520
Hi Timo,
Thanks for your reply.
If we aim for the option 1, it makes sense for me to include the change in this
FLIP as the option 1 does not change any public API. I'll update the FLIP page
to illustrate this.
Best,
Wei
> 在 2020年3月9日，17:58，Timo Walther 写道：
>
> Hi Wei,
>
> I agree with
Hi Hequn,
Thanks for driving this. +1 for the metrics support for Python UDF, which makes
it much easier for users to monitor the execution of Python UDFs.
Best,
Wei
> 在 2020年3月10日，15:32，Xingbo Huang 写道：
>
> Hi Hequn,
> thanks for drafting the FLIP and kicking off the discussion.
>
> +1
Thanks Wenlong ~
For PROPERTIES Hint Error handling
Actually we have no way to figure out whether a error prone hint is a
PROPERTIES hint, for example, if use writes a hint like ‘PROPERTIAS’, we do not
know if this hint is a PROPERTIES hint, what we know is that the hint name was
not
Hi Hequn,
thanks for drafting the FLIP and kicking off the discussion.
+1 for this feature.
I think this feature will be extremely convenient for PyFlink users.
Best,
Xingbo
Hequn Cheng 于2020年3月9日周一 上午11:32写道：
> Hi everyone,
>
> FLIP-58 adds the support for Python UDFs, but user-defined
Hi Danny, thanks for the proposal. +1 for adding table hints, it is really
a necessary feature for flink sql to integrate with a catalog.
For error handling, I think it would be more natural to throw an
exception when error table hint provided, because the properties in hint
will be merged and
To Weike: About the Error Handing
To be consistent with other SQL vendors, the default is to log warnings and if
there is any error (invalid hint name or options), the hint is just ignored. I
have already addressed in the wiki.
To Timo: About the PROPERTIES Table Hint
• The properties hints
65 matches
Mail list logo
