Yun Gao created FLINK-16582:
---
Summary: NettyBufferPoolTest may have warns on NettyBuffer leak
Key: FLINK-16582
URL: https://issues.apache.org/jira/browse/FLINK-16582
Project: Flink
Issue Type:
Jingsong Lee created FLINK-16581:
Summary: Minibatch deduplication lack state TTL
Key: FLINK-16581
URL: https://issues.apache.org/jira/browse/FLINK-16581
Project: Flink
Issue Type: Bug
Would you mind to share more information about why the task executor
is killed? If it is killed by Yarn, you might get such info in Yarn
NM/RM logs.
Best,
Yangze Guo
Best,
Yangze Guo
On Fri, Mar 13, 2020 at 12:31 PM DONG, Weike wrote:
>
> Hi,
>
> Recently I have encountered a strange behavior
Thanks for the reminder Stephan and the inputs/discussion Andrey and
Xintong. I totally agree that we should try best not to introduce new
issues when resolving the existing one. Will watch closely about the
discussion and conclusion around metaspace configuration.
>From the point as a developer
Hi,
Recently I have encountered a strange behavior of Flink on YARN, which is
that when I try to cancel a Flink job running in per-job mode on YARN using
commands like
"cancel -m yarn-cluster
-yid application_1559388106022_9412 ed7e2e0ab0a7316c1b65df6047bc6aae"
the client happily found and
李开青 created FLINK-16580:
---
Summary: flink-connector-kafka desrializer
Key: FLINK-16580
URL: https://issues.apache.org/jira/browse/FLINK-16580
Project: Flink
Issue Type: Wish
Components:
I think you can modify the operator’s parallelism. It is only if you have
set maxParallelism, and while restoring from a checkpoint, you shouldn’t
modify the maxParallelism. Otherwise, I believe the state will be lost.
-
Sivaprasanna
On Fri, 13 Mar 2020 at 9:01 AM, LakeShen wrote:
> Hi
Thanks a lot!, tison
tison 于2020年3月12日周四 下午5:56写道：
> The StoppableFunction is gone.
>
> See also https://issues.apache.org/jira/browse/FLINK-11889
>
> Best,
> tison.
>
>
> LakeShen 于2020年3月12日周四 下午5:44写道：
>
>> Hi community,
>> now I am seeing the FLIP-45 , as I see the stop command
Hi community,
I have a question is that I cancel the flink task and retain the
checkpoint dir, then restore from the checkpoint dir ,can I change the
flink operator's parallelism,in my thoughts, I think I can't change the
flink operator's parallelism,but I am not sure.
Thanks to your
Jark Wu created FLINK-16579:
---
Summary: Upgrade Calcite version to 1.23 for Flink SQL
Key: FLINK-16579
URL: https://issues.apache.org/jira/browse/FLINK-16579
Project: Flink
Issue Type: Task
+1 (non-binding)
Checkpoint timeout in cases of backpressure is hard to tune. I and our
users ever spent lots of time on that. It is great to have this feature.
Arvid Heise 于2020年3月10日周二 下午9:33写道：
> Hi all,
>
> I would like to start the vote for FLIP-76 [1], which is discussed and
> reached a
Shuo Cheng created FLINK-16578:
--
Summary: join lateral table function with condition fails with
exception
Key: FLINK-16578
URL: https://issues.apache.org/jira/browse/FLINK-16578
Project: Flink
Shuo Cheng created FLINK-16577:
--
Summary: Exception will be thrown when computing columnInterval
relmetadata in some case
Key: FLINK-16577
URL: https://issues.apache.org/jira/browse/FLINK-16577
Project:
Hi Robert,
+1 to drop it but maybe not 1.11.
ORC has not been supported on StreamingFileSink. I have seen lots of users
run ORC in the bucketing sink.
Best,
Jingsong Lee
On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman wrote:
> Sorry, I meant FLIP-46.
>
> Seth
>
> On Thu, Mar 12, 2020 at 11:52
I think in FLINK-16406 we have not only increased the default value for
metaspace, but also increased the 'process.size' in default
'flink-conf.yaml'.
That means:
- For people using the default 'flink-conf.yaml', it's not really taking
memory from heap / managed for metaspace.
- For
Hi Hequn,
+1, thank you for this discussion, and metrics are very important for
monitoring the running state of Python UDF.
Best,
Jincheng
Hequn Cheng 于2020年3月12日周四 下午11:39写道：
> @Dian Fu Thanks a lot for the advice. The FLIP
> page has been updated.
>
> Best,
> Hequn
>
> On Thu, Mar 12,
Nico Kruber created FLINK-16576:
---
Summary: State inconsistency on restore with memory state backends
Key: FLINK-16576
URL: https://issues.apache.org/jira/browse/FLINK-16576
Project: Flink
Thanks Gyula,
Looking forward to your comments.
Just to let you know, I would not like having a method that in some
cases works as expected and in some other ones it does not. It would
be nice if we could expose consistent behaviour to the users.
On Thu, Mar 12, 2020 at 8:44 PM Gyula Fóra
I added a roadmap section to the FLIP as suggested by Yu and Roman.
Unless someone objects, I'd still consider the voting period to end
tomorrow. For me, the roadmap is only a clarification of already written
and discussed points.
We already have enough binding votes, but there may be concerns
Thanks Kostas, I have to review the possible limitations with the Executor
before I can properly answer.
Regarding you comments for the listener pattern, we proposed in the
document to include the getPipeline() in the JobClient itself as you
suggested to fit the pattern :) For not always being
Hi again,
Just to clarify, I am not against exposing the Pipeline if this will
lead to a "clean" solution.
And, I. forgot to say that the last solution, if adopted, would have
to work on the JobGraph, which may not be that desirable.
Kostas
On Thu, Mar 12, 2020 at 8:26 PM Kostas Kloudas wrote:
Hi all,
I do not have a strong opinion on the topic yet, but I would like to
share my thoughts on this.
In the solution proposing a wrapping AtlasExecutor around the Flink
Executors, if we allow the user to use the CLI to submit jobs, then
this means that the CLI code may have to change so that
Bowen Li created FLINK-16575:
Summary: develop HBaseCatalog to integrate HBase metadata into
Flink
Key: FLINK-16575
URL: https://issues.apache.org/jira/browse/FLINK-16575
Project: Flink
Issue
static-max created FLINK-16574:
--
Summary: StreamingFileSink should rename files or fail if
destination file already exists
Key: FLINK-16574
URL: https://issues.apache.org/jira/browse/FLINK-16574
Sorry, I meant FLIP-46.
Seth
On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman wrote:
> I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we
> want to drop the bucketing sink.
>
> Seth
>
> On Thu, Mar 12, 2020 at 10:53 AM David Anderson
> wrote:
>
>> The BucketingSink is
I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we
want to drop the bucketing sink.
Seth
On Thu, Mar 12, 2020 at 10:53 AM David Anderson wrote:
> The BucketingSink is still somewhat widely used, I think in part because of
> shortcomings in the StreamingFileSink.
>
> I
+1 (non-binding)
I think the PoC result has shown the effect on reducing checkpoint time
when back-pressure occurs, and I totally agree with that the feature could be
implemented in steps.
--
From:Roman Khachatryan
Send
The BucketingSink is still somewhat widely used, I think in part because of
shortcomings in the StreamingFileSink.
I would hope that in tandem with removing the bucketing sink we could also
address some of these issues. I'm thinking in particular of issues that are
waiting on FLIP-46 [1].
@Dian Fu Thanks a lot for the advice. The FLIP
page has been updated.
Best,
Hequn
On Thu, Mar 12, 2020 at 4:19 PM Dian Fu wrote:
> Hi Hequn,
>
> Thanks for driving this. +1 to this feature.
>
> Just one minor comment: It seems that we will add an API get_metric_group
> for the Python class
Thanks for driving this discussion, Robert!
This e2e test really fails frequently. +1 to drop bucketing sink, it is not
worth paying more efforts since deprecated.
Best,
Zhijiang
--
From:Jeff Zhang
Send Time:2020 Mar. 12
Good idea! +1 for dropping the BucketingSink.
Best,
Hequn
> On Mar 12, 2020, at 10:40 PM, Robert Metzger wrote:
>
> Hi all,
>
> I'm currently investigating a failing end to end test for the bucketing
> sink [1].
> The bucketing sink has been deprecated in the 1.9 release [2], because we
>
+1
Aljoscha
+1， dropping deprecated api is always necessary for a sustainable project.
Kostas Kloudas 于2020年3月12日周四 下午11:06写道：
> Hi Robert,
>
> +1 for dropping the BucketingSink.
> In any case, it has not been maintained for quite some time.
>
> Cheers,
> Kostas
>
> On Thu, Mar 12, 2020 at 3:41 PM Robert
Hi Robert,
+1 for dropping the BucketingSink.
In any case, it has not been maintained for quite some time.
Cheers,
Kostas
On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger wrote:
>
> Hi all,
>
> I'm currently investigating a failing end to end test for the bucketing
> sink [1].
> The bucketing
Hi all,
I'm currently investigating a failing end to end test for the bucketing
sink [1].
The bucketing sink has been deprecated in the 1.9 release [2], because we
have the new StreamingFileSink [3] for quite a while.
Before putting any effort into fixing the end to end test for the sink, I
On 12.03.20 14:05, Flavio Pompermaier wrote:
There's also a related issue that I opened a long time ago
https://issues.apache.org/jira/browse/FLINK-10879 that could be closed once
implemented this FLIP (or closed immediately and referenced as duplicated
by the new JIRA ticket that would be
Hi Gyula!
My main motivation was to try and avoid mixing an internal interface
(Pipeline) with public API. It looks like this is trying to go "public
stable", but doesn't really do it exactly because of mixing "pipeline" into
this.
You would need to cast "Pipeline" and work on internal classes in
No need to revert it now - I am not saying it should not go into 1.10.1, I
am just saying this is not clear to me yet.
We have to trade off the fact that we may break some deployments with the
fact that we will "safe" a lot of new deployments.
I simply lack the data points / insight at the moment
@Danny sounds good.
Maybe it is worth listing all the classes of problems that you want to
address and then look at each class and see if hints are a good default
solution or a good optional way of simplifying things?
The discussion has grown a lot and it is starting to be hard to distinguish
the
Hi Stephan!
Thanks for checking this out. I agree that wrapping the other
PipelineExecutors with a delegating AtlasExecutor would be a good
alternative approach to implement this but I actually feel that it suffers
even more problems than exposing the Pipeline instance in the JobListener.
The
Thanks a lot for moving this FLIP forward. Looking forward to the JIRAs.
Best,
Yang
Flavio Pompermaier 于2020年3月12日周四 下午9:05写道：
> +1 (non-binding).
> There's also a related issue that I opened a long time ago
> https://issues.apache.org/jira/browse/FLINK-10879 that could be closed
> once
>
Thanks Stephan ~
We can remove the support for properties that may change the semantics of
query if you think that is a trouble.
How about we support the /*+ properties() */ hint only for those optimize
parameters, such as the fetch size of source or something like that, does
that make sense?
+1 (non-binding).
There's also a related issue that I opened a long time ago
https://issues.apache.org/jira/browse/FLINK-10879 that could be closed once
implemented this FLIP (or closed immediately and referenced as duplicated
by the new JIRA ticket that would be created
On Thu, Mar 12, 2020 at
Maximilian Michels created FLINK-16573:
--
Summary: Kinesis consumer does not properly shutdown RecordFetcher
threads
Key: FLINK-16573
URL: https://issues.apache.org/jira/browse/FLINK-16573
> - For 1.10.1 I am not completely sure, because users expect to upgrade
> that without config adjustments. That might not be possible with that
> change.
Ok, makes sense, I will revert it for 1.10 and only try to improve error
message and docs.
> On 12 Mar 2020, at 13:15, Stephan Ewen
Aljoscha Krettek created FLINK-16572:
Summary: CheckPubSubEmulatorTest is flaky on Azure
Key: FLINK-16572
URL: https://issues.apache.org/jira/browse/FLINK-16572
Project: Flink
Issue
@Andrey about the increase in metaspace size
- I have no concerns for 1.11.0.
- For 1.10.1 I am not completely sure, because users expect to upgrade
that without config adjustments. That might not be possible with that
change.
On Thu, Mar 12, 2020 at 12:55 PM Andrey Zagrebin
wrote:
>
> >
Hi,
Looks like that the general consent is to have a release now / very soon
for Flink Stateful Functions.
Since there are not objections so far, I'll proceed to kick off the release
(I'm volunteering to manage this release :)).
I'll cut off a feature branch, release-2.0, next Monday (Mar.
> About "FLINK-16142 Memory Leak causes Metaspace OOM error on repeated job”
My understanding that the issue is basically covered by:
- [FLINK-16225] Metaspace Out Of Memory should be handled as Fatal Error in
TaskManager
no full consensus there but improving error message for existing
created FLINK-16571:
Summary: Throw exception when current key are out of range
Key: FLINK-16571
URL: https://issues.apache.org/jira/browse/FLINK-16571
Project: Flink
Issue Type: Improvement
I think Bowen has actually put it very well.
(1) Hints that change semantics looks like trouble waiting to happen. For
example Kafka offset handling should be in filters. The Kafka source should
support predicate pushdown.
(2) Hints should not be a workaround for current shortcomings. A lot of
Thank you all, for chiming in!
I like the ideas suggested to update the website, I am also +1 for an
update the the main image and make sure we show. Maybe something like [1],
which would also help to increase SQL visibility.
I guess that would be a bit of a bigger effort, and I would kick off
Hi all,
Thanks for the votes.
We have 3 binding +1 votes: Tison, Rong Rong, and Aljoscha and 0 negative ones.
I will move the FLIP to accepted and will start opening JIRAs.
Thanks a lot,
Kostas
On Thu, Mar 12, 2020 at 11:17 AM Aljoscha Krettek wrote:
>
> +1 (binding)
>
> Aljoscha
Hi all!
In general, nice idea to support this integration with Atlas.
I think we could make this a bit easier/lightweight with a small change.
One of the issues that is not super nice is that this starts exposing the
(currently empty) Pipeline interface in the public API.
The Pipeline is an SPI
Fabian Paul created FLINK-16570:
---
Summary: Difficulties to select correct metric with long name in
dropdown of Flink UI task menu
Key: FLINK-16570
URL: https://issues.apache.org/jira/browse/FLINK-16570
+1 (binding)
Aljoscha
Tzu-Li (Gordon) Tai created FLINK-16569:
---
Summary: Allow empty keys in Kafka egress messages for Statefun
Python SDK
Key: FLINK-16569
URL: https://issues.apache.org/jira/browse/FLINK-16569
The StoppableFunction is gone.
See also https://issues.apache.org/jira/browse/FLINK-11889
Best,
tison.
LakeShen 于2020年3月12日周四 下午5:44写道：
> Hi community,
> now I am seeing the FLIP-45 , as I see the stop command only suit
> for the sources that implement the StoppableFunction
Rui Li created FLINK-16568:
--
Summary: Cannot call Hive UDAF that requires constant arguments
Key: FLINK-16568
URL: https://issues.apache.org/jira/browse/FLINK-16568
Project: Flink
Issue Type: Bug
Hi community,
now I am seeing the FLIP-45 , as I see the stop command only suit
for the sources that implement the StoppableFunction interface.
So I have a question is that if I use StopWithSavepoint command to
stop my flink task , just like './flink stop -p xxx ...', this command
It seems this FLIP's name is somewhat misleading. From my understanding,
this FLIP is trying to
address the dynamic parameter issue, and table hints is the way we wan to
choose. I think we should
be focus on "what's the right way to solve dynamic property" instead of
discussing "whether table
jinhai created FLINK-16567:
--
Summary: Get the API error of the StreamQueryConfig on Page "Query
Configuration"
Key: FLINK-16567
URL: https://issues.apache.org/jira/browse/FLINK-16567
Project: Flink
Yangze Guo created FLINK-16566:
--
Summary: Change the log level of the launching command and dynamic
properties from DEBUG to INFO in Mesos integration
Key: FLINK-16566
URL:
Hi Hequn,
Thanks for driving this. +1 to this feature.
Just one minor comment: It seems that we will add an API get_metric_group for
the Python class FunctionContext, could you update the FLIP reflecting this?
Thanks,
Dian
> 在 2020年3月10日，下午3:38，Wei Zhong 写道：
>
> Hi Hequn,
>
> Thanks for
Hequn Cheng created FLINK-16565:
---
Summary: Make Pipeline Json compitable between Java and Python if
all Pipelinestage are Java ones
Key: FLINK-16565
URL: https://issues.apache.org/jira/browse/FLINK-16565
Zhijiang created FLINK-16564:
Summary: HadoopS3RecoverableWriterITCase fails with
NullPointerException
Key: FLINK-16564
URL: https://issues.apache.org/jira/browse/FLINK-16564
Project: Flink
Xintong Song created FLINK-16563:
Summary: CommandLineParser should fail with explicit error message
when parsing recognized arguments.
Key: FLINK-16563
URL: https://issues.apache.org/jira/browse/FLINK-16563
Zili Chen created FLINK-16562:
-
Summary: Handle JobManager termination future in place
Key: FLINK-16562
URL: https://issues.apache.org/jira/browse/FLINK-16562
Project: Flink
Issue Type:
Thanks for launching this discussion @Stephan!
+1 for the idea of linking stateful functions in Flink website. It is time to
increase the exposure of this secret weapon for attracting more attentions.
It is benefit for interested users to try it out in desired scenarios and also
benefit for
Thanks for the reminder Jark. Will keep an eye on these two.
Best Regards,
Yu
On Thu, 12 Mar 2020 at 12:32, Jark Wu wrote:
> Thanks for driving this release, Yu!
> +1 to start 1.10.1 release cycle.
>
> From the Table SQL module, I think we should also try to get in the
> following issues:
> -
70 matches
Mail list logo
