Kostas Kloudas created FLINK-15805:
--
Summary: StreamingFileSink on S3 fails when firing processing time
timers after recovery.
Key: FLINK-15805
URL: https://issues.apache.org/jira/browse/FLINK-15805
Hi all,
I agree with Stephan that it is reasonable to assume that multiple
independent jobs should not be in the same "main()" or "application"
or "program" or environment.
But if/when dataStream subsumes dataSet, then batch programs will also
have checkpoints/savepoints and in that case it is
Congratulations Yu and welcome!
On Thu, Jan 23, 2020 at 2:28 PM Till Rohrmann wrote:
>
> Congrats Yu :-)
>
> On Thu, Jan 23, 2020 at 2:02 PM Yang Wang wrote:
>
> > Congratulations, Yu.
> >
> >
> > Best,
> > Yang
> >
> > tison 于2020年1月23日周四 下午7:07写道：
> >
> > > Congratulations!
> > >
> > > Best,
Kostas Kloudas created FLINK-15687:
--
Summary: Potential test instabilities due to concurrent access to
TaskSlotTable.
Key: FLINK-15687
URL: https://issues.apache.org/jira/browse/FLINK-15687
Project
Hi all,
I am writing here as the discussion on the Google Doc seems to be a
bit difficult to follow.
I think that in order to be able to make progress, it would be helpful
to focus on per-job mode for now.
The reason is that:
1) making the (unique) JobSubmitHandler responsible for creating the
Kostas Kloudas created FLINK-15292:
--
Summary: Rename Executor interface to PipelineExecutor
Key: FLINK-15292
URL: https://issues.apache.org/jira/browse/FLINK-15292
Project: Flink
Issue Type
Hi all,
With the feature-freeze for the release-1.10 already past us, it is
time to focus a little bit on documenting the new features that the
community added to this release, and improving the already existing
documentation based on questions that we see in Flink's mailing lists.
To this end,
Kostas Kloudas created FLINK-15279:
--
Summary: Document new `executeAsync()` method and the newly
introduced `JobClient`
Key: FLINK-15279
URL: https://issues.apache.org/jira/browse/FLINK-15279
Kostas Kloudas created FLINK-15278:
--
Summary: Update StreamingFileSink documentation
Key: FLINK-15278
URL: https://issues.apache.org/jira/browse/FLINK-15278
Project: Flink
Issue Type: Sub
Kostas Kloudas created FLINK-15276:
--
Summary: Update CLI documentation to reflect the addition of the
ExecutorCLI
Key: FLINK-15276
URL: https://issues.apache.org/jira/browse/FLINK-15276
Project
Kostas Kloudas created FLINK-15275:
--
Summary: Update CLI documentation to include only current valid
options
Key: FLINK-15275
URL: https://issues.apache.org/jira/browse/FLINK-15275
Project: Flink
Kostas Kloudas created FLINK-15274:
--
Summary: Update Filesystem documentation to reflect changes in
shading
Key: FLINK-15274
URL: https://issues.apache.org/jira/browse/FLINK-15274
Project: Flink
Kostas Kloudas created FLINK-15273:
--
Summary: Complete and Update Documentation
Key: FLINK-15273
URL: https://issues.apache.org/jira/browse/FLINK-15273
Project: Flink
Issue Type: Bug
Kostas Kloudas created FLINK-15260:
--
Summary: Uniformize Kubernetes executor name with the rest
Key: FLINK-15260
URL: https://issues.apache.org/jira/browse/FLINK-15260
Project: Flink
Issue
Kostas Kloudas created FLINK-15195:
--
Summary: Remove unu
Key: FLINK-15195
URL: https://issues.apache.org/jira/browse/FLINK-15195
Project: Flink
Issue Type: Sub-task
Reporter
Kostas Kloudas created FLINK-15179:
--
Summary: Kubernetes should not have a CustomCommandLine.
Key: FLINK-15179
URL: https://issues.apache.org/jira/browse/FLINK-15179
Project: Flink
Issue
Kostas Kloudas created FLINK-15120:
--
Summary: Make FlinkYarnCli#isActive() & #getApplicationId()
respect config APPLICATION_ID
Key: FLINK-15120
URL: https://issues.apache.org/jira/browse/FLINK-1
Kostas Kloudas created FLINK-15119:
--
Summary: Remove PlanExecutor and its subclasses
Key: FLINK-15119
URL: https://issues.apache.org/jira/browse/FLINK-15119
Project: Flink
Issue Type: Sub
Kostas Kloudas created FLINK-15118:
--
Summary: Make flink-scala-shell use Executors
Key: FLINK-15118
URL: https://issues.apache.org/jira/browse/FLINK-15118
Project: Flink
Issue Type: Sub
Kostas Kloudas created FLINK-15042:
--
Summary: Fix python compatibility by excluding the
Env.executeAsync() (FLINK-14854)
Key: FLINK-15042
URL: https://issues.apache.org/jira/browse/FLINK-15042
Kostas Kloudas created FLINK-15041:
--
Summary: Remove default close() implementation from JobClient
Key: FLINK-15041
URL: https://issues.apache.org/jira/browse/FLINK-15041
Project: Flink
Kostas Kloudas created FLINK-15039:
--
Summary: Remove default close() implementation from ClusterClient
Key: FLINK-15039
URL: https://issues.apache.org/jira/browse/FLINK-15039
Project: Flink
Kostas Kloudas created FLINK-15033:
--
Summary: Remove unused RemoteEnvirnment.executeRemotely()
(FLINK-11048)
Key: FLINK-15033
URL: https://issues.apache.org/jira/browse/FLINK-15033
Project: Flink
Kostas Kloudas created FLINK-14972:
--
Summary: Implement RemoteExecutor as a new Executor
Key: FLINK-14972
URL: https://issues.apache.org/jira/browse/FLINK-14972
Project: Flink
Issue Type
Kostas Kloudas created FLINK-14851:
--
Summary: Make the (Stream)ContextEnvironment use the Executors
Key: FLINK-14851
URL: https://issues.apache.org/jira/browse/FLINK-14851
Project: Flink
Kostas Kloudas created FLINK-14850:
--
Summary: Refactor Executor interface and introduce a minimal
JobClient interface
Key: FLINK-14850
URL: https://issues.apache.org/jira/browse/FLINK-14850
Project
+1 (binding)
On Tue, Nov 12, 2019 at 10:20 AM tison wrote:
>
> +1 (binding)
>
> Best,
> tison.
>
>
> Aljoscha Krettek 于2019年11月12日周二 下午4:37写道：
>
> > +1 (binding)
> >
> > > On 8. Nov 2019, at 09:45, Danny Chan wrote:
> > >
> > > Thanks for driving on this, David.
> > >
> > > +1(non-binding)
Kostas Kloudas created FLINK-14758:
--
Summary: Add Executor-related interfaces and utilities
Key: FLINK-14758
URL: https://issues.apache.org/jira/browse/FLINK-14758
Project: Flink
Issue Type
Kostas Kloudas created FLINK-14657:
--
Summary: Generalize and move YarnConfigUtils from flink-yarn to
flink-core
Key: FLINK-14657
URL: https://issues.apache.org/jira/browse/FLINK-14657
Project: Flink
; >>>>
> >>>>> On 23. Oct 2019, at 10:41, Zili Chen wrote:
> >>>>>
> >>>>> Thanks for starting this voting process. I have looked at the FLIP
> >> and
> >>>> the
> >>>>> disc
Kostas Kloudas created FLINK-14548:
--
Summary: Many tests in the nightly fail during cancellation.
Key: FLINK-14548
URL: https://issues.apache.org/jira/browse/FLINK-14548
Project: Flink
Hi all,
This is the voting thread for FLIP-81, as the title says.
The FLIP can be found in [1] and the discussion thread in [2].
As per the bylaws, the vote will stay open until Friday 26-10 (3 days)
or until it gets 3 votes.
Thank you,
Kostas
[1]
> experimental and clearly describing what they do and imply.
>
> Best,
> Aljoscha
>
> > On 22. Oct 2019, at 14:26, Kostas Kloudas wrote:
> >
> > Hi Dawid,
> >
> > For the first comment, I am also up for re-using as many options as
> > possible. My
Kostas Kloudas created FLINK-14502:
--
Summary: Make ClusterDescriptor/ClusterSpecification creation
configuration-based.
Key: FLINK-14502
URL: https://issues.apache.org/jira/browse/FLINK-14502
Kostas Kloudas created FLINK-14501:
--
Summary: Move ClusteDescriptor/ClusterSpecification creation from
the CustomCommandLine to a ClusterClientFactory
Key: FLINK-14501
URL: https://issues.apache.org/jira/browse
it more
> > prominent that we don’t like the “attached”, “shutdown-on-attached-exit”,
> > and “dynamic-properties” and that we only add them for backwards
> > compatibility.
> >
> > Best,
> > Aljoscha
> >
> >> On 21. Oct 2019, at 17:48,
+1 (binding)
On Mon, Oct 21, 2019 at 5:18 PM Aljoscha Krettek wrote:
>
> +1 (binding)
>
> Aljoscha
>
> > On 21. Oct 2019, at 16:18, Thomas Weise wrote:
> >
> > +1 (binding)
> >
> >
> > On Mon, Oct 21, 2019 at 7:10 AM Timo Walther wrote:
> >
> >> +1 (binding)
> >>
> >> Thanks,
> >> Timo
> >>
>
Hi all,
As part of FLIP-73 (the Executors effort), we would like to introduce
some new configuration options. These are needed in order to be able
to map all the options that the user can specify either
programmatically or through the CLI into configuration options.
The bylaws specify that every
M Timo Walther wrote:
>
> Hi Kostas,
>
> can we still discuss the naming of the properties? For me, having
> "execution" and "exector" as prefixes might be confusing in the future
> and difficult to identify if you scan through a list of properties.
>
> How a
StringEncoder<>("UTF-8"))
.withBucketAssigner(assignerB)
.build();
myInputFromKafka.addSink(sinkA)
myInputFromKafka.addSink(sinkB)
Cheers,
Kostas
-- Forwarded message -
From: amran dean
Date: Wed, Oct 16, 2019 at 8:43 PM
Subject: Re: Verifying correctness of StreamingFileSink
Hi all,
Thanks for opening the discussion!
I like the idea, so +1 from my side and actually this is aligned with
our intensions for the FLIP-73 effort.
For the naming convention of the parameters introduced in the FLIP, my
proposal would be have the full word "execution" instead of the
shorter
Hi all,
Although in the initial thread I said that, in general, I would prefer
having one repository, I understand that arguments presented here and
I think it makes sense for such a young project to have its own
repository.
So +1 from my side, with an asterisk about hoping that eventually the
Hi Enrico,
Thanks for opening the discussion!
One thing to note that may help s that the hadoop S3 FS tries to
imitate a filesystem on top of S3:
- before writing a key it checks if the "parent directory" exists by
checking for a key with the prefix up to the last "/"
- it creates empty marker
Hi Aljoscha,
Given that config option keys are user-facing and any change there is
breaking, I think there should be a discussion about them and a FLIP,
where people have to actually vote for it seems to be the right place.
I understand that this is tedious (and actually I will also have to
open
Hi all,
Big +1 for contributing Stateful Functions to Flink and as for the
main question at hand, I would vote for putting it in the main
repository.
I understand that this can couple the release cadence of Flink and
Stateful Functions although I think the pros of having a "you break
it,
you fix
Kostas Kloudas created FLINK-14377:
--
Summary: Translate ProgramOptions relevant for job execution to
ConfigOptions.
Key: FLINK-14377
URL: https://issues.apache.org/jira/browse/FLINK-14377
Project
Kostas Kloudas created FLINK-14376:
--
Summary: Introduce the Executor abstraction (FLIP-73)
Key: FLINK-14376
URL: https://issues.apache.org/jira/browse/FLINK-14376
Project: Flink
Issue Type
uded from the FLIP. But
> to emphasize that we make the decision to exclude it I add it to reject
> alternatives.
>
> (2) Updated FLIP to reflect the consensus :-)
>
> Best,
> tison.
>
>
> Kostas Kloudas 于2019年10月11日周五 下午5:12写道：
>
> > Hi all,
> >
>
Hi all,
I only have two minor comments before voting and they have to do with
the following:
1) In the discussion, we agreed to remove the cancelWithSavepoint()
from the JobClient as this is deprecated in the rest API. This is not
in the FLIP.
2) The section "ClusterDescriptor or
er client or a job client? As discussed previously
> I think conceptually it is a job client?
>
> Best,
> tison.
>
>
> Kostas Kloudas 于2019年10月10日周四 下午5:08写道：
>
> > Hi Tison,
> >
> > I would say that as a first step, and until we see that the interfaces
> &g
as Weise wrote:
> >
> > +1
> >
> >
> > On Fri, Oct 4, 2019 at 8:56 AM Zili Chen wrote:
> >
> >> Thanks for your works Kostas!
> >>
> >> +1 for FLIP-73
> >>
> >> Best,
> >> tison
> >>
> >>
> &g
o return a CompletableFuture and to completely
> > > remove
> > > >> the “detached” logic from ClusterClient. That way, the new components
> > > make
> > > >> no distinction between “detached” and “attached” but we can still do
> > it
> > &g
Hi all,
I would like to start the vote for FLIP-73 [1], which is discussed and
reached a consensus in the discussion thread[2].
Given that it is Friday, the vote will be open until Oct. 9th (72h
starting on Monday), unless there is an objection or not enough votes.
Thanks,
Kostas
[1]
gt;>>>
> >>>>>>>>> Besides, refer to the discussion with Till[1], it would be
> >>> helpful
> >>>> we
> >>>>>>>>> follow the same process
> >>>>>>>>> of ses
Hi Tison,
I see. Then I would say that as a first step, and to see if people are
happy with the result,
integration with the production code can be through a new method
executeAsync() in the Executor
that we discussed earlier.
This method could potentially be exposed to ExecutionEnvironment as a
configured on Executor created
> >> and
> >> let this one overwhelm that one.
> >>
> >> I can see it is useful that conceptually we can create an Executor for a
> >> series jobs
> >> to the same cluster but with different job configuration per pi
and use it to execute multiple pipelines, if
in the future we choose to do so.
Let me know what do you think,
Kostas
On Wed, Oct 2, 2019 at 7:23 PM Kostas Kloudas wrote:
>
> Hi all,
>
> I agree with Tison that we should disentangle threads so that people
> can work independently.
Hi all,
I agree with Tison that we should disentangle threads so that people
can work independently.
For FLIP-73:
- for Preview/OptimizedPlanEnv: I think they are orthogonal to the
Executors work, as they are using the exexute() method because this is
the only "entry" to the user program. To
Hi all,
I would also like to give a +1 for supporting lists as config options
with the delimeter being a parameter (if we cannot find a consensus).
To some extent the current codebase has already solved the issue by
already having lists as options, but the problem is that so far there
was no
ml/b2e22a45aeb94a8d06b50c4de078f7b23d9ff08b8226918a1a903768@%3Cdev.flink.apache.org%3E
> [2]
> https://lists.apache.org/x/thread.html/240582148eda905a772d59b2424cb38fa16ab993647824d178cacb02@%3Cdev.flink.apache.org%3E
> [3]
> https://docs.google.com/document/d/1E-8UjOLz4QPUTxetGWbU23OlsIH9V
Hi Tison,
I shared some thoughts on this topic in the FLIP-74 thread, as some of
the questions are related to that.
Cheers,
Kostas
On Wed, Sep 25, 2019 at 12:30 PM Zili Chen wrote:
>
> Hi all,
>
> While drafting FLIP-74 I notice that a job level client(called JobClient) is
> always retrieved
Hi Tison,
Thanks for the FLIP and launching the discussion!
As a first note, big +1 on providing/exposing a JobClient to the users!
Some points that would be nice to be clarified:
1) You mention that we can get rid of the DETACHED mode: I agree that
at a high level, given that everything will
it. What do you think?
>
> Best,
> Aljoscha
>
> > On 25. Sep 2019, at 10:11, Kostas Kloudas wrote:
> >
> > Hi Thomas and Zili,
> >
> > As you both said the Executor is a new addition so there are no
> > compatibility concerns.
> > Backwards compati
d compatibility, it is on (Stream)ExecutionEnvironment#execute.
> You're right that we don't stick to blocking to return a JobExecutionResult in
> Executor aspect but implementing env.execute with a unique
>
> Executor#execute(or with suffix Async): CompletableFuture
>
> what do you th
Hi all,
In the context of the discussion about introducing the Job Client API [1],
there was a side-discussion about refactoring the way users submit jobs in
Flink. There were many different interesting ideas on the topic and 3
design documents that were trying to tackle both the issue about code
Kostas Kloudas created FLINK-14157:
--
Summary: Temporarily remove S3 StreamingFileSink end-to-end test
Key: FLINK-14157
URL: https://issues.apache.org/jira/browse/FLINK-14157
Project: Flink
t;> of
>>> >> concept implementation for the new JobClient interface, maybe with a new
>>> >> method executeAsync() in the environment that returns the JobClient and
>>> >> implement the methods to see how that works and to see where we get.
>>
Hi Enrico,
Sorry for the late reply. I think your understanding is correct.
The best way to do it is to write your own ParquetBulkWriter and the
corresponding factory.
Out of curiosity, I guess that in the BucketingSink you were using the
AvroKeyValueSinkWriter, right?
Cheers,
Kostas
On Fri,
Qin
> Send Time:2019 Sep. 9 (Mon.) 10:47
> To:dev
> Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC
>
> Congrats, Kostas!
>
> On Sun, Sep 8, 2019 at 11:48 PM myasuka wrote:
>
> > Congratulations Kostas!
> >
> > Best
> > Yun Tang
>
Hi Gary,
Thanks for kicking off the feature discussion.
+1 for Gary and Yu as release managers.
Cheers,
Kostas
On Fri, Sep 6, 2019 at 5:06 PM Gary Yao wrote:
>
> Hi community,
>
> Since Apache Flink 1.9.0 has been released more than 2 weeks ago, I want to
> start kicking off the discussion
Kostas Kloudas created FLINK-13972:
--
Summary: Move PreviewPlanEnvironment to flink-tests
Key: FLINK-13972
URL: https://issues.apache.org/jira/browse/FLINK-13972
Project: Flink
Issue Type
Kostas Kloudas created FLINK-13959:
--
Summary: Consolidate DetachedEnvironment and ContextEnvironment
Key: FLINK-13959
URL: https://issues.apache.org/jira/browse/FLINK-13959
Project: Flink
, 2019 at 11:46 AM Kostas Kloudas wrote:
>
> Great idea Tison!
>
> I will create the umbrella issue and post it here so that we are all
> on the same page!
>
> Cheers,
> Kostas
>
> On Wed, Sep 4, 2019 at 11:36 AM Zili Chen wrote:
> >
> > Hi Kostas & A
Kostas Kloudas created FLINK-13954:
--
Summary: Clean up ExecutionEnvironment / JobSubmission code paths
Key: FLINK-13954
URL: https://issues.apache.org/jira/browse/FLINK-13954
Project: Flink
ili Chen 于2019年8月31日周六 下午12:52写道：
>>
>> Great Kostas! Looking forward to your POC!
>>
>> Best,
>> tison.
>>
>>
>> Jeff Zhang 于2019年8月30日周五 下午11:07写道：
>>>
>>> Awesome, @Kostas Looking forward your POC.
>>>
>>> Kosta
Kostas Kloudas created FLINK-13946:
--
Summary: Remove deactivated JobSession-related code.
Key: FLINK-13946
URL: https://issues.apache.org/jira/browse/FLINK-13946
Project: Flink
Issue Type
> in the future.
> So should we update the fixVersion of FLINK-13940?
>
> Best,
> Jark
>
> On Tue, 3 Sep 2019 at 21:33, Kostas Kloudas wrote:
>
> > Thanks for waiting!
> >
> > A fix for FLINK-13940 has been merged on 1.8, 1.9 and the master under
> > FLIN
very important bug, And great thanks for
> helping fix it Kostas!
>
> Best, Jincheng
>
> Kostas Kloudas 于2019年9月2日周一 下午7:20写道：
>
> > Hi all,
> >
> > I think this should be also considered a blocker
> > https://issues.apache.org/jira/browse/FLINK-13940.
Kostas Kloudas created FLINK-13941:
--
Summary: Prevent data-loss by not cleaning up small part files
from S3.
Key: FLINK-13941
URL: https://issues.apache.org/jira/browse/FLINK-13941
Project: Flink
Hi all,
I think this should be also considered a blocker
https://issues.apache.org/jira/browse/FLINK-13940.
It is not a regression but it can result to data loss.
I think I can have a quick fix by tomorrow.
Cheers,
Kostas
On Mon, Sep 2, 2019 at 12:01 PM jincheng sun wrote:
>
> Thanks for all
ClusterEntryPoint actually uses the user code class loader
> > > with child first class loading [1]. Hence, I hope that this problem will
> > be
> > > resolved soon.
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-13840
> > >
Hi all,
I agree with Arvid, although for point 2 I would be less strict.
@Piotr, for the side note you mentioned, and from the description you
mention in the mail for example I,
it seems that the need to pass parameters in the build() is not an
inherent need of the build pattern but it
can be
Hi all,
On the topic of web submission, I agree with Till that it only seems
to complicate things.
It is bad for security, job isolation (anybody can submit/cancel jobs), and its
implementation complicates some parts of the code. So, if it were to
redesign the
WebUI, maybe this part could be left
Thanks a lot!
We have the 3 +1's so we can move forward.
Cheers,
Kostas
On Wed, Aug 21, 2019 at 1:30 PM Chesnay Schepler wrote:
>
> +1
>
> On 21/08/2019 13:23, Timo Walther wrote:
> > +1
> >
> > Am 21.08.19 um 13:21 schrieb Stephan Ewen:
> >> +1
> >
Hi all,
Following the FLIP process, this is a voting thread dedicated to the FLIP-52.
As shown from the corresponding discussion thread [1], we seem to agree that
the Program interface can be removed, so let's make it also official
with a vote.
Cheers,
Kostas
[1]
t,
> > > tison.
> > >
> > > [1]
> > >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/REST-API-JarRunHandler-More-flexibility-for-launching-jobs-td31026.html#a31168
> > >
> > >
> > > Stephan Ewen 于2019年8月14日周三 下午7:5
Congratulations Andrey!
Well deserved!
Kostas
On Wed, Aug 14, 2019 at 4:04 PM Yun Tang wrote:
>
> Congratulations Andrey.
>
> Best
> Yun Tang
>
> From: Xintong Song
> Sent: Wednesday, August 14, 2019 21:40
> To: Oytun Tez
> Cc: Zili Chen ; Till Rohrmann ;
>
Kostas Kloudas created FLINK-13716:
--
Summary: Remove Package-related chinese documentation
Key: FLINK-13716
URL: https://issues.apache.org/jira/browse/FLINK-13716
Project: Flink
Issue Type
Kostas Kloudas created FLINK-13715:
--
Summary: Remove Package-related english documentation.
Key: FLINK-13715
URL: https://issues.apache.org/jira/browse/FLINK-13715
Project: Flink
Issue Type
Kostas Kloudas created FLINK-13714:
--
Summary: Remove Package-related code.
Key: FLINK-13714
URL: https://issues.apache.org/jira/browse/FLINK-13714
Project: Flink
Issue Type: Sub-task
users of Flink don’t actually know it exists and
> > it is only making our code more complicated. Going forward with the new
> > Client API discussions will be a lot easier without it as well.
> >
> > Best,
> > Aljoscha
> >
> > > On 14. Aug
Kostas Kloudas created FLINK-13713:
--
Summary: Remove legacy Package interface.
Key: FLINK-13713
URL: https://issues.apache.org/jira/browse/FLINK-13713
Project: Flink
Issue Type: Bug
Hi all,
It is nice to have this discussion.
I am totally up for removing the unused Program interface, as this will
simplify a lot of other code paths in the ClusterClient and elsewhere.
Also about the easier integration of Flink with other frameworks, there
is another discussion in the mailing
obgraph.png
> [2] https://transfer.sh/iDg1A/watermarks.png
>
> On 8/7/19 3:04 PM, Kostas Kloudas wrote:
> > But are they chained together? Could you provide the code from your job,
> at
> > least until operator A?
> >
> > On Wed, Aug 7, 2019 at 3:03 PM Jan Lu
But are they chained together? Could you provide the code from your job, at
least until operator A?
On Wed, Aug 7, 2019 at 3:03 PM Jan Lukavský wrote:
> Actually, operator A is intermediate, source is preceding it.
>
> On 8/7/19 2:44 PM, Kostas Kloudas wrote:
> > Hi Jan,
> &
rote:
> Hi Kostas,
>
> thanks for reaction, comments inline.
>
> On 8/7/19 1:59 PM, Kostas Kloudas wrote:
> > Hi Jan,
> >
> > Two pointers that may help you explain the behavior are the following:
> >
> > 1) If you have a custom watermark generator, I do no
Hi Jan,
Two pointers that may help you explain the behavior are the following:
1) If you have a custom watermark generator, I do not think that Flink
checks if it emits only monotonically
increasing watermarks. This is the responsibility of the generator itself.
This means that although you
Kostas Kloudas created FLINK-13605:
--
Summary: AsyncDataStreamITCase. testUnorderedWait failed on Travis
Key: FLINK-13605
URL: https://issues.apache.org/jira/browse/FLINK-13605
Project: Flink
Kostas Kloudas created FLINK-13440:
--
Summary: Add test that fails job when sync savepoint is discarded.
Key: FLINK-13440
URL: https://issues.apache.org/jira/browse/FLINK-13440
Project: Flink
Hi Elkhan,
As Till pointed out, the fact that your files remain in-progress is the
expected behaviour as the StreamingFileSink assumes
checkpointing is enabled in order to work. There are no plans to change
this behaviour but an issue that may be relevant
for you is
101 - 200 of 450 matches
Mail list logo
