To: dev@flink.apache.org
Subject: Re: Force enabling checkpoints for iterative streaming
jobs
I disagree. Not having checkpointed operators inside the
iteration
still
breaks the guarantees.
It is not about the states it is about the loop itself.
On Wed, Jun 10
@flink.apache.org
Subject: Re: Force enabling checkpoints for iterative streaming jobs
I disagree. Not having checkpointed operators inside the iteration
still
breaks the guarantees.
It is not about the states it is about the loop itself.
On Wed, Jun 10, 2015 at 10:12 AM Aljoscha Krettek
whether there is a nice way
to
make
iterations more structured.
Paris
From: Gyula Fóra gyula.f...@gmail.com
Sent: Wednesday, June 10, 2015 10:19 AM
To: dev@flink.apache.org
Subject: Re: Force enabling checkpoints
: Force enabling checkpoints for iterative streaming jobs
I disagree. Not having checkpointed operators inside the iteration
still
breaks the guarantees.
It is not about the states it is about the loop itself.
On Wed, Jun 10, 2015 at 10:12 AM Aljoscha Krettek
aljos
gyula.f...@gmail.com
Sent: Wednesday, June 10, 2015 10:19 AM
To: dev@flink.apache.org
Subject: Re: Force enabling checkpoints for iterative streaming jobs
I disagree. Not having checkpointed operators inside the iteration still
breaks the guarantees.
It is not about the states it is about
I disagree. Not having checkpointed operators inside the iteration still
breaks the guarantees.
It is not about the states it is about the loop itself.
On Wed, Jun 10, 2015 at 10:12 AM Aljoscha Krettek aljos...@apache.org
wrote:
This is the answer I gave on the PR (we should have one place for
iterations
more structured.
Paris
From: Gyula Fóra gyula.f...@gmail.com
Sent: Wednesday, June 10, 2015 10:19 AM
To: dev@flink.apache.org
Subject: Re: Force enabling checkpoints for iterative streaming jobs
I disagree. Not having checkpointed operators inside
On 10 Jun 2015, at 14:29, Gyula Fóra gyula.f...@gmail.com wrote:
Max suggested that I add this feature slightly hidden to the execution
config instance.
The problem then is that I either make a public field in the config or once
again add a method.
Any ideas?
I thought about this as
Then I suggest we leave it in the environment along with the other
checkpointing methods.
I updated my PR so it includes hints how to force enable checkpoints (and
the reduced guarantees) when an error is thrown for iterative jobs.
On Wed, Jun 10, 2015 at 2:46 PM, Aljoscha Krettek
We could add a method on the ExecutionConfig but mark it as deprecated
and explain that it will go away once the interplay of iterations,
state and so on is properly figured out.
On Wed, Jun 10, 2015 at 2:36 PM, Ufuk Celebi u...@apache.org wrote:
On 10 Jun 2015, at 14:29, Gyula Fóra
This is the answer I gave on the PR (we should have one place for
discussing this, though):
I would be against merging this in the current form. What I propose is
to analyse the topology to verify that there are no checkpointed
operators inside iterations. Operators before and after iterations
I agree that for the sake of the above mentioned use cases it is reasonable
to add this to the release with the right documentation, for machine
learning potentially loosing one round of feedback data should not matter.
Let us not block prominent users until the next release on this.
On Wed, Jun
: Force enabling checkpoints for iterative streaming jobs
I disagree. Not having checkpointed operators inside the iteration still
breaks the guarantees.
It is not about the states it is about the loop itself.
On Wed, Jun 10, 2015 at 10:12 AM Aljoscha Krettek aljos...@apache.org
wrote
Let's mark the method of the environment as deprecated like Aljoscha
suggested. Then I think we could merge it.
On Wed, Jun 10, 2015 at 2:50 PM, Gyula Fóra gyula.f...@gmail.com wrote:
Then I suggest we leave it in the environment along with the other
checkpointing methods.
I updated my PR so
Done, I will merge it after travis passes.
Maximilian Michels m...@apache.org ezt írta (időpont: 2015. jún. 10., Sze,
15:25):
Let's mark the method of the environment as deprecated like Aljoscha
suggested. Then I think we could merge it.
On Wed, Jun 10, 2015 at 2:50 PM, Gyula Fóra
As for people currently suffering from it:
An application King is developing requires iterations, and they need
checkpoints. Practically all SAMOA programs would need this.
It is very likely that the state interfaces will be changed after the
release, so this is not something that we can just
Hey Gyula,
I understand your reasoning, but I don't think its worth to rush this into the
release.
As you've said, we cannot give precise guarantees. But this is arguably one of
the key requirements for any fault tolerance mechanism. Therefore I disagree
that this is better than not having
17 matches
Mail list logo
