Hi Greg,

at the moment the serialization of savepoints costs the same as the
serialization of checkpoints, because they use the same serialization
logic. In fact, with Ufuk's changes [1], a savepoint is a checkpoint with
special properties. However, in the future we will probably have different
serialization formats. The savepoint would use a generalized format which
allows to restore the savepoint with a different state backend. When
drawing a checkpoint, we don't have to do this, because we know that the
state backend won't change. Thus, we could store the checkpoint in a more
compressed format which exploits the characteristics of the respective
state backend. But this is not implemented yet.

[1] https://github.com/apache/flink/pull/2608


On Wed, Oct 12, 2016 at 2:35 PM, Greg Hogan <c...@greghogan.com> wrote:

> Sorry, I haven't followed this development, but roughly how much more
> costly is the new serialization for savepoints?
> On Wed, Oct 12, 2016 at 5:51 AM, SHI Xiaogang <shixiaoga...@gmail.com>
> wrote:
> > Hi all,
> >
> > Currently, savepoints are exactly the completed checkpoints, and Flink
> > provides commands (save/run) to allow saving and restoring jobs. But in
> the
> > near future, savepoints will be very different from checkpoints because
> > they will have common serialization formats and allow recover from major
> > updates. The saving and restoring based on savepoints will be more
> costly.
> >
> > To provide efficient saving and restoring of jobs, we propose to add two
> > more commands in Flink: SUSPEND and RESUME which are based on
> checkpoints.
> >
> > As the implementation of checkpoints depends on the backends (and many
> > other components in Flink), suspending and resuming may not work if there
> > exist major changes in the job or Flink (e.g., different backends). But
> as
> > the implementation is based on checkpoints instead of savepoints, they
> are
> > supposed to be more efficient.
> >
> > The details of the design can be viewed in the Google Doc: Support
> Resuming
> > and Suspending of Flink Jobs
> > <https://docs.google.com/document/d/1c3vUOTrNlCu2uhfi5ZNYpAguoFR03
> > NgQWZpDTkSxVjg/edit?usp=sharing>
> > .
> >
> > Look forward to your comments. Any feedback is appreciated. :)
> >
> > Thanks,
> > Xiaogang
> >

Reply via email to