[
https://issues.apache.org/jira/browse/FLINK-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brendan MacDonell updated FLINK-14109:
--------------------------------------
Description:
My team at Sight Machine just finished building a custom HA backend for our
infrastructure. The process was mostly painless, but we ran into a few pain
points on the way:
* {{CompletedCheckpointStore#getLatestCheckpoint}} is not marked as
{{@Nullable}}, so there was some confusion about whether the Javadocs or (lack
of) annotation is correct. The interface would be clearer if the annotation was
present.
* The javadocs for {{CompletedCheckpointStore#recover}} disagree with the
documentation for {{ZooKeeperCompletedCheckpointStore#recover}}. It's not
immediately clear to someone working on the code that the ZK javadoc is
outdated and the interface documentation is correct.
* -The base {{CompletedCheckpointStore}} tests only work with
high-availability backends that keep a list of checkpoints in memory. If the
backend persists and retrieves data from another source the tests will fail.
It's fairly simple to change the tests to lift this requirement though. See
[this gist|https://gist.github.com/bremac/1b3365bc0257dfbd33bcd0b7a7627c00] for
an example.-
We've got patches for the points above that we'd be happy to contribute. :)
was:
My team at Sight Machine just finished building a custom HA backend for our
infrastructure. The process was mostly painless, but we ran into a few pain
points on the way:
* {{CompletedCheckpointStore#getLatestCheckpoint}} is not marked as
\{{@Nullable}}, so there was some confusion about whether the Javadocs or (lack
of) annotation is correct. The interface would be clearer if the annotation was
present.
* The javadocs for {{CompletedCheckpointStore#recover}} disagree with the
documentation for {{ZooKeeperCompletedCheckpointStore#recover}}. It's not
immediately clear to someone working on the code that the ZK javadoc is
outdated and the interface documentation is correct.
* The base {{CompletedCheckpointStore}} tests only work with high-availability
backends that keep a list of checkpoints in memory. If the backend persists and
retrieves data from another source the tests will fail. It's fairly simple to
change the tests to lift this requirement though. See [this
gist|https://gist.github.com/bremac/1b3365bc0257dfbd33bcd0b7a7627c00] for an
example.
We've got patches for the points above that we'd be happy to contribute. :)
> Improve javadocs and tests for high-availability backend
> --------------------------------------------------------
>
> Key: FLINK-14109
> URL: https://issues.apache.org/jira/browse/FLINK-14109
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: Brendan MacDonell
> Assignee: Brendan MacDonell
> Priority: Minor
>
> My team at Sight Machine just finished building a custom HA backend for our
> infrastructure. The process was mostly painless, but we ran into a few pain
> points on the way:
> * {{CompletedCheckpointStore#getLatestCheckpoint}} is not marked as
> {{@Nullable}}, so there was some confusion about whether the Javadocs or
> (lack of) annotation is correct. The interface would be clearer if the
> annotation was present.
> * The javadocs for {{CompletedCheckpointStore#recover}} disagree with the
> documentation for {{ZooKeeperCompletedCheckpointStore#recover}}. It's not
> immediately clear to someone working on the code that the ZK javadoc is
> outdated and the interface documentation is correct.
> * -The base {{CompletedCheckpointStore}} tests only work with
> high-availability backends that keep a list of checkpoints in memory. If the
> backend persists and retrieves data from another source the tests will fail.
> It's fairly simple to change the tests to lift this requirement though. See
> [this gist|https://gist.github.com/bremac/1b3365bc0257dfbd33bcd0b7a7627c00]
> for an example.-
> We've got patches for the points above that we'd be happy to contribute. :)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
