MalcolmSanders created FLINK-11811:
--------------------------------------
Summary: Implement etcd based RunningJobsRegistry and
SubmittedJobGraphStore
Key: FLINK-11811
URL: https://issues.apache.org/jira/browse/FLINK-11811
Project: Flink
Issue Type: Sub-task
Components: Runtime / Coordination
Reporter: MalcolmSanders
Assignee: MalcolmSanders
Zookeeper based RunningJobsRegistry uses zookeeper as a key-value store to keep
the latest job statuses using persistent zookeeper nodes. So etcd based
RunningJobsRegistry can use io.etcd.jetcd.KV to achieve the same goal.
Zookeeper based SubmittedJobGraphStore applies ZooKeeperStateHandleStore to
store submitted each job graph in a dfs file while records its metadata, such
as dfs location, to a zookeeper node in a zookeeper directory. etcd based
SubmittedJobGraphStore will use EtcdSubmittedJobGraphStore to fulfill the
target.
In addition, SubmittedJobGraphStore needs to watch this zookeeper directory to
get notifications once znodes in such directory is created or deleted. Previous
etcd API io.etcd.jetcd.Watch can watch a key as prefix by setting
io.etcd.jetcd.options.WatchOption#withPrefix() which is just like watching a
directory in zookeeper.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
