[
https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018741#comment-15018741
]
ASF GitHub Bot commented on STORM-885:
--------------------------------------
Github user knusbaum commented on a diff in the pull request:
https://github.com/apache/storm/pull/838#discussion_r45515660
--- Diff:
storm-core/src/clj/org/apache/storm/pacemaker/pacemaker_state_factory.clj ---
@@ -0,0 +1,124 @@
+;; Licensed to the Apache Software Foundation (ASF) under one
+;; or more contributor license agreements. See the NOTICE file
+;; distributed with this work for additional information
+;; regarding copyright ownership. The ASF licenses this file
+;; to you under the Apache License, Version 2.0 (the
+;; "License"); you may not use this file except in compliance
+;; with the License. You may obtain a copy of the License at
+;;
+;; http://www.apache.org/licenses/LICENSE-2.0
+;;
+;; Unless required by applicable law or agreed to in writing, software
+;; distributed under the License is distributed on an "AS IS" BASIS,
+;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+;; See the License for the specific language governing permissions and
+;; limitations under the License.
+
+(ns org.apache.storm.pacemaker.pacemaker-state-factory
+ (:require [org.apache.storm.pacemaker pacemaker]
+ [backtype.storm.cluster-state [zookeeper-state-factory :as
zk-factory]]
+ [backtype.storm
+ [config :refer :all]
+ [cluster :refer :all]
+ [log :refer :all]
+ [util :as util]])
+ (:import [backtype.storm.generated
+ HBExecutionException HBNodes HBRecords
+ HBServerMessageType HBMessage HBMessageData HBPulse]
+ [backtype.storm.cluster_state zookeeper_state_factory]
+ [backtype.storm.cluster ClusterState]
+ [org.apache.storm.pacemaker PacemakerClient])
+ (:gen-class
+ :implements [backtype.storm.cluster.ClusterStateFactory]))
+
+;; So we can mock the client for testing
+(defn makeClient [conf]
+ (PacemakerClient. conf))
+
+(defn makeZKState [conf auth-conf acls context]
+ (.mkState (zookeeper_state_factory.) conf auth-conf acls context))
+
+(def max-retries 10)
+
+(defn -mkState [this conf auth-conf acls context]
+ (let [zk-state (makeZKState conf auth-conf acls context)
+ pacemaker-client (makeClient conf)]
+
+ (reify
+ ClusterState
+ ;; Let these pass through to the zk-state. We only want to handle
heartbeats.
+ (register [this callback] (.register zk-state callback))
+ (unregister [this callback] (.unregister zk-state callback))
+ (set_ephemeral_node [this path data acls] (.set_ephemeral_node
zk-state path data acls))
+ (create_sequential [this path data acls] (.create_sequential
zk-state path data acls))
+ (set_data [this path data acls] (.set_data zk-state path data acls))
+ (delete_node [this path] (.delete_node zk-state path))
+ (get_data [this path watch?] (.get_data zk-state path watch?))
+ (get_data_with_version [this path watch?] (.get_data_with_version
zk-state path watch?))
+ (get_version [this path watch?] (.get_version zk-state path watch?))
+ (get_children [this path watch?] (.get_children zk-state path
watch?))
+ (mkdirs [this path acls] (.mkdirs zk-state path acls))
+ (node_exists [this path watch?] (.node_exists zk-state path watch?))
+ (add_listener [this listener] (.add_listener zk-state listener))
+ (sync_path [this path] (.sync_path zk-state path))
+
+ (set_worker_hb [this path data acls]
+ (util/retry-on-exception
+ max-retries
+ "set_worker_hb"
+ #(let [response
+ (.send pacemaker-client
+ (HBMessage. HBServerMessageType/SEND_PULSE
+ (HBMessageData/pulse
+ (doto (HBPulse.)
+ (.set_id path)
+ (.set_details data)))))]
+ (if (= (.get_type response)
HBServerMessageType/SEND_PULSE_RESPONSE)
+ :ok
+ (throw (HBExecutionException. "Invalid Response Type"))))))
--- End diff --
It's part of the ClusterState interface spec.
> Heartbeat Server (Pacemaker)
> ----------------------------
>
> Key: STORM-885
> URL: https://issues.apache.org/jira/browse/STORM-885
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Kyle Nusbaum
>
> Large highly connected topologies and large clusters write a lot of data into
> ZooKeeper. The heartbeats, that make up the majority of this data, do not
> need to be persisted to disk. Pacemaker is intended to be a secure
> replacement for storing the heartbeats without changing anything within the
> heartbeats. In the future as more metrics are added in, we may want to look
> into switching it over to look more like Heron, where a metrics server is
> running for each node/topology. And can be used to aggregate/per-aggregate
> them in a more scalable manor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
