----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29044/ -----------------------------------------------------------
(Updated Dec. 18, 2014, 8:09 a.m.) Review request for sentry, bc Wong, Lenni Kuff, Prasad Mujumdar, and Sravya Tirukkovalur. Changes ------- Updated with documentation Repository: sentry Description ------- This patch includes another implementation of the PersistentSentryStore Decorator that, in addition to logging operations locally, will also replicate the logs to other peers in a cluster. It uses Hazelcast for the following : 1) It uses Hazelcast's implementation of a distributed topic. A SentryStore peer receiving a write/update request will replicate the write to its peers by publishing the record to the topic. A distributed topic can be configured for globalOrdering thereby ensuring that writes are ordered. 2) This is coupled with a distributed counter to ensure monotonically increasing seqence numbers for the log records. Consistency considerations: This deisgn implies that the SentryStore cluster is Eventually consistent since a Replica will have to wait for the log entry from the peer that initiated the write. The current HA design of Sentry selects a SentryService in a round-robin manner, but if this is changed to 'STICKY' (clients will make requests / 'stick' to one of the available Sentry servers and will switch only incase the node has gone down or any other network issue), it would make the service strongly consistent. Why use HazelCast instead of Zookeeper ? 1) Zookeeper APIs, even using curator is slighlty more complex than HazelCast'sm t 2) Hazelcast can trivially be dropped-in as a library and implements java.util.concurrent and collection classes 3) Zookeeper is less optimized for large data transfers (and clients can miss zookeeper watches which complicates the code.. while hazelcast guarantees topic message delivery to connected clients). There is a possibility that full store snapshots might have to be sent to a node when it starts up More decription on the JIRA : issues.apache.org/jira/browse/SENTRY-567 Diffs (updated) ----- sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/DistributedUtils.java PRE-CREATION sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/persistent/SentryStoreWithReplicatedLog.java PRE-CREATION sentry-provider/sentry-provider-db/src/test/java/org/apache/sentry/provider/db/service/persistent/TestSentryStoreWithReplicatedLog.java PRE-CREATION Diff: https://reviews.apache.org/r/29044/diff/ Testing ------- Thanks, Arun Suresh
