Clint Byrum wrote:
Excerpts from Mike Perez's message of 2015-07-31 10:40:04 -0700:
On Fri, Jul 31, 2015 at 8:56 AM, Joshua Harlow<harlo...@outlook.com> wrote:
...random thought here, skip as needed... in all honesty orchestration
solutions like mesos
(http://mesos.apache.org/assets/img/documentation/architecture3.jpg),
map-reduce solutions like hadoop, stream processing systems like apache
storm (...), are already using zookeeper and I'm not saying we should just
use it cause they are, but the likelihood that they just picked it for no
reason are imho slim.
I'd really like to see focus cross project. I don't want Ceilometer to
depend on Zoo Keeper, Cinder to depend on etcd, etc. This is not ideal
for an operator to have to deploy, learn and maintain each of these
solutions.
I think this is difficult when you consider everyone wants options of
their preferred DLM. If we went this route, we should pick one.
Regardless, I want to know if we really need a DLM. Does Ceilometer
really need a DLM? Does Cinder really need a DLM? Can we just use a
hash ring solution where operators don't even have to know or care
about deploying a DLM and running multiple instances of Cinder manager
just works?
So in the Ironic case, if two conductors decide they both own one IPMI
controller, _chaos_ can ensue. They may, at different times, read that
the power is up, or down, and issue power control commands that may take
many seconds, and thus on the next status run of the other command may
cause the conductor to react by reversing, and they'll just fight over
the node in a tug-o-war fashion.
Oh wait, except, thats not true. Instead, they use the database as a
locking mechanism, and AFAIK, no nodes have been torn limb from limb by
two conductors thus far.
But, a DLM would be more efficient, and actually simplify failure
recovery for Ironic's operators. The database locks suffer from being a
little too conservative, and sometimes you just have to go into the DB
and delete a lock after something explodes (this was true 6 months ago,
it may have better automation sometimes now, I don't know).
A point of data, using kazoo, and zk-shell (python library and python
zookeeper shell like interface), just to show how much introspection can
be done with zookeeper when a kazoo lock is created (tooz locks when
used with zookeeper use this same/similar code).
(session #1)
>>> from kazoo import client
>>> c = client.KazooClient()
>>> c.start()
>>> lk = c.Lock()
>>> lk = c.Lock('/resourceX')
>>> lk.acquire()
True
(session #2)
$ zk-shell
Welcome to zk-shell (1.1.0)
(DISCONNECTED) /> connect
(DISCONNECTED) /> connect localhost:2181
(CLOSED) />
(CONNECTED) /> ls /resourceX
75ef011db92a44bfabf5dbf25fe2965c__lock__0000000000
(CONNECTED) /> stat
/resourceX/75ef011db92a44bfabf5dbf25fe2965c__lock__0000000000
Stat(
czxid=8103
mzxid=8103
ctime=1438383904513
mtime=1438383904513
version=0
cversion=0
aversion=0
ephemeralOwner=0x14ed0a76f850002
dataLength=0
numChildren=0
pzxid=8103
)
(CONNECTED) /> stat /resourceX/
Stat(
czxid=8102
mzxid=8102
ctime=1438383904494
mtime=1438383904494
version=0
cversion=1
aversion=0
ephemeralOwner=0x0
dataLength=0
numChildren=1
pzxid=8103
)
### back to session #1 lk.release() lock in first session
(CONNECTED) /> ls /resourceX/
(CONNECTED) />
The above shows creation times, who is waiting on the lock, modification
times, the owner.... Anyways I digress, if anyone really wants to know
more about zookeeper let me know or drop into the #zookeeper channel on
freenode (I'm one of the core maintainers of kazoo).
-Josh
Anyway, I'm all for the simplest possible solution. But, don't make it
_too_ simple.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
