Hi, Eric

Thanks a lot for your comments.

On 06.03.15 06:21, Eric Windisch wrote:
On Wed, Mar 4, 2015 at 12:10 PM, ozamiatin <ozamia...@mirantis.com <mailto:ozamia...@mirantis.com>> wrote:


    By this e-mail I'd like to start a discussion about current zmq
    driver internal design problems I've found out.
    I wish to collect here all proposals and known issues. I hope this
    discussion will be continued on Liberty design summit.
    And hope it will drive our further zmq driver development efforts.

    ZMQ Driver issues list (I address all issues with # and references
    are in []):

    1. ZMQContext per socket (blocker is neutron improper usage of
    messaging via fork) [3]
    2. Too many different contexts.
        We have InternalContext used for ZmqProxy, RPCContext used in
    ZmqReactor, and ZmqListener.
        There is also zmq.Context which is zmq API entity. We need to
    consider a possibility to unify their usage over inheritance
    (maybe stick to RPCContext)
        or to hide them as internal entities in their modules (see
    refactoring #6)

The code, when I abandoned it, was moving toward fixing these issues, but for backwards compatibility was doing so in a staged fashion across the stable releases.

I agree it's pretty bad. Fixing this now, with the driver in a less stable state should be easier, as maintaining compatibility is of less importance.

    3. Topic related code everywhere. We have no topic entity. It is
    all string operations.
        We need some topics management entity and topic itself as an
    entity (not a string).
        It causes issues like [4], [5]. (I'm already working on it).
        There was a spec related [7].

Good! It's ugly. I had proposed a patch at one point, but I believe the decision was that it was better and cleaner to move toward the oslo.messaging abstraction as we solve the topic issue. Now that oslo.messaging exists, I agree it's well past time to fix this particular ugliness.

    4. Manual implementation of messaging patterns.
       Now we can observe poor usage of zmq features in zmq driver.
    Almost everything is implemented over PUSH/PULL.

        4.1 Manual polling - use zmq.Poller (listening and replying
    for multiple sockets)
        4.2 Manual request/reply implementation for call [1].
            Using of REQ/REP (ROUTER/DEALER) socket solves many
    issues. A lot of code may be reduced.
        4.3 Timeouts waiting

There are very specific reasons for the use of PUSH/PULL. I'm firmly of the belief that it's the only viable solution for an OpenStack RPC driver. This has to do with how asynchronous programming in Python is performed, with how edge-triggered versus level-triggered events are processed, and general state management for REQ/REP sockets.

I could be proven wrong, but I burned quite a bit of time in the beginning of the ZMQ effort looking at REQ/REP before realizing that PUSH/PULL was the only reasonable solution. Granted, this was over 3 years ago, so I would not be too surprised if my assumptions are no longer valid.

I agree that REQ/REP is very limited because of their synchronous nature and 1-to-1 connection possibility. But there are ROUTER/DEALER proxy sockets recommended to use with REQ/REP to compose 1-to-N and N-to-N asynchronous patterns. I'm in research of that now and I didn't finally decide yet. When everything will be clear for me I'll come with a spec on that.

    5. Add possibility to work without eventlet [2]. #4.1 is also
    related here, we can reuse many of the implemented solutions
       like zmq.Poller over asynchronous sockets in one separate
    thread (instead of spawning on each new socket).
       I will update the spec [2] on that.

Great. This was one of the motivations behind oslo.messaging and it would be great to see this come to fruition.

    6. Put all zmq driver related stuff (matchmakers, most classes
    from zmq_impl) into a separate package.
       Don't keep all classes (ZmqClient, ZmqProxy, Topics management,
    ZmqListener, ZmqSocket, ZmqReactor)
       in one impl_zmq.py module.

Seems fine. In fact, I think a lot of code could be shared with an AMQP v1 driver...

I'll check what can be shared. Actually I didn't yet dig into AMQP v1 driver enough.

    7. Need more technical documentation on the driver like [6].
       I'm willing to prepare a current driver architecture overview
    with some graphics UML charts, and to continue discuss the driver

Documentation has always been a sore point. +2
Eric Windisch

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Oleksii Zamiatin
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Reply via email to