Re: [openstack-dev] [oslo][kite] oslo.messaging changes for message security
On 11/14/2014 08:38 AM, Doug Hellmann wrote: > > On Nov 13, 2014, at 8:47 PM, Jamie Lennox wrote: > >> Hi all, >> >> To implement kite we need the ability to sign and encrypt the message and the >> message data. This needs to happen at a very low level in the oslo.messaging >> stack. The existing message security review >> (https://review.openstack.org/#/c/109806/) isn't going to be sufficient. It >> allows us to sign/encrypt only the message data ignoring the information in >> the >> context and not allowing us to sign the message as a whole. It would also >> intercept and sign notifications which is not something that kite can do. >> >> Mostly this is an issue of how the oslo.messaging library is constructed. The >> choice of how data is serialized for transmission (including things like how >> you arrange context and message data in the payload) is handled individually >> by >> the driver layer rather than in a common higher location. All the drivers use >> the same helper functions for this and so it isn't a problem in practice. >> >> Essentially I need a stateful serializing/deserializing object (I need to >> store >> keys and hold things like a connection to the kite server) that either >> extends >> or replaces oslo.messaging._drivers.common.serialize_msg and deserialize_msg >> and their exception counterparts. >> >> There are a couple of ways I can see to do what I need: >> >> 1. Kite becomes a more integral part of oslo.messaging and the marshalling >> and >> verification code becomes part of the existing RPC path. This is how it was >> initially proposed, it does not provide a good story for future or >> alternative >> implementations. Oslo.messaging would either have a dependency on kiteclient, >> implement its own ways of talking to the server, or have some hack that >> imports >> kiteclient if available. >> >> 2. Essentially I add a global object loaded from conf to the existing common >> RPC file. Pro: The existing drivers continue to work as today, Con: global >> state held by a library. However given the way oslo messaging works i'm not >> really sure how much of a problem this is. We typically load transport from a >> predefined location in the conf file and we're not really in a situation >> where >> you might want to construct different transports with different parameters in >> the same project. >> >> 3. I create a protocol object out of the RPC code that kite can subclass and >> the protocol can be chosen by CONF when the transport/driver is created. This >> still touches a lot of places as the protocol object would need to be passed >> to >> all messages, consumers etc. It involves changing the interface of the >> drivers >> to accept this new object and changes in each of the drivers to work with the >> new protocol object rather than the existing helpers. >> >> 4. As the last option requires changing the driver interface anyway we try >> and >> correct the driver interfaces completely. The driver send and receive >> functions >> that currently accept a context and args parameters should only accept a >> generic object/string consisting of already marshalled data. The code that >> handles serializing and deserializing gets moved to a higher level and kite >> would be pluggable there with the current RPC being default. >> >> None of these options involve changing the public facing interfaces nor the >> messages emitted on the wire (when kite is not used). >> >> I've been playing a little with option 3 and I don't think it's worth it. >> There >> is a lot of code change and additional object passing that I don't think >> improves the library in general. >> >> Before I go too far down the path with option 4 I'd like to hear the thoughts >> of people more familiar with the library. >> >> Is there a reason that the drivers currently handle marshalling rather than >> the >> RPC layer? > > It may have been an artifact of the evolution of that code, but I seem to > remember at some point that one of the drivers had a limitation either in the > byte-values allowed or the number of bytes allowed in a message. They all > seem to be doing roughly the same thing to construct the messages now, > though, so I’m not sure if that’s really true. I believe you're thinking of the issue related to this: https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_qpid.py#L346 I know when I originally made that change concerns were raised about how it would impact secure messaging, but at the time the conclusion was that it would be fine. I _think_ that should still be the case since it only affects the wire format of Qpid messages. Either Qpid directly serializes the message dict, or it dumps it to json and serializes the resulting string. Either way the content of the message doesn't change. Changing the interface to pass in a serialized string would actually simplify things. That said, there were also backwards compatibility concerns with this change
Re: [openstack-dev] [oslo][kite] oslo.messaging changes for message security
On Nov 13, 2014, at 8:47 PM, Jamie Lennox wrote: > Hi all, > > To implement kite we need the ability to sign and encrypt the message and the > message data. This needs to happen at a very low level in the oslo.messaging > stack. The existing message security review > (https://review.openstack.org/#/c/109806/) isn't going to be sufficient. It > allows us to sign/encrypt only the message data ignoring the information in > the > context and not allowing us to sign the message as a whole. It would also > intercept and sign notifications which is not something that kite can do. > > Mostly this is an issue of how the oslo.messaging library is constructed. The > choice of how data is serialized for transmission (including things like how > you arrange context and message data in the payload) is handled individually > by > the driver layer rather than in a common higher location. All the drivers use > the same helper functions for this and so it isn't a problem in practice. > > Essentially I need a stateful serializing/deserializing object (I need to > store > keys and hold things like a connection to the kite server) that either extends > or replaces oslo.messaging._drivers.common.serialize_msg and deserialize_msg > and their exception counterparts. > > There are a couple of ways I can see to do what I need: > > 1. Kite becomes a more integral part of oslo.messaging and the marshalling and > verification code becomes part of the existing RPC path. This is how it was > initially proposed, it does not provide a good story for future or alternative > implementations. Oslo.messaging would either have a dependency on kiteclient, > implement its own ways of talking to the server, or have some hack that > imports > kiteclient if available. > > 2. Essentially I add a global object loaded from conf to the existing common > RPC file. Pro: The existing drivers continue to work as today, Con: global > state held by a library. However given the way oslo messaging works i'm not > really sure how much of a problem this is. We typically load transport from a > predefined location in the conf file and we're not really in a situation where > you might want to construct different transports with different parameters in > the same project. > > 3. I create a protocol object out of the RPC code that kite can subclass and > the protocol can be chosen by CONF when the transport/driver is created. This > still touches a lot of places as the protocol object would need to be passed > to > all messages, consumers etc. It involves changing the interface of the drivers > to accept this new object and changes in each of the drivers to work with the > new protocol object rather than the existing helpers. > > 4. As the last option requires changing the driver interface anyway we try and > correct the driver interfaces completely. The driver send and receive > functions > that currently accept a context and args parameters should only accept a > generic object/string consisting of already marshalled data. The code that > handles serializing and deserializing gets moved to a higher level and kite > would be pluggable there with the current RPC being default. > > None of these options involve changing the public facing interfaces nor the > messages emitted on the wire (when kite is not used). > > I've been playing a little with option 3 and I don't think it's worth it. > There > is a lot of code change and additional object passing that I don't think > improves the library in general. > > Before I go too far down the path with option 4 I'd like to hear the thoughts > of people more familiar with the library. > > Is there a reason that the drivers currently handle marshalling rather than > the > RPC layer? It may have been an artifact of the evolution of that code, but I seem to remember at some point that one of the drivers had a limitation either in the byte-values allowed or the number of bytes allowed in a message. They all seem to be doing roughly the same thing to construct the messages now, though, so I’m not sure if that’s really true. > > I know there is ongoing talk about evolving the oslo.messaging library, I > unfortunately didn't make it to the sessions at summit. Has this problem been > raised? How would it affect those talks? Security did come up, but I don’t see where we recorded any decisions in the etherpad: https://etherpad.openstack.org/p/kilo-oslo-oslo.messaging Our bigger concern was lack of reviewers for the library, which will make any large-scale refactoring problematic. > > Is there explicit/implicit support for out of tree drivers that would disallow > changing these interfaces? There’s implicit support, but we haven’t formally called those APIs stable. > > Does anyone have alternative ideas on how to organize the library for message > security? > > Thanks for the help. > > > Jamie > > ___ > OpenStack-dev mailing list > OpenStack-dev@lis
