Re: Using script-native drivers for Proton messenger bindings
On Wed, Nov 14, 2012 at 10:36 PM, Ted Ross wrote: > > On 11/14/2012 12:56 PM, Rafael Schloming wrote: > >> On Wed, Nov 14, 2012 at 6:37 PM, Ted Ross wrote: >> >> On 11/14/2012 11:49 AM, Rafael Schloming wrote: >>> >>> How does the packaging prevent this? 1) Develop the non-blocking aspects of the API. This might just involve a few additions to the current API or possibly adding a distinct non-blocking API which would be a layer underneath the current API. Either way, this non-blocking API then becomes the top half of the messenger engine. This is good, but I claim that the blocking/non-blocking of the >>> messenger >>> API is an orthogonal concern. The style of API used is related to the >>> programming model used in the application. I think that users will want >>> to >>> have the "native" driver and use the blocking messenger API. >>> >> >> Yes, but unless I'm missing something, the blocking calls will need to be >> "native" as well since they will need to call into the native driver to >> block, and so they will need to be built on top of a non blocking API. >> > > I'll use a concrete example. Assume we have a python program using the > current messaging API (which has three blocking calls). Assume further > that the driver is written in Python and uses a blocking mechanism > appropriate for the in-use threading library. Since the only place proton > *ever* blocks is in pn_driver_wait, and pn_driver_wait is written in > python, everything works correctly even though the program uses the > "blocking" messenger api. Yes, but those 3 blocking messenger calls are implemented in C. They do call pn_driver_wait, as you say, but the call the C pn_driver_wait. For what you're describing to work you'd need to somehow get them to call the python pn_driver_wait. That's why I think you'd need to re-implement the blocking messenger calls in the python part of the binding for this to work (so you're not calling python -> C -> python, or ruby -> C -> ruby). I think re-implementing the blocking messenger calls can be made quite trivial if the non blocking API is constructed appropriately, and I'm guessing easier than constructing the above call chain in each binding. --Rafael
Re: Using script-native drivers for Proton messenger bindings
On 11/14/2012 12:56 PM, Rafael Schloming wrote: On Wed, Nov 14, 2012 at 6:37 PM, Ted Ross wrote: On 11/14/2012 11:49 AM, Rafael Schloming wrote: How does the packaging prevent this? 1) Develop the non-blocking aspects of the API. This might just involve a few additions to the current API or possibly adding a distinct non-blocking API which would be a layer underneath the current API. Either way, this non-blocking API then becomes the top half of the messenger engine. This is good, but I claim that the blocking/non-blocking of the messenger API is an orthogonal concern. The style of API used is related to the programming model used in the application. I think that users will want to have the "native" driver and use the blocking messenger API. Yes, but unless I'm missing something, the blocking calls will need to be "native" as well since they will need to call into the native driver to block, and so they will need to be built on top of a non blocking API. I'll use a concrete example. Assume we have a python program using the current messaging API (which has three blocking calls). Assume further that the driver is written in Python and uses a blocking mechanism appropriate for the in-use threading library. Since the only place proton *ever* blocks is in pn_driver_wait, and pn_driver_wait is written in python, everything works correctly even though the program uses the "blocking" messenger api. -Ted
Re: Using script-native drivers for Proton messenger bindings
On Wed, Nov 14, 2012 at 6:37 PM, Ted Ross wrote: > > On 11/14/2012 11:49 AM, Rafael Schloming wrote: > >> >> How does the packaging prevent this? >> >> >>1) Develop the non-blocking aspects of the API. This might just >> involve a >> few additions to the current API or possibly adding a distinct >> non-blocking >> API which would be a layer underneath the current API. Either way, this >> non-blocking API then becomes the top half of the messenger engine. >> > > This is good, but I claim that the blocking/non-blocking of the messenger > API is an orthogonal concern. The style of API used is related to the > programming model used in the application. I think that users will want to > have the "native" driver and use the blocking messenger API. Yes, but unless I'm missing something, the blocking calls will need to be "native" as well since they will need to call into the native driver to block, and so they will need to be built on top of a non blocking API. --Rafael
Re: Using script-native drivers for Proton messenger bindings
On 11/14/2012 11:49 AM, Rafael Schloming wrote: How does the packaging prevent this? 1) Develop the non-blocking aspects of the API. This might just involve a few additions to the current API or possibly adding a distinct non-blocking API which would be a layer underneath the current API. Either way, this non-blocking API then becomes the top half of the messenger engine. This is good, but I claim that the blocking/non-blocking of the messenger API is an orthogonal concern. The style of API used is related to the programming model used in the application. I think that users will want to have the "native" driver and use the blocking messenger API. 2) Develop the bottom half API for the messenger engine. This would in part simply be an API that would provide access to a set of transport objects along with connection details. This would presumably also need to expose a way for the messenger to listen (or ask the driver to listen) on a given host/port also in order to support the full range of behaviour that messenger currently does. This latter part might be as simple as providing access to the list of subscriptions. --Rafael
Re: Using script-native drivers for Proton messenger bindings
On 11/14/2012 12:24 PM, Andrew Stitcher wrote: On Wed, 2012-11-14 at 17:49 +0100, Rafael Schloming wrote: ... How does the packaging prevent this? Not to put words in Ted's mouth, but I think what he is talking about here is that the messenger API code itself depends on the driver code. Andrew Yes, that is correct. If you are using just the engine API, you can ignore the built-in driver and provide your own. But for messenger, this is not possible. -Ted
Re: Using script-native drivers for Proton messenger bindings
On Wed, 2012-11-14 at 17:49 +0100, Rafael Schloming wrote: > ... > How does the packaging prevent this? Not to put words in Ted's mouth, but I think what he is talking about here is that the messenger API code itself depends on the driver code. Andrew
Re: Using script-native drivers for Proton messenger bindings
On Wed, Nov 14, 2012 at 4:54 PM, Ted Ross wrote: > One of the problems we've had for a long time with wrapped bindings is the > mismatch of threading models. For example, a Ruby program using an > extension module written in C/C++ will hang if a call into the extension > blocks. This is because the extension uses pthreads and the host program > uses "greenthreads" (i.e. some layered threading mechanism in the scripting > language). This is also an issue for Python (and probably every other > scripting language). > > The proper solution for proton/messenger (which necessarily has a few > blocking calls) is to provide (or allow the developer to provide) a driver > written in the scripting language using the same threading library that > their program uses. > > The proton code structure allows for such a strategy, but the packaging > does not. I think that the proton project should make the necessary > adjustments to encourage developers to contribute "native" drivers for > Ruby, Python-eventlet, etc. > How does the packaging prevent this? I'm interested in others' thoughts on this and what would be necessary to > make this happen. I think what's needed is to "engine-ify" Messenger, which involves two steps: 1) Develop the non-blocking aspects of the API. This might just involve a few additions to the current API or possibly adding a distinct non-blocking API which would be a layer underneath the current API. Either way, this non-blocking API then becomes the top half of the messenger engine. 2) Develop the bottom half API for the messenger engine. This would in part simply be an API that would provide access to a set of transport objects along with connection details. This would presumably also need to expose a way for the messenger to listen (or ask the driver to listen) on a given host/port also in order to support the full range of behaviour that messenger currently does. This latter part might be as simple as providing access to the list of subscriptions. --Rafael
Re: Using script-native drivers for Proton messenger bindings
On Wed, Nov 14, 2012 at 10:54:45AM -0500, Ted Ross wrote: > One of the problems we've had for a long time with wrapped bindings > is the mismatch of threading models. For example, a Ruby program > using an extension module written in C/C++ will hang if a call into > the extension blocks. This is because the extension uses pthreads > and the host program uses "greenthreads" (i.e. some layered > threading mechanism in the scripting language). This is also an > issue for Python (and probably every other scripting language). > > The proper solution for proton/messenger (which necessarily has a > few blocking calls) is to provide (or allow the developer to > provide) a driver written in the scripting language using the same > threading library that their program uses. > > The proton code structure allows for such a strategy, but the > packaging does not. I think that the proton project should make the > necessary adjustments to encourage developers to contribute "native" > drivers for Ruby, Python-eventlet, etc. > > I'm interested in others' thoughts on this and what would be > necessary to make this happen. I'm not sure I follow what is meant by "the packaging does not". Can you be more specific there? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpVIestdYpTE.pgp Description: PGP signature
