Re: Review Request 12289: QPID-4327: HA added PrimaryTransactionObserver.

[Alan Conway]

> On July 8, 2013, 12:10 p.m., Gordon Sim wrote:
> > /trunk/qpid/cpp/src/qpid/broker/TransactionObserver.h, line 48
> > 
> >
> > It's really the enqueue and dequeue operations that are transactional. 
> > It might be better to define this interface in terms of those.
> > 
> > It would also seem - on the surface at least - a little more consistent 
> > to have a set of TransactionObservers at the broker level (along side those 
> > for queues and connections), rather than a factory that is then used to 
> > attach different observer instances to transaction buffers.
> > 
> > E.g.
> > 
> > class TransactionObserver {
> >   void enqueued(TransactionContext&, const Message&, const 
> > boost::shared_ptr) = 0;
> >   void dequeued(TransactionContext&, const Message&, const 
> > boost::shared_ptr) = 0;
> >   void started(TransactionContext&) = 0;
> >   void prepared(TransactionContext&) = 0;
> >   void committed(TransactionContext&) = 0;
> >   void rolledback(TransactionContext&) = 0;
> > }
> > 
> > Perhaps adding a 'std::string getId()' method to TransactionContext if 
> > needed.
> > 
> > Though 1.0 doesn't yet support transactions, DeliveryRecords is an 0-10 
> > specific class so the interface you have here would need to change for 1.0 
> > anyway I suspect.
> 
> Alan Conway wrote:
> Regarding "It would also seem - on the surface at least - a little more 
> consistent to have a set of TransactionObservers at the broker level (along 
> side those for queues and connections), rather than a factory that is then 
> used to attach different observer instances to transaction buffers."
> 
> The way I have it is more consistent with e.g. QueueObserver - an 
> individual observer for each queue. It avoids another layer of maps by going 
> direct to the correct observer. There is no QueueObserverFactory because a 
> plugin can register a ConfigObserver and be informed of queue creation, so 
> can attach it's observer. There isn't such an intercept point for 
> transactions (that I know of) so I added the factory. Maybe there's a more 
> consistent way to add such a point - ConfigObserver doesn't seem the right 
> place though. Maybe ConfigObserver could be renamed to something more generic 
> (BrokerEventObserver? yuck.) 
> 
> Regarding your interface above I'll rework things on those lines...
> 
> Gordon Sim wrote:
> What about having a broker level set of observers, but adding those to 
> each TransactionContext/TransactionBuffer and allowing code to attach 
> additional observers to specific TransactionContexts/TransactionBuffers if 
> desired?

I think the changes in diff 4 address these issues: 
- TranactionObserver doesn't use DeliveryRecords, it uses queue position and 
replication SequenceNumbers to ID a message.
- ConfigObserver has been renamed BrokerObserver and provides startTx and 
startDtx methods to observe creation of a transaction.
- Plugins that wish to can add a TransactionObserver to the transaction to see 
accept, publish, prepare, commit and rollback.


- Alan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12289/#review22810
---


On July 10, 2013, 10:28 p.m., Alan Conway wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12289/
> ---
> 
> (Updated July 10, 2013, 10:28 p.m.)
> 
> 
> Review request for qpid, Andrew Stitcher and Gordon Sim.
> 
> 
> Repository: qpid
> 
> 
> Description
> ---
> 
> QPID-4327: HA added PrimaryTransactionObserver.
> 
> Initial implementation only logs some trace messages.
> 
> QPID-4327: TransactionObserver interface.
> 
> Added TransactionObserver interface, called at each point in a transaction's
> lifecycle. Currently only a single observer can be associated with a
> transaction.
> 
> Added startTx, startDtx to BrokerObserver so plugins can observe transactions
> starting and set a TransactionObserver.
> 
> QPID-4327: Renamed ConfigurationObserver as BrokerObserver.
> 
> This class really was intended as a observer for broker-level events which
> includes configuration but may in future include other non-configuration 
> events
> such as transactions.
> 
> QPID-4327: Refactor to simplify TxAccept.
> 
> Removed un-necessary RangeOps layers.
> 
> 
> Diffs
> -
> 
>   /trunk/qpid/cpp/src/CMakeLists.txt 1501768 
>   /trunk/qpid/cpp/src/Makefile.am 1501768 
>   /trunk/qpid/cpp/src/ha.mk 1501768 
>   /trunk/qpid/cpp/src/qpid/broker/Broker.h 1501768 
>   /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1501768 
>   /trunk/qpid/cpp/src/qpid/broker/BrokerObserver.h PRE-CREATION 
>   /

Re: Review Request 12289: QPID-4327: HA added PrimaryTransactionObserver.

[Alan Conway]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12289/
---

(Updated July 10, 2013, 10:28 p.m.)


Review request for qpid, Andrew Stitcher and Gordon Sim.


Changes
---

Modified TransactionObserver interface to use only queue and replication 
sequence numbers, not DeliveryRecords.


Repository: qpid


Description (updated)
---

QPID-4327: HA added PrimaryTransactionObserver.

Initial implementation only logs some trace messages.

QPID-4327: TransactionObserver interface.

Added TransactionObserver interface, called at each point in a transaction's
lifecycle. Currently only a single observer can be associated with a
transaction.

Added startTx, startDtx to BrokerObserver so plugins can observe transactions
starting and set a TransactionObserver.

QPID-4327: Renamed ConfigurationObserver as BrokerObserver.

This class really was intended as a observer for broker-level events which
includes configuration but may in future include other non-configuration events
such as transactions.

QPID-4327: Refactor to simplify TxAccept.

Removed un-necessary RangeOps layers.


Diffs (updated)
-

  /trunk/qpid/cpp/src/CMakeLists.txt 1501768 
  /trunk/qpid/cpp/src/Makefile.am 1501768 
  /trunk/qpid/cpp/src/ha.mk 1501768 
  /trunk/qpid/cpp/src/qpid/broker/Broker.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/BrokerObserver.h PRE-CREATION 
  /trunk/qpid/cpp/src/qpid/broker/BrokerObservers.h PRE-CREATION 
  /trunk/qpid/cpp/src/qpid/broker/ConfigurationObserver.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/ConfigurationObservers.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/DtxAck.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/ExchangeRegistry.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/Queue.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/QueueRegistry.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/RecoveredDequeue.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/RecoveredEnqueue.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/SemanticState.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/TransactionObserver.h PRE-CREATION 
  /trunk/qpid/cpp/src/qpid/broker/TxAccept.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/TxAccept.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/TxBuffer.h 1501768 
  /trunk/qpid/cpp/src/qpid/broker/TxBuffer.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/broker/TxOp.h 1501768 
  /trunk/qpid/cpp/src/qpid/ha/HaBroker.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/ha/Primary.h 1501768 
  /trunk/qpid/cpp/src/qpid/ha/Primary.cpp 1501768 
  /trunk/qpid/cpp/src/qpid/ha/PrimaryTransactionObserver.h PRE-CREATION 
  /trunk/qpid/cpp/src/qpid/ha/PrimaryTransactionObserver.cpp PRE-CREATION 
  /trunk/qpid/cpp/src/qpid/ha/QueueSnapshots.h 1501768 
  /trunk/qpid/cpp/src/qpid/ha/RemoteBackup.h 1501768 
  /trunk/qpid/cpp/src/qpid/ha/RemoteBackup.cpp 1501768 
  /trunk/qpid/cpp/src/tests/CMakeLists.txt 1501768 
  /trunk/qpid/cpp/src/tests/Makefile.am 1501768 
  /trunk/qpid/cpp/src/tests/TransactionObserverTest.cpp PRE-CREATION 
  /trunk/qpid/cpp/src/tests/TxMocks.h 1501768 
  /trunk/qpid/cpp/src/tests/brokertest.py 1501768 
  /trunk/qpid/cpp/src/tests/ha_tests.py 1501768 
  /trunk/qpid/cpp/src/tests/test_env.sh.in 1501768 
  /trunk/qpid/cpp/src/tests/test_tools.h 1501768 

Diff: https://reviews.apache.org/r/12289/diff/


Testing
---

It compiles


Thanks,

Alan Conway

Re: Swigged python package name...

[Darryl L. Pierce]

On Wed, Jul 10, 2013 at 09:23:01PM +0100, Gordon Sim wrote:
> Right now the python-qpid fedora package, to use a concrete example,
> installs into sitepackages (qpid/messaging.py etc). Are you
> proposing to change that?

No, I think I've combined how Ruby and Python work in this case. With
Ruby gems you can have multiple versions of a gem installed and require
only the version you need in your app. When skimming over the Python
docs on eggs I mistook some of what it said in there as being the
equivalent to what Ruby can do.

> >Not so much nicer as more consistent across the supported set of
> >languages. In Ruby we have Qpid::Messaging, in Perl it's
> >qpid::messaging, in C++ it's qpid::messaging. It would be nice to have
> >Python also be qpid.messaging
> 
> It is! And has been long before the Ruby or Perl equivalents even existed.
> 
> The issue is that we now have two *different* messaging libraries
> for python...
> 
> >and not something different, to keep it as
> >intuitive as possible.
> 
> ... I think trying to force the two into the same namespace is far
> from intuitive.

I think your previous suggestion to have a compatibility import might be
the better way to go. How about something like this:

1. move the pure Python bindings to the qpid.pure package
2. leave the Swig bindings in cqpid for the time being
3. create a new module that lives in qpid.messaging and which will
import either qpid.pure or cqpid depending on a setting from the
application importing them?

This, I think, comes close to what you were proposing.

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/



pgpzF9vh4ovj8.pgp
Description: PGP signature

[jira] [Commented] (QPID-4984) Linear File Allocation for the Linux AIO Store

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/QPID-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13705111#comment-13705111
 ] 

ASF subversion and git services commented on QPID-4984:
---

Commit 1502018 from [~kpvdr]
[ https://svn.apache.org/r1502018 ]

QPID-4984: Continuing work on the linearstoreutils lib

> Linear File Allocation for the Linux AIO Store
> --
>
> Key: QPID-4984
> URL: https://issues.apache.org/jira/browse/QPID-4984
> Project: Qpid
>  Issue Type: Improvement
>  Components: C++ Broker
>Reporter: Ted Ross
>
> This Jira is for a proposed change to the AIO "legacy" store in the Qpid C++ 
> Broker.
> Rather than pre-allocating store files in a fixed-size ring buffer (per 
> queue), the linear store allocates files to queue journals on an as-needed 
> basis, freeing them to a free-pool when no longer needed (i.e. all stored 
> messages have been dequeued).
> The development is ongoing in a branch called "linearstore".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Re: Swigged python package name...

[Gordon Sim]

On 07/10/2013 06:24 PM, Darryl L. Pierce wrote:

On Wed, Jul 10, 2013 at 05:59:43PM +0100, Gordon Sim wrote:

On 07/10/2013 04:51 PM, Darryl L. Pierce wrote:

On Wed, Jul 10, 2013 at 03:54:44PM +0100, Gordon Sim wrote:

On 07/10/2013 01:45 PM, Darryl L. Pierce wrote:

On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:

I would think that downstream repos would package one or the other
but not both.  It would be nice if downstream, it was called
qpid.messagingregardless of the implementation used.


Agreed.


I disagree. I can see cases where it would be beneficial to have
both installed. E.g. you have existing applications that you don't
want to switch (if it ain't broke etc), but you also want to try 1.0
(or just the c++ impl) for some new use cases.


I can see someone wanting to try out 1.0, but they'll still eventually
have to make a decision, right?


The point is that the decision taken for one use case shouldn't
necessarily force the same choice across the board.


+1


Regarding not wanting to refactor existing apps that work, etc., that
can be handled by the administrators by putting the old Python bindings
in a separate location from the newer bindings by naming the eggs
differently; i.e., python-qpid is the old API set while
python-qpid_messaging is the newer set. Using pkg_resources the app
developer can have both APIs living on the same machine, using the same
packages, without collision.


I think its important to keep the different notions of package
separate. On the one hand there is the python namespace. On the
other hand there is the platform specific installation approach in
use (e.g. rpms or whatever).

In the case of the latter, I think it is decidedly odd to e.g. yum
install python-qpid, yum install python-qpid_messaging and have the
installation of the second package overwrite artefacts of the first
one (even assuming this doesn't totally break things).


That wouldn't be the case given that the two can be installed using
separate names, both for the RPM itself also the underlying egg file
shared via other Python methods. The was the collision avoidance I was
talking about above; i.e., installing python-qpid (which ships the qpid
egg's contents) would live in directories named python-qpid-VEV, while
installing python-qpid_messaging (whose foundation is a different egg
named qpid_messaging) would live in directories named
python-qpid_messaging-VER. Both very sheltered from each other.


Right now the python-qpid fedora package, to use a concrete example, 
installs into sitepackages (qpid/messaging.py etc). Are you proposing to 
change that?



The app developer can then use something similar to your suggestion
about conditional imports and, instead of catching a raised exception to
pick a different package, could use:

pkg_resource.require("qpid==VER")

or:

pkg_resource.require("qpid_messaging=VER")

to import the desired implementation of qpid.messaging to get the APIs
they want.


Those statements go in the application code? If so how is that better 
than simply importing from one or the other namespace?


Is package_resource not a distribution/deployment tool rather than 
something applications would directly invoke in their code?



I think it ought to be a goal to have the Swigged bindings be a
drop-in replacement for the pure Python bindings. The users shouldn't care
which implementation they use.


Some won't, some will.

Though the APIs are getting very close, there are still - and I
believe will always be - behavioural differences. Even apart from
those, we have had users using eventlet and monkey-patching and they
are going to care which implementation they have.


I'm not sure how to respond to that. The behavioral differences will
result in large differences in how the APIs are expected to work?


There are differences, and those differences could mean that an
application that works with one does not work with the other.

To state the obvious, the swigged implementation is a different
library offering (broadly) the same API. It is not an evolution of
the existing implementation.

It is clearer to give these two different implementations two
different names.

I can certainly also see cases where users don't want to explicitly
choose one or the other and want their code to run against either
without change (I have that need myself as explained in an earlier
mail). However I think that can be done by a simple (5 line) module
with a third name.

That way users can pick one, or the other, or explicitly decide they
don't care and want to pick whichever is available based on path.


Understood. See what I wrote above as another way of approaching this.


I don't really understand what you wrote above.


My main issue is looking at the pure Python bindings and limiting what
feels, to me at least, the package name for the swigged bindings due to
that.  I think we can overcome that without having extra packages on top
of them.


I don't really understand what you are saying

[jira] [Commented] (QPID-4984) Linear File Allocation for the Linux AIO Store

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/QPID-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704984#comment-13704984
 ] 

ASF subversion and git services commented on QPID-4984:
---

Commit 1501945 from [~tedross]
[ https://svn.apache.org/r1501945 ]

QPID-4984 - Added a utilities library for linearstore and its tools.

> Linear File Allocation for the Linux AIO Store
> --
>
> Key: QPID-4984
> URL: https://issues.apache.org/jira/browse/QPID-4984
> Project: Qpid
>  Issue Type: Improvement
>  Components: C++ Broker
>Reporter: Ted Ross
>
> This Jira is for a proposed change to the AIO "legacy" store in the Qpid C++ 
> Broker.
> Rather than pre-allocating store files in a fixed-size ring buffer (per 
> queue), the linear store allocates files to queue journals on an as-needed 
> basis, freeing them to a free-pool when no longer needed (i.e. all stored 
> messages have been dequeued).
> The development is ongoing in a branch called "linearstore".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Created] (QPID-4984) Linear File Allocation for the Linux AIO Store

[Ted Ross (JIRA)]

Ted Ross created QPID-4984:
--

 Summary: Linear File Allocation for the Linux AIO Store
 Key: QPID-4984
 URL: https://issues.apache.org/jira/browse/QPID-4984
 Project: Qpid
  Issue Type: Improvement
  Components: C++ Broker
Reporter: Ted Ross


This Jira is for a proposed change to the AIO "legacy" store in the Qpid C++ 
Broker.

Rather than pre-allocating store files in a fixed-size ring buffer (per queue), 
the linear store allocates files to queue journals on an as-needed basis, 
freeing them to a free-pool when no longer needed (i.e. all stored messages 
have been dequeued).

The development is ongoing in a branch called "linearstore".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Re: Swigged python package name...

[Darryl L. Pierce]

On Wed, Jul 10, 2013 at 05:59:43PM +0100, Gordon Sim wrote:
> On 07/10/2013 04:51 PM, Darryl L. Pierce wrote:
> >On Wed, Jul 10, 2013 at 03:54:44PM +0100, Gordon Sim wrote:
> >>On 07/10/2013 01:45 PM, Darryl L. Pierce wrote:
> >>>On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:
> I would think that downstream repos would package one or the other
> but not both.  It would be nice if downstream, it was called
> qpid.messagingregardless of the implementation used.
> >>>
> >>>Agreed.
> >>
> >>I disagree. I can see cases where it would be beneficial to have
> >>both installed. E.g. you have existing applications that you don't
> >>want to switch (if it ain't broke etc), but you also want to try 1.0
> >>(or just the c++ impl) for some new use cases.
> >
> >I can see someone wanting to try out 1.0, but they'll still eventually
> >have to make a decision, right?
> 
> The point is that the decision taken for one use case shouldn't
> necessarily force the same choice across the board.

+1

> >Regarding not wanting to refactor existing apps that work, etc., that
> >can be handled by the administrators by putting the old Python bindings
> >in a separate location from the newer bindings by naming the eggs
> >differently; i.e., python-qpid is the old API set while
> >python-qpid_messaging is the newer set. Using pkg_resources the app
> >developer can have both APIs living on the same machine, using the same
> >packages, without collision.
> 
> I think its important to keep the different notions of package
> separate. On the one hand there is the python namespace. On the
> other hand there is the platform specific installation approach in
> use (e.g. rpms or whatever).
> 
> In the case of the latter, I think it is decidedly odd to e.g. yum
> install python-qpid, yum install python-qpid_messaging and have the
> installation of the second package overwrite artefacts of the first
> one (even assuming this doesn't totally break things).

That wouldn't be the case given that the two can be installed using
separate names, both for the RPM itself also the underlying egg file
shared via other Python methods. The was the collision avoidance I was
talking about above; i.e., installing python-qpid (which ships the qpid
egg's contents) would live in directories named python-qpid-VEV, while
installing python-qpid_messaging (whose foundation is a different egg
named qpid_messaging) would live in directories named
python-qpid_messaging-VER. Both very sheltered from each other.

The app developer can then use something similar to your suggestion
about conditional imports and, instead of catching a raised exception to
pick a different package, could use:

pkg_resource.require("qpid==VER")

or:

pkg_resource.require("qpid_messaging=VER")

to import the desired implementation of qpid.messaging to get the APIs
they want.

> >>>I think it ought to be a goal to have the Swigged bindings be a
> >>>drop-in replacement for the pure Python bindings. The users shouldn't care
> >>>which implementation they use.
> >>
> >>Some won't, some will.
> >>
> >>Though the APIs are getting very close, there are still - and I
> >>believe will always be - behavioural differences. Even apart from
> >>those, we have had users using eventlet and monkey-patching and they
> >>are going to care which implementation they have.
> >
> >I'm not sure how to respond to that. The behavioral differences will
> >result in large differences in how the APIs are expected to work?
> 
> There are differences, and those differences could mean that an
> application that works with one does not work with the other.
> 
> To state the obvious, the swigged implementation is a different
> library offering (broadly) the same API. It is not an evolution of
> the existing implementation.
> 
> It is clearer to give these two different implementations two
> different names.
> 
> I can certainly also see cases where users don't want to explicitly
> choose one or the other and want their code to run against either
> without change (I have that need myself as explained in an earlier
> mail). However I think that can be done by a simple (5 line) module
> with a third name.
>
> That way users can pick one, or the other, or explicitly decide they
> don't care and want to pick whichever is available based on path.

Understood. See what I wrote above as another way of approaching this.

> >My main issue is looking at the pure Python bindings and limiting what
> >feels, to me at least, the package name for the swigged bindings due to
> >that.  I think we can overcome that without having extra packages on top
> >of them.
> 
> I don't really understand what you are saying here. Are you saying
> you want the name qpid.messaging for the swigged implementation
> because it's a nicer name than alternatives?

Not so much nicer as more consistent across the supported set of
languages. In Ruby we have Qpid::Messaging, in Perl it's
qpid::messaging, in C++ it's qpid::messaging. It would be n

Re: Swigged python package name...

[Gordon Sim]

On 07/10/2013 04:51 PM, Darryl L. Pierce wrote:

On Wed, Jul 10, 2013 at 03:54:44PM +0100, Gordon Sim wrote:

On 07/10/2013 01:45 PM, Darryl L. Pierce wrote:

On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:

I would think that downstream repos would package one or the other
but not both.  It would be nice if downstream, it was called
qpid.messagingregardless of the implementation used.


Agreed.


I disagree. I can see cases where it would be beneficial to have
both installed. E.g. you have existing applications that you don't
want to switch (if it ain't broke etc), but you also want to try 1.0
(or just the c++ impl) for some new use cases.


I can see someone wanting to try out 1.0, but they'll still eventually
have to make a decision, right?


The point is that the decision taken for one use case shouldn't 
necessarily force the same choice across the board.


[...]

Regarding not wanting to refactor existing apps that work, etc., that
can be handled by the administrators by putting the old Python bindings
in a separate location from the newer bindings by naming the eggs
differently; i.e., python-qpid is the old API set while
python-qpid_messaging is the newer set. Using pkg_resources the app
developer can have both APIs living on the same machine, using the same
packages, without collision.


I think its important to keep the different notions of package separate. 
On the one hand there is the python namespace. On the other hand there 
is the platform specific installation approach in use (e.g. rpms or 
whatever).


In the case of the latter, I think it is decidedly odd to e.g. yum 
install python-qpid, yum install python-qpid_messaging and have the 
installation of the second package overwrite artefacts of the first one 
(even assuming this doesn't totally break things).



I think it ought to be a goal to have the Swigged bindings be a
drop-in replacement for the pure Python bindings. The users shouldn't care
which implementation they use.


Some won't, some will.

Though the APIs are getting very close, there are still - and I
believe will always be - behavioural differences. Even apart from
those, we have had users using eventlet and monkey-patching and they
are going to care which implementation they have.


I'm not sure how to respond to that. The behavioral differences will
result in large differences in how the APIs are expected to work?


There are differences, and those differences could mean that an 
application that works with one does not work with the other.


To state the obvious, the swigged implementation is a different library 
offering (broadly) the same API. It is not an evolution of the existing 
implementation.


It is clearer to give these two different implementations two different 
names.


I can certainly also see cases where users don't want to explicitly 
choose one or the other and want their code to run against either 
without change (I have that need myself as explained in an earlier 
mail). However I think that can be done by a simple (5 line) module with 
a third name.


That way users can pick one, or the other, or explicitly decide they 
don't care and want to pick whichever is available based on path.


[...]

My main issue is looking at the pure Python bindings and limiting what
feels, to me at least, the package name for the swigged bindings due to
that.  I think we can overcome that without having extra packages on top
of them.


I don't really understand what you are saying here. Are you saying you 
want the name qpid.messaging for the swigged implementation because it's 
a nicer name than alternatives?


-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Re: Swigged python package name...

[Darryl L. Pierce]

On Wed, Jul 10, 2013 at 03:54:44PM +0100, Gordon Sim wrote:
> On 07/10/2013 01:45 PM, Darryl L. Pierce wrote:
> >On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:
> >>I would think that downstream repos would package one or the other
> >>but not both.  It would be nice if downstream, it was called
> >>qpid.messagingregardless of the implementation used.
> >
> >Agreed.
> 
> I disagree. I can see cases where it would be beneficial to have
> both installed. E.g. you have existing applications that you don't
> want to switch (if it ain't broke etc), but you also want to try 1.0
> (or just the c++ impl) for some new use cases.

I can see someone wanting to try out 1.0, but they'll still eventually
have to make a decision, right? If they go with the swigged APIs then
they should be able to drop in the updated APIs with as little impact on
their code as possible, IMO. 

Regarding not wanting to refactor existing apps that work, etc., that
can be handled by the administrators by putting the old Python bindings
in a separate location from the newer bindings by naming the eggs
differently; i.e., python-qpid is the old API set while
python-qpid_messaging is the newer set. Using pkg_resources the app
developer can have both APIs living on the same machine, using the same
packages, without collision.

> >I think it ought to be a goal to have the Swigged bindings be a
> >drop-in replacement for the pure Python bindings. The users shouldn't care
> >which implementation they use.
> 
> Some won't, some will.
> 
> Though the APIs are getting very close, there are still - and I
> believe will always be - behavioural differences. Even apart from
> those, we have had users using eventlet and monkey-patching and they
> are going to care which implementation they have.

I'm not sure how to respond to that. The behavioral differences will
result in large differences in how the APIs are expected to work?

For the latter case, I see what you're saying. It'll be an issue, and
I'd love to get some of them involved in discussions regarding this.

> >WRT the name space, I'm still wanting to keep it as qpid.messaging
> >since, in the long run, that would require developers to not change
> >their imports and usage.
> 
> I want to keep qpid.messaging for the existing python client, have
> the swigged implementation available through another name and
> optionally have a third that simply tries one and falls back on the
> other based on what is available (for those use cases that want to
> be able to switch between the two).

My main issue is looking at the pure Python bindings and limiting what
feels, to me at least, the package name for the swigged bindings due to
that.  I think we can overcome that without having extra packages on top
of them.

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/



pgp1xqkryeOUz.pgp
Description: PGP signature

Re: Swigged python package name...

[Gordon Sim]

On 07/10/2013 01:45 PM, Darryl L. Pierce wrote:

On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:

I would think that downstream repos would package one or the other
but not both.  It would be nice if downstream, it was called
qpid.messagingregardless of the implementation used.


Agreed.


I disagree. I can see cases where it would be beneficial to have both 
installed. E.g. you have existing applications that you don't want to 
switch (if it ain't broke etc), but you also want to try 1.0 (or just 
the c++ impl) for some new use cases.



I think it ought to be a goal to have the Swigged bindings be a
drop-in replacement for the pure Python bindings. The users shouldn't care
which implementation they use.


Some won't, some will.

Though the APIs are getting very close, there are still - and I believe 
will always be - behavioural differences. Even apart from those, we have 
had users using eventlet and monkey-patching and they are going to care 
which implementation they have.


[...]

WRT the name space, I'm still wanting to keep it as qpid.messaging
since, in the long run, that would require developers to not change
their imports and usage.


I want to keep qpid.messaging for the existing python client, have the 
swigged implementation available through another name and optionally 
have a third that simply tries one and falls back on the other based on 
what is available (for those use cases that want to be able to switch 
between the two).


-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Resolved] (QPID-4976) [AMQP 1.0] support standard lifetime policies

[Gordon Sim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/QPID-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gordon Sim resolved QPID-4976.
--

Resolution: Fixed

> [AMQP 1.0] support standard lifetime policies
> -
>
> Key: QPID-4976
> URL: https://issues.apache.org/jira/browse/QPID-4976
> Project: Qpid
>  Issue Type: Bug
>  Components: C++ Broker, C++ Client
>Affects Versions: 0.22
>Reporter: Gordon Sim
>Assignee: Gordon Sim
> Fix For: 0.23
>
>
> I.e. delete-on-close, delete-on-no-links, delete-on-no-messages and 
> delete-on-no-links-or-messages

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (QPID-4976) [AMQP 1.0] support standard lifetime policies

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/QPID-4976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704578#comment-13704578
 ] 

ASF subversion and git services commented on QPID-4976:
---

Commit 1501768 from [~gsim]
[ https://svn.apache.org/r1501768 ]

QPID-4976: support standard lifetime policies

> [AMQP 1.0] support standard lifetime policies
> -
>
> Key: QPID-4976
> URL: https://issues.apache.org/jira/browse/QPID-4976
> Project: Qpid
>  Issue Type: Bug
>  Components: C++ Broker, C++ Client
>Affects Versions: 0.22
>Reporter: Gordon Sim
>Assignee: Gordon Sim
> Fix For: 0.23
>
>
> I.e. delete-on-close, delete-on-no-links, delete-on-no-messages and 
> delete-on-no-links-or-messages

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Updated] (QPID-4530) Provide Documentation for JCA/XA/Cluster configuration

[Weston M. Price (JIRA)]

 [ 
https://issues.apache.org/jira/browse/QPID-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Weston M. Price updated QPID-4530:
--

Fix Version/s: (was: 0.21)
   0.23

> Provide Documentation for JCA/XA/Cluster configuration
> --
>
> Key: QPID-4530
> URL: https://issues.apache.org/jira/browse/QPID-4530
> Project: Qpid
>  Issue Type: Improvement
>  Components: JCA
>Affects Versions: 0.18
> Environment: All supported OS platforms, all supported JEE platforms.
>Reporter: Weston M. Price
>Assignee: Weston M. Price
> Fix For: 0.23
>
>
> Currently we are lacking any sort of documentation on configuration, 
> limitations etc when running JCA in a cluster environment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Re: Swigged python package name...

[Darryl L. Pierce]

On Tue, Jul 09, 2013 at 04:01:02PM -0400, Ted Ross wrote:
> I would think that downstream repos would package one or the other
> but not both.  It would be nice if downstream, it was called
> qpid.messagingregardless of the implementation used.

Agreed. I think it ought to be a goal to have the Swigged bindings be a
drop-in replacement for the pure Python bindings. The users shouldn't care
which implementation they use.

>  Is there a way
> to parameterize it such that the namespace could be overridden when
> built and packaged?

Well, the name space is defined by the directory structure when the
modules are installed. So wherever the .py files are installed defines
what the package name will be. So I did a small experiment (see below)
just to verify that we can use different package names without having to
munge the code.
 
> For example, call the wrapped client cqpid.messaging but make it
> easy for a distro to build and package it as qpid.messaging.

We can accomplish this by just having a qpid/messaging directory tree
under the site packages location and in __init__.py there do:

from cqpid import *

This would put the entire cqpid.* set into the qpid/messaging namespace.

Regarding the module name coming out of Swig, I don't think we can use a
segmented name. Looking at the documentation [1] it appears that's
solely used to name the shared library and how that name is interpreted
is dynamic language dependant.

Unless you were suggesting using cqpid.messaging similar to what I
described above.

WRT the name space, I'm still wanting to keep it as qpid.messaging
since, in the long run, that would require developers to not change
their imports and usage. If, down the road, a group decided to maintain
the pure Python bindings separately they could use a different
namespace, like we saw with the evolution of qpid::client to
qpid::messaging in the C++ code itself?

[1] http://www.swig.org/Doc2.0/Modules.html

-- 
Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc.
Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/



pgpxTqiP0Y2Ix.pgp
Description: PGP signature

Re: Swigged python package name...

[Gordon Sim]

On 07/09/2013 08:34 PM, Darryl L. Pierce wrote:

One of the issues that needs to be fixed for this is the package name
for the code. Currently it's "cqpid", which isn't close to what we have
in other languages (which all use a variable on qpid.messaging).

However, we can't use that since it's what our pure Python code uses.

Or can we?


I don't think we should. They should each have their own name to clearly 
identify them (the name refers to the implementation, not just the API 
as it does in say c++).


I agree that cqpid is not a good name for the swigged module; it is too 
ambiguous. I would suggest something like qpidc_messaging or 
qpid_cmessaging instead. However changes such as those need to be 
discussed on the user list.



That's the question: how do we package up the code? I've experimented
with the examples we have and they seem to work just fine with cqpid
substituted for qpid.messaging for the imports.


I have been working to get (at least some of) the python tests running 
over the swigged implementation. My motivation here was initially to add 
some new tests for the AMQP 1.0 functionality using python. However it 
would also be desirable to be able to run some of the existing tests 
over 1.0 as well as continuing to be able to run over 0-10.


I hope to have a patch up for review before too long, but by adding 
another module (I called it qpid_messaging.py) that tries to import from 
the swigged module, then falls back to importing qpid.messaging, I was 
able to have one import statement in tests of interest and then run them 
over either swigged or pure implementations based on whether the former 
was on the path or not.


So in this scheme you have qpid.messaging for the exiting, pure python 
implementation (only supports 0-10 at present), cqpid (which I agree is 
a poor name, but see above) for the swigged implementation that supports 
0-10 and 1.0, and qpid_messaging that can be used in contexts where 
switching between the two without editing files is desirable (e.g. tests 
and perhaps examples).



-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Commented] (QPID-4983) [Java Broker] Move store implementations to broker plugins

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/QPID-4983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13704342#comment-13704342
 ] 

ASF subversion and git services commented on QPID-4983:
---

Commit 1501682 from [~godfrer]
[ https://svn.apache.org/r1501682 ]

QPID-4983 : [Java Broker] Move store implementations to broker plugins

> [Java Broker] Move store implementations to broker plugins
> --
>
> Key: QPID-4983
> URL: https://issues.apache.org/jira/browse/QPID-4983
> Project: Qpid
>  Issue Type: Improvement
>  Components: Java Broker
>Reporter: Rob Godfrey
>Assignee: Rob Godfrey
>Priority: Minor
>
> Move the java broker store implementations to separate plugins to enable 
> deployment of brokers which do not offer a particular store type if it is not 
> required / desired.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

[jira] [Created] (QPID-4983) [Java Broker] Move store implementations to broker plugins

[Rob Godfrey (JIRA)]

Rob Godfrey created QPID-4983:
-

 Summary: [Java Broker] Move store implementations to broker plugins
 Key: QPID-4983
 URL: https://issues.apache.org/jira/browse/QPID-4983
 Project: Qpid
  Issue Type: Improvement
  Components: Java Broker
Reporter: Rob Godfrey
Assignee: Rob Godfrey
Priority: Minor


Move the java broker store implementations to separate plugins to enable 
deployment of brokers which do not offer a particular store type if it is not 
required / desired.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org