Re: [NOTICE / DISCUSS] migrating Git repositories to gitbox.apache.org
+1 from me for early migration too, albeit from my not-so-lofty status as a very occasional committer :) On Mon, 10 Dec 2018 at 16:10, Robbie Gemmell wrote: > Hi Folks, > > Per the below mail sent by the ASF Infrastructure team (to dev@), our Git > repositories will be migrated from the original/old git-wip-us service on > to > the newer gitbox.apache.org service by February 7th. > > This is definitely happening, but we have some say over when/how it does: > 1. Document consensus on list and voluntarily request to move by Jan 9th. > 2. Wait for mandated move between Jan 9th and Feb 6th, coordinate with > infra >to proceed once it has been. > 3. Do nothing, ignore the mandate, and be forcibly moved on Feb 7th. > > I'm in favour of volunteering and doing it sooner than later, so to > that end here is my lazy consensus statement: > > Unless there is discussion here resulting in decision to delay, I will > raise the migration request JIRA with Infra on Thursday 13th Dec, > hopefully allowing completion of the process by the end of this week > if they have bandwidth to do so. > > Robbie > > On Fri, 7 Dec 2018 at 16:52, Daniel Gruno wrote: > > > > [NOTICE] Mandatory relocation of Apache git repositories on > git-wip-us.apache.org > > > > > > [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE > > DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS] > > > > Hello Apache projects, > > > > I am writing to you because you may have git repositories on the > > git-wip-us server, which is slated to be decommissioned in the coming > > months. All repositories will be moved to the new gitbox service which > > includes direct write access on github as well as the standard ASF > > commit access via gitbox.apache.org. > > > > ## Why this move? ## > > The move comes as a result of retiring the git-wip service, as the > > hardware it runs on is longing for retirement. In lieu of this, we > > have decided to consolidate the two services (git-wip and gitbox), to > > ease the management of our repository systems and future-proof the > > underlying hardware. The move is fully automated, and ideally, nothing > > will change in your workflow other than added features and access to > > GitHub. > > > > ## Timeframe for relocation ## > > Initially, we are asking that projects voluntarily request to move > > their repositories to gitbox, hence this email. The voluntary > > timeframe is between now and January 9th 2019, during which projects > > are free to either move over to gitbox or stay put on git-wip. After > > this phase, we will be requiring the remaining projects to move within > > one month, after which we will move the remaining projects over. > > > > To have your project moved in this initial phase, you will need: > > > > - Consensus in the project (documented via the mailing list) > > - File a JIRA ticket with INFRA to voluntarily move your project repos > >over to gitbox (as stated, this is highly automated and will take > >between a minute and an hour, depending on the size and number of > >your repositories) > > > > To sum up the preliminary timeline; > > > > - December 9th 2018 -> January 9th 2019: Voluntary (coordinated) > >relocation > > - January 9th -> February 6th: Mandated (coordinated) relocation > > - February 7th: All remaining repositories are mass migrated. > > > > This timeline may change to accommodate various scenarios. > > > > ## Using GitHub with ASF repositories ## > > When your project has moved, you are free to use either the ASF > > repository system (gitbox.apache.org) OR GitHub for your development > > and code pushes. To be able to use GitHub, please follow the primer > > at: https://reference.apache.org/committer/github > > > > > > We appreciate your understanding of this issue, and hope that your > > project can coordinate voluntarily moving your repositories in a > > timely manner. > > > > All settings, such as commit mail targets, issue linking, PR > > notification schemes etc will automatically be migrated to gitbox as > > well. > > > > With regards, Daniel on behalf of ASF Infra. > > > > PS:For inquiries, please reply to us...@infra.apache.org, not your > > project's dev list :-). > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org > For additional commands, e-mail: dev-h...@qpid.apache.org > >
Re: Welcome Roddie Kieley as an Apache Qpid committer
Welcome Roddie! On Tue, 27 Nov 2018 at 15:14, Jakub Scholz wrote: > Congrats and welcome to the community ... and thanks for all your > contributions Roddie! > > Jakub > > On Tue, Nov 27, 2018 at 3:38 PM Robbie Gemmell > wrote: > > > The Qpid PMC have voted to grant commit rights to Roddie Kieley in > > recognition of continued contributions to the project. > > > > Welcome, Roddie! > > > > Robbie > > > > - > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > >
Re: Client to broker connection status using C++ API
On Wed, 17 Oct 2018 at 18:00, Gordon Sim wrote: > On 17/10/18 17:15, trivedi_ravi13 wrote: > > Hi, > > > >I've a requirement to monitor if a particular client is connected > > successfully to the broker or not periodically. Is there a way using C++ > > APIs to get the status of all connected clients on Qpid 0.24 ? > > Yes. Different brokers have different schemas and management protocols, > but I think all of them allow management via some message based protocol. > > The qpid c++ broker, it uses a protocol called QMF (qpid management > framework), which is a message based protocol that you can use with any > amqp client. > > Attached is a simple example using the qpid::messaging API for listing > connections (or indeed other types) using qmf. You could also use the > proton cpp api sending the same request message. > > > Further to Gordon's example code, I'd like to mention an open-source library I've written which does exactly this: https://github.com/fourceu/fourc-qpid-manager This is based on the QMF protocol Gordon mentioned. There's an example which gets all bindings: https://github.com/fourceu/fourc-qpid-manager/blob/master/examples/Bindings.cxx Substituting "Connections" for "Bindings" here should give you what you're after. I have tested it only on the later 1.36-1.38 versions but there's no reason I know of that would prevent it from working with 0.24. > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org
Re: Proposal to deprecate qpid legacystore
Regarding the actual "following through" on the deprecation: If I understand correctly, the linearstore ORM code relies on classes from the Berkeley DB library; with legacystore gone, the only purpose of the dependency on BDB would be for this mapping even though BDB storage would no longer be in use anywhere. The current-day upshot of this is that BDB is required even if linearstore is the only store included in the build. It would be nice if this dependency on BDB from linearstore could be removed as part of the deprecation of legacystore; the only problem is that this increases the complexity of the task by an order of magnitude and means that quite some refactoring will be required to the relatively stable linearstore code, which then might not be quite so stable. Perhaps there's a more acceptable migration path that would achieve all goals and also maintain QA? Apologies if I'm harping on about an already well-known issue. On Fri, 8 Jun 2018 at 18:01, Kim van der Riet wrote: > On 06/06/2018 01:04 PM, Robbie Gemmell wrote: > > Seems reasonable. I assume you mean warning at startup only if they > > are actually using the legacy store. > > Yes, correct. Sorry I was not more clear. If the legacystore module is > loaded by qpid, then a warning-level deprecation notice will appear when > it is initialized. > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > >
Re: Note: Heads up regarding Ubuntu QPID-CPP cmake issue
Hi Paul, On 17 May 2018 at 15:30, Flores, Paul A.wrote: > UBUNTU uses uuidgen rather then uuid_generate. This causes issues with > src/CMakeList.txt "block" from lines 296 thru 320. > If I'm not mistaken, uuidgen is the command-line tool while uuid_generate is the name of the library function (wrapped by uuidgen) which the cmake script is looking for in the section you reference. Can you be more specific about what issues you're seeing? Reporting error messages would be helpful. It would also be instructive to know what version of Ubuntu you're using and whether or not you have the "uuid-dev" package installed as the latter affects the execution path of the script. Since I've been using Qpid on Ubuntu for several years and haven't had any issues in this area, I wonder if there might be something unusual about your environment; perhaps if for example you've installed something (possibly uuid-related) from source that's affecting things or your build environment has otherwise become inconsistent (broken apt packages etc)? Build logs might be informative on this point. > > Immediate resolution is straightforward, alter the script to refer to > uuidgen (search and replace). > > > I would like to suggest a long term resolution is to rely on > IdentifyOS.cmake to record the OS into a variable and use accordingly. > Overall scope of such a change is beyond my understanding of the existing > cmake structure/organization. > The section of CMake script you mention attempts (correctly, I think) to discover the relevant interface in an OS-agnostic manner, relying instead on the libraries and headers it is actually able to locate. Given that even with knowledge of the precise OS variant the available library interfaces are still not known, I would say this is the correct approach. > > > > > > This communication (including any attachments) may contain information > that is proprietary, confidential or exempt from disclosure. If you are not > the intended recipient, please note that further dissemination, > distribution, use or copying of this communication is strictly prohibited. > Anyone who received this message in error should notify the sender > immediately by telephone or by return email and delete it from his or her > computer. >
Re: QPid-proton cpp 0.21 - Crash
A couple of suggestions spring to mind - I've experienced problems with timers in other libraries where a timer fires after (or indeed during) its callback or associated data has been deleted, resulting in a segfault. Could this be relevant? Probably capturing a core dump and inspecting with gdb would be enlightening and would be my first port of call. Another approach might be some code introspection to verify that timers are cancelled and that their handlers have completed before relevant garbage collection takes place. Rather general comments I'm afraid but I thought it might be worth consideration. Chris On 17 April 2018 at 16:36, Baptiste Robertwrote: > Hello, > > When I create a proton::container and use it, I have a crash when I delete > the proton object: > > void pn_proactor_free(pn_proactor_t *p) { > -> DeleteTimerQueueEx(p->timer_queue, INVALID_HANDLE_VALUE); > > I'm using proton 0.21 compiled in CXX03 mode. > > Is anyone have an idea ? > > Thank you, > > Baptiste >
Re: [ANNOUNCE] Apache Qpid C++ 1.38.0 released
FYI, Gentoo ebuilds for this release (together with required Proton versions) are available here: https://github.com/fourceu/qpid-portage-overlay On 4 April 2018 at 00:20, Justin Ross <jr...@apache.org> wrote: > The Apache Qpid community is pleased to announce the immediate availability > of Apache Qpid C++ 1.38.0. > > Apache Qpid (https://qpid.apache.org) is a cross-platform messaging > solution > that implements the Advanced Message Queuing Protocol (AMQP, > https://www.amqp.org). It provides message brokers written in C++ > and Java, a message router, and messaging APIs for C, C++, Java, > JMS, Python, and Ruby. > > This release provides an update of the Qpid C++ broker and the > C++ Qpid Messaging API with its related language bindings, incorporating > various bug fixes and improvements. > > This release incorporates a number of defect fixes and enhancements. > Release notes are available at: > > http://qpid.apache.org/releases/qpid-cpp-1.38.0/release-notes.html > > The release is available now from our website: > > http://qpid.apache.org/download.html > > Thanks to all the users and contributors who have helped to improve > Apache Qpid C++. > > Justin > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: [C++ binding] Complex types: defining an array of arrays of differing types
Is using boost::variant an option? Then you could use something like #include ... typedef boost::variant<uint8_t, bool> elem_type; std::vector array1 = {uint8_t(0), uint8_t(1)}; std::vector array2 = {true, false}; std::vector<std::vector> array3 = {array1, array2}; This at least compiles, but I don't know if it will produce the results you want from a QITS perspective. Chris On 28 March 2018 at 17:23, Kim van der Riet <kim.vdr...@redhat.com> wrote: > This is part of the Qpid Interop Test Suite, a test written for complex > type interoperability. > > AMQP allows arrays of arrays of differing types, I am wanting to test this > using the different bindings against each other - this is what the QITS > does. > > In the case of the C++ binding, the default encoding for an AMQP array > occurs when a container (std::vector, std::deque, etc.) is of a c++ type > rather than using proton::value, which would cause the encoding to be an > AMQP list. > > Certainly I can see that the non-homogenous nature of the declaration is > the issue, I just don't know how to get around it in a way that the Proton > encoder will accept. I know there is a way to force a > std::vector to be encoded as an array, but I don't know how > to do that. > > Kim > > > On 03/28/2018 11:49 AM, Chris Richardson wrote: > >> The short answer is "you can't", at least not along the lines you're >> thinking. The std::vector type is a homogeneous container, see eg: this >> for >> more info: https://isocpp.org/wiki/faq/containers#heterogeneous-contain >> ers >> >> The upshot is that the elements of the vectors must have a common (base) >> type, which clearly uint8_t and bool do not. The usual way to get around >> this is exactly as you've outlined - to use some sort of variant-style >> container such as the proton::value or boost::variant etc. The >> alternatives >> (as mentioned in the above article) are almost too terrible to contemplate >> and would involve lists of void pointers and risky reinterpret_casts >> scattered through the code . >> >> In some situations it can be useful to use a tuple type, then you could >> have some construct like >> std::tuple<std::vector, std::vector> >> but if your list isn't of a fixed size known at compile-time (or >> specifically needs to be a vector) this probably isn't what you're after >> either. >> >> Can you say more about what you're trying to do and why proton::value >> isn't >> sufficient? >> >> Chris >> >> >> >> On 28 March 2018 at 15:45, Kim van der Riet <kim.vdr...@redhat.com> >> wrote: >> >> By default, using std::vector<c++-type> is encoded as an AMQP array, >>> whereas std::vector is encoded as an AMQP list. So, to >>> create an array of lists of differing types, for example: >>> >>> std::vector list1 = {uint8_t(0), uint8_t(1)}; >>> std::vector list2 = {true, false}; >>> std::vector<std::vector > array = {list1, list2}; >>> >>> works ok. However, how does one define an array of arrays of differing >>> types? For example: >>> >>> std::vector array1 = {uint8_t(0), uint8_t(1)}; >>> std::vector array2 = {true, false}; >>> std::vector<std::vector<##> > array3 = {array1, array2}; >>> >>> What is ##? Unless all the sub-arrays are of the same type, it is not >>> possible to define a single type here. If I use proton::value (which >>> would >>> make it indistinguishable from the array of lists above), I get a compile >>> error: >>> >>> could not convert '{array1, array2}' from '>> list' to 'std::vector<std::vector >' >>> >>> What is the correct way to create an array of arrays of differing types? >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org >>> For additional commands, e-mail: users-h...@qpid.apache.org >>> >>> >>> >> >> > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: [C++ binding] Complex types: defining an array of arrays of differing types
The short answer is "you can't", at least not along the lines you're thinking. The std::vector type is a homogeneous container, see eg: this for more info: https://isocpp.org/wiki/faq/containers#heterogeneous-containers The upshot is that the elements of the vectors must have a common (base) type, which clearly uint8_t and bool do not. The usual way to get around this is exactly as you've outlined - to use some sort of variant-style container such as the proton::value or boost::variant etc. The alternatives (as mentioned in the above article) are almost too terrible to contemplate and would involve lists of void pointers and risky reinterpret_casts scattered through the code . In some situations it can be useful to use a tuple type, then you could have some construct like std::tuple<std::vector, std::vector> but if your list isn't of a fixed size known at compile-time (or specifically needs to be a vector) this probably isn't what you're after either. Can you say more about what you're trying to do and why proton::value isn't sufficient? Chris On 28 March 2018 at 15:45, Kim van der Riet <kim.vdr...@redhat.com> wrote: > By default, using std::vector<c++-type> is encoded as an AMQP array, > whereas std::vector is encoded as an AMQP list. So, to > create an array of lists of differing types, for example: > > std::vector list1 = {uint8_t(0), uint8_t(1)}; > std::vector list2 = {true, false}; > std::vector<std::vector > array = {list1, list2}; > > works ok. However, how does one define an array of arrays of differing > types? For example: > > std::vector array1 = {uint8_t(0), uint8_t(1)}; > std::vector array2 = {true, false}; > std::vector<std::vector<##> > array3 = {array1, array2}; > > What is ##? Unless all the sub-arrays are of the same type, it is not > possible to define a single type here. If I use proton::value (which would > make it indistinguishable from the array of lists above), I get a compile > error: > > could not convert '{array1, array2}' from ' list' to 'std::vector<std::vector >' > > What is the correct way to create an array of arrays of differing types? > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Install on mac osx
Hi Joshua, A similar issue was raised a few weeks ago, you might find that thread useful: http://qpid.2158936.n2.nabble.com/Error-installing-python-qpid-proton-0-20-0-tp7672530p7672660.html Unfortunately I'm no more able to test this on a Mac than Ken, so that information is all I can offer! I think you'll need to attempt an installation from source rather than via pip. Chris On 20 February 2018 at 05:08, Joshua Agudo <joshua.ag...@gmail.com> wrote: > Hi, > > Is it possible to install python qpid proton on a mac? > I ran "pip3 install python-qpid-proton" but failed with: > > "/private/var/folders/f0/8mtk875s79jcgcy1l590z69mgn > /T/pip-build-Er8aX6/python-qpid-proton/proton-c/src/ > proactor/epoll.c:48:10: > fatal error: 'sys/timerfd.h' file not found" > > Is qpid proton only supported on linux/windows? > > > Regards > Josh > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Type of node with Proton library
Hi Daniel, It looks to me like you've almost answered your own question, or perhaps that I've misunderstood it... however I'll forge ahead regardless! I have some code which does this here: https://github.com/fourceu/fourc-qpid-manager/blob/master/include/fourc/fmf/BrokerAgentTemplate.h See line 532, "sendMessage". If you look past the Boost details you'll see that it passes the node type in the sender address exactly as you have specified with spout. There is a second way to provide address attributes by passing a nested Variant::Map structure to the address. This has been useful to me when using other address attributes such as a link selector. I don't have an example of this I can readily share but the pseudo-code might look something like: qpid::messaging::Session session; // = connection.createSession etc qpid::messaging::Message message; // the message to be sent qpid::messaging::types::Variant::Map node_type; qpid::messaging::types::Variant::Map options; node_type["type"] = "topic"; options["node"] = node_type; options["assert"] = "always"; // I've found that this is important or the node type specification may be ignored qpid::messaging::Address address("amq.topic"); address.setOptions(options); auto sender = session.createSender(address); sender.send(message); HTH Chris On 7 February 2018 at 12:37, Gavrila, Daniel <d.gavr...@selex-es-gmbh.com> wrote: > Hi , > > I was reading this thread > > http://qpid.2158936.n2.nabble.com/Queue-and-exchange-with- > the-same-name-td7579722.html > > and I would like to know how I can specify the type of node( queue or > topic) in C++ Proton like here: > > spout 'amq.topic; {node: {type:topic}}' > > spout 'amq.topic; {node: {type:queue}}' > > Best regards, > Daniel > > Selex ES GmbH > Sitz der Gesellschaft / Registered Office: Neuss > Registergericht / Register Court: Neuss HRB 17453 > Gesch?ftsf?hrer / Managing Director: Ulrich Nellen > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Qpid PPA releases for qpid-cpp-1.37/qpid-proton-0.18
Could we please have an update to the very useful PPA at https://launchpad.net/~qpid/+archive/ubuntu/released? It would be really useful to have qpid-cpp-1.37 and qpid-proton-0.18 in there. I realise I mentioned this here previously, but in doing so I hijacked another thread which showed rather poor etiquette on my part and I was deservedly ignored! Many thanks -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: New user questions: Broker/AMQP versions
On 22 January 2018 at 14:17, Justin Ross <justin.r...@gmail.com> wrote: > On Mon, Jan 22, 2018 at 6:05 AM, gewesp <gew...@gmail.com> wrote: > > > Hi all, > > > > I'm consulting with a company in the security business and we're > evaluating > > the use of QPID as a messaging platform. > > > > Q1: Supported AMQP versions of the C++ broker (qpidd): > > * According to https://qpid.apache.org/components/cpp-broker/index. > html > > <https://qpid.apache.org/components/cpp-broker/index.html> , > > it supports 0-10 and 1.0. > > * According to > > https://qpid.apache.org/releases/qpid-cpp-1.37.0/cpp-broker/book/AMQP- > > Compatibility.html > > <https://qpid.apache.org/releases/qpid-cpp-1.37.0/cpp-broker/book/AMQP- > > Compatibility.html> > > , > > it supports 0-9 and 0-10 . > > Which one is it? > > > > The former, 0-10 and 1.0. The C++ broker book needs a lot of work! > > > > Q2: APIs: As far as I understand, there are two C++ APIs (Messaging and > > Proton). Which one of > > these is the preferred one and will be supported in the long run? > > > > Q3: Python3 support: AFAICS, there is Python3 support for the Proton API, > > but not for Messaging (neither > > is it planned?). Can I deduce from that that the Proton API is the > > preferred one? > > > > Yes, Proton is the way to go. > > > > FWIW, we're using Ubuntu 16.04 (LTS) and we'd like to go with standard > > Ubuntu packages > > as far as practicable. The project requires the use of AMQP 1.0. > > > > The best way to get recent Qpid in Ubuntu is the Qpid PPA. The default > distro packages tend to be really old. > > https://launchpad.net/~qpid/+archive/ubuntu/released Apropos the PPA, we are eagerly anticipating the addition of packages for Proton 18/19 and Qpid-cpp 1.37 :) -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Unexplained b" and " around message body with python and java, but not with C++
Hi Olivier, The 'b' prefix in this case is part of the automatic character encoding of the language in use, indicating that the string is a "bytes" object rather than a Unicode string which is the default expectation of the C++. Have a browse of the relevant character encoding documentation for the language in question, for instance https://www.pythoncentral.io/python-unicode-encode-decode-strings-python-2x/ for python. For the simple case of a string literal you could try assigning your string with Message(u"[MESSAGE]"). Hope that helps Chris On 21 December 2017 at 16:51, Olivier Delbeke <olivier.delb...@awtce.be> wrote: > Hi all, > > When I send messages to a queue using the java or python APIs, the message > body I receive on the other side (C++ binding) is preceded by b" and > appended with ". > When I send the same message with the C++ API, I do not have this > behaviour. > > Example: > [C++] sender.send( proton::message("[MESSAGE]") ) > --> results in [MESSAGE] in C++ ( to_string(message.body()) ) > > [python] sender.send( Message(body="[MESSAGE]") ) > --> results in b"[MESSAGE]" in C++ ( to_string(message.body()) ) => b and > " are added at the beginning, and " is added at the end. > > I tried many things like forcing the "content_type" and the > "content_encoding", or changing the "inferred" flag, but nothing helped. > These annoying b" and " are still there. > Looking at the source code of qpid proton, I also do not find where they > could be added or why. > > (BTW, I use qpid-proton 0.18.0) > > Any idea ? > > Olivier > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Compiling proton-c for Linux Alpine
On 21 November 2017 at 17:41, Alessio Gottardo <tappo...@yahoo.it.invalid> wrote: > Thank you Chris, > > I've been reading about this new world (to me) of the reasons behind glibc > and ulibc. In some way there are similar issues in "toy languages / higher > level languages" like java or javascript (node.js) when shipping the > equivalent of executables with tons of dependencies (and transitive > dependencies) making the artefact huge in terms of disk space. > > Anyway, I've added the following to the docker file (using RUN statements) > in order to include the binary package of glibc to Linux Alpine (cf. > https://github.com/sgerrand/alpine-pkg-glibc#installing): > > > apk --no-cache add ca-certificates wget > wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent. > com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub > wget https://github.com/sgerrand/alpine-pkg-glibc/releases/ > download/2.25-r0/glibc-2.25-r0.apk > apk add glibc-2.25-r0.apk > > However I get the same compilation error. > > I believe that's because somehow I need to point the cmake/make stuff to > glibc (instead of the default/system ulibc) hardcoding somewhere the path? > If that's the case then I am not sure where I should change this stuff and > which path I should add. > Please let me know if this rings any bell. > I've followed these installation steps and now realise that the glibc package above is runtime only and does not support local compilation - were it to do so it would probably be called "glibc-dev" by convention. This means that you can only run packages that were compiled elsewhere, which is presumably the case for JDKs etc. In retrospect an obvious choice when the target is a minimalist OS so apologies for leading you down a bit of a blind alley. In terms of progressing further I would suggest you might look into extending a build system such as sgerrand's (the one that generates the glibc.apk package) in order to either build a glibc-dev package (this will probably result in a bloated Alpine image) or to build qpid-proton and export it as a .apk file. The latter might be the most satisfactory solution since you would then not require many of the build dependencies you already have (git, cmake, gcc etc) in order to install qpid-proton in the live environment. The down-side is that it's quite and involved process and certainly not one for the faint of heart! Another alternative is that we make a more official version of the source code hack I previously mentioned to make Proton compatible with the reduced (I'm not judging) musl libc. Any thoughts, Messrs. Stitcher, Conway et al? Thanks > Alessio > > > > > > On Monday, 20 November 2017, 19:34:50 GMT, Chris Richardson < > c...@fourc.eu> wrote: > > Hi Alessio, > The problem here appears to be that qpid-proton assumes glibc is installed > on your system (it is on "most"(tm) linuces), but alpine doesn't appear to > contain it, nor yet is it available in the default apk repositories. The > pthread-stub package which provides /usr/include/pthread.h does not include > the expected PTHREAD_MUTEX_ADAPTIVE_NP definition. > The easiest option might be to use a docker image which does include > glibc, try one of the documented solutions here http://wiki.alpinelinux. > org/wiki/Running_glibc_programs or here https://github.com/sgerrand. > This is a bit of a hack and comes with a huge disclaimer as I haven't > tested it at all, but if glibc isn't an option you could change > the PTHREAD_MUTEX_ADAPTIVE_NP to PTHREAD_MUTEX_NORMAL. I _think_ the only > side effect would be that some sub-optimal system calls might take place so > the thread locking would not be as efficient.I've attached a patch to do > this which you can apply withpatch -p1 < qpid-proton-0.18.0-pthread- > stub.patch > I hope this helps to point you in the right direction. > Chris > > On 20 November 2017 at 17:11, Alessio Gottardo <tappo...@yahoo.it.invalid> > wrote: > > Hi there, > I am trying to compile Qpid Proton on Linux Alpine (within a docker image). > I am getting this error: > /go/src/qpid-proton/proton-c/ src/proactor/epoll.c: In function > 'pmutex_init':/go/src/qpid- proton/proton-c/src/proactor/ epoll.c:104:36: > error: 'PTHREAD_MUTEX_ADAPTIVE_NP' undeclared (first use in this function) > pthread_mutexattr_settype(& attr, PTHREAD_MUTEX_ADAPTIVE_NP); > ^/go/ > src/qpid-proton/proton-c/src/ proactor/epoll.c:104:36: note: each > undeclared identifier is reported only once for each function it appears > inmake[2]: *** [proton-c/CMakeFiles/qpid- proton.dir/build.make:695: > proton-c/CMakeFiles/qpid- proton.dir/src/proactor/epoll. c.o] Error > 1make[1]: *** [CMakeFiles/Make
Re: [VOTE] Release Apache Qpid CPP 1.37.0
+1 - Verified signatures and checksums - Built from source on Ubuntu 17 (GCC 7.2); ran "make test" with success - Built from source on Gentoo x64 On 21 November 2017 at 18:20, Robbie Gemmell <robbie.gemm...@gmail.com> wrote: > Hi folks, > > I have put together a spin for a Qpid CPP 1.37.0 release, please give > it a test out and vote accordingly. > > The source archive can be grabbed from: > https://dist.apache.org/repos/dist/dev/qpid/cpp/1.37.0-rc1/ > > The JIRAs currently assigned are: > https://issues.apache.org/jira/secure/ReleaseNote.jspa? > projectId=12310520=12338808 > > It is tagged as 1.37.0-rc1. > > Regards, > Robbie > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Compiling proton-c for Linux Alpine
Hi Alessio, The problem here appears to be that qpid-proton assumes glibc is installed on your system (it is on "most"(tm) linuces), but alpine doesn't appear to contain it, nor yet is it available in the default apk repositories. The pthread-stub package which provides /usr/include/pthread.h does not include the expected PTHREAD_MUTEX_ADAPTIVE_NP definition. The easiest option might be to use a docker image which does include glibc, try one of the documented solutions here http://wiki.alpinelinux.org/wiki/Running_glibc_programs or here https://github.com/sgerrand. This is a bit of a hack and comes with a huge disclaimer as I haven't tested it at all, but if glibc isn't an option you could change the PTHREAD_MUTEX_ADAPTIVE_NP to PTHREAD_MUTEX_NORMAL. I _think_ the only side effect would be that some sub-optimal system calls might take place so the thread locking would not be as efficient. I've attached a patch to do this which you can apply with patch -p1 < qpid-proton-0.18.0-pthread-stub.patch I hope this helps to point you in the right direction. Chris On 20 November 2017 at 17:11, Alessio Gottardo <tappo...@yahoo.it.invalid> wrote: > Hi there, > I am trying to compile Qpid Proton on Linux Alpine (within a docker image). > I am getting this error: > /go/src/qpid-proton/proton-c/src/proactor/epoll.c: In function > 'pmutex_init':/go/src/qpid-proton/proton-c/src/proactor/epoll.c:104:36: > error: 'PTHREAD_MUTEX_ADAPTIVE_NP' undeclared (first use in this function) > pthread_mutexattr_settype(, PTHREAD_MUTEX_ADAPTIVE_NP); > ^/go/ > src/qpid-proton/proton-c/src/proactor/epoll.c:104:36: note: each > undeclared identifier is reported only once for each function it appears > inmake[2]: *** [proton-c/CMakeFiles/qpid-proton.dir/build.make:695: > proton-c/CMakeFiles/qpid-proton.dir/src/proactor/epoll.c.o] Error > 1make[1]: *** [CMakeFiles/Makefile2:1069: > proton-c/CMakeFiles/qpid-proton.dir/all] > Error 2make: *** [Makefile:141: all] Error 2 > The relevant part of the Dockerfile (FROM golang:1.8.3-alpine3.6): > ```# download Apache QPIDWORKDIR /go/src/RUN git clone --progress > --verbose http://git.apache.org/qpid-proton.git > # dependencies for Apache QPID (check https://pkgs.alpinelinux.org/ > packages)RUN apk add gcc=6.3.0-r4 g++=6.3.0-r4 cmake=3.8.1-r0 > make=4.2.1-r0RUN apk add python2=2.7.13-r1RUN apk add libssl1.0=1.0.2m-r0 > cyrus-sasl=2.1.26-r10RUN apk add swig=3.0.10-r0 > # compile and install Apache QPIDWORKDIR /go/src/qpid-protonRUN git > checkout tags/0.18.0WORKDIR /go/src/qpid-proton/buildRUN cmake .. > -DCMAKE_INSTALL_PREFIX=/usr -DSYSINSTALL_BINDINGS=ONRUN make install``` > The previous steps are based on the details I found here: > https://github.com/apache/qpid-proton/blob/master/INSTALL.md > The reason for doing this is to be able to run Go applications using > Docker and the Electron library for QPID. > I am not familiar with compiling C projects and I am not able to proceed > based on that error message.It seems some header files are missing or > something like that but I can not figure out which dependency is > missing.Any hint or help on how to compile this library for Linux Alpine? > Thank youAlessio > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!* diff --git a/proton-c/src/proactor/epoll.c b/proton-c/src/proactor/epoll.c index d5a323c1..71743583 100644 --- a/proton-c/src/proactor/epoll.c +++ b/proton-c/src/proactor/epoll.c @@ -101,7 +101,7 @@ static void pmutex_init(pthread_mutex_t *pm){ pthread_mutexattr_t attr; pthread_mutexattr_init(); - pthread_mutexattr_settype(, PTHREAD_MUTEX_ADAPTIVE_NP); + pthread_mutexattr_settype(, PTHREAD_MUTEX_NORMAL); if (pthread_mutex_init(pm, )) { perror("pthread failure"); abort(); - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Welcome Chris Richardson as an Apache Qpid committer
Thanks all! I do promise to execute this office to the best of my ability etc etc so help me Qpid. On 15 November 2017 at 13:58, Chuck Rolke <cro...@redhat.com> wrote: > What he said. Welcome, Chris! > > - Original Message - > > From: "Robbie Gemmell" <robbie.gemm...@gmail.com> > > To: users@qpid.apache.org > > Sent: Wednesday, November 15, 2017 5:19:23 AM > > Subject: Welcome Chris Richardson as an Apache Qpid committer > > > > The Qpid PMC have voted to grant commit rights to Chris Richardson in > > recognition of continued contributions to the project. > > > > Welcome, Chris! > > > > - > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Memory keeping increasing when using Qpid, can any one help?
Fank, You should bear in mind that the messaging interfaces are concurrent - in the case of your sender, the sender.send(message) call is non-blocking and will queue messages up to the capacity of the sender (default: 50) and will then block until the queue has a slot available. This behaviour can be changed (if you wish) by calling sender.send(message, true) to make each call block. You can read more about it here: https://qpid.apache.org/ releases/qpid-cpp-1.36.0/messaging-api/book/replay.html Similarly, the receiver will pre-fetch messages from the broker up to its capacity setting, but the default is zero so this probably isn't relevant. What may be relevant is that session.acknowledge() is by default non-blocking (and can be changed to a blocking call with session.acknowledge(true)). While you might expect that each message would be destroyed as it goes out of scope in your loop, it so happens that this may also not behave as expected: the Message type is a wrapper on a smart pointer which I strongly suspect will be in use by the session until the background session.acknowledge() operation has finished with it. Just a small caveat on the implementation of your example sender - because the send is a background operation, you will probably find your connection gets closed and your process terminates before some of the last 50 messages are sent. Calling session.sync(true) before closing the connection will prevent this. Would these considerations explain the memory usage you're seeing? On 19 October 2017 at 11:13, question <46970...@qq.com> wrote: > Dear all, > > Sorry to disturb you. I am new to Qpid. > When I tried to write a simple sender and receiver by splitting the > HelloWorld example in Qpid cpp 1.36. I met memory problem. > > HelloWord1 is responsible receiving data as below: > >Connection connection(broker, connectionOptions); > connection.open(); > > Session session = connection.createSession(); > Receiver receiver = session.createReceiver(address); > > while (true) > { > > Message message; > message.setContentObject("Hello world!"); > message.getContentObject().setEncoding("utf8"); > // sender.send(message); > > message = receiver.fetch(Duration::FOREVER); > std::cout << message.getContent() << std::endl; > session.acknowledge(); > > } > connection.close(); > > HelloWorld2 is responsible sending data as below: > > Connection connection(broker, connectionOptions); > connection.open(); > Session session = connection.createSession(); > Sender sender = session.createSender(address); > for (int i = 0; i < 5; i++) > { > Message message; > message.setContentObject("Hello world!"); > message.getContentObject().setEncoding("utf8"); > sender.send(message); > > } > connection.close(); > > And I use Java qpid-broker-6.1.1 as broker. > > I saw the both processes HelloWord1.exe and HelloWorld2.exe memory usage > increase quickly. > Am I using the Qpid in a wrong way? Maybe I am too stupid. Can anyone > help? :-( > > The two cpp files are attached... > > Thanks, > Frank. > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Seeking help with some Qpid C++ issues
Since it's been a bit quiet in this thread I've added a couple of comments to the code review, carefully picking the low-hanging fruit of the initialisation issues and avoiding the awkward test failure :) As per review comments, I hope I have contributed more than just stating the obvious and apologies if I have not. Chris On 10 October 2017 at 18:09, Robbie Gemmell <robbie.gemm...@gmail.com> wrote: > Any takers? > > On 28 September 2017 at 19:08, Justin Ross <justin.r...@gmail.com> wrote: > > Hi, all. I've been attempting to get Qpid C++ in shape for a release. I > > need some review for some of the changes I've proposed (got some already > > from Robbie - thanks) and some help with test failures. > > > > The change that needs particular review attention: > > > > > > https://github.com/apache/qpid-cpp/pull/10/commits/ > 84e8001023e3a12a1032eafd7a8ea7719c47d627 > > > > The broader set of changes I have lined up: > > > > https://github.com/apache/qpid-cpp/pull/10 > > > > The test failures I'm seeing on Fedora 26: > > > > On Fedora 26 x86-64 > > > > The following tests FAILED: > > 18 - ha_tests (Failed) > > 26 - ssl_tests (Failed) > > > > ## Details for 'ha_tests' > > > > 18: ha_tests.ReplicationTests.test_auth > > > . > > fail > > 18: Error during test: Traceback (most recent call last): > > 18: File "/usr/bin/qpid-python-test", line 340, in run > > 18: phase() > > 18: File "/home/jross/code/qpid-cpp-ssorj/bld/src/tests/ha_tests. > py", > > line 633, in test_auth > > 18: s0.sender("ex").send("foo"); > > 18: File > > "/home/jross/code/qpid-cpp-ssorj/bld/bindings/qpid/ > python/qpid_messaging.py", > > line 812, in sender > > 18: s = self._sender(target) > > 18: File > > "/home/jross/code/qpid-cpp-ssorj/bld/bindings/qpid/ > python/qpid_messaging.py", > > line 771, in _sender > > 18: return _qpid_messaging.Session__sender(self, *args) > > 18: UnauthorizedAccess: Link detached by peer with > > amqp:unauthorized-access: ACL denied access request to ex from > > anonymous@QPID > > (/home/jross/code/qpid-cpp-ssorj/src/qpid/broker/amqp/Authorise.cpp:152) > > > > 2017-09-28 10:29:55 [Security] info SASL: Mechanism list: ANONYMOUS PLAIN > > 2017-09-28 10:29:55 [Security] info SASL: Starting authentication with > > mechanism: ANONYMOUS > > 2017-09-28 10:29:55 [Security] info qpid.0.0.0.0:53173-127.0.0.1:58096 > > Authenticated as anonymous@QPID > > 2017-09-28 10:29:55 [HA] info aba36a70(active) Accepted client connection > > qpid.0.0.0.0:53173-127.0.0.1:58096 qpid-python-tes(7561) > > 2017-09-28 10:29:55 [HA] debug aba36a70(active) Created queue q > > replication: all > > 2017-09-28 10:29:55 [HA] debug aba36a70(active) Guard of q at > 56c9dd56@tcp: > > 127.0.0.1:57887: Guarded: front 1, back 0, guarded 1 > > 2017-09-28 10:29:55 [HA] debug aba36a70(active) Subscription to q at > > 56c9dd56@tcp:127.0.0.1:57887: Subscribed: primary [1,0]={ }, guarded 1, > > backup (keep { }, drop { }) > > 2017-09-28 10:29:55 [HA] debug aba36a70(active) Subscription to q at > > 56c9dd56@tcp:127.0.0.1:57887: Caught up at 1 > > 2017-09-28 10:29:55 [HA] debug aba36a70(active) Created exchange ex > > replication: all > > 2017-09-28 10:29:55 [Protocol] error Error on attach: ACL denied access > > request to ex from anonymous@QPID > > (/home/jross/code/qpid-cpp-ssorj/src/qpid/broker/amqp/Authorise.cpp:152) > > > > ## Details for 'ssl_tests' > > > > 26: ++ /home/jross/code/qpid-cpp-ssorj/src/tests/ping_broker -b amqps:// > > 127.0.0.1:37029 > > --ssl-trustfile=/home/jross/code/qpid-cpp-ssorj/bld/src/ > tests/test_cert_dir/ca_cert.pem > > 26: + Failed: ConnectError - '[SSL:' 'CERTIFICATE_VERIFY_FAILED]' > > certificate verify failed '(_ssl.c:661)' > > 26: /home/jross/code/qpid-cpp-ssorj/bld/src/tests/run_ssl_tests: line > 299: > > Failed:: command not found > > > > (The test code here is broken. It's trying to execute the error string.) > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Compiling a minimal lightweight 32 bit cpp broker for and old linux
One hint that might save you some time later - I think I'm right in saying that the c++ broker requires Berkeley DB for persistence even when using the linearstore plugin; I believe this is because it internally uses BDB classes for its ORM irrespective of storage backend. If you're trying to build on a minimal system you might trip over this one. On 25 September 2017 at 12:43, Gordon Sim <g...@redhat.com> wrote: > On 25/09/17 10:15, Marco wrote: > >> Hi I am trying to compile a c++broker for an old 32 bit minimal linux, I >> do >> not need ssl or acl but I need to have persistency, I am not so strong >> with >> CMake so I am bit lost, has someone tried this ? What is the best strategy >> in your opinion ? >> > > As a first step, run ccmake and you can interactively turn off bits you > don't want. If the current options are good enough, thats easy to do (e.g. > you can turn off SSL and SASL, but at present not acls - but then the acl's > aren't enforced by default so its whether the small extra overhead in > library size is something you really care about). > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Qpid-cpp 1.37 release request
Hi, Short version: Please can I request/vote/beg for a release of qpid-cpp-1.37 soon? There are pending fixes that we would very much like to have in production. Long version (read: sob-story) One such example is that we're suffering at the hands of this issue: https://issues.apache.org/jira/browse/QPID-7693 concerning leaked file descriptors. It's a particular problem in our environment because we run inter-broker routes over unreliable network links which may be unavailable for extended periods of time. This causes the client broker to segfault due to leaked file descriptors and subsequently other applications local to the crashed broker to die due of the same affliction. As a temporary measure I've back-ported the above fix into our qpid-cpp-1.36 ebuild here https://github.com/fourceu/qpid-portage-overlay but clearly this is a gentoo-only fix. Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: cmake install seems to tamper with interpreter directive
autotools Unable to believe CMake could possibly be this rubbish (I am a devotee I'm afraid) I've just spent the last couple of hours sifting through the CMake's own source code to track this one down. It turns out that you have collectively conspired to besmirch the good name of CMake and an immediate and full retraction is required! ;) It seems the problem here is not with CMake but actually with the qpid-dispatch code, which has duplicate install routines for the qdstat/qdmanager files. One is as Gordon described using install(PROGRAMS...) and the other is via the python distutils method using /python/setup.py.in; it is in fact distutils that munges the shebang as described and not CMake at all! Removing qdstat/qdmanager from setup.py.in fixes the problem. CMake: 1, distutils: nil. On 6 September 2017 at 20:00, Ken Giusti <kgiu...@redhat.com> wrote: > Alan - > > Hate cmake? Maybe you'd be interested in automake instead. > > /me ducks > > ;) > > On Wed, Sep 6, 2017 at 2:44 PM, Alan Conway <acon...@redhat.com> wrote: > > On Wed, Sep 6, 2017 at 12:58 PM, Gordon Sim <g...@redhat.com> wrote: > > > >> It seems that the install directive for PROGRAMS in cmake somehow > >> evaluates and rewrites the interpreter directive in some way. > >> > >> E.g. in dispatch there is: > >> > >> install(PROGRAMS > >>> ${CMAKE_CURRENT_SOURCE_DIR}/tools/qdstat > >>> ${CMAKE_CURRENT_SOURCE_DIR}/tools/qdmanage > >>> DESTINATION bin) > >>> > >> > >> The first line of both those scripts is: > >> > >> #!/usr/bin/env python > >>> > >> > >> But when you install, it rewrites that to whatever that evaluates to. > This > >> is not really what is wanted when building an install image for > >> installation in some other system (e.g. an rpm or a docker image), where > >> the setup may be different. > >> > >> Does anyone know of a simple way to prevent this? > >> > >> > > That boggles my mind. I tried install(FILES...) and it does the same > > thing!!@# > > The cmake docs do not mention anything about such transformations, > > anywhere. The string "#!" does not even occur in the docs. The install > doc > > says it simply copies the files, which is clearly a lie. Since there's no > > mention that this "feature" even exists, I've no idea how to turn it off > - > > install won't allow a COPYONLY tag like configure_file, I tried. Sigh. > Did > > I mention I hate cmake? > > > > -- > -K > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Address options grammar
I was using the Address constructor Address(const std::string& name, const std::string& subject, const qpid::types::Variant::Map& options, const std::string& type = ""); to provide the name and options. It seems the behaviour is not the same though as your method also works for me. Problem solved, thanks very much :) C On 26 August 2017 at 14:30, Jakub Scholz <ja...@scholz.cz> wrote: > How exactly do you create the address and what do you have in you client / > broker log? > > When I try to do something like this: > Address myAddress; > Variant::Map options; > Variant::Map selector; > selector["selector"] = "color='blue'"; > options["link"] = selector; > Variant::Map node_type; > node_type["type"] = "queue"; > options["node"] = node_type; > options["assert"] = "always"; > myAddress.setOptions(options); > myAddress.setName("myTest"); > > For me it seems to work properly. The client log correctly creates the > address: > 2017-08-26 13:28:14 [Client] debug treating source address as queue: > myTest;{assert:always, link:{selector:color='blue'}, node:{type:queue}} > The client sends only QueueDeclare and QueueQuery and the only ACL right > needed seem to be queue/access and queue/consume. > > Regards > Jakub > > On Sat, Aug 26, 2017 at 1:08 PM, Chris Richardson <c...@fourc.eu> wrote: > > > Hi, > > > > Using the qpid-cpp-1.36 broker/client I am trying to specify two clauses > on > > a receiver node (queue) address; one is a selector and looks something > like > > this: > > > > Variant::Map options; > > Variant::Map selector; > > selector["selector"] = ""; > > options["link"] = selector; > > > > the other is a node type specification and looks like > > > > Variant::Map node_type; > > node_type["type"] = "queue"; > > options["node"] = node_type; // using "options" from previous example > > options["assert"] = "always"; > > > > Either one of these works on their own, but when combined the second > clause > > seems to be ignored and an attempt to access the queue name as an > exchange > > is made, which gets denied by the ACL. > > > > Am I doing something nonsensical like trying to combine > > AMQP-0.10/AMQP-1.0/JMS grammar? Ref: > > https://qpid.apache.org/releases/qpid-cpp-1.36.0/ > > messaging-api/book/section-addresses.html, > > it appears that the "selector" keyword does not exist at all - my use of > > this is based on this previous question > > http://qpid.2158936.n2.nabble.com/Property-names-in- > > subscription-selectors-td7625162.html#a7625166 > > (and nested links). > > > > Some clarification would be much appreciated! > > > > Chris > > > > -- > > > > *Chris Richardson*, System Architect > > c...@fourc.eu > > > > > > *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, > Norwaywww.fourc.eu > > <http://www.fourc.eu/>* > > > > *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook > > <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter > > <http://bit.ly/fourctw>!* > > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Address options grammar
Hi, Using the qpid-cpp-1.36 broker/client I am trying to specify two clauses on a receiver node (queue) address; one is a selector and looks something like this: Variant::Map options; Variant::Map selector; selector["selector"] = ""; options["link"] = selector; the other is a node type specification and looks like Variant::Map node_type; node_type["type"] = "queue"; options["node"] = node_type; // using "options" from previous example options["assert"] = "always"; Either one of these works on their own, but when combined the second clause seems to be ignored and an attempt to access the queue name as an exchange is made, which gets denied by the ACL. Am I doing something nonsensical like trying to combine AMQP-0.10/AMQP-1.0/JMS grammar? Ref: https://qpid.apache.org/releases/qpid-cpp-1.36.0/messaging-api/book/section-addresses.html, it appears that the "selector" keyword does not exist at all - my use of this is based on this previous question http://qpid.2158936.n2.nabble.com/Property-names-in-subscription-selectors-td7625162.html#a7625166 (and nested links). Some clarification would be much appreciated! Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Multiple connections with ssl client auth does not work as expected
Thanks to both of you Gordon and Jakub for your feedback. Following Gordon's comment showing a different NSS version from mine I've done some testing and it seems it is indeed down to the version of NSS. In particular it looks like NSS 3.28 does not work and NSS 3.31 does work. Here's what I did in a bit more detail: * Started with a fresh Ubuntu 16.04.2 server image * Installed apt packages 'build-essential cmake-curses-gui python-dev gyp ninja-build zlib1g-dev ruby libboost-program-options-dev libboost-system-dev uuid-dev libsasl2-dev libssl-dev pkg-config swig' * Installed (from source) NSS 3.31 * Installed (from source) qpid-proton 0.16 and qpid-python 1.36 * Installed qpid-cpp master branch (test passed) * Installed qpid-cpp 1.36.x branch (test passed) * Downgraded NSS to 3.28 (apt packages 'libnss3-dev libnss3-tools') * Re-installed the same qpid components (1.36.x branch) - TEST FAILED * Upgraded back to NSS 3.1, re-installed qpid-cpp (1.36.x) - test passed. Thanks again! Chris On 22 August 2017 at 22:39, Jakub Scholz <ja...@scholz.cz> wrote: > I did some more digging through my old work emails. We had a customer with > this kind of problem in June 2015. At that time I was able to reproduce > this behaviour on Linux (I believe they were Windows users, so it should > have affected also Windows). The problem was exactly the same as described > by Chris. > > I was able to find a very primitive workaround at that time ... the > "caching" was done based on hostname. So when two different hostnames were > used the caching did not happen and the connections worked fine. So the > workaround was to 1) create for each connection an artificial /etc/hosts > record to which will route to the same broker, 2) Disable SSL hostname > verification and 3) open each connection through a different hostname. Hey > it was ugly and a bit insecure ... but it worked as a quick fix :-). > > Later the customer got back to us with following info: > "After much investigation and experimentation, we have identified a > potential issue with the Qpid library implementation of SSL connections, > where the SSL session id might occasionally be "cached" or "reused". We > believe this is the cause of the ACL exceptions, since when the session Id > is cached and reused in an application that connects with multiple SSL > client certificates, the previously used client certificate information of > that previous SSL session might be reused as well for the next SSL > connection attempt. We have implemented a fix in our Qpid library and > application (by essentially disabling the caching of SSL session id for > each SSL connection attempt), and deployed them in our UAT environment > (user acceptance test). It seems to be working, as our application is able > to run with multiple SSL certificates now." > > I asked them to provide the patch to the Qpid community. But I'm not sure > they ever did - at least I cannot find anything. I'm not sure why I didn't > raised a QPID JIRA for it my self. :-( > > I probably haven't tried it since 2015. So if it works for Gordon it might > be already fixed in master. > > Regards > Jakub > > On Mon, Aug 21, 2017 at 11:18 PM, Jakub Scholz <ja...@scholz.cz> wrote: > >> This is definitely not new in 1.36. I think this is present "since the >> beginning". :-( >> >> If I remember correctly the problem was that the NSS library which handles >> the SSL certificates is initialised only once even when you create multiple >> connections. I was quite sure this was already discussed in some issue or >> on the mailing list but I have problems finding it now - I will keep >> looking. >> >> Regards >> Jakub >> >> On Mon, Aug 21, 2017 at 10:46 PM, Chris Richardson <c...@fourc.eu> wrote: >> >>> Hi, >>> >>> I've encountered an issue with the C++ broker (1.36) where it appears >>> that multiple connections within the same process using SSL client >>> auth do not use the "ssl-cert-name" property provided when the >>> connection instance is created. Rather, it appears the second >>> connection will use auth details of the first connection. >>> >>> Is this expected behaviour? >>> >>> Setup to show this happening is rather involved so I've created a jira >>> issue with some attached material to demonstrate the problem: >>> https://issues.apache.org/jira/browse/QPID-7894. >>> >>> Regards >>> >>> -- >>> Chris Richardson, System Architect >>> c...@fourc.eu >>> >>> FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway >>
Multiple connections with ssl client auth does not work as expected
Hi, I've encountered an issue with the C++ broker (1.36) where it appears that multiple connections within the same process using SSL client auth do not use the "ssl-cert-name" property provided when the connection instance is created. Rather, it appears the second connection will use auth details of the first connection. Is this expected behaviour? Setup to show this happening is rather involved so I've created a jira issue with some attached material to demonstrate the problem: https://issues.apache.org/jira/browse/QPID-7894. Regards -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu Follow us on LinkedIn, Facebook, Google+ and Twitter! - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
C++ management library 0.2.0 for C++ broker released
Hi all, For anyone who might find this useful, I've just released version 0.2.0 of our open C++ broker management library which now includes support for managing inter-broker routes (mirroring qpid-route functionality). Source: https://github.com/fourceu/fourc-qpid-manager/releases/tag/v0.2.0 Background: http://qpid.2158936.n2.nabble.com/C-broker-management-from-a-C-library-td7653957.html Regards -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: QMF Request from Qpid JMS
Hi mottese, I think I've actually read your post properly this time ;) You might find some info in this post from last December (this assumes you're using the C++ broker): http://qpid.2158936.n2.nabble.com/Qpid-QMF-with-qpid-jms-client-0-10-0-td7656627.html In that thread I attached a java class which does something similar for queues and exchanges using JMS and QMF. The OP also posted some feedback with an account of their experiences. Chris On 7 June 2017 at 21:03, mottese <mott...@sandia.gov> wrote: > I am using JMS 0.11.1 (Have to use Java 7) and was wondering how I can use > QMF to get a list of bindings from the broker. Using C++ I would have done: > > Sender s = session.createSender("qmf.default.direct/broker"); > > Message request; > request.setContentType("amqp/map"); > request.setProperty("x-amqp-0-10.app-id", "qmf2"); > request.setProperty("qmf.opcode", "_query_request"); > Variant::Map schemaId; > schemaId["_class_name"] = "binding"; > Variant::Map content; > content["_what"] = "OBJECT"; > content["_schema_id"] = schemaId; > > request.setContentObject(content); > s.send(request); > > How would I do this via JMS? Thanks > > > > -- > View this message in context: > http://qpid.2158936.n2.nabble.com/QMF-Request-from-Qpid-JMS-tp7663863.html > Sent from the Apache Qpid users mailing list archive at Nabble.com. > > ----- > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu Follow us on LinkedIn, Facebook, Google+ and Twitter! - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: C++ Messaging API: getReceiver(topic/subject)
I think you might mean to use createReceiver (to create a new receiver), not getReceiver (to get an existing one). Note the documentation for the getReceiver call specifies "@exception KeyError if there is none for that name." HTH Chris On 1 June 2017 at 16:13, mottese <mott...@sandia.gov> wrote: > Hi, I'm trying to use getReceiver, but am not getting the results I'm > expecting. For example, this is what I'm doing: > > Session session = con.getSession(); > Address addr("amq.topic/mySubject"); > session.createReceiver(addr); > ... > ... > Receiver r = session.getReceiver("amq.topic/mySubject"); > > This is throwing a KeyError and I can't figure out why. Any help would be > appreciated. > > > > > -- > View this message in context: > http://qpid.2158936.n2.nabble.com/C-Messaging-API-getReceiver-topic-subject-tp7663715.html > Sent from the Apache Qpid users mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu Follow us on LinkedIn, Facebook, Google+ and Twitter! - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Using QMF to know about consumers in a topic exchange
I wrote a library to support this type of thing a while ago. I've just added an example equivalent to Gordon's (there may be some slight similarities ;) ) to show the retrieval of bindings: https://github.com/fourceu/fourc-qpid-manager/blob/master/examples/Bindings.cxx The library hides the complications of the QMF RPC and hides complications like responses split across several messages (which will happen when you have more than ~100 bindings on your broker). Peer review of the library is more than welcome :) Chris On 26 May 2017 at 19:40, Gordon Sim <g...@redhat.com> wrote: > On 26/05/17 18:44, mottese wrote: >> >> Gordon Sim wrote >>> >>> If you run with >>> QPID_LOG_ENABLE=trace+ it should give some more insight into what was >>> going wrong over 1.0. >> >> >> I turned on trace+, but all I seem to be seeing is a whole bunch of the >> same >> message: >> >> [Broker] trace Dispatching to xxx#_yyy: 0 >> [Broker] trace Can't deliver to xxx#_yyy >> >> I looked earlier in the log and it looks like it was able to successfully >> create the response queue. > > > Sorry, my mistake. Not sure why the protocol was to 1.0 in that, it looks > like it was only used on 0-10. (It was a program I had lying around from a > previous request). > > Attached is a revised version that works with AMQP 1.0. It uses the > Receiver::getAddress() to correctly set the reply to address. I also updated > it to use the content object methods for the message rather than the 0-10 > specific encode/decode methods. > > I've tried it out myself this time before sending(!) and it works as > expected. > > > ----- > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu Follow us on LinkedIn, Facebook, Google+ and Twitter! - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Qpid tools with python vs qpid-cpp messaging
On 13 February 2017 at 14:55, Alan Conwaywrote: > On Thu, 2017-02-09 at 15:30 +, Gordon Sim wrote: >> On 09/02/17 15:20, Gordon Sim wrote: >> > It seems that the swigged c++ implementation is now prefered to the >> > pure >> > python on. Not sure why that was done; I hadn't even realised it >> > had >> > been done. >> >> This was done as the fix for >> https://issues.apache.org/jira/browse/QPID-7432, i.e. in order to >> allow >> management tools to work over AMQP 1.0. >> >> However, it does not take into account some differences between the >> c++ >> and python implementations, particularly for SSL configuration. >> >> It would be nice to make the preference for the swigged version >> somehow >> conditional. E.g. through a command line option that specifically >> requests AMQP 1.0. > > Gah, I had not realized that the SWIG client was not a drop-in > replacement even aside from protocol differences. > > If it was just the URL format, we could easily have the SWIG client use > the python URL parser instead of the C++ one, but if there are more > differences (SASL config??) then it is not simple at all. > > Can you make a list of all the things that differ between the clients > so we can figure out how to deal with this? It doesn't sounds as simple > as a command line flag for AMQP 1.0 if you need to replace your SASL > configuration as well just to change the protocol the management client > speaks. > I don't think that question was directed at me but I'll chuck in my tuppence-worth of "user experience" anyway... SSL config is definitely an issue, but luckily (for us) not a big one since as we already use a mix of the c++, python and Java clients I write our certificates to an NSS DB and PEM files (and JKS keystore/truststore files). This includes the SASL with SSL client auth config. With that already in place it's easy enough to choose a URL based on the result of attempting to use the python tools in bash eg: if [[ "$(qpid-config -b amqps://localhost)" =~ "Invalid URL" ]]; then # use the c++ client url syntax and SSL config from QPID_SSL* env vars else # use the python client url syntax and PEM files named in cli switches fi A bit hacky, but it works... albeit with AMQP1.0 in the c++ client and AMQP-0-10 in the python (I think). Gordon mentioned switching between the qpid_messaging and qpid.messaging clients; I was slightly confused by this but hadn't got around to inquiring further - more specific details would be really useful. Presumably one of them is the native python client and the other is the swigged wrapper? Are we talking about only qpid-cpp here or are is the build type of proton also relevant? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Qpid tools with python vs qpid-cpp messaging
Hi, Since the integration of the qpid-tools with the main qpid-cpp project in version 1.35 we've had a problem where in some environments the tools (tested: qpid-stat, qpid-config) now require the c++ client url format of : rather than the python format of amqp[s]://:, resulting in errors such as: $ qpid-stat -b amqp://localhost -e Failed: ConnectionError - Invalid URL: amqp://localhost (/home/chrisr/projects/qpid/qpid-cpp/qpid-cpp-1.36.0/src/qpid/Url.cpp:248) Confusingly, this hasn't happened on all environments so I'm assuming some factor such as installation order, previous versions or the selected Swig bindings affects the outcome. Changing to the c++ syntax fixes the problem for plaintext connections but I'm unsure how to formulate an ssl connection since there is no way I can see to pass "--connection-options={transport:ssl}" as one would for a c++ client and the following python-esque syntax also fails: $ qpid-stat -b localhost:5671 --ssl-certificate cert.pem --ssl-key key.pem -e Failed: MessagingError - Invalid option: ssl_certfile not recognised (/home/chrisr/projects/qpid/qpid-cpp/qpid-cpp-1.36.0/src/qpid/client/amqp0_10/ConnectionImpl.cpp:195) There seems to be an API mismatch here and even if there wasn't I can't see how it would work since I assume the c++ client would expect to get its certs from an NSS db specified by the QPID_SSL_* env vars. Could you please shed some light on what's going on here and how we should proceed? Thanks in advance -- Chris Richardson, System Architect c...@fourc.eu FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norway www.fourc.eu Follow us on LinkedIn, Facebook, Google+ and Twitter! - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Patch + jira vs pull request
I have been wondering about this too and would definitely give a +1 to the pull request approach if it's an option. /C On 28 January 2017 at 08:51, Adel Boutros <adelbout...@live.com> wrote: > Hello, > > For some time we have been submitting jira issues with patches for > problems we have detected. However, it is also possible to perform a pull > request on the github repositories. > > So I was wondering which method you prefer and why? > > Regards, > Adel > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: qpid-cpp-1.36.0 build error with nss-3.28
On 18 January 2017 at 10:24, Gordon Sim <g...@redhat.com> wrote: > On 17/01/17 18:35, Chris Richardson wrote: > >> I'd like to request 2 changes in qpid to ease the issue, if I could... >> > > Could you create a JIRA (or perhaps two of them) and attach the patches to > those? Thanks! > Done! https://issues.apache.org/jira/browse/QPID-7629 https://issues.apache.org/jira/browse/QPID-7630 > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Qpid QMF with qpid-jms-client 0.10.0
Hi Filip, I think I might have had the same problem as you in the past because the methods (I think) are required are not available on the JMS interface. My solution was a rather hacky approach of casting the JMS instance to the actual Qpid type, as in the following examples: private void method(String method, Map arguments, String address) throws JMSException {Destination reply_to = session.createTemporaryQueue();MessageConsumer receiver = session.createConsumer(reply_to);JmsMapMessage request = (JmsMapMessage) session.createMapMessage(); request.setValidatePropertyNames(false); request.setStringProperty("x-amqp-0-10.app-id", "qmf2"); request.setStringProperty("qmf.opcode", "_method_request"); request.setStringProperty("method", "request");Map object_id_map = new HashMap(); object_id_map.put("_object_name", address); request.setString("_method_name", method);// Can't set map directly on object, so we'll hack it hereJmsMapMessageFacade facade = (JmsMapMessageFacade)request.getFacade(); facade.put("_object_id", object_id_map); facade.put("_arguments", arguments);int correlation_id = sendMessage(request, reply_to);List response = awaitResponse(receiver, correlation_id, 10 * 1000);if (response != null && response.size() > 0) {new ResponseDecoder().decodeResponse(response);} else { throw new JMSException("No response received");}} private int sendRequest(String opcode, Map<String, ?> query, Destination replyAddress) throws JMSException {JmsMapMessage request = (JmsMapMessage) session.createMapMessage(); request.setValidatePropertyNames(false); request.setStringProperty("x-amqp-0-10.app-id", "qmf2"); request.setStringProperty("qmf.opcode", opcode);// Can't set map directly on object, so we'll hack it here JmsMapMessageFacade facade = (JmsMapMessageFacade)request.getFacade(); for (Map.Entry<String, ?> entry : query.entrySet()) { facade.put(entry.getKey(), entry.getValue());}return sendMessage(request, replyAddress);} Unfortunately I haven't structured our Java implementation of the QMF2 interface as well as I could have done and it's rather entrenched in another of our libraries but I'm sure no one will mind if I share the attached implementation (based on the Qpid Python implementation) for reference. /Chris On 19 December 2016 at 14:14, Filip Nguyen <fngu...@redhat.com> wrote: > We currently use Java JMS qpid-client 0.32. It is quite easy to use QMF > [1] just by using MapMessage etc. > > With new Java Qpid JMS client org.apache.qpid:qpid-jms-client:jar:0.10.0 > I couldn't find a way how to use Qpid QMF. Is there any existing usage or > example how to do that? > > [1] > > MapMessage request = session.createMapMessage(); > request.setJMSReplyTo(responseQueue); > request.setStringProperty("x-amqp-0-10.app-id", "qmf2"); > request.setStringProperty("qmf.opcode", "_query_request"); > > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!* /* * Copyright (C) 2015 FourC AS, http://www.fourc.eu/ * All Rights Reserved. */ package eu.fourc.messenger.fmf.impl.java; import com.google.common.base.MoreObjects; import eu.fourc.messenger.fmf.Exchange; import eu.fourc.messenger.fmf.Node; import eu.fourc.messenger.fmf.Queue; import eu.fourc.messenger.impl.ConnectionImpl; import org.apache.qpid.jms.JmsTopic; import org.apache.qpid.jms.message.JmsMapMessage; import org.apache.qpid.jms.message.JmsMessage; import org.apache.qpid.jms.message.facade.JmsMapMessageFacade; import org.apache.qpid.jms.provider.amqp.message.AmqpJmsMessageFacade; import javax.jms.*; import java.util.HashMap; import java.util.LinkedList; import java.util.List; import java.util.Map; public class BrokerAgent { private static final String TARGET_ADDRESS = "qmf.default.direct"; private static final String TARGET_SUBJECT = "broker"; private Session session; private int correlator = 1; // Maybe use atomic int? public BrokerAgent(Connection connection) throws JMSException { Connection conn = connection instanceof ConnectionImpl ? ((ConnectionImpl)co
Re: [ANNOUNCE] Apache Qpid C++ 1.36.0 released
FYI Gentoo Portage ebuilds available in the same place as my earlier comment for the Proton 16 release: https://github.com/fourceu/ qpid-portage-overlay Chris On 14 December 2016 at 20:00, Robbie Gemmell <rob...@apache.org> wrote: > The Apache Qpid community is pleased to announce the immediate > availability of Apache Qpid C++ 1.36.0. > > Apache Qpid (http://qpid.apache.org) is a cross-platform enterprise > messaging solution which implements the Advanced Message Queuing Protocol > (AMQP, http://www.amqp.org). It provides message brokers written in C++ > and > Java, a message router, and messaging APIs for C, C++, Java, JavaScript, > JMS, .NET, Perl, Python, and Ruby. > > This release provides an update of the Qpid C++ broker and the > C++ Qpid Messaging API with its related language bindings. > > The release is available now from our website: > > http://qpid.apache.org/download.html > > The release includes many bug fixes and improvements: > > http://qpid.apache.org/releases/qpid-cpp-1.36.0/release-notes.html > > Thanks to all involved, > Robbie > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: [ANNOUNCE] Apache Qpid Proton 0.16.0 released
FYI also: Gentoo Portage ebuilds available here: https://github.com/fourceu/qpid-portage-overlay Chris On 13 December 2016 at 19:37, Ken Giusti <kgiu...@redhat.com> wrote: > FYI: > > The python bindings are available via pypi: > > https://pypi.python.org/pypi/python-qpid-proton/0.16.0 > > -K > > > - Original Message - > > From: "Robbie Gemmell" <rob...@apache.org> > > To: annou...@apache.org, d...@qpid.apache.org, users@qpid.apache.org > > Sent: Tuesday, December 13, 2016 1:57:06 PM > > Subject: [ANNOUNCE] Apache Qpid Proton 0.16.0 released > > > > The Apache Qpid (http://qpid.apache.org) community is pleased to > announce > > the immediate availability of Apache Qpid Proton 0.16.0. > > > > Apache Qpid Proton is a messaging library for the Advanced Message > Queuing > > Protocol 1.0 (AMQP 1.0, ISO/IEC 19464, http://www.amqp.org). It can be > used > > in a wide range of messaging applications including brokers, clients, > > routers, bridges, proxies, and more. > > > > The release is available now from our website: > > http://qpid.apache.org/download.html > > > > Proton-J is also available via Maven Central: > > http://qpid.apache.org/maven.html > > > > Release notes can be found at: > > http://qpid.apache.org/releases/qpid-proton-0.16.0/ > release-notes.html > > > > Thanks to all involved, > > Robbie > > > > - > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > > > -- > -K > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: qpid-cpp-1.35 linearstore build error
Since this is more a question about cmake/c++ and not strictly a question about Qpid I'll break userlist etiquette (again) and, with apologies to the real Qpid team, stick my nose in where it doesn't belong... I've had this same problem installing on Gentoo since if your BerkeleyDB is not installed in one of the following paths (as yours is not) it will not be found by the qpid-cpp build system: /usr/local/include/db4 /usr/local/include/libdb4 /usr/local/include /usr/include/db4 /usr/include/libdb4 /usr/include You can see how I've solved this on Gentoo here: https://github.com/fourceu/qpid-portage-overlay/blob/master/net-misc/qpid-cpp/qpid-cpp-0.34-r1.ebuild Line 56 begins a block which finds the relevant headers and passes them to the Qpid build system. In your case a similar solution might be to add "-DDB_CXX_INCLUDE_DIR=/usr/include/db43" to your cmake command line, or set the variable in the CMake GUI if that's what you're using. HTH /Chris On 7 December 2016 at 00:53, rammohan ganapavarapu <rammohanga...@gmail.com> wrote: > Hi, > > I am trying to build linear-store for qpid-cpp-1.35 version and i am > getting bellow error. > > -- Legacystore requires BerkeleyDB for C++ which is absent. > -- Legacystore is excluded from build. > -- Linearstore requires BerkeleyDB for C++ which is absent. > CMake Error at src/linearstore.cmake:65 (message): > Linearstore requires BerkeleyDB for C++ which is absent. > Call Stack (most recent call first): > src/CMakeLists.txt:1274 (include) > > > > I have installed necessary rpms (db4), is there a way for cmake to tell > where to find db4 libraries? any options i can use to make it work? > > > ec2-user@ip-10-17-8-126:~/files/rpmbuild/SPECS> rpm -ql db43-4.3.29-125.17 > /usr/lib64/libdb-4.3.so > /usr/lib64/libdb_cxx-4.3.so > /usr/share/doc/packages/db43 > /usr/share/doc/packages/db43/LICENSE > /usr/share/doc/packages/db43/README > /usr/share/doc/packages/db43/images > /usr/share/doc/packages/db43/images/api.gif > /usr/share/doc/packages/db43/images/next.gif > /usr/share/doc/packages/db43/images/prev.gif > /usr/share/doc/packages/db43/images/ps.gif > /usr/share/doc/packages/db43/images/ref.gif > /usr/share/doc/packages/db43/images/sleepycat.gif > /usr/share/doc/packages/db43/index.html > /usr/share/doc/packages/db43/sleepycat > /usr/share/doc/packages/db43/sleepycat/legal.html > /usr/share/doc/packages/db43/sleepycat/license.html > ec2-user@ip-10-17-8-126:~/files/rpmbuild/SPECS> rpm -ql > db43-devel-4.3.29-125.17 > /usr/include/db43 > /usr/include/db43/db.h > /usr/include/db43/db_185.h > /usr/include/db43/db_cxx.h > /usr/lib64/libdb-4.3.a > /usr/lib64/libdb_cxx-4.3.a > > Thanks, > Ram > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: qpid-cpp-1.35 rpm build for SUSE
Ram, I've tried to reproduce your problem with no success. Here are the steps I took: * Created a new EC2 instance of SLES 12 SP1, ami-cae4b7dd on am3.large dual core instance type * Installed some prerequisites sudo zypper install cmake sudo zypper install gcc-c++ * Downloaded and built qpid-proton-0-14 as follows: mkdir -p proton/build cd proton wget http://archive.apache.org/dist/qpid/proton/0.14.0/qpid-proton-0.14.0.tar.gz tar zxf qpid-proton-0.14.0.tar.gz cd build/ cmake ../qpid-proton-0.14.0 make -j3 This built with no problems. Then I noticed from the pathnames in your original post that you're using rpmbuild so I tried that too. I haven't used rpmbuild before but it seems you need a specfile, so I hashed one up (see attached). Then I ran $rpmbuild -ba qpid-proton-0.14.0.spec and the build succeded again, terminating with the messages: Wrote: /home/ec2-user/rpmbuild/SRPMS/qpid-proton-0.14.0-1.src.rpm Wrote: /home/ec2-user/rpmbuild/RPMS/x86_64/qpid-proton-0.14.0-1.x86_64.rpm In both cases the compilation of connection.cpp (which failed in your build) happened earlier in my output (27%) than in yours so I guess you're including some extra cmake settings you haven't mentioned. This could affect the build outcome. Another possibility common to c++ it that your build system is "dirty" in some way, for instance if you have a previous version of proton already installed. That could result in the compiler picking up a conflicting version of a header file from eg: /usr/include instead of the one it should find in your build tree. Might be worth trying this again on a clean system. Hope that helps! Chris On 21 November 2016 at 16:32, rammohan ganapavarapu <rammohanga...@gmail.com > wrote: > Chris, > > Thanks for trying to help me, below are my env details. > > OS: SUSE Linux Enterprise Server 12 SP1 (x86_64) - Kernel \r (\l). > cmake -version > cmake version 2.8.12.1 > gcc version 4.8.5 (SUSE Linux) > > > > Ram > > On Fri, Nov 18, 2016 at 6:16 PM, Chris Richardson <c...@fourc.eu> wrote: > > > Hi Ram, > > > > It looks like you're not entirely alone with this problem: > > http://stackoverflow.com/questions/39708294/error-changes-meaning-when- > > installing-apache-qpid > > notably also SUSE, unfortunately no solution posted. > > > > May I suggest you post some more info about your environment, > particularly > > arch (whether you're on 32 or 64 bit) and what compiler (incl. version) > > you're using. Steps to reproduce would also help. Unfortunately I don't > > have SUSE but I'd be happy to test on the distros I do have if it's of > any > > benefit. > > > > /Chris > > > > > > > > On 18 November 2016 at 23:34, rammohan ganapavarapu < > > rammohanga...@gmail.com > > > wrote: > > > > > Hi, > > > > > > If any one built qpid-cpp rpms for suse please share your experience, > if > > > you already have a rpms to download please share. > > > > > > Ram > > > > > > On Tue, Nov 8, 2016 at 8:19 AM, rammohan ganapavarapu < > > > rammohanga...@gmail.com> wrote: > > > > > > > Hi, > > > > > > > > I am trying to build qpid-cpp v1.35 rpms for SUSE, while i am > building > > > > qpid-proton rpm i am getting this compilation error any idea why i am > > > > getting this error and how to fix it? > > > > > > > > [ 34%] Building CXX object proton-c/bindings/cpp/ > > > > CMakeFiles/qpid-proton-cpp.dir/src/connection.cpp.o > > > > In file included from /mnt/ec2-user/rpmbuild/BUILD/ > > > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > > > ././link.hpp:31, > > > > from /mnt/ec2-user/rpmbuild/BUILD/ > > > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > > > > ./receiver.hpp:27, > > > > from /mnt/ec2-user/rpmbuild/BUILD/ > > > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > > > session.hpp:27, > > > > from /mnt/ec2-user/rpmbuild/BUILD/ > > > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/ > > > connection.hpp:28, > > > > from /mnt/ec2-user/rpmbuild/BUILD/ > > > > qpid-proton-0.14.0/proton-c/bindings/cpp/src/connection.cpp:24: > > > > /mnt/ec2-user/rpmbuild/BUILD/qpid-proton-0.14.0/proton-c/ > > > > bindings/cpp/include/proton/././././sender_options.hpp:87: error: > > > > declaration of ‘proton::sender_options& proton::sender_options:: > > > > delivery_mode(proton::delivery_mode)’ > &g
Re: qpid-cpp-1.35 rpm build for SUSE
Hi Ram, It looks like you're not entirely alone with this problem: http://stackoverflow.com/questions/39708294/error-changes-meaning-when-installing-apache-qpid notably also SUSE, unfortunately no solution posted. May I suggest you post some more info about your environment, particularly arch (whether you're on 32 or 64 bit) and what compiler (incl. version) you're using. Steps to reproduce would also help. Unfortunately I don't have SUSE but I'd be happy to test on the distros I do have if it's of any benefit. /Chris On 18 November 2016 at 23:34, rammohan ganapavarapu <rammohanga...@gmail.com > wrote: > Hi, > > If any one built qpid-cpp rpms for suse please share your experience, if > you already have a rpms to download please share. > > Ram > > On Tue, Nov 8, 2016 at 8:19 AM, rammohan ganapavarapu < > rammohanga...@gmail.com> wrote: > > > Hi, > > > > I am trying to build qpid-cpp v1.35 rpms for SUSE, while i am building > > qpid-proton rpm i am getting this compilation error any idea why i am > > getting this error and how to fix it? > > > > [ 34%] Building CXX object proton-c/bindings/cpp/ > > CMakeFiles/qpid-proton-cpp.dir/src/connection.cpp.o > > In file included from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > ././link.hpp:31, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > > ./receiver.hpp:27, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > session.hpp:27, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/ > connection.hpp:28, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/src/connection.cpp:24: > > /mnt/ec2-user/rpmbuild/BUILD/qpid-proton-0.14.0/proton-c/ > > bindings/cpp/include/proton/././././sender_options.hpp:87: error: > > declaration of ‘proton::sender_options& proton::sender_options:: > > delivery_mode(proton::delivery_mode)’ > > /mnt/ec2-user/rpmbuild/BUILD/qpid-proton-0.14.0/proton-c/ > > bindings/cpp/include/proton/./././././delivery_mode.hpp:30: error: > > changes meaning of ‘delivery_mode’ from ‘struct proton::delivery_mode’ > > In file included from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > ././link.hpp:32, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > > ./receiver.hpp:27, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/./ > session.hpp:27, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/include/proton/ > connection.hpp:28, > > from /mnt/ec2-user/rpmbuild/BUILD/ > > qpid-proton-0.14.0/proton-c/bindings/cpp/src/connection.cpp:24: > > /mnt/ec2-user/rpmbuild/BUILD/qpid-proton-0.14.0/proton-c/ > > bindings/cpp/include/proton/././././receiver_options.hpp:83: error: > > declaration of ‘proton::receiver_options& proton::receiver_options:: > > delivery_mode(proton::delivery_mode)’ > > /mnt/ec2-user/rpmbuild/BUILD/qpid-proton-0.14.0/proton-c/ > > bindings/cpp/include/proton/./././././delivery_mode.hpp:30: error: > > changes meaning of ‘delivery_mode’ from ‘struct proton::delivery_mode’ > > make[2]: *** [proton-c/bindings/cpp/CMakeFiles/qpid-proton-cpp. > dir/src/connection.cpp.o] > > Error 1 > > make[1]: *** [proton-c/bindings/cpp/CMakeFiles/qpid-proton-cpp.dir/all] > > Error 2 > > > > > > Ram > > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
C++ broker management from a C++ library
Hi, I noticed recently there was a question about how to do this ( http://qpid.2158936.n2.nabble.com/How-from-within-a-C-application-is-it-possible-to-examine-and-manage-a-broker-td7652451.html) which happened to be posted at exactly the time I'd finished implementing a small feature subset in C++. This was based on the python implementation exactly as Steve mentioned in the above post. Since it seemed we at FourC are not the only ones interested in this functionality I've carved the code out of our libraries, expanded the scope to include as much of the QMF RPC interface as I could and released it under LGPL3 as a library here https://github.com/fourceu/fourc-qpid-manager >From a user perspective it's simply a case of creating an instance of a fourc::fmf::BrokerAgent, which requires a connected qpid::messaging::Session for its constructor. The interface of this class is similar to the Qpid Python one and should be pretty self-explanatory. The functionality offered includes making ACL queries, triggering an ACL reload, querying for broker, connection, exchange, link, memory, session and subscriber stats and querying/creating/deleting exchanges, queues and bindings. Stub implementations of the Cluster and HaBroker classes are included but they don't do anything because I haven't yet built a cluster to work with! Internally the library uses the QMF interface and is very closely modelled on the Python implementation; after that things get ugly as of course C++ doesn't support some of the nice Python deserialisation features, so the rest of the library is pairs of classes and related decoders. It includes a fairly comprehensive suite of tests which execute on Travis CI so my hope is that even though the code is very new it should actually work! If there's sufficient interest I can add more documentation and some examples. One feature we would really like which is not present is the ability to modify ACLs "in memory", so to speak, rather than making modifications to the ACL file and then triggering a reload. Could anyone provide more information about whether this is possible and if so how? I hope this library will be of use to someone but of course the main reason for publishing it is to end up with the best possible implementation so feedback and contributions will be very warmly welcomed! -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Re: [Qpid C++ Broker] Why broker link state retains connecting when I setup Broker Federation in windows using qpid-route ?
I had some trouble setting this up too and one of my mistakes was that I was using server-only certificates (as you state you are doing in your original post). When a broker establishes a link it uses its configured cert as a _client_ cert, so the cert must be valid for that purpose also. HTH On 13 November 2016 at 05:08, lei@airlink-software.com < lei@airlink-software.com> wrote: > Hi, > > Thank you for the reply. > > As you said, I tried to set "auth=on" and provide an ACL with only one > line "acl allow all all", but got the same result. > I also tried to read the logs, and found that when use linux version, > there are logs like: > > 2016-11-11 18:27:34 [Broker] info [0x7f13f308e700] > /home/alex/qpid-cpp-1.35.0/src/qpid/broker/Link.cpp:240: Inter-broker > link established to 172.16.28.144:5672 > 2016-11-11 18:27:34 [Model] debug [0x7f13f308e700] > /home/alex/qpid-cpp-1.35.0/src/qpid/broker/amqp_0_10/Connection.cpp:329: > Create connection. user:@QPID rhost:qpid.tcp:172.16.28.144:5672 > 2016-11-11 18:27:34 [System] debug [0x7f13f308e700] > /home/alex/qpid-cpp-1.35.0/src/qpid/sys/AsynchIOHandler.cpp:102: SENT > [qpid.tcp:172.16.28.144:5672]: INIT(0-10) > > But when use windows version, these logs disappeared. > It seems that the destination node didn't send protocol negotiation to the > source node for some reasons in windows, but I don't know what the reasons > are. > Now I'm trying to read the source code of Qpid C++ Broker but it's a > little difficult to me... > > Regards, > > > Lei Dai > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Possible thread sync error in proton/qpid-cpp
Hi, As per the subject I'm experiencing a crash which I am _reeaasonably_ sure I've pinpointed to a problem with one of the C/C++ Qpid client libraries. Of course I could also be doing something goofy in which case I apologise profusely! The problem manifests in a number of different ways including a double-free and stack trace, an error message (when built in debug) reading proton-c/src/object/object.c:236: pn_object_decref: Assertion `head->refcount > 0' failed. and a simple segfault with no further info. I've examined some core dumps which give some further hints about the problem but I won't include results here - instead I've tried to distil the problem down to the attached source. Two things seem to be key - using AMQP1.0 (0-10 is fine) and creating and closing sessions very fast while sending some message over the link. I realise this may not be typical usage but it does happen in my integration testing and causes my tests to crash! The versions I've tested with are: qpid-proton-0.9.1-rc1 / qpid-cpp-0-34 qpid-proton-0.14.0 / qpid-cpp-1.35 Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0 20160609 I've been testing against c++ broker 0.34 and 1.35 fairly indiscriminately on the assumption it's not really relevant to the crash. Sample code - build with eg: g++ -g -std=c++11 -o qpid-test main.cxx /usr/local/lib/libqpidmessaging.so /usr/local/lib/libqpidtypes.so Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!* #include #include #include #include #include #include #include const std::string CONNECTION_URL = ""; const std::string CONNECTION_OPTIONS = "{transport:ssl, protocol:amqp1.0}"; const std::string SENDER_ARGS = ";{ create: always, node: { type: queue, x-declare: { auto-delete: true }}}"; const std::string QUEUE_NAME = "TestQueue"; int main(int, char**) { qpid::messaging::Connection connection(CONNECTION_URL, CONNECTION_OPTIONS); connection.open(); int i = 0; while (true) { std::cout << "Creating session, itr: " << ++i << std::endl; auto session = connection.createSession(); qpid::messaging::Message message; auto sender = session.createSender(QUEUE_NAME + SENDER_ARGS); auto receiver = session.createReceiver(QUEUE_NAME); sender.send(message); receiver.fetch(message); session.sync(); session.close(); } } - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Setting log level on running broker
That looks perfect (and much more aesthetically pleasing than my version). Can we have an official release soon? On 20 January 2016 at 17:12, Gordon Sim <g...@redhat.com> wrote: > On 01/20/2016 05:05 PM, Chris Richardson wrote: > >> Hi, >> >> It would be really nice to have a command-line method for setting the >> log level on a running broker (c++), as discussed here: >> http://grokbase.com/t/qpid/users/11bxbmbxq3/set-log-level >> >> The suggested solution with qpid-ctrl tool does work, but since this is, >> as mentioned in the article, a test tool and also doesn't support ssl >> connections, it doesn't quite fit the bill for many scenarios. >> >> Would it be possible to add this feature to qpid-config, which already >> has almost everything that's required in place? I've attached a patch to >> qpid-tools-0.32 which fulfils at least my wishes - as you can see only a >> handful of actual code lines. >> > > Actually Alan committed something similar early last year: > > http://svn.apache.org/viewvc?view=revision=1676333 > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Setting log level on running broker
Hi, It would be really nice to have a command-line method for setting the log level on a running broker (c++), as discussed here: http://grokbase.com/t/qpid/users/11bxbmbxq3/set-log-level The suggested solution with qpid-ctrl tool does work, but since this is, as mentioned in the article, a test tool and also doesn't support ssl connections, it doesn't quite fit the bill for many scenarios. Would it be possible to add this feature to qpid-config, which already has almost everything that's required in place? I've attached a patch to qpid-tools-0.32 which fulfils at least my wishes - as you can see only a handful of actual code lines. Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!* diff -ur qpid-tools-0.32/src/py/qpid-config qpid-tools-0.32-loglevel/src/py/qpid-config --- qpid-tools-0.32/src/py/qpid-config 2015-02-11 20:43:53.0 + +++ qpid-tools-0.32-loglevel/src/py/qpid-config 2016-01-20 16:51:36.508006692 + @@ -48,6 +48,8 @@ qpid-config [OPTIONS] add [--argument =] qpid-config [OPTIONS] del qpid-config [OPTIONS] list [--show-property ] +qpid-config [OPTIONS] get-loglevel +qpid-config [OPTIONS] set-loglevel qpid-config [OPTIONS] shutdown""" description = """ @@ -57,6 +59,7 @@ $ qpid-config add exchange direct d -a localhost:5672 $ qpid-config exchanges -b 10.1.1.7:1 $ qpid-config queues -b guest/guest@broker-host:1 +$ qpid-config set-loglevel trace+:Security Add Exchange values: @@ -78,6 +81,14 @@ none - no replication configuration - replicate queue and exchange existence and bindings, but not messages. all- replicate configuration and messages + +Log levels: +A log level is a set of comma-separated rules, where a rule is in the form 'LEVEL[+-][:PATTERN]'. +LEVEL is one of: trace debug info notice warning error critical +PATTERN is a logging category name, or a namespace-qualified function name or name fragment. +Logging category names are: +Security Broker Management Protocol System HA Messaging Store +Network Test Client Application Model Unspecified """ REPLICATE_LEVELS= ["none", "configuration", "all"] @@ -611,7 +622,7 @@ declArgs[r[0]] = get_value(r) if config._durable: -# allow the default fileCount and fileSize specified +# allow the default fileCount and fileSize specified # in qpid config file to take prededence if config._fileCount: declArgs[FILECOUNT] = config._fileCount @@ -666,8 +677,6 @@ Usage() qname = args[0] self.broker.delQueue(qname, if_empty=config._if_empty, if_unused=config._if_unused) - - def Bind(self, args): if len(args) < 2: @@ -716,6 +725,12 @@ key = args[2] self.broker.unbind(ename, qname, key) +def GetLogLevel(self): +print self.broker.getLogLevel()['level'] + +def SetLogLevel(self, level): + self.broker.setLogLevel(level) + def ReloadAcl(self): try: self.broker.reloadAclFile() @@ -798,6 +813,13 @@ bm.Unbind(args[1:]) elif cmd == "reload-acl": bm.ReloadAcl() +elif cmd == "get-loglevel": +bm.GetLogLevel() +elif cmd == "set-loglevel": +if modifier != "": +bm.SetLogLevel(modifier) +else: +Usage() elif cmd == "list" and len(args) > 1: # fetch objects objects = bm.broker.list(modifier) diff -ur qpid-tools-0.32/src/py/qpidtoollibs/broker.py qpid-tools-0.32-loglevel/src/py/qpidtoollibs/broker.py --- qpid-tools-0.32/src/py/qpidtoollibs/broker.py 2014-06-02 14:56:12.0 +0100 +++ qpid-tools-0.32-loglevel/src/py/qpidtoollibs/broker.py 2016-01-20 15:40:19.955012325 + @@ -222,7 +222,7 @@ def getLogLevel(self): """Get the log level""" -return self._method('getLogLevel') +return self._method('getLogLevel', {}) def setTimestampConfig(self, receive): """Set the message timestamping configuration""" - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Setting log level on running broker
Hi Chuck, Unfortunately that's only partially true - while the Logger.cpp changes were indeed released in qpid-cpp-0.34, the python file changes are key to this functionality and are part of the qpid-tools package, which has not been released since version 0.32. What I'm after is a 0.34 release of the python components (client and tools). These releases would be doubly appreciated because we're waiting on another issue which is affecting us and is fixed upstream from 0.32 ( https://issues.apache.org/jira/browse/QPID-6326). Cheers Chris On 20 January 2016 at 21:39, Chuck Rolke <cro...@redhat.com> wrote: > Alan's commit db7b30e was released in qpid-cpp-0.34 July, 2015. > > Further improvements to that file by Ken are in commit 2d9f712 > on trunk. They have not been in a formal release. > > - Original Message - > > From: "Chris Richardson" <c...@fourc.eu> > > To: "users" <users@qpid.apache.org> > > Sent: Wednesday, January 20, 2016 12:21:41 PM > > Subject: Re: Setting log level on running broker > > > > That looks perfect (and much more aesthetically pleasing than my > version). > > Can we have an official release soon? > > > > On 20 January 2016 at 17:12, Gordon Sim <g...@redhat.com> wrote: > > > > > On 01/20/2016 05:05 PM, Chris Richardson wrote: > > > > > >> Hi, > > >> > > >> It would be really nice to have a command-line method for setting the > > >> log level on a running broker (c++), as discussed here: > > >> http://grokbase.com/t/qpid/users/11bxbmbxq3/set-log-level > > >> > > >> The suggested solution with qpid-ctrl tool does work, but since this > is, > > >> as mentioned in the article, a test tool and also doesn't support ssl > > >> connections, it doesn't quite fit the bill for many scenarios. > > >> > > >> Would it be possible to add this feature to qpid-config, which already > > >> has almost everything that's required in place? I've attached a patch > to > > >> qpid-tools-0.32 which fulfils at least my wishes - as you can see > only a > > >> handful of actual code lines. > > >> > > > > > > Actually Alan committed something similar early last year: > > > > > > http://svn.apache.org/viewvc?view=revision=1676333 > > > > > > > > > - > > > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > > > For additional commands, e-mail: users-h...@qpid.apache.org > > > > > > > > > > > > -- > > > > *Chris Richardson*, System Architect > > c...@fourc.eu > > > > > > *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, > Norwaywww.fourc.eu > > <http://www.fourc.eu/>* > > > > *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook > > <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter > > <http://bit.ly/fourctw>!* > > > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
AMQP 0.10 subscription node type
Hi, Can you please give me some help with the address syntax for specifying node type in amqp 0.10? What I've tried is queue-name; {node: {type: queue}} but if I understand correctly this is an amqp 1.0 syntax...? I've tried various combinations of x-declare, x-subscribe and x-bindings but haven't managed to strike gold yet. Background: The problem I'm having is very similar to this one Jakub helped me with recently: http://qpid.2158936.n2.nabble.com/AMQP-1-0-appears-to-access-exchange-as-queue-even-after-successful-ACL-match-td7633652.html only now I'm using the python client (so amqp-0.10 instead of amqp1.0) and I'm trying to avoid the subscription process attempting to access my queue name as an exchange rather than visa versa. Any help very much appreciated! -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: AMQP 0.10 subscription node type
Thanks Robbie, that looks like it - I've tested and the problem appears to be fixed in trunk. On 8 January 2016 at 12:05, Robbie Gemmell <robbie.gemm...@gmail.com> wrote: > I know little about the Python client but I recalled seeing something > mentioned along these lines. After a bit of searching I'm thinking > this looks like https://issues.apache.org/jira/browse/QPID-6326 ? > > It hasn't made it into a release yet and the JIRA isnt marked > resolved, but there is a commit against it so trying the code from > trunk (or seeing if the patch applies to what you are using) to see if > it helps might be an idea. > > Robbie > > On 8 January 2016 at 11:06, Chris Richardson <c...@fourc.eu> wrote: > > Hi, > > > > Can you please give me some help with the address syntax for specifying > > node type in amqp 0.10? What I've tried is > > queue-name; {node: {type: queue}} > > but if I understand correctly this is an amqp 1.0 syntax...? > > > > I've tried various combinations of x-declare, x-subscribe and x-bindings > > but haven't managed to strike gold yet. > > > > Background: > > The problem I'm having is very similar to this one Jakub helped me with > > recently: > > > http://qpid.2158936.n2.nabble.com/AMQP-1-0-appears-to-access-exchange-as-queue-even-after-successful-ACL-match-td7633652.html > > only now I'm using the python client (so amqp-0.10 instead of amqp1.0) > and > > I'm trying to avoid the subscription process attempting to access my > queue > > name as an exchange rather than visa versa. > > > > Any help very much appreciated! > > > > -- > > > > *Chris Richardson*, System Architect > > c...@fourc.eu > > > > > > *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, > Norwaywww.fourc.eu > > <http://www.fourc.eu/>* > > > > *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook > > <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter > > <http://bit.ly/fourctw>!* > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
AMQP 1.0 appears to access exchange as queue even after successful ACL match
Hi, I'd like to tentatively suggest there may be a bug in the process of qpid-send transferring a message to a broker using AMQP 1.0. This has come to light because I'm using a fairly locked-down ACL which is intended to allow clients only exactly the access they require. The simplified situation is that I have a 0.34 broker with a topic exchange ("test.topic") and an ACL which allows my user to access and publish to the exchange, something like this: #start acl acl allow client@QPID access exchange name=test.topic acl allow client@QPID publish exchange name=test.topic acl deny-log all all #end acl I'm sending a message to the broker with: qpid-send -b broker:5671 --connection-options "{transport:ssl,protocol:amqp1.0}" -a "test.topic" --content-string "hello" and getting the error message: qpid-send: Link detached by peer with amqp:unauthorized-access: ACL denied access request to test.topic from client@QPID (/src/qpid/broker/amqp/Authorise.cpp:144) The broker log shows: [Security] info qpid.192.168.0.221:5671-192.168.0.221:56953 Authenticated as client@QPID [Security] debug ACL: Lookup for id:client@QPID action:access objectType:exchange name:test.topic with params { } [Security] debug ACL: checking rule [rule 1 ruleMode = allow props{ name=test.topic }] [Security] debug ACL: lookup name 'test.topic' matched with rule name 'test.topic' [Security] debug ACL: Successful match, the decision is:allow [Security] debug ACL: Lookup for id:client@QPID action:access objectType:queue name:test.topic with params { } [Security] debug ACL: No successful match, defaulting to the decision mode deny-log Notice that access is correctly approved with object type 'exchange', but that the process nevertheless continues to try with object type 'queue' instead, which then fails. I've tried being explicit with the node type by using the address ' test.topic;{node:{type:exchange}}' but that produced exactly the same results. I've found 2 ways to make this work - firstly by changing the protocol to amqp0-10 in the connection options (this does not allow the node type specification in the address, but seems to work anyway), and secondly by adding in an ACL entry to allow access to the non-existent queue "test.topic": acl allow client@QPID access queue name=test.topic whereupon the broker log looks like this: [Security] info qpid.192.168.0.221:5671-192.168.0.221:56955 Authenticated as client@QPID [Security] debug ACL: Lookup for id:client@QPID action:access objectType:exchange name:test.topic with params { } [Security] debug ACL: checking rule [rule 1 ruleMode = allow props{ name=test.topic }] [Security] debug ACL: lookup name 'test.topic' matched with rule name 'test.topic' [Security] debug ACL: Successful match, the decision is:allow [Security] debug ACL: Lookup for id:client@QPID action:access objectType:queue name:test.topic with params { } [Security] debug ACL: checking rule [rule 2 ruleMode = allow props{ name=test.topic }] [Security] debug ACL: lookup name 'test.topic' matched with rule name 'test.topic' [Security] debug ACL: Successful match, the decision is:allow [Security] debug ACL: Lookup for id:client@QPID action:access objectType:exchange name:test.topic with params { durable=true type=topic } [Security] debug ACL: checking rule [rule 1 ruleMode = allow props{ name=test.topic }] [Security] debug ACL: lookup name 'test.topic' matched with rule name 'test.topic' [Security] debug ACL: Successful match, the decision is:allow [Security] debug ACL: Lookup for id:client@QPID action:publish objectType:exchange exchange name:test.topic with routing key [Security] debug ACL: checking rule [rule 3 ruleMode = allow props{ name=test.topic }] [Security] debug ACL: Rule: 3 lookup exchange name 'test.topic' matched with rule name 'test.topic' [Security] debug ACL: Rule: 3 Successful match, the decision is:allow I wonder if this is related to this issue: https://issues.apache.org/jira/browse/QPID-5299, namely the security concerns about applying ACL rules before revealing the existence of objects. Have I missed anything? Can this legitimately be called a bug? Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: AMQP 1.0 appears to access exchange as queue even after successful ACL match
That works! Brilliant, many thanks. I thought I'd thought of everything :p On 12 November 2015 at 14:50, Jakub Scholz <ja...@scholz.cz> wrote: > I think you have to use topic instead of exchange in the address, e.g.: > test.topic; { node: { type: topic} } > > That should get your client working. > > J. > > On Thu, Nov 12, 2015 at 3:29 PM, Chris Richardson <c...@fourc.eu> wrote: > > > test.topic;{node:{type:exchange}} > > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Broker federation with SSL client auth and SASL EXTERNAL
On 19 October 2015 at 15:37, Gordon Sim <g...@redhat.com> wrote: > On 10/17/2015 01:36 AM, Chris Richardson wrote: > >> Hi, >> >> I'm attempting to set up a broker federation topology using purely SSL >> client authentication and the EXTERNAL SASL mechanism on qpid-cpp 0.34. >> This seems to be within an iota of working but I can't quite get the >> configuration correct for the inter-broker routes. >> >> The point I have arrived at is that I have 2 brokers, both of which are >> configured to accept only connections over SSL with client cert >> authentication. Both python (qpid-stat et al) and c++ (qpid-send/receive) >> clients work perfectly - however the route between the brokers does not >> work because the broker establishing the connection does not use a >> suitable >> certificate. The connection fails with >> Inter-broker link disconnected from broker-2:5671 Failed: SSL peer cannot >> verify your certificate. [-12271] >> > > Was the broker certificate signed by a trusted CA as for the client > certificates? I.e. does the broker accepting the incoming inter-broker > connection trust the other broker? The short answer to that question is _mostly_ "yes" and it got me thinking along the right lines. I didn't realise that the broker's certificate is the one used as a client certificate in the context of the route (kind of obvious in retrospect). I was using certificates with only the "serverAuth" usage extension on both brokers, which was therefore being rejected by the broker receiving the route connection. Adding clientAuth to the usage list (and extending my ACL to match) fixed the problem. Thanks for the hint! > > > >> I've found I can fix this by setting the QPID_SSL_CERT_DB, >> QPID_SSL_CERT_PASSWORD_FILE and QPID_SSL_CERT_NAME variables in the >> environment of the source broker process, but c++ client connections to >> this broker then fail with >> Failed to connect: Failed: NSS error [-8101] >> >> (/var/tmp/portage/net-misc/qpid-cpp-0.34/work/qpid-cpp-0.34/src/qpid/sys/ssl/SslSocket.cpp:177) >> According to the NSS documentation this error is due to an invalid use of >> an SSL certificate (eg: server auth cert being used for client auth) but >> this is the same certificate which previously worked fine. Python client >> connections are unaffected. >> >> I have a swathe of configuration data and logs which I can share if >> needed, >> but to begin with can you tell me if this is something which should, at >> least in principle, work? >> >> Thanks in advance >> >> > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Broker federation with SSL client auth and SASL EXTERNAL
Hi, I'm attempting to set up a broker federation topology using purely SSL client authentication and the EXTERNAL SASL mechanism on qpid-cpp 0.34. This seems to be within an iota of working but I can't quite get the configuration correct for the inter-broker routes. The point I have arrived at is that I have 2 brokers, both of which are configured to accept only connections over SSL with client cert authentication. Both python (qpid-stat et al) and c++ (qpid-send/receive) clients work perfectly - however the route between the brokers does not work because the broker establishing the connection does not use a suitable certificate. The connection fails with Inter-broker link disconnected from broker-2:5671 Failed: SSL peer cannot verify your certificate. [-12271] I've found I can fix this by setting the QPID_SSL_CERT_DB, QPID_SSL_CERT_PASSWORD_FILE and QPID_SSL_CERT_NAME variables in the environment of the source broker process, but c++ client connections to this broker then fail with Failed to connect: Failed: NSS error [-8101] (/var/tmp/portage/net-misc/qpid-cpp-0.34/work/qpid-cpp-0.34/src/qpid/sys/ssl/SslSocket.cpp:177) According to the NSS documentation this error is due to an invalid use of an SSL certificate (eg: server auth cert being used for client auth) but this is the same certificate which previously worked fine. Python client connections are unaffected. I have a swathe of configuration data and logs which I can share if needed, but to begin with can you tell me if this is something which should, at least in principle, work? Thanks in advance -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu <http://www.fourc.eu/>* *Follow us on LinkedIn <http://bit.ly/fourcli>, Facebook <http://bit.ly/fourcfb>, Google+ <http://bit.ly/fourcgp> and Twitter <http://bit.ly/fourctw>!*
Re: Qpid packages for Ubuntu, was Re: [RESULT] [VOTE] Release Qpid CPP 0.34
would be interested in (or even has the resources for)? +1 We have something slightly analogous in the qpid c++ broker for building the windows installer: http://svn.apache.org/viewvc/qpid/trunk/qpid/packaging/windows/ We should keep a set of scripts in the source tree for building .debs (as well as rpms, etc). Wouldn't this duplicate the existing functionality offered by CMake, which can create deb, rpm and various standard archive types (not to mention NullSoft installer for Windoze)? I've used this approach to create debs myself and although it was a while back (0.28) I seem to recall it did everything it should have done. Outside of that, I wrote a small guide in an issue of the debian packaging repo for proton (see: https://github.com/mcpierce/debian-qpid-proton/issues/4). Perhaps we could work together with Darryl to put together a more complete release guide, and begin to share the responsibility? Cheers, Matt - Original Message - From: Matt Broadstone mbroa...@gmail.com javascript:; To: users users@qpid.apache.org javascript:; Sent: Tuesday, June 30, 2015 7:59:35 AM Subject: Re: [RESULT] [VOTE] Release Qpid CPP 0.34 On Mon, Jun 29, 2015 at 10:16 AM, Gordon Sim g...@redhat.com javascript:; wrote: On 06/29/2015 02:19 PM, Robbie Gemmell wrote: This vote is now closed. There were 6 binding +1 votes, with no other votes received. The vote has passed. I will add the files to the dist release svn repo. The website will be updated later after the artifacts have had time to sync to the mirrors. Thanks for standing in as the release manager for this, Robbie! Fantastics news! Is there any idea yet when the qpid-testing packages will be updated to 0.34 in Darryl's ubuntu PPA? Regards, Matt - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org javascript:; For additional commands, e-mail: users-h...@qpid.apache.org javascript:; -- -K -- -K - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- -K - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- -K - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Qpid broker hooks
Hi, We're interested in customising some of the behaviour of the broker, particularly in the context of routing and setting custom properties on messages as they transit the broker. Can you tell me if there's mechanism eg: hooks/plugins to allow us to do this, or any plans to implement such a thing? Thanks Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: [ANNOUNCE] Qpid Proton 0.9 released
Something seems to have gone a little awry with the release page in 0.9 - most links under the Documentation section here http://qpid.apache.org/components/messenger/index.html appear to be broken (0.8 equivalents are ok) (eg: http://qpid.apache.org/releases/qpid-proton-0.9/protocol-engine/c/api/messenger_8h.html ) I minor oversight I'm sure ;) Chris On 3 April 2015 at 04:48, Rafael Schloming r...@apache.org wrote: Hi Everyone, Qpid Proton 0.9 is now officially available. You can find it here: http://qpid.apache.org/releases/qpid-proton-0.9/index.html In addition to numerous bug fixes and improvements, the 0.9 release includes a new reactive API that makes it significantly easier to integrate AMQP into an existing application, or to build custom AMQP applications from scratch. The release also includes a number of fixes and/or enhancements resulting in API changes, some of these are noted below: - pn_buffer_t is no longer public API - pn_bytes_t.bytes is renamed to pn_bytes_t.start - pn_sasl_{client|server}() is replaced by pn_transport_set_server() and client being the default. There is also an important Windows SSL/TLS functionality change: The Proton SSL/TLS module for 0.8 using the native Microsoft SChannel libraries was not configurable and successful handshakes resulted solely from having the appropriate CA certificate in the official Windows Trusted Root CA store. It is now possible to specify alternate trusted root CA databases and to turn off certificate checking altogether, using the same Proton API conventions as for OpenSSL on Posix systems. In particular, Proton applications in Windows will not check server certificates at all in 0.9 unless the capability is explicitly enabled using the pn_ssl_domain_set_trusted_ca_db() function. To use the system Trusted Root CA store: pn_ssl_domain_set_trusted_ca_db(d, sys:root) Or to use a file based PKCS#12 certificate store: pn_ssl_domain_set_trusted_ca_db(d, mycerts.p12) For more information, see the detailed release notes posted on the release page at the URL above. --Rafael -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: 'Error reading socket: Success(0)'
Not sure if this will turn out to be helpful or spam but I've previously encountered this error message and added the following to our kb: This is related to an invalid SSL certificate and has been observed in a situation where the address used to contact the server did not match that of its certificate, eg: using localhost instead of localhost.localdomain or using a DNS alias which does not exist in the certificate. Depending on the client, the error may be accompanied by a message on the client such as [Security] warning Connect failed: Failed: Unable to communicate securely with peer: requested domain name does not match the server's certificate. [-12276] HTH! On 6 March 2015 at 16:08, Gordon Sim g...@redhat.com wrote: On 03/06/2015 04:02 PM, Gordon Sim wrote: Doh, I was looking at the wrong code. Of course, its the NSPR based socket implementation, which has an alternative way of retrieving the error message. I'll raise a JIRA. Just fyi for anyone interested: https://issues.apache.org/ jira/browse/QPID-6435 - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Gentoo ebuilds for Qpid components
Hi list, We (FourC) need gentoo ebuilds for qpid components but I've been unable to find existing complete/maintained packages. The packages I could find are: https://bugs.gentoo.org/show_bug.cgi?id=389027 - reported problems and not updated since qpid-cpp-0.14 in 2012. https://github.com/armills/overlay - includes only the base python package and qpid-cpp (presumably all the author needed); not updated since 0.24. I hope I didn't miss any more complete or up-to-date packages... Since we need up-to-date versions we've created our own overlay which we intend to maintain. We're publishing these in the hope they will be of use to others but naturally because we would love others to contribute and ultimately have the maintenance taken over by some enthusiastic party! Our Portage overlay can be found here: https://github.com/fourceu/qpid-portage-overlay I am no Portage expert so I'm sure there is room for improvement, but they seem to be fundamentally working. We do have tests for them on our internal CI although I can't easily publish the results. Any feedback would be gratefully received! /Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
TLS without SASL in C++ broker
Hi, I've just found a problem configuring a 0.30 broker with TLS but without SASL - the broker rejects connections as unencrypted while a packet trace shows that the connection is, in fact, decidedly encrypted (as with a working system, certificate transfer can be seen in packet 6 after inital TCP connection and handshake; thereafter no plain text packets are transfered). The error message from the security system is Rejected un-encrypted connection. The problem came about after I inadvertently built the broker without the dependent SASL libararies available; rebuilding with SASL fixed the problem with no adjustments to my configuration or SSL database. The assumption appears to be that SSL is not possible without SASL, which I'm not sure is correct...? The only place I found this error in the code was src/qpid/broker/SaslAuthenticator.cpp:203: if (encrypt) { #if HAVE_SASL // encryption required - check to see if we are running over an // encrypted SSL connection. SecuritySettings external = connection.getExternalSecuritySettings(); sasl_ssf_t external_ssf = (sasl_ssf_t) external.ssf; if (external_ssf 1)// 1 == unencrypted #endif { QPID_LOG(error, Rejected un-encrypted connection.); throw ConnectionForcedException(Connection must be encrypted.); } } I think this may be related to https://issues.apache.org/jira/browse/QPID-1899 - I couldn't see any other related issues. Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: [VOTE] Release revised 0.30 python, qmf, tests, and tools components
Hi, I'm looking at the 0.30 mirror at ftp://www.mirrorservice.org/sites/ftp.apache.org/qpid/0.30/ and it appears still to be missing the SHA1 checksums. Could this be fixed please? As a point of personal preference I'd hope for a single SHA1SUM file for all archives, then I won't have to modify my scripts to read them from individual files :P /Chris On 23 September 2014 14:52, Ken Giusti kgiu...@redhat.com wrote: - Original Message - From: Justin Ross jr...@apache.org To: users@qpid.apache.org Sent: Friday, September 19, 2014 8:42:33 AM Subject: [VOTE] Release revised 0.30 python, qmf, tests, and tools components Hi, folks. The RC 1 versions of the python components were missing license and notice files. See Robbie's comments and the attendant fixes: http://qpid.2158936.n2.nabble.com/VOTE-Release-Qpid-0-30-components-td7613719.html https://issues.apache.org/jira/browse/QPID-6110 This is a respin of the following source components only: qpid-python +1 qpid-qmf +1 qpid-tests qpid-tools +1 The other components, qpid-cpp and qpid-java and qpid-java-qmf-tools, are unaffected. The only change versus RC 1 is the addition of the missing files. For these components only, the previous vote is withdrawn. The new RC 2 artifacts are available here: https://dist.apache.org/repos/dist/dev/qpid/0.30-rc2/ Please indicate (or re-indicate) your vote for each component below. If you favor proceeding to GA with these RC bits, vote +1. If you have reason to think a component is not ready for release, vote -1. - Qpid Python (qpid-python-0.30.tar.gz) - Qpid QMF (qpid-qmf-0.30.tar.gz) - Qpid tests (qpid-tests-0.30.tar.gz) - Qpid tools (qpid-tools-0.30.tar.gz) Thanks! Justin --- 0.30 release page: https://cwiki.apache.org/confluence/display/qpid/0.30+Release -- -K - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
C++ client connection fails with SSL and AMQP1.0
Hi, It seems that the combination of SSL and AMQP 1.0 does not work in 0.30 beta (connection fails). The issue looks exactly the same as https://issues.apache.org/jira/browse/QPID-5648, which was reported fixed in trunk and 0.28. I'm running a 0.30 beta broker with ssl enabled. Without any QPID_SSL settings present in my environment, I can successfully send a message like this: QPID_SSL_CERT_DB=/var/lib/qpidd/ssl/client-db spout -b localhost:5671 --content hello --connection-options {transport:ssl} amq.topic Notes - /var/lib/qpidd/ssl/client-db holds an NSS database with the CA cert for the broker. This example uses the default AMQP 0.10. When I add the extra protocol option to use AMQP 1.0: QPID_SSL_CERT_DB=/var/lib/qpidd/ssl/client-db spout -b localhost:5671 --content hello --connection-options {protocol:amqp1.0, transport:ssl} amq.topic I get the error Connect failed to amqp:ssl:localhost:5671: Reconnect disabled. If I run this in a debugger I can get some more info on the error (reported in the callback at qpid::messaging::amqp::SslTransport::failed()) which is NSS error [-12268]. According to the NSS documentation this is SSL_ERROR_SSL_DISABLED. I've done some experimentation and if I call NSS_InitContext() referencing my NSS db before opening the AMQP connection, the SSL handshake (with AMQP1.0) is successful. My first guess would be that there is some difference in the way qpid::messaging:amqp::ConnectoinHandle (used for AMQP1.0 connections) and qpid::client::amqp0_10::ConnectionImpl (used for AMQP0.10 connections) initialise the NSS subsystem. Hopefully that and not an error on my part... Could you please look into this further? Many thanks -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: qpid-route TLS connection to destination broker
Hello again, I'm a bit concerned I'm approaching this in the wrong way - should I have logged this directly in Jira? Also I hope I didn't structure it in a way that caused offense, I was aiming for consise rather than rude! ;) Regards /Chris On 14 August 2014 11:46, Chris Richardson c...@fourc.eu wrote: Hi, I have another SSL problem and potential fix, this time to the qpid-python package. Any chance this could make it into the 0.30 release? -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
qpid-route TLS connection to destination broker
Hi, I have another SSL problem and potential fix, this time to the qpid-python package. Any chance this could make it into the 0.30 release? In summary, qpid-route is unable to connect to destination brokers with TLS, which implies in practice that a federated topology cannot have require-encrypted set. Circumstances are: * Configure a broker for both plain and TLS access * Access plaintext service with eg: qpid-route route list - should show an (empty) list of routes * Access TLS service with eg: qpid-route route list amqps://localhost - connection will fail with Failed: ConnectionFailed - (None, 'connection aborted') This happens because qpid/framer.py:93 reads the 8-byte AMQP protocol header from the socket (waiting for sufficient data in a loop) with s = self.sock.recv(n) This is fine in plaintext when sufficient data is present, but with TLS enabled the header is fragmented and it reads from the socket twice, actually reading 9 bytes (1 byte into the next part of the connection handshake). This byte is discarded and the handshake later fails. Replacing the read with s = self.sock.recv(n - len(self.rx_buf)) appears to fix the problem. This error does not affect all python tools eg: qpid-stat, which use the qpid::messaging::Connection instead of the qpid::connection::Connection class. Regards -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: Trouble building the .Net bindings in Windows.
On 14 August 2014 03:28, erikb erik.beu...@redcrater.co.nz wrote: Anyway... I've installed all the prerequisites mentioned in the Install-Windows document under the What to install section. When I (try to) configure cmake, I get the following error: CMake Error at src/CMakeLists.txt:90 (message): Can't locate ruby, needed to generate amqp 0-10 framing code. I do have ruby installed. In fact I went to a lot of trouble to find a windows installed for V1.9.3, as the document mentioned it needed to be pre-version 2. I can offer a non-authoritative hint here since this is a standard Windows development problem rather than a qpid-specific problem: you've installed something (ruby) but there's no mechanism for your build process to find it. I'd suggest you either 1. add your ruby bin directory to your system path; this can be a pain on windows - don't forget to ensure that whatever process is looking for ruby (cmake) is started from an environment that has picked up the changes; if in doubt reboot :p. By ruby bin I mean the directory in your ruby installation that contains ruby.exe. or 2. After running cmake's configure stage (if you're using the gui), you'll find a variable called RUBY_EXECUTABLE which can be set to the full path to ruby.exe. If you can't find the setting, make sure you're viewing advanced variables. The same setting can be set on the command line with -DRUBY_EXECUTABLE=full path to ruby.exe I don't have a windows box to test this on but I hope this sends you in the right direction. Good luck! /Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: Inter-broker link with SSL
I think I've tracked this one down. My theory is that the qpid::amqp_1_10::Connection::decode method, which contains the code if (isClient !initialized) { //read in protocol header framing::ProtocolInitiation pi; if (pi.decode(in)) { if(!(pi==version)) throw Exception(QPID_MSG(Unsupported version: pi supported version version)); QPID_LOG(trace, RECV [ identifier ]: INIT( pi )); } initialized = true; } can incorrectly set the connection status to initialised when the pi.decode() method fails (which it does if 8 bytes are available). This does not happen in practice without SSL, but when SSL is switched on I have seen zero, 1 and 4 bytes turn up in the input buffer on first invocation. Moving the initialized = true line to inside the if (pi.decode(int) { block seems to fix the issue (this is because subsequent calls to the Connection::decode() method (when sufficient data is available in the input buffer) will result in the pi.decode() method correctly consuming the AMQP... header). A further optimisation might be to skip the decode() method body if insufficient data is available thereby avoiding the overhead of the various objects that are created and destroyed and the exceptions that are thrown... but that's extra lines of code and less is more, as they say ;) Could a qpid dev please confirm this fix and, if approved, could we be informed of when a fix will be released? Thanks Chris On 4 August 2014 13:22, Chris Richardson c...@fourc.eu wrote: Hi, As you see from the subject I'm trying to set up an SSL link between two brokers (version 0.28). To cut to the chase, both brokers are able to communicate locally over SSL - for instance I can use qpid-stat -b amqps://localhost:5671 -e and qpid-stat -b amqps://localhost:5674 -e (these being my broker SSL ports) to retrieve lists of exchanges. This works fine. The problem comes when I attempt to add a route. To simplify, I'm just trying to add just the link here with qpid-route -t ssl link add localhost:5673 hostname:5671 Note that the destination is using a non-ssl port as per Gordon's suggestion in this thread http://qpid.2158936.n2.nabble.com/SSL-between-c-brokers-td7581914.html and that hostname matches the cn= part of my ssl-cert's subject. Sadly that thread related to version 0.14 and culminated in a bug fix related to that version and did not continue along the same lines as my problem. Following the addition of the link, the command qpid-route link list localhost:5673 shows HostPortTransport Durable State Last Error = hostname 5671ssl N Waiting Closed by peer and the source broker repeatedly reports [System] error framing-error: Framing version unsupported (AMQPFrame.cpp:93) I've done some digging into the state of the broker around this area of the code, specifically in the AMQPFrame::decode method. It seems that the framing version is derived from a flags byte and should always be zero, but in this case has a value of 1. The byte stream being decoded is 134 bytes long and begins (in hex) {41 4d 51 50 01 01 00 0a 0f 00 00 7e 00 00 00 00 00 00 00 00 01 01 07 00 00 00 00 3f 00 00 00 0 13 71 70 69 64 2e 66 65 64 65 ... ...} This appears to be a decrypted version of the frame as it begins AMQP, 1, 1, 0, 10... which matches the specification exactly down to the class, instance and protocol version of the message being received. However, the flags byte from which the frame version is derived has a value of 41 ie. appears to be incorrectly read from the A of AMQP... clearly something has gone awry! Other packets arriving at the AMQPFrame::decode method appear to have had the header already removed and begin { 0f ...} (assuming those flags are set). At this point I thought it may be time to hand over to someone with a better understanding of the broker internals. Help... please? :) /Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!* -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Inter-broker link with SSL
Hi, As you see from the subject I'm trying to set up an SSL link between two brokers (version 0.28). To cut to the chase, both brokers are able to communicate locally over SSL - for instance I can use qpid-stat -b amqps://localhost:5671 -e and qpid-stat -b amqps://localhost:5674 -e (these being my broker SSL ports) to retrieve lists of exchanges. This works fine. The problem comes when I attempt to add a route. To simplify, I'm just trying to add just the link here with qpid-route -t ssl link add localhost:5673 hostname:5671 Note that the destination is using a non-ssl port as per Gordon's suggestion in this thread http://qpid.2158936.n2.nabble.com/SSL-between-c-brokers-td7581914.html and that hostname matches the cn= part of my ssl-cert's subject. Sadly that thread related to version 0.14 and culminated in a bug fix related to that version and did not continue along the same lines as my problem. Following the addition of the link, the command qpid-route link list localhost:5673 shows HostPortTransport Durable State Last Error = hostname 5671ssl N Waiting Closed by peer and the source broker repeatedly reports [System] error framing-error: Framing version unsupported (AMQPFrame.cpp:93) I've done some digging into the state of the broker around this area of the code, specifically in the AMQPFrame::decode method. It seems that the framing version is derived from a flags byte and should always be zero, but in this case has a value of 1. The byte stream being decoded is 134 bytes long and begins (in hex) {41 4d 51 50 01 01 00 0a 0f 00 00 7e 00 00 00 00 00 00 00 00 01 01 07 00 00 00 00 3f 00 00 00 0 13 71 70 69 64 2e 66 65 64 65 ... ...} This appears to be a decrypted version of the frame as it begins AMQP, 1, 1, 0, 10... which matches the specification exactly down to the class, instance and protocol version of the message being received. However, the flags byte from which the frame version is derived has a value of 41 ie. appears to be incorrectly read from the A of AMQP... clearly something has gone awry! Other packets arriving at the AMQPFrame::decode method appear to have had the header already removed and begin { 0f ...} (assuming those flags are set). At this point I thought it may be time to hand over to someone with a better understanding of the broker internals. Help... please? :) /Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Dispatch router multi-hop topology
Hi, I'm trying to get an example of a multi-hop topology working with qpid-dispatch 0.2 built against qpid-proton 0.28. The scenario is probably most simply described by plagiarising the config files from the tests/config-3-linear directory, which sets up a series of 3 sequentially connected routers. From the test topology.txt: +--+ +--+ +--+ |QDR.A | |QDR.B | |QDR.C | |port: | |port: | |port: | | 20001|-| 20002|-| 20003| | | | | | | | | | | | | +--+ +--+ +--+ * The direction of the arrow shows the direction of the connection setup Connector -- Listener To each of the config files in this example I've added a listener: listener { addr: 0.0.0.0 port: 5672 sasl-mechanisms: ANONYMOUS } with the port incremented to 5673 and 5674 in config files B and C (so it can be run on a single machine). I'm using drain/spout to send messages with the following syntax: drain --connection-options {protocol:amqp1.0} -b 0.0.0.0:5672 -f myAddress spout --connection-options {protocol:amqp1.0} -b 0.0.0.0:5674 myAddress By varying the port number I can of course control which router the messages are being sent/received from. When the drain process connects I can see the mobile address registered on all three routers. Sending messages via any pair of adjacent routers works yet I am not able to send an address from router A to router C or visa versa - spout does not return, presumably since it has not had confirmation of delivery. Am I doing something wrong? Thanks Chris -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: Dispatch router multi-hop topology
Aha, x-amqp-to works a treat. I'd tried adding the x-opt-qd.to option mentioned here https://qpid.apache.org/releases/qpid-dispatch-0.2/book/amqp-mapping.html but in retrospect that does look more like an internal router property. Many thanks! On 17 June 2014 13:29, Ted Ross tr...@redhat.com wrote: Hi Chris, You may have run afoul of https://issues.apache.org/jira/browse/DISPATCH-1 which was fixed after 0.2. Spout doesn't set the to field in the messages it sends, it puts the address in the target of the link. You can work around this by adding the following to the spout command line: --property x-amqp-to=myAddress Alternatively, you can try running Dispatch from trunk (you will need Proton 0.7 for this). -Ted On 06/17/2014 04:59 AM, Chris Richardson wrote: Hi, I'm trying to get an example of a multi-hop topology working with qpid-dispatch 0.2 built against qpid-proton 0.28. The scenario is probably most simply described by plagiarising the config files from the tests/config-3-linear directory, which sets up a series of 3 sequentially connected routers. From the test topology.txt: +--+ +--+ +--+ |QDR.A | |QDR.B | |QDR.C | |port: | |port: | |port: | | 20001|-| 20002|-| 20003| | | | | | | | | | | | | +--+ +--+ +--+ * The direction of the arrow shows the direction of the connection setup Connector -- Listener To each of the config files in this example I've added a listener: listener { addr: 0.0.0.0 port: 5672 sasl-mechanisms: ANONYMOUS } with the port incremented to 5673 and 5674 in config files B and C (so it can be run on a single machine). I'm using drain/spout to send messages with the following syntax: drain --connection-options {protocol:amqp1.0} -b 0.0.0.0:5672 -f myAddress spout --connection-options {protocol:amqp1.0} -b 0.0.0.0:5674 myAddress By varying the port number I can of course control which router the messages are being sent/received from. When the drain process connects I can see the mobile address registered on all three routers. Sending messages via any pair of adjacent routers works yet I am not able to send an address from router A to router C or visa versa - spout does not return, presumably since it has not had confirmation of delivery. Am I doing something wrong? Thanks Chris - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org -- *Chris Richardson*, System Architect c...@fourc.eu *FourC AS, Vestre Rosten 81, Trekanten, NO-7075 Tiller, Norwaywww.fourc.eu http://www.fourc.eu/* *Follow us on LinkedIn http://bit.ly/fourcli, Facebook http://bit.ly/fourcfb, Google+ http://bit.ly/fourcgp and Twitter http://bit.ly/fourctw!*
Re: Server-client message relaying with dynamic routes
On 8 April 2014 15:41, Gordon Sim g...@redhat.com wrote: snip We are particularly interested in your comment ...qpidd (which supports establishing basic AMQP 1.0 links to/from other processes)...; Chris've tried things like adding a link from a broker to a router with qpid-route link add... but this produces errors relating to SASL on the router side and version errors on the broker side. We haven't pursued this further as we're not sure this is how it's supposed to work. Is there any documentation or examples you could refer us to? In the case of qpidd this is indeed (as Fraser pointed out) via a slightly different mechanism than the existing 'federation' feature (which is more tied to AMQP 0-10). Specifically you would create a 'domain' to represent the router (or whatever external AMQP 1.0 compliant process you wish to connect to), then create 'incoming' or 'outgoing' links for messages to be transferred over. I'm afraid although I've described it on this list a few times, it isn't properly documented at all yet, sorry! I should also point out that its a relatively raw 'feature' at present. Specific known limitations at present include no automatic reconnect for broker established 1.0 connections and no mechanism for preventing message looping. However as an example, say you swapped Dispatch Router in for Broker B, you would then create a 'domain' for that on both Broker A and Broker C, using qpid-config (on latest qpid release): qpid-config add domain BrokerB --argument url=public-ip or qpidt test tool[1] on earlier releases: qpidt create domain url=public-ip (You can also specify sasl details if you want authentication, but as dispatch router doesn't yet support that, I've left it out). Then you would either add 'incoming' or 'outgoing' links from/to that domain, e.g. on BrokerA qpid-config add outgoing my-test-link source=abc target=def and on BrokerC qpid-config add incoming my-test-link source=def target=xyz which should cause messages sent to abc on BrokerA to be received by subscribers to xyz on BrokerC, with messages queued in abc and xyz. I've set this scenario up on my test system (I had to add --argument domain=BrokerB to the qpid-config add incoming|outgoing ... commands...) and it looks promising, except if I understand correctly there is no queuing on the relaying node since it's a dispatch router. Unfortunately we absolutely need this. (You can also send/receive to/from e.g. my-random-address@BrokerB which will dynamically create the links as needed between A/C and B. However in this scheme you wouldn't get any store and forward ability, so there may not be any value to A and C being qpidd instances at all). But to sum up, are there at all any best practise ways to solve dynamic routing for spontaneously connected clients without public IPs with today's (or tomorrow's :-) ) tool-set? You should be able to set this up using the 'old' federation and qpidd, though it would only use AMQP 0-10. You would need to use push routes as well as the standard pull routes. However even with dynamic routing you would need one route in each direction per exchange, and the same would apply to 'static' routes if you didn't need the traffic to be optimised to avoid unnecessary transfers[2], and static routes do allow push routes (where the flow of messages is to the process that originally established the tcp connection). I think this is the best solution today if you have to have some store-and-forward ability (i.e. allow senders to forget about messages they sent and rely on the messaging intermediaries to keep trying to deliver until successful). Tomorrows solution would I think be based either on the new mechanisms being added to Dispatch Router, allowing store-and-forward capability to be added for addresses through an associated broker to which the router will manage connections, or on adding reconnect/redelivery capabilities to messenger. You are just a little bit ahead of us at present! We're obviously very interested in these next developments. Can you give us a rough idea of when we can hope to see these implemented, at least in beta? Thanks, Chris
Re: Server-client message relaying with dynamic routes
Thanks very much Gordon. I have had some previous endeavours with the dispatch router (in fact I started there before switching to the broker) and will return to that plan of attack. I think we will still need brokers in this topology in order to queue messages for offline clients, otherwise the router would be the better option. I look forward to future releases with the improved broker integration! /Chris 2014-03-28 15:52 GMT+00:00 Gordon Sim g...@redhat.com: On 03/28/2014 01:36 PM, Chris Richardson wrote: Ah, now we are really opening Pandora's box! ;) The term client here is actually quite a simplified term and actually refers loosely speaking to a system managing a number of products on that... client. Both the management system and the products should be individually identifiable and addressable over AMQP; it should also be possible to address them in groups. Furthermore the client system will typically be connected over an unreliable network link and the messaging system is expected to guarantee delivery. Given these constraints I envisage each client system having at least one local exchange (and queues to cope with disconnections), and most probably it/they will be topic exchanges to support the addressing functionality. However I'm very new to the AMQP routing and addressing concepts so I may be misguided. Even with dynamic routing, you need to manually set it up per exchange (what it then does is manage the set of routing keys in use for inter-broker links, based on the interest expressed by subscriptions). If you are using topic exchanges you might be able to simply forward all messages for a given exchange through from A-B-C, which you can do by simple static routes per exchange (not actually any more management operations than making setting up dynamic routing for the same exchange). The static routes *should* work ok with 'push' routes (though I will warn that those are not as well used), allowing you to choose the direction of the TCP connection independent of the direction of message flow. Also, without wanting to confuse the picture too much, I'll just note in passing that the recent developments around AMQP 1.0 may be of some interest to your scenario. First off there is the 'Dispatch Router' component. This is superficially similar to a broker, but it only forwards messages it never accepts responsibility for them. It could be used as the 'server' broker in your setup, with the 'clients' both connecting in and sending or receiving to/from that. You could use this in conjunction with qpidd (which supports establishing basic AMQP 1.0 links to/from other processes), or depending on your use case there may not be a real need for brokers at all. (A future release of the router will be able to establish links out to other 1.0 capable brokers (or other types of 'thing') itself making it even more flexible/powerful). - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Static routes might be ok for a prototype, but a production system would have many hundreds or even thousands of clients frequently being added and removed. My assumption is that a static configuration would incur a much higher management overhead? 2014-03-28 10:51 GMT+00:00 Gordon Sim g...@redhat.com: On 03/27/2014 03:33 PM, Chris Richardson wrote: Hi mailinglist, I'm trying to set up a broker federation topology with a server and (for prototyping) two clients and I need to send messages from one client to the other, routed via the server broker since the clients will be firewalled/NATed and can not communicate directly. My understanding is that the way to do this is with dynamic routing and I've followed the discussion on the following thread with some success: http://qpid.2158936.n2.nabble.com/Dynamic-routing-between- disconnected-exchanges-td7598100.html. That article describes three nodes A, B and C, and relaying from node A to C via B. So far so good - I can use drain on node C's test-topic/C exchange/routing key and spout to write to node A's test-topic/C exchange/routing key and the message is transferred via the server (node B). The problem I have is that this setup relies on TCP links being established in both directions between each node. In my client-server scenario this is clearly not possible and with the network restriction in place the dynamic routing fails. As the documentation states, A dynamic exchange route is always a pull route. It can never be a push route.. Does this mean that the underlying broker link must be established in the same direction as the route, or is there some way to override this or get the route from the server to utilize the existing link from the client? Solutions involving VPNs and tunnels are not allowed. What are the message flows here, i.e. what 'topics' or 'queues' will be involved? Do you really need dynamic federation or could a static configuration work? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Ah, now we are really opening Pandora's box! ;) The term client here is actually quite a simplified term and actually refers loosely speaking to a system managing a number of products on that... client. Both the management system and the products should be individually identifiable and addressable over AMQP; it should also be possible to address them in groups. Furthermore the client system will typically be connected over an unreliable network link and the messaging system is expected to guarantee delivery. Given these constraints I envisage each client system having at least one local exchange (and queues to cope with disconnections), and most probably it/they will be topic exchanges to support the addressing functionality. However I'm very new to the AMQP routing and addressing concepts so I may be misguided. 2014-03-28 13:00 GMT+00:00 Gordon Sim g...@redhat.com: On 03/28/2014 12:53 PM, Gordon Sim wrote: On 03/28/2014 11:22 AM, Chris Richardson wrote: Static routes might be ok for a prototype, but a production system would have many hundreds or even thousands of clients frequently being added and removed. My assumption is that a static configuration would incur a much higher management overhead? It really depends on the messaging patterns between the clients. At one extreme, if each client was using the same 'topic' (whether sending or receiving), then the static configuration would be very simple. At the other extreme if each client would require extra routes to be defined, then yes, that would be an unwieldy solution most likely. Putting the question a better way, how many different exchanges will you be using and what will be the type of the exchange(s)? - To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org For additional commands, e-mail: users-h...@qpid.apache.org
Server-client message relaying with dynamic routes
Hi mailinglist, I'm trying to set up a broker federation topology with a server and (for prototyping) two clients and I need to send messages from one client to the other, routed via the server broker since the clients will be firewalled/NATed and can not communicate directly. My understanding is that the way to do this is with dynamic routing and I've followed the discussion on the following thread with some success: http://qpid.2158936.n2.nabble.com/Dynamic-routing-between-disconnected-exchanges-td7598100.html. That article describes three nodes A, B and C, and relaying from node A to C via B. So far so good - I can use drain on node C's test-topic/C exchange/routing key and spout to write to node A's test-topic/C exchange/routing key and the message is transferred via the server (node B). The problem I have is that this setup relies on TCP links being established in both directions between each node. In my client-server scenario this is clearly not possible and with the network restriction in place the dynamic routing fails. As the documentation states, A dynamic exchange route is always a pull route. It can never be a push route.. Does this mean that the underlying broker link must be established in the same direction as the route, or is there some way to override this or get the route from the server to utilize the existing link from the client? Solutions involving VPNs and tunnels are not allowed. Just for info, I've included some more info from my prototype showing information firstly from the working setup: Client A - 192.168.0.12 Server - 192.168.0.13 Client B - 192.168.0.14 *Server:* *~$ qpid-route link list* HostPortTransport Durable State Last Error = 192.168.0.125672tcp N Operational 192.168.0.145672tcp N Operational *~$ sudo netstat -np | grep 5672* tcp0 0 192.168.0.13:43022 192.168.0.12:5672 ESTABLISHED 1477/qpidd tcp0 0 192.168.0.13:5672 192.168.0.12:38576 ESTABLISHED 1477/qpidd tcp0 0 192.168.0.13:5672 192.168.0.14:46242 ESTABLISHED 1477/qpidd tcp0 0 192.168.0.13:34905 192.168.0.14:5672 ESTABLISHED 1477/qpidd *Client A:* *~$ qpid-route link list* HostPortTransport Durable State Last Error = 192.168.0.135672tcp N Operational *~$ sudo netstat -np | grep 5672* tcp0 0 192.168.0.12:5672 192.168.0.13:43022 ESTABLISHED 1467/qpidd tcp0 0 192.168.0.12:38576 192.168.0.13:5672 ESTABLISHED 1467/qpidd *Client B:* Equivalent to Client A. Then, with some (not so) cunning use of iptables, I blocked incoming connections from the server to the clients: *~$ sudo iptables -vnL* Chain INPUT (policy ACCEPT 2157 packets, 424K bytes) pkts bytes target prot opt in out source destination 4 240 REJECT tcp -- * * 192.168.0.13 0.0.0.0/0tcp dpt:5672 reject-with icmp-port-unreachable and the server-side links show failure, as one would expect: *~$ qpid-route link list* HostPortTransport Durable State Last Error = 192.168.0.125672tcp N Waiting Connection refused 192.168.0.145672tcp N Waiting Connection refused Any suggestions on how to solve this problem would be greatly appreciated! Thanks in advance Chris