Hello again folks.OK, so having managed to "repair" the Address created by the slightly broken AddressParser for C++ (see below) I thought that I'd try the same with Perl, but no!! Despite my Pidjin Perl I *nearly* got there, only to be thwarted at the last hurdle!!!
It appears that the SWIG Perl binding hides the Variant data type pretty much completely. I spent most of Sunday tearing my hair out trying various things to encode the argument value, but I think it's pretty much impossible to do it in Perl given the existing SWIG binding (if anyone can figure it out I'd love to know!!!!).
Not one for admitting defeat :-) I took a crash course in SWIG courtesy of Mr Google.
I've attached for your delectation/humour a C++ binding file to repair the address and to allow setting properties as UTF8 plus the SWIG interface file, a Makefile and demo Perl producer/consumer clients.
This stuff allows C++, Perl and Java all to interoperate using the headers exchange by properly using UTF8 strings as the Address binding arguments and Message properties.
I hope this stuff is useful - I'm on a mission to promote interoperability :-D
BTW I know that the right approach is to actually fix the AddressParser, but I wanted to start with a fix in "user space" to allow C++/Perl clients to get working without relying on a patch to the Qpid client runtime code.
Cheers, Frase Fraser Adams wrote:
I agree, this is a bug. I've raised a JIRA and will get that fixed: https://issues.apache.org/jira/browse/QPID-3492Hi all,For everyone who's been following this thread and have been bitten by this issue, in lieu of a patch to the AddressParser itself the following code just might help you out. It's a basic "client side" code block that, in essence, "repairs" an Address created by the broken AddressParser by working its way through the various Address blocks until it gets to the x-bindings arguments block it then iterates through each argument explicitly setting the encoding of VAR_STRING valued arguments to utf8.My C++ is somewhat rusty so there might well be more elegant/succinct ways to find the arguments block, feel free to post back something nicer, but the code below seems a fair starter for ten. I've tried it with a basic Java producer and the "repaired" bindings match nicely.Hope this is some use, Frase. /* // Example Usage:string address = "testqueue; {create: receiver, node: {x-declare: {arguments: {'qpid.policy_type': ring, 'qpid.max_size': 500000000}}, x-bindings: [{exchange: 'amq.match', queue: 'testqueue', key: 'data1', arguments: {x-match: all, data-service: amqp-delivery, item-owner: fadams}}]}}";Address addr(address); addr = utf8EncodeAddress(addr); Connection connection(broker, connectionOptions); try { connection.open(); Session session = connection.createSession(); Receiver receiver = session.createReceiver(addr); receiver.setCapacity(100); // Enable receiver prefetch ......etc..... */ /*There is a bug in the Qpid C++ AddressParser code up to at least version 0.12 whereby strings values get encoded as raw binary values as opposed to UTF8. In many circumstances this doesn't cause significant problems, however for the case of bindings to the headers exchange it has the potential to create a serious interoperability problem as Java producers in particular will certainly set header strings as UTF8 Java Strings.These methods "repair" an Address by re-encoding x-bindings argument values as UTF8 strings so that they work in an interoperable way with the Headers exchange. Note well the use of references as we're changing theunderlying Address passed to the method. */ /*This method looks for the "x-bindings" Map within a "node" or "link" block and if one is found it then iterates through the bindings. For each binding that is found the "arguments" Map is looked up and if an "arguments" Map is found the method iterates though each argument explicitly setting the encodingof VAR_STRING valued arguments to utf8. */ void utf8EncodeBlock(Variant::Map& block) { Variant::Map::iterator i = block.find("x-bindings"); if (i != block.end()) { Variant::List& bindings = i->second.asList();for (Variant::List::iterator li = bindings.begin(); li != bindings.end(); li++) {Variant::Map& binding = li->asMap(); i = binding.find("arguments"); if (i != binding.end()) { Variant::Map& arguments = i->second.asMap(); for (i = arguments.begin(); i != arguments.end(); i++) { if (i->second.getType() == VAR_STRING) { i->second.setEncoding("utf8"); } } } } } } /*This method extracts the "node" and "link" Maps from the options part of the Address and passes references to them to the utf8EncodeBlock method in order to repair the contents of the block.*/ Address& utf8EncodeAddress(Address& addr) { Variant::Map& options = addr.getOptions(); Variant::Map::iterator i = options.find("node"); if (i != options.end()) { utf8EncodeBlock(i->second.asMap()); } i = options.find("link"); if (i != options.end()) { utf8EncodeBlock(i->second.asMap()); } return addr; } --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
#include <qpid/messaging/Address.h>
#include <qpid/messaging/Message.h>
#include <qpid/types/Variant.h>
using namespace qpid::messaging;
using namespace qpid::types;
/*
There is a bug in the Qpid C++ AddressParser code up to at least version 0.12 whereby strings values get encoded
as raw binary values as opposed to UTF8. In many circumstances this doesn't cause significant problems, however
for the case of bindings to the headers exchange it has the potential to create a serious interoperability problem
as Java producers in particular will certainly set header strings as UTF8 Java Strings.
These methods "repair" an Address by re-encoding x-bindings argument values as UTF8 strings so that they work
in an interoperable way with the Headers exchange. Note well the use of references as we're changing the
underlying Address passed to the method.
There is also a setUtf8Property method to allow Perl to send a correctly UTF8 encoded header
*/
/*
This method looks for the "x-bindings" Map within a "node" or "link" block and if one is found it then
iterates through the bindings. For each binding that is found the "arguments" Map is looked up and if
an "arguments" Map is found the method iterates though each argument explicitly setting the encoding
of VAR_STRING valued arguments to utf8.
*/
void utf8EncodeBlock(Variant::Map& block) {
Variant::Map::iterator i = block.find("x-bindings");
if (i != block.end()) {
Variant::List& bindings = i->second.asList();
for (Variant::List::iterator li = bindings.begin(); li != bindings.end(); li++) {
Variant::Map& binding = li->asMap();
i = binding.find("arguments");
if (i != binding.end()) {
Variant::Map& arguments = i->second.asMap();
for (i = arguments.begin(); i != arguments.end(); i++) {
if (i->second.getType() == VAR_STRING) {
i->second.setEncoding("utf8");
}
}
}
}
}
}
/*
This method extracts the "node" and "link" Maps from the options part of the Address and passes references
to them to the utf8EncodeBlock method in order to repair the contents of the block.
*/
Address& utf8EncodeAddress(Address& addr) {
Variant::Map& options = addr.getOptions();
Variant::Map::iterator i = options.find("node");
if (i != options.end()) {
utf8EncodeBlock(i->second.asMap());
}
i = options.find("link");
if (i != options.end()) {
utf8EncodeBlock(i->second.asMap());
}
return addr;
}
/*
This method properly encodes the value parameter as a UTF8 Variant type then sets the Property on the
Message passed in as a parameter.
*/
void setUtf8Property(Message& message, const char* key, const char* value) {
Variant utf8Value(value);
utf8Value.setEncoding("utf8");
message.setProperty(key, utf8Value);
}
/* cqpid_perl_utf8.i */
%module cqpid_perl_utf8
%{
#include <qpid/messaging/Address.h>
#include <qpid/messaging/Message.h>
extern qpid::messaging::Address& utf8EncodeAddress(qpid::messaging::Address&
addr);
extern void setUtf8Property(qpid::messaging::Message& message, const char* key,
const char* value);
%}
extern qpid::messaging::Address& utf8EncodeAddress(qpid::messaging::Address&
addr);
extern void setUtf8Property(qpid::messaging::Message& message, const char* key,
const char* value);
item_consumer.pl
Description: Perl program
item_producer.pl
Description: Perl program
REMOVE = -rm
#----------------------- Compiler Options ---------------------
COMPILER = g++
CFLAGS = -O6 -march=native -Wall -posix
CDEFINES = -D_POSIX_C_SOURCE=199506L\
-D_XOPEN_SOURCE\
-D_XOPEN_SOURCE_EXTENDED\
-D_REENTRANT\
-D_THREAD_SAFE
# -------------------------------------------------------------
CINCLUDES = \
-I.\
all : cqpid_perl_utf8.so
cqpid_perl_utf8.so : cqpid_perl_utf8.i
swig -c++ -perl cqpid_perl_utf8.i
$(COMPILER) $(CFLAGS) $(CDEFINES) -fpic -c cqpid_perl_utf8.cpp
$(COMPILER) $(CFLAGS) $(CDEFINES) `perl -MConfig -e 'print join(" ",
@Config{qw(ccflags optimize cccdlflags)}, "-I$$Config{archlib}/CORE")'` -c
cqpid_perl_utf8_wrap.cxx
$(COMPILER) $(CFLAGS) $(CDEFINES) -shared cqpid_perl_utf8.o
cqpid_perl_utf8_wrap.o -lqpidmessaging -o cqpid_perl_utf8.so
clean :
$(REMOVE) *~
$(REMOVE) *.~*
$(REMOVE) *.o
$(REMOVE) .objects
$(REMOVE) cqpid_perl_utf8.pm
$(REMOVE) cqpid_perl_utf8.so
$(REMOVE) cqpid_perl_utf8_wrap.*
--------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
