Data interface in Proton-J
Hello everybody, I've been following this list for quite a while but so far had no reason to take part in discussions. However, these days I'm porting some C++ code that is using Proton-C to talk to peers to Java. The C++ code uses pn_data_put_* functions from codec module to compose the body of the message and pn_data_get_* counterparts to decompose it at the other end. In Java, the Proton-Api's Data interface (org.apache.qpid.proton.codec) seems to provide the same functionality. But as far as I could see no class is actually implementing the interface, and the DataFactoryImpl.createData(long) is only throwing the exception which indicates that this operation is not supported. I've seen this in both 0.4 package and on the Git's trunk. Is this a part of functionality that is merely not implemented yet, or something that is not planned to get implemented at all? In either case, what would be the alternate methods to put together the message body in a way that could be understood by C++ code? And vice versa? The use of Java bindings to Proton-C is not the preferred way to go. Thanks and best regards, Leon
Re: Data interface in Proton-J
That would be great! Let me have a dig around over the weekend and see if I can find it (I have *lots* of machines where it might be on :-( I should probably be more organised). Cheers, Rob On 14 June 2013 16:01, Leon Mlakar l...@digiverse.si wrote: Hi Rob, thanks for reply. It got me thinking ... since Data interface is not a history and will be around, to do things The Right Way I'll have to implement parts of it myself anyway. So I might as well take a shot in adding missing 70%, or at least some part of them, on top of your 30% :-). Is there any chance for you to you dig up what you've got so far and get it over to me somehow? Best regards, Leon Hi Leon, As you have noticed a pure Java implementation of the Data interface is not yet available (I think I have a 30% complete implementation sitting somewhere on an old laptop, but I never got round to finishing it). It will get implemented eventually, but it's not currently being worked on as far as I know. The Java implementation was built upon a codec which serializes objects in Java form into AMQP binary data. On the Java implementation of the Message class there is a method setBody(Section body). So, in Java, to send an amqp value message containing a map, you could do something like: import org.apache.qpid.proton.amqp.**messaging.AmqpValue; import org.apache.qpid.proton.amqp.**Symbol; ... Map map = new LinkedHashMap(); map.put(symbol, Symbol.valueOf(a symbol)); map.put(int, 21); ... AmqpValue body = new AmqpBody(map); message.setBody(body); Hope this helps, Rob On 14 June 2013 15:22, Leon Mlakar l...@digiverse.si wrote: Hello everybody, I've been following this list for quite a while but so far had no reason to take part in discussions. However, these days I'm porting some C++ code that is using Proton-C to talk to peers to Java. The C++ code uses pn_data_put_* functions from codec module to compose the body of the message and pn_data_get_* counterparts to decompose it at the other end. In Java, the Proton-Api's Data interface (org.apache.qpid.proton.codec) seems to provide the same functionality. But as far as I could see no class is actually implementing the interface, and the DataFactoryImpl.createData(long) is only throwing the exception which indicates that this operation is not supported. I've seen this in both 0.4 package and on the Git's trunk. Is this a part of functionality that is merely not implemented yet, or something that is not planned to get implemented at all? In either case, what would be the alternate methods to put together the message body in a way that could be understood by C++ code? And vice versa? The use of Java bindings to Proton-C is not the preferred way to go. Thanks and best regards, Leon
[MESSENGER] multilingual docs - reviews welcome
What I'm doing with Messenger Tutorial Docs -- Reviews Welcome
==
{
1. How it works
--
{
There will be a custom cmake target, probably called 'docs'.
When you 'make' this target, it runs a bunch of simple Messenger
tests in all supported languages.
The purpose of these tests is not the same as the other tests
that already exist; they are not trying to prove correctness
of all the Messenger features. Their purpose is to provide
the documents with good code snippets -- that so the same thing
in each language.
If any of these tests fail, we stop right there and the docs
do not get made.
After the tests run successfully, the code snippets are all
extracted.
The docs I write are kind of ... templates for docs. Each doc
gets expanded into L distinct docs, where L is the number of
languages that Messenger supports. So you get the cross-product
of docs and languages.
Each doc has little markers in it, like
pmdocproc 13
that tell the processor which snippet to put there.
Each program has little snippet-markers like:
/* pmdocproc 13 c */
code ( goes, here );
/* pmdocproc 13 end */
to tell the processor where to get the snippet from, and what
language it's in.
The markers that are inthe code are always comments -- however that
language makes its comments -- and must be on a line by themselves.
}
2. Why it works that way
{
We get two cool benefits out of this:
* You can see the whole doc tree in your favorite language,
and compare languages.
* The code snippets will never go out of date. If they
quite working, the docs don't get built.
}
3. Where I am Now
--
{
* the python version of the doc-maker is working
* I know how to integrate with cmake, and doing that now.
* I'll be away next week, but I would like to check stuff in
shortly after returning.
}
4. Feedback I would like
-
{
Anything you are inspired to volunteer, about any aspect of this.
I am attaching the Python doc maker below just in case anyone wants
to look at it. ( I was so uncomfortable with Python that I prototyped
the project in C, so I don't know how *pythonic* my code is )
Thanks in advance!
And please send any comments to the list.
}
5. The Python doc processor
--
{
#! /usr/bin/python
import os
import subprocess
import shutil
#
# First, run all the tests for each language.
# We will not create any documents if any of
# these tests fail.
#
def run_examples(languages):
saved_dir = os.getcwd()
for language in languages:
test_dir = './doc_examples/' + language
os.chdir ( test_dir )
subprocess.check_call ( ./run_all )
print -
print Tests in , test_dir , were successful.
print -
os.chdir ( saved_dir )
print \n=
print All language example tests were successful.
print =\n
#
# Make new output dirs for each language.
# These will hold final docs.
#
def make_output_dirs ( output_dir, html_dir, languages ):
if os.path.exists ( output_dir ):
shutil.rmtree ( output_dir )
os.mkdir ( output_dir )
if os.path.exists ( html_dir ):
shutil.rmtree ( html_dir )
os.mkdir ( html_dir )
for language in languages:
os.mkdir ( output_dir + '/' + language )
os.mkdir ( html_dir + '/' + language )
#
# For each example program name, there
# should be an instance of it in each
# example/language directory.
# I.e., example program foo should exist as
# doc_examples/c/foo.c
# doc_examples/rb/foo.rb
# doc_examples/py/foo.py
#
def make_example_file_names(example_dir, languages, example_names,
example_file_names ):
for language in languages:
RE: Deadlock in pn_messenger_stop? (C Qpid Library)
Hmm - perhaps I spoke too soon... This looked like it worked in that little test application but when I plugged it into my actual application, it seemed to work some of the times and not others. I then took your advice and introduced a join in the test application and as soon as I do so, stopping the subscriber on the subscriber thread stopped working (perhaps it never did work properly and this just flagged it). I have attached the latest source for it with the subscriber living entirely on its own thread, and with a join now introduced in main(). It deadlocks with the same backtrace as originally reported when trying to run pn_messenger_stop. Cheers, Frank -Original Message- From: Frank Quinn Sent: 14 June 2013 13:49 To: proton@qpid.apache.org Subject: RE: Deadlock in pn_messenger_stop? (C Qpid Library) Thanks Ted, I did try inserting a sleep of 1s between the send and the stop (I suspected the same thing you did), but the behaviour persisted so I don't think the problem was the time for the message to land. However, your suggestion to stop it in the subscriber thread did seem to work (still unsure why, but I'll not question it for now), thanks! Cheers, Frank -Original Message- From: Ted Ross [mailto:tr...@redhat.com] Sent: 14 June 2013 13:09 To: proton@qpid.apache.org Subject: Re: Deadlock in pn_messenger_stop? (C Qpid Library) I would suggest that you not stop the subscriber messenger in the main thread. Rather, stop it in the subscriber thread right before it exits. Alternatively (preferably), you should pthread_join the thread in main before stopping the messengers. It looks to me that you've got a race condition where main is stopping mSub at the same time the thread is processing messages on mSub. Keep in mind that pn_messenger_send is only going to block until the message has been pushed to the socket. It will not wait for the message to be received and processed by the other thread. -Ted On 06/14/2013 07:10 AM, Frank Quinn wrote: Hi Folks, See attached code: I'm encountering a deadlock when I try to stop messengers. The general workflow is: 1. Create pub and sub Messengers 2. Start the Messengers 3. Thread sub off onto its own thread as recv is a blocking call 4. Publish round trip from the pub messenger to the sub messenger with a destroy subject (recv is uninteruptable at the moment so this is our only to interrupt it) 5. Stop the messengers When I try and stop the messengers, the application deadlocks with the following backtrace (there is only one thread running at this point as the subscribe thread has since exited): Thread 1 (Thread 0x7f38181a4840 (LWP 6688)): #0 0x003518ce99ad in poll () at ../sysdeps/unix/syscall-template.S:81 #1 0x00309c226a1c in poll (__timeout=optimized out, __nfds=optimized out, __fds=optimized out) at /usr/include/bits/poll2.h:46 #2 pn_driver_wait_2 (d=d@entry=0x1a81140, timeout=optimized out, timeout@entry=-1) at /usr/src/debug/qpid-proton-0.4/proton-c/src/posix/driver.c:752 #3 0x00309c226c42 in pn_driver_wait (d=0x1a81140, timeout=timeout@entry=-1) at /usr/src/debug/qpid-proton-0.4/proton-c/src/posix/driver.c:807 #4 0x00309c2242d3 in pn_messenger_tsync (messenger=0x1a81050, predicate=0x309c222d80 pn_messenger_stopped, timeout=optimized out) at /usr/src/debug/qpid-proton-0.4/proton-c/src/messenger.c:623 #5 0x00400ffb in main () at qpid_deadlock_repro.c:123 Is this the correct workflow for this or am I missing a flush or unlock step somewhere along the way? Cheers, Frank -- -- Please consider the environment before printing this e-mail. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient or have received this e-mail in error, please advise the sender immediately by reply e-mail and delete this message and any attachments without retaining a copy. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden. -- -- *Please consider the environment before printing this email.* *Visit our website at http://www.nyse.com ** *** Note: The information contained in this message and any attachment to it is privileged, confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to the message, and please delete it from your system. Thank you. NYSE Euronext. *
