There is a zero copy API for JZMQ with this commit: https://github.com/zeromq/jzmq/commit/336a8a1d3269483ba67bb903c8dc0b1482079a33
Cheers On Sat, Feb 16, 2013 at 5:47 PM, Trevor Bernard <[email protected]> wrote: > @gonzalo Did you do anything special on the client side to read the > direct bytebuffer? My recvZeroCopy isn't working ATM. > > On Sat, Feb 16, 2013 at 4:28 PM, Trevor Bernard > <[email protected]> wrote: >> Also you'll need to use ByteBuffer.allocateDirect(...) in order for it >> to work. I explicitly didn't check GetDirectBufferAddress for perf >> reasons. >> >> For the JZMQ api, I used sendZeroCopy and recvZeroCopy for a lack of a >> better/fancier name. ;) >> >> -Trev >> >> On Sat, Feb 16, 2013 at 4:26 PM, Trevor Bernard >> <[email protected]> wrote: >>> I added support for zero copy send and recv. >>> >>> https://github.com/zeromq/jzmq/issues/170 >>> >>> I haven't had time to test this but it compiles and the unit tests pass. >>> >>> -Trev >>> >>> On Fri, Feb 15, 2013 at 5:26 PM, The Carrolls <[email protected]> >>> wrote: >>>> Awesome! I'll be watching for it then. >>>> >>>> On 02/15/13 13:25, Trevor Bernard wrote: >>>>> This looks simple enough. I should be able to port this into a jzmq >>>>> branch fairly easily. >>>>> >>>>> I'll likely do it over the weekend. >>>>> >>>>> -Trev >>>>> >>>>> On Fri, Feb 15, 2013 at 1:55 PM, gonzalo diethelm <[email protected]> >>>>> wrote: >>>>>> I remembered that I did implement this for a tentative Java binding for >>>>>> Crossroads IO. The relevant methods where send() and recv(). This all >>>>>> worked and performed MUCH better that current binding. Some details >>>>>> follow; if more is required, let me know. >>>>>> >>>>>> Usage: >>>>>> >>>>>> int size = 128; >>>>>> ByteBuffer bb = ByteBuffer.allocateDirect(size); >>>>>> ... >>>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0); >>>>>> ... >>>>>> int rc = xs.xs_recv(sock, bb, 0, size, 0); >>>>>> >>>>>> --------------------- >>>>>> In Java: >>>>>> >>>>>> public class XsLibrary { >>>>>> ... >>>>>> public native int xs_send(long socket, >>>>>> ByteBuffer buffer, >>>>>> int offset, >>>>>> int length, >>>>>> int flags); >>>>>> public native int xs_recv(long socket, >>>>>> ByteBuffer buffer, >>>>>> int offset, >>>>>> int length, >>>>>> int flags); >>>>>> } >>>>>> >>>>>> --------------------- >>>>>> In C (the JNI part): >>>>>> >>>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1send(JNIEnv* >>>>>> env, >>>>>> >>>>>> jobject obj, >>>>>> jlong >>>>>> socket, >>>>>> >>>>>> jobject buffer, >>>>>> jint >>>>>> offset, >>>>>> jint >>>>>> length, >>>>>> jint >>>>>> flags) >>>>>> { >>>>>> void* sock = 0; >>>>>> jbyte* buf = 0; >>>>>> int ret = 0; >>>>>> >>>>>> sock = (void*) socket; >>>>>> XS_ASSERT(sock); >>>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer); >>>>>> XS_ASSERT(buf); >>>>>> ret = xs_send(sock, buf, length, flags); >>>>>> return ret; >>>>>> } >>>>>> >>>>>> JNIEXPORT jint JNICALL Java_io_crossroads_jni_XsLibrary_xs_1recv(JNIEnv* >>>>>> env, >>>>>> >>>>>> jobject obj, >>>>>> jlong >>>>>> socket, >>>>>> >>>>>> jobject buffer, >>>>>> jint >>>>>> offset, >>>>>> jint >>>>>> length, >>>>>> jint >>>>>> flags) >>>>>> { >>>>>> void* sock = 0; >>>>>> jbyte* buf = 0; >>>>>> int ret = 0; >>>>>> >>>>>> sock = (void*) socket; >>>>>> XS_ASSERT(sock); >>>>>> buf = (jbyte*) (*env)->GetDirectBufferAddress(env, buffer); >>>>>> XS_ASSERT(buf); >>>>>> ret = xs_recv(sock, buf, length, flags); >>>>>> return ret; >>>>>> } >>>>>> >>>>>> --------------------- >>>>>> >>>>>> -- >>>>>> Gonzalo Diethelm >>>>>> DCV Chile >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: [email protected] [mailto:zeromq-dev- >>>>>>> [email protected]] On Behalf Of Trevor Bernard >>>>>>> Sent: Friday, February 15, 2013 2:02 PM >>>>>>> To: [email protected]; ZeroMQ development list >>>>>>> Subject: Re: [zeromq-dev] Zero Copy in Java jzmq >>>>>>> >>>>>>>> I'm thinking of extending jzmq to optionally allow the zero-copy >>>>>>>> technique using the concepts outlined by Martin Thompson here: >>>>>>> That's awesome >>>>>>> >>>>>>>> I'm not 100% sure I'll do it, I need some more experimentation first, >>>>>>>> but is there any interest or words of advice if someone's tried this >>>>>>>> already? >>>>>>> I'd love for send/recv to have a ByteBuffer API. >>>>>>> >>>>>>> Some resources and general suggestions: >>>>>>> >>>>>>> Have a peak at zmq_msg_t in zmq.h. >>>>>>> >>>>>>> Defines ZMTP/2.0 Spec: >>>>>>> * http://rfc.zeromq.org/spec:15 >>>>>>> >>>>>>> Also depending on the size of the msg, it may be allocated on the stack >>>>>>> or >>>>>>> heap. >>>>>>> * http://api.zeromq.org/3-2:zmq-msg-init-size >>>>>>> >>>>>>> At some point you'll have to access a malloc'ed array from Java/JNI. >>>>>>> * >>>>>>> http://docs.oracle.com/javase/6/docs/technotes/guides/jni/spec/functions. >>>>>>> html#nio_support >>>>>>> >>>>>>> Something like this would do the trick. This wraps a native pointer in a >>>>>>> ByteBufer >>>>>>> >>>>>>> void *data = ... >>>>>>> jobject bb = (*env)->NewDirectByteBuffer(env, (void*) data, >>>>>>> sizeof(zmq_msg_t)); >>>>>>> >>>>>>> Here is another option: Use a library like javolution to do your >>>>>>> mapping: >>>>>>> >>>>>>> http://javolution.org/target/site/apidocs/javolution/io/Struct.html >>>>>>> >>>>>>> class Message extends Struct { >>>>>>> Unsigned8 _ = new Unsigned8(32); >>>>>>> Message() { >>>>>>> setByteBuffer(Message.nativeBuffer(), 0); >>>>>>> } >>>>>>> private static native ByteBuffer nativeBuffer(); } >>>>>>> >>>>>>> Something along those lines. >>>>>>> >>>>>>> Hope this helps. >>>>>>> >>>>>>> -Trev >>>>>>> _______________________________________________ >>>>>>> zeromq-dev mailing list >>>>>>> [email protected] >>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>> >>>>>> ----------------------------------------- >>>>>> Declaración de confidencialidad: Este Mensaje esta destinado para >>>>>> el uso de la o las personas o entidades a quien ha sido dirigido y >>>>>> puede contener información reservada y confidencial que no puede >>>>>> ser divulgada, difundida, ni aprovechada en forma alguna. El uso no >>>>>> autorizado de la información contenida en este correo podrá ser >>>>>> sancionado de conformidad con la ley chilena. >>>>>> Si usted ha recibido este correo electrónico por error, le pedimos >>>>>> eliminarlo junto con los archivos adjuntos y avisar inmediatamente >>>>>> al remitente, respondiendo este mensaje. >>>>>> >>>>>> "Before printing this e-mail think if is really necesary". >>>>>> Disclosure: This Message is to be used by the individual, >>>>>> individuals or entities that it is addressed to and may include >>>>>> private and confidential information that may not be disclosed, >>>>>> made public nor used in any way at all. Unauthorized use of the >>>>>> information in this electronic mail message may be subject to the >>>>>> penalties set forth by Chilean law. >>>>>> If you have received this electronic mail message in error, we ask >>>>>> you to destroy the message and its attached file(s) and to >>>>>> immediately notify the sender by answering this message. >>>> >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
