Re: Need advice on sys5 shm and zero copy sockets
Don't have original message, i reply to a reply sorry. On 2/8/13 4:22 AM, gary mazzaferro wrote: Hi, I was told to post this question here (Ken Merry), it would be a good place to get some help. I'm not sure this is doable without a kernel module, which I don't want to add. I'll explain what I'm attempting.. I'm designing a high speed rest motor for cloud execution environment. 1) I'd like to eliminate copy from the tcp stack to the application(s). 2) I'm also sharing the buffers across processes and jails. So I'd like to preserve the zero-copy in a msg pipe/unix socket 3) Some buffers will go to disk file systems. Wish list: 4) I'd like it to work with sctp because I like it for local networking :) 5) I'd like to provision memory pools on a per application/connection/ip port basis. Ultimate Goal: 6) Additionally, I'm injecting code from a foreign process into the workflow of another process (state machine). The connection between them will be a signal and shared state information. I'm assuming item (6) is a separate issue, but it may impact the direction.. I've tried shm with zero copy sockets with linux and it just will not work !! BTW, I'm returning to freebsd after far too many years cheers, gary For question (1) Are you using secure rest with TLS/SSL? Does your rest implementation uses openssl? If yes to both, then you don't need to use tcp. SSL already does all the work tcp does, resending lost/damaged packets and reordering received packets, so you can use plain udp without much pain. Openssl has since 0.9.8 this capability implemented with udp, don't remember if udplite. Check RFC 6347 HTH --- --- Eduardo Morras emorr...@yahoo.es ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need advice on sys5 shm and zero copy sockets
On 2/8/13 4:22 AM, gary mazzaferro wrote: Hi, I was told to post this question here (Ken Merry), it would be a good place to get some help. I'm not sure this is doable without a kernel module, which I don't want to add. I'll explain what I'm attempting.. I'm designing a high speed rest motor for cloud execution environment. 1) I'd like to eliminate copy from the tcp stack to the application(s). 2) I'm also sharing the buffers across processes and jails. So I'd like to preserve the zero-copy in a msg pipe/unix socket 3) Some buffers will go to disk file systems. Wish list: 4) I'd like it to work with sctp because I like it for local networking :) 5) I'd like to provision memory pools on a per application/connection/ip port basis. Ultimate Goal: 6) Additionally, I'm injecting code from a foreign process into the workflow of another process (state machine). The connection between them will be a signal and shared state information. I'm assuming item (6) is a separate issue, but it may impact the direction.. I've tried shm with zero copy sockets with linux and it just will not work !! BTW, I'm returning to freebsd after far too many years cheers, gary ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org this sound somewhat like what I did back in the 90s with BSD4.3 unfortunately it was not done with TCP (or sctp of course) what we did was to create a special shared memeory device driver. Then we added ioctls to the disk driver layer to write named blocks of memory from that device to the raw device (we didin't use a filesystem). We also changed the network drivers to use named blocks of memory in that device for packet reception. We added a special protocol which used recvmsg() and and sendmesg() to pass ownership of those named blocks between the process that had mmapped them and the protocol. The protocol had an IP header but also a small protocol specific header of our own.. we sent packets that were larger than a page. zero copy from disk-process, process-network and network-disk (and reverse of course) of course it was all on proprietary protocols so not of use to you. julian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need advice on sys5 shm and zero copy sockets
On 2/8/13 4:22 AM, gary mazzaferro wrote: Hi, I was told to post this question here (Ken Merry), it would be a good place to get some help. I'm not sure this is doable without a kernel module, which I don't want to add. I'll explain what I'm attempting.. I'm designing a high speed rest motor for cloud execution environment. 1) I'd like to eliminate copy from the tcp stack to the application(s). 2) I'm also sharing the buffers across processes and jails. So I'd like to preserve the zero-copy in a msg pipe/unix socket 3) Some buffers will go to disk file systems. Wish list: 4) I'd like it to work with sctp because I like it for local networking :) 5) I'd like to provision memory pools on a per application/connection/ip port basis. Ultimate Goal: 6) Additionally, I'm injecting code from a foreign process into the workflow of another process (state machine). The connection between them will be a signal and shared state information. I'm assuming item (6) is a separate issue, but it may impact the direction.. I've tried shm with zero copy sockets with linux and it just will not work !! BTW, I'm returning to freebsd after far too many years cheers, gary ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org this sound somewhat like what I did back in the 90s with BSD4.3 unfortunately it was not done with TCP (or sctp of course) what we did was to create a special shared memeory device driver. Then we added ioctls to the disk driver layer to write named blocks of memory from that device to the raw device (we didin't use a filesystem). We also changed the network drivers to use named blocks of memory in that device for packet reception. We added a special protocol which used recvmsg() and and sendmesg() to pass ownership of those named blocks between the process that had mmapped them and the protocol. The protocol had an IP header but also a small protocol specific header of our own.. we sent packets that were larger than a page. zero copy from disk-process, process-network and network-disk (and reverse of course) of course it was all on proprietary protocols so not of use to you. julian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need advice on sys5 shm and zero copy sockets
On 2 April 2013 20:07, Julian Elischer jul...@freebsd.org wrote: this sound somewhat like what I did back in the 90s with BSD4.3 unfortunately it was not done with TCP (or sctp of course) what we did was to create a special shared memeory device driver. Then we added ioctls to the disk driver layer to write named blocks of memory from that device to the raw device (we didin't use a filesystem). Funny that. I have to do something like this for this software radio NIC, that does a hundred or so megabytes a second of DMA. Eek. Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need advice on sys5 shm and zero copy sockets
On Monday, March 04, 2013 1:24:08 pm gary mazzaferro wrote: Hi, Thanks for all the help.. Looks like I'll move forward with recommending linux as a base for our new cloud execution containers. Personally, I thought freebsd would be a technically superior and longer term solution for scientific grid and cloud, but if I can't get support on best architectural practices. I'll need to move to something I that will be supported during eval, design and prototyping processes. There is not anything in stock FreeBSD that currently does zero-copy sockets for TCP. You could add such a thing, but you would have to implement your own. :( Some of the building blocks are in place. For example, you can create POSIX shared memory objects via shm_open() (and FreeBSD has an extension where a path of SHM_ANON creates anonymous, unnamed objects) and pass that fd into the kernel where an ioctl handler can map it into KVA using shm_map() and shm_unmap(). You'd have to change TCP to do something useful with this buffer however. -- John Baldwin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Need advice on sys5 shm and zero copy sockets
Hi, Thanks for all the help.. Looks like I'll move forward with recommending linux as a base for our new cloud execution containers. Personally, I thought freebsd would be a technically superior and longer term solution for scientific grid and cloud, but if I can't get support on best architectural practices. I'll need to move to something I that will be supported during eval, design and prototyping processes. -g On Thu, Feb 7, 2013 at 1:22 PM, gary mazzaferro ga...@oedata.com wrote: Hi, I was told to post this question here (Ken Merry), it would be a good place to get some help. I'm not sure this is doable without a kernel module, which I don't want to add. I'll explain what I'm attempting.. I'm designing a high speed rest motor for cloud execution environment. 1) I'd like to eliminate copy from the tcp stack to the application(s). 2) I'm also sharing the buffers across processes and jails. So I'd like to preserve the zero-copy in a msg pipe/unix socket 3) Some buffers will go to disk file systems. Wish list: 4) I'd like it to work with sctp because I like it for local networking :) 5) I'd like to provision memory pools on a per application/connection/ip port basis. Ultimate Goal: 6) Additionally, I'm injecting code from a foreign process into the workflow of another process (state machine). The connection between them will be a signal and shared state information. I'm assuming item (6) is a separate issue, but it may impact the direction.. I've tried shm with zero copy sockets with linux and it just will not work !! BTW, I'm returning to freebsd after far too many years cheers, gary ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
