Hi Stan, thanks. I already got the adapters. In case I encounter a problem, I will post a specific questions.
Best regards, Christoph Von: Smith, Stan [mailto:[email protected]] Gesendet: Montag, 19. August 2013 18:12 An: Fab Tillier; Christoph Müller Cc: '[email protected]' Betreff: RE: Use of I/O completion port with Winverbs Christoph, Located in the winOFED src tree under ulp\dapl2\dapl\NetworkDirect\NdHelper.cpp, specifically NdOpenAdapter(). Also in dapl_nd_device.c : dapls_ib_open_hca() could provide some NDv2 device handling insights. W.r.t. to which ND device, NdQueryAddressList() will return a list of ND devices on the system; also see dapli_GetIPaddr(). For the DAPL case, I match a device name from dat.conf ( hca0 ) and use the device name trailing instance number as an index into the NdQueryAddressList() of ND devices. Stan. From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Fab Tillier Sent: Sunday, August 18, 2013 12:19 PM To: Christoph Müller Cc: '[email protected]' Subject: Re: [ofw] Use of I/O completion port with Winverbs Ah, I didn’t realize Mellanox included the headers in their redist, ok. The docs for NDv2 are in the pipeline, but there’s quite a backlog for our tech writers so I don’t have an ETA unfortunately. The combination of NDv1 and NDK docs should get you on the right path in the meantime, and you can always send me questions. Provider discovery is largely the same as NDv1, though you call DllGetClassObject for the IND2Provider interface directly, rather than going through the IClassFactory interface. You should be able to interop between ND and librdma code, as well as between Windows and Linux no problem. If it doesn’t work then there’s a bug in the drivers somewhere. Of course you’ll be limited by the capabilities of NDv2 (no remote atomic, only reliable connected, no immediate data, etc.) -Fab From: Christoph Müller [mailto:[email protected]] Sent: Sunday, August 18, 2013 12:01 PM To: Fab Tillier Cc: '[email protected]' Subject: AW: Use of I/O completion port with Winverbs Hi Fab, thanks. I already got access to the IDN2Provider on Friday - the headers are also included in Mellanox' distribution just as Tzachi said. The problem on my side was that I was looking for some import libraries. I finally found some NDv1 documentation on MSDN explaining how to get the root object via the Winsock provider catalogue. However, there does not seem to be any NDv2 documentation yet, but the differences seem to be subtle. As I have only limited time, I will probably not be able to rewrite all of my code at once. I think that my NDv2 code will be able to talk to librdma peers as long as I use only basic functionality like remote read/write or inline send/receive. Is that correct? Best regards, Christoph Von meinem Windows Phone gesendet ________________________________ Von: Fab Tillier<mailto:[email protected]> Gesendet: 18.08.2013 07:38 An: Christoph Müller<mailto:[email protected]> Cc: '[email protected]'<mailto:[email protected]> Betreff: RE: Use of I/O completion port with Winverbs Hi Christoph, You can get the ND header files (included in the NetworkDirect DDK zip file) here: http://www.microsoft.com/en-us/download/details.aspx?id=36043 There is no dependency on the HPC Pack - you just need drivers that support ND (ideally NDv2, which the Mellanox 4.40 release does). Using ND should give you the greatest hardware independence too, though by design ND exposes the lowest common denominator functionality of iWARP and InfiniBand, so some functionality is not available (e.g. RDMA atomics). Don’t hesitate to ask if you have questions. -Fab From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Christoph Müller Sent: Thursday, August 15, 2013 6:04 AM To: Tzachi Dar; [email protected]<mailto:[email protected]> Subject: Re: [ofw] Use of I/O completion port with Winverbs Thank you. I was not aware that the SDK including NDv2 is distributed with Mellanox’ drivers. In this case, I will give it a try. Thanks again, Christoph Von: Tzachi Dar [mailto:[email protected]] Gesendet: Donnerstag, 15. August 2013 14:43 An: Christoph Müller; [email protected]<mailto:[email protected]> Betreff: RE: Use of I/O completion port with Winverbs You only need to download Mellanox drivers 4.40 for windows, and you will have nd support. http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers Thanks Tzachi From: Christoph Müller [mailto:[email protected]] Sent: Thursday, August 15, 2013 3:32 PM To: Tzachi Dar; [email protected]<mailto:[email protected]> Subject: AW: Use of I/O completion port with Winverbs ndinstall is part of the HPC pack distribution? My main problem with ND is to find out how to get it... Best regards, Christoph Von meinem Windows Phone gesendet ________________________________ Von: Tzachi Dar<mailto:[email protected]> Gesendet: 15.08.2013 14:28 An: Christoph Müller<mailto:[email protected]>; [email protected]<mailto:[email protected]> Betreff: RE: Use of I/O completion port with Winverbs Please note that the windows APIs are changing from time to time and ND is probably the one that will stay longer since it is supported by MS. I don’t think that you need to install the hpc pack for it. You simply need to run ndinstall and then you can start running nd programs. Thanks Tzachi From: Christoph Müller [mailto:[email protected]] Sent: Thursday, August 15, 2013 3:21 PM To: Tzachi Dar; [email protected]<mailto:[email protected]> Subject: AW: Use of I/O completion port with Winverbs Actually, I am in the process of deciding which API to use (although I have existing librdma code). NetworkDirect indeed looks promising, but as I understand, I need HPC Pack 2012 for that which I cannot install on our cluster in the short term. Best regards, Christoph Von meinem Windows Phone gesendet ________________________________ Von: Tzachi Dar<mailto:[email protected]> Gesendet: 15.08.2013 14:15 An: Christoph Müller<mailto:[email protected]>; [email protected]<mailto:[email protected]> Betreff: RE: Use of I/O completion port with Winverbs can you write your application with Network Direct? If so, there are overlapped objects there that you should be able to connect to the completion ports. Thanks Tzachi From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Christoph Müller Sent: Thursday, August 15, 2013 3:10 PM To: [email protected]<mailto:[email protected]> Subject: [ofw] Use of I/O completion port with Winverbs Hi all, I was wondering whether it is possible to use CreateIoCompletionPort etc. with IWVConnectEndpoints. I can see that the implementation inherits a handle from CWVBase, but I do not see how I could use this information for creating an IOCP and, of course, I do not know whether it would work. Furthermore, I would prefer continuing to use the RDMA library, but I see absolutely no way to create an IOCP here. So me questions are: Is it possible to use IOCPs with winverbs? Is it possible to use IOCPs with the RDMA library? Is there sample code for either of it? Thanks in advance, Christoph -- Visualisierungsinstitut der Universität Stuttgart Allmandring 19, 70569 Stuttgart E-Mail: [email protected]<mailto:[email protected]> Telefon: (07 11) 685 - 88626
_______________________________________________ ofw mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
