Re: [ofa-general] Re: [PATCH RFC] RDMA/CMA: Allocate PS_TCP ports from the host TCP port space.
From: Roland Dreier [EMAIL PROTECTED] Date: Mon, 20 Aug 2007 18:16:54 -0700 And direct data placement really does give you a factor of two at least, because otherwise you're stuck receiving the data in one buffer, looking at some of the data at least, and then figuring out where to copy it. And memory bandwidth is if anything becoming more valuable; maybe LRO + header splitting + page remapping tricks can get you somewhere but as NCPUS grows then it seems the TLB shootdown cost of page flipping is only going to get worse. As Herbert has said already, people can code for this just like they have to code for RDMA. There is no fundamental difference from converting an application to sendfile or similar. The only thing this needs is a recvmsg_I_dont_care_where_the_data_is() call. There are no alignment issues unless you are trying to push this data directly into the page cache. Couple this with a card that makes sure that on a per-page basis, only data for a particular flow (or group of flows) will accumulate. People already make cards that can do stuff like this, it can be done statelessly with an on-chip dynamically maintained flow table. And best yet it doesn't turn off every feature in the networking nor bypass it for the actual protocol processing. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
From: Krishna Kumar2 [EMAIL PROTECTED] Date: Fri, 17 Aug 2007 11:36:03 +0530 I ran 3 iterations of 45 sec tests (total 1 hour 16 min, but I will run a longer one tonight). The results are (results in KB/s, and %): I ran a 8.5 hours run with no batching + another 8.5 hours run with batching (Buffer sizes: 32 128 512 4096 16384, Threads: 1 8 32, Each test run time: 3 minutes, Iterations to average: 5). TCP seems to get a small improvement. Using 16K buffer size really isn't going to keep the pipe full enough for TSO. And realistically applications queue much more data at a time. Also, with smaller buffer sizes can have negative effects for the dynamic receive and send buffer growth algorithm the kernel uses, it might consider the connection application limited for too long. I would really prefer to see numbers that use buffer sizes more in line with the amount of data that is typically inflight on a 1G connection on a local network. Do a tcpdump during the height of the transfer to see about what this value is. When an ACK comes in, compare the sequence number it's ACK'ing with the sequence number of the most recently sent frame. The difference is approximately the pipe size at maximum congestion window assuming a loss free local network. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 5/7] osm: QoS - compiling policy file parser
Michael S. Tsirkin wrote: Quoting Michael S. Tsirkin [EMAIL PROTECTED]: Subject: Re: [PATCH 5/7] osm: QoS - compiling policy file parser Quoting Yevgeny Kliteynik [EMAIL PROTECTED]: Subject: [PATCH 5/7] osm: QoS - compiling policy file parser Compiling QoS policy file parser. C H files are generated from Lex Yacc files and the generated files are compiled. Generated files are not included in git, but they do get included in RPM as they are also created by 'make dist'. Signed-off-by: Yevgeny Kliteynik [EMAIL PROTECTED] --- opensm/include/Makefile.am |2 ++ opensm/opensm/Makefile.am | 28 +++- opensm/opensm/configure.in |2 ++ 3 files changed, 31 insertions(+), 1 deletions(-) diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am index ea62507..b83205f 100644 --- a/opensm/include/Makefile.am +++ b/opensm/include/Makefile.am @@ -90,6 +90,8 @@ EXTRA_DIST = \ $(srcdir)/opensm/osm_state_mgr_ctrl.h \ $(srcdir)/opensm/osm_perfmgr.h \ $(srcdir)/opensm/osm_perfmgr_db.h \ + $(srcdir)/opensm/osm_qos_policy.h \ + $(srcdir)/opensm/osm_qos_parser_y.h \ $(srcdir)/complib/cl_thread_osd.h \ $(srcdir)/complib/cl_packon.h \ $(srcdir)/complib/cl_atomic_osd.h \ diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am index 6dfa824..6c7208d 100644 --- a/opensm/opensm/Makefile.am +++ b/opensm/opensm/Makefile.am @@ -56,7 +56,21 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \ osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \ osm_vl15intf.c osm_vl_arb_rcv.c \ st.c osm_perfmgr.c osm_perfmgr_db.c \ -osm_event_plugin.c osm_dump.c +osm_event_plugin.c osm_dump.c \ +osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c + +osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y $(srcdir)/../include/opensm/osm_qos_policy.h + $(YACC) -y -d $(srcdir)/osm_qos_parser.y + mv y.tab.c osm_qos_parser_y.c + mv y.tab.h osm_qos_parser_y.h + cp -f osm_qos_parser_y.c $(srcdir)/ + cp -f osm_qos_parser_y.h $(srcdir)/../include/opensm/ + +osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l $(srcdir)/../include/opensm/osm_qos_policy.h + $(LEX) $(srcdir)/osm_qos_parser.l + mv lex.yy.c osm_qos_parser_l.c + cp -f osm_qos_parser_l.c $(srcdir)/ + Why do you use these mv tricks? Can you not use the -o flag to generate the output file with the proper name? Good idea Alternatively, if you do not want to use flex/bison extensions, you can use -t flag to have lex output to standard output, and -b flag to supply prefix to yacc. BTW, I think it's a good idea to also use -p prefix to reduce the global namespace pollution by yacc. OK. It affects patches 2/7 and 5/7 - I'll repost them. -- Yevgeny ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] ofa_1_2_kernel 20070821-0100 daily build status
This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2 Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.12 Passed on powerpc with linux-2.6.18 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on ia64 with linux-2.6.13 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.19 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.20 Passed on ppc64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.13 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on ppc64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.16.21-0.8-default Failed: ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] SDP overhead
Hello, Can somebody tell me what the overhead is of an SDP packet when the packet is completely filled with data? The numbers I want to have are: max data length of 1 packet total header length of 1 packet Regards, Koen Segers *** Disclaimer *** Vlaamse Radio- en Televisieomroep Auguste Reyerslaan 52, 1043 Brussel nv van publiek recht BTW BE 0244.142.664 RPR Brussel http://www.vrt.be/disclaimer ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH - 11] ipoib - add LSO support
Eli Cohen wrote: On Mon, 2007-08-20 at 18:48 +0300, Or Gerlitz wrote: I see that the patch adds the NETIF_F_TSO flag to the device features but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? if yes, what would it take SW wise to support it? UDP LSO is not supported by the HW. I see. Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and thinking about the connectx TCP segmentation offloading a little further I am somehow confused: With the ipoib connected mode a typical MTU exposed to the OS is 64K, where data goes over IB RC (soon to be UC) transport meaning that the HCA does the fragmentation / reassembly at the QP (IB L4) level allowing to send one 64K IB packet over a path whose Link layer (IB L2) MTU is 2k. So TCP can use segments size upto 64K and UDP can send datagram's whose size is upto 64K without IP fragmentation. With all this at hand, what does LSO buys you at all? Or. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] ofa_1_2_c_kernel 20070821-0200 daily build status
This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_2/linux-2.6.git git_branch: ofed_1_2_c Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-mlx4-mod --with-core-mod --with-addr_trans-mod --with-rds-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on powerpc with linux-2.6.15 Passed on x86_64 with linux-2.6.16 Passed on powerpc with linux-2.6.12 Passed on x86_64 with linux-2.6.18 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.13 Passed on x86_64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.14 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.12 Passed on ppc64 with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on ia64 with linux-2.6.19 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.22 Passed on ppc64 with linux-2.6.14 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.22 Passed on ppc64 with linux-2.6.18-8.el5 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on x86_64 with linux-2.6.9-22.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.9-34.ELsmp Failed: Build failed on powerpc with linux-2.6.19 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of â-â /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of â-â /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of â-â make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.19_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.19' make: *** [kernel] Error 2 -- Build failed on powerpc with linux-2.6.17 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of â-â /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:834: error: invalid type argument of â-â /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:835: error: invalid type argument of â-â make[4]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband/hw/ehca] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.17_powerpc_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/powerpc/linux-2.6.17' make: *** [kernel] Error 2 -- Build failed on powerpc with linux-2.6.18 Log: /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers/infiniband/hw/ehca/ehca_main.c:831: error: invalid type argument of â-â /home/vlad/tmp/ofa_1_2_c_kernel-20070821-0200_linux-2.6.18_powerpc_check/drivers
[ofa-general] Re: [PATCH - 11] ipoib - add LSO support
Quoting Or Gerlitz [EMAIL PROTECTED]: Subject: Re: [PATCH - 11] ipoib - add LSO support Eli Cohen wrote: On Mon, 2007-08-20 at 18:48 +0300, Or Gerlitz wrote: I see that the patch adds the NETIF_F_TSO flag to the device features but not the NETIF_F_UFO flag. Is UDP LSO supported by the connectX HW? if yes, what would it take SW wise to support it? UDP LSO is not supported by the HW. I see. Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and thinking about the connectx TCP segmentation offloading a little further I am somehow confused: With the ipoib connected mode a typical MTU exposed to the OS is 64K, where data goes over IB RC (soon to be UC) transport meaning that the HCA does the fragmentation / reassembly at the QP (IB L4) level allowing to send one 64K IB packet over a path whose Link layer (IB L2) MTU is 2k. So TCP can use segments size upto 64K and UDP can send datagram's whose size is upto 64K without IP fragmentation. With all this at hand, what does LSO buys you at all? If you read these patches closely, you'll see that LSO is disabled for connected mode. -- MST ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH - 11] ipoib - add LSO support
On Tue, 2007-08-21 at 12:51 +0300, Or Gerlitz wrote: Reading http://en.wikipedia.org/wiki/TCP_segmentation_offload and thinking about the connectx TCP segmentation offloading a little further I am somehow confused: With the ipoib connected mode a typical MTU exposed to the OS is 64K, where data goes over IB RC (soon to be UC) transport meaning that the HCA does the fragmentation / reassembly at the QP (IB L4) level allowing to send one 64K IB packet over a path whose Link layer (IB L2) MTU is 2k. So TCP can use segments size upto 64K and UDP can send datagram's whose size is upto 64K without IP fragmentation. With all this at hand, what does LSO buys you at all? You have the option to work in either connected or datagram mode, and some might prefer to use datagram mode for various reasons. In these cases you will want the benefits that LSO provides. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH - 11] ipoib - add LSO support
Michael S. Tsirkin wrote: Quoting Or Gerlitz [EMAIL PROTECTED]: With the ipoib connected mode a typical MTU exposed to the OS is 64K, where data goes over IB RC (soon to be UC) transport meaning that the HCA does the fragmentation / reassembly at the QP (IB L4) level allowing to send one 64K IB packet over a path whose Link layer (IB L2) MTU is 2k. So TCP can use segments size upto 64K and UDP can send datagram's whose size is upto 64K without IP fragmentation. With all this at hand, what does LSO buys you at all? If you read these patches closely, you'll see that LSO is disabled for connected mode. This makes me more confused: if someone buys connectX and uses IPoIB, since connected mode provides the best performance for both TCP and UDP unicast traffic they would not bother to use the datagram mode. And if they have UDP multicast traffic, they would use the UD QP of IPoIB (which means they use datagram mode for multicast). So the only case where I see usage of the LSO stateless offloading is when working against device which does not support connected mode, such as IB/Ethernet IP gateways, but as you may know, very much of the traffic over these gateways is IP multicast based... Or. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 1/7] osm: QoS - adding new PathRecord fields
Hi Sasha,
Sasha Khapyorsky wrote:
Hi Yevgeny!
On 15:03 Mon 20 Aug , Yevgeny Kliteynik wrote:
Adding QoS fields to PathRecord.
Signed-off-by: Yevgeny Kliteynik [EMAIL PROTECTED]
Some comments are below.
---
infiniband-diags/src/saquery.c |8 +-
opensm/include/iba/ib_types.h | 158 +++
opensm/opensm/osm_helper.c |8 +-
opensm/opensm/osm_sa_multipath_record.c |2 +-
opensm/opensm/osm_sa_path_record.c |4 +-
opensm/osmtest/osmtest.c|2 +-
6 files changed, 154 insertions(+), 28 deletions(-)
diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c
index 522399e..f085854 100644
--- a/infiniband-diags/src/saquery.c
+++ b/infiniband-diags/src/saquery.c
@@ -188,7 +188,7 @@ static void
print_path_record(ib_path_rec_t *p_pr)
{
printf(PathRecord dump:\n
- \t\tresv0...0x%016 PRIx64 \n
+ \t\tservice_id..0x%016 PRIx64 \n
\t\tdgid0x%016 PRIx64 :
0x%016 PRIx64 \n
\t\tsgid0x%016 PRIx64 :
@@ -199,7 +199,7 @@ print_path_record(ib_path_rec_t *p_pr)
\t\ttclass..0x%X\n
\t\tnum_path_revers.0x%X\n
\t\tpkey0x%X\n
- \t\tsl..0x%X\n
+ \t\tqos_class_sl0x%X\n
I think it is more useful to have separate printouts for sl and
qos_class.
Agree
\t\tmtu.0x%X\n
\t\trate0x%X\n
\t\tpkt_life0x%X\n
@@ -207,7 +207,7 @@ print_path_record(ib_path_rec_t *p_pr)
\t\tresv2...0x%X\n
\t\tresv3...0x%X\n
,
- *(uint64_t*)p_pr-resv0,
+ cl_ntoh64( p_pr-service_id ),
cl_ntoh64( p_pr-dgid.unicast.prefix ),
cl_ntoh64( p_pr-dgid.unicast.interface_id ),
cl_ntoh64( p_pr-sgid.unicast.prefix ),
@@ -218,7 +218,7 @@ print_path_record(ib_path_rec_t *p_pr)
p_pr-tclass,
p_pr-num_path,
cl_ntoh16( p_pr-pkey ),
- cl_ntoh16( p_pr-sl ),
+ cl_ntoh16( p_pr-qos_class_sl ),
p_pr-mtu,
p_pr-rate,
p_pr-pkt_life,
diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h
index 490e3fc..f96893a 100644
--- a/opensm/include/iba/ib_types.h
+++ b/opensm/include/iba/ib_types.h
@@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN
const uint8_t class_code)
#define IB_SMINFO_STATE_MASTER 3
/**/
+/d* IBA Base: Constants/IB_PATH_REC_SL_MASK
+* NAME
+* IB_PATH_REC_SL_MASK
+*
+* DESCRIPTION
+* Mask for the sl field for path record
+*
+* SOURCE
+*/
+#define IB_PATH_REC_SL_MASK0x000F
+
+/d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK
+* NAME
+* IB_PATH_REC_QOS_CLASS_MASK
+*
+* DESCRIPTION
+* Mask for the QoS class field for path record
+*
+* SOURCE
+*/
+#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0
+
/d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK
* NAME
* IB_PATH_REC_SELECTOR_MASK
@@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid)
*/
#include complib/cl_packon.h
typedef struct _ib_path_rec {
- uint8_t resv0[8];
+ ib_net64_t service_id;
ib_gid_t dgid;
ib_gid_t sgid;
ib_net16_t dlid;
@@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec {
uint8_t tclass;
uint8_t num_path;
ib_net16_t pkey;
- ib_net16_t sl;
+ ib_net16_t qos_class_sl;
uint8_t mtu;
uint8_t rate;
uint8_t pkt_life;
@@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec {
#include complib/cl_packoff.h
/*
* FIELDS
-* resv0
-* Reserved bytes.
+* service_id
+* Service ID for QoS.
*
* dgid
* GID of destination port.
@@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec {
* pkey
* Partition key (P_Key) to use on this path.
*
-* resv1
-* Reserved byte.
-*
-* sl
-* Service level to use on this path.
+* qos_class_sl
+* QoS class and service level to use on this path.
*
* mtu
* MTU and MTU selector fields to use on this path
@@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec {
*/
/* Path Record Component Masks */
+#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)1))
#define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)2))
#define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)3))
#define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)4))
@@ -2372,7 +2392,7 @@ typedef struct
[ofa-general] [PATCH 1/7 V2] osm: QoS - adding new PathRecord fields
Adding QoS fields to PathRecord.
Signed-off-by: Yevgeny Kliteynik [EMAIL PROTECTED]
---
infiniband-diags/src/saquery.c |8 +-
opensm/include/iba/ib_types.h | 149 ---
opensm/opensm/osm_helper.c |8 +-
opensm/opensm/osm_sa_multipath_record.c |3 +-
opensm/opensm/osm_sa_path_record.c |6 +-
opensm/osmtest/osmtest.c|4 +-
6 files changed, 154 insertions(+), 24 deletions(-)
diff --git a/infiniband-diags/src/saquery.c b/infiniband-diags/src/saquery.c
index 522399e..a9a8da4 100644
--- a/infiniband-diags/src/saquery.c
+++ b/infiniband-diags/src/saquery.c
@@ -188,7 +188,7 @@ static void
print_path_record(ib_path_rec_t *p_pr)
{
printf(PathRecord dump:\n
- \t\tresv0...0x%016 PRIx64 \n
+ \t\tservice_id..0x%016 PRIx64 \n
\t\tdgid0x%016 PRIx64 :
0x%016 PRIx64 \n
\t\tsgid0x%016 PRIx64 :
@@ -199,6 +199,7 @@ print_path_record(ib_path_rec_t *p_pr)
\t\ttclass..0x%X\n
\t\tnum_path_revers.0x%X\n
\t\tpkey0x%X\n
+ \t\tqos_class...0x%X\n
\t\tsl..0x%X\n
\t\tmtu.0x%X\n
\t\trate0x%X\n
@@ -207,7 +208,7 @@ print_path_record(ib_path_rec_t *p_pr)
\t\tresv2...0x%X\n
\t\tresv3...0x%X\n
,
- *(uint64_t*)p_pr-resv0,
+ cl_ntoh64( p_pr-service_id ),
cl_ntoh64( p_pr-dgid.unicast.prefix ),
cl_ntoh64( p_pr-dgid.unicast.interface_id ),
cl_ntoh64( p_pr-sgid.unicast.prefix ),
@@ -218,7 +219,8 @@ print_path_record(ib_path_rec_t *p_pr)
p_pr-tclass,
p_pr-num_path,
cl_ntoh16( p_pr-pkey ),
- cl_ntoh16( p_pr-sl ),
+ ib_path_rec_qos_class( p_pr ),
+ ib_path_rec_sl( p_pr ),
p_pr-mtu,
p_pr-rate,
p_pr-pkt_life,
diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h
index 490e3fc..ada4428 100644
--- a/opensm/include/iba/ib_types.h
+++ b/opensm/include/iba/ib_types.h
@@ -1647,6 +1647,28 @@ static inline boolean_t OSM_API ib_class_is_rmpp(IN
const uint8_t class_code)
#define IB_SMINFO_STATE_MASTER 3
/**/
+/d* IBA Base: Constants/IB_PATH_REC_SL_MASK
+* NAME
+* IB_PATH_REC_SL_MASK
+*
+* DESCRIPTION
+* Mask for the sl field for path record
+*
+* SOURCE
+*/
+#define IB_PATH_REC_SL_MASK0x000F
+
+/d* IBA Base: Constants/IB_PATH_REC_QOS_CLASS_MASK
+* NAME
+* IB_PATH_REC_QOS_CLASS_MASK
+*
+* DESCRIPTION
+* Mask for the QoS class field for path record
+*
+* SOURCE
+*/
+#define IB_PATH_REC_QOS_CLASS_MASK 0xFFF0
+
/d* IBA Base: Constants/IB_PATH_REC_SELECTOR_MASK
* NAME
* IB_PATH_REC_SELECTOR_MASK
@@ -2286,7 +2308,7 @@ ib_gid_get_guid(IN const ib_gid_t * const p_gid)
*/
#include complib/cl_packon.h
typedef struct _ib_path_rec {
- uint8_t resv0[8];
+ ib_net64_t service_id;
ib_gid_t dgid;
ib_gid_t sgid;
ib_net16_t dlid;
@@ -2295,7 +2317,7 @@ typedef struct _ib_path_rec {
uint8_t tclass;
uint8_t num_path;
ib_net16_t pkey;
- ib_net16_t sl;
+ ib_net16_t qos_class_sl;
uint8_t mtu;
uint8_t rate;
uint8_t pkt_life;
@@ -2306,8 +2328,8 @@ typedef struct _ib_path_rec {
#include complib/cl_packoff.h
/*
* FIELDS
-* resv0
-* Reserved bytes.
+* service_id
+* Service ID for QoS.
*
* dgid
* GID of destination port.
@@ -2335,11 +2357,8 @@ typedef struct _ib_path_rec {
* pkey
* Partition key (P_Key) to use on this path.
*
-* resv1
-* Reserved byte.
-*
-* sl
-* Service level to use on this path.
+* qos_class_sl
+* QoS class and service level to use on this path.
*
* mtu
* MTU and MTU selector fields to use on this path
@@ -2360,6 +2379,7 @@ typedef struct _ib_path_rec {
*/
/* Path Record Component Masks */
+#define IB_PR_COMPMASK_SERVICEID (CL_HTON64(((uint64_t)1)1))
#define IB_PR_COMPMASK_DGID (CL_HTON64(((uint64_t)1)2))
#define IB_PR_COMPMASK_SGID (CL_HTON64(((uint64_t)1)3))
#define IB_PR_COMPMASK_DLID (CL_HTON64(((uint64_t)1)4))
@@ -2372,7 +2392,7 @@ typedef struct _ib_path_rec {
#define IB_PR_COMPMASK_REVERSIBLE(CL_HTON64(((uint64_t)1)11))
#define IB_PR_COMPMASK_NUMBPATH (CL_HTON64(((uint64_t)1)12))
#define IB_PR_COMPMASK_PKEY (CL_HTON64(((uint64_t)1)13))
[ofa-general] [PATCH 3/7 V2] osm: QoS policy C H files
QoS policy data structures and functions
Signed-off-by: Yevgeny Kliteynik [EMAIL PROTECTED]
---
opensm/include/opensm/osm_qos_policy.h | 188 +++
opensm/opensm/osm_qos_policy.c | 901
2 files changed, 1089 insertions(+), 0 deletions(-)
create mode 100644 opensm/include/opensm/osm_qos_policy.h
create mode 100644 opensm/opensm/osm_qos_policy.c
diff --git a/opensm/include/opensm/osm_qos_policy.h
b/opensm/include/opensm/osm_qos_policy.h
new file mode 100644
index 000..05b8dc9
--- /dev/null
+++ b/opensm/include/opensm/osm_qos_policy.h
@@ -0,0 +1,188 @@
+/*
+ * Copyright (c) 2004-2006 Voltaire, Inc. All rights reserved.
+ * Copyright (c) 2002-2005 Mellanox Technologies LTD. All rights reserved.
+ * Copyright (c) 1996-2003 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer in the documentation and/or other materials
+ *provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+/*
+ * Abstract:
+ *Declaration of OSM QoS Policy data types and functions.
+ *
+ * Environment:
+ *Linux User Mode
+ *
+ * Author:
+ *Yevgeny Kliteynik, Mellanox
+ */
+
+#ifndef OSM_QOS_POLICY_H
+#define OSM_QOS_POLICY_H
+
+#include stdio.h
+#include assert.h
+#include stdlib.h
+#include string.h
+#include ctype.h
+#include iba/ib_types.h
+#include opensm/osm_log.h
+#include opensm/osm_opensm.h
+
+#define YYSTYPE char *
+#define OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH 128
+#define OSM_QOS_POLICY_DEFAULT_LEVEL_NAME default
+
+/***/
+
+typedef struct osm_qos_port_group_t_ {
+ char *name; /* single string (this port group name) */
+ char *use; /* single string (description) */
+ cl_list_t port_name_list; /* list of port names (.../.../...) */
+ uint64_t **guid_range_arr; /* array of guid ranges (pair of 64-bit
guids) */
+ unsigned guid_range_len;/* num of guid ranges in the array */
+ cl_list_t partition_list; /* list of partition names */
+ boolean_t node_type_ca;
+ boolean_t node_type_switch;
+ boolean_t node_type_router;
+ boolean_t node_type_self;
+} osm_qos_port_group_t;
+
+osm_qos_port_group_t *osm_qos_policy_port_group_create();
+void osm_qos_policy_port_group_destroy();
+
+/***/
+
+typedef struct osm_qos_vlarb_scope_t_ {
+ cl_list_t group_list; /* list of group names (strings) */
+ cl_list_t across_list; /* list of 'across' group names (strings) */
+ cl_list_t vlarb_high_list; /* list of num pairs (n:m,...), 32-bit
values */
+ cl_list_t vlarb_low_list; /* list of num pairs (n:m,...), 32-bit
values */
+ uint32_t vl_high_limit; /* single integer */
+ boolean_t vl_high_limit_set;
+} osm_qos_vlarb_scope_t;
+
+osm_qos_vlarb_scope_t *osm_qos_policy_vlarb_scope_create();
+void osm_qos_policy_vlarb_scope_destroy();
+
+/***/
+
+typedef struct osm_qos_sl2vl_scope_t_ {
+ cl_list_t group_list; /* list of strings (port group names) */
+ boolean_t from[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH];
+ boolean_t to[OSM_QOS_POLICY_MAX_PORTS_ON_SWITCH];
+ cl_list_t across_from_list; /* list of strings (port group names) */
+ cl_list_t across_to_list; /* list of strings (port group names) */
+ uint8_t sl2vl_table[16];/* array of sl2vl values */
+ boolean_t sl2vl_table_set;
+} osm_qos_sl2vl_scope_t;
+
+osm_qos_sl2vl_scope_t *osm_qos_policy_sl2vl_scope_create();
+void osm_qos_policy_sl2vl_scope_destroy();
+
+/***/
+
+typedef struct
[ofa-general] [PATCH 5/7 V2] osm: QoS - compiling policy file parser
Compiling QoS policy file parser.
C H files are generated from Lex Yacc files and the
generated files are compiled.
Generated files are not included in git, but they do get
included in RPM as they are also created by 'make dist'.
Signed-off-by: Yevgeny Kliteynik [EMAIL PROTECTED]
---
opensm/include/Makefile.am |2 ++
opensm/opensm/Makefile.am | 16 +++-
opensm/opensm/configure.in |2 ++
3 files changed, 19 insertions(+), 1 deletions(-)
diff --git a/opensm/include/Makefile.am b/opensm/include/Makefile.am
index ea62507..b83205f 100644
--- a/opensm/include/Makefile.am
+++ b/opensm/include/Makefile.am
@@ -90,6 +90,8 @@ EXTRA_DIST = \
$(srcdir)/opensm/osm_state_mgr_ctrl.h \
$(srcdir)/opensm/osm_perfmgr.h \
$(srcdir)/opensm/osm_perfmgr_db.h \
+ $(srcdir)/opensm/osm_qos_policy.h \
+ $(srcdir)/opensm/osm_qos_parser_y.h \
$(srcdir)/complib/cl_thread_osd.h \
$(srcdir)/complib/cl_packon.h \
$(srcdir)/complib/cl_atomic_osd.h \
diff --git a/opensm/opensm/Makefile.am b/opensm/opensm/Makefile.am
index 6dfa824..31e09f7 100644
--- a/opensm/opensm/Makefile.am
+++ b/opensm/opensm/Makefile.am
@@ -56,7 +56,15 @@ opensm_SOURCES = main.c osm_console.c osm_db_files.c \
osm_ucast_lash.c osm_ucast_file.c osm_ucast_ftree.c \
osm_vl15intf.c osm_vl_arb_rcv.c \
st.c osm_perfmgr.c osm_perfmgr_db.c \
-osm_event_plugin.c osm_dump.c
+osm_event_plugin.c osm_dump.c \
+osm_qos_parser_y.c osm_qos_parser_l.c osm_qos_policy.c
+
+osm_qos_parser_y.c: $(srcdir)/osm_qos_parser.y
$(srcdir)/../include/opensm/osm_qos_policy.h
+ $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o
$(srcdir)/osm_qos_parser_y.c
--defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h
--name-prefix=__qos_parser_
+
+osm_qos_parser_l.c: $(srcdir)/osm_qos_parser.l
$(srcdir)/../include/opensm/osm_qos_policy.h
+ $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c
$(srcdir)/osm_qos_parser.l
+
if OSMV_OPENIB
opensm_CFLAGS = -Wall $(OSMV_CFLAGS) -fno-strict-aliasing
-DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP $(DBGFLAGS) -D_XOPEN_SOURCE=600
-D_BSD_SOURCE=1
opensm_CXXFLAGS = -Wall $(OSMV_CFLAGS) -DVENDOR_RMPP_SUPPORT -DDUAL_SIDED_RMPP
$(DBGFLAGS) -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1
@@ -104,3 +112,9 @@ install-exec-hook:
if test -L $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; then rm
$(DESTDIR)/$(libdir)/libopensm-$(VERSION).so; fi; \
lname=`\ls -l $(DESTDIR)/$(libdir)/libopensm.so | awk '{print $$NF}'`; \
ln -s $$lname $(DESTDIR)/$(libdir)/libopensm-$(VERSION).so
+
+# generate c and h files from the lex and yacc files
+dist-hook: $(srcdir)/osm_qos_parser.y $(srcdir)/osm_qos_parser.l
+ $(YACC) -y -d $(srcdir)/osm_qos_parser.y -o
$(srcdir)/osm_qos_parser_y.c
--defines=$(srcdir)/../include/opensm/osm_qos_parser_y.h
--name-prefix=__qos_parser_
+ $(LEX) --prefix=__qos_parser_ -o $(srcdir)/osm_qos_parser_l.c
$(srcdir)/osm_qos_parser.l
+
diff --git a/opensm/opensm/configure.in b/opensm/opensm/configure.in
index afb20b3..a49538d 100644
--- a/opensm/opensm/configure.in
+++ b/opensm/opensm/configure.in
@@ -22,6 +22,8 @@ AC_PROG_INSTALL
AC_PROG_LN_S
AC_PROG_MAKE_SET
AC_PROG_LIBTOOL
+AM_PROG_LEX
+AC_PROG_YACC
dnl Checks for libraries
--
1.5.1.4
___
general mailing list
general@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: Using 16K buffer size really isn't going to keep the pipe full enough for TSO. Why the comparison with TSO (or GSO for that matter)? Seems to me that is only valid/fair if you have a single flow. Batching is multi-flow focused (or i should say flow-unaware). cheers, jamal ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] RE: OFA website edits
Hey Jeff, Can I get ownership of /var/www/openfabrics.org/downloads/cxgb3? That way I can publish libcxgb3 releases, which I maintain. Right now ralfc owns this instead of swise. Thanks, Steve. Jeff Becker wrote: Hi. I created most of the requested directory/owner pairs in /var/www/openfabrics.org/downloads. I left out the various MPI directories, figuring the appropriate web pages will be linked from somewhere (possibly the downloads web page). I gave Stan Smith an account. Stan, please contact me to get the account info. I'm still working out how to do the dynamic web page stuff, but at least people can start populating their directories. Thanks. -jeff On 7/25/07, Arlin Davis [EMAIL PROTECTED] wrote: I would like to propose adding project directories under http://www.openfabrics.org/downloads/ where appropriate and give maintainers access. For example: Jeff, please add the following directories with maintainer access as follow (or grant access at a maintainer group level): http://www.openfabrics.org/downloads/verbs (rdreier) http://www.openfabrics.org/downloads/rdmacm (shefty) http://www.openfabrics.org/downloads/dapl (ardavis) http://www.openfabrics.org/downloads/sdp (eitan) http://www.openfabrics.org/downloads/utils (eitan) http://www.openfabrics.org/downloads/management (sashak) http://www.openfabrics.org/downloads/OFED (vlad) http://www.openfabrics.org/downloads/archives (vlad) http://www.openfabrics.org/downloads/WinOF (ssmith) (Stan Smith will need an account) http://www.openfabrics.org/downloads/hw/mthca (rdreir) http://www.openfabrics.org/downloads/hw/mlx4 (rdreir) http://www.openfabrics.org/downloads/hw/ehca (raisch) http://www.openfabrics.org/downloads/hw/ipath (ralphc) http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc) http://www.openfabrics.org/downloads/mpi/mvapich (pasha) http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland) http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres) Let us know when these directories are created and the maintainers, who want to expose their packages via the webpage, will create a README that details the contents of the directory along with WEB_README that provides a short description for the webpage. Will this format allow you to auto configure the download webpage sufficiently? The idea is to only add links/descriptions to those project sub-directories with WEB_README files present. Please advise if something on the list is wrong or we missed a project. Thanks, -arlin ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking
Sasha Khapyorsky wrote: Hi Ira, On 09:34 Mon 20 Aug , Ira Weiny wrote: Sasha, Should this be applied to 1.2? Yes, I think it should, although don't know when next 1.2.x release will be. Sasha Ira, Till (and if) we have another 1.2.x release you can apply the patch on your local OFED copy using the ofed_patch.sh script that is located under OFED docs. Tziporet ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] [Fwd: [GIT PULL] ofed-1.3 - libcxgb3 refresh]
resending to lists... Original Message Subject: [GIT PULL] ofed-1.3 - libcxgb3 refresh Date: Tue, 21 Aug 2007 09:17:23 -0500 From: Steve Wise [EMAIL PROTECTED] To: Vladimir Sokolovsky [EMAIL PROTECTED] CC: OpenFabricsEWG [EMAIL PROTECTED], OpenFabrics General general@lists.openfabrics.org Vlad, I have a new release of libcxgb3 that I want included in ofed-1.3. Changes since ofed-1.2 are 1 bug fix (bug 703), some cleanup on the spec file as well as adding a ChangeLog file (bug 707). This is release 1.0.1 of libcxgb3... Pull from git://git.openfabrics.org/~swise/libcxgb3 ofed_1_3 Thanks, Steve. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] Query on ibv_post_recv and IBV_SEND_INLINE
Hi. Bharath Ramesh wrote: I have a question on using IBV_SEND_INLINE. From what I understand if I am using IBV_SEND_INLINE I technically neednt register the buffer with ibv_reg_mr as I dont need the L_Key. But still in the receiver end I would have to have the buffer registered to receive the inline data that has been posted. If I have an application where I am using a lot of small messages, then memory registration would be expensive. Is there any other option for receiving inline messages with out actually registering the receive buffer. The flag IBV_SEND_INLINE describe to the low level driver how the data will be read from the memory buffer (the driver will read the data and give it to the HCA or the HCA will fetch the data from the memory using the gather list). The data on the wire for inline messages looks the same like data which was fetched by the HCA. The receive side don't have any feature which can work with memory buffer without any registration. thanks Dotan ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH - 11] ipoib - add LSO support
Can you provide a pointer to some Mellanox high level doc/faq explaining the possible benefits from the stateless offloading features of connectX? The benefits of the stateless offload features can best be described by numbers. I have been able to push throughput in datagram mode to 814 MB/s. Details of the offload features are in the ConnectX prm. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] does ibv_modify_qp() check error ?
Hi, if anyone can answer, I have a question about ibv_modify_qp() behavior. I use ibv_modify_qp() to move QP state from INIT to RTR and then RTS. when INIT--RTR, I need to provide peer's qp number and port lid. If qp number and port lid are not correct (for example, both QPs are connected to different switch, and there is no bridge link between the switches), my code is silently hanging. Does ibv_modify_qp() check such error, or is there other way to detect such error ? so my code can error out, not hanging ? Thanks. --CQ, HP MPI ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] [PATCH] opensm: fix outstanding mad counters tracking
On Tue, 21 Aug 2007 17:46:33 +0300 Tziporet Koren [EMAIL PROTECTED] wrote: Sasha Khapyorsky wrote: Hi Ira, snip Yes, I think it should, although don't know when next 1.2.x release will be. Ira, Till (and if) we have another 1.2.x release you can apply the patch on your local OFED copy using the ofed_patch.sh script that is located under OFED docs. Thanks, we have already applied using patch to our local repo here. I think we have seen this issue with our 1.2 here and I wondered if this patch would be applicable to 1.2. Sasha, I will let you know if there are issues. Thanks, Ira ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: does ibv_modify_qp() check error ?
Quoting Tang, Changqing [EMAIL PROTECTED]: Subject: does ibv_modify_qp() check error ? Hi, if anyone can answer, I have a question about ibv_modify_qp() behavior. I use ibv_modify_qp() to move QP state from INIT to RTR and then RTS. when INIT--RTR, I need to provide peer's qp number and port lid. If qp number and port lid are not correct (for example, both QPs are connected to different switch, and there is no bridge link between the switches), my code is silently hanging. Does ibv_modify_qp() check such error, or is there other way to detect such error ? so my code can error out, not hanging ? Thanks. Try posting a WR, e.g. a 0 size RDMA write. You'll get a CQE with error if the QPN is wrong. -- MST ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
From: jamal [EMAIL PROTECTED] Date: Tue, 21 Aug 2007 08:30:22 -0400 On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: Using 16K buffer size really isn't going to keep the pipe full enough for TSO. Why the comparison with TSO (or GSO for that matter)? Because TSO does batching already, so it's a very good tit for tat comparison of the new batching scheme vs. an existing one. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Do you think the QoS new code will make it for 2.6.24?
Thanks Tziporet ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
On Tue, 2007-21-08 at 11:51 -0700, David Miller wrote: Because TSO does batching already, so it's a very good tit for tat comparison of the new batching scheme vs. an existing one. Fair enough - I may have read too much into your email then;- For bulk type of apps (where TSO will make a difference) this a fair test. Hence i agree the 16KB buffer size is not sensible if the goal is to simulate such an app. However (and this is where i read too much into what you were saying) that the test by itself is insufficient comparison. You gotta look at the other side of the coin i.e. at apps where TSO wont buy much. Examples, a busy ssh or irc server and you could go as far as looking at the most pre-dominant app on the wild west, http (average page size from a few years back was in the range of 10-20K and can be simulated with good ole netperf/iperf). cheers, jamal ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] IPOIB subnet on IB fabric with routers
In order for an ipoib subnet to work on an IB fabric consisting of multiple IB subnets, the MGIDs used for multicast can't have link-local scope otherwise packets will get dropped by the first IB router. Currently, the scope is hard-coded to link-local in the ip*_ib_mc_map() routines. A reasonable first step is to modify said routines to use the scope from the link level broadcast address of the ipoib device in question, and I have a proposed patch which I will post right away. The second step is to make the scope in the link level broadcast address configurable, but I'm not sure what the best way to accomplish that would be. The ip command supports changing the link level broadcast address, but it uses an ioctl which limits the hardware address to 14 bytes (six bytes short) ... so it fails. Even if that approach would work it would be a little error prone, we only care about four bits out of a 20 byte value. My current favourite approach is to add a broadcast_scope attribute under /sys/class/net/ibX/ that could be configured before bringing up the link. Does anybody have a better solution? Rolf ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] [PATCH] use scope from ipoib device link-level broadcast address in MGIDs
An ipoib subnet on an IB fabric that spans multiple IB subnets can't use
link-local scope in multicast GIDs. This patch takes the scope from the
link level broadcast address of the ipoib device when mapping to an IB
multicast address.
Signed-off-by: Rolf Manderscheid [EMAIL PROTECTED]
---
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 9ffb998..2fd1b30 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -2560,7 +2560,7 @@ static void cma_set_mgid(struct rdma_id_private *id_priv,
/* IPv6 address is an SA assigned MGID. */
memcpy(mgid, sin6-sin6_addr, sizeof *mgid);
} else {
- ip_ib_mc_map(sin-sin_addr.s_addr, mc_map);
+ ip_ib_mc_map(sin-sin_addr.s_addr, dev_addr-broadcast, mc_map);
if (id_priv-id.ps == RDMA_PS_UDP)
mc_map[7] = 0x01; /* Use RDMA CM signature */
mc_map[8] = ib_addr_get_pkey(dev_addr) 8;
diff --git a/include/net/if_inet6.h b/include/net/if_inet6.h
index 3ec7d07..b636221 100644
--- a/include/net/if_inet6.h
+++ b/include/net/if_inet6.h
@@ -268,14 +268,17 @@ static inline void ipv6_arcnet_mc_map(const struct
in6_addr *addr, char *buf)
buf[0] = 0x00;
}
-static inline void ipv6_ib_mc_map(struct in6_addr *addr, char *buf)
+static inline void ipv6_ib_mc_map(const struct in6_addr *addr,
+ const unsigned char *broadcast, char *buf)
{
+ unsigned char scope = broadcast[5] 0xF;
+
buf[0] = 0;/* Reserved */
buf[1] = 0xff; /* Multicast QPN */
buf[2] = 0xff;
buf[3] = 0xff;
buf[4] = 0xff;
- buf[5] = 0x12; /* link local scope */
+ buf[5] = 0x10 | scope; /* scope from broadcast address */
buf[6] = 0x60; /* IPv6 signature */
buf[7] = 0x1b;
buf[8] = 0;/* P_Key */
diff --git a/include/net/ip.h b/include/net/ip.h
index abf2820..fbb4dd0 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -266,16 +266,18 @@ static inline void ip_eth_mc_map(__be32 naddr, char *buf)
* Leave P_Key as 0 to be filled in by driver.
*/
-static inline void ip_ib_mc_map(__be32 naddr, char *buf)
+static inline void ip_ib_mc_map(__be32 naddr, const unsigned char *broadcast,
char *buf)
{
__u32 addr;
+ unsigned char scope = broadcast[5] 0xF;
+
buf[0] = 0;/* Reserved */
buf[1] = 0xff; /* Multicast QPN */
buf[2] = 0xff;
buf[3] = 0xff;
addr= ntohl(naddr);
buf[4] = 0xff;
- buf[5] = 0x12; /* link local scope */
+ buf[5] = 0x10 | scope; /* scope from broadcast address */
buf[6] = 0x40; /* IPv4 signature */
buf[7] = 0x1b;
buf[8] = 0;/* P_Key */
diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c
index 9ab9d53..feb643c 100644
--- a/net/ipv4/arp.c
+++ b/net/ipv4/arp.c
@@ -214,7 +214,7 @@ int arp_mc_map(__be32 addr, u8 *haddr, struct net_device
*dev, int dir)
ip_tr_mc_map(addr, haddr);
return 0;
case ARPHRD_INFINIBAND:
- ip_ib_mc_map(addr, haddr);
+ ip_ib_mc_map(addr, dev-broadcast, haddr);
return 0;
default:
if (dir) {
diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
index 0358e60..8275625 100644
--- a/net/ipv6/ndisc.c
+++ b/net/ipv6/ndisc.c
@@ -308,7 +308,7 @@ int ndisc_mc_map(struct in6_addr *addr, char *buf, struct
net_device *dev, int d
ipv6_arcnet_mc_map(addr, buf);
return 0;
case ARPHRD_INFINIBAND:
- ipv6_ib_mc_map(addr, buf);
+ ipv6_ib_mc_map(addr, dev-broadcast, buf);
return 0;
default:
if (dir) {
___
general mailing list
general@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re:
Strategy INF files. inform how AGP USB Core PCI ISA PnP that we took crawled web.The rich Patches Updates NES SNES Game Boy Advance Sega Master Megadrive Engine Neo Geo grabber CDROM bgimg.gif___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
David Miller [EMAIL PROTECTED] wrote on 08/22/2007 12:21:43 AM: From: jamal [EMAIL PROTECTED] Date: Tue, 21 Aug 2007 08:30:22 -0400 On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: Using 16K buffer size really isn't going to keep the pipe full enough for TSO. Why the comparison with TSO (or GSO for that matter)? Because TSO does batching already, so it's a very good tit for tat comparison of the new batching scheme vs. an existing one. I am planning to do more testing on your suggestion over the weekend, but I had a comment. Are you saying that TSO and batching should be mutually exclusive so hardware that doesn't support TSO (like IB) only would benefit? But even if they can co-exist, aren't cases like sending multiple small skbs better handled with batching? Thanks, - KK ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] Re: [PATCH 0/9 Rev3] Implement batching skb API and support in IPoIB
From: Krishna Kumar2 [EMAIL PROTECTED] Date: Wed, 22 Aug 2007 09:41:52 +0530 David Miller [EMAIL PROTECTED] wrote on 08/22/2007 12:21:43 AM: From: jamal [EMAIL PROTECTED] Date: Tue, 21 Aug 2007 08:30:22 -0400 On Tue, 2007-21-08 at 00:18 -0700, David Miller wrote: Using 16K buffer size really isn't going to keep the pipe full enough for TSO. Why the comparison with TSO (or GSO for that matter)? Because TSO does batching already, so it's a very good tit for tat comparison of the new batching scheme vs. an existing one. I am planning to do more testing on your suggestion over the weekend, but I had a comment. Are you saying that TSO and batching should be mutually exclusive so hardware that doesn't support TSO (like IB) only would benefit? But even if they can co-exist, aren't cases like sending multiple small skbs better handled with batching? I'm not making any suggestions, so don't read that into anything I've said :-) I think the jury is still out, but seeing TSO perform even slightly worse with the batching changes in place would be very worrysome. This applies to both throughput and cpu utilization. ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[ofa-general] nightly osm_sim report 2007-08-22:normal completion
OSM Simulation Regression Summary [Generated mail - please do NOT reply] OpenSM rev = Mon_Aug_20_12:20:05_2007 [a6cfde5efb96f6b3858c26e6934554ded57f89cb] ibutils rev = Thu_Aug_9_15:39:12_2007 [d1681ad8afb35441ac1741392d9e2e94a9d7e22f] Total=560 Pass=559 Fail=1 Pass: 42 Stability IS1-16.topo 42 OsmTest IS1-16.topo 42 OsmStress IS1-16.topo 42 Multicast IS1-16.topo 42 LidMgr IS1-16.topo 41 Pkey IS1-16.topo 14 Stability IS3-loop.topo 14 Stability IS3-128.topo 14 Pkey IS3-128.topo 14 OsmTest IS3-loop.topo 14 OsmTest IS3-128.topo 14 OsmStress IS3-128.topo 14 Multicast IS3-loop.topo 14 Multicast IS3-128.topo 14 LidMgr IS3-128.topo 14 FatTree merge-roots-4-ary-2-tree.topo 14 FatTree merge-root-4-ary-3-tree.topo 14 FatTree gnu-stallion-64.topo 14 FatTree blend-4-ary-2-tree.topo 14 FatTree RhinoDDR.topo 14 FatTree FullGnu.topo 14 FatTree 4-ary-2-tree.topo 14 FatTree 2-ary-4-tree.topo 14 FatTree 12-node-spaced.topo 14 FTreeFail 4-ary-2-tree-missing-sw-link.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-2.topo 14 FTreeFail 4-ary-2-tree-links-at-same-rank-1.topo 14 FTreeFail 4-ary-2-tree-diff-num-pgroups.topo Failures: 1 Pkey IS1-16.topo ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
