Sean: in such situations, where there is mystery traffic, a packet capture is in order. Wireshark is friendlier but snoop comes with Solaris. You can limit the capture to 30 packets. We will want to know if its an ARP storm or maybe its bridge packets or is something doing retransmits in IP....the possibilities are quite numerous but a packet capture will tell us which it is.
----- Original Message ----- From: [email protected] <[email protected]> To: Octave Orgeron <[email protected]>; [email protected] <[email protected]> Sent: Tue Apr 05 08:06:10 2011 Subject: Re: [ldoms-discuss] link based IPMP and LDOMS The problem doesn't seem to be on the guest because the throughput on the vnet remains minimal. The traffic on the IO domain however jumps. The traffic is the same regardless (subnet traffic, broadcast etc) of whether there is a guest assigned or not, but the through put just seems to jump as soon as a guest has the aggregate assigned to it. I performed a dladm show-aggr and the traffic appears to only utilise 2 of the 4 interfaces for input and the remaining 2 for output (this might be correct): key:1 ipackets rbytes opackets obytes %ipkts %opkts Total 4441 1545061 15 3250 nxge0 2477 1173436 0 0 55.8 0.0 nxge1 1961 371311 0 0 44.2 0.0 nxge2 1 124 15 3250 0.0 100.0 nxge3 2 190 0 0 0.0 0.0 When I remove the client, the through put drops.... key:1 ipackets rbytes opackets obytes %ipkts %opkts Total 23 1680 0 0 nxge0 17 1084 0 0 73.9 - nxge1 2 216 0 0 8.7 - nxge2 1 124 0 0 4.3 - nxge3 3 256 0 0 13.0 - Cheers Sean -----Original Message----- From: Octave Orgeron [mailto:[email protected]] Sent: 31 March 2011 16:24 To: Burke Sean; [email protected] Subject: Re: [ldoms-discuss] link based IPMP and LDOMS What kind of traffic do you see from the guest through snoop? *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* Octave J. Orgeron Solaris Virtualization Architect and Consultant Web: http://unixconsole.blogspot.com E-Mail: [email protected] *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* ----- Original Message ---- From: Burke Sean <[email protected]> To: "[email protected]" <[email protected]> Sent: Thu, March 31, 2011 9:12:08 AM Subject: Re: [ldoms-discuss] link based IPMP and LDOMS I have a weird situation when I go the dladm route in that I constantly have network traffic going over the aggregate as soon as I assign the vnet to a guest as a secondary path... The setup is a split bus T5240 with the guests primary vnet coming from the primary IO's 10Gb card. See below for nicstat capture of aggregate on the secondary IO domain Time Int rKB/s wKB/s rPk/s wPk/s rAvs wAvs %Util Sat 16:04:59 aggr1 1.02 0.40 12.18 2.79 85.66 146.3 0.00 0.00 ldm add-vnet linkprop=phys-state guest-vnet1 secondary-vsw0 guest 16:05:04 aggr1 204.7 0.49 878.8 4.18 238.6 119.2 0.17 0.00 16:05:09 aggr1 1304.5 0.28 3579.4 2.00 373.2 145.4 1.07 0.00 16:05:14 aggr1 3601.7 0.29 4218.0 2.00 874.4 147.0 2.95 0.00 16:05:19 aggr1 2938.8 0.30 3365.1 1.99 894.3 154.2 2.41 0.00 ldm remove-vnet guest-vnet1 guest 16:05:24 aggr1 1.29 0.40 16.79 3.60 78.63 114.6 0.00 0.00 16:05:29 aggr1 1.00 0.35 12.17 2.39 84.23 150.3 0.00 0.00 bash-3.00# dladm show-aggr -L key: 1 (0x0001) policy: L2 address: 0:21:28:b0:8c:8e (auto) LACP mode: active LACP timer: short device activity timeout aggregatable sync coll dist defaulted expired nxge0 active short yes yes yes yes no no nxge1 active short yes yes yes yes no no nxge2 active short yes yes yes yes no no nxge3 active short yes yes yes yes no no Any thoughts? Cheers Sean -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Hudes, Dana Sent: 31 March 2011 16:01 To: '[email protected]'; '[email protected]' Subject: Re: [ldoms-discuss] link based IPMP and LDOMS Instead of ipmp do dladm link aggregation and all your traffic uses both links. In case of failure you have degraded performance for all rather than failure for some. With solaris 11 and crossbow we will have traffic shaping to throttle bandwidth per sub interface but I haven't got that running yet ----- Original Message ----- From: [email protected] <[email protected]> To: [email protected] <[email protected]> Sent: Thu Mar 31 08:09:24 2011 Subject: Re: [ldoms-discuss] link based IPMP and LDOMS > I see no benefit from doing IPMP in each LDOM instead > of doing IPMP at the platform level and giving the > resulting virtual interface to the vnet to the LDOM. OK picture this. you have a domain controller with two interfaces e1000g0 and nxge0 both are on the same vlan 192.168.1.x joined together with IPMP. active interface is e1000g0. we have two different LDOMS that are going to be using this network LDOMa and LDOMb. each LDOM has 4 zones under it. This gives us five vnets on each LDOM ten in total. Since IPMP is active/passive all of your traffic from ten different vnets is going down your e1000g0 pipe. now lets say we do our IPMP down inside the LDOMS/zones this would require that I create 5 vnets on LDOMa under e1000g0 and 5 vnets on LDOMa under nxge0. 5 vnets on LDOMb under e1000g0 and 5 vnets on LDOMb under nxge0. Now down inside the LDOMS/zones I alternate between which vnet is the primary. This allows me to double the network bandwidth down inside the LDOMS/zones than doing all of the IPMP at the domain controller level. Next: My environment has 4 to 6 LDOMS on a given domain controller all running 4 zones inside of them. Each of the LDOMS is tied to a vlan that I do not want or need on the domain controller. I do not want a interface plumbed up on the domain controller just to run IPMP on it instead of the LDOMS. -- This message posted from opensolaris.org _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss Stryker SA / Registered number: CH-550-0054924-4 / Registered office: Cité-Centre, Grand rue 90, 1820 Montreux / VAT number: . 291987 Les informations contenues dans ce courriel ainsi que les éventuelles pièces jointes sont exclusivement destinées à l’usage du ou des destinataires prévus. Ce courriel peut contenir des informations privilégiées ou confidentielles aux yeux de la loi ou le produit d’un travail soumis au secret professionnel de l’avocat ou autrement protégé contre la divulgation. Aucune renonciation à un quelconque privilège n’est envisagée. Si vous n’êtes pas le destinataire prévu, toute interception, communication, diffusion ou copie est strictement interdite et pourra donner lieu à des poursuites pénales ou civiles à votre encontre. Si vous avez reçu ce courriel par erreur, veuillez en avertir l’expéditeur par téléphone ou par retour de courriel et confirmer que vous détruirez ou effacerez de votre système toutes les exemplaires de ce message et des éventuelles pièces jointes. The information contained in this email message and any attachments is exclusively for the use of the intended recipient(s). It may contain legally privileged, confidential information or work product subject to attorney-client privilege or otherwise protected from disclosure. No waiver of any privilege is intended. If you are not the intended recipient, any interception, dissemination, distribution or copying is strictly prohibited, and may subject you to criminal and/or civil penalties. If you have received this email message in error, please notify the sender by telephone or by email reply and confirm that you will destroy or delete all copies of the message and any attachments from your system. _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss Stryker SA / Registered number: CH-550-0054924-4 / Registered office: Cité-Centre, Grand rue 90, 1820 Montreux / VAT number: . 291987 Les informations contenues dans ce courriel ainsi que les éventuelles pièces jointes sont exclusivement destinées à l’usage du ou des destinataires prévus. Ce courriel peut contenir des informations privilégiées ou confidentielles aux yeux de la loi ou le produit d’un travail soumis au secret professionnel de l’avocat ou autrement protégé contre la divulgation. Aucune renonciation à un quelconque privilège n’est envisagée. Si vous n’êtes pas le destinataire prévu, toute interception, communication, diffusion ou copie est strictement interdite et pourra donner lieu à des poursuites pénales ou civiles à votre encontre. Si vous avez reçu ce courriel par erreur, veuillez en avertir l’expéditeur par téléphone ou par retour de courriel et confirmer que vous détruirez ou effacerez de votre système toutes les exemplaires de ce message et des éventuelles pièces jointes. The information contained in this email message and any attachments is exclusively for the use of the intended recipient(s). It may contain legally privileged, confidential information or work product subject to attorney-client privilege or otherwise protected from disclosure. No waiver of any privilege is intended. If you are not the intended recipient, any interception, dissemination, distribution or copying is strictly prohibited, and may subject you to criminal and/or civil penalties. If you have received this email message in error, please notify the sender by telephone or by email reply and confirm that you will destroy or delete all copies of the message and any attachments from your system. _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
