Re: [zones-discuss] ip_restrict_interzone_loopback again
Hi, Do you have more details on your zone configuration? If you are using exclusive stack zones then this is expected. Rgds, Jon Christine Tran wrote: Hi, Has anyone *actually* observe that you can communicate between zones with the cable removed when /dev/ip ip_restrict_interzone_loopback is set to 0? Here's my setup, s10u5. global: 192.168.1.60/24 e1000g0, cabled zone1: 192.168.1.61/24 e1000g1, cabled zone2: 192.168.1.62/24 e1000g2, not cabled by default ip_restrict_interzone_loopback is 0, no need to do anything there. zone2 is pretty much by himself, unable to communicate to global or zone1. If I understand correctly, he should be able to communicate with the global zone and zone1, even without the cable. This is not the case. CT ___ zones-discuss mailing list zones-discuss@opensolaris.org -- Jon Anderson Solaris Revenue Product Engineering Sun Microsystems Inc. SPARC House (UK) Tel: ++44 (0)1252 421 868 Mob: ++44 (0)7747 180 910 ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
On Fri, Jan 23, 2009 at 4:27 AM, Jon Anderson jon.ander...@sun.com wrote: Hi, Do you have more details on your zone configuration? If you are using exclusive stack zones then this is expected. Hmm, I thought the exact opposite. zones of type exclusive-ip type, plumbed on different interfaces, will drive their traffic out one IF and into the other. I believe Steffen Weiberle did some test to measure the delay as opposed to using the internal loopback mechanism. Unless ip_restrict_interzone_loopback is 0 (the default is 1 on OS). You can have zones of type exclusive-ip plumbed on different interfaces but not cabled up if this parameter is set to 0. This is what I gleaned from reading what little there is about this param. Otherwise ... how could you ever have traffic going from zone1:e1000g1 to zone2:e1000g2 without a cable? Anyhow, this may be a JASS problem because JASS will enable ipfilter on the global zone but JASS's mod to ipf.conf does not pass lo0 traffic. And yet, ... my understanding is that the internal loopback mechanism does not really involve lo0. By the time we got it sorted out last night we were all out of gas, so we didn't get to the bottom of it. If you can describe exactly how I can get traffic from zone1:e1000g1 to zone2:e1000g2 without cabling up the interfaces, that would solve my problem. Thanks. CT ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
I may be wrong on this as I haven't looked for quite a long while now (and things change rapidly) but Exclusive stack zones means you get a separate IP stack and, therefore, a separate routing table. This means that we don't know anything about interfaces which are 'local' to other zones on the node. In a shared-stack zone, we know whether the destination is local or not (as we can look up the IRE_LOCAL for the destination as it's in the same stack). ip_restrict_interzone_loopback only affects shared stack zones for this reason. Even then, when it is 'on', it is only enforced if the source and destination ills are different(to detect an L2 loopback). I'm pretty sure that this was how things were when this integrated. Zones with an exclusive stack are more like virtual machines. Unless ip_restrict_interzone_loopback is 0 (the default is 1 on OS). You can have zones of type exclusive-ip plumbed on different interfaces but not cabled up if this parameter is set to 0. Where is this documented? This is what I gleaned from reading what little there is about this param. Otherwise ... how could you ever have traffic going from zone1:e1000g1 to zone2:e1000g2 without a cable? With exclusive stack zones, I don't _think_ you can. Perhaps someone will correct me if this is possible. If you can describe exactly how I can get traffic from zone1:e1000g1 to zone2:e1000g2 without cabling up the interfaces, that would solve my problem. You definitely can with shared stack zones. Thanks. CT ___ zones-discuss mailing list zones-discuss@opensolaris.org -- Jon Anderson Solaris Revenue Product Engineering Sun Microsystems Inc. SPARC House (UK) Tel: ++44 (0)1252 421 868 Mob: ++44 (0)7747 180 910 ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
Unless ip_restrict_interzone_loopback is 0 (the default is 1 on OS). You can have zones of type exclusive-ip plumbed on different interfaces but not cabled up if this parameter is set to 0. Where is this documented? This is what started the whole kerfuffle for me, https://www.opensolaris.org/jive/thread.jspa?threadID=84543tstart=-1 particularly which have their local IP addresses on different interfaces, I zeroed in on this and conveniently ignored the shared-stack zone which I'm noticing just now. But how could that be ... shared-stack zone with IP address on different interface? This thing cannot exist? Here it is. I need exclusive stack so I can snoop traffic when bad things happen. When bad things are not happening, traffic must not be snoopable, sayeth the people in charge. I have this brilliant idea (based on what I read) that I can conviently shunt traffic to the NIC or internally, at will, using this nifty param. Bad things happen, shunt it outside to observe. Bad things go away, shunt it back inside, remove the cable. This cannot work, you say? This is S10U5, OS is not an option. CT ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
But how could that be ... shared-stack zone with IP address on different interface? This thing cannot exist? You can add multiple physicals to a shared stack zone, they are just added as logicals. You need the underlying interface plumbed in the global zone though. An exclusive stack doesn't know anything about other zones' network configuration. Here it is. I need exclusive stack so I can snoop traffic when bad things happen. When bad things are not happening, traffic must not be snoopable, sayeth the people in charge. I have this brilliant idea (based on what I read) that I can conviently shunt traffic to the NIC or internally, at will, using this nifty param. Bad things happen, shunt it outside to observe. Bad things go away, shunt it back inside, remove the cable. This cannot work, you say? This is S10U5, OS is not an option. You can have shared stack zones with ip_restrict_interzone_loopback disabled. When you have 'problems' you can enable this to shunt traffic outside the box. This means it will need to be cabled up to a switch though at these times - you seem to indicate this won't be an issue. One issue would be if the ill for source and destination was the same then we would still send via loopback. You should be able to avoid this if the zone IP addresses are configured on different physical interfaces. CT -- Jon Anderson Solaris Revenue Product Engineering Sun Microsystems Inc. SPARC House (UK) Tel: ++44 (0)1252 421 868 Mob: ++44 (0)7747 180 910 ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Zones and network
You can't disable loopback traffic routing on a shared stack zone if the source and destination are on the same interface. This would be a data link loopback which most L2 devices wouldn't handle properly (i.e. you are trying to send from e1000g0 to a an address on e1000g0) -- This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
You can add multiple physicals to a shared stack zone, they are just added as logicals. You need the underlying interface plumbed in the global zone though. An exclusive stack doesn't know anything about other zones' network configuration. OK, I'm beginning to see. Like this, you mean? global zone: plumb e1000g1 0.0.0.0 plumb e1000g2 0.0.0.0 zone1: e1000g1:1 192.168.1.61 zone2: e1000g2:1 192.168.1.62 ip_restrict_interzone_loopback = 0 traffic from zone1 - zone2 shunted internally ip_restrict_interzone_loopback = 1, cabled to a switch traffic from zone1 - zone2 forced out the NIC, and observable with snoop One issue would be if the ill for source and destination was the same then we would still send via loopback. You mean if zone1 and zone2 were plumbed on e1000g1:1, and e1000g1:2, traffic will never be observable no matter what. I can live with this. Did I get this right? CT ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] ip_restrict_interzone_loopback again
You mean if zone1 and zone2 were plumbed on e1000g1:1, and e1000g1:2, traffic will never be observable no matter what. I can live with this. All IP packets can be observed as of Nevada build 103; see http://blogs.sun.com/seb/ for some examples with zones. -- meem ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] Zone Networking and Backups
Zones Group, A colleague and I have been trying to find a definitive answer for a Zones networking question as we're trying to solidify how we backup large Oracle databases running inside of Zones. Basically the scenario is this: If I Initiate a client backup from the local zone to the storage node software running in the global zone, and they share their ethernet interface - no shared IP instance - does the backup traffic hit the physical wire? Or, does it stay inside the kernel/memory... thereby achieving a very fast in memory transfer to the backup server running in the global zone? Thanks so much, I really appreciate the help! -Matthew -- This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Zone Networking and Backups
For ip-type=shared the communication stays in the system. O(memory speed). Very fast with little IP processing. This is the default. For ip-type=exclusive the communication will have to go out the interface assigned to the nonglobal zone and routed externally to an interface on the global zone. It is visible and the transfer is O(slowest interface). Bob Sent from my iPhone On Jan 23, 2009, at 1:41 PM, Matt Walburn m...@railwave.com wrote: Zones Group, A colleague and I have been trying to find a definitive answer for a Zones networking question as we're trying to solidify how we backup large Oracle databases running inside of Zones. Basically the scenario is this: If I Initiate a client backup from the local zone to the storage node software running in the global zone, and they share their ethernet interface - no shared IP instance - does the backup traffic hit the physical wire? Or, does it stay inside the kernel/memory... thereby achieving a very fast in memory transfer to the backup server running in the global zone? Thanks so much, I really appreciate the help! -Matthew -- This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Zone Networking and Backups
On 01/23/09 14:41, Matt Walburn wrote: Zones Group, A colleague and I have been trying to find a definitive answer for a Zones networking question as we're trying to solidify how we backup large Oracle databases running inside of Zones. Basically the scenario is this: If I Initiate a client backup from the local zone to the storage node software running in the global zone, and they share their ethernet interface - no shared IP instance - does the backup traffic hit the physical wire? Or, does it stay inside the kernel/memory... thereby achieving a very fast in memory transfer to the backup server running in the global zone? Thanks so much, I really appreciate the help! -Matthew The traffic must not go out the ethernet interface if using the single interface you mentioned. Otherwise it would be an spec violation as ethernet frames going to the switch can't be sent directly back to the system. You mention not using shared IP, yet only one interface. I mention that since if you were to configure VNICs on a single physical interface, the same rules apply, even if the zones are configured with exclusive IP Instances. The VNIC 'switch' will not have frames leave the system. Actually, the more I read the question, the more confused I am. Are the zones on the same IP subnet? Are you using one NIC *and* using exclusive IP Instances (ip-type=exclusive)? If not using VNICs (SX-CE 105) are you using VLANs? Steffen PS. for some scenarios with VNICs or VLANs, see blogs.sun.com/stw/ ___ zones-discuss mailing list zones-discuss@opensolaris.org
