On Tue, Oct 24, 2006 at 01:41:32PM -0700, Erik Nordmark wrote:
> Edward Pilatowicz wrote:
> >- how about the opposite scenario. a user configures a zone with
> > without and an exclusive ip instance and sets up a bunch of network
> > configuration inside that zone that would normally only apply to a
> > zone with an exclusive ip instance. then the user boots the zone.
> > does this result in any errors during boot?
>
> The same thing would happen as if you today did
> echo 10.1.2.3 >/etc/hostname.bge0
> echo 10.1.1.1 >/etc/defaultrouter
> reboot
> in a zone. This is silently ignored.
> (Or, as I've seen some scripts do, put IP configuration into
> $ZROOT/etc/sysidcfg; also silently ignored today.)
>
> The way the smf methods are structured in S10 they skip the IP level
> configuration. The only change introduced by IP instances is the
> predicate used to test whether to skip.
> For example, net-physical has these udiffs:
> -smf_is_globalzone || exit $SMF_EXIT_OK
> +smf_configure_ip || exit $SMF_EXIT_OK
>
perhaps some kind of warning message should be generated in this
scenario instead?
something like:
Ignoring zone network configuration specified: /etc/hostname.bge0
Current network configuration is dictated by the global zone.
To use the network configuration specified within this zone it
must have an exclusive-IP instance allocated to it by the global
zone.
> >- you mention that no GLD or layer 2 changes will be supported inside
> > zones. your example then mentions aggregations. i'm rusty on
> > remembering all the layers so i'll just ask, are vlans and ip
> > tunnels supported
>
> Aggregations can be configured using dladm in the global zone, and then
> the aggr<N> datalink names can be given to exclusive-IP zones.
>
> VLANs will be supported for GLDv3 devices. (We can't get e.g. the ce
> driver to have a separate /dev/ce<N> entry per instance and per VLAN.)
>
> IP tunnels work today, but they are not devices but a funny construct
> using a tunneling streams module.
> The clearview tunnel project will turn tunnels into GLDv3 links, and
> they are aware of the implications of IP instances. (I use separate IP
> Instances on the laptop punched into separate punchin points just for fun.)
>
good to know.
> >- if an a zone is configured with an exclusive stack, will the
> > /dev (and eventually /dev/net) devices associated with that stack
> > also be automatically imported into that zone? if so, then how does
> > this interact with non-native zones? (linux won't be expecting
> > /dev/bge0 in it's namespace. linux uses network devices like
> > /dev/net0.)
>
> Yes.
>
> The BrandZ merge was done recently, hence is not covered in the design
> document.
>
> One part is getting things like /dev/ip show up in the exclusive-IP
> zones, and not in the shared-IP zones.
> This is done by extensions to platform.xml in the form of
> + <!-- Devices to create in exclusive IP zone only -->
> + <device match="ip" iptype="exclusive" />
>
> The other part is e.g. /dev/bge33000, which is done with
> di_prof_add_dev() as part of the vplat bringup.
>
> Should we just limit that to certain brands for the time being? Or we
> can disable the ability to setup exclusive-IP with certain brands.
> Suggestions?
>
nit: in zonecfg it's "ip-type" were as above there's no dash.
i looked into it a bit more and as it turns out in linux network devices
don't actually have any /dev entries. but we can't simply tell non-native
brands not to map exclusive-IP network devices because this could break
other zones. (think sn1, belinix, nexenta, etc.)
so here's a question. in an exclusive-IP zone, do we have to have
network /dev entries in to be able to configure network interfaces?
or can all the necessary configuration be done via socket operations?
i guess if we need to have access to the device nodes then the easiest
thing to do will be to simply map them in and hopefully linux apps will
ignore them. if linux apps don't ignore them then we'll have to
create /dev and /native/dev as seperate namespaces. (currently
they are the same.)
if we don't need/want the nodes then we'll have to add some kind of
brand callback such that the lx brand can indicate this when you
attempt to add in the interfaces.
lastly, we'll probably have to add some kind or mechanism that will
allow a brand to iterate over all the network devices which have been
exclusively allocated to it. (so when the linux brand does
"ifconfig eth0 plumb" we can look at all the available interfaces and
plumb one up.)
> >- wrt to brandz, supporting an exclusive ip instance in an an lx
> > branded zone will require the implementation of network configuration
> > interfaces within the lx brand. it will probably involve translating
> > a bunch of ioctls and socket operations. also, looking at a centos
> > machine i see that it uses iptables instead of ipfilters. so all
> > the iptables configuration system calls would need to be translated
> > into their corresponding ipfilters commands.
>
> I've asked Darren Reed whether it is even feasible to do iptables to IP
> Filter mappings.
>
> > will this project do this work? if not it seems that users should be
> > unable to create such a configuration in zonecfg.
>
> No.
>
> We can certainly disable IP instances with non-native brands in zonecfg.
> But I assume we'd want to support sn-1, thus "non-native" might not be
> the right test.
>
then this seems like something that should be controlled via
the brand configuration verification callback. currently the lx
brand already looks for unsupported zonecfg options and reports
errors if there are any. it seems like it should be enhanced to
detect an exclusive-IP config and not allow it. (see the verify
subcommand to lx_support.c.)
> >- have you considered providing a way to configure an ip instance
> > assigned to a zone from the global zone?
>
> Yes. Whether it is sensible depends what the definition is of
> "configure". If all we are talking about is static configuration i.e. an
> ifconfig and route command done at boot time, then I'm sure the one can
> put together a solution in many different ways.
>
> However, I doubt the desire stops at static configuration.
> If there is a need to do dynamic configuration, whether a DHCP client or
> a routing daemon, then I think such an approach becomes intractable.
> In such a case the non-global zone has to have some observability and
> interaction with the daemon. For instance being able to start it and
> stop it, but also be able to extract non-IP configuration from the DHCP
> client (DNS configuration, timezone, etc.)
>
> I doubt "static only" would be a stable state. For the dynamic approach
> I suspect it is less work, short-term as even more so as brandZ evolves,
> in doing the ioctl translation and running the regular Linux daemons.
>
hm. i spend most my time thinking about the NAT case so i hadn't
considered dhcp. i guess that will require running the dhcp daemon
inside the linux zone which will require supporting whatever
daemon/kernel interfaces they use. (probably socket operations.)
ugh.
> > this would help with the situation above where the lx branded zone
> > can't configure itself.
> >
> > another reason that i ask this is because currently you plan to make
> > SMF startup scripts detect if a zone has an exclusive configuration
> > or not.
>
> Yes, all encapsulated in the form of smf_configure_ip.
>
> > this becomes more complicated when it comes to modifying
> > linux startup scripts. currently we disable most of them. if we
> > take the same approach for non-native zones then we'll have to stop
> > disabling the script and patch them to be stacked instances aware
> > as well?
>
> Do you disable them, or do they check "has IP already been configured"
> and just skip doing the work themselves?
>
currently we just disable them by moving them aside.
i guess we could patch them up by adding a line to the top like:
/native/usr/lib/brand/lx/lx_configure_ip || exit 0
where lx_configure_ip is a script that runs a native binary and
tells us if we're configured with an exclusive-IP stack or not.
> >- you mention that this will help us get NAT support for zones, then
> > you also mention vSwitch. so i'm a little confused. after ip
> > instances goes back will we be able to use the global zone as a NAT
> > for local zones or not? (rfe 6338667) if so, could you provide an
> > example configuration to illustrate how this might be setup using
> > ip instances? if not, could you explain a little more what additional
> > functionality is still required?
>
> What this project provides by itself is IP isolation when different
> zones are connected to different (V)LANs.
>
> To easily do NAT in the global zone there is a need for an internal-only
> Ethernet software construct.
> The crossbow project is building that as a combination of what is
> currently called a vSwitch and a VNIC.
>
> The VNIC support is already there in the crossbow bits folks can
> download from opensolaris.org.
> The vSwitch, which isn't much code at all, remains to be done.
>
> >- will dladm show-* subcommands work in a zone? will they display the
> > interfaces associated with exclusive ip instances assigned to that
> > zone?
>
> dladm is restricted to PRIV_SYS_NET_CONFIG, hence only works in the
> global zone.
>
so it sounds like this is a clearview issue where they will probalby be
changing this behavior.
> >- clearview will allow interfaces to be renamed via dladm, so once an
> > interface has been assigned exclusively to a zone will zone
> > administrators be able to rename it via dladm?
>
> Yes. But what problem are you trying to solve?
>
> The clearview team realizes that renaming as a way to replace an
> Ethernet is a fine thing, but other forms of renaming require changing
> other configuration that use the name.
> Thus if you use dladm vanity naming to name bge0 to be net3, then later
> you can rename e1000g2 to be net3 without any effect on the rest of the
> system. Other forms of rename will require changes elsewhere (ipf.conf,
> zonecfg).
>
with the question above i'm not trying to solve any problem.
i'm trying to anticipate problems that other peoples solutions
might generate. ;)
ed
_______________________________________________
zones-discuss mailing list
zones-discuss@opensolaris.org
