Re: [zones-discuss] ipfilter (ipf.conf) entries in zonecfg?
On Fri, Nov 7, 2008 at 12:13 PM, Tommy McNeely [EMAIL PROTECTED] wrote: Hello Zones experts, We are attempting to create a new data center architecture that favors virtualization with zones. Previously, if we wanted to have zones from different security contexts (front-end, back-end, internet, etc), they had to be in different physical machines (or LDOMS). Now that we have the ability (ok, as of s10u4, but we have been busy) to use ipfilter between zones on the same host, we believe there may be enough separation to have zones in different security contexts on the same global-zone. I would like to get people's feedback on what they would think of creating the ability to have ipfilter rules, that would normally be located in ipf.conf in the global zone, inside the zonecfg. When the zone is brought online it could pipe the rules into ipf -f - or something. I am thinking the zonecfg seems like a good place to store them because when I want to move a zone from one machine to another, I would prefer the firewall came along with the zone. We have discussed using vnic interfaces (crossbow?), but I don't believe thats integrated yet? Besides, we don't really trust the application administrator (zone administrator) with the firewall, so we'd like to keep its configuration in the global zone, which I assume would still work even with vnic's. QUESTION: If we put the firewall (ipf.conf) inside the zone and use a private IP instance, can they can put a pass out quick on vnic0 keep state and they have the ability to connect to any other zone on the same machine? I know that rule in the global zone makes it that way, but maybe ip stack instances fix that? Crossbow is not a feature in S10. You mentioned the use of S10U4 above. In that context, the simple answer to your question is no, because VNICs don't exist. --JeffV ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] using load balancers with zones
Continuing on my zones in different subnets theme, but starting a new thread to improve readability... The other problem I have been experiencing can be solved with the right ipf.conf incantation (I think). Lets pretend the host (zone) mail-fe-1 initiates a connection to dirsrv-lb (load balancer) on port 389/tcp. The Load balancer decides to send that connection (dnat) to host (zone) dirsrv-1, which happens to be on the same physical server (global zone) as the mail-fe-1 zone... (all networks below are /24) So, according to mail-fe-1: LOCAL: mail-fe-1 (172.16.1.51) REMOTE: dirsrv-lb (172.16.1.30) ... and according to dirsrv-1: LOCAL: dirsrv-1 (172.16.3.31) REMOTE: mail-fe-1 (172.16.1.51) If these are all separate physical machines (or at least one physical machine per subnet), there would be no problems, because in order to get back to mail-fe-1 (172.16.1.51), the host dirsrv-1 (172.16.3.31) would use its default router (the load balancer), which would fix (snat) the source address to be dirsrv-lb (172.16.1.30), and everyone would be happy. Since they are on the same physical machine, even though they are on separate physical interfaces, they find each other as the packet is flowing out e1000g3, and comes back in e1000g1, but never really gets back because the Local/Remote addresses don't match, so mail-fe-1 never sees the S/A. The host based firewall (ipf) gets scared and confused and blocks the packet, but I don't think that it would even matter if IPF wasn't being used, because I had this problem in the past without ipf or any firewall with everything on the same subnet. Same issue, the packet was returned directly instead of via the wire where it would have (presumably) flowed through the load balancer and gotten fixed. I can obviously workaround the issue by using source-nat in the load balancer, causing dirsrv-1 to think that the connection came FROM the load balancer (or one of its nat-pool IP addresses) to force the packet onto the wire, but that makes logs useless on dirsrv-1 and troubleshooting a who made what changes issue a nightmare. I am sure there must be a way to tell ipf to force packets from the be- net to the fe-net to go out on the wire, presumably using dup-to but I was unable to make it work, so I am using source-nat at the moment. I look forward to hearing that someone else has already figured this out and here is how they did it, but for now I need to work on installing software on mail-fe-1 because it was supposed to be working last Tuesday ;) Thanks in advance, Tommy ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Can't create an exclusive IP zone in Open Solaris snv_99
You probably can't use bge0 because its being used in the global zone. Try plugging a cable into bge1 (net1) and use that. Exclusive means exclusive, it needs its own network interface. -- This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Zone Statistics: monitoring resource use of zones
On Sun, Nov 9, 2008 at 7:54 PM, Jeff Victor [EMAIL PROTECTED] wrote:
It has become clear that there is a need to monitor resource
consumption of workloads in zones, and an easy method to compare
consumption to resource controls. In order to understand how a
software tool could fulfill this need, I created an OpenSolaris
project and a prototype to get started. If this sounds interesting,
you can find the project and Perl script at:
http://opensolaris.org/os/project/zonestat/ .
If you have any comments, or suggestions for improvement, please let
me know on this e-mail list or via private e-mail.
I've had such needs for a while and have developed some tools to help
my organization with that. Unfortunately, I'm not able to share that
code. I am able to share suggestions...
I am in a habit of:
#! /usr/bin/perl -w
use strict;
That catches a lot of mistakes that may be masked by:
close STDERR;
which I never do. :)
Please do not use /etc/release as a test of kernel functionality.
Those that patch to an equivalent level as the update release have a
similar level of functionality. A better mechanism would be to check
for specific kernel patches. See
http://blogs.sun.com/patch/entry/solaris_10_kernel_patchid_progression
for kernel patch ID's through Update 6. The blog entry was posted
just before U5 shipped, so the U6 info should be checked for accuracy.
# Get amount and cap of memory locked by processes in each zone.
# kstat -p caps:*:lockedmem_zone_* conveniently summarizes all zones for us.
#
open(KSTAT, /usr/bin/kstat -p caps:*:lockedmem_zone_* |);
while (KSTAT) {
You could just use Sun::Solaris::Kstat rather than forking another perl script.
My feeling on capped memory is that if it becomes an issue and capped
swap is not really close to capped memory, the over-consumptive zone
has too high of a chance of causing horrible I/O problems for all
zones. That is, the cap is likely to do more harm than good. This
may change if swap can go onto solid state disk. I only mention this,
because I don't see a purpose in capping RSS, rather I cap swap.
FWIW, I tend to use the term reserved memory instead of swap
because that is less confusing to most people.
For CPU related stats, take a look at a discussion I started a while back:
http://mail.opensolaris.org/pipermail/perf-discuss/2005-November/002048.html
One project I would like to kick off sometime is doing per user, per
project, and per zone microstate accounting. Presumably this data
would be available through kstat. The tricky part here is to not
introduce a big load on the system in the process of doing this. The
above discussion and/or others in a similar vein have led me to think
that collecting stats as processes exit and periodically through a
kernel thread would be the way to go. This approach won't be accurate
to subsecond intervals, but generally speaking you don't need better
data than per minute. Such a thread should have no more impact on the
system than a single user running prstat or top with a similar
interval. Further, it would be good data for prstat (e.g. -a) to use.
A follow-on to that would be to have a way to track usage of kernel
taskq work. As more in-kernel functionality comes into existence, it
becomes harder to see where the utilization is. For example, a kstat
that counted the relative amount of time in zfs crypto versus zfs
gzip9 operations would be helpful to the support person that is trying
to answer the call why does vmstat say my system is pegged while
prstat shows no processes consuming CPU? Sure, dtrace can get that
information - but it is hard for the typical person to write and
pretty expensive to run as regular monitoring. But, now I'm a bit off
topic.
The use of mdb rules out use by most users. I dislike tools that make
users (e.g. application owners) ask me for root access. I haven't
looked closely to see which, if any, of the other commands also
require some elevated privileges.
Most (all?) other commands have the full path set but mdb doesn't.
Perhaps $ENV{PATH} = ... would be a good thing to add.
I didn't have a chance to check logic closely or run it on a test
system. I'll offer more feedback if needed when I get a chance to
test it. It is a great start and I can't wait to see it progress.
--
Mike Gerdts
http://mgerdts.blogspot.com/
___
zones-discuss mailing list
zones-discuss@opensolaris.org
Re: [zones-discuss] Zones hosting providers?
Sorry that I don't have an answer to your question, but I have some thoughts that might help :) I see zones hosting as two potentially different services... 1. Virtual Dedicated Solaris Server - You are provided a virtual dedicated server that runs (Open)Solaris. You know its a zone because you type zonename and its not global .. but other than that, they don't really need to disclose what kind of virtualization they choose to use. It probably wouldn't be marketed as zones hosting or anything to do with zones. As an admin, you SSH in and do your thing, just like any other virtual-dedicated-server product. 2. ZONE hosting - You somehow provide a hosting provider with a filesystem (zfs shapshot?) of your pre-made zone to run in the cloud ... Maybe something like network.com or amazon EC (if they were on Solaris). The provider would probably have to provide you a virtualbox copy of their hosting environment with the exact OS/patches, so you could build your zone and it would zoneadm -z BLAH attach without much fuss when you sent it to them. I don't know whether you are thinking of #1 or #2 or both, but in your questioning of service providers, I think if I were you, I would simplify by asking them if they provide Solaris virtual dedicated servers, rather than zones hosting, as your typical CSR won't know what the heck zones are or what they have to do with Solaris or otherwise. ~tommy -- This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] Zone Statistics: monitoring resource use of zones
It has become clear that there is a need to monitor resource consumption of workloads in zones, and an easy method to compare consumption to resource controls. In order to understand how a software tool could fulfill this need, I created an OpenSolaris project and a prototype to get started. If this sounds interesting, you can find the project and Perl script at: http://opensolaris.org/os/project/zonestat/ . If you have any comments, or suggestions for improvement, please let me know on this e-mail list or via private e-mail. --JeffV ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] static routes vs default routes (zones in different subnets)
Hello, As I previously mentioned, I am working on the possibility of putting zones from different security contexts (front-end, application, back- end) into the same physical server, which is effectively putting zones in more than one subnet. We also like to use a load balancer, at least on the front-end net, and increasingly on the back-end nets as well. In order to use a load balancer, the general idea is that you set your default route on the real server to go through the load balancer. I am able to do this fine (s10u6), even without a home brew SMF service to add default routes after zones come up like I have had to use in the past. The problem I ran into this time was that static routes don't seem to use the same intelligence that default routes do. For example, in the global zone I have 4 default routers, each pointing to the default router of a local subnet (172.16.1.254/24, 172.16.2.254/24, and 172.16.3.254/24 (4th is not important at the moment). Inside a local- zone with an interface on the 172.16.1.0/24 network, it only sees the one default route that it can use (172.16.1.254). That's perfect. The problem comes in when I start looking at the admin network. I don't want to setup a load balancer service for SSH into each zone, so I generally set a static route in to get to the admin network using a different gateway (hard-firewall)... route -p add -net 172.16.250.0/24 172.16.1.1 ... of course if I have multiple subnets, I also have... route -p add -net 172.16.250.0/24 172.16.2.1 route -p add -net 172.16.250.0/24 172.16.3.1 this confuses zones in the .2 and .3 subnets, as they see all three routes and try to use the first one, even though they cannot reach 172.16.1.1. I know this would be made simpler with vnic and private IP stack, but in my world, shared-ip is the only thing available (feasible). Thanks in advance, Tommy ___ zones-discuss mailing list zones-discuss@opensolaris.org
