Hi Marcus, Somesh, Paul
Thanks for responding and yes, I think in some aspects I may be asking too
much but this isn't just me asking for extra/additional/new features per se.
I think it's easier if I split up the issues that I perceive to be problems
into four distinct areas. There's overlap between them but it might help me
explain better:
1. New features
2. Features that are already there but that can’t be used
3. Features that are turned on that can't be turned off
4. Features that work but which could do with a more granular implementation
In more detail:
1. New features:
Yes this is a contentious one (QoS, Dynamic Routing, OpenVPN etc). I think
that this is the greedy ask and it's not feasible or even fair to ask for
stuff for free. Believe it or not, I'm not asking "for CloudStack to provide
and maintain a fully fledged and featured router distribution in its
provided virtual router". If customer demand pays for something new and cool
to be developed and it's contributed back then great but this sort of thing
can't be an expectation by default. For those that do want more, the
solution here though might be, IMHO the ability to either:
a) Provide a relatively simple way for users (not devs) to customise
the VR
themselves, adding/swapping/deleting packages, orchestration agents (puppet
ansible salt etc) and giving people access to the VR for configuration
without the risk of deletion on recreation; making the VR a 'first-class
citizen' vm instead of a disposable commodity.
b) Allow the use of third party router/firewall VMs to be used instead
of
the default VR. Obviously, this would be at the sacrifice of functions that
a 3rd party might not provide such as userdata and most other functions
would have to be managed manually on the VR without the orchestration
functionality that ACS provides with the official system VMs. There appears
to already be a way to do this but I can’t find much information on this so
I might be very wrong. The router.template.<hypervisor> setting lets me
create my own VR template but things went very wrong when I tried using it
so I gave up. This setting also applies to every single vr in the entire
zone which is not going to fit for many, especially if the replacement is a
commercially licensed product. This is what I gather Somesh alluded to in
his reply. It would be great if we could offer a choice of built-in VR, a
Vyatta VR or a Cisco VR depending on the customer requirements without
having to code a separate network provider for each, get it committed and
wait for a new release.
2. Features that are already there but that can’t be used
These are things like being able to configure the IPsec VPNs, iptables
firewall/NAT rules, or routing tables on VRs in more detail. At the moment,
if it can’t be orchestrated by ACS, it can’t be done at all. If I were to
suggest that configuration of all VMs including user VMs had to be done by
ACS most would likely state that as being utterly ridiculous but yet we
accept that limitation with the VR VMs. Also, some of the hardcoded rules
that validate configuration changes to VRs are simply broken. For example,
yes it is possible and perfectly normal to have two routes with destinations
that overlap if the subnet mask is different - it's called route
summarisation. Another example is that I might want to set my default route
on a VPC VR to the private gateway but because it doesn’t fit with the
specific use-case that whoever designed that aspect had at the time, it's
not allowed. One last one which would need more but not much, is being able
to use DHCP relay functionality already built into DNSMasq - That one would
solve a few problems for hybrid or private cloud deployments where IPAM is
managed with Active Directory integrated tools.
3. Features that are turned on that can't be turned off
This one is a personal annoyance and while there's likely other examples,
one that gets me is the fact that you can’t turn off source NAT on a VPC VR.
Perhaps a sensible *default* might be to use source NAT but there are
perfectly valid reasons why some people might want to turn it off. Some
people might want public IPs inside one or more tiers, some people might be
doing source NAT further up in the network and the 'public' network is
merely a transit network (default route on the VR). These sort of things are
where I actually want ACS to do less and not to try to second guess what I
want - just let me have a little more control over the basics without making
things 'easier' for me (and enforcing it).
4. Features that work but which could do with a more granular implementation
A good example of this would be the fact that you have to specify a CIDR for
a VPC that cannot be changed or added to once configured. Nobody in their
right mind would ever place a constraint like this on an on-premise network.
Yes if you're doing something very simple or you create a VPC for each
application that you build it's less of an issue but if you're using ACS as
part of your more traditional corporate infrastructure, for DR purposes, for
hybrid cloud purposes etc, it's a very notable WTF to any network engineer
when they first see it. This example may be in place to make things simpler
elsewhere (such as routes, firewall rules & VPN connections) but in my mind
is simply brushing additional complexity in the future under the
metaphorical carpet.
Trying to flip the dev/networking approach here, it would be like creating a
really cool automated way of installing and configuring MySQL on every VM
but stopping anyone from uninstalling it or using anything other than
InnoDB. The network guy might say "Why do you need anything other than
MySQL/InnoDB and if you don’t want it, don’t use it". The dev/ops/devops guy
would say they'd rather have a plain VM with no DB at all and install
themselves if needed than have the really cool automation. Let users decide
whether they want ACS to do cool stuff for them or whether they just want
the basics and they'll do the rest (if more is even needed).
In summary, while my initial rant may have come across as wanting more for
nothing, priority 1 for me is actually the option to have *less* but the
ability for me to tweak stuff myself instead of having ACS enforce its view
on how things should be done. Leave the templated network provisioning
procedures alone for where they fit, perhaps leave them as defaults, but don’t
enforce them or assume that everyone wants them. I don't think it is safe to
assume that "Cloud consumers are end users, web developers, application
developers". IMHO, making that assumption, is the cause of the same effect.
If we reject that assumption, we may find ACS to be a bit more welcoming to
others. I don’t believe that more is necessarily needed, in fact less.
Adrian
</rant>
-----Original Message-----
From: Marcus [mailto:shadow...@gmail.com]
Sent: 22 February 2015 07:08
To: d...@cloudstack.apache.org
Cc: users@cloudstack.apache.org
Subject: Re: Network QoS (not bandwidth limiting)
The points raised are certainly valid from an enterprise networking
standpoint, and don't fall on deaf ears, but we should keep things in
perspective. To provide the aforementioned features would be relatively
uncharted territory in the cloud orchestration world (at least not
considering vendor provided networking solutions that only handle the
network part of the equation), so while it would be good to aspire to
providing those things, it should be no surprise that the platform works
that way and lacks such features.
For further perspective, keep in mind that cloud orchestration in general
has been a pitch to software developers and management for "easy
infrastructure". Cloud consumers are end users, web developers, application
developers, so again it should be no surprise that the product provides
features that cater to that, rather than providing the bells and whistles
that a network admin would want to see in their infrastructure. CloudStack
was never built to be pitched to network teams as a cure for managing their
infra deployments, the only cloud product providers doing that are network
vendors who have cloud networking products. This is of course why a VPC
needs IPs defined, as applications care more about how to serve up a web
page than network engineering and managing distinct layer 2 and 3, so the
whole network stack is sandwiched into a simple orchestration mechanism that
gets the application what it needs.
In designing and deploying cloud, the most common complaint I see from
people who are infrastructure maintainers is "why can't I just build the
infrastructure the way I want and then have it orchestrated?".
Unfortunately, we can't just automate and integrate with anyone's pet
design. CloudStack supports many novel and custom network designs simply by
allowing the option of letting you manage the network hardware and being
hands-off (shared/public networks), while also being pluggable to allow
vendors to take over whatever features and they wish. I've seen some pretty
advanced overlay networking provided through third party plugins to
CloudStack that take over all network functionality and provide more.
What's really being asked for here is for CloudStack to provide and maintain
a fully fledged and featured router distribution in its provided virtual
router. It's an admirable project to have if we can get support for it. My
guess is there's a bit of a disconnect in interest though, because many (but
not all) enterprises who want CloudStack for infrastructure automation are
skeptical about a VM as software router and prefer to bring in
aforementioned enterprise vendors who have their own plugins. People who
provide cloud hosting and other services tend to use the routers, but their
interest in enterprise level routing and redundancy varies greatly, and
their customers are designing their apps to be resilient to infrastructure
loss (e.g. most AWS customers). That's of course not entirely the whole
truth, as is evidenced by the work we are seeing on redundant routers, but I
do believe that's why we haven't seen these things from the beginning. They
just haven't been all that important to the target customers, even though
infrastructure engineers are used to providing them.
So now comes my philosophy. In the end, I think the great thing about open
source communities is that if there's the right level of interest, it will
happen. I'm the kind of person who feels a pang of stress at the idea that
something I work on can't be all things to all people, but after building a
hosting business over the last few years I've begun to realize that it's
really only practical to try to be good for a subset of the market and focus
on that. You'll never please everyone, there are limits to what you can
accomplish, and sometimes it's OK to just concede that your product is not
going to work for everyone. If you don't, you'll spread yourself too thin
and fail everyone. In order to make something great you have to have a limit
on your scope. That's not to say you don't listen to your customers, but you
sometimes have to make hard choices on who to listen to and who to upset.
None of this should be taken as a discouragement to the topics at hand, but
again as someone to takes it personally when I don't deliver I wanted to
provide some follow up to address the "rant" and try to provide perspective
on why the things are the way they are.
On Sat, Feb 21, 2015 at 1:58 PM, Somesh Naidu <somesh.na...@citrix.com>
wrote:
> Adrian,
>
> Rant or not, I believe you have raised a valid point and reflect certain
> group of peoples requirement.
>
> Based on your requirement, I believe you are looking for something like
> Vyatta.
>
> Regards,
> Somesh
>
> -----Original Message-----
> From: Adrian Lewis [mailto:adr...@alsiconsulting.co.uk]
> Sent: Friday, February 20, 2015 8:50 PM
> To: users@cloudstack.apache.org
> Subject: RE: Network QoS (not bandwidth limiting)
>
> Tempted to suggest some sort of special interest group where
> networking people can have some input into the dev process despite not
> necessarily being able to produce any code themselves. As an example,
> Schuberg Philis have recently done some great work on the redundant
> VPC VR but to a network person, this sort of functionality is almost
> taken for granted (please don't take this as a lack of appreciation).
> Similarly, the lack of end-to-end QoS for applications running on ACS
> seems to me at least to be a fairly significant oversight. ACS is
> known as having very flexible networking compared with some of the
> alternatives but there does still appear to be an enterprise focus on most
> elements that a 'typical'
> developer (dare I say it, web developer) faces but more of a home
> network approach to the networking side (aside from some pretty
> impressive niche features).
>
> We shouldn't need to rely on proprietary 3rd party products to provide
> a similar level of versatility for networking in ACS in my opinion. It
> seems bizarre to me that we have load balancing, distributed routing &
> ACLs with the OVS controller, PVLANs for isolation, etc, but yet
> still don't have what I would consider basic functions such as better
> control over NAT, firewalling, routing (no dynamic routing protocols
> at all), IPsec, having to specify IP related attributes to what should
> simply be L2 constructs (why does a VPC need to be given a CIDR?!?)
> etc. AWS had a similar issue that lead to the VPC being introduced -
> enterprises consistently rejected the weird and illogical way that
> they did networking back in the day that was overly focussed on web/cloudy
> workloads.
>
> This sounds like a rant and to an extent it is but I'd like to turn it
> into a positive. I feel fairly helpless when the typical response to
> feedback like this is that I should just contribute code. There are a
> number of people that embrace the concept that the community should be
> a collective of not just developers, but at the same time it's pretty
> difficult to feel part of a community that's run almost uniquely by
> developers; it's even a bit intimidating at times. I've seen too many
> commercial companies that abandon innovation in favour of satisfying
> the 'large account' RFC/RFPs and in my opinion the same may apply to a
> project driven largely by the needs of those that can contribute code.
>
> To flip the concept on its head, it would be like a network guy
> creating an amazing cloud orchestration platform but where you can
> only run centos
> 6 with a LAMP stack - yes this might work for a lot of people (and it
> would likely only be adopted by those people) but for those that just
> want to do something a bit different, it would be a fairly frustrating
> experience.
>
> Am I simply being a spoilt kid here or is there room for input that
> might be constructive? Is there anyone here on the list with a
> networking focus that can corroborate these concerns?
>
> Adrian
>
> -----Original Message-----
> From: Somesh Naidu [mailto:somesh.na...@citrix.com]
> Sent: 20 February 2015 18:31
> To: users@cloudstack.apache.org
> Subject: RE: Network QoS (not bandwidth limiting)
>
> I don't think we can. QoS in CS is mostly throttling traffic on the
> virtual interface.
>
> Regards,
> Somesh
>
>
> -----Original Message-----
> From: len.bellem...@alternativenetworks.com
> [mailto:len.bellem...@alternativenetworks.com]
> Sent: Friday, February 20, 2015 5:18 AM
> To: users@cloudstack.apache.org
> Subject: Network QoS (not bandwidth limiting)
>
> Hi All,
>
> Does anyone know if it's possible to do network QoS in Cloudstack? I
> don't mean bandwidth limiting, but rather, prioritising different
> traffic types for voice, etc.
>
> Thanks
> Len
