On 26/11/12 22:18, Alon Bar-Lev wrote:
> ----- Original Message -----
>> From: "Livnat Peer" <lp...@redhat.com>
>> To: "Shu Ming" <shum...@linux.vnet.ibm.com>
>> Cc: "Alon Bar-Lev" <abar...@redhat.com>, "VDSM Project Development" 
>> <vdsm-devel@lists.fedorahosted.org>
>> Sent: Monday, November 26, 2012 2:57:19 PM
>> Subject: Re: [vdsm] Future of Vdsm network configuration - Thread mid-summary
>> On 26/11/12 03:15, Shu Ming wrote:
>>> Livnat,
>>> Thanks for your summary.  I got comments below.
>>> 2012-11-25 18:53, Livnat Peer:
>>>> Hi All,
>>>> We have been discussing $subject for a while and I'd like to
>>>> summarized
>>>> what we agreed and disagreed on thus far.
>>>> The way I see it there are two related discussions:
>>>> 1. Getting VDSM networking stack to be distribution agnostic.
>>>> - We are all in agreement that VDSM API should be generic enough
>>>> to
>>>> incorporate multiple implementation. (discussed on this thread:
>>>> Alon's
>>>> suggestion, Mark's patch for adding support for netcf etc.)
>>>> - We would like to maintain at least one implementation as the
>>>> working/up-to-date implementation for our users, this
>>>> implementation
>>>> should be distribution agnostic (as we all acknowledge this is an
>>>> important goal for VDSM).
>>>> I also think that with the agreement of this community we can
>>>> choose to
>>>> change our focus, from time to time, from one implementation to
>>>> another
>>>> as we see fit (today it can be OVS+netcf and in a few months we'll
>>>> use
>>>> the quantum based implementation if we agree it is better)
>>>> 2. The second discussion is about persisting the network
>>>> configuration
>>>> on the host vs. dynamically retrieving it from a centralized
>>>> location
>>>> like the engine. Danken raised a concern that even if going with
>>>> the
>>>> dynamic approach the host should persist the management network
>>>> configuration.
>>> About dynamical retrieving from a centralized location,  when will
>>> the
>>> retrieving start? Just in the very early stage of host booting
>>> before
>>> network functions?  Or after the host startup and in the normal
>>> running
>>> state of the host?  Before retrieving the configuration,  how does
>>> the
>>> host network connecting to the engine? I think we need a basic well
>>> known network between hosts and the engine first.  Then after the
>>> retrieving, hosts should reconfigure the network for later
>>> management.
>>> However, the timing to retrieve and reconfigure are challenging.
>> We did not discuss the dynamic approach in details on the list so far
>> and I think this is a good opportunity to start this discussion...
>> From what was discussed previously I can say that the need for a well
>> known network was raised by danken, it was referred to as the
>> management
>> network, this network would be used for pulling the full host network
>> configuration from the centralized location, at this point the
>> engine.
>> About the timing for retrieving the configuration, there are several
>> approaches. One of them was described by Alon, and I think he'll join
>> this discussion and maybe put it in his own words, but the idea was
>> to
>> 'keep' the network synchronized at all times. When the host have
>> communication channel to the engine and the engine detects there is a
>> mismatch in the host configuration, the engine initiates 'apply
>> network
>> configuration' action on the host.
>> Using this approach we'll have a single path of code to maintain and
>> that would reduce code complexity and bugs - That's quoting Alon Bar
>> Lev
>> (Alon I hope I did not twisted your words/idea).
>> On the other hand the above approach makes local tweaks on the host
>> (done manually by the administrator) much harder.
>> Any other approaches ?
>> I'd like to add a more general question to the discussion what are
>> the
>> advantages of taking the dynamic approach?
>> So far I collected two reasons:
>> -It is a 'cleaner' design, removes complexity on VDSM code, easier to
>> maintain going forward, and less bug prone (I agree with that one, as
>> long as we keep the retrieving configuration mechanism/algorithm
>> simple).
>> -It adheres to the idea of having a stateless hypervisor - some more
>> input on this point would be appreciated
>> Any other advantages?
>> discussing the benefits of having the persisted
>> Livnat
> Sorry for the delay. Some more expansion.
> After boot a host running vdsm is able to receive communication from engine.
> This means that host has legitimate layer 2 configuration and layer 3 
> configuration for the interface used to communicate to engine.
> Reduce complexity of implementation, so that only one algorithm is used in 
> order to reach to operative state as far as networking is concerned.
> (Storage is extremely similar I can s/network/storage/ and still be relevant).

For reaching the mission above we can also use the approach suggested by
Adam. start from a clean configuration and execute setup network to set
the host networking configuration. In Adam's proposal VDSM itself is
issuing the setupNetwork and in your approach the engine does.

> Host running vdsm is a complete slave of its master, will it be ovirt-engine 
> or other engine.
> Having a complete slave ease implementation:
>  1. Master always apply the setting as-is.
>  2. No need to consider slave state.
>  3. No need to implement AI to reach from unknown state X to known state Y + 
> delta.
>  4. After reboot (or fence) host is always in known state.
> A. Given communication to vdsm, construct required vlan, bonding, bridge 
> setup on machine.
> B. Reboot/Fence - host is reset, apply A.
> C. Network configuration is changed at engine:
>   (1) Drop all resources that are not used by active VMs.

I'm not sure what you mean by the above, drop all resources *not* used
by VMs?

>   (2) Apply A.
> D. Host in maintenance - network configuration can be changed, will be 
> applied when host go into active, apply C (no resources are used by VMs, all 
> resources are dropped).
> E. Critical network is down (Host not operative) - network configuration is 
> not changed.
> F. Host unreachable (None responsive) - network configuration cannot be 
> changed.

What happens if we have a host that is added to the engine (or used to
be non-operational and now returns to up) and reports a network
configuration different than what is configured in the engine?

> Single deterministic algorithm to apply network configuration.
> Pre-defined state after host reboot/fence, host always reachable, previous 
> network configuration that may be malformed is not in effect.
> Easy to integrate with various network management solution, can it be 
> primitive iproute, brctl implementation, NetworkManager, OVS or any other 
> configuration, as Linux is Linux is Linux, the ability to interact with the 
> kernel is single, while in order to persist implementation requires to 
> interact with the distribution.
> Moreover, a stateless implementation may be integrated with larger set of 
> network management tools, as no assumption of persistence is added to the 
> requirements, so if OVS is non-persistent, we use it as-is.
> We should aspire to reach to a state in which ovirt-node or any other similar 
> solution is totally stateless, adding a new node to a cluster should be some 
> blade rebooting from PXE, each persistence layer we drop, the closer we reach 
> to managing a large data center built on huge number of machines go up/down 
> as required joining different clusters.
> While discussing clusters, we should also consider autonomic clusters that 
> enforces policy even if ovirt-engine is unreachable, in this mode we would 
> like a primitive manager to be able to enforce policy including networking, 
> while allowing adding/removing nodes without performing any local 
> configuration.
> System administrator will not be allowed to modify 'by hand' any of the 
> network settings (except of this basic engine reachability).
> Special settings can be set in the master, which will apply them via the 
> master->vdsm protocol, which in turn use the network management interface in 
> order to push them, this method should be generic enough to allow pushing 
> most of the configuration setting allowed (key=value). This approach will 
> also help replacing/adding nodes in cluster and/or mass deployment.
> Edge conditions can be handled by executing some script on host machine, 
> allowing administrator to override network configuration upon network 
> configuration event.
> Assuming the host running vdsm as a complete slave and stateless will enable 
> us to provide better control over that host in the short and long run.
> Manual intervention on hosts serving as hypervisors has the flexibility 
> argument. However at mass deployment, large data-center or dynamic 
> environment this flexibility argument becomes liability.
> Thank you,
> Alon Bar-Lev

vdsm-devel mailing list

Reply via email to