Hi guys,

It’s really exciting that we are moving to one flexible solution to have the IP 
and interface tables separately. Based on this, I have some ideas for the 
change of the Traffic Monitor(TM). Actually, the TM have two different purposes 
to polling the cache servers. One is to make sure the connectivity is ok, the 
other is to query the stat of the interface. The connectivity is actually for 
the IP (with port) connectivity, while the stat is for the interface. Since we 
will have multiple interfaces and may have multiple IPs for each interface, I 
think the polling to the cache server should be like:
1) Check the connectivity of all the configured IPs of the cache server
  This function works like the keep alive to make sure all the configured IPs 
is reachable.
2) Query the stat for all the configured interfaces of the cache server
     This function is to query the interfaces stat, it doesn’t care which IP 
will be used to query the stat.

So I suggest the change to the current TM to be like:
1) Separate the current polling of cache servers into two different pollings, 
one for the keep alive, the other for the stat query.
  For the keep alive polling, we can use some specific url to check if the IP 
with port is opened or not; For the stat query, we will reuse the astats 
plugin, but need a little change to the uri to support returning multiple 
interfaces stat in the same request.
  If we don’t separate it, we will have to issue each stat query for each IP, 
and all the IPs of one interface will return the same big response, which is 
unnecessary, and will bring much more unnecessary load to the ATS and TM. If we 
separate it, we could make the keep alive check very light weighted, and also 
can use one request to query all the configured interfaces stat. Besides, we 
could also use different polling intervals for the keep alive and stat query.

2) The health availability of the cache server will be based on the configured 
IPs of the cache server
     Currently, we only have the availability of one cache server, but it will 
not be enough when we add multiple interfaces and multiple IPs support. We need 
record the availability for each configured IP so that if it’s assigned, the 
router can check if it can redirect the client request to that assigned IP or 

3) The max available bandwidth check should be changed to per interface.
  Currently, the max available bandwidth is only for the primary interface of 
the cache server, if we have multiple interfaces support, we should check the 
bandwidth availability for each interface. If one interface is overload, we 
will mark all the IPs of that interface to be unavailable.

4) We will pick up one available IP to query the stat.
     Currently, we use the fqdn of the cache server to query the stat, since we 
will have multiple IPs, we could pick up one available IP to query.

Best regards,

On 4/4/18, 1:23 PM, "Nir Sopher" <n...@qwilt.com> wrote:

    Note that beyond the DB, the change should also be reflected into the
    As I see it, a flexible model may be built of the below items:
    1 - Edge server.
    2- Interface
    3. IPs
    The Interface (or should it be called "delivery unit") is the element we
    redirect the traffic to and which is monitored by the traffic-monitor:
    * Each server may have multiple interfaces
    * Each interface may have multiple IPs
    * Interfaces has priorities (abstraction for primary/secondary)
    * Each interface is given a seperate DNS name by the router. Single name
    for the multiple IPs.
    * Each interface is monitored and reported seperately by the traffic
    monitor, including health an stats.
    The router "redirect target decision" may look as follows
    0. Select cache as we do today taking into account the consistent hash. A
    server is in the selection group only if one of its interfaces is found to
    be healthy
    1. Once we have server selected, select an interface out of all interfaces
    of the server with max available priority.
    An additional improvement, may assign DS to interfaces instead of servers.
    A server serves DS X iff one of its interfaces is assigned to the DS.
    On Apr 4, 2018 6:56 AM, "Zhilin Huang (zhilhuan)" <zhilh...@cisco.com>
    Updated the DB schema in section
    On 04/04/2018, 11:02 AM, "Zhilin Huang (zhilhuan)" <zhilh...@cisco.com>
        Good points. I am happy to make this change in the design doc.
        On 03/04/2018, 8:17 PM, "Eric Friedrich (efriedri)" <efrie...@cisco.com>
            I would prefer a consistent way to store all interface and IP
    address information. Its good database design practice to store similar
    information in similar tables (i.e. all IP info in 1 table) rather than
    keep some IPs in the server table and some IPs in another table.
            I also think this refactoring will give us greater flexibility for
    more changes in the future. Outside of this particular use case, we might
    have additional features like sharing edges between public/private networks
    or having multiple (equal priority) streaming interfaces on a cache.
            These future features would be easier if the interface data and IP
    data is all organized into separate tables.
            I’d also like to see the delivery service to IP mapping be a many
    to many mapping in the DB. For this particular feature we will only assign
    a single IP (and we can restrict that in the API if we want), but I am near
    certain that in the future we would like the ability to assign a DS to
    multiple IPs on the same cache.
            > On Apr 3, 2018, at 2:42 AM, Zhilin Huang (zhilhuan) <
    zhilh...@cisco.com> wrote:
            > Hi Mark,
            > Thanks for your comments. Please check my reply in another thread:
            > If we all agreed to use unified tables for all IPs and/or
    interfaces: primary, management, secondary, then there need to be two
    tables: IP and interface.
            > And in the server table, we need to replace the original
    "interface_xxx", "ip_xxx", "ip6_xxx" fields with a "primary_ip_id" field.
    And do similar things to management IP.
            > Thanks,
            > Zhilin
            > On 03/04/2018, 7:08 AM, "Mark Torluemke" <mtorlue...@apache.org>
            >    I would support an 'interfaces' table (adding some sort of a
    'type' column)
            >    that would include moving the management and lights out
            >    interfaces to that table as well.
            >    Cheers,
            >    Mark
            >    On Mon, Apr 2, 2018 at 2:39 PM, Nir Sopher <n...@qwilt.com>
            >> Hi Zhilin,
            >> I took a quick look into the spec. Hope to have the opportunity
    to dive
            >> deeper into it soon so we can further discuss it.
            >> For now I have a 2 questions.
            >> In the spec, you refer to "secondary interfaces", and you have a
    list of
            >> secondary interfaces added.
            >> IIUC the secondary interfaces are used as long as they are
    available, and
            >> when down, you move to the primary interface.
            >> Why not, instead of holding a secondary interfaces table, move
            >> interfaces to a separate table? Primary and secondary.
            >> For each interface you can hold:
            >>   - Server id
            >>   - name (e.g. eth0)
            >>   - IPv6
            >>   - IPv4
            >>   - Priority (Integer as flexible as you wish: e.g. "1" for
            >>   "2" for "primary" in your example,)
            >> Additionally, it is not clear to me what happens if one of the
            >> fails?
            >> Does every interface has a unique DNS name? If an interface
    fails, are
            >> redirects
            >> sent only to the available (secondary) interfaces?
            >> Thanks,
            >> Nir
            >> On Mon, Apr 2, 2018 at 10:21 AM, Zhilin Huang (zhilhuan) <
            >> zhilh...@cisco.com
            >>> wrote:
            >>> Hi Guys,
            >>> This was originally posted in another discussion. Resend this
    in a
            >>> standalone topic to catch more awareness. The link for the
    design doc:
            >>> -ZS9nSsd4/edit?usp=sharing
            >>> Short summary for the feature design:
            >>> ---
            >>> There is feature request from market to add secondary IPs
    support on edge
            >>> cache servers, and the functionality to assign a delivery
    service to a
            >>> secondary IP of an edge cache.
            >>> This feature requires Traffic Ops implementation to support
    secondary IP
            >>> configuration for edge cache, and delivery service assignment to
            >> secondary
            >>> IP.
            >>> Traffic Monitor should also monitor connectivity of secondary
            >>> configured. And Traffic Router needs support to resolve
    streamer FQDN to
            >>> secondary IP assigned in a delivery service.
            >>> Traffic Server should record the IP serving client request. And
            >>> reject request to an unassigned IP for a delivery service.
            >>> This design has taken compatibility into consideration: if no
            >> IP
            >>> configured, or some parts of the system has not been upgraded
    to the
            >>> version supports this feature, the traffic will be served by
    primary IPs
            >> as
            >>> before.
            >>> ---
            >>> Much appreciated and welcome to any comments. If no major
    objections, we
            >>> planned to start coding this week.
            >>> Thanks,
            >>> Zhilin

Reply via email to