Hi,
I'm making a paper proposal for the LISA system administrators
conference. Below is my current draft abstract for the paper. I need
comments worst case by Wednesday. By or before Tuesday evening (9:30 PM
US mountain time) is best.
I'm looking first and foremost for impact. Eliminating less important
things can increase impact (as does shortening in general). Spelling,
grammar, style, whatever... Tell me what you think. Feel free to reply
to me or the mailing list.
I'm putting it inline to make it easer to comment by email...
The Assimilation Monitoring System -- Cloud-Scale Discovery and Monitoring
*Abstract*
This paper provides a work-in-progress report on the /Assimilation
Monitoring System/ - an extremely scalable discovery-driven
exception-monitoring system designed from the ground up for cloud-scale
data centers -- whether they are clouds or simply very large data
centers. This system continually discovers servers, services,
dependencies and switch connections in the normal course of operation -
without setting off network security alarms.
Because it discovers the set of all servers and what services are
running on monitored servers, it is a straightforward matter to discover
the set of servers and services which are /not/ being monitored --
improving the quality of service being provided. In addition, because
of the detailed knowledge of servers and services, a significant
potential for greatly reduced effort in initial and ongoing
configuration. Because it also discovers dependencies between servers,
this information is useful in understanding the operation of services in
the data center, and for finding the root causes of cascading failures.
The discovery and monitoring data is stored in the Neo4j graph database,
making queries about interconnections and dependencies straightforward
and fast.
The scalability comes about from a novel neighbor monitoring technique.
This technique performs the overwhelming majority of monitoring work in
/*O*/(1) time complexity for all servers in the network -- creating an
architecture which is fundamentally more scalable than conventional
monitoring systems. In addition, this monitoring technique is naturally
topology-sensitive, with more than 90% of the network traffic staying
within a switch. Since the neighbor monitoring topology corresponds to
the network topology, it is straightforward to distinguish server
failures from network failures. Because network topologies are typically
modeled after physical topologies, it is able to monitor multiple data
centers with minimal WAN traffic.
The discovery system is straightforward, and creates no network traffic
to perform discovery. It is also easily extensible by system
administrators, with the ability to discover arbitrary kinds of
information about a server through a simple scripting interface.
The result is a system which provides a rich tapestry of details about
what is actually in the data center, what is monitored, what is not, and
what is working and what is not -- with the ability to scale from a
handful of servers to hundreds of thousands of servers -- with minimal
server and network utilization.
This paper describes the architecture, implementation, current status of
the Assimilation Monitoring System along with future challenges and
opportunities.
--
Alan Robertson <[email protected]> - @OSSAlanR
"Openness is the foundation and preservative of friendship... Let me claim
from you at all times your undisguised opinions." - William Wilberforce
_______________________________________________
Assimilation mailing list - Discovery-Driven Monitoring
[email protected]
http://lists.community.tummy.com/cgi-bin/mailman/listinfo/assimilation
http://assimmon.org/
