I would like to start a discussion around adding a REST API to Metron. I believe this is well worth the effort and would provide several benefits including:
- Giving users a uniform, well-tested and well-documented interface for interacting with Metron - Providing a layer of security (role-based, ACL, etc) around Metron functions - Enabling developers to build tools and interfaces on top of Metron without needing detailed knowledge of Metron internals The purpose of this discussion is to gather requirements around what the REST API should include and the features it should provide. Assuming we decide a REST API is a worthy addition and can agree on a comprehensive list of requirements, I will start a separate more technical discussion around architecture and design. I will start it off with a list of requirements that I think are important. The REST API should: - Be secure and adhere to modern web security standards - Provide a pluggable authentication mechanism - Be written in Java since the majority of Metron is written in Java - Be powered by a mature, open source web development framework with strong community support - Be as comprehensive as possible, meaning any task or function in Metron should be available in the REST API (are there exceptions?) - Have detailed documentation that always reflects the current version - Have the same level of installation tooling as Metron platform The following is a initial list of functions I think should be included: - CRUD functions for parser, enrichment, and indexing zookeeper configurations - A function to retrieve versioning and auditing information for zookeeper configs - Functions for starting/stopping/enabling/disabling various Storm topologies - A function to retrieve sample data from various Kafka topics - A function to test Grok statements - A function to test parsers against sample data - A function to test enrichments against sample data - Functions that give status for the various components in Metron (HBase, Storm topologies, enrichments, etc) - A function that tests and runs Stellar statements against sample data. I will continue to add to these lists as I think of more. Looking forward to hearing everyone’s ideas and input. Ryan Merriman
