I left a comment on the JIRA. I think your design is promising. One other thing I would suggest is for us to crowd source requirements around model management. Specifically:
Model versioning and retention Model access controls management Requirements around concept drift Requirements around model output Any model audit and logging requirements What model metrics need to be exposed Requirements around failure modes 03.07.2016, 14:00, "Casey Stella" <[email protected]>: > Hi all, > > I think we are at the point where we should try to tackle Model as a > service for Metron. As such, I created a JIRA and proposed an architecture > for accomplishing this within Metron. > > My inclination is to be data science language/library agnostic and to > provide a general purpose REST infrastructure for managing and serving > models trained on historical data captured from Metron. The assumption is > that we are within the hadoop ecosystem, so: > > - Models stored on HDFS > - REST Model Services resource-managed via Yarn > - REST Model Services discovered via Zookeeper. > > I would really appreciate community comment on the JIRA ( > https://issues.apache.org/jira/browse/METRON-265). The proposed > architecture is attached as a document to that JIRA. > > I look forward to feedback! > > Best, > > Casey ------------------- Thank you, James Sirota PPMC- Apache Metron (Incubating) jsirota AT apache DOT org
