[
https://issues.apache.org/jira/browse/METRON-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633469#comment-15633469
]
Jon Zeolla commented on METRON-195:
-----------------------------------
I would add to the "Single Pane of Glass" point that this UI should be able to
query across the indexed (Elasticsearch/SOLR) and HDFS stores. METRON-542 is
intended to lay the foundation for this ability.
> Next Generation Metron UI
> -------------------------
>
> Key: METRON-195
> URL: https://issues.apache.org/jira/browse/METRON-195
> Project: Metron
> Issue Type: Wish
> Reporter: George Vetticaden
> Labels: ForwardLookingEpic, METRON_UI
> Attachments: Metron Next Gen UI.png
>
>
> !Metron Next Gen UI.png|width=800, height=400!
> Over the last few weeks, we have had the chance to interview over 20
> different SOC/Security personnel from various organizations. These folks
> represented the following personas:
> * Tier 1, 2 and 3 SOC Analyst
> * SOC Manager
> * Security Platform Architects
> * SIEM Engineers
> * Data Scientists
> The interviews ranged from 10 minute to 2 hour conversations and some of the
> interview questions can be found on the following wiki page:
> https://cwiki.apache.org/confluence/display/METRON/SOC+Interview+Discussion+Guide
> In addition to these questions, we had the opportunity to watch them use
> their current security tooling and also watch them use the current Metron
> based Kibana UI (for folks that had Metron deployed)
> The user interviews are documented here:
> https://cwiki.apache.org/confluence/display/METRON/User+Interviews
> This provides us a valuable trove of requirements and insights about the
> challenges faced by SOC personnel.
> While the current Metron UI based on Kibana 3/4 has some advantages including
> the ability to connect to different elastic indexes, dashboard and panel
> support, and rich support for different panel types, it was clear the
> current Metron UI lacks some key capabilities that most SOC personnel would
> require. Some key limitations identified based on these interviews included:
> * Clunky unintuitive user interface. Someone has to to teach how to use
> Metron UI couple of times for a user to understand even how to start with it.
> * All things around searching was very difficult to understand. This included
> things like
> ** Difference between search and filtering
> ** How to create a pinned/named query
> ** How to associate pinned queries to panels
> ** Search DSL was difficult to use
> * No Authentication or RBAC support
> * Alert Triaging not supported
> In addition to these challenges, the following were key requirements that the
> SOC personnel deemed critical for Metron to have:
> * Support for Alert Triaging. The ability for the user to group/search alerts
> and their related telemetries and create a case/ticket from it.
> * Case Workflow Management
> * Alert Queues that SOC analysts can pick up alerts to work on
> * Authentication & Authorization/RBAC Support
> * Make Search easier for a novice easier
> * Faceting Support
> * More robust pivoting functionality.
> * Help the advanced user write more complicated queries/searches
> * More advanced visualization support including link analysis / Knowledge
> Graphs
> * Curated/Filtered views of Alerts for SOC 1 Analysts
> * KPIs that provides accountability (e/g: average time spent on case, top 10
> critical alerts, SLA violations..)
> * Better/ more intelligent grouping of alerts into a "Meta-Alert"
> * Drill down of Meta-Alert into the set of alerts the meta consists of.
> Drilling down on a raw alert to the the telemetry data sources
> * Management UI Tooling around provisioning, monitoring and managing Metron.
> Based on this data, some key design principles started to emerge that the
> Metron UI should adhere to:
> # A Single Pane of Glass - One interface to monitor and act Telemetry feeds,
> Behavior anomalies, Alerts and Meta-Alerts, Metron system health,
> Investigate, slice and dice, hunt and seek, search & filter, Collaborate and
> share info and Identify and collect data
> # Find Needles in the Haystack - Find that tiny detail that helps you create
> a better defense. Meta-Alerts are doorways to the data details Meta-Alerts
> can contain hundreds of smaller related alerts. Search and Facet to dive
> deeper
> # Same Data, Many Angles - One data set can be visualized in many ways. Not
> only do we need to show many views of a data set out of the box. We must also
> allow for customization.
> # It’s All Collaborative - I can share anything I see. I can preserve it for
> later. I can save it for myself or send to someone else. Once a collaboration
> begins, we can annotate and categorize data for future use and investigation.
> Oversight. All activity is audited. So what I do and the amount of time I
> spend doing it can be backtracked and replayed somehow.
> # It Evolves - Off the shelve, the system will be powerful. However, it can
> grow and evolve with threat intelligence feeds, Behavior Modeling, App store
> style add-ons, and Learning Models. Learning Models use machine learning to
> predict anomalous behavior. It takes a pattern and alerts when similar
> patterns emerge. Behavior Modeling observers the average entity behavior and
> creates an expected range of behavior. The system learns from what happens
> as well as what you feed it.
> # A Tailored Fit - The system is personalized and customizable. Based on
> role, a user may get a specific interface. Based on experience, a user can
> modify interfaces to suit there personal preferences.
> # I’m Never Lost - No matter how deep a user dives into the data, they always
> know where they are. I can move back through my diving to a previous result.
> The complexity of these systems can be immense. We need to design for
> interruption and the resuming of their place by providing some way-finding
> tools to guide users.
> # It Teaches You - The system teaches novice users to become advanced users.
> As you use the system if gives you all the hints you need to become an
> expert. Complex things are made easy enough for anyone to be able to do.
> # Think for Them - The system anticipates user needs and does the thinking
> for them. We put the burden of complexity on the software so that we simplify
> the user experience. Make the experience as human as possible, guide and help
> users achieve outcomes. Shorten the number clicks by decreasing complexity.
> Hence, based on these challenges and requirements discovered from these
> interviews, this is a proposal to build a set of next generation user
> interfaces for Apache Metron that caters to the different SOC personas that
> will use Metron. Fore more details on SOC personnas see here:
> https://cwiki.apache.org/confluence/display/METRON/Metron+User+Personas+And+Benefits
> As the below above illustrates, the proposal is to move away from Kibana and
> introduce a number of different views/UIs that caters to different types of
> SOC users across three Phases:
> * Phase 1 - This phase focuses on SOC analysts and Managers. They includes
> the following different Views/UIs:
> ** Metron Investigator UI - Ability for SOC analyst to do alert triaging,
> threat hunting and searching. It should also provide the ability to export a
> view based on the filters and seed a new Metron Case with that context.
> ** Case Management Workflow - The ability to manage Metron cases. This
> entails supporting different workflow statuses (new, pending, in-progress,
> escalate, etc..) It also should provide Metron case queue management
> functionality.
> KPI and Situational Awareness Dashboards - Set of Dashboards that provides
> situational awareness of the environment as well key performance indicators
> for managers.
> * Phase 2 - The primary focus is on the Platform Engineers / Dev Ops that
> will manage the Metron application. This involves interfaces to manage
> topologies, enrichments, threat intel data sources, configuration, ec..
> * Phase 3 - This phase is all about the data scientist and providing an
> integrated analytical environment that uses powerful tools such as Zeppelin,
> Jupiter, Python etc..
>
> This initial high level proposal will need to be fleshed out as the community
> provides their feedback.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
