Repository: metron Updated Branches: refs/heads/master b005563fe -> eaae51507
METRON-1206 Make alerts UI conform to ops UI for install (merrimanr) closes apache/metron#773 Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/eaae5150 Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/eaae5150 Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/eaae5150 Branch: refs/heads/master Commit: eaae515076ee51be9709753855df35bf204808c1 Parents: b005563 Author: merrimanr <[email protected]> Authored: Wed Sep 27 17:39:49 2017 -0500 Committer: merrimanr <[email protected]> Committed: Wed Sep 27 17:39:49 2017 -0500 ---------------------------------------------------------------------- metron-interface/metron-alerts/README.md | 90 ++++++----- metron-interface/metron-alerts/alerts_ui.yml | 21 +++ metron-interface/metron-alerts/assembly.xml | 8 +- .../metron-alerts/scripts/alerts-server.js | 34 ++-- .../metron-alerts/scripts/metron-alerts-ui | 159 +++++++++++++++++++ .../metron-alerts/scripts/start-dev.sh | 2 +- .../metron-alerts/scripts/start_alerts_ui.sh | 24 --- 7 files changed, 249 insertions(+), 89 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/README.md ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/README.md b/metron-interface/metron-alerts/README.md index db7f038..765ad83 100644 --- a/metron-interface/metron-alerts/README.md +++ b/metron-interface/metron-alerts/README.md @@ -6,15 +6,64 @@ - [Installing on an existing Cluster](#installing-on-an-existing-cluster) ## Caveats -* UI doesn't have an authentication module yet * UI uses local storage to save all the data. A middleware needs to be designed and developed for persisting the data ## Prerequisites -* Elastic search should be up and running and should have some alerts populated by metron topologies +* The Metron REST application should be up and running and Elasticsearch should have some alerts populated by Metron topologies +* The Management UI should be installed (which includes [Express](https://expressjs.com/)) * The alerts can be populated using Quick Dev, Full Dev or any other setup * UI is developed using angular4 and uses angular-cli * node.JS >= 7.8.0 +## Installation + +### From Source + +1. Package the application with Maven: + + ``` + cd metron-interface/metron-alerts + mvn clean package + ``` + +1. Untar the archive in the $METRON_HOME directory. The directory structure will look like: + + ``` + bin + metron-alerts-ui + web + expressjs + alerts-server.js + alerts-ui + web assets (html, css, js, ...) + ``` + +1. Copy the `$METRON_HOME/bin/metron-alerts-ui` script to `/etc/init.d/metron-alerts-ui` + +1. [Express](https://expressjs.com/) is installed at `$METRON_HOME/web/expressjs/` as part of the Management UI installation process. The Management UI should be installed first on the same host as the Alerts UI. + +## Configuration + +The Alerts UI is configured in the `$METRON_HOME/config/alerts_ui.yml` file. Create this file and set the values to match your environment: + +``` +port: port the alerts UI will run on + +rest: + host: REST application host + port: REST applciation port +``` + +## Usage + +After configuration is complete, the Management UI can be managed as a service: + +``` +service metron-alerts-ui start +``` + +The application will be available at http://host:4201 assuming the port is set to `4201`. Logs can be found at `/var/log/metron/metron-alerts-ui.log`. + ## Development Setup 1. Install all the dependent node_modules using the following command @@ -26,9 +75,9 @@ ``` ./scripts/start-dev.sh ``` -1. You can view the GUI @http://localhost:4200 . The default credentials for login are admin/password +1. You can view the GUI @http://localhost:4201. The default credentials for login are admin/password -**NOTE**: *In the development mode ui by default connects to ES at http://node1:9200 for fetching data. If you wish to change it you can change the ES url at metron/metron-interface/metron-alerts/proxy.conf.json* +**NOTE**: *In the development mode ui by default connects to REST at http://node1:8082 for fetching data. If you wish to change it you can change the REST url at metron/metron-interface/metron-alerts/proxy.conf.json* ## E2E Tests @@ -49,36 +98,3 @@ An expressjs server is available for mocking the elastic search api. 1. E2E tests uses data from full-dev wherever applicable. The tests assume rest-api's are available @http://node1:8082 **NOTE**: *e2e tests covers all the general workflows and we will extend them as we need* - -## Mpack Integration -Yet to come - -## Installing on an existing Cluster -1. Build Metron: - ``` - mvn clean package -DskipTests - ``` - -1. Copy `metron/metron-interface/metron-alerts/target/metron-alerts-METRON_VERSION-archive.tar.gz` to the desired host. - -1. Untar the archive in the target directory. The directory structure will look like: - ``` - bin - start_alerts_ui.sh - web - alerts-ui - package.json - server.js - web assets (html, css, js, ...) - ``` - -1. [Expressjs](https://github.com/expressjs/express) webserver script is included in the build that will serve the application. (The script has few rewrite rules and we can replace expressjs with any other webserver) - -1. Then start the application with the script: - ``` - ./bin/start_alerts_ui.sh - Usage: server.js -p [port] -r [restUrl] - Options: - -p Port to run metron alerts ui [required] - -r, --resturl Url where elastic search rest api is available [required] - ``` http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/alerts_ui.yml ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/alerts_ui.yml b/metron-interface/metron-alerts/alerts_ui.yml new file mode 100644 index 0000000..4481fa9 --- /dev/null +++ b/metron-interface/metron-alerts/alerts_ui.yml @@ -0,0 +1,21 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +port: 4201 + +rest: + host: localhost + port: 8080 http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/assembly.xml ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/assembly.xml b/metron-interface/metron-alerts/assembly.xml index 63e7380..f392a66 100644 --- a/metron-interface/metron-alerts/assembly.xml +++ b/metron-interface/metron-alerts/assembly.xml @@ -22,24 +22,24 @@ <directory>${project.basedir}/dist</directory> <outputDirectory>/web/alerts-ui</outputDirectory> <excludes> + <exclude>**/.npmignore</exclude> </excludes> <fileMode>0644</fileMode> </fileSet> <fileSet> <directory>${project.basedir}/scripts</directory> - <outputDirectory>/web/alerts-ui</outputDirectory> + <outputDirectory>web/expressjs</outputDirectory> <includes> - <include>package.json</include> <include>alerts-server.js</include> </includes> <fileMode>0644</fileMode> </fileSet> <fileSet> <directory>${project.basedir}/scripts</directory> - <outputDirectory>/bin</outputDirectory> + <outputDirectory>bin</outputDirectory> <useDefaultExcludes>true</useDefaultExcludes> <includes> - <include>start_alerts_ui.sh</include> + <include>metron-alerts-ui</include> </includes> <fileMode>0755</fileMode> <lineEnding>unix</lineEnding> http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/scripts/alerts-server.js ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/scripts/alerts-server.js b/metron-interface/metron-alerts/scripts/alerts-server.js index 6a999f2..8c0025b 100644 --- a/metron-interface/metron-alerts/scripts/alerts-server.js +++ b/metron-interface/metron-alerts/scripts/alerts-server.js @@ -27,28 +27,22 @@ var serveStatic = require('serve-static'); var favicon = require('serve-favicon'); var proxy = require('http-proxy-middleware'); var argv = require('optimist') - .demand(['p', 'r']) - .usage('Usage: alert-server.js -p [port] -r [restUrl]') - .describe('p', 'Port to run metron management ui') - .describe('r', 'Url where metron rest application is available') + .demand(['c']) + .alias('c', 'config_file') + .usage('Usage: alerts-server.js -c [config_file]') + .describe('c', 'Path to alerts_ui.yml') .argv; +var YAML = require('yamljs'); -var port = argv.p; var metronUIAddress = ''; var ifaces = os.networkInterfaces(); -var restUrl = argv.r || argv.resturl; -var conf = { - "rest": { - "target": restUrl, - "secure": false - } -}; +var uiConfig = YAML.load(argv.c); Object.keys(ifaces).forEach(function (dev) { ifaces[dev].forEach(function (details) { if (details.family === 'IPv4') { metronUIAddress += '\n'; - metronUIAddress += 'http://' + details.address + ':' + port; + metronUIAddress += 'http://' + details.address + ':' + uiConfig.port; } }); }); @@ -60,17 +54,11 @@ function setCustomCacheControl (res, path) { res.setHeader("Expires", new Date(Date.now() + 2592000000).toUTCString()); } -var rewriteSearchProxy = proxy({ - target: restUrl, - ws: true, - pathRewrite: { - '^/search' : '' - } -}); - app.use(compression()); -app.use('/api', proxy(conf.rest)); +var restUrl = 'http://' + uiConfig.rest.host + ':' + uiConfig.rest.port; +app.use('/api/v1', proxy(restUrl)); +app.use('/logout', proxy(restUrl)); app.use(favicon(path.join(__dirname, '../alerts-ui/favicon.ico'))); @@ -83,6 +71,6 @@ app.get('*', function(req, res){ res.sendFile(path.resolve('../alerts-ui/index.html')); }); -app.listen(port, function(){ +app.listen(uiConfig.port, function(){ console.log("Metron alerts ui is listening on " + metronUIAddress); }); http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/scripts/metron-alerts-ui ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/scripts/metron-alerts-ui b/metron-interface/metron-alerts/scripts/metron-alerts-ui new file mode 100644 index 0000000..bb8e67c --- /dev/null +++ b/metron-interface/metron-alerts/scripts/metron-alerts-ui @@ -0,0 +1,159 @@ +#!/usr/bin/env bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# metron alerts UI service +# chkconfig: - 20 80 +# description: Alerts UI +# processname: metron-alerts-ui +# + +# all LSB compliant distributions provide the following +# http://refspecs.linuxbase.org/LSB_3.0.0/LSB-PDA/LSB-PDA/iniscrptfunc.html +if [ -f /lib/lsb/init-functions ]; then + . /lib/lsb/init-functions +fi + +NAME=metron-alerts-ui +DESC="Metron Alerts UI" +METRON_VERSION=${project.version} +METRON_HOME=/usr/metron/$METRON_VERSION +METRON_LOG_DIR="/var/log/metron" +METRON_PID_DIR="/var/run/metron" +METRON_USER="metron" +METRON_GROUP="metron" +METRON_SYSCONFIG="/etc/default/metron" +if [ -f "$METRON_SYSCONFIG" ]; then + set -a + . "$METRON_SYSCONFIG" +fi + +PIDFILE="$METRON_PID_DIR/$NAME.pid" + +DAEMON="node $METRON_HOME/web/expressjs/alerts-server.js -c $METRON_HOME/config/alerts_ui.yml" + +# +# start the rest application +# +start() { + + # if pidfile exists, do not start another + if [ -f $PIDFILE ]; then + PID=`cat $PIDFILE` + printf "OK [$PID]\n" + return + fi + + if [ ! -d "$METRON_LOG_DIR" ]; then + mkdir -p "$METRON_LOG_DIR" && chown "$METRON_USER":"$METRON_GROUP" "$METRON_LOG_DIR" + fi + + if [ ! -d "$METRON_PID_DIR" ]; then + mkdir -p "$METRON_PID_DIR" && chown "$METRON_USER":"$METRON_GROUP" "$METRON_PID_DIR" + fi + + # kick-off the daemon + CMD="$DAEMON >> $METRON_LOG_DIR/$NAME.log 2>&1 & echo \$!" + PID=`su -c "$CMD" $METRON_USER` + + if [ -z $PID ]; then + printf "Fail\n" + else + echo $PID > $PIDFILE + printf "Ok [$PID]\n" + fi +} + +# +# stop the rest application +# +stop() { + if [ -f $PIDFILE ]; then + PID=`cat $PIDFILE` + while sleep 1 + echo -n "." + kill -0 $PID >/dev/null 2>&1 + do + kill $PID + done + rm -f $PIDFILE + printf "%s\n" "Ok" + else + printf "%s\n" "Not running" + fi +} + +# +# status check of the rest application +# +status() { + if [ -f $PIDFILE ]; then + PID=`cat $PIDFILE` + if [ -z "`ps axf | grep ${PID} | grep -v grep`" ]; then + printf "%s\n" "Process dead but pidfile exists" + else + echo "Running" + fi + else + printf "%s\n" "Service not running" + fi +} + +case "$1" in + + ############################################################################## + # start + # + start) + printf "%-50s \n" "Starting $NAME..." + start + ;; + + ############################################################################## + # status + # + status) + printf "%-50s \n" "Checking $NAME..." + status + ;; + + ############################################################################## + # stop + # + stop) + printf "%-50s \n" "Stopping $NAME..." + stop + ;; + + ############################################################################## + # restart + # + restart) + $0 stop + $0 start + ;; + + ############################################################################## + # reload + # + reload) + ;; + + *) + echo "Usage: $0 {status|start|stop|restart}" + exit 1 +esac http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/scripts/start-dev.sh ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/scripts/start-dev.sh b/metron-interface/metron-alerts/scripts/start-dev.sh index 5f6941a..86814a5 100755 --- a/metron-interface/metron-alerts/scripts/start-dev.sh +++ b/metron-interface/metron-alerts/scripts/start-dev.sh @@ -16,4 +16,4 @@ # limitations under the License. # SCRIPTS_ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -$SCRIPTS_ROOT/../node_modules/@angular/cli/bin/ng serve --proxy-config proxy.conf.json $@ +$SCRIPTS_ROOT/../node_modules/@angular/cli/bin/ng serve --proxy-config proxy.conf.json --port 4201 $@ http://git-wip-us.apache.org/repos/asf/metron/blob/eaae5150/metron-interface/metron-alerts/scripts/start_alerts_ui.sh ---------------------------------------------------------------------- diff --git a/metron-interface/metron-alerts/scripts/start_alerts_ui.sh b/metron-interface/metron-alerts/scripts/start_alerts_ui.sh deleted file mode 100755 index 732ea08..0000000 --- a/metron-interface/metron-alerts/scripts/start_alerts_ui.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/bash -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -METRON_VERSION=${project.version} -METRON_HOME=/usr/metron/$METRON_VERSION - -cd $METRON_HOME/web/alerts-ui -npm install -node $METRON_HOME/web/alerts-ui/alerts-server.js $*
