[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball closed the pull request at: https://github.com/apache/metron/pull/ ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r213434094
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
Sure that's fine.
---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198883 --- Diff: metron-interface/metron-alerts-host/pom.xml --- @@ -0,0 +1,143 @@ + --- End diff -- Done, and agreed, it should be a formal style guide addition, and part of the formatting configs on the contributor wiki really. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198753 --- Diff: metron-interface/metron-alerts-host/src/main/java/org/apache/metron/ui/AlertsApplication.java --- @@ -0,0 +1,31 @@ +/** --- End diff -- Done ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213198659 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Done, I moved the spring versions up to interface. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213124491 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Makes sense. I guess I'm half thinking about the idea of 3rd party extensions, but we can at least be consistent within the official bits and let people override if they absolutely for some strange reason have to definitely use something different. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r213124377
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
Right, my mistake. We should change it and carry through. Would it make
sense to do that as part of the follow on PR for the mpack work, since that
will be where the bulk of the changes for this would be?
---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r213093853 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Sure if you think certain versions should be configured separately. At the very least I think we want the spring boot version to be consistent. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r213093319
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
This environment variable is sourced from /etc/default/metron which is
bound to the spring options setting in the REST tab in Ambari. I tested it in
full dev and the option is indeed added to alerts and config UI start commands.
---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r212275406
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
They're actually in different environments, so didn't feel the need to
change the convention. These scripts take all their environment from Ambari
launchers in a subsequent PR, rather than the /etc/defaults approach, so there
is no actual collision. I can see an argument for changing them all for
clarity, but could also see the argument for consistency in the context of
potentially growing number of services like this.
---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212270804 --- Diff: metron-interface/metron-rest-client/bin/pom.xml --- @@ -0,0 +1,52 @@ + --- End diff -- Yes. Will remove. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212225176 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Which properties do you mean? I can see a case for the very general ones for sure, but the spring cloud and jwt versions could very much differ from service to service. Should those live in the metron-ui-host and metron-ui-security project respectively you think? ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212108451 --- Diff: metron-interface/metron-ui-host/pom.xml --- @@ -0,0 +1,142 @@ + + +http://maven.apache.org/POM/4.0.0; +xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; +xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + +metron-ui-host +jar + +Metron Generic UI Host +Spring Server to host config ui + + +org.apache.metron +metron-interface +0.5.1 + + + --- End diff -- Ideally these properties would be defined at the metron-interface module level. Do you mind changing in this PR? Should be an easy fix and we should do it eventually. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212107810 --- Diff: metron-interface/metron-rest-client/bin/pom.xml --- @@ -0,0 +1,52 @@ + --- End diff -- Is adding this file a mistake? ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/#discussion_r212107439
--- Diff:
metron-interface/metron-alerts-host/src/main/scripts/metron-alerts.sh ---
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+if [ -z "${METRON_SSL_PASSWORD}" ]; then
+echo "METRON_SSL_PASSWORD unset."
+fi
+
+METRON_VERSION=${project.version}
+METRON_HOME="${METRON_HOME:-/usr/metron/${METRON_VERSION}}"
+METRON_SYSCONFIG="${METRON_SYSCONFIG:-/etc/default/metron}"
+
+echo "METRON_VERSION=${METRON_VERSION}"
+echo "METRON_HOME=${METRON_HOME}"
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+
+if [ -f "$METRON_SYSCONFIG" ]; then
+echo "METRON_SYSCONFIG=${METRON_SYSCONFIG}"
+set -a
+. "$METRON_SYSCONFIG"
+fi
+
+echo "METRON_SPRING_PROFILES_ACTIVE=${METRON_SPRING_PROFILES_ACTIVE}"
+
+METRON_CONFIG_LOCATION="
--spring.config.location=classpath:/application.yml,$METRON_HOME/config/alerts_ui.yml"
+echo "METRON_CONFIG_LOCATION=${METRON_CONFIG_LOCATION}"
+METRON_SPRING_OPTIONS+=${METRON_CONFIG_LOCATION}
--- End diff --
I'm not sure if this is intentional but this property is also used by REST
(METRON_SPRING_OPTIONS). I can't see many cases where we would want this to be
shared.
---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212106486 --- Diff: metron-interface/metron-alerts-host/pom.xml --- @@ -0,0 +1,143 @@ + --- End diff -- I'm not even sure this is accurate according to our style guide but almost every xml file follows this. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212106080 --- Diff: metron-interface/metron-alerts-host/src/main/java/org/apache/metron/ui/AlertsApplication.java --- @@ -0,0 +1,31 @@ +/** --- End diff -- Our formatting for Java is generally 2 spaces for indents. I don't feel like it's much effort and might as well make it consistent. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/#discussion_r212104536 --- Diff: metron-interface/metron-alerts-host/pom.xml --- @@ -0,0 +1,143 @@ + --- End diff -- Our xml files generally follow the formatting style of 4 character indents. Reformatting should be easy and might as well do it now. ---
[GitHub] metron pull request #1111: METRON-1665 Host UIs on Spring Boot and add SSO a...
GitHub user simonellistonball opened a pull request: https://github.com/apache/metron/pull/ METRON-1665 Host UIs on Spring Boot and add SSO authentication This is part of a move towards using KnoxSSO, and LDAP as an authentication method for Metron UIs. The current UIs are served up by very lightweight nodejs applications, which serve the static bundle files produced by the angular build process, and proxies the rest api. This PR replaces this with Spring Boot applications, using the Spring Security stack. It also provides a shared project used as the base for the UI host application, which ensures consistent security across all UIs (metron-interface/metron-ui-security) an abstract web hosting application with zuul embedded for proxying rest to avoid cors problems (metron-interface/metron-ui-host) and two further concrete applications to do the actual hosting, the angular front-ends have been subtley changed to package themselves in jars which are included in the hosting applications, aiding the ease and integrity of the front end code on deployment. Some tests in the rest api have been changed slightly, and a fix has been applied to AlertsUiController to allow users access to delete their own settings. The metron-rest project has also been adapted to work with metron-ui-security, and so inherits the same authentication benefits. A follow on PR will address changes to the mpack to implement necessary config changes. Nodejs has been removed from the RPMs, and the deployment process, since it is no longer needed beyond the build stage. An additional feature is the filter provided to allow JWT cookies to be passed to a backend through zuul as Authorization Bearer headers, allowing simpler stateless interaction with the rest api for example, which opens the possibility of deploying each layer in a load balanced or HA mode using ribbon config. To test at this stage, you will need to manually configure and launch the metron-alerts and metron-config jars, which behave like init.d scripts. This can be done by launching the jars through the bin/ scripts, but would be a lot easier with the follow on work on mpack elements to configure them. For now, integration tests have been included for key function on the security layer and should be reviewed. ## Pull Request Checklist Thank you for submitting a contribution to Apache Metron. Please refer to our [Development Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235) for the complete guide to follow for contributions. Please refer also to our [Build Verification Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview) for complete smoke testing guides. In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). - [x] Does your PR title start with METRON- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? ### For code changes: - [x] Have you included steps to reproduce the behavior or problem that is being changed or addressed? - [x] Have you included steps or a guide to how the change may be verified and tested manually? - [x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via: ``` mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh ``` - [x] Have you written or updated unit tests and or integration tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via `site-book/target/site/index.html`: ``` cd site-book mvn site ``` Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. It is also recommended that
