METRON-1844 Allow for LDAP to be used for authentication and roles (justinleet) closes apache/metron#1246
Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/fefbb376 Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/fefbb376 Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/fefbb376 Branch: refs/heads/master Commit: fefbb376fad0526116ed91035408bfeb72059d9c Parents: d0411f6 Author: justinjleet <[email protected]> Authored: Tue Oct 23 13:29:32 2018 -0400 Committer: justinjleet <[email protected]> Committed: Fri Nov 2 09:24:08 2018 -0400 ---------------------------------------------------------------------- dependencies_with_url.csv | 2 + metron-deployment/development/README.md | 34 +++++++ .../development/knox-demo-ldap.ldif | 101 +++++++++++++++++++ .../CURRENT/configuration/metron-rest-env.xml | 7 +- .../configuration/metron-security-env.xml | 57 ++++++++++- .../package/scripts/alerts_ui_commands.py | 1 - .../package/scripts/management_ui_commands.py | 1 - .../package/scripts/params/params_linux.py | 12 ++- .../CURRENT/package/scripts/rest_commands.py | 4 +- .../METRON/CURRENT/package/templates/metron.j2 | 2 + .../METRON/CURRENT/themes/metron_theme.json | 30 ++++++ metron-interface/metron-rest/README.md | 38 +++++++ .../src/main/config/rest_application.yml | 15 +-- .../metron/rest/config/WebSecurityConfig.java | 40 +++----- .../rest/controller/AlertsUIController.java | 1 - .../metron/rest/controller/UserController.java | 13 +++ .../src/main/resources/application-vagrant.yml | 3 +- .../metron-rest/src/main/scripts/metron-rest.sh | 23 +++++ 18 files changed, 337 insertions(+), 47 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/dependencies_with_url.csv ---------------------------------------------------------------------- diff --git a/dependencies_with_url.csv b/dependencies_with_url.csv index 2e1eedd..7bc0e9b 100644 --- a/dependencies_with_url.csv +++ b/dependencies_with_url.csv @@ -337,6 +337,8 @@ org.springframework:spring-web:jar:5.0.5.RELEASE:compile,The Apache Software Lic org.springframework:spring-webmvc:jar:3.2.6.RELEASE:compile,The Apache Software License, Version 2.0,https://github.com/SpringSource/spring-framework org.springframework:spring-webmvc:jar:4.3.3.RELEASE:compile,The Apache Software License, Version 2.0,https://github.com/spring-projects/spring-framework org.springframework:spring-webmvc:jar:5.0.5.RELEASE:compile,The Apache Software License, Version 2.0,https://github.com/spring-projects/spring-framework +org.springframework.ldap:spring-ldap-core:jar:2.3.2.RELEASE:compile,ASLv2,https://spring.io/projects/spring-ldap +org.springframework.security:spring-security-ldap:jar:5.1.1.RELEASE:compile,ASLv2,https://spring.io/projects/spring-security org.tukaani:xz:jar:1.0:compile,Public Domain,http://tukaani.org/xz/java.html org.xerial.snappy:snappy-java:jar:1.0.4.1:compile,The Apache Software License, Version 2.0,http://code.google.com/p/snappy-java/ org.xerial.snappy:snappy-java:jar:1.1.1.7:compile,The Apache Software License, Version 2.0,https://github.com/xerial/snappy-java http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/development/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/development/README.md b/metron-deployment/development/README.md index 2a04e5f..b86a5c4 100644 --- a/metron-deployment/development/README.md +++ b/metron-deployment/development/README.md @@ -27,3 +27,37 @@ This directory contains environments useful for Metron developers. These enviro ## Vagrant Cachier recommendations The development boxes are designed to be spun up and destroyed on a regular basis as part of the development cycle. In order to avoid the overhead of re-downloading many of the heavy platform dependencies, Vagrant can use the [vagrant-cachier](http://fgrehm.viewdocs.io/vagrant-cachier/) plugin to store package caches between builds. If the plugin has been installed to your vagrant it will be used, and packages will be cached in ~/.vagrant/cache. + +## Knox Demo LDAP + +The development environment can be set up to authenticate against Knox's demo LDAP. + +A couple notes +* A custom LDIF file is used to setup users. This is to get the roles and passwords setup correctly. +* The demo LDAP uses plaintext passwords with no encryption prefix (e.g. {SSHA}). +* You may need or want to shut down any or all of the topologies. This is optional, but clears some room + +To setup this up, start full dev. +* In Ambari, add the Knox service (Actions -> +Add Service). Accept all defaults and let it install. The configs that will be set how we need by default are: + * LDAP URL = ldap://localhost:33389 + * User dn pattern = uid={0},ou=people,dc=hadoop,dc=apache,dc=org + * LDAP user searchbase = ou=people,dc=hadoop,dc=apache,dc=org + * Group Search Base = ou=groups,dc=hadoop,dc=apache,dc=org + * Group Search Filter = member={0} + * User Base DN = uid=admin,ou=people,dc=hadoop,dc=apache,dc=org + * User Search Filter is empty + * User password attribute = userPassword + * LDAP group role attribute = cn + * Bind User = uid=admin,ou=people,dc=hadoop,dc=apache,dc=org + * LDAP Truststore is empty + * LDAP Truststore Password is empty + +* In the Knox configuration, go to "Advanced users-ldif". We have a custom ldif file "knox-demo-ldap.ldif" in "metron-deployment/development" that contains a customized variant of the users and groups defined here. Replace the default ldif configuration with the contents of "knox-demo-ldap.ldif" +* Start the Demo LDAP (In Knox, "Service Actions -> Start Demo LDAP) +* In Metron's configs, we're going to make two changes + * Set "LDAP Enabled" to "On" + * In Security, set "Bind user password" to match the admin user's password from the ldif file (admin-password). +* Restart the REST application + +Now, when you go to Swagger or the UIs, you should be able to give a user and password. +"admin" will have the roles ROLE_ADMIN and ROLE_USER, which can be verified via the "/whoami/roles" endpoint in Swagger. Similarly, there is a user "sam" that only has ROLE_USER. A third user, "tom" has neither role. http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/development/knox-demo-ldap.ldif ---------------------------------------------------------------------- diff --git a/metron-deployment/development/knox-demo-ldap.ldif b/metron-deployment/development/knox-demo-ldap.ldif new file mode 100644 index 0000000..3097a64 --- /dev/null +++ b/metron-deployment/development/knox-demo-ldap.ldif @@ -0,0 +1,101 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +version: 1 + +# Please replace with site specific values +dn: dc=hadoop,dc=apache,dc=org +objectclass: organization +objectclass: dcObject +o: Hadoop +dc: hadoop + +# Entry for a sample people container +# Please replace with site specific values +dn: ou=people,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:organizationalUnit +ou: people + +# Entry for a sample end user +# Please replace with site specific values +dn: uid=guest,ou=people,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:person +objectclass:organizationalPerson +objectclass:inetOrgPerson +cn: Guest +sn: User +uid: guest +userPassword:guest-password + + +# entry for sample user admin +dn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:person +objectclass:organizationalPerson +objectclass:inetOrgPerson +cn: Admin +sn: Admin +uid: admin +userPassword:admin-password + +# entry for sample user sam +dn: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:person +objectclass:organizationalPerson +objectclass:inetOrgPerson +cn: sam +sn: sam +uid: sam +userPassword:sam-password + +# entry for sample user tom +dn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:person +objectclass:organizationalPerson +objectclass:inetOrgPerson +cn: tom +sn: tom +uid: tom +userPassword:tom-password + +# create FIRST Level groups branch +dn: ou=groups,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass:organizationalUnit +ou: groups +description: generic groups branch + +# create the admin group under groups +dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass: groupofnames +cn: admin +description:admin group +member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org + +# create the user group under groups +dn: cn=user,ou=groups,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass: groupofnames +cn: user +description: user group +member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org +member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-rest-env.xml ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-rest-env.xml b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-rest-env.xml index ab2491b..55b880f 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-rest-env.xml +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-rest-env.xml @@ -35,9 +35,12 @@ </property> <property> <name>metron_spring_profiles_active</name> - <description>Active Spring profiles. 'jdbc' is the default profiler for authentication. 'ldap' can is also available.</description> + <description>Active Spring profiles. 'ldap' is used to enable authentication via LDAP.</description> <display-name>Active Spring profiles</display-name> - <value>jdbc</value> + <value/> + <value-attributes> + <empty-value-valid>true</empty-value-valid> + </value-attributes> </property> <property> <name>metron_jdbc_driver</name> http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-security-env.xml ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-security-env.xml b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-security-env.xml index b96653e..ab1fe6c 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-security-env.xml +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-security-env.xml @@ -19,9 +19,31 @@ --> <configuration supports_final="true"> <property> + <name>metron.ldap.enabled</name> + <display-name>LDAP Enabled</display-name> + <value>false</value> + <description>Enable LDAP for Authentication</description> + <value-attributes> + <type>value-list</type> + <entries> + <entry> + <value>true</value> + <label>On</label> + </entry> + <entry> + <value>false</value> + <label>Off</label> + </entry> + </entries> + <selection-cardinality>1</selection-cardinality> + <overridable>false</overridable> + </value-attributes> + <on-ambari-upgrade add="true"/> + </property> + <property> <name>metron.ldap.url</name> <display-name>LDAP URL</display-name> - <value>ldap://localhost:33369</value> + <value>ldap://localhost:33389</value> <description>LDAP Server URL</description> <value-attributes> <overridable>false</overridable> @@ -40,9 +62,9 @@ </property> <property> <name>metron.ldap.user.searchbase</name> - <display-name>Group Search Base</display-name> + <display-name>User Search Base</display-name> <value>ou=people,dc=hadoop,dc=apache,dc=org</value> - <description>LDAP group searchbase</description> + <description>LDAP user searchbase</description> <value-attributes> <overridable>false</overridable> <empty-value-valid>true</empty-value-valid> @@ -105,8 +127,9 @@ </property> <property> <name>metron.ldap.group.roleattribute</name> + <display-name>LDAP group role attribute</display-name> <value>cn</value> - <description>LDAP group role attribute</description> + <description>The LDAP group attribute to be used for determining roles</description> <value-attributes> <overridable>false</overridable> </value-attributes> @@ -116,7 +139,7 @@ <name>metron.ldap.bind.dn</name> <display-name>Bind User</display-name> <value>uid=admin,ou=people,dc=hadoop,dc=apache,dc=org</value> - <description>Full distinguished name (DN), including common name (CN), of an LDAP user account that has privileges to search for users. </description> + <description>Full distinguished name (DN), of an LDAP user account that has privileges to search for users. </description> <on-ambari-upgrade add="true"/> <value-attributes> <overridable>false</overridable> @@ -136,4 +159,28 @@ <on-ambari-upgrade add="true"/> </property> + <property> + <name>metron.ldap.ssl.truststore</name> + <display-name>LDAP Truststore</display-name> + <value></value> + <description>Path of truststore with SSL certs for LDAP</description> + <value-attributes> + <overridable>false</overridable> + <empty-value-valid>true</empty-value-valid> + </value-attributes> + <on-ambari-upgrade add="true"/> + </property> + <property> + <name>metron.ldap.ssl.truststore.password</name> + <display-name>LDAP Truststore Password</display-name> + <value></value> + <property-type>PASSWORD</property-type> + <description>Password for the truststore with SSL certs for LDAP</description> + <value-attributes> + <overridable>false</overridable> + <empty-value-valid>true</empty-value-valid> + </value-attributes> + <on-ambari-upgrade add="true"/> + </property> + </configuration> \ No newline at end of file http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/alerts_ui_commands.py ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/alerts_ui_commands.py b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/alerts_ui_commands.py index e173f8d..f837043 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/alerts_ui_commands.py +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/alerts_ui_commands.py @@ -23,7 +23,6 @@ from resource_management.core.resources.system import Execute, File import metron_service - # Wrap major operations and functionality in this class class AlertsUICommands: __params = None http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_commands.py ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_commands.py b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_commands.py index c918587..4e81df6 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_commands.py +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_commands.py @@ -22,7 +22,6 @@ from resource_management.core.logger import Logger from resource_management.core.resources.system import Execute, File from resource_management.core.exceptions import ExecutionFailed from resource_management.libraries.functions.get_user_call_output import get_user_call_output -from resource_management.libraries.functions.format import format import metron_service http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py index aef7f6d..75f68fc 100755 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/params_linux.py @@ -58,7 +58,14 @@ metron_management_ui_port = status_params.metron_management_ui_port metron_alerts_ui_host = status_params.metron_alerts_ui_host metron_alerts_ui_port = status_params.metron_alerts_ui_port metron_jvm_flags = config['configurations']['metron-rest-env']['metron_jvm_flags'] -metron_spring_profiles_active = config['configurations']['metron-rest-env']['metron_spring_profiles_active'] + +# Construct the profiles as a temp variable first. Only the first time it's set will carry through +metron_spring_profiles_temp = config['configurations']['metron-rest-env']['metron_spring_profiles_active'] +if config['configurations']['metron-security-env']['metron.ldap.enabled']: + metron_spring_profiles_active = metron_spring_profiles_temp + ',ldap' +else: + metron_spring_profiles_active = metron_spring_profiles_temp + metron_jdbc_driver = config['configurations']['metron-rest-env']['metron_jdbc_driver'] metron_jdbc_url = config['configurations']['metron-rest-env']['metron_jdbc_url'] metron_jdbc_username = config['configurations']['metron-rest-env']['metron_jdbc_username'] @@ -279,7 +286,8 @@ metron_ldap_user_searchfilter = config['configurations']['metron-security-env'][ metron_ldap_group_searchbase = config['configurations']['metron-security-env']['metron.ldap.group.searchbase'] metron_ldap_group_searchfilter = config['configurations']['metron-security-env']['metron.ldap.group.searchfilter'] metron_ldap_group_role = config['configurations']['metron-security-env']['metron.ldap.group.roleattribute'] -metron_ldap = metron_ldap_url != "" +metron_ldap_ssl_truststore = config['configurations']['metron-security-env']['metron.ldap.ssl.truststore'] +metron_ldap_ssl_truststore_password = config['configurations']['metron-security-env']['metron.ldap.ssl.truststore.password'] # Management UI metron_rest_host = default("/clusterHostInfo/metron_rest_hosts", [hostname])[0] http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py index 674b744..bab9129 100755 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py @@ -146,11 +146,11 @@ class RestCommands: "export HDP_VERSION={hdp_version};" "export METRON_RA_INDEXING_WRITER={ra_indexing_writer};" "export METRON_LDAP_PASSWORD={metron_ldap_password!p};" - "export METRON_LDAP_USER_PASSWORD={metron_ldap_user_password!p};" + "export METRON_LDAP_SSL_TRUSTSTORE_PASSWORD={metron_ldap_ssl_truststore_password!p};" "{metron_home}/bin/metron-rest.sh;" "unset METRON_JDBC_PASSWORD;" "unset METRON_LDAP_PASSWORD;" - "unset METRON_LDAP_USER_PASSWORD;" + "unset METRON_LDAP_SSL_TRUSTSTORE_PASSWORD;" )) Execute(cmd, http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/metron.j2 ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/metron.j2 b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/metron.j2 index 9ceaa09..08d4281 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/metron.j2 +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/templates/metron.j2 @@ -37,10 +37,12 @@ METRON_LDAP_USERDN="{{metron_ldap_userdn}}" METRON_LDAP_USER_PATTERN="{{metron_ldap_user_pattern}}" METRON_LDAP_USER_DNBASE="{{metron_ldap_user_dnbase}}" METRON_LDAP_USER_SEARCHBASE="{{metron_ldap_user_searchbase}}" +METRON_LDAP_USER_PASSWORD="{{metron_ldap_user_password}}" METRON_LDAP_USER_SEARCHFILTER="{{metron_ldap_user_searchfilter}}" METRON_LDAP_GROUP_SEARCHBASE="{{metron_ldap_group_searchbase}}" METRON_LDAP_GROUP_SEARCHFILTER="{{metron_ldap_group_searchfilter}}" METRON_LDAP_GROUP_ROLE="{{metron_ldap_group_role}}" +METRON_LDAP_SSL_TRUSTSTORE="{{metron_ldap_ssl_truststore}}" ZOOKEEPER="{{zookeeper_quorum}}" BROKERLIST="{{kafka_brokers}}" http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json index d3ef96a..1d7b6c5 100644 --- a/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json +++ b/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/themes/metron_theme.json @@ -885,6 +885,10 @@ "subsection-name": "subsection-alerts-ui" }, { + "config": "metron-security-env/metron.ldap.enabled", + "subsection-name": "subsection-security-ldap" + }, + { "config": "metron-security-env/metron.ldap.url", "subsection-name": "subsection-security-ldap" }, @@ -925,6 +929,14 @@ "subsection-name": "subsection-security-ldap" }, { + "config": "metron-security-env/metron.ldap.ssl.truststore", + "subsection-name": "subsection-security-ldap" + }, + { + "config": "metron-security-env/metron.ldap.ssl.truststore.password", + "subsection-name": "subsection-security-ldap" + }, + { "config": "metron-pcap-env/pcap_topology_workers", "subsection-name": "subsection-pcap" }, @@ -1608,6 +1620,12 @@ } }, { + "config": "metron-security-env/metron.ldap.enabled", + "widget": { + "type": "toggle" + } + }, + { "config": "metron-security-env/metron.ldap.url", "widget": { "type": "text-field" @@ -1626,6 +1644,18 @@ } }, { + "config": "metron-security-env/metron.ldap.ssl.truststore", + "widget": { + "type": "text-field" + } + }, + { + "config": "metron-security-env/metron.ldap.ssl.truststore.password", + "widget": { + "type": "password" + } + }, + { "config": "metron-security-env/metron.ldap.user.dnpattern", "widget": { "type": "text-field" http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/README.md ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/README.md b/metron-interface/metron-rest/README.md index 7f00cde..f85707f 100644 --- a/metron-interface/metron-rest/README.md +++ b/metron-interface/metron-rest/README.md @@ -194,6 +194,44 @@ METRON_PRINCIPAL_NAME="[email protected]" METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" ``` +### LDAP + +Metron REST can be configured to use LDAP for authentication and roles. Configuration can be performed via Ambari in the "Security" tab. + +Configuration will default to matching Knox's Demo LDAP for convenience. This should only be used for development purposes. Manual instructions for setting up demo LDAP and finalizing configuration (e.g. setting up the user LDIF file) can be found in the [Development README](../../metron-deployment/development/README.md#knox-demo-ldap). + +#### LDAPS +There is configuration to provide a path to a truststore with SSL certificates and provide a password. Users should import certificates as needed to appropriate truststores. An example of doing this is: +``` +keytool -import -alias <alias> -file <certificate> -keystore <keystore_file> -storepass <password> +``` + + +#### Roles +Roles used by Metron are ROLE_ADMIN and ROLE_USER. Metron will use a property in a group containing the appropriate role to construct this. + +For example, our ldif file could create this group: +``` +dn: cn=admin,ou=groups,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass: groupofnames +cn: admin +description:admin group +member: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org +``` + +If we are using "cn" as our role attribute, Metron will give the "admin" user the role "ROLE_ADMIN". + +Similarly, we could give a user "sam" ROLE_USER with the following group: +``` +dn: cn=user,ou=groups,dc=hadoop,dc=apache,dc=org +objectclass:top +objectclass: groupofnames +cn: user +description: user group +member: uid=sam,ou=people,dc=hadoop,dc=apache,dc=org +``` + ## Spring Profiles The REST application comes with a few [Spring Profiles](http://docs.spring.io/autorepo/docs/spring-boot/current/reference/html/boot-features-profiles.html) to aid in testing and development. http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/config/rest_application.yml ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/config/rest_application.yml b/metron-interface/metron-rest/src/main/config/rest_application.yml index 25851e2..7ab7b73 100644 --- a/metron-interface/metron-rest/src/main/config/rest_application.yml +++ b/metron-interface/metron-rest/src/main/config/rest_application.yml @@ -55,13 +55,14 @@ pcap: yarn.queue: ${PCAP_YARN_QUEUE} finalizer.threadpool.size: ${PCAP_FINALIZER_THREADPOOL_SIZE} -datasource: - driverClassName: ${METRON_JDBC_DRIVER} - url: ${METRON_JDBC_URL} - username: ${METRON_JDBC_USERNAME} - password: ${METRON_JDBC_PASSWORD} - platform: ${METRON_JDBC_PLATFORM} - continue-on-error: true +spring: + datasource: + driverClassName: ${METRON_JDBC_DRIVER} + url: ${METRON_JDBC_URL} + username: ${METRON_JDBC_USERNAME} + password: ${METRON_JDBC_PASSWORD} + platform: ${METRON_JDBC_PLATFORM} + continue-on-error: true ldap: provider: http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java index 1f4b490..7ca3a46 100644 --- a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java +++ b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/config/WebSecurityConfig.java @@ -36,7 +36,7 @@ import org.springframework.security.config.annotation.method.configuration.Enabl import org.springframework.security.config.annotation.web.builders.HttpSecurity; import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter; -import org.springframework.security.crypto.factory.PasswordEncoderFactories; +import org.springframework.security.crypto.password.LdapShaPasswordEncoder; import org.springframework.security.crypto.password.NoOpPasswordEncoder; import org.springframework.security.crypto.password.PasswordEncoder; import org.springframework.security.web.authentication.logout.HttpStatusReturningLogoutSuccessHandler; @@ -55,6 +55,9 @@ public class WebSecurityConfig extends WebSecurityConfigurerAdapter { @Autowired private Environment environment; + @Autowired + private DataSource dataSource; + @Value("${ldap.provider.url}") private String providerUrl; @Value("${ldap.provider.userdn}") @@ -108,13 +111,7 @@ public class WebSecurityConfig extends WebSecurityConfigurerAdapter { } @Autowired - private DataSource dataSource; - - @Autowired - private PasswordEncoder passwordEncoder; - - @Override - public void configure(AuthenticationManagerBuilder auth) throws Exception { + public void configureJdbc(AuthenticationManagerBuilder auth) throws Exception { // Note that we can switch profiles on the fly in Ambari. List<String> activeProfiles = Arrays.asList(environment.getActiveProfiles()); if (activeProfiles.contains(MetronRestConstants.LDAP_PROFILE)) { @@ -132,28 +129,23 @@ public class WebSecurityConfig extends WebSecurityConfigurerAdapter { .managerPassword(providerPassword) .and() .passwordCompare() - .passwordEncoder(passwordEncoder) + .passwordEncoder(new LdapShaPasswordEncoder()) .passwordAttribute(passwordAttribute); - } else if (!activeProfiles.contains(MetronRestConstants.LDAP_PROFILE) && - (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) || - activeProfiles.contains(MetronRestConstants.TEST_PROFILE))) { - LOG.debug("Setting up dev/test JDBC authentication."); - auth.jdbcAuthentication().dataSource(dataSource) - .passwordEncoder(passwordEncoder) - .withUser("user").password("{noop}password").roles(SECURITY_ROLE_USER).and() - .withUser("user1").password("{noop}password").roles(SECURITY_ROLE_USER).and() - .withUser("user2").password("{noop}password").roles(SECURITY_ROLE_USER).and() - .withUser("admin").password("{noop}password") - .roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN); + } else if (activeProfiles.contains(MetronRestConstants.DEV_PROFILE) || + activeProfiles.contains(MetronRestConstants.TEST_PROFILE)) { + auth.jdbcAuthentication() + .dataSource(dataSource) + .withUser("user").password("password").roles(SECURITY_ROLE_USER).and() + .withUser("user1").password("password").roles(SECURITY_ROLE_USER).and() + .withUser("user2").password("password").roles(SECURITY_ROLE_USER).and() + .withUser("admin").password("password").roles(SECURITY_ROLE_USER, SECURITY_ROLE_ADMIN); } else { - LOG.debug("Setting up JDBC authentication."); - // TODO what are we supposed to do here? - auth.jdbcAuthentication().dataSource(dataSource).passwordEncoder(NoOpPasswordEncoder.getInstance()); + auth.jdbcAuthentication().dataSource(dataSource); } } @Bean public PasswordEncoder passwordEncoder() { - return PasswordEncoderFactories.createDelegatingPasswordEncoder(); + return NoOpPasswordEncoder.getInstance(); } } http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/AlertsUIController.java ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/AlertsUIController.java b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/AlertsUIController.java index c3b125a..fe2968f 100644 --- a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/AlertsUIController.java +++ b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/AlertsUIController.java @@ -74,7 +74,6 @@ public class AlertsUIController { } } - // TODO need roles to make this work. No tests covering it, but it'll break. @Secured({"ROLE_" + SECURITY_ROLE_ADMIN}) @ApiOperation(value = "Retrieves all users' settings. Only users that are part of " + "the \"ROLE_ADMIN\" role are allowed to get all user settings.") http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/UserController.java ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/UserController.java b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/UserController.java index b5f7765..24c781c 100644 --- a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/UserController.java +++ b/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/UserController.java @@ -19,6 +19,11 @@ package org.apache.metron.rest.controller; import io.swagger.annotations.ApiOperation; import io.swagger.annotations.ApiResponse; +import java.util.List; +import java.util.stream.Collectors; +import org.springframework.security.access.annotation.Secured; +import org.springframework.security.core.context.SecurityContextHolder; +import org.springframework.security.core.userdetails.UserDetails; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RestController; @@ -35,4 +40,12 @@ public class UserController { public String user(Principal user) { return user.getName(); } + + @Secured("IS_AUTHENTICATED_FULLY") + @RequestMapping(path = "/whoami/roles", method = RequestMethod.GET) + public List<String> user() { + UserDetails userDetails = (UserDetails) SecurityContextHolder.getContext(). + getAuthentication().getPrincipal(); + return userDetails.getAuthorities().stream().map(ga -> ga.getAuthority()).collect(Collectors.toList()); + } } http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/resources/application-vagrant.yml ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/resources/application-vagrant.yml b/metron-interface/metron-rest/src/main/resources/application-vagrant.yml index 5f21add..05ea571 100644 --- a/metron-interface/metron-rest/src/main/resources/application-vagrant.yml +++ b/metron-interface/metron-rest/src/main/resources/application-vagrant.yml @@ -52,7 +52,6 @@ storm: randomaccess.script.path: /usr/metron/${metron.version}/bin/start_elasticsearch_topology.sh batch.script.path: /usr/metron/${metron.version}/bin/start_hdfs_topology.sh - kerberos: enabled: false principal: [email protected] @@ -62,7 +61,7 @@ ldap: provider: url: ldap://node1:33389 userdn: uid=admin,ou=people,dc=hadoop,dc=apache,dc=org - password: "{noop}admin-password" + password: "admin-password" user: dn.patterns: uid={0},ou=people,dc=hadoop,dc=apache,dc=org passwordAttribute: userPassword http://git-wip-us.apache.org/repos/asf/metron/blob/fefbb376/metron-interface/metron-rest/src/main/scripts/metron-rest.sh ---------------------------------------------------------------------- diff --git a/metron-interface/metron-rest/src/main/scripts/metron-rest.sh b/metron-interface/metron-rest/src/main/scripts/metron-rest.sh index 2c8c7e6..b454b8e 100644 --- a/metron-interface/metron-rest/src/main/scripts/metron-rest.sh +++ b/metron-interface/metron-rest/src/main/scripts/metron-rest.sh @@ -113,6 +113,24 @@ METRON_REST_CLASSPATH+=":${indexing_files[0]}" echo "METRON_REST_CLASSPATH=${METRON_REST_CLASSPATH}" +echo "METRON_JDBC_DRIVER=${METRON_JDBC_DRIVER}" +echo "METRON_JDBC_URL=${METRON_JDBC_URL}" +echo "METRON_JDBC_USERNAME=${METRON_JDBC_USERNAME}" +echo "METRON_JDBC_PLATFORM=${METRON_JDBC_PLATFORM}" + +echo "METRON_LDAP_URL=${METRON_LDAP_URL}" +echo "METRON_LDAP_USERDN=${METRON_LDAP_USERDN}" + +echo "METRON_LDAP_USER_PATTERN=${METRON_LDAP_USER_PATTERN}" +echo "METRON_LDAP_USER_PASSWORD=${METRON_LDAP_USER_PASSWORD}" +echo "METRON_LDAP_USER_SEARCHBASE=${METRON_LDAP_USER_SEARCHBASE}" +echo "METRON_LDAP_USER_SEARCHFILTER=${METRON_LDAP_USER_SEARCHFILTER}" + +echo "METRON_LDAP_GROUP_SEARCHBASE=${METRON_LDAP_GROUP_SEARCHBASE}" +echo "METRON_LDAP_GROUP_SEARCHFILTER=${METRON_LDAP_GROUP_SEARCHFILTER}" +echo "METRON_LDAP_GROUP_ROLE=${METRON_LDAP_GROUP_ROLE}" +echo "METRON_LDAP_SSL_TRUSTSTORE=${METRON_LDAP_SSL_TRUSTSTORE}" + #Use Solr daos if ra indexing writer set to Solr if [[ ${METRON_RA_INDEXING_WRITER} == "Solr" ]]; then METRON_INDEX_DAO=" --index.dao.impl=org.apache.metron.solr.dao.SolrDao,org.apache.metron.indexing.dao.HBaseDao" @@ -126,6 +144,11 @@ if [[ ${METRON_RA_INDEXING_WRITER} == "Solr" ]]; then METRON_SPRING_OPTIONS+=${METRON_WRITER_NAME} fi +if [ -n "${METRON_LDAP_SSL_TRUSTSTORE}" ]; then + METRON_JVMFLAGS+=" -Djavax.net.ssl.trustStore=${METRON_LDAP_SSL_TRUSTSTORE}" + METRON_JVMFLAGS+=" -Djavax.net.ssl.trustStorePassword=${METRON_LDAP_SSL_TRUSTSTORE_PASSWORD}" +fi + echo "Starting application" ${JAVA_HOME}/bin/java -Dhdp.version=${HDP_VERSION} ${METRON_JVMFLAGS} \ -cp ${METRON_REST_CLASSPATH} \
