Repository: aurora Updated Branches: refs/heads/master c7600955f -> 32cd1d56f
Document Aurora security features. Bugs closed: AURORA-817 Reviewed at https://reviews.apache.org/r/33537/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/32cd1d56 Tree: http://git-wip-us.apache.org/repos/asf/aurora/tree/32cd1d56 Diff: http://git-wip-us.apache.org/repos/asf/aurora/diff/32cd1d56 Branch: refs/heads/master Commit: 32cd1d56f82f8612f52f1c1817a72f4cbf33f3bd Parents: c760095 Author: Kevin Sweeney <[email protected]> Authored: Tue Apr 28 12:02:11 2015 -0700 Committer: Kevin Sweeney <[email protected]> Committed: Tue Apr 28 12:02:11 2015 -0700 ---------------------------------------------------------------------- docs/deploying-aurora-scheduler.md | 5 + docs/security.md | 278 ++++++++++++++++++++++++++++++++ 2 files changed, 283 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/aurora/blob/32cd1d56/docs/deploying-aurora-scheduler.md ---------------------------------------------------------------------- diff --git a/docs/deploying-aurora-scheduler.md b/docs/deploying-aurora-scheduler.md index 70d6e3d..7c09b3d 100644 --- a/docs/deploying-aurora-scheduler.md +++ b/docs/deploying-aurora-scheduler.md @@ -13,6 +13,7 @@ machines. This guide helps you get the scheduler set up and troubleshoot some c - [Storage Performance Considerations](#storage-performance-considerations) - [Network considerations](#network-considerations) - [Considerations for running jobs in docker](#considerations-for-running-jobs-in-docker) + - [Security Considerations](#security-considerations) - [Running Aurora](#running-aurora) - [Maintaining an Aurora Installation](#maintaining-an-aurora-installation) - [Monitoring](#monitoring) @@ -183,6 +184,10 @@ For example, monit can be configured with assuming you set `-http_port=8081`. +## Security Considerations + +See [security.md](security.md). + ### Maintaining an Aurora Installation ### Monitoring http://git-wip-us.apache.org/repos/asf/aurora/blob/32cd1d56/docs/security.md ---------------------------------------------------------------------- diff --git a/docs/security.md b/docs/security.md new file mode 100644 index 0000000..db2e924 --- /dev/null +++ b/docs/security.md @@ -0,0 +1,278 @@ +Aurora integrates with [Apache Shiro](http://shiro.apache.org/) to provide security +controls for its API. In addition to providing some useful features out of the box, Shiro +also allows Aurora cluster administrators to adapt the security system to their organizationâs +existing infrastructure. + +- [Enabling Security](#enabling-security) +- [Authentication](#authentication) + - [HTTP Basic Authentication](#http-basic-authentication) + - [Server Configuration](#server-configuration) + - [Client Configuration](#client-configuration) + - [HTTP SPNEGO Authentication (Kerberos)](#http-spnego-authentication-kerberos) + - [Server Configuration](#server-configuration-1) + - [Client Configuration](#client-configuration-1) +- [Authorization](#authorization) + - [Using an INI file to define security controls](#using-an-ini-file-to-define-security-controls) + - [Caveats](#caveats) +- [Implementing a Custom Realm](#implementing-a-custom-realm) + - [Packaging a realm module](#packaging-a-realm-module) +- [Known Issues](#known-issues) + +# Enabling Security + +By default, the Aurora scheduler will not authenticate users, and therefore is unsecured. +To enable security, the following scheduler command line argument must be set: + +``` +-enable_api_security=true +``` + +This instructs the scheduler to use *a* security mechanism, but it does not +define *which*. There are two major components of security: [authentication +and authorization](http://en.wikipedia.org/wiki/Authentication#Authorization). A cluster +administrator may choose the approach used for each, and may also implement custom mechanisms +for either. Later sections describe the options available. + +# Authentication + +The scheduler must be configured with instructions for how to process authentication +credentials at a minimum. There are currently two built-in authentication schemes - +[HTTP Basic Authentication](http://en.wikipedia.org/wiki/Basic_access_authentication), and +[SPNEGO](http://en.wikipedia.org/wiki/SPNEGO) (Kerberos). + +## HTTP Basic Authentication + +Basic Authentication is a very quick way to add *some* security. It is supported +by all major browsers and HTTP client libraries with minimal work. However, +before relying on Basic Authentication you should be aware of the [security +considerations](http://tools.ietf.org/html/rfc2617#section-4). + +### Server Configuration + +At a minimum you need to set 4 command-line flags on the scheduler: + +``` +-enable_api_security=true +-http_authentication_mechanism=BASIC +-shiro_realm_modules=org.apache.aurora.scheduler.http.api.security.IniShiroRealmModule +-shiro_ini_path=path/to/security.ini +``` + +And create a security.ini file like so: + +``` +[users] +sally = apple, admin + +[roles] +admin = * +``` + +The details of the security.ini file are explained below. Note that this file contains plaintext, +unhashed passwords. + +### Client Configuration + +To configure the client for HTTP Basic authentication, add an entry to ~/.netrc with your credentials + +``` +% cat ~/.netrc +# ... + +machine aurora.example.com +login sally +password apple + +# ... +``` + +No changes are required to `clusters.json`. + +## HTTP SPNEGO Authentication (Kerberos) + +### Server Configuration +At a minimum you need to set 6 command-line flags on the scheduler: + +``` +-enable_api_security=true +-http_authentication_mechanism=NEGOTIATE +-shiro_realm_modules=org.apache.aurora.scheduler.http.api.security.Kerberos5ShiroRealmModule,org.apache.aurora.scheduler.http.api.security.IniShiroRealmModule +-kerberos_server_principal=HTTP/[email protected] +-kerberos_server_keytab=path/to/aurora.example.com.keytab +-shiro_ini_path=path/to/security.ini +``` + +And create a security.ini file like so: + +``` +% cat path/to/security.ini +[users] +sally = _, admin + +[roles] +admin = * +``` + +What's going on here? First, Aurora must be configured to request Kerberos credentials when presented with an +unauthenticated request. This is achieved by setting + +``` +-enable_api_security=true +-http_authentication_mechanism=NEGOTIATE +``` + +Next, a Realm module must be configured to **authenticate** the current request using the Kerberos +credentials that were requested. Aurora ships with a realm module that can do this + +``` +-shiro_realm_modules=org.apache.aurora.scheduler.http.api.security.Kerberos5ShiroRealmModule[,...] +``` + +The Kerberos5Realm requires a keytab file and a server principal name. The principal name will usually +be in the form `HTTP/[email protected]`. + +``` +-kerberos_server_principal=HTTP/[email protected] +-kerberos_server_keytab=path/to/aurora.example.com.keytab +``` + +The Kerberos5 realm module is authentication-only. For scheduler security to work you must also +enable a realm module that provides an Authorizer implementation. For example, to do this using the +IniShiroRealmModule: + +``` +-shiro_realm_modules=org.apache.aurora.scheduler.http.api.security.Kerberos5ShiroRealmModule,org.apache.aurora.scheduler.http.api.security.IniShiroRealmModule +``` + +You can then configure authorization using a security.ini file as described below +(the password field is ignored). You must configure the realm module with the path to this file: + +``` +-shiro_ini_path=path/to/security.ini +``` + +### Client Configuration +To use Kerberos on the client-side you must build Kerberos-enabled client binaries. Do this with + +``` +./pants binary src/main/python/apache/aurora/client/cli:kaurora +./pants binary src/main/python/apache/aurora/admin:kaurora_admin +``` + +You must also configure each cluster where you've enabled Kerberos on the scheduler +to use Kerberos authentication. Do this by setting `auth_mechanism` to `KERBEROS` +in `clusters.json`. + +``` +% cat ~/.aurora/clusters.json +{ + "devcluser": { + "auth_mechanism": "KERBEROS", + ... + }, + ... +} +``` + +# Authorization +Given a means to authenticate the entity a client claims they are, we need to define what privileges they have. + +## Using an INI file to define security controls + +The simplest security configuration for Aurora is an INI file on the scheduler. For small +clusters, or clusters where the users and access controls change relatively infrequently, this is +likely the preferred approach. However you may want to avoid this approach if access permissions +are rapidly changing, or if your access control information already exists in another system. + +You can enable INI-based configuration with following scheduler command line arguments: + +``` +-enable_api_security=true +-http_authentication_mechanism=BASIC +-shiro_ini_path=path/to/security.ini +``` + +*note* As the argument name reveals, this is using Shiroâs +[IniRealm](http://shiro.apache.org/configuration.html#Configuration-INIConfiguration) behind +the scenes. + +The INI file will contain two sections - users and roles. Hereâs an example for what might +be in security.ini: + +``` +[users] +sally = apple, admin +jim = 123456, accounting +becky = letmein, webapp +larry = 654321,accounting +steve = password + +[roles] +admin = * +accounting = thrift.AuroraAdmin:setQuota +webapp = thrift.AuroraSchedulerManager:*:webapp +``` + +The users section defines user user credentials and the role(s) they are members of. These lines +are of the format `<user> = <password>[, <role>...]`. As you probably noticed, the passwords are +in plaintext and as a result read access to this file should be restricted. + +In this configuration, each user has different privileges for actions in the cluster because +of the roles they are a part of: + +* admin is granted all privileges +* accounting may adjust the amount of resource quota for any role +* webapp represents a collection of jobs that represents a service, and its members may create and modify any jobs owned by it + +### Caveats +You might find documentation on the Internet suggesting there are additional sections in `shiro.ini`, +like `[main]` and `[urls]`. These are not supported by Aurora as it uses a different mechanism to configure +those parts of Shiro. Think of Aurora's `security.ini` as a subset with only `[users]` and `[roles]` sections. + +# Implementing a Custom Realm + +Since Auroraâs security is backed by [Apache Shiro](https://shiro.apache.org), you can implement a +custom [Realm](http://shiro.apache.org/realm.html) to define organization-specific security behavior. + +In addition to using Shiro's standard APIs to implement a Realm you can link against Aurora to +access the type-safe Permissions Aurora uses. See the Javadoc for `org.apache.aurora.scheduler.spi` +for more information. + +## Packaging a realm module +Package your custom Realm(s) with a Guice module that exposes a `Set<Realm>` multibinding. + +```java +package com.example; + +import com.google.inject.AbstractModule; +import com.google.inject.multibindings.Multibinder; +import org.apache.shiro.realm.Realm; + +public class MyRealmModule extends AbstractModule { + @Override + public void configure() { + Realm myRealm = new MyRealm(); + + Multibinder.newSetBinder(binder(), Realm.class).addBinding().toInstance(myRealm); + } + + static class MyRealm implements Realm { + // Realm implementation. + } +} +``` + +# Known Issues + +While the APIs and SPIs we ship with are stable as of 0.8.0, we are aware of several incremental +improvements. Please follow, vote, or send patches. + +Relevant tickets: +* [AURORA-343](https://issues.apache.org/jira/browse/AURORA-343): HTTPS support +* [AURORA-1248](https://issues.apache.org/jira/browse/AURORA-1248): Client retries 4xx errors +* [AURORA-1279](https://issues.apache.org/jira/browse/AURORA-1279): Remove kerberos-specific build targets +* [AURORA-1290](https://issues.apache.org/jira/browse/AURORA-1290): Allow specifying a shorthand for "well-known" Module FQCNs +* [AURORA-1291](https://issues.apache.org/jira/browse/AURORA-1291): Collapse enable_api_security into http_authentication_mechanism +* [AURORA-1293](https://issues.apache.org/jira/browse/AURORA-1291): Consider defining a JSON format in place of INI +* [AURORA-1179](https://issues.apache.org/jira/browse/AURORA-1179): Supported hashed passwords in security.ini +* [AURORA-1295](https://issues.apache.org/jira/browse/AURORA-1295): Support security for the ReadOnlyScheduler service
