[
https://issues.apache.org/jira/browse/AMBARI-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandor Magyari updated AMBARI-13431:
------------------------------------
Description:
This task tracks the required changes in the handling code for the Blueprint
.json and the Cluster Creation Template .json files in order to allow the user
to request that a given cluster be Kerberized.
The most natural place for this configuration will likely be in the Cluster
Creation template, which would then allow a given Blueprint to be referenced
via secure and non-secure cluster creation requests.
Based on feedback from Product Management, a customer should be able to
indicate that a cluster is to be Kerberized in either the Blueprint .json or
the Cluster Creation template .json.
This feature should support enabling Kerberos at the level of the Blueprint or
the Cluster Creation template. In either JSON document, the user should be
able to indicate a security tag that looks like:
{code}
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
}
{code}
The "type" field in the new "security" map should be set to "KERBEROS" in order
to indicate that Kerberos should be supported.
The "kerberos_descriptor_reference" field in the "security" map could be used
to refer to an existing Kerberos descriptor that has been POST-ed to the Ambari
REST API.
If the user wishes to embed the Kerberos descriptor in the Blueprint or Cluster
Creation template, then the "kerberos_descriptor" field in the "security" map
should be set to the contents of that descriptor.
The "security" map could eventually also include other configuration items
pertaining to the security of a given cluster. While Kerberos is the initial
support being added, other security mechanisms may evolve over time, and we
should be able to use the same configuration structures in order to eventually
integrate with these technologies as well.
*Note: The user should typically only specify a "kerberos_descriptor_reference"
or a "kerberos_descriptor". If both are set, the Blueprint processor should
treat this as an error condition.*
This new JSON element should exist at the top-level of the Cluster Creation
Template and Blueprint documents.
The following example shows what a Cluster Creation template might look like in
this scenario:
{code}
{
"blueprint" : "blueprint-ha",
"default_password" : "default",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
},
"host_groups" :[
{
"name" : "host_group_1",
"hosts" : [
{
"fqdn" : "c6401.ambari.apache.org"
}
]
},
...
]
}
{code}
The following example shows what a Blueprint that requires Kerberos support
should look like:
{code}
{
"host_groups": [
{
"name": "master",
"configurations": [
...
],
"components": [
{
"name": "NAMENODE"
},
{
"name": "SECONDARY_NAMENODE"
},
{
"name": "RESOURCEMANAGER"
},
{
"name": "HISTORYSERVER"
},
{
"name": "APP_TIMELINE_SERVER"
},
{
"name": "ZOOKEEPER_SERVER"
}
],
"cardinality": "1"
},
...
],
"Blueprints": {
"blueprint_name": "multi-simple-yarn",
"stack_name": "HDP",
"stack_version": "2.2",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
}
}
}
{code}
In the example above, the "type" field is included in the "security" map
section of the Blueprint document, embedded within the "Blueprints" map. This
is the most natural place for the Blueprint itself, since it contains the
metadata that should be associated with the Blueprint deployment, outside of
the configuration and components.
h2. Priority Ordering
Since the Kerberos setting will be supported in either the Blueprint or the
Cluster Creation template, this new support will need to handle the cases where
the setting is chosen in both documents.
# If a security type of "KERBEROS" is not selected in a Blueprint, then the
Cluster Creation template used by override this setting by including "type" :
"KERBEROS" in the template. This allows us to support deploying a Blueprint in
both Kerberized and non-Kerberized environments. This implies that any
Kerberos-specific configuration would need to be included in the Cluster
Creation template, but this is already supported by the Blueprints
configuration overrides.
# If a security type of "KERBEROS" is selected, then the Cluster Creation
template should not be able to override this setting to less-secure mode. If
the Cluster Creation template is configured to use a different security
mechanism, (For example: "type" : "OFF"), then the Blueprints processor should
treat this as an error condition. If the Cluster creation template does not
specify a "security" tag, then the "security" setting in the Blueprint should
be honored. In general, we should allow overrides to a more-secure cluster,
and forbid overrides for a less-secure cluster.
h2. Blueprint Database Table Changes
These additions to the Blueprint .json and Cluster Creation Template .json
structure will likely require changes to the Blueprint entity database tables,
already defined in ambari-server.
This current task will encompass any Database table changes needed to make
these additions, and will also likely require some ambari-server Upgrade
handling. This will involve using the existing Ambari Upgrade utilities to
support moving from older Ambari installs to Ambari 2.2. The main work here
will be updating existing database tables to support the new structure.
h2. Backwards compatibility
Any Blueprints that worked in previous versions of Ambari (non-Kerberized)
should work as-is in Ambari 2.2, in order to preserve backwards compatibility.
This means that these new configuration tags must not be required in a
non-Kerberized environment.
h2. Blueprint Validation
The Blueprint validator should be updated to check on the value of the security
"type" field, when it is present. Once we determine the accepted set of
possible values ("OFF" and "KERBEROS", for now), the validator should check
this, and return a reasonable error to the REST client if an invalid value is
set.
The kerberos.json (either referenced or embedded) descriptor must be saved to
the cluster’s artifacts resource prior to Kerberization.
In order to setup Kerberos, KDC admin credentials must be available, users can
choose the store *type* when submitting KDC admin credentials inside Cluster
template:
{code:java}
"credentials" : [
{
"alias" : "kdc.admin.credential",
"principal" : "(PRINCIPAL)",
"key" : "(PASSWORD)",
"type" : "(temporary | persistent)"
}
]
{code}
They can either set type=PERSISTENT or TEMPORARY. Temporary admin credentials
are valid 90 minutes or until server restart. Should they want to add new hosts
after temporary credentials are expired they had to submit credentials using
credential API:
POST /api/v1/clusters/:clusterName/credentials/kdc.admin.credential
{code:java}
{
"Credential" : {
"principal" : "(PRINCIPAL)",
"key" : "(PASSWORD)",
"type" : "(persisted|temporary)"
}
}
{code}
was:
This task tracks the required changes in the handling code for the Blueprint
.json and the Cluster Creation Template .json files in order to allow the user
to request that a given cluster be Kerberized.
The most natural place for this configuration will likely be in the Cluster
Creation template, which would then allow a given Blueprint to be referenced
via secure and non-secure cluster creation requests.
Based on feedback from Product Management, a customer should be able to
indicate that a cluster is to be Kerberized in either the Blueprint .json or
the Cluster Creation template .json.
This feature should support enabling Kerberos at the level of the Blueprint or
the Cluster Creation template. In either JSON document, the user should be
able to indicate a security tag that looks like:
{code}
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
}
{code}
The "type" field in the new "security" map should be set to "KERBEROS" in order
to indicate that Kerberos should be supported.
The "kerberos_descriptor_reference" field in the "security" map could be used
to refer to an existing Kerberos descriptor that has been POST-ed to the Ambari
REST API.
If the user wishes to embed the Kerberos descriptor in the Blueprint or Cluster
Creation template, then the "kerberos_descriptor" field in the "security" map
should be set to the contents of that descriptor.
The "security" map could eventually also include other configuration items
pertaining to the security of a given cluster. While Kerberos is the initial
support being added, other security mechanisms may evolve over time, and we
should be able to use the same configuration structures in order to eventually
integrate with these technologies as well.
*Note: The user should typically only specify a "kerberos_descriptor_reference"
or a "kerberos_descriptor". If both are set, the Blueprint processor should
treat this as an error condition.*
This new JSON element should exist at the top-level of the Cluster Creation
Template and Blueprint documents.
The following example shows what a Cluster Creation template might look like in
this scenario:
{code}
{
"blueprint" : "blueprint-ha",
"default_password" : "default",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
},
"host_groups" :[
{
"name" : "host_group_1",
"hosts" : [
{
"fqdn" : "c6401.ambari.apache.org"
}
]
},
...
]
}
{code}
The following example shows what a Blueprint that requires Kerberos support
should look like:
{code}
{
"host_groups": [
{
"name": "master",
"configurations": [
...
],
"components": [
{
"name": "NAMENODE"
},
{
"name": "SECONDARY_NAMENODE"
},
{
"name": "RESOURCEMANAGER"
},
{
"name": "HISTORYSERVER"
},
{
"name": "APP_TIMELINE_SERVER"
},
{
"name": "ZOOKEEPER_SERVER"
}
],
"cardinality": "1"
},
...
],
"Blueprints": {
"blueprint_name": "multi-simple-yarn",
"stack_name": "HDP",
"stack_version": "2.2",
"security" : {
"type" : "KERBEROS",
"kerberos_descriptor_reference" : "kd1",
"kerberos_descriptor" : {
...
}
}
}
}
{code}
In the example above, the "type" field is included in the "security" map
section of the Blueprint document, embedded within the "Blueprints" map. This
is the most natural place for the Blueprint itself, since it contains the
metadata that should be associated with the Blueprint deployment, outside of
the configuration and components.
h2. Priority Ordering
Since the Kerberos setting will be supported in either the Blueprint or the
Cluster Creation template, this new support will need to handle the cases where
the setting is chosen in both documents.
# If a security type of "KERBEROS" is not selected in a Blueprint, then the
Cluster Creation template used by override this setting by including "type" :
"KERBEROS" in the template. This allows us to support deploying a Blueprint in
both Kerberized and non-Kerberized environments. This implies that any
Kerberos-specific configuration would need to be included in the Cluster
Creation template, but this is already supported by the Blueprints
configuration overrides.
# If a security type of "KERBEROS" is selected, then the Cluster Creation
template should not be able to override this setting to less-secure mode. If
the Cluster Creation template is configured to use a different security
mechanism, (For example: "type" : "OFF"), then the Blueprints processor should
treat this as an error condition. If the Cluster creation template does not
specify a "security" tag, then the "security" setting in the Blueprint should
be honored. In general, we should allow overrides to a more-secure cluster,
and forbid overrides for a less-secure cluster.
h2. Blueprint Database Table Changes
These additions to the Blueprint .json and Cluster Creation Template .json
structure will likely require changes to the Blueprint entity database tables,
already defined in ambari-server.
This current task will encompass any Database table changes needed to make
these additions, and will also likely require some ambari-server Upgrade
handling. This will involve using the existing Ambari Upgrade utilities to
support moving from older Ambari installs to Ambari 2.2. The main work here
will be updating existing database tables to support the new structure.
h2. Backwards compatibility
Any Blueprints that worked in previous versions of Ambari (non-Kerberized)
should work as-is in Ambari 2.2, in order to preserve backwards compatibility.
This means that these new configuration tags must not be required in a
non-Kerberized environment.
h2. Blueprint Validation
The Blueprint validator should be updated to check on the value of the security
"type" field, when it is present. Once we determine the accepted set of
possible values ("OFF" and "KERBEROS", for now), the validator should check
this, and return a reasonable error to the REST client if an invalid value is
set.
The kerberos.json (either referenced or embedded) descriptor must be saved to
the cluster’s artifacts resource prior to Kerberization.
In order to setup Kerberos, KDC admin credentials must be available, users can
choose the store *type* when submitting KDC admin credentials inside Cluster
template:
{code:java}
"credentials" : [
{
"alias" : "kdc.admin.credential",
"principal" : "(PRINCIPAL)",
"key" : "(PASSWORD)",
"type" : "(temporary | persistent)"
}
], ....
{code}
They can either set type=PERSISTENT or TEMPORARY. Temporary admin credentials
are valid 90 minutes or until server restart. Should they want to add new hosts
after temporary credentials are expired they had to submit credentials using
credential API:
POST /api/v1/clusters/:clusterName/credentials/kdc.admin.credential
{code:java}
{
"Credential" : {
"principal" : "(PRINCIPAL)",
"key" : "(PASSWORD)",
"type" : "(persisted|temporary)"
}
}
{code}
> Blueprints: Configuration to select Kerberos
> --------------------------------------------
>
> Key: AMBARI-13431
> URL: https://issues.apache.org/jira/browse/AMBARI-13431
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Reporter: Sandor Magyari
> Assignee: Sandor Magyari
> Fix For: 2.1.3
>
> Attachments: AMBARI-13431-v6.patch, AMBARI-13431-v7-branch-2.1.patch,
> AMBARI-13431-v7.patch
>
>
> This task tracks the required changes in the handling code for the Blueprint
> .json and the Cluster Creation Template .json files in order to allow the
> user to request that a given cluster be Kerberized.
> The most natural place for this configuration will likely be in the Cluster
> Creation template, which would then allow a given Blueprint to be referenced
> via secure and non-secure cluster creation requests.
> Based on feedback from Product Management, a customer should be able to
> indicate that a cluster is to be Kerberized in either the Blueprint .json or
> the Cluster Creation template .json.
> This feature should support enabling Kerberos at the level of the Blueprint
> or the Cluster Creation template. In either JSON document, the user should
> be able to indicate a security tag that looks like:
> {code}
> "security" : {
> "type" : "KERBEROS",
> "kerberos_descriptor_reference" : "kd1",
> "kerberos_descriptor" : {
> ...
> }
> }
> {code}
> The "type" field in the new "security" map should be set to "KERBEROS" in
> order to indicate that Kerberos should be supported.
> The "kerberos_descriptor_reference" field in the "security" map could be used
> to refer to an existing Kerberos descriptor that has been POST-ed to the
> Ambari REST API.
> If the user wishes to embed the Kerberos descriptor in the Blueprint or
> Cluster Creation template, then the "kerberos_descriptor" field in the
> "security" map should be set to the contents of that descriptor.
> The "security" map could eventually also include other configuration items
> pertaining to the security of a given cluster. While Kerberos is the initial
> support being added, other security mechanisms may evolve over time, and we
> should be able to use the same configuration structures in order to
> eventually integrate with these technologies as well.
> *Note: The user should typically only specify a
> "kerberos_descriptor_reference" or a "kerberos_descriptor". If both are set,
> the Blueprint processor should treat this as an error condition.*
> This new JSON element should exist at the top-level of the Cluster Creation
> Template and Blueprint documents.
> The following example shows what a Cluster Creation template might look like
> in this scenario:
> {code}
> {
> "blueprint" : "blueprint-ha",
> "default_password" : "default",
> "security" : {
> "type" : "KERBEROS",
> "kerberos_descriptor_reference" : "kd1",
> "kerberos_descriptor" : {
> ...
> }
> },
> "host_groups" :[
> {
> "name" : "host_group_1",
> "hosts" : [
> {
> "fqdn" : "c6401.ambari.apache.org"
> }
> ]
> },
> ...
> ]
> }
> {code}
> The following example shows what a Blueprint that requires Kerberos support
> should look like:
> {code}
> {
> "host_groups": [
> {
> "name": "master",
> "configurations": [
> ...
> ],
> "components": [
> {
> "name": "NAMENODE"
> },
> {
> "name": "SECONDARY_NAMENODE"
> },
> {
> "name": "RESOURCEMANAGER"
> },
> {
> "name": "HISTORYSERVER"
> },
> {
> "name": "APP_TIMELINE_SERVER"
> },
> {
> "name": "ZOOKEEPER_SERVER"
> }
> ],
> "cardinality": "1"
> },
> ...
> ],
> "Blueprints": {
> "blueprint_name": "multi-simple-yarn",
> "stack_name": "HDP",
> "stack_version": "2.2",
> "security" : {
> "type" : "KERBEROS",
> "kerberos_descriptor_reference" : "kd1",
> "kerberos_descriptor" : {
> ...
> }
> }
> }
> }
> {code}
> In the example above, the "type" field is included in the "security" map
> section of the Blueprint document, embedded within the "Blueprints" map.
> This is the most natural place for the Blueprint itself, since it contains
> the metadata that should be associated with the Blueprint deployment, outside
> of the configuration and components.
> h2. Priority Ordering
> Since the Kerberos setting will be supported in either the Blueprint or the
> Cluster Creation template, this new support will need to handle the cases
> where the setting is chosen in both documents.
> # If a security type of "KERBEROS" is not selected in a Blueprint, then the
> Cluster Creation template used by override this setting by including "type" :
> "KERBEROS" in the template. This allows us to support deploying a Blueprint
> in both Kerberized and non-Kerberized environments. This implies that any
> Kerberos-specific configuration would need to be included in the Cluster
> Creation template, but this is already supported by the Blueprints
> configuration overrides.
> # If a security type of "KERBEROS" is selected, then the Cluster Creation
> template should not be able to override this setting to less-secure mode. If
> the Cluster Creation template is configured to use a different security
> mechanism, (For example: "type" : "OFF"), then the Blueprints processor
> should treat this as an error condition. If the Cluster creation template
> does not specify a "security" tag, then the "security" setting in the
> Blueprint should be honored. In general, we should allow overrides to a
> more-secure cluster, and forbid overrides for a less-secure cluster.
> h2. Blueprint Database Table Changes
> These additions to the Blueprint .json and Cluster Creation Template .json
> structure will likely require changes to the Blueprint entity database
> tables, already defined in ambari-server.
> This current task will encompass any Database table changes needed to make
> these additions, and will also likely require some ambari-server Upgrade
> handling. This will involve using the existing Ambari Upgrade utilities to
> support moving from older Ambari installs to Ambari 2.2. The main work here
> will be updating existing database tables to support the new structure.
> h2. Backwards compatibility
> Any Blueprints that worked in previous versions of Ambari (non-Kerberized)
> should work as-is in Ambari 2.2, in order to preserve backwards
> compatibility. This means that these new configuration tags must not be
> required in a non-Kerberized environment.
> h2. Blueprint Validation
> The Blueprint validator should be updated to check on the value of the
> security "type" field, when it is present. Once we determine the accepted
> set of possible values ("OFF" and "KERBEROS", for now), the validator should
> check this, and return a reasonable error to the REST client if an invalid
> value is set.
> The kerberos.json (either referenced or embedded) descriptor must be saved to
> the cluster’s artifacts resource prior to Kerberization.
> In order to setup Kerberos, KDC admin credentials must be available, users
> can choose the store *type* when submitting KDC admin credentials inside
> Cluster template:
> {code:java}
> "credentials" : [
> {
> "alias" : "kdc.admin.credential",
> "principal" : "(PRINCIPAL)",
> "key" : "(PASSWORD)",
> "type" : "(temporary | persistent)"
> }
> ]
> {code}
> They can either set type=PERSISTENT or TEMPORARY. Temporary admin credentials
> are valid 90 minutes or until server restart. Should they want to add new
> hosts after temporary credentials are expired they had to submit credentials
> using credential API:
> POST /api/v1/clusters/:clusterName/credentials/kdc.admin.credential
> {code:java}
> {
> "Credential" : {
> "principal" : "(PRINCIPAL)",
> "key" : "(PASSWORD)",
> "type" : "(persisted|temporary)"
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
