----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63965/ -----------------------------------------------------------
Review request for Ambari, Jayush Luniya, Swapan Shridhar, and Vitalyi Brodetskyi. Bugs: AMBARI-22487 https://issues.apache.org/jira/browse/AMBARI-22487 Repository: ambari Description ------- Create new service config version POST api/v2/clusters/<cluster-name>/servicegroups/<service-group-name>/services/<service-name>/configurations/service_config_versions Diffs ----- ambari-server/src/main/java/org/apache/ambari/server/api/services/ConfigurationService.java 79128a2374 ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceConfigVersionService.java c6e320121f ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java b5bc813e9c ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 875d9b6f9f ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceConfigVersionRequest.java 29a7a421cd ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceConfigVersionResourceProvider.java cbc08ebdf4 Diff: https://reviews.apache.org/r/63965/diff/1/ Testing ------- mvn clean install -DskipTests=true -Dmaven.test.skip=true -X [INFO] Reactor Summary: [INFO] [INFO] Ambari Main ........................................ SUCCESS [ 20.681 s] [INFO] Apache Ambari Project POM .......................... SUCCESS [ 0.047 s] [INFO] Ambari Web ......................................... SUCCESS [ 59.888 s] [INFO] Ambari Views ....................................... SUCCESS [ 1.454 s] [INFO] Ambari Admin View .................................. SUCCESS [ 9.794 s] [INFO] ambari-utility ..................................... SUCCESS [ 4.461 s] [INFO] ambari-metrics ..................................... SUCCESS [ 1.585 s] [INFO] Ambari Metrics Common .............................. SUCCESS [ 5.107 s] [INFO] Ambari Metrics Hadoop Sink ......................... SUCCESS [ 3.697 s] [INFO] Ambari Metrics Flume Sink .......................... SUCCESS [ 3.057 s] [INFO] Ambari Metrics Kafka Sink .......................... SUCCESS [ 3.088 s] [INFO] Ambari Metrics Storm Sink .......................... SUCCESS [ 2.462 s] [INFO] Ambari Metrics Storm Sink (Legacy) ................. SUCCESS [ 2.196 s] [INFO] Ambari Metrics Collector ........................... SUCCESS [ 20.644 s] [INFO] Ambari Metrics Monitor ............................. SUCCESS [ 2.597 s] [INFO] Ambari Metrics Grafana ............................. SUCCESS [ 5.799 s] [INFO] Ambari Metrics Host Aggregator ..................... SUCCESS [ 5.548 s] [INFO] Ambari Metrics Assembly ............................ SUCCESS [01:48 min] [INFO] Ambari Service Advisor ............................. SUCCESS [ 0.614 s] [INFO] Ambari Server ...................................... SUCCESS [03:49 min] [INFO] Ambari Functional Tests ............................ SUCCESS [ 0.494 s] [INFO] Ambari Agent ....................................... SUCCESS [ 50.352 s] [INFO] Ambari Client ...................................... SUCCESS [ 0.100 s] [INFO] Ambari Python Client ............................... SUCCESS [ 0.896 s] [INFO] Ambari Groovy Client ............................... SUCCESS [ 1.600 s] [INFO] Ambari Shell ....................................... SUCCESS [ 0.101 s] [INFO] Ambari Python Shell ................................ SUCCESS [ 0.699 s] [INFO] Ambari Groovy Shell ................................ SUCCESS [ 0.756 s] [INFO] ambari-logsearch ................................... SUCCESS [ 3.196 s] [INFO] Ambari Logsearch Appender .......................... SUCCESS [ 0.303 s] [INFO] Ambari Logsearch Config Api ........................ SUCCESS [ 0.242 s] [INFO] Ambari Logsearch Config Zookeeper .................. SUCCESS [ 0.352 s] [INFO] Ambari Logsearch Log Feeder ........................ SUCCESS [ 4.504 s] [INFO] Ambari Logsearch Web ............................... SUCCESS [01:10 min] [INFO] Ambari Logsearch Server ............................ SUCCESS [ 8.842 s] [INFO] Ambari Logsearch Assembly .......................... SUCCESS [ 0.206 s] [INFO] Ambari Logsearch Integration Test .................. SUCCESS [ 0.299 s] [INFO] ambari-infra ....................................... SUCCESS [ 0.198 s] [INFO] Ambari Infra Solr Client ........................... SUCCESS [ 2.149 s] [INFO] Ambari Infra Solr Plugin ........................... SUCCESS [ 0.445 s] [INFO] Ambari Infra Manager ............................... SUCCESS [ 8.802 s] [INFO] Ambari Infra Solr Assembly ......................... SUCCESS [ 0.214 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 10:47 min [INFO] Finished at: 2017-11-20T13:48:32-08:00 [INFO] Final Memory: 411M/1854M [INFO] ------------------------------------------------------------------------ Create a new service config version and corresponding configurations - EG : Initial configurations for HDFS =========================================================================================================== POST http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions Request : ```json { "ServiceConfigVersion": { "service_config_version_note": "Initial configurations for HDFS", "stack_id": "HDP-3.0.0" }, "configurations": [{ "Config": { "type": "core-site", "tag": "version1", "properties": { "hadoop.proxyuser.hdfs.groups": "*", "hadoop.proxyuser.root.groups": "*", "hadoop.proxyuser.hdfs.hosts": "*", "hadoop.proxyuser.root.hosts": "mradha-1.openstacklocal", "fs.defaultFS": "hdfs://mradha-1.openstacklocal:8020", "fs.trash.interval": "360", "net.topology.script.file.name": "/etc/hadoop/conf/topology_script.py" }, "service_config_version_note": "Initial configurations for HDFS", "properties_attributes": { "final": { "fs.defaultFS": "true" }, "password": {}, "user": {}, "group": {}, "text": {}, "additional_user_property": {}, "not_managed_hdfs_path": {}, "value_from_property_file": {} } } }, { "Config": { "type": "hadoop-env", "tag": "version1", "properties": { "content": "abdef", "dtnode_heapsize": "1024m", "nfsgateway_heapsize": "1024", "hdfs_user": "hdfs", "proxyuser_group": "users" } } }, { "Config": { "type": "hadoop-metrics2.properties", "tag": "version1", "properties": { "content": "abcdde" } } }, { "Config": { "type": "hadoop-policy", "tag": "version1", "properties": { "security.admin.operations.protocol.acl": "hadoop", "security.client.datanode.protocol.acl": "*", "security.client.protocol.acl": "*", "security.datanode.protocol.acl": "*", "security.inter.datanode.protocol.acl": "*", "security.refresh.usertogroups.mappings.protocol.acl": "hadoop" } } }, { "Config": { "type": "hdfs-log4j", "tag": "version1", "properties": { "content": "xyzedfg", "hadoop_log_max_backup_size": "256", "hadoop_log_number_of_backup_files": "10", "hadoop_security_log_max_backup_size": "256", "hadoop_security_log_number_of_backup_files": "20" } } }, { "Config": { "type": "hdfs-logsearch-conf", "tag": "version1", "properties": { "component_mappings": "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "content": "some content info", "service_name": "HDFS" } } }, { "Config": { "type": "hdfs-site", "tag": "version1", "properties": { "dfs.block.access.token.enable": "true", "dfs.blockreport.initialDelay": "120", "dfs.blocksize": "134217728", "nfs.exports.allowed.hosts": "* rw", "nfs.file.dump.dir": "/tmp/.hdfs-nfs" }, "properties_attributes": { "final": { "dfs.datanode.data.dir": "true", "dfs.datanode.failed.volumes.tolerated": "true", "dfs.namenode.http-address": "true", "dfs.namenode.name.dir": "true", "dfs.support.append": "true", "dfs.webhdfs.enabled": "true" }, "password": {}, "user": {}, "group": {}, "text": {}, "additional_user_property": {}, "not_managed_hdfs_path": {}, "value_from_property_file": {} } } }, { "Config": { "type": "ranger-hdfs-audit", "tag": "version1", "properties": {} } }, { "Config": { "type": "ranger-hdfs-plugin-properties", "tag": "version1", "properties": {} } }, { "Config": { "type": "ranger-hdfs-policymgr-ssl", "tag": "version1", "properties": {} } }, { "Config": { "type": "ranger-hdfs-security", "tag": "version1", "properties": {} } }, { "Config": { "type": "ssl-client", "tag": "version1", "properties": { "ssl.client.keystore.location": "/etc/security/clientKeys/keystore.jks", "ssl.client.keystore.password": "bigdata", "ssl.client.keystore.type": "jks", "ssl.client.truststore.location": "/etc/security/clientKeys/all.jks", "ssl.client.truststore.password": "bigdata", "ssl.client.truststore.reload.interval": "10000", "ssl.client.truststore.type": "jks" } } }, { "Config": { "type": "ssl-server", "tag": "version1", "properties": { "ssl.server.keystore.keypassword": "bigdata", "ssl.server.keystore.location": "/etc/security/serverKeys/keystore.jks", "ssl.server.keystore.password": "bigdata", "ssl.server.keystore.type": "jks", "ssl.server.truststore.location": "/etc/security/serverKeys/all.jks", "ssl.server.truststore.password": "bigdata", "ssl.server.truststore.reload.interval": "10000", "ssl.server.truststore.type": "jks" } } } ] } ``` Response : ```json { "resources" : [ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/configurations/service_config_versions?service_name=HDFS&service_config_version=1", "configurations" : [ { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "core-site", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "fs.defaultFS" : "hdfs://mradha-1.openstacklocal:8020", "ha.failover-controller.active-standby-elector.zk.op.retries" : "120", "hadoop.security.authentication" : "simple", "ipc.server.tcpnodelay" : "true", }, "configAttributes" : { "final" : { "fs.defaultFS" : "true" } }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hadoop-env", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "proxyuser_group" : "users", "hadoop_heapsize" : "1024", "hadoop_root_logger" : "INFO,RFA", "dtnode_heapsize" : "1024m", "namenode_backup_dir" : "/tmp/upgrades", "namenode_opt_permsize" : "128m" }, "configAttributes" : { }, "propertiesTypes" : { "NOT_MANAGED_HDFS_PATH" : [ "hdfs_tmp_dir" ], "USER" : [ "hdfs_user" ], "KERBEROS_PRINCIPAL" : [ "hdfs_principal_name" ], "GROUP" : [ "proxyuser_group" ] } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hadoop-metrics2.properties", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "content" : "some content info" }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hadoop-policy", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "security.inter.datanode.protocol.acl" : "*", "security.refresh.usertogroups.mappings.protocol.acl" : "hadoop", "security.client.datanode.protocol.acl" : "*", "security.admin.operations.protocol.acl" : "hadoop", "security.datanode.protocol.acl" : "*", "security.inter.tracker.protocol.acl" : "*", "security.job.client.protocol.acl" : "*", "security.job.task.protocol.acl" : "*", "security.client.protocol.acl" : "*", "security.refresh.policy.protocol.acl" : "hadoop", "security.namenode.protocol.acl" : "*" }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hdfs-log4j", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "hadoop_security_log_max_backup_size" : "256", "hadoop_security_log_number_of_backup_files" : "20", "hadoop_log_number_of_backup_files" : "10", "content" : "some content info", "hadoop_log_max_backup_size" : "256" }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hdfs-logsearch-conf", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "component_mappings" : "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "service_name" : "HDFS", "content" : "somecontentinfo" }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "hdfs-site", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "dfs.replication" : "3", "dfs.namenode.audit.log.async" : "true", "dfs.namenode.checkpoint.dir" : "/hadoop/hdfs/namesecondary", "dfs.namenode.avoid.read.stale.datanode" : "true", "dfs.namenode.checkpoint.period" : "21600", "dfs.datanode.http.address" : "0.0.0.0:50075" }, "configAttributes" : { "final" : { "dfs.webhdfs.enabled" : "true", "dfs.support.append" : "true", "dfs.namenode.http-address" : "true", "dfs.namenode.name.dir" : "true", "dfs.datanode.failed.volumes.tolerated" : "true", "dfs.datanode.data.dir" : "true" } }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ranger-hdfs-audit", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ranger-hdfs-plugin-properties", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { }, "configAttributes" : { }, "propertiesTypes" : { "PASSWORD" : [ "REPOSITORY_CONFIG_PASSWORD" ] } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ranger-hdfs-policymgr-ssl", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { }, "configAttributes" : { }, "propertiesTypes" : { "PASSWORD" : [ "xasecure.policymgr.clientssl.truststore.password", "xasecure.policymgr.clientssl.keystore.password" ] } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ranger-hdfs-security", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { }, "configAttributes" : { }, "propertiesTypes" : { } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ssl-client", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "ssl.client.truststore.reload.interval" : "10000", "ssl.client.keystore.location" : "/etc/security/clientKeys/keystore.jks", "ssl.client.truststore.password" : "SECRET:ssl-client:1:ssl.client.truststore.password", "ssl.client.truststore.type" : "jks", "ssl.client.truststore.location" : "/etc/security/clientKeys/all.jks", "ssl.client.keystore.password" : "SECRET:ssl-client:1:ssl.client.keystore.password", "ssl.client.keystore.type" : "jks" }, "configAttributes" : { }, "propertiesTypes" : { "PASSWORD" : [ "ssl.client.truststore.password", "ssl.client.keystore.password" ] } }, { "clusterName" : "cl1", "stackId" : { "stackName" : "HDP", "stackVersion" : "3.0.0", "stackId" : "HDP-3.0.0" }, "type" : "ssl-server", "versionTag" : "version1", "version" : 1, "serviceId" : 2, "serviceGroupId" : 2, "serviceConfigVersions" : null, "configs" : { "ssl.server.truststore.type" : "jks", "ssl.server.truststore.password" : "SECRET:ssl-server:1:ssl.server.truststore.password", "ssl.server.keystore.keypassword" : "SECRET:ssl-server:1:ssl.server.keystore.keypassword", "ssl.server.truststore.reload.interval" : "10000", "ssl.server.truststore.location" : "/etc/security/serverKeys/all.jks", "ssl.server.keystore.type" : "jks", "ssl.server.keystore.location" : "/etc/security/serverKeys/keystore.jks", "ssl.server.keystore.password" : "SECRET:ssl-server:1:ssl.server.keystore.password" }, "configAttributes" : { }, "propertiesTypes" : { "PASSWORD" : [ "ssl.server.truststore.password", "ssl.server.keystore.keypassword", "ssl.server.keystore.password" ] } } ], "group_id" : -1, "group_name" : "Default", "service_config_version" : 1, "service_group_id" : 2, "service_group_name" : "test-hdp", "service_id" : 2, "service_name" : "HDFS", "ServiceConfigVersion" : { "service_config_version_note" : "Initial configurations for HDFS" } } ] } ``` Change a particular config. EG: Change HDFS log4j maxnumber of backup files ============================================================================ POST http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions Request : ```json { "ServiceConfigVersion": { "service_config_version_note": "HDFS number of log backup files from 10 to 12", "stack_id": "HDP-3.0.0" }, "configurations": [{ "Config": { "type": "hdfs-log4j", "tag": "version2", "properties": { "content": "somecontent", "hadoop_log_max_backup_size": "256", "hadoop_log_number_of_backup_files": "12", "hadoop_security_log_max_backup_size": "256", "hadoop_security_log_number_of_backup_files": "20" } } } ] } ``` Revert back to an old config version. EG: Initial config version ================================================================== POST http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions Request : ```json { "service_config_version": 1 } ``` Fetch all the service config versions for a service. EG: HDFS ================================================================== GET http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions Response : ```json { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions", "items" : [ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/configurations/service_config_versions?service_name=HDFS&service_config_version=1", "cluster_name" : "cl1", "configurations" : [ { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "core-site", "tag" : "version1", "version" : 1, "properties" : { "fs.defaultFS" : "hdfs://mradha-1.openstacklocal:8020", "fs.trash.interval" : "360", "ipc.client.connection.maxidletime" : "30000", "ipc.client.idlethreshold" : "8000", "ipc.server.tcpnodelay" : "true", "mapreduce.jobtracker.webinterface.trusted" : "false", "net.topology.script.file.name" : "/etc/hadoop/conf/topology_script.py" }, "properties_attributes" : { "final" : { "fs.defaultFS" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-env", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "dtnode_heapsize" : "1024m", "hadoop_heapsize" : "1024", "namenode_opt_maxnewsize" : "128m", "namenode_opt_maxpermsize" : "256m", "namenode_opt_newsize" : "128m", "namenode_opt_permsize" : "128m", "nfsgateway_heapsize" : "1024", "proxyuser_group" : "users" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-metrics2.properties", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-policy", "tag" : "version1", "version" : 1, "properties" : { "security.admin.operations.protocol.acl" : "hadoop", "security.client.datanode.protocol.acl" : "*", "security.namenode.protocol.acl" : "*", "security.refresh.policy.protocol.acl" : "hadoop", "security.refresh.usertogroups.mappings.protocol.acl" : "hadoop" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-log4j", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "hadoop_log_max_backup_size" : "256", "hadoop_log_number_of_backup_files" : "10", "hadoop_security_log_max_backup_size" : "256", "hadoop_security_log_number_of_backup_files" : "20" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-logsearch-conf", "tag" : "version1", "version" : 1, "properties" : { "component_mappings" : "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "content" : "some content", "service_name" : "HDFS" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-site", "tag" : "version1", "version" : 1, "properties" : { "dfs.block.access.token.enable" : "true", "dfs.blockreport.initialDelay" : "120", "dfs.blocksize" : "134217728", "dfs.permissions.enabled" : "true", "dfs.permissions.superusergroup" : "hdfs", "dfs.replication" : "3", "nfs.file.dump.dir" : "/tmp/.hdfs-nfs" }, "properties_attributes" : { "final" : { "dfs.webhdfs.enabled" : "true", "dfs.support.append" : "true", "dfs.namenode.http-address" : "true", "dfs.namenode.name.dir" : "true", "dfs.datanode.failed.volumes.tolerated" : "true", "dfs.datanode.data.dir" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-audit", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-plugin-properties", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-policymgr-ssl", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-security", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-client", "tag" : "version1", "version" : 1, "properties" : { "ssl.client.keystore.location" : "/etc/security/clientKeys/keystore.jks", "ssl.client.keystore.password" : "SECRET:ssl-client:1:ssl.client.keystore.password", "ssl.client.truststore.reload.interval" : "10000", "ssl.client.truststore.type" : "jks" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-server", "tag" : "version1", "version" : 1, "properties" : { "ssl.server.keystore.keypassword" : "SECRET:ssl-server:1:ssl.server.keystore.keypassword", "ssl.server.keystore.location" : "/etc/security/serverKeys/keystore.jks", "ssl.server.keystore.password" : "SECRET:ssl-server:1:ssl.server.keystore.password", "ssl.server.keystore.type" : "jks", "ssl.server.truststore.location" : "/etc/security/serverKeys/all.jks", "ssl.server.truststore.password" : "SECRET:ssl-server:1:ssl.server.truststore.password", "ssl.server.truststore.reload.interval" : "10000", "ssl.server.truststore.type" : "jks" }, "properties_attributes" : { } } ], "createtime" : 1510956751144, "group_id" : -1, "group_name" : "Default", "hosts" : [ ], "is_cluster_compatible" : true, "is_current" : false, "service_config_version" : 1, "service_group_name" : "test-hdp", "service_name" : "HDFS", "user" : "admin", "ServiceConfigVersion" : { "service_config_version_note" : "Initial configurations for HDFS", "stack_id" : "HDP-3.0.0" } }, { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/configurations/service_config_versions?service_name=HDFS&service_config_version=2", "cluster_name" : "cl1", "configurations" : [ { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "core-site", "tag" : "version1", "version" : 1, "properties" : { "fs.defaultFS" : "hdfs://mradha-1.openstacklocal:8020", "fs.trash.interval" : "360", "ha.failover-controller.active-standby-elector.zk.op.retries" : "120", "ipc.server.tcpnodelay" : "true", "mapreduce.jobtracker.webinterface.trusted" : "false", "net.topology.script.file.name" : "/etc/hadoop/conf/topology_script.py" }, "properties_attributes" : { "final" : { "fs.defaultFS" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-env", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "dtnode_heapsize" : "1024m", "hadoop_heapsize" : "1024", "hadoop_pid_dir_prefix" : "/var/run/hadoop", "namenode_opt_newsize" : "128m", "namenode_opt_permsize" : "128m", "nfsgateway_heapsize" : "1024", "proxyuser_group" : "users" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-metrics2.properties", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-policy", "tag" : "version1", "version" : 1, "properties" : { "security.admin.operations.protocol.acl" : "hadoop", "security.client.datanode.protocol.acl" : "*", "security.job.task.protocol.acl" : "*", "security.namenode.protocol.acl" : "*", "security.refresh.policy.protocol.acl" : "hadoop", "security.refresh.usertogroups.mappings.protocol.acl" : "hadoop" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-logsearch-conf", "tag" : "version1", "version" : 1, "properties" : { "component_mappings" : "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "content" : "some content", "service_name" : "HDFS" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-site", "tag" : "version1", "version" : 1, "properties" : { "dfs.block.access.token.enable" : "true", "dfs.blockreport.initialDelay" : "120", "dfs.blocksize" : "134217728", "dfs.datanode.http.address" : "0.0.0.0:50075", "hadoop.caller.context.enabled" : "true", "nfs.exports.allowed.hosts" : "* rw", "nfs.file.dump.dir" : "/tmp/.hdfs-nfs" }, "properties_attributes" : { "final" : { "dfs.webhdfs.enabled" : "true", "dfs.support.append" : "true", "dfs.namenode.http-address" : "true", "dfs.namenode.name.dir" : "true", "dfs.datanode.failed.volumes.tolerated" : "true", "dfs.datanode.data.dir" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-audit", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-plugin-properties", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-policymgr-ssl", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-security", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-client", "tag" : "version1", "version" : 1, "properties" : { "ssl.client.keystore.location" : "/etc/security/clientKeys/keystore.jks", "ssl.client.keystore.password" : "SECRET:ssl-client:1:ssl.client.keystore.password", "ssl.client.keystore.type" : "jks", "ssl.client.truststore.location" : "/etc/security/clientKeys/all.jks", "ssl.client.truststore.password" : "SECRET:ssl-client:1:ssl.client.truststore.password", "ssl.client.truststore.reload.interval" : "10000", "ssl.client.truststore.type" : "jks" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-server", "tag" : "version1", "version" : 1, "properties" : { "ssl.server.keystore.keypassword" : "SECRET:ssl-server:1:ssl.server.keystore.keypassword", "ssl.server.keystore.location" : "/etc/security/serverKeys/keystore.jks", "ssl.server.truststore.type" : "jks" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-log4j", "tag" : "version2", "version" : 2, "properties" : { "content" : "some content", "hadoop_log_max_backup_size" : "256", "hadoop_log_number_of_backup_files" : "12", "hadoop_security_log_max_backup_size" : "256", "hadoop_security_log_number_of_backup_files" : "20" }, "properties_attributes" : { } } ], "createtime" : 1510958422471, "group_id" : -1, "group_name" : "Default", "hosts" : [ ], "is_cluster_compatible" : true, "is_current" : false, "service_config_version" : 2, "service_group_name" : "test-hdp", "service_name" : "HDFS", "user" : "admin", "ServiceConfigVersion" : { "service_config_version_note" : "HDFS number of log backup files from 10 to 12", "stack_id" : "HDP-3.0.0" } }, { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/configurations/service_config_versions?service_name=HDFS&service_config_version=3", "cluster_name" : "cl1", "configurations" : [ { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "core-site", "tag" : "version1", "version" : 1, "properties" : { "fs.defaultFS" : "hdfs://mradha-1.openstacklocal:8020", "fs.trash.interval" : "360", }, "properties_attributes" : { "final" : { "fs.defaultFS" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-env", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "dtnode_heapsize" : "1024m", "keyserver_host" : " ", "keyserver_port" : "", "namenode_backup_dir" : "/tmp/upgrades", "proxyuser_group" : "users" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-metrics2.properties", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-policy", "tag" : "version1", "version" : 1, "properties" : { "security.admin.operations.protocol.acl" : "hadoop", "security.job.client.protocol.acl" : "*", "security.job.task.protocol.acl" : "*", "security.namenode.protocol.acl" : "*", "security.refresh.policy.protocol.acl" : "hadoop", "security.refresh.usertogroups.mappings.protocol.acl" : "hadoop" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-log4j", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "hadoop_log_max_backup_size" : "256", "hadoop_log_number_of_backup_files" : "10", "hadoop_security_log_max_backup_size" : "256", "hadoop_security_log_number_of_backup_files" : "20" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-logsearch-conf", "tag" : "version1", "version" : 1, "properties" : { "component_mappings" : "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "content" : "soemcontent", "service_name" : "HDFS" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-site", "tag" : "version1", "version" : 1, "properties" : { "dfs.block.access.token.enable" : "true", "dfs.blockreport.initialDelay" : "120", "dfs.blocksize" : "134217728", "nfs.exports.allowed.hosts" : "* rw", "nfs.file.dump.dir" : "/tmp/.hdfs-nfs" }, "properties_attributes" : { "final" : { "dfs.webhdfs.enabled" : "true", "dfs.support.append" : "true", "dfs.namenode.http-address" : "true", "dfs.namenode.name.dir" : "true", "dfs.datanode.failed.volumes.tolerated" : "true", "dfs.datanode.data.dir" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-audit", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-plugin-properties", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-policymgr-ssl", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-security", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-client", "tag" : "version1", "version" : 1, "properties" : { "ssl.client.keystore.location" : "/etc/security/clientKeys/keystore.jks", "ssl.client.keystore.password" : "SECRET:ssl-client:1:ssl.client.keystore.password", "ssl.client.keystore.type" : "jks", "ssl.client.truststore.location" : "/etc/security/clientKeys/all.jks", "ssl.client.truststore.password" : "SECRET:ssl-client:1:ssl.client.truststore.password", "ssl.client.truststore.reload.interval" : "10000", "ssl.client.truststore.type" : "jks" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-server", "tag" : "version1", "version" : 1, "properties" : { "ssl.server.keystore.keypassword" : "SECRET:ssl-server:1:ssl.server.keystore.keypassword", "ssl.server.keystore.location" : "/etc/security/serverKeys/keystore.jks", }, "properties_attributes" : { } } ], "createtime" : 1510962642204, "group_id" : -1, "group_name" : "Default", "hosts" : [ ], "is_cluster_compatible" : true, "is_current" : true, "service_config_version" : 3, "service_group_name" : "test-hdp", "service_name" : "HDFS", "user" : "admin", "ServiceConfigVersion" : { "service_config_version_note" : null, "stack_id" : "HDP-3.0.0" } } ] } ``` Fetch the latest config version for a service. EG: HDFS ======================================================== GET http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions?is_current=true Response : ```json { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/servicegroups/test-hdp/services/HDFS/configurations/service_config_versions?is_current=true", "items" : [ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1/configurations/service_config_versions?service_name=HDFS&service_config_version=3", "cluster_name" : "cl1", "configurations" : [ { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "core-site", "tag" : "version1", "version" : 1, "properties" : { "fs.defaultFS" : "hdfs://mradha-1.openstacklocal:8020", "fs.trash.interval" : "360", "ha.failover-controller.active-standby-elector.zk.op.retries" : "120", "ipc.server.tcpnodelay" : "true", "mapreduce.jobtracker.webinterface.trusted" : "false", "net.topology.script.file.name" : "/etc/hadoop/conf/topology_script.py" }, "properties_attributes" : { "final" : { "fs.defaultFS" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-env", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "dtnode_heapsize" : "1024m", "namenode_opt_newsize" : "128m", "namenode_opt_permsize" : "128m", "nfsgateway_heapsize" : "1024", "proxyuser_group" : "users" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-metrics2.properties", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hadoop-policy", "tag" : "version1", "version" : 1, "properties" : { "security.admin.operations.protocol.acl" : "hadoop", "security.client.datanode.protocol.acl" : "*", "security.client.protocol.acl" : "*", "security.refresh.policy.protocol.acl" : "hadoop", "security.refresh.usertogroups.mappings.protocol.acl" : "hadoop" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-log4j", "tag" : "version1", "version" : 1, "properties" : { "content" : "some content", "hadoop_log_max_backup_size" : "256", "hadoop_log_number_of_backup_files" : "10", "hadoop_security_log_max_backup_size" : "256", "hadoop_security_log_number_of_backup_files" : "20" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-logsearch-conf", "tag" : "version1", "version" : 1, "properties" : { "component_mappings" : "NAMENODE:hdfs_namenode;DATANODE:hdfs_datanode;SECONDARY_NAMENODE:hdfs_secondarynamenode;JOURNALNODE:hdfs_journalnode;ZKFC:hdfs_zkfc;NFS_GATEWAY:hdfs_nfs3", "content" : "some content", "service_name" : "HDFS" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "hdfs-site", "tag" : "version1", "version" : 1, "properties" : { "dfs.block.access.token.enable" : "true", "dfs.blockreport.initialDelay" : "120", "dfs.blocksize" : "134217728", "dfs.client.read.shortcircuit" : "true", "dfs.namenode.safemode.threshold-pct" : "1", "dfs.namenode.secondary.http-address" : "mradha-2.openstacklocal:50090", "dfs.namenode.stale.datanode.interval" : "30000", "nfs.file.dump.dir" : "/tmp/.hdfs-nfs" }, "properties_attributes" : { "final" : { "dfs.webhdfs.enabled" : "true", "dfs.support.append" : "true", "dfs.namenode.http-address" : "true", "dfs.namenode.name.dir" : "true", "dfs.datanode.failed.volumes.tolerated" : "true", "dfs.datanode.data.dir" : "true" } } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-audit", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-plugin-properties", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-policymgr-ssl", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ranger-hdfs-security", "tag" : "version1", "version" : 1, "properties" : { }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-client", "tag" : "version1", "version" : 1, "properties" : { "ssl.client.keystore.location" : "/etc/security/clientKeys/keystore.jks", "ssl.client.keystore.password" : "SECRET:ssl-client:1:ssl.client.keystore.password", "ssl.client.keystore.type" : "jks", "ssl.client.truststore.location" : "/etc/security/clientKeys/all.jks", "ssl.client.truststore.password" : "SECRET:ssl-client:1:ssl.client.truststore.password", "ssl.client.truststore.reload.interval" : "10000", "ssl.client.truststore.type" : "jks" }, "properties_attributes" : { } }, { "Config" : { "cluster_name" : "cl1", "stack_id" : "HDP-3.0.0" }, "type" : "ssl-server", "tag" : "version1", "version" : 1, "properties" : { "ssl.server.keystore.keypassword" : "SECRET:ssl-server:1:ssl.server.keystore.keypassword", "ssl.server.keystore.location" : "/etc/security/serverKeys/keystore.jks", "ssl.server.keystore.password" : "SECRET:ssl-server:1:ssl.server.keystore.password", "ssl.server.keystore.type" : "jks", "ssl.server.truststore.location" : "/etc/security/serverKeys/all.jks", "ssl.server.truststore.password" : "SECRET:ssl-server:1:ssl.server.truststore.password", "ssl.server.truststore.reload.interval" : "10000", "ssl.server.truststore.type" : "jks" }, "properties_attributes" : { } } ], "createtime" : 1510962642204, "group_id" : -1, "group_name" : "Default", "hosts" : [ ], "is_cluster_compatible" : true, "is_current" : true, "service_config_version" : 3, "service_group_name" : "test-hdp", "service_name" : "HDFS", "user" : "admin", "ServiceConfigVersion" : { "service_config_version_note" : null, "stack_id" : "HDP-3.0.0" } } ] } ``` Trimmed down some config properties to reduce size. Thanks, Madhuvanthi Radhakrishnan