Repository: ambari Updated Branches: refs/heads/branch-2.5 ce1ed6e1a -> 6b93dc182
AMBARI-20220 - Parameterize the Startup Web Server Timeout Default Value (jonathanhurley) Project: http://git-wip-us.apache.org/repos/asf/ambari/repo Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/6b93dc18 Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/6b93dc18 Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/6b93dc18 Branch: refs/heads/branch-2.5 Commit: 6b93dc18270a1cd1d7ef23335e62870d09e671a0 Parents: ce1ed6e Author: Jonathan Hurley <[email protected]> Authored: Mon Feb 27 20:16:05 2017 -0500 Committer: Jonathan Hurley <[email protected]> Committed: Wed Mar 1 16:13:44 2017 -0500 ---------------------------------------------------------------------- ambari-server/docs/configuration/index.md | 19 +++++++++----- .../server/configuration/Configuration.java | 18 ++++++++++---- .../python/ambari_server/serverConfiguration.py | 22 +++++++++++++++++ .../src/main/python/ambari_server_main.py | 24 +++++++++--------- .../src/test/python/TestAmbariServer.py | 26 ++++++++++++++++++-- 5 files changed, 84 insertions(+), 25 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/ambari/blob/6b93dc18/ambari-server/docs/configuration/index.md ---------------------------------------------------------------------- diff --git a/ambari-server/docs/configuration/index.md b/ambari-server/docs/configuration/index.md index e0a01dc..58eb7c7 100644 --- a/ambari-server/docs/configuration/index.md +++ b/ambari-server/docs/configuration/index.md @@ -44,6 +44,7 @@ The following are the properties which can be used to configure Ambari. | agent.check.remote.mounts | Determines whether the Ambari Agents will use the `df` or `df -l` command when checking disk mounts for capacity issues. Auto-mounted remote directories can cause long delays. |`false` | | agent.package.install.task.timeout | The time, in seconds, before package installation commands are killed. |`1800` | | agent.package.parallel.commands.limit | The maximum number of tasks which can run within a single operational request. If there are more tasks, then they will be broken up between multiple operations. |`100` | +| agent.service.check.task.timeout | The time, in seconds, before agent service check commands are killed. |`0` | | agent.ssl | Determines whether SSL is used to communicate between Ambari Server and Ambari Agents. |`true` | | agent.stack.retry.on_repo_unavailability | Determines whether agents should retrying installation commands when the repository is not available. This can prevent false installation errors with repositories that are sporadically inaccessible. |`false` | | agent.stack.retry.tries | The number of times an Ambari Agent should retry package installation when it fails due to a repository error. <br/><br/> This property is related to `agent.stack.retry.on_repo_unavailability`. |`5` | @@ -61,6 +62,7 @@ The following are the properties which can be used to configure Ambari. | ambari.post.user.creation.hook | The location of the post user creation hook on the ambari server hosting machine. |`/var/lib/ambari-server/resources/scripts/post-user-creation-hook.sh` | | ambari.post.user.creation.hook.enabled | Indicates whether the post user creation is enabled or not. By default is false. |`false` | | ambari.python.wrap | The name of the shell script used to wrap all invocations of Python by Ambari. |`ambari-python-wrap` | +| ambariserver.metrics.disable | Global disable flag for AmbariServer Metrics. |`false` | | anonymous.audit.name | The name of the user given to requests which are executed without any credentials. |`_anonymous` | | api.authenticated.user | The username of the default user assumed to be executing API calls. When set, authentication is not required in order to login to Ambari or use the REST APIs. | | | api.csrfPrevention.enabled | Determines whether Cross-Site Request Forgery attacks are prevented by looking for the `X-Requested-By` header. |`true` | @@ -144,6 +146,9 @@ The following are the properties which can be used to configure Ambari. | kerberos.operation.retry.timeout | The time to wait (in seconds) between failed kerberos operations retries. |`10` | | ldap.sync.username.collision.behavior | Determines how to handle username collision while updating from LDAP.<br/><br/>The following are examples of valid values:<ul><li>`skip`<li>`convert`</ul> |`convert` | | log4j.monitor.delay | Indicates the delay, in milliseconds, for the log4j monitor to check for changes |`300000` | +| logsearch.metadata.cache.expire.timeout | The time, in hours, that the Ambari Server will hold Log File metadata in its internal cache before making a request to the LogSearch Portal to get the latest metadata. |`24` | +| logsearch.portal.connect.timeout | The time, in milliseconds, that the Ambari Server will wait while attempting to connect to the LogSearch Portal service. |`5000` | +| logsearch.portal.read.timeout | The time, in milliseconds, that the Ambari Server will wait while attempting to read a response from the LogSearch Portal service. |`5000` | | metadata.path | The location on the Ambari Server where the stack resources exist.<br/><br/>The following are examples of valid values:<ul><li>`/var/lib/ambari-server/resources/stacks`</ul> | | | metrics.retrieval-service.cache.timeout | The amount of time, in minutes, that JMX and REST metrics retrieved directly can remain in the cache. |`30` | | metrics.retrieval-service.request.ttl | The number of seconds to wait between issuing JMX or REST metric requests to the same endpoint. This property is used to throttle requests to the same URL being made too close together<br/><br/> This property is related to `metrics.retrieval-service.request.ttl.enabled`. |`5` | @@ -166,7 +171,7 @@ The following are the properties which can be used to configure Ambari. | repo.validation.suffixes.ubuntu | The suffixes to use when validating Ubuntu repositories. |`/dists/%s/Release` | | resources.dir | The location on the Ambari Server where all resources exist, including common services, stacks, and scripts. |`/var/lib/ambari-server/resources/` | | rolling.upgrade.skip.packages.prefixes | A comma-separated list of packages which will be skipped during a stack upgrade. | | -| security.agent.hostname.validate | Determines whether the Ambari Agent host names should be validated against a regular expression to ensure that they are well-formed.<br><br>WARNING: By setting this value to false, host names will not be validated, allowing a possible security vulnerability as described in CVE-2014-3582. See https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Vulnerabilities for more information.|`true` | +| security.agent.hostname.validate | Determines whether the Ambari Agent host names should be validated against a regular expression to ensure that they are well-formed.<br><br>WARNING: By setting this value to false, host names will not be validated, allowing a possible security vulnerability as described in CVE-2014-3582. See https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Vulnerabilities for more information. |`true` | | security.master.key.location | The location on the Ambari Server of the master key file. This is the key to the master keystore. | | | security.master.keystore.location | The location on the Ambari Server of the master keystore file. | | | security.server.cert_name | The name of the file located in the `security.server.keys_dir` directory where certificates will be generated when Ambari uses the `openssl ca` command. |`ca.crt` | @@ -228,20 +233,22 @@ The following are the properties which can be used to configure Ambari. | server.jdbc.user.passwd | The password for the user when logging into the database. |`bigdata` | | server.locks.profiling | Enable the profiling of internal locks. |`false` | | server.metrics.retrieval-service.thread.priority | The priority of threads used by the service which retrieves JMX and REST metrics directly from their respective endpoints. |`5` | -| server.metrics.retrieval-service.threadpool.size.core | The core number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`8` | -| server.metrics.retrieval-service.threadpool.size.max | The maximum number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`16` | -| server.metrics.retrieval-service.threadpool.worker.size | The number of queued requests allowed for JMX and REST metrics before discarding old requests which have not been fullfilled. |`160` | +| server.metrics.retrieval-service.threadpool.size.core | The core number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`16` | +| server.metrics.retrieval-service.threadpool.size.max | The maximum number of threads used to retrieve JMX and REST metrics directly from their respective endpoints. |`32` | +| server.metrics.retrieval-service.threadpool.worker.size | The number of queued requests allowed for JMX and REST metrics before discarding old requests which have not been fullfilled. |`320` | | server.operations.retry-attempts | The number of retry attempts for failed API and blueprint operations. |`0` | | server.os_family | The operating system family for all hosts in the cluster. This is used when bootstrapping agents and when enabling Kerberos.<br/><br/>The following are examples of valid values:<ul><li>`redhat`<li>`ubuntu`</ul> | | | server.os_type | The operating system version for all hosts in the cluster. This is used when bootstrapping agents and when enabling Kerberos.<br/><br/>The following are examples of valid values:<ul><li>`6`<li>`7`</ul> | | | server.persistence.type | The type of database connection being used. Unless using an embedded PostgresSQL server, then this should be `remote`.<br/><br/>The following are examples of valid values:<ul><li>`local`<li>`remote`</ul> |`local` | | server.property-provider.threadpool.completion.timeout | The maximum time, in milliseconds, that federated requests for data can execute before being terminated. Increasing this value could result in degraded performanc from the REST APIs. |`5000` | -| server.property-provider.threadpool.size.core | The core number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`8` | -| server.property-provider.threadpool.size.max | The maximum number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`16` | +| server.property-provider.threadpool.size.core | The core number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`16` | +| server.property-provider.threadpool.size.max | The maximum number of threads that will be used to retrieve data from federated datasources, such as remote JMX endpoints. |`32` | | server.property-provider.threadpool.worker.size | The maximum size of pending federated datasource requests, such as those to JMX endpoints, which can be queued before rejecting new requests. |`2147483647` | +| server.script.threads | The number of threads that should be allocated to run external script. |`4` | | server.script.timeout | The time, in milliseconds, until an external script is killed. |`5000` | | server.stage.command.execution_type | How to execute commands in one stage |`STAGE` | | server.stages.parallel | Determines whether operations in different execution requests can be run concurrently. |`true` | +| server.startup.web.timeout | The time, in seconds, that the ambari-server Python script will wait for Jetty to startup before returning an error code. |`50` | | server.task.timeout | The time, in seconds, before a server-side operation is terminated. |`1200` | | server.timeline.metrics.cache.catchup.interval | The time, in milliseconds, that Ambari Metrics intervals should use when extending the boundaries of the original request.<br/><br/> This property is related to `server.timeline.metrics.cache.disabled`. |`300000` | | server.timeline.metrics.cache.connect.timeout.millis | The time, in milliseconds, to wait while attempting to connect to Ambari Metrics.<br/><br/> This property is related to `server.timeline.metrics.cache.disabled`. |`5000` | http://git-wip-us.apache.org/repos/asf/ambari/blob/6b93dc18/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java index e414aa1..27ac82c 100644 --- a/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java +++ b/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java @@ -2557,6 +2557,14 @@ public class Configuration { public static final ConfigurationProperty<Integer> LOGSEARCH_METADATA_CACHE_EXPIRE_TIMEOUT = new ConfigurationProperty<>( "logsearch.metadata.cache.expire.timeout", 24); + /** + * The time, in seconds, that the ambari-server Python script will wait for + * Jetty to startup before returning an error code. + */ + @Markdown(description = "The time, in seconds, that the ambari-server Python script will wait for Jetty to startup before returning an error code.") + public static final ConfigurationProperty<Integer> SERVER_STARTUP_WEB_TIMEOUT = new ConfigurationProperty<>( + "server.startup.web.timeout", 50); + private static final Logger LOG = LoggerFactory.getLogger( Configuration.class); @@ -2748,7 +2756,7 @@ public class Configuration { H2("h2"); private static final Map<String, DatabaseType> m_mappedTypes = - new HashMap<String, Configuration.DatabaseType>(5); + new HashMap<>(5); static { for (DatabaseType databaseType : EnumSet.allOf(DatabaseType.class)) { @@ -2827,12 +2835,12 @@ public class Configuration { public Configuration(Properties properties) { this.properties = properties; - agentConfigsMap = new HashMap<String, String>(); + agentConfigsMap = new HashMap<>(); agentConfigsMap.put(CHECK_REMOTE_MOUNTS.getKey(), getProperty(CHECK_REMOTE_MOUNTS)); agentConfigsMap.put(CHECK_MOUNTS_TIMEOUT.getKey(), getProperty(CHECK_MOUNTS_TIMEOUT)); agentConfigsMap.put(ENABLE_AUTO_AGENT_CACHE_UPDATE.getKey(), getProperty(ENABLE_AUTO_AGENT_CACHE_UPDATE)); - configsMap = new HashMap<String, String>(); + configsMap = new HashMap<>(); configsMap.putAll(agentConfigsMap); configsMap.put(AMBARI_PYTHON_WRAP.getKey(), getProperty(AMBARI_PYTHON_WRAP)); configsMap.put(SRVR_AGENT_HOSTNAME_VALIDATE.getKey(), getProperty(SRVR_AGENT_HOSTNAME_VALIDATE)); @@ -4090,7 +4098,7 @@ public class Configuration { public Map<String, String> getAmbariProperties() { Properties properties = readConfigFile(); - Map<String, String> ambariPropertiesMap = new HashMap<String, String>(); + Map<String, String> ambariPropertiesMap = new HashMap<>(); for(String key : properties.stringPropertyNames()) { ambariPropertiesMap.put(key, properties.getProperty(key)); @@ -5650,7 +5658,7 @@ public class Configuration { // Get and process the configured user type values to convert the comma-delimited string of // user types into a ordered (as found in the comma-delimited value) list of UserType values. String userTypes = getProperty(KERBEROS_AUTH_USER_TYPES); - List<UserType> orderedUserTypes = new ArrayList<UserType>(); + List<UserType> orderedUserTypes = new ArrayList<>(); String[] types = userTypes.split(","); for (String type : types) { http://git-wip-us.apache.org/repos/asf/ambari/blob/6b93dc18/ambari-server/src/main/python/ambari_server/serverConfiguration.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/python/ambari_server/serverConfiguration.py b/ambari-server/src/main/python/ambari_server/serverConfiguration.py index 078a941..3dd165b 100644 --- a/ambari-server/src/main/python/ambari_server/serverConfiguration.py +++ b/ambari-server/src/main/python/ambari_server/serverConfiguration.py @@ -181,6 +181,9 @@ VIEWS_DIR_PROPERTY = "views.dir" ACTIVE_INSTANCE_PROPERTY = "active.instance" +# web server startup timeout +WEB_SERVER_STARTUP_TIMEOUT = "server.startup.web.timeout" + #Common setup or upgrade message SETUP_OR_UPGRADE_MSG = "- If this is a new setup, then run the \"ambari-server setup\" command to create the user\n" \ "- If this is an upgrade of an existing setup, run the \"ambari-server upgrade\" command.\n" \ @@ -978,6 +981,25 @@ def remove_password_file(filename): return 0 +def get_web_server_startup_timeout(properties): + """ + Gets the time, in seconds, that the startup script should wait for the web server to bind to + the configured port. If this value is too low, then the startup script will return an + error code even though Ambari is actually starting up. + :param properties: + :return: The timeout value, in seconds. The default is 50. + """ + # get the timeout property and strip it if it exists + timeout = properties[WEB_SERVER_STARTUP_TIMEOUT] + timeout = None if timeout is None else timeout.strip() + + if timeout is None or timeout == "": + timeout = 50 + else: + timeout = int(timeout) + return timeout + + def get_original_master_key(properties, options = None): input = True masterKey = None http://git-wip-us.apache.org/repos/asf/ambari/blob/6b93dc18/ambari-server/src/main/python/ambari_server_main.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/main/python/ambari_server_main.py b/ambari-server/src/main/python/ambari_server_main.py index 66678b0..0cd19cc 100644 --- a/ambari-server/src/main/python/ambari_server_main.py +++ b/ambari-server/src/main/python/ambari_server_main.py @@ -23,22 +23,21 @@ import sys import logging from ambari_commons.exceptions import FatalException -from ambari_commons.logging_utils import get_debug_mode, print_warning_msg, print_info_msg, \ - set_debug_mode_from_options +from ambari_commons.logging_utils import get_debug_mode, print_warning_msg, print_info_msg, set_debug_mode_from_options from ambari_commons.os_check import OSConst from ambari_commons.os_family_impl import OsFamilyFuncImpl, OsFamilyImpl -from ambari_commons.os_utils import is_root, run_os_command +from ambari_commons.os_utils import is_root from ambari_server.ambariPath import AmbariPath from ambari_server.dbConfiguration import ensure_dbms_is_running, ensure_jdbc_driver_is_installed from ambari_server.serverConfiguration import configDefaults, find_jdk, get_ambari_properties, \ - get_conf_dir, get_is_persisted, get_is_secure, get_java_exe_path, get_original_master_key, read_ambari_user, \ - get_is_active_instance, update_properties, get_ambari_server_ui_port, \ - PID_NAME, SECURITY_KEY_ENV_VAR_NAME, SECURITY_MASTER_KEY_LOCATION, \ - SETUP_OR_UPGRADE_MSG, check_database_name_property, parse_properties_file, get_missing_properties + get_java_exe_path, read_ambari_user, \ + get_is_active_instance, update_properties, get_ambari_server_ui_port, PID_NAME, \ + check_database_name_property, parse_properties_file, get_missing_properties + +from ambari_server.serverConfiguration import get_web_server_startup_timeout from ambari_server.serverUtils import refresh_stack_hash from ambari_server.setupHttps import get_fqdn -from ambari_server.setupSecurity import generate_env, \ - ensure_can_start_under_current_user +from ambari_server.setupSecurity import generate_env, ensure_can_start_under_current_user from ambari_server.utils import check_reverse_lookup, save_pid, locate_file, locate_all_file_paths, looking_for_pid, \ save_main_pid_ex, check_exitcode, get_live_pids_count, wait_for_ui_start from ambari_server.serverClassPath import ServerClassPath @@ -104,7 +103,6 @@ SERVER_START_CMD_DEBUG_WINDOWS = "{0} " \ SERVER_START_TIMEOUT = 5 #seconds SERVER_START_RETRIES = 4 -WEB_UI_INIT_TIME = 50 #seconds SERVER_PING_TIMEOUT_WINDOWS = 5 SERVER_PING_ATTEMPTS_WINDOWS = 4 @@ -229,9 +227,11 @@ def wait_for_server_start(pidFile, scmStatus): exception = None if server_started: ambari_server_ui_port = get_ambari_server_ui_port(properties) - if not wait_for_ui_start(int(ambari_server_ui_port), WEB_UI_INIT_TIME): + web_server_startup_timeout = get_web_server_startup_timeout(properties) + + if not wait_for_ui_start(int(ambari_server_ui_port), web_server_startup_timeout): exception = FatalException(1, "Server not yet listening on http port " + ambari_server_ui_port + \ - " after " + str(WEB_UI_INIT_TIME) + " seconds. Exiting.") + " after " + str(web_server_startup_timeout) + " seconds. Exiting.") elif get_live_pids_count(pids) <= 0: exitcode = check_exitcode(os.path.join(configDefaults.PID_DIR, EXITCODE_NAME)) exception = FatalException(-1, AMBARI_SERVER_DIE_MSG.format(exitcode, configDefaults.SERVER_OUT_FILE)) http://git-wip-us.apache.org/repos/asf/ambari/blob/6b93dc18/ambari-server/src/test/python/TestAmbariServer.py ---------------------------------------------------------------------- diff --git a/ambari-server/src/test/python/TestAmbariServer.py b/ambari-server/src/test/python/TestAmbariServer.py index ed99533..529c0d8 100644 --- a/ambari-server/src/test/python/TestAmbariServer.py +++ b/ambari-server/src/test/python/TestAmbariServer.py @@ -4401,8 +4401,8 @@ class TestAmbariServer(TestCase): @patch("os.chown") @patch("ambari_server.setupSecurity.get_master_key_location") @patch("ambari_server.setupSecurity.save_master_key") - @patch("ambari_server_main.get_is_persisted") - @patch("ambari_server_main.get_is_secure") + @patch("ambari_server.setupSecurity.get_is_persisted") + @patch("ambari_server.setupSecurity.get_is_secure") @patch('os.chmod', autospec=True) @patch("ambari_server.serverConfiguration.write_property") @patch("ambari_server.serverConfiguration.get_validated_string_input") @@ -8590,6 +8590,28 @@ class TestAmbariServer(TestCase): self.assertTrue(is_server_runing_method.called) pass + + def test_web_server_startup_timeout(self): + from ambari_server.serverConfiguration import get_web_server_startup_timeout + from ambari_server.serverConfiguration import WEB_SERVER_STARTUP_TIMEOUT + + properties = Properties() + timeout = get_web_server_startup_timeout(properties) + self.assertEquals(50, timeout) + + properties.process_pair(WEB_SERVER_STARTUP_TIMEOUT, "") + timeout = get_web_server_startup_timeout(properties) + self.assertEquals(50, timeout) + + properties.process_pair(WEB_SERVER_STARTUP_TIMEOUT, "120") + timeout = get_web_server_startup_timeout(properties) + self.assertEquals(120, timeout) + + properties.process_pair(WEB_SERVER_STARTUP_TIMEOUT, "120 ") + timeout = get_web_server_startup_timeout(properties) + self.assertEquals(120, timeout) + + def _create_empty_options_mock(self): options = MagicMock() options.ldap_url = None
