Repository: aurora Updated Branches: refs/heads/master d8d1c8d68 -> 8a2fc4c97
Command line flag to change FrameworkInfo.name This commit introduces a new command line flag `-framework_name` and changes the value from the hardcoded 'TwitterScheduler' to 'aurora'. This change is non-intrusive as https://issues.apache.org/jira/browse/MESOS-2614 has been fixed. The new value will be shown on the UI and used for Mesos DNS. Bugs closed: AURORA-945 Reviewed at https://reviews.apache.org/r/46596/ Project: http://git-wip-us.apache.org/repos/asf/aurora/repo Commit: http://git-wip-us.apache.org/repos/asf/aurora/commit/8a2fc4c9 Tree: http://git-wip-us.apache.org/repos/asf/aurora/tree/8a2fc4c9 Diff: http://git-wip-us.apache.org/repos/asf/aurora/diff/8a2fc4c9 Branch: refs/heads/master Commit: 8a2fc4c97de740a62e8ed10ca14cfdc6f289044f Parents: d8d1c8d Author: Stephan Erb <[email protected]> Authored: Wed May 4 00:25:59 2016 +0200 Committer: Stephan Erb <[email protected]> Committed: Wed May 4 00:25:59 2016 +0200 ---------------------------------------------------------------------- RELEASE-NOTES.md | 11 ++++++++--- docs/features/service-discovery.md | 12 +++++------- docs/reference/scheduler-configuration.md | 18 ++++++++++++++---- .../mesos/CommandLineDriverSettingsModule.java | 15 ++++++++++----- .../CommandLineDriverSettingsModuleTest.java | 6 +++++- 5 files changed, 42 insertions(+), 20 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/aurora/blob/8a2fc4c9/RELEASE-NOTES.md ---------------------------------------------------------------------- diff --git a/RELEASE-NOTES.md b/RELEASE-NOTES.md index 7a37d0d..8d5cbed 100644 --- a/RELEASE-NOTES.md +++ b/RELEASE-NOTES.md @@ -3,20 +3,25 @@ ### New/updated: +- Upgraded Mesos to 0.27.2 - Added a new optional [Apache Curator](https://curator.apache.org/) backend for performing scheduler leader election. You can enable this with the new `-zk_use_curator` scheduler argument. -- Adding --nosetuid-health-checks flag to control whether the executor runs health checks as the job's - role's user. -- Upgraded Mesos to 0.27.2 +- Adding --nosetuid-health-checks flag to control whether the executor runs health checks as the + job's role's user. - New scheduler command line argument `-offer_filter_duration` to control the time after which we expect Mesos to re-offer unused resources. A short duration improves scheduling performance in smaller clusters, but might lead to resource starvation for other frameworks if you run multiple ones in your cluster. Uses the Mesos default of 5s. +- New scheduler command line option `-framework_name` to change the name used for registering + the Aurora framework with Mesos. The current default value is 'TwitterScheduler'. ### Deprecations and removals: - Deprecated `--restart-threshold` option in the `aurora job restart` command to match the job updater behavior. This option has no effect now and will be removed in the future release. +- Deprecated `-framework_name` default argument 'TwitterScheduler'. In a future release this + will change to 'aurora'. Please be aware that depending on your usage of Mesos, this will + be a backward incompatible change. For details, see MESOS-703. 0.13.0 ------ http://git-wip-us.apache.org/repos/asf/aurora/blob/8a2fc4c9/docs/features/service-discovery.md ---------------------------------------------------------------------- diff --git a/docs/features/service-discovery.md b/docs/features/service-discovery.md index f242730..36823c8 100644 --- a/docs/features/service-discovery.md +++ b/docs/features/service-discovery.md @@ -28,17 +28,15 @@ An example is using [Mesos-DNS](https://github.com/mesosphere/mesos-dns), which records. With current implementation, the example job with key `devcluster/vagrant/test/http-example` generates at least the following: -1. An A record for `http_example.test.vagrant.twitterscheduler.mesos` (which only includes IP address); +1. An A record for `http_example.test.vagrant.aurora.mesos` (which only includes IP address); 2. A [SRV record](https://en.wikipedia.org/wiki/SRV_record) for - `_http_example.test.vagrant._tcp.twitterscheduler.mesos`, which includes IP address and every port. This should only + `_http_example.test.vagrant._tcp.aurora.mesos`, which includes IP address and every port. This should only be used if the service has one port. -3. A SRV record `_{port-name}._http_example.test.vagrant._tcp.twitterscheduler.mesos` for each port name +3. A SRV record `_{port-name}._http_example.test.vagrant._tcp.aurora.mesos` for each port name defined. This should be used when the service has multiple ports. Things to note: 1. The domain part (".mesos" in above example) can be configured in [Mesos DNS](http://mesosphere.github.io/mesos-dns/docs/configuration-parameters.html); -2. The `twitterscheduler` part is the lower-case of framework name, which is not configurable right now (see - [TWITTER_SCHEDULER_NAME](https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java#L98)); -3. Right now, portmap and port aliases in announcer object are not reflected in DiscoveryInfo, therefore not visible in - Mesos DNS records either. This is because they are only resolved in thermos executors. \ No newline at end of file +2. Right now, portmap and port aliases in announcer object are not reflected in DiscoveryInfo, therefore not visible in + Mesos DNS records either. This is because they are only resolved in thermos executors. http://git-wip-us.apache.org/repos/asf/aurora/blob/8a2fc4c9/docs/reference/scheduler-configuration.md ---------------------------------------------------------------------- diff --git a/docs/reference/scheduler-configuration.md b/docs/reference/scheduler-configuration.md index d2262f7..5f898a8 100644 --- a/docs/reference/scheduler-configuration.md +++ b/docs/reference/scheduler-configuration.md @@ -16,6 +16,10 @@ Required flags: Directory to store backups under. Will be created if it does not exist. -cluster_name [not null] Name to identify the cluster being served. +-db_max_active_connection_count [must be > 0] + Max number of connections to use with database via MyBatis +-db_max_idle_connection_count [must be > 0] + Max number of idle connections to the database via MyBatis -framework_authentication_file Properties file which contains framework credentials to authenticate with Mesosmaster. Must contain the properties 'aurora_authentication_principal' and 'aurora_authentication_secret'. -mesos_master_address [not null] @@ -30,8 +34,6 @@ Required flags: Path to the thermos executor entry point. -tier_config [file must be readable] Configuration file defining supported task tiers, task traits and behaviors. --zk_digest_credentials - user:password to use when authenticating with ZooKeeper. -zk_endpoints [must have at least 1 item] Endpoint specification for the ZooKeeper servers. @@ -83,9 +85,11 @@ Optional flags: -flapping_task_threshold (default (5, mins)) A task that repeatedly runs for less than this time is considered to be flapping. -framework_announce_principal (default false) - When 'framework_authentication_file' flag is set, the FrameworkInfo registered with the mesos master will also contain the principal. This is necessary if you intend to use mesos authorization via mesos ACLs. The default will change in a future release. + When 'framework_authentication_file' flag is set, the FrameworkInfo registered with the mesos master will also contain the principal. This is necessary if you intend to use mesos authorization via mesos ACLs. The default will change in a future release. Changing this value is backwards incompatible. For details, see MESOS-703. -framework_failover_timeout (default (21, days)) Time after which a framework is considered deleted. SHOULD BE VERY HIGH. +-framework_name (default TwitterScheduler) + Name used to register the Aurora framework with Mesos. Changing this value can be backwards incompatible. For details, see MESOS-703. -global_container_mounts (default []) A comma separated list of mount points (in host:container form) to mount into all (non-mesos) containers. -history_max_per_job_threshold (default 100) @@ -154,6 +158,8 @@ Optional flags: The timeout for doing log appends and truncations. -native_log_zk_group_path A zookeeper node for use by the native log to track the master coordinator. +-offer_filter_duration (default (5, secs)) + Duration after which we expect Mesos to re-offer unused resources. A short duration improves scheduling performance in smaller clusters, but might lead to resource starvation for other frameworks if you run many frameworks in your cluster. -offer_hold_jitter_window (default (1, mins)) Maximum amount of random jitter to add to the offer hold time window. -offer_reservation_duration (default (3, mins)) @@ -180,7 +186,7 @@ Optional flags: If false, Docker tasks may run without an executor (EXPERIMENTAL) -shiro_ini_path Path to shiro.ini for authentication and authorization configuration. --shiro_realm_modules (default [org.apache.aurora.scheduler.app.MoreModules$1@2d3379b4]) +-shiro_realm_modules (default [org.apache.aurora.scheduler.app.MoreModules$1@13c9d689]) Guice modules for configuring Shiro Realms. -sla_non_prod_metrics (default []) Metric categories collected for non production tasks. @@ -216,9 +222,13 @@ Optional flags: URL prefix for job container stats. -zk_chroot_path chroot path to use for the ZooKeeper connections +-zk_digest_credentials + user:password to use when authenticating with ZooKeeper. -zk_in_proc (default false) Launches an embedded zookeeper server for local testing causing -zk_endpoints to be ignored if specified. -zk_session_timeout (default (4, secs)) The ZooKeeper session timeout. +-zk_use_curator (default false) + Uses Apache Curator as the zookeeper client; otherwise a copy of Twitter commons/zookeeper (the legacy library) is used. ------------------------------------------------------------------------- ``` http://git-wip-us.apache.org/repos/asf/aurora/blob/8a2fc4c9/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java b/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java index 5c6cdd2..93bc672 100644 --- a/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java +++ b/src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java @@ -72,9 +72,15 @@ public class CommandLineDriverSettingsModule extends AbstractModule { help = "When 'framework_authentication_file' flag is set, the FrameworkInfo " + "registered with the mesos master will also contain the principal. This is " + "necessary if you intend to use mesos authorization via mesos ACLs. " - + "The default will change in a future release.") + + "The default will change in a future release. Changing this value is backwards " + + "incompatible. For details, see MESOS-703.") private static final Arg<Boolean> FRAMEWORK_ANNOUNCE_PRINCIPAL = Arg.create(false); + @CmdLine(name = "framework_name", + help = "Name used to register the Aurora framework with Mesos. Changing this value can be " + + "backwards incompatible. For details, see MESOS-703.") + private static final Arg<String> FRAMEWORK_NAME = Arg.create("TwitterScheduler"); + @CmdLine(name = "executor_user", help = "User to start the executor. Defaults to \"root\". " + "Set this to an unprivileged user if the mesos master was started with " @@ -94,9 +100,6 @@ public class CommandLineDriverSettingsModule extends AbstractModule { + "resources in offer.") private static final Arg<String> MESOS_ROLE = Arg.create(); - // TODO(wfarner): Figure out a way to change this without risk of fallout (MESOS-703). - private static final String TWITTER_FRAMEWORK_NAME = "TwitterScheduler"; - @Override protected void configure() { Optional<Protos.Credential> credentials = getCredentials(); @@ -110,6 +113,7 @@ public class CommandLineDriverSettingsModule extends AbstractModule { MESOS_MASTER_ADDRESS.get(), credentials, buildFrameworkInfo( + FRAMEWORK_NAME.get(), EXECUTOR_USER.get(), principal, FRAMEWORK_FAILOVER_TIMEOUT.get(), @@ -143,6 +147,7 @@ public class CommandLineDriverSettingsModule extends AbstractModule { @VisibleForTesting static FrameworkInfo buildFrameworkInfo( + String frameworkName, String executorUser, Optional<String> principal, Amount<Long, Time> failoverTimeout, @@ -150,8 +155,8 @@ public class CommandLineDriverSettingsModule extends AbstractModule { Optional<String> role) { FrameworkInfo.Builder infoBuilder = FrameworkInfo.newBuilder() + .setName(frameworkName) .setUser(executorUser) - .setName(TWITTER_FRAMEWORK_NAME) // Require slave checkpointing. Assumes slaves have '--checkpoint=true' arg set. .setCheckpoint(true) .setFailoverTimeout(failoverTimeout.as(Time.SECONDS)); http://git-wip-us.apache.org/repos/asf/aurora/blob/8a2fc4c9/src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java ---------------------------------------------------------------------- diff --git a/src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java b/src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java index dc964b8..448d57a 100644 --- a/src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java +++ b/src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java @@ -73,6 +73,7 @@ public class CommandLineDriverSettingsModuleTest { @Test public void testFrameworkInfoNoRevocable() { Protos.FrameworkInfo info = CommandLineDriverSettingsModule.buildFrameworkInfo( + "aurora", "user", Optional.absent(), Amount.of(1L, Time.MINUTES), @@ -86,6 +87,7 @@ public class CommandLineDriverSettingsModuleTest { @Test public void testFrameworkInfoRevocable() { Protos.FrameworkInfo info = CommandLineDriverSettingsModule.buildFrameworkInfo( + "aurora", "user", Optional.absent(), Amount.of(1L, Time.MINUTES), @@ -100,6 +102,7 @@ public class CommandLineDriverSettingsModuleTest { @Test public void testFrameworkInfoNoRevocableWithAnnouncedPrincipal() { Protos.FrameworkInfo info = CommandLineDriverSettingsModule.buildFrameworkInfo( + "aurora", "user", Optional.of("auroraprincipal"), Amount.of(1L, Time.MINUTES), @@ -113,8 +116,9 @@ public class CommandLineDriverSettingsModuleTest { @Test public void testFrameworkInfoRevocableWithAnnouncedPrincipalAndRole() { Protos.FrameworkInfo info = CommandLineDriverSettingsModule.buildFrameworkInfo( + "aurora", "user", - Optional.of("auroraprincipal"), + Optional.of("auroraprincipal"), Amount.of(1L, Time.MINUTES), true, Optional.of(TEST_ROLE));
