Repository: mesos-site Updated Branches: refs/heads/asf-site 33d26065d -> e5950137b
Updated the website built from mesos SHA: be2ee83. Project: http://git-wip-us.apache.org/repos/asf/mesos-site/repo Commit: http://git-wip-us.apache.org/repos/asf/mesos-site/commit/e5950137 Tree: http://git-wip-us.apache.org/repos/asf/mesos-site/tree/e5950137 Diff: http://git-wip-us.apache.org/repos/asf/mesos-site/diff/e5950137 Branch: refs/heads/asf-site Commit: e5950137ba4e6e1d7e9195fe52be21fe869dcd0d Parents: 33d2606 Author: jenkins <bui...@apache.org> Authored: Thu Jul 12 03:26:14 2018 +0000 Committer: jenkins <bui...@apache.org> Committed: Thu Jul 12 03:26:14 2018 +0000 ---------------------------------------------------------------------- .../app-framework-development-guide/index.html | 361 ++----------------- .../app-framework-development-guide/index.html | 361 ++----------------- 2 files changed, 52 insertions(+), 670 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/mesos-site/blob/e5950137/content/documentation/app-framework-development-guide/index.html ---------------------------------------------------------------------- diff --git a/content/documentation/app-framework-development-guide/index.html b/content/documentation/app-framework-development-guide/index.html index 9dd6aff..fb3db8e 100644 --- a/content/documentation/app-framework-development-guide/index.html +++ b/content/documentation/app-framework-development-guide/index.html @@ -119,276 +119,29 @@ and Scala.</p> <h2>Create your Framework Scheduler</h2> +<h3>API</h3> + <p>If you are writing a scheduler against Mesos 1.0 or newer, it is recommended to use the new <a href="/documentation/latest/./scheduler-http-api/">HTTP API</a> to talk to Mesos.</p> -<p>If your framework needs to talk to Mesos 0.28.0 or older, you can write the -scheduler in C++, Java/Scala, or Python. Your framework scheduler should inherit -from the <code>Scheduler</code> class (see API below). Your scheduler should create a SchedulerDriver +<p>If your framework needs to talk to Mesos 0.28.0 or older, or you have not updated to the +<a href="/documentation/latest/./scheduler-http-api/">HTTP API</a>, you can write the scheduler in C++, Java/Scala, or Python. +Your framework scheduler should inherit from the <code>Scheduler</code> class +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/scheduler.hpp#L58-L177">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/Scheduler.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L34-L137">Python</a>). Your scheduler should create a SchedulerDriver (which will mediate +communication between your scheduler and the Mesos master) and then call <code>SchedulerDriver.run()</code> +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/scheduler.hpp#L180-L317">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/SchedulerDriver.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L140-L278">Python</a>).</p> + +<p>Your scheduler should create a SchedulerDriver (which will mediate communication between your scheduler and the Mesos master) and then call -<code>SchedulerDriver.run()</code>.</p> - -<h3>Scheduler API</h3> - -<p>Callback interface to be implemented by framework schedulers.</p> - -<p>Declared in <code>MESOS_HOME/include/mesos/scheduler.hpp</code></p> - -<pre><code class="{.cpp}">/* - * Invoked when the scheduler successfully registers with a Mesos - * master. A unique ID (generated by the master) used for - * distinguishing this framework from others and `MasterInfo` - * with the ip and port of the current master are provided as arguments. - */ -virtual void registered( - SchedulerDriver* driver, - const FrameworkID& frameworkId, - const MasterInfo& masterInfo); - -/* - * Invoked when the scheduler reregisters with a newly elected Mesos master. - * This is only called when the scheduler has previously been registered. - * `MasterInfo` containing the updated information about the elected master - * is provided as an argument. - */ -virtual void reregistered( - SchedulerDriver* driver, - const MasterInfo& masterInfo); - -/* - * Invoked when the scheduler becomes "disconnected" from the master - * (e.g., the master fails and another is taking over). - */ -virtual void disconnected(SchedulerDriver* driver); - -/* - * Invoked when resources have been offered to this framework. A - * single offer will only contain resources from a single slave. - * Resources associated with an offer will not be re-offered to - * _this_ framework until either (a) this framework has rejected - * those resources (see SchedulerDriver::launchTasks) or (b) those - * resources have been rescinded (see Scheduler::offerRescinded). - * Note that resources may be concurrently offered to more than one - * framework at a time (depending on the allocator being used). In - * that case, the first framework to launch tasks using those - * resources will be able to use them while the other frameworks - * will have those resources rescinded (or if a framework has - * already launched tasks with those resources then those tasks will - * fail with a TASK_LOST status and a message saying as much). - */ -virtual void resourceOffers( - SchedulerDriver* driver, - const std::vector<Offer>& offers); - -/* - * Invoked when an offer is no longer valid (e.g., the slave was - * lost or another framework used resources in the offer). If for - * whatever reason an offer is never rescinded (e.g., dropped - * message, failing over framework, etc.), a framework that attempts - * to launch tasks using an invalid offer will receive TASK_LOST - * status updates for those tasks (see Scheduler::resourceOffers). - */ -virtual void offerRescinded(SchedulerDriver* driver, const OfferID& offerId); - -/* - * Invoked when the status of a task has changed (e.g., a slave is - * lost and so the task is lost, a task finishes and an executor - * sends a status update saying so, etc). If implicit - * acknowledgements are being used, then returning from this - * callback _acknowledges_ receipt of this status update! If for - * whatever reason the scheduler aborts during this callback (or - * the process exits) another status update will be delivered (note, - * however, that this is currently not true if the slave sending the - * status update is lost/fails during that time). If explicit - * acknowledgements are in use, the scheduler must acknowledge this - * status on the driver. - */ -virtual void statusUpdate(SchedulerDriver* driver, const TaskStatus& status); +<code>SchedulerDriver.run()</code>:</p> -/* - * Invoked when an executor sends a message. These messages are best - * effort; do not expect a framework message to be retransmitted in - * any reliable fashion. - */ -virtual void frameworkMessage( - SchedulerDriver* driver, - const ExecutorID& executorId, - const SlaveID& slaveId, - const std::string& data); - -/* - * Invoked when a slave has been determined unreachable (e.g., - * machine failure, network partition). Most frameworks will need to - * reschedule any tasks launched on this slave on a new slave. - * - * NOTE: This callback is not reliably delivered. If a host or - * network failure causes messages between the master and the - * scheduler to be dropped, this callback may not be invoked. - */ -virtual void slaveLost(SchedulerDriver* driver, const SlaveID& slaveId); - -/* - * Invoked when an executor has exited/terminated. Note that any - * tasks running will have TASK_LOST status updates automagically - * generated. - * - * NOTE: This callback is not reliably delivered. If a host or - * network failure causes messages between the master and the - * scheduler to be dropped, this callback may not be invoked. - */ -virtual void executorLost( - SchedulerDriver* driver, - const ExecutorID& executorId, - const SlaveID& slaveId, - int status); - -/* - * Invoked when there is an unrecoverable error in the scheduler or - * scheduler driver. The driver will be aborted BEFORE invoking this - * callback. - */ -virtual void error(SchedulerDriver* driver, const std::string& message); -</code></pre> +<h3>High Availability</h3> -<h3>Scheduler Driver API</h3> - -<p>The Scheduler Driver is responsible for managing the scheduler’s lifecycle -(e.g., start, stop, or wait to finish) and interacting with Mesos Master -(e.g., launch tasks, kill tasks, etc.).</p> - -<p>Note that this interface is usually not implemented by a framework itself, -but it describes the possible calls a framework scheduler can make to -interact with the Mesos Master.</p> - -<p>Please note that usage of this interface requires an instantiated -MesosSchedulerDiver. -See <code>src/examples/test_framework.cpp</code> for an example of using the -MesosSchedulerDriver.</p> - -<p>Declared in <code>MESOS_HOME/include/mesos/scheduler.hpp</code></p> - -<pre><code class="{.cpp}">// Starts the scheduler driver. This needs to be called before any -// other driver calls are made. -virtual Status start(); - -// Stops the scheduler driver. If the 'failover' flag is set to -// false then it is expected that this framework will never -// reconnect to Mesos. So Mesos will unregister the framework and -// shutdown all its tasks and executors. If 'failover' is true, all -// executors and tasks will remain running (for some framework -// specific failover timeout) allowing the scheduler to reconnect -// (possibly in the same process, or from a different process, for -// example, on a different machine). -virtual Status stop(bool failover = false); - -// Aborts the driver so that no more callbacks can be made to the -// scheduler. The semantics of abort and stop have deliberately been -// separated so that code can detect an aborted driver (i.e., via -// the return status of SchedulerDriver::join, see below), and -// instantiate and start another driver if desired (from within the -// same process). Note that 'stop()' is not automatically called -// inside 'abort()'. -virtual Status abort(); - -// Waits for the driver to be stopped or aborted, possibly -// _blocking_ the current thread indefinitely. The return status of -// this function can be used to determine if the driver was aborted -// (see mesos.proto for a description of Status). -virtual Status join(); - -// Starts and immediately joins (i.e., blocks on) the driver. -virtual Status run(); - -// Requests resources from Mesos (see mesos.proto for a description -// of Request and how, for example, to request resources from -// specific slaves). Any resources available are offered to the -// framework via Scheduler::resourceOffers callback, asynchronously. -virtual Status requestResources(const std::vector<Request>& requests); - -// Launches the given set of tasks. Any remaining resources (i.e., -// those that are not used by the launched tasks or their executors) -// will be considered declined. Note that this includes resources -// used by tasks that the framework attempted to launch but failed -// (with `TASK_ERROR`) due to a malformed task description. The -// specified filters are applied on all unused resources (see -// mesos.proto for a description of Filters). Available resources -// are aggregated when multiple offers are provided. Note that all -// offers must belong to the same slave. Invoking this function with -// an empty collection of tasks declines offers in their entirety -// (see Scheduler::declineOffer). -virtual Status launchTasks( - const std::vector<OfferID>& offerIds, - const std::vector<TaskInfo>& tasks, - const Filters& filters = Filters()); - -// Kills the specified task. Note that attempting to kill a task is -// currently not reliable. If, for example, a scheduler fails over -// while it was attempting to kill a task it will need to retry in -// the future. Likewise, if unregistered / disconnected, the request -// will be dropped (these semantics may be changed in the future). -virtual Status killTask(const TaskID& taskId); - -// Accepts the given offers and performs a sequence of operations on -// those accepted offers. See Offer.Operation in mesos.proto for the -// set of available operations. Any remaining resources (i.e., those -// that are not used by the launched tasks or their executors) will -// be considered declined. Note that this includes resources used by -// tasks that the framework attempted to launch but failed (with -// `TASK_ERROR`) due to a malformed task description. The specified -// filters are applied on all unused resources (see mesos.proto for -// a description of Filters). Available resources are aggregated -// when multiple offers are provided. Note that all offers must -// belong to the same slave. -virtual Status acceptOffers( - const std::vector<OfferID>& offerIds, - const std::vector<Offer::Operation>& operations, - const Filters& filters = Filters()); - -// Declines an offer in its entirety and applies the specified -// filters on the resources (see mesos.proto for a description of -// Filters). Note that this can be done at any time, it is not -// necessary to do this within the Scheduler::resourceOffers -// callback. -virtual Status declineOffer( - const OfferID& offerId, - const Filters& filters = Filters()); - -// Removes all filters previously set by the framework (via -// launchTasks()). This enables the framework to receive offers from -// those filtered slaves. -virtual Status reviveOffers(); - -// Inform Mesos master to stop sending offers to the framework. The -// scheduler should call reviveOffers() to resume getting offers. -virtual Status suppressOffers(); - -// Acknowledges the status update. This should only be called -// once the status update is processed durably by the scheduler. -// Not that explicit acknowledgements must be requested via the -// constructor argument, otherwise a call to this method will -// cause the driver to crash. -virtual Status acknowledgeStatusUpdate(const TaskStatus& status); - -// Sends a message from the framework to one of its executors. These -// messages are best effort; do not expect a framework message to be -// retransmitted in any reliable fashion. -virtual Status sendFrameworkMessage( - const ExecutorID& executorId, - const SlaveID& slaveId, - const std::string& data); - -// Allows the framework to query the status for non-terminal tasks. -// This causes the master to send back the latest task status for -// each task in 'statuses', if possible. Tasks that are no longer -// known will result in a TASK_LOST update. If statuses is empty, -// then the master will send the latest status for each task -// currently known. -virtual Status reconcileTasks(const std::vector<TaskStatus>& statuses); -</code></pre> - -<h3>Handling Failures</h3> - -<p>How to build Mesos frameworks that remain available in the face of failures is +<p>How to build Mesos frameworks that are highly available in the face of failures is discussed in a <a href="/documentation/latest/./high-availability-framework-guide/">separate document</a>.</p> <h2>Working with Executors</h2> @@ -462,78 +215,16 @@ between tasks and processes.</p> to use the new <a href="/documentation/latest/./executor-http-api/">HTTP API</a> to talk to Mesos.</p> <p>If writing against Mesos 0.28.0 or older, your framework executor must inherit -from the Executor class. It must override the launchTask() method. You can use +from the Executor class (see (see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/executor.hpp#L60-L137">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/Executor.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L280-L344">Python</a>). It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where -Mesos is running from.</p> - -<h4>Executor API</h4> - -<p>Declared in <code>MESOS_HOME/include/mesos/executor.hpp</code></p> - -<pre><code class="{.cpp}">/* - * Invoked once the executor driver has been able to successfully - * connect with Mesos. In particular, a scheduler can pass some - * data to its executors through the `FrameworkInfo.ExecutorInfo`'s - * data field. - */ -virtual void registered( - ExecutorDriver* driver, - const ExecutorInfo& executorInfo, - const FrameworkInfo& frameworkInfo, - const SlaveInfo& slaveInfo); - -/* - * Invoked when the executor reregisters with a restarted slave. - */ -virtual void reregistered(ExecutorDriver* driver, const SlaveInfo& slaveInfo); - -/* - * Invoked when the executor becomes "disconnected" from the slave - * (e.g., the slave is being restarted due to an upgrade). - */ -virtual void disconnected(ExecutorDriver* driver); - -/* - * Invoked when a task has been launched on this executor (initiated - * via Scheduler::launchTasks). Note that this task can be realized - * with a thread, a process, or some simple computation, however, no - * other callbacks will be invoked on this executor until this - * callback has returned. - */ -virtual void launchTask(ExecutorDriver* driver, const TaskInfo& task); - -/* - * Invoked when a task running within this executor has been killed - * (via SchedulerDriver::killTask). Note that no status update will - * be sent on behalf of the executor, the executor is responsible - * for creating a new TaskStatus (i.e., with TASK_KILLED) and - * invoking ExecutorDriver::sendStatusUpdate. - */ -virtual void killTask(ExecutorDriver* driver, const TaskID& taskId); - -/* - * Invoked when a framework message has arrived for this - * executor. These messages are best effort; do not expect a - * framework message to be retransmitted in any reliable fashion. - */ -virtual void frameworkMessage(ExecutorDriver* driver, const std::string& data); - -/* - * Invoked when the executor should terminate all of its currently - * running tasks. Note that after a Mesos has determined that an - * executor has terminated any tasks that the executor did not send - * terminal status updates for (e.g., TASK_KILLED, TASK_FINISHED, - * TASK_FAILED, etc) a TASK_LOST status update will be created. - */ -virtual void shutdown(ExecutorDriver* driver); - -/* - * Invoked when a fatal error has occurred with the executor and/or - * executor driver. The driver will be aborted BEFORE invoking this - * callback. - */ -virtual void error(ExecutorDriver* driver, const std::string& message); -</code></pre> +Mesos is running from. Your executor should create an ExecutorDriver (which will +mediate communication between your executor and the Mesos agent) and then call +<code>ExecutorDriver.run()</code> +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/executor.hpp#L140-L188">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/ExecutorDriver.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L348-L401">Python</a>).</p> <h4>Install your custom Framework Executor</h4> http://git-wip-us.apache.org/repos/asf/mesos-site/blob/e5950137/content/documentation/latest/app-framework-development-guide/index.html ---------------------------------------------------------------------- diff --git a/content/documentation/latest/app-framework-development-guide/index.html b/content/documentation/latest/app-framework-development-guide/index.html index 6d68f6a..75bad4c 100644 --- a/content/documentation/latest/app-framework-development-guide/index.html +++ b/content/documentation/latest/app-framework-development-guide/index.html @@ -119,276 +119,29 @@ and Scala.</p> <h2>Create your Framework Scheduler</h2> +<h3>API</h3> + <p>If you are writing a scheduler against Mesos 1.0 or newer, it is recommended to use the new <a href="/documentation/latest/./scheduler-http-api/">HTTP API</a> to talk to Mesos.</p> -<p>If your framework needs to talk to Mesos 0.28.0 or older, you can write the -scheduler in C++, Java/Scala, or Python. Your framework scheduler should inherit -from the <code>Scheduler</code> class (see API below). Your scheduler should create a SchedulerDriver +<p>If your framework needs to talk to Mesos 0.28.0 or older, or you have not updated to the +<a href="/documentation/latest/./scheduler-http-api/">HTTP API</a>, you can write the scheduler in C++, Java/Scala, or Python. +Your framework scheduler should inherit from the <code>Scheduler</code> class +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/scheduler.hpp#L58-L177">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/Scheduler.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L34-L137">Python</a>). Your scheduler should create a SchedulerDriver (which will mediate +communication between your scheduler and the Mesos master) and then call <code>SchedulerDriver.run()</code> +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/scheduler.hpp#L180-L317">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/SchedulerDriver.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L140-L278">Python</a>).</p> + +<p>Your scheduler should create a SchedulerDriver (which will mediate communication between your scheduler and the Mesos master) and then call -<code>SchedulerDriver.run()</code>.</p> - -<h3>Scheduler API</h3> - -<p>Callback interface to be implemented by framework schedulers.</p> - -<p>Declared in <code>MESOS_HOME/include/mesos/scheduler.hpp</code></p> - -<pre><code class="{.cpp}">/* - * Invoked when the scheduler successfully registers with a Mesos - * master. A unique ID (generated by the master) used for - * distinguishing this framework from others and `MasterInfo` - * with the ip and port of the current master are provided as arguments. - */ -virtual void registered( - SchedulerDriver* driver, - const FrameworkID& frameworkId, - const MasterInfo& masterInfo); - -/* - * Invoked when the scheduler reregisters with a newly elected Mesos master. - * This is only called when the scheduler has previously been registered. - * `MasterInfo` containing the updated information about the elected master - * is provided as an argument. - */ -virtual void reregistered( - SchedulerDriver* driver, - const MasterInfo& masterInfo); - -/* - * Invoked when the scheduler becomes "disconnected" from the master - * (e.g., the master fails and another is taking over). - */ -virtual void disconnected(SchedulerDriver* driver); - -/* - * Invoked when resources have been offered to this framework. A - * single offer will only contain resources from a single slave. - * Resources associated with an offer will not be re-offered to - * _this_ framework until either (a) this framework has rejected - * those resources (see SchedulerDriver::launchTasks) or (b) those - * resources have been rescinded (see Scheduler::offerRescinded). - * Note that resources may be concurrently offered to more than one - * framework at a time (depending on the allocator being used). In - * that case, the first framework to launch tasks using those - * resources will be able to use them while the other frameworks - * will have those resources rescinded (or if a framework has - * already launched tasks with those resources then those tasks will - * fail with a TASK_LOST status and a message saying as much). - */ -virtual void resourceOffers( - SchedulerDriver* driver, - const std::vector<Offer>& offers); - -/* - * Invoked when an offer is no longer valid (e.g., the slave was - * lost or another framework used resources in the offer). If for - * whatever reason an offer is never rescinded (e.g., dropped - * message, failing over framework, etc.), a framework that attempts - * to launch tasks using an invalid offer will receive TASK_LOST - * status updates for those tasks (see Scheduler::resourceOffers). - */ -virtual void offerRescinded(SchedulerDriver* driver, const OfferID& offerId); - -/* - * Invoked when the status of a task has changed (e.g., a slave is - * lost and so the task is lost, a task finishes and an executor - * sends a status update saying so, etc). If implicit - * acknowledgements are being used, then returning from this - * callback _acknowledges_ receipt of this status update! If for - * whatever reason the scheduler aborts during this callback (or - * the process exits) another status update will be delivered (note, - * however, that this is currently not true if the slave sending the - * status update is lost/fails during that time). If explicit - * acknowledgements are in use, the scheduler must acknowledge this - * status on the driver. - */ -virtual void statusUpdate(SchedulerDriver* driver, const TaskStatus& status); +<code>SchedulerDriver.run()</code>:</p> -/* - * Invoked when an executor sends a message. These messages are best - * effort; do not expect a framework message to be retransmitted in - * any reliable fashion. - */ -virtual void frameworkMessage( - SchedulerDriver* driver, - const ExecutorID& executorId, - const SlaveID& slaveId, - const std::string& data); - -/* - * Invoked when a slave has been determined unreachable (e.g., - * machine failure, network partition). Most frameworks will need to - * reschedule any tasks launched on this slave on a new slave. - * - * NOTE: This callback is not reliably delivered. If a host or - * network failure causes messages between the master and the - * scheduler to be dropped, this callback may not be invoked. - */ -virtual void slaveLost(SchedulerDriver* driver, const SlaveID& slaveId); - -/* - * Invoked when an executor has exited/terminated. Note that any - * tasks running will have TASK_LOST status updates automagically - * generated. - * - * NOTE: This callback is not reliably delivered. If a host or - * network failure causes messages between the master and the - * scheduler to be dropped, this callback may not be invoked. - */ -virtual void executorLost( - SchedulerDriver* driver, - const ExecutorID& executorId, - const SlaveID& slaveId, - int status); - -/* - * Invoked when there is an unrecoverable error in the scheduler or - * scheduler driver. The driver will be aborted BEFORE invoking this - * callback. - */ -virtual void error(SchedulerDriver* driver, const std::string& message); -</code></pre> +<h3>High Availability</h3> -<h3>Scheduler Driver API</h3> - -<p>The Scheduler Driver is responsible for managing the scheduler’s lifecycle -(e.g., start, stop, or wait to finish) and interacting with Mesos Master -(e.g., launch tasks, kill tasks, etc.).</p> - -<p>Note that this interface is usually not implemented by a framework itself, -but it describes the possible calls a framework scheduler can make to -interact with the Mesos Master.</p> - -<p>Please note that usage of this interface requires an instantiated -MesosSchedulerDiver. -See <code>src/examples/test_framework.cpp</code> for an example of using the -MesosSchedulerDriver.</p> - -<p>Declared in <code>MESOS_HOME/include/mesos/scheduler.hpp</code></p> - -<pre><code class="{.cpp}">// Starts the scheduler driver. This needs to be called before any -// other driver calls are made. -virtual Status start(); - -// Stops the scheduler driver. If the 'failover' flag is set to -// false then it is expected that this framework will never -// reconnect to Mesos. So Mesos will unregister the framework and -// shutdown all its tasks and executors. If 'failover' is true, all -// executors and tasks will remain running (for some framework -// specific failover timeout) allowing the scheduler to reconnect -// (possibly in the same process, or from a different process, for -// example, on a different machine). -virtual Status stop(bool failover = false); - -// Aborts the driver so that no more callbacks can be made to the -// scheduler. The semantics of abort and stop have deliberately been -// separated so that code can detect an aborted driver (i.e., via -// the return status of SchedulerDriver::join, see below), and -// instantiate and start another driver if desired (from within the -// same process). Note that 'stop()' is not automatically called -// inside 'abort()'. -virtual Status abort(); - -// Waits for the driver to be stopped or aborted, possibly -// _blocking_ the current thread indefinitely. The return status of -// this function can be used to determine if the driver was aborted -// (see mesos.proto for a description of Status). -virtual Status join(); - -// Starts and immediately joins (i.e., blocks on) the driver. -virtual Status run(); - -// Requests resources from Mesos (see mesos.proto for a description -// of Request and how, for example, to request resources from -// specific slaves). Any resources available are offered to the -// framework via Scheduler::resourceOffers callback, asynchronously. -virtual Status requestResources(const std::vector<Request>& requests); - -// Launches the given set of tasks. Any remaining resources (i.e., -// those that are not used by the launched tasks or their executors) -// will be considered declined. Note that this includes resources -// used by tasks that the framework attempted to launch but failed -// (with `TASK_ERROR`) due to a malformed task description. The -// specified filters are applied on all unused resources (see -// mesos.proto for a description of Filters). Available resources -// are aggregated when multiple offers are provided. Note that all -// offers must belong to the same slave. Invoking this function with -// an empty collection of tasks declines offers in their entirety -// (see Scheduler::declineOffer). -virtual Status launchTasks( - const std::vector<OfferID>& offerIds, - const std::vector<TaskInfo>& tasks, - const Filters& filters = Filters()); - -// Kills the specified task. Note that attempting to kill a task is -// currently not reliable. If, for example, a scheduler fails over -// while it was attempting to kill a task it will need to retry in -// the future. Likewise, if unregistered / disconnected, the request -// will be dropped (these semantics may be changed in the future). -virtual Status killTask(const TaskID& taskId); - -// Accepts the given offers and performs a sequence of operations on -// those accepted offers. See Offer.Operation in mesos.proto for the -// set of available operations. Any remaining resources (i.e., those -// that are not used by the launched tasks or their executors) will -// be considered declined. Note that this includes resources used by -// tasks that the framework attempted to launch but failed (with -// `TASK_ERROR`) due to a malformed task description. The specified -// filters are applied on all unused resources (see mesos.proto for -// a description of Filters). Available resources are aggregated -// when multiple offers are provided. Note that all offers must -// belong to the same slave. -virtual Status acceptOffers( - const std::vector<OfferID>& offerIds, - const std::vector<Offer::Operation>& operations, - const Filters& filters = Filters()); - -// Declines an offer in its entirety and applies the specified -// filters on the resources (see mesos.proto for a description of -// Filters). Note that this can be done at any time, it is not -// necessary to do this within the Scheduler::resourceOffers -// callback. -virtual Status declineOffer( - const OfferID& offerId, - const Filters& filters = Filters()); - -// Removes all filters previously set by the framework (via -// launchTasks()). This enables the framework to receive offers from -// those filtered slaves. -virtual Status reviveOffers(); - -// Inform Mesos master to stop sending offers to the framework. The -// scheduler should call reviveOffers() to resume getting offers. -virtual Status suppressOffers(); - -// Acknowledges the status update. This should only be called -// once the status update is processed durably by the scheduler. -// Not that explicit acknowledgements must be requested via the -// constructor argument, otherwise a call to this method will -// cause the driver to crash. -virtual Status acknowledgeStatusUpdate(const TaskStatus& status); - -// Sends a message from the framework to one of its executors. These -// messages are best effort; do not expect a framework message to be -// retransmitted in any reliable fashion. -virtual Status sendFrameworkMessage( - const ExecutorID& executorId, - const SlaveID& slaveId, - const std::string& data); - -// Allows the framework to query the status for non-terminal tasks. -// This causes the master to send back the latest task status for -// each task in 'statuses', if possible. Tasks that are no longer -// known will result in a TASK_LOST update. If statuses is empty, -// then the master will send the latest status for each task -// currently known. -virtual Status reconcileTasks(const std::vector<TaskStatus>& statuses); -</code></pre> - -<h3>Handling Failures</h3> - -<p>How to build Mesos frameworks that remain available in the face of failures is +<p>How to build Mesos frameworks that are highly available in the face of failures is discussed in a <a href="/documentation/latest/./high-availability-framework-guide/">separate document</a>.</p> <h2>Working with Executors</h2> @@ -462,78 +215,16 @@ between tasks and processes.</p> to use the new <a href="/documentation/latest/./executor-http-api/">HTTP API</a> to talk to Mesos.</p> <p>If writing against Mesos 0.28.0 or older, your framework executor must inherit -from the Executor class. It must override the launchTask() method. You can use +from the Executor class (see (see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/executor.hpp#L60-L137">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/Executor.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L280-L344">Python</a>). It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where -Mesos is running from.</p> - -<h4>Executor API</h4> - -<p>Declared in <code>MESOS_HOME/include/mesos/executor.hpp</code></p> - -<pre><code class="{.cpp}">/* - * Invoked once the executor driver has been able to successfully - * connect with Mesos. In particular, a scheduler can pass some - * data to its executors through the `FrameworkInfo.ExecutorInfo`'s - * data field. - */ -virtual void registered( - ExecutorDriver* driver, - const ExecutorInfo& executorInfo, - const FrameworkInfo& frameworkInfo, - const SlaveInfo& slaveInfo); - -/* - * Invoked when the executor reregisters with a restarted slave. - */ -virtual void reregistered(ExecutorDriver* driver, const SlaveInfo& slaveInfo); - -/* - * Invoked when the executor becomes "disconnected" from the slave - * (e.g., the slave is being restarted due to an upgrade). - */ -virtual void disconnected(ExecutorDriver* driver); - -/* - * Invoked when a task has been launched on this executor (initiated - * via Scheduler::launchTasks). Note that this task can be realized - * with a thread, a process, or some simple computation, however, no - * other callbacks will be invoked on this executor until this - * callback has returned. - */ -virtual void launchTask(ExecutorDriver* driver, const TaskInfo& task); - -/* - * Invoked when a task running within this executor has been killed - * (via SchedulerDriver::killTask). Note that no status update will - * be sent on behalf of the executor, the executor is responsible - * for creating a new TaskStatus (i.e., with TASK_KILLED) and - * invoking ExecutorDriver::sendStatusUpdate. - */ -virtual void killTask(ExecutorDriver* driver, const TaskID& taskId); - -/* - * Invoked when a framework message has arrived for this - * executor. These messages are best effort; do not expect a - * framework message to be retransmitted in any reliable fashion. - */ -virtual void frameworkMessage(ExecutorDriver* driver, const std::string& data); - -/* - * Invoked when the executor should terminate all of its currently - * running tasks. Note that after a Mesos has determined that an - * executor has terminated any tasks that the executor did not send - * terminal status updates for (e.g., TASK_KILLED, TASK_FINISHED, - * TASK_FAILED, etc) a TASK_LOST status update will be created. - */ -virtual void shutdown(ExecutorDriver* driver); - -/* - * Invoked when a fatal error has occurred with the executor and/or - * executor driver. The driver will be aborted BEFORE invoking this - * callback. - */ -virtual void error(ExecutorDriver* driver, const std::string& message); -</code></pre> +Mesos is running from. Your executor should create an ExecutorDriver (which will +mediate communication between your executor and the Mesos agent) and then call +<code>ExecutorDriver.run()</code> +(see: <a href="https://github.com/apache/mesos/blob/1.6.0/include/mesos/executor.hpp#L140-L188">C++</a>, +<a href="http://mesos.apache.org/api/latest/java/org/apache/mesos/ExecutorDriver.html">Java</a>, +<a href="https://github.com/apache/mesos/blob/1.6.0/src/python/interface/src/mesos/interface/__init__.py#L348-L401">Python</a>).</p> <h4>Install your custom Framework Executor</h4>