[mesos] 01/02: Added `futureTracker` to the `SlaveOptions` in tests.
This is an automated email from the ASF dual-hosted git repository.
abudnik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit dee4b849c8179ea46947c8ea4dd031f6eb37b659
Author: Andrei Budnik
AuthorDate: Fri Sep 6 17:01:56 2019 +0200
Added `futureTracker` to the `SlaveOptions` in tests.
`PendingFutureTracker` is shared across both Mesos containerizer and
the agent, so we need to add an option to be able to start a slave in
tests with an instance of the `futureTrack` as a parameter.
Review: https://reviews.apache.org/r/71454
---
src/tests/cluster.cpp | 20 +---
src/tests/cluster.hpp | 5 +
src/tests/mesos.cpp | 1 +
src/tests/mesos.hpp | 8
4 files changed, 27 insertions(+), 7 deletions(-)
diff --git a/src/tests/cluster.cpp b/src/tests/cluster.cpp
index 1646516..f7bc882 100644
--- a/src/tests/cluster.cpp
+++ b/src/tests/cluster.cpp
@@ -416,6 +416,7 @@ Try> Slave::create(
const Option& qosController,
const Option& secretGenerator,
const Option& providedAuthorizer,
+const Option& futureTracker,
bool mock)
{
process::Owned slave(new Slave());
@@ -447,10 +448,15 @@ Try> Slave::create(
}
#endif // __WINDOWS__
- Try futureTracker = PendingFutureTracker::create();
- if (futureTracker.isError()) {
-return Error(
-"Failed to create pending future tracker: " + futureTracker.error());
+ // If the future tracker is not provided, create a default one.
+ if (futureTracker.isNone()) {
+Try _futureTracker = PendingFutureTracker::create();
+if (_futureTracker.isError()) {
+ return Error(
+ "Failed to create pending future tracker: " +
_futureTracker.error());
+}
+
+slave->futureTracker.reset(_futureTracker.get());
}
// If the containerizer is not provided, create a default one.
@@ -468,7 +474,7 @@ Try> Slave::create(
gc.getOrElse(slave->gc.get()),
nullptr,
volumeGidManager,
- futureTracker.get());
+ futureTracker.getOrElse(slave->futureTracker.get()));
if (_containerizer.isError()) {
return Error("Failed to create containerizer: " +
_containerizer.error());
@@ -621,7 +627,7 @@ Try> Slave::create(
qosController.getOrElse(slave->qosController.get()),
secretGenerator.getOrElse(slave->secretGenerator.get()),
volumeGidManager,
-futureTracker.get(),
+futureTracker.getOrElse(slave->futureTracker.get()),
authorizer));
} else {
slave->slave.reset(new slave::Slave(
@@ -636,7 +642,7 @@ Try> Slave::create(
qosController.getOrElse(slave->qosController.get()),
secretGenerator.getOrElse(slave->secretGenerator.get()),
volumeGidManager,
-futureTracker.get(),
+futureTracker.getOrElse(slave->futureTracker.get()),
authorizer));
}
diff --git a/src/tests/cluster.hpp b/src/tests/cluster.hpp
index c04ee14..415a60f 100644
--- a/src/tests/cluster.hpp
+++ b/src/tests/cluster.hpp
@@ -170,6 +170,7 @@ public:
const Option& qosController = None(),
const Option& secretGenerator = None(),
const Option& authorizer = None(),
+ const Option& futureTracker = None(),
bool mock = false);
~Slave();
@@ -227,6 +228,10 @@ private:
// of who created it).
slave::Containerizer* containerizer = nullptr;
+ // Pending future tracker must be destroyed last since there may be
+ // pending requests related to the dependant objects declared below.
+ process::Owned futureTracker;
+
// Dependencies that are created by the factory method.
process::Owned authorizer;
process::Owned ownedContainerizer;
diff --git a/src/tests/mesos.cpp b/src/tests/mesos.cpp
index e77db22..664c302 100644
--- a/src/tests/mesos.cpp
+++ b/src/tests/mesos.cpp
@@ -347,6 +347,7 @@ Try> MesosTest::StartSlave(const
SlaveOptions& options)
options.qosController,
options.secretGenerator,
options.authorizer,
+ options.futureTracker,
options.mock);
if (slave.isSome() && !options.mock) {
diff --git a/src/tests/mesos.hpp b/src/tests/mesos.hpp
index ecde518..73b1866 100644
--- a/src/tests/mesos.hpp
+++ b/src/tests/mesos.hpp
@@ -186,6 +186,13 @@ struct SlaveOptions
return *this;
}
+ SlaveOptions& withFutureTracker(
+ const Option& futureTracker)
+ {
+this->futureTracker = futureTracker;
+return *this;
+ }
+
mesos::master::detector::MasterDetector* detector;
bool mock;
Option flags;
@@ -197,6 +204,7 @@ struct SlaveOptions
Option qosController;
Option secretGenerator;
Option authorizer;
+ Option futureTracker;
};
[mesos] branch master updated (ce801bd -> 1122674)
This is an automated email from the ASF dual-hosted git repository. abudnik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/mesos.git. from ce801bd Fixed documentation on master fault domains. new dee4b84 Added `futureTracker` to the `SlaveOptions` in tests. new 1122674 Implemented an integration test for /containerizer/debug endpoint. The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/tests/cluster.cpp | 20 -- src/tests/cluster.hpp | 5 ++ src/tests/mesos.cpp | 1 + src/tests/mesos.hpp | 8 +++ src/tests/slave_tests.cpp | 158 ++ 5 files changed, 185 insertions(+), 7 deletions(-)
[mesos] 02/02: Implemented an integration test for /containerizer/debug endpoint.
This is an automated email from the ASF dual-hosted git repository.
abudnik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
commit 1122674a5c03894e4552d46cfa26dca0557a8f68
Author: Andrei Budnik
AuthorDate: Fri Sep 6 13:25:35 2019 +0200
Implemented an integration test for /containerizer/debug endpoint.
This test starts an agent with the MockIsolator to intercept calls to
its `prepare` method, then it launches a task, which gets stuck.
We check that the /containerizer/debug endpoint returns a non-empty
list of pending futures including `MockIsolator::prepare`. After
setting the promise for the `prepare`, the task successfully starts
and we expect for the /containerizer/debug endpoint to return an
empty list of pending operations.
Review: https://reviews.apache.org/r/71455
---
src/tests/slave_tests.cpp | 158 ++
1 file changed, 158 insertions(+)
diff --git a/src/tests/slave_tests.cpp b/src/tests/slave_tests.cpp
index c147bfc..fd4fd6b 100644
--- a/src/tests/slave_tests.cpp
+++ b/src/tests/slave_tests.cpp
@@ -63,6 +63,7 @@
#endif // USE_SSL_SOCKET
#include "common/build.hpp"
+#include "common/future_tracker.hpp"
#include "common/http.hpp"
#include "common/protobuf_utils.hpp"
@@ -83,6 +84,8 @@
#include "slave/containerizer/fetcher_process.hpp"
#include "slave/containerizer/mesos/containerizer.hpp"
+#include "slave/containerizer/mesos/isolator_tracker.hpp"
+#include "slave/containerizer/mesos/launcher.hpp"
#include "tests/active_user_test_helper.hpp"
#include "tests/containerizer.hpp"
@@ -94,6 +97,7 @@
#include "tests/resources_utils.hpp"
#include "tests/utils.hpp"
+#include "tests/containerizer/isolator.hpp"
#include "tests/containerizer/mock_containerizer.hpp"
using namespace mesos::internal::slave;
@@ -112,7 +116,9 @@ using mesos::master::detector::StandaloneMasterDetector;
using mesos::v1::resource_provider::Event;
using mesos::slave::ContainerConfig;
+using mesos::slave::ContainerLaunchInfo;
using mesos::slave::ContainerTermination;
+using mesos::slave::Isolator;
using mesos::v1::scheduler::Call;
using mesos::v1::scheduler::Mesos;
@@ -2787,6 +2793,158 @@ TEST_F(SlaveTest, ContainersEndpoint)
}
+// This test ensures that `/containerizer/debug` endpoint returns a non-empty
+// list of pending futures when an isolator becomes unresponsive during
+// container launch.
+TEST_F(SlaveTest, ROOT_ContainerizerDebugEndpoint)
+{
+ Try> master = StartMaster();
+ ASSERT_SOME(master);
+
+ slave::Flags flags = CreateSlaveFlags();
+
+ Try _launcher = SubprocessLauncher::create(flags);
+ ASSERT_SOME(_launcher);
+
+ Owned launcher(_launcher.get());
+
+ MockIsolator* mockIsolator = new MockIsolator();
+
+ Future prepare;
+ Promise> promise;
+
+ EXPECT_CALL(*mockIsolator, recover(_, _))
+.WillOnce(Return(Nothing()));
+
+ // Simulate a long prepare from the isolator.
+ EXPECT_CALL(*mockIsolator, prepare(_, _))
+.WillOnce(DoAll(FutureSatisfy(),
+Return(promise.future(;
+
+ EXPECT_CALL(*mockIsolator, update(_, _))
+.WillOnce(Return(Nothing()));
+
+ // Wrap `mockIsolator` in `PendingFutureTracker`.
+ Try _futureTracker = PendingFutureTracker::create();
+ ASSERT_SOME(_futureTracker);
+
+ Owned futureTracker(_futureTracker.get());
+
+ Owned isolator = Owned(new IsolatorTracker(
+ Owned(mockIsolator), "MockIsolator", futureTracker.get()));
+
+ Fetcher fetcher(flags);
+
+ Try> provisioner = Provisioner::create(flags);
+ ASSERT_SOME(provisioner);
+
+ Try create = MesosContainerizer::create(
+ flags,
+ true,
+ ,
+ nullptr,
+ launcher,
+ provisioner->share(),
+ {isolator});
+
+ ASSERT_SOME(create);
+
+ Owned containerizer(create.get());
+
+ Owned detector = master.get()->createDetector();
+
+ Try> slave =
+StartSlave(SlaveOptions(detector.get())
+ .withContainerizer(containerizer.get())
+ .withFutureTracker(futureTracker.get()));
+
+ ASSERT_SOME(slave);
+
+ MockScheduler sched;
+ MesosSchedulerDriver driver(
+ , DEFAULT_FRAMEWORK_INFO, master.get()->pid, DEFAULT_CREDENTIAL);
+
+ EXPECT_CALL(sched, registered(, _, _));
+
+ Future> offers;
+ EXPECT_CALL(sched, resourceOffers(, _))
+.WillOnce(FutureArg<1>())
+.WillRepeatedly(Return());// Ignore subsequent offers.
+
+ driver.start();
+
+ AWAIT_READY(offers);
+ ASSERT_FALSE(offers->empty());
+
+ const Offer& offer = offers.get()[0];
+
+ // Launch a task and wait until it is in RUNNING status.
+ TaskInfo task = createTask(
+ offer.slave_id(),
+ Resources::parse("cpus:1;mem:32").get(),
+ SLEEP_COMMAND(1000));
+
+ Future statusStarting;
+ Future statusRunning;
+ EXPECT_CALL(sched, statusUpdate(, _))
+.WillOnce(FutureArg<1>())
+.WillOnce(FutureArg<1>());
+
+ driver.launchTasks(offer.id(), {task});
+
+ AWAIT_READY(prepare);
+
[mesos] branch master updated: Fixed documentation on master fault domains.
This is an automated email from the ASF dual-hosted git repository.
bbannier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/mesos.git
The following commit(s) were added to refs/heads/master by this push:
new ce801bd Fixed documentation on master fault domains.
ce801bd is described below
commit ce801bd6b3f56788eb4d3179959fa7290220432d
Author: Dominik Dary
AuthorDate: Tue Sep 24 10:45:58 2019 +0200
Fixed documentation on master fault domains.
---
docs/fault-domains.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/fault-domains.md b/docs/fault-domains.md
index 08f13b5..1a48acc 100644
--- a/docs/fault-domains.md
+++ b/docs/fault-domains.md
@@ -33,9 +33,9 @@ agent node. The value of this flag must be a file path or a
JSON dictionary
with the key `fault_domain` and subkeys `region` and `zone` mapping to
arbitrary strings:
-mesos-master --domain='{"fault_domain": {"region": "eu", "zone": "rack1"}}'
+mesos-master --domain='{"fault_domain": {"region": {"name":"eu"}, "zone":
{ "name":"rack1"}}}'
-mesos-agent --domain='{"fault_domain": {"region": "eu", "zone": "rack2"}}'
+mesos-agent --domain='{"fault_domain": {"region": {"name":"eu"}, "zone":
{"name":"rack2"}}}'
Frameworks can learn about the domain of an agent by inspecting the `domain`
field in the received offer, which contains a `DomainInfo` that has the
@@ -103,4 +103,4 @@ updated so that they can schedule tasks in the cloud if
required.
Non-region aware frameworks will now only receive offers from agents within
the data center, where the master nodes reside. Region-aware frameworks are
supposed to know when and if they should place their tasks in the data center
-or with the cloud provider.
\ No newline at end of file
+or with the cloud provider.
