Have you tried the remedy steps included in the error message?
------------------------------------------------------------
To remedy this do as follows:
Step 1: rm -f /tmp/mesos/meta/slaves/latest
This ensures slave doesn't recover old live executors.
Step 2: Restart the slave.
On Fri, Apr 8, 2016 at 11:29 AM, Stefano Bianchi <[email protected]>
wrote:
> Hi i would like to enter in this mailing list.
> i'm currently doing my Master Thesis on Mesos and Calico.
> I'm working at INFN, institute of nuclear physics. The goal of the thesis
> is to build a PaaS where mesos is the scheduler and Calico must allow the
> interconnection between multiple datacenters linked to the CERN.
>
> I'm exploiting an IaaS based on Openstack, here i have created 6 Virtual
> Machines, 3 Masters and 3 Slaves, on one slave is running Mesos-DNS from
> Marathon.
> All is perfectly working, since i am on another network i changed
> correctly the hostnames such that on mesos are resolvable and i tried to
> run from marathon a simple http server which is scalable on all my machine.
> So all is fine and working.
>
> The only thing that i don't like is that each 3 slaves have 1 CPU 10 GB of
> disk memory and 2GB of RAM, but mesos currently show for each one only 5 GB
> of disk memory and 900MB of RAM.
> So checking in documentation i found the command to manage the resources.
> I stopped Slave1, for instance, and i have run this command:
>
> mesos-slave --master=MASTER_ADDRESS:5050
> --resources='cpu:1;mem:2000;disk:9000'
>
> where i want set 2000 GB of RAM and 9000GB of disk memory.
> The output is the following:
>
> I0408 15:11:00.915324 7892 main.cpp:215] Build: 2016-03-10 20:32:58 by root
>
> I0408 15:11:00.915436 7892 main.cpp:217] Version: 0.27.2
>
> I0408 15:11:00.915448 7892 main.cpp:220] Git tag: 0.27.2
>
> I0408 15:11:00.915459 7892 main.cpp:224] Git SHA:
> 3c9ec4a0f34420b7803848af597de00fedefe0e2
>
> I0408 15:11:00.923334 7892 systemd.cpp:236] systemd version `219` detected
>
> I0408 15:11:00.923384 7892 main.cpp:232] Inializing systemd state
>
> I0408 15:11:00.950050 7892 systemd.cpp:324] Started systemd slice
> `mesos_executors.slice`
>
> I0408 15:11:00.951529 7892 containerizer.cpp:143] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
>
> I0408 15:11:00.963232 7892 linux_launcher.cpp:101] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
>
> I0408 15:11:00.965541 7892 main.cpp:320] Starting Mesos slave
>
> I0408 15:11:00.966008 7892 slave.cpp:192] Slave started on
> 1)@192.168.100.56:5051
>
> I0408 15:11:00.966023 7892 slave.cpp:193] Flags at startup:
> --appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins"
> --docker="docker" --docker_auth_server="https://auth.docker.io";
> --docker_kill_orphans="true" --docker_puller_timeout="60"
> --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins"
> --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
> --logbufsecs="0" --logging_level="INFO" --master="192.168.100.55:5050"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="1secs" --resources="cpu:1;mem:2000;disk:9000"
> --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox"
> --strict="true" --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/mesos"
>
> I0408 15:11:00.967485 7892 slave.cpp:463] Slave resources: cpu(*):1;
> mem(*):2000; disk(*):9000; cpus(*):1; ports(*):[31000-32000]
>
> I0408 15:11:00.967547 7892 slave.cpp:471] Slave attributes: [ ]
>
> I0408 15:11:00.967560 7892 slave.cpp:476] Slave hostname:
> slave1.openstacklocal
>
> I0408 15:11:00.971304 7893 state.cpp:58] Recovering state from
> '/tmp/mesos/meta'
>
> *Failed to perform recovery: Incompatible slave info detected*.
>
> ------------------------------------------------------------
>
> Old slave info:
>
> hostname: "*slave_public_IP_i_set*"
>
> resources {
>
> name: "cpus"
>
> type: SCALAR
>
> scalar {
>
> value: 1
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "mem"
>
> type: SCALAR
>
> scalar {
>
> value: 920
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "disk"
>
> type: SCALAR
>
> scalar {
>
> value: 5112
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "ports"
>
> type: RANGES
>
> ranges {
>
> range {
>
> begin: 31000
>
> end: 32000
>
> }
>
> }
>
> role: "*"
>
> }
>
> id {
>
> value: "ad490064-1a6e-415c-8536-daef0d8e3572-S7"
>
> }
>
> checkpoint: true
>
> port: 5051
>
> ------------------------------------------------------------
>
> New slave info:
>
> hostname: "
>
> slave1.openstacklocal
>
> "
>
> resources {
>
> name: "cpu"
>
> type: SCALAR
>
> scalar {
>
> value: 1
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "mem"
>
> type: SCALAR
>
> scalar {
>
> value: 2000
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "disk"
>
> type: SCALAR
>
> scalar {
>
> value: 9000
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "cpus"
>
> type: SCALAR
>
> scalar {
>
> value: 1
>
> }
>
> role: "*"
>
> }
>
> resources {
>
> name: "ports"
>
> type: RANGES
>
> ranges {
>
> range {
>
> begin: 31000
>
> end: 32000
>
> }
>
> }
>
> role: "*"
>
> }
>
> id {
>
> value: "ad490064-1a6e-415c-8536-daef0d8e3572-S7"
>
> }
>
> checkpoint: true
>
> port: 5051
>
> ------------------------------------------------------------
>
> To remedy this do as follows:
>
> Step 1: rm -f /tmp/mesos/meta/slaves/latest
>
> This ensures slave doesn't recover old live executors.
>
> Step 2: Restart the slave.
>
>
>
> I can notice two things:
>
>
> 1)the message of failure;
>
> 2)the hostname is changed; the right one is a public IP i have set in order
> to resolve the hostname for mesos.
>
> As a consequence, when i start the slave, the resources are exaclty the same,
> nothing is changed.
>
> Can you please help me?
>
>
> Thanks!
>
>
>
