Hi i would like to enter in this mailing list. i'm currently doing my Master Thesis on Mesos and Calico. I'm working at INFN, institute of nuclear physics. The goal of the thesis is to build a PaaS where mesos is the scheduler and Calico must allow the interconnection between multiple datacenters linked to the CERN.
I'm exploiting an IaaS based on Openstack, here i have created 6 Virtual Machines, 3 Masters and 3 Slaves, on one slave is running Mesos-DNS from Marathon. All is perfectly working, since i am on another network i changed correctly the hostnames such that on mesos are resolvable and i tried to run from marathon a simple http server which is scalable on all my machine. So all is fine and working. The only thing that i don't like is that each 3 slaves have 1 CPU 10 GB of disk memory and 2GB of RAM, but mesos currently show for each one only 5 GB of disk memory and 900MB of RAM. So checking in documentation i found the command to manage the resources. I stopped Slave1, for instance, and i have run this command: mesos-slave --master=MASTER_ADDRESS:5050 --resources='cpu:1;mem:2000;disk:9000' where i want set 2000 GB of RAM and 9000GB of disk memory. The output is the following: I0408 15:11:00.915324 7892 main.cpp:215] Build: 2016-03-10 20:32:58 by root I0408 15:11:00.915436 7892 main.cpp:217] Version: 0.27.2 I0408 15:11:00.915448 7892 main.cpp:220] Git tag: 0.27.2 I0408 15:11:00.915459 7892 main.cpp:224] Git SHA: 3c9ec4a0f34420b7803848af597de00fedefe0e2 I0408 15:11:00.923334 7892 systemd.cpp:236] systemd version `219` detected I0408 15:11:00.923384 7892 main.cpp:232] Inializing systemd state I0408 15:11:00.950050 7892 systemd.cpp:324] Started systemd slice `mesos_executors.slice` I0408 15:11:00.951529 7892 containerizer.cpp:143] Using isolation: posix/cpu,posix/mem,filesystem/posix I0408 15:11:00.963232 7892 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher I0408 15:11:00.965541 7892 main.cpp:320] Starting Mesos slave I0408 15:11:00.966008 7892 slave.cpp:192] Slave started on 1)@192.168.100.56:5051 I0408 15:11:00.966023 7892 slave.cpp:193] Flags at startup: --appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_auth_server="https://auth.docker.io"; --docker_kill_orphans="true" --docker_puller_timeout="60" --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos" --logbufsecs="0" --logging_level="INFO" --master="192.168.100.55:5050" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="1secs" --resources="cpu:1;mem:2000;disk:9000" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/mesos" I0408 15:11:00.967485 7892 slave.cpp:463] Slave resources: cpu(*):1; mem(*):2000; disk(*):9000; cpus(*):1; ports(*):[31000-32000] I0408 15:11:00.967547 7892 slave.cpp:471] Slave attributes: [ ] I0408 15:11:00.967560 7892 slave.cpp:476] Slave hostname: slave1.openstacklocal I0408 15:11:00.971304 7893 state.cpp:58] Recovering state from '/tmp/mesos/meta' *Failed to perform recovery: Incompatible slave info detected*. ------------------------------------------------------------ Old slave info: hostname: "*slave_public_IP_i_set*" resources { name: "cpus" type: SCALAR scalar { value: 1 } role: "*" } resources { name: "mem" type: SCALAR scalar { value: 920 } role: "*" } resources { name: "disk" type: SCALAR scalar { value: 5112 } role: "*" } resources { name: "ports" type: RANGES ranges { range { begin: 31000 end: 32000 } } role: "*" } id { value: "ad490064-1a6e-415c-8536-daef0d8e3572-S7" } checkpoint: true port: 5051 ------------------------------------------------------------ New slave info: hostname: " slave1.openstacklocal " resources { name: "cpu" type: SCALAR scalar { value: 1 } role: "*" } resources { name: "mem" type: SCALAR scalar { value: 2000 } role: "*" } resources { name: "disk" type: SCALAR scalar { value: 9000 } role: "*" } resources { name: "cpus" type: SCALAR scalar { value: 1 } role: "*" } resources { name: "ports" type: RANGES ranges { range { begin: 31000 end: 32000 } } role: "*" } id { value: "ad490064-1a6e-415c-8536-daef0d8e3572-S7" } checkpoint: true port: 5051 ------------------------------------------------------------ To remedy this do as follows: Step 1: rm -f /tmp/mesos/meta/slaves/latest This ensures slave doesn't recover old live executors. Step 2: Restart the slave. I can notice two things: 1)the message of failure; 2)the hostname is changed; the right one is a public IP i have set in order to resolve the hostname for mesos. As a consequence, when i start the slave, the resources are exaclty the same, nothing is changed. Can you please help me? Thanks!
