Re: [VOTE] Release Apache Mesos 1.5.0 (rc2)

2018-02-02 Thread Jie Yu 
+1

Verified in our internal CI that `sudo make check` passed in CentOS 6,
CentOS7, Debian 8, Ubuntu 14.04, Ubuntu 16.04 (both w/ or w/o SSL enabled).


On Thu, Feb 1, 2018 at 5:36 PM, Gilbert Song  wrote:

> Hi all,
>
> Please vote on releasing the following candidate as Apache Mesos 1.5.0.
>
> 1.5.0 includes the following:
> 
> 
>   * Support Container Storage Interface (CSI).
>   * Agent reconfiguration policy.
>   * Auto GC docker images in Mesos Containerizer.
>   * Standalone containers.
>   * Support gRPC client.
>   * Non-leading VOTING replica catch-up.
>
>
> The CHANGELOG for the release is available at:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_p
> lain;f=CHANGELOG;hb=1.5.0-rc2
> 
> 
>
> The candidate for Mesos 1.5.0 release is available at:
> https://dist.apache.org/repos/dist/dev/mesos/1.5.0-rc2/mesos-1.5.0.tar.gz
>
> The tag to be voted on is 1.5.0-rc2:
> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=1.5.0-rc2
>
> The MD5 checksum of the tarball can be found at:
> https://dist.apache.org/repos/dist/dev/mesos/1.5.0-rc2/mesos
> -1.5.0.tar.gz.md5
>
> The signature of the tarball can be found at:
> https://dist.apache.org/repos/dist/dev/mesos/1.5.0-rc2/mesos
> -1.5.0.tar.gz.asc
>
> The PGP key used to sign the release is here:
> https://dist.apache.org/repos/dist/release/mesos/KEYS
>
> The JAR is in a staging repository here:
> https://repository.apache.org/content/repositories/orgapachemesos-1222
>
> Please vote on releasing this package as Apache Mesos 1.5.0!
>
> The vote is open until Tue Feb  6 17:35:16 PST 2018 and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Mesos 1.5.0
> [ ] -1 Do not release this package because ...
>
> Thanks,
> Jie and Gilbert
>

Re: Struggling with running Docker container on Windows agent

2018-02-02 Thread Andrew Schwartzmeyer 
 Hello,

Would you please provide me with the executor's stderr log? This can be
found in the work directory on the agent, it should give us a bit more
information as to why it failed to start the task.

It'll be deeply nested, something like:

c:mesoswork_dirslaves7dc02270-a4e1-4f59-9ad7-56bad5182ea4-S3frameworkseb32cef4-c503-4ab7-85d4-8d4577e6a3bf-executorsnotepad.fcf078d1-084a-11e8-8f77-02421c3bc93crunslateststderr
(and stdout)

Thanks,

Andy

On 02/02/2018 1:30 pm, ajkf9uvxc ajkf9uvxc wrote: 

> Hi, 
> 
> I am trying to get a job in DCOS to run a docker container on a Windows agent 
> machine. DCOS was installed using the AWS CF template here: 
> https://downloads.dcos.io/dcos/stable/aws.html [1] (single master). 
> 
> The Windows agent is added: 
> 
> C:mesosmesosbuildsrcmesos-agent.exe --attributes=os:windows 
> --containerizers=docker,mesos --hostname=10.19.10.206 --IP=10.19.10.206 
> --master=zk://10.22.1.94:2181/mesos [2] --work_dir=c:mesoswork_dir 
> --launcher_dir=c:mesosmesosbuildsrc --log_dir=c:mesoslogs 
> 
> And a simple job works: 
> 
> dcos.activestate.com [3] -> Job -> New 
> 
> { 
> 
> "id": "mywindowstest01", 
> 
> "labels": {}, 
> 
> "run": { 
> 
> "cpus": 0.01, 
> 
> "mem": 128, 
> 
> "disk": 0, 
> 
> "cmd": "C:\Windows\System32\cmd.exe /c echo helloworld > 
> c:\mesos\work_dir\helloworld2", 
> 
> "env": {}, 
> 
> "placement": { 
> 
> "constraints": [ 
> 
> { 
> 
> "attribute": "os", 
> 
> "operator": "EQ", 
> 
> "value": "windows" 
> 
> } 
> 
> ] 
> 
> }, 
> 
> "artifacts": [], 
> 
> "maxLaunchDelay": 3600, 
> 
> "volumes": [], 
> 
> "restart": { 
> 
> "policy": "NEVER" 
> 
> } 
> 
> }, 
> 
> "schedules": [] 
> 
> } 
> 
> creates: "c:\mesos\work_dir\helloworld2" 
> 
> The Windows agent has DockerCE installed and is set to run Windows containers 
> (tried with Linux containers as well and getting the same problem, but for 
> the purpose of this question let's stick to Windows containers) 
> 
> I confirmed that it's possible to run a Windows container manually, directly 
> on Windows 10 by starting a Powershell as Administrator and running: 
> 
> docker run -ti microsoft/windowsservercore 
> and 
> 
> docker run microsoft/windowsservercore 
> 
> Both commands create a new container (verified with "docker ps" , besides I 
> get a cmd.exe shell in the conatiner for the first command) 
> 
> Now the problem: 
> 
> trying to run a container from DCOS does not work: 
> 
> dcos job add a.json 
> 
> with the json: 
> 
> { 
> "id": "myattempt11", 
> "labels": {}, 
> "run": { 
> "env": {}, 
> "cpus": 1.00, 
> "mem": 512, 
> "disk": 1000, 
> "placement": { 
> "constraints": [ 
> { 
> "attribute": "os", 
> "operator": "EQ", 
> "value": "windows" 
> } 
> ] 
> }, 
> "artifacts": [], 
> "maxLaunchDelay": 3600, 
> "docker": { 
> "image": "microsoft/windowsservercore" 
> }, 
> "restart": { 
> "policy": "NEVER" 
> } 
> }, 
> "schedules": [] 
> } 
> 
> Does not work: 
> 
> # dcos job add a.json 
> 
> # dcos job run myattempt11 
> Run ID: 20180202203339zVpxc 
> 
> The log on the Mesos Agent on Windows shows activity but not much information 
> about the problem (see "TASK_FAILED" at the end below): 
> 
> Log file created at: 2018/02/02 12:52:47 
> Running on machine: DESKTOP-JJK06UJ 
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg 
> I0202 12:52:47.330880 8388 logging.cpp:201] INFO level logging started! 
> I0202 12:52:47.335886 8388 main.cpp:365] Build: 2017-12-20 23:35:42 UTC by 
> Anne S Bell 
> I0202 12:52:47.335886 8388 main.cpp:366] Version: 1.5.0 
> I0202 12:52:47.337895 8388 main.cpp:373] Git SHA: 
> 327726d3c7272806c8f3c3b7479758c26e55fd43 
> I0202 12:52:47.35 8388 resolver.cpp:69] Creating default secret resolver 
> I0202 12:52:47.574883 8388 containerizer.cpp:304] Using isolation { 
> windows/cpu, filesystem/windows, windows/mem, environment_secret } 
> I0202 12:52:47.577883 8388 provisioner.cpp:299] Using default backend 'copy' 
> I0202 12:52:47.596886 3348 slave.cpp:262] Mesos agent started on 
> (1)@10.19.10.206:5051 [4] 
> I0202 12:52:47.597883 3348 slave.cpp:263] Flags at startup: 
> --appc_simple_discovery_uri_prefix="http://; 
> --appc_store_dir="C:UsersactiveitAppDataLocalTempmesosstoreappc" 
> --attributes="os:windows" --authenticate_http_readonly="false" 
> --authenticate_http_readwrite="false" --authenticatee="crammd5" 
> --authentication_backoff_factor="1secs" --authorizer="local" 
> --container_disk_watch_interval="15secs" --containerizers="docker,mesos" 
> --default_role="*" --disk_watch_interval="1mins" --docker="docker" 
> --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io 
> [5]" --docker_remove_delay="6hrs" --docker_socket="//./pipe/docker_engine" 
> --docker_stop_timeout="0ns" 
> --docker_store_dir="C:UsersactiveitAppDataLocalTempmesosstoredocker" 
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
> --enforce_container_disk_quota="false" 
> --executor_registration_timeout="1mins" 
>

Struggling with running Docker container on Windows agent

2018-02-02 Thread ajkf9uvxc ajkf9uvxc 
Hi,
I am trying to get a job in DCOS to run a docker container on a Windows agent 
machine. DCOS was installed using the AWS CF template here: 
https://downloads.dcos.io/dcos/stable/aws.html (single master).
The Windows agent is added:
C:\mesos\mesos\build\src\mesos-agent.exe --attributes=os:windows 
--containerizers=docker,mesos --hostname=10.19.10.206 --IP=10.19.10.206 
--master=zk://10.22.1.94:2181/mesos --work_dir=c:\mesos\work_dir 
--launcher_dir=c:\mesos\mesos\build\src --log_dir=c:\mesos\logs

And a simple job works:

dcos.activestate.com -> Job -> New



{

  "id": "mywindowstest01",

  "labels": {},

  "run": {

    "cpus": 0.01,

    "mem": 128,

    "disk": 0,

    "cmd": "C:\\Windows\\System32\\cmd.exe /c echo helloworld > 
c:\\mesos\\work_dir\\helloworld2",

    "env": {},

    "placement": {

  "constraints": [

    {

  "attribute": "os",

  "operator": "EQ",

  "value": "windows"

    }

  ]

    },

    "artifacts": [],

    "maxLaunchDelay": 3600,

    "volumes": [],

    "restart": {

  "policy": "NEVER"

    }

  },

  "schedules": []

}

creates: "c:\\mesos\\work_dir\\helloworld2"

The Windows agent has DockerCE installed and is set to run Windows containers 
(tried with Linux containers as well and getting the same problem, but for the 
purpose of this question let's stick to Windows containers)
I confirmed that it's possible to run a Windows container manually, directly on 
Windows 10 by starting a Powershell as Administrator and running:
docker run -ti microsoft/windowsservercoreand docker run 
microsoft/windowsservercore
Both commands create a new container (verified with "docker ps" , besides I get 
a cmd.exe shell in the conatiner for the first command)
Now the problem:
trying to run a container from DCOS does not work:

dcos job add a.json

with the json:

{  "id": "myattempt11",  "labels": {},  "run": {    "env": {},    "cpus": 1.00, 
   "mem": 512,    "disk": 1000,    "placement": {      "constraints": [        
{          "attribute": "os",          "operator": "EQ",          "value": 
"windows"        }      ]    },    "artifacts": [],    "maxLaunchDelay": 3600,  
  "docker": {      "image": "microsoft/windowsservercore"    },    "restart": { 
     "policy": "NEVER"    }  },  "schedules": []}
Does not work:
# dcos job add a.json
# dcos job run myattempt11 
Run ID: 20180202203339zVpxc
The log on the Mesos Agent on Windows shows activity but not much information 
about the problem (see "TASK_FAILED" at the end below):
Log file created at: 2018/02/02 12:52:47Running on machine: DESKTOP-JJK06UJLog 
line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msgI0202 
12:52:47.330880  8388 logging.cpp:201] INFO level logging started!I0202 
12:52:47.335886  8388 main.cpp:365] Build: 2017-12-20 23:35:42 UTC by Anne S 
BellI0202 12:52:47.335886  8388 main.cpp:366] Version: 1.5.0I0202 
12:52:47.337895  8388 main.cpp:373] Git SHA: 
327726d3c7272806c8f3c3b7479758c26e55fd43I0202 12:52:47.35  8388 
resolver.cpp:69] Creating default secret resolverI0202 12:52:47.574883  8388 
containerizer.cpp:304] Using isolation { windows/cpu, filesystem/windows, 
windows/mem, environment_secret }I0202 12:52:47.577883  8388 
provisioner.cpp:299] Using default backend 'copy'I0202 12:52:47.596886  3348 
slave.cpp:262] Mesos agent started on (1)@10.19.10.206:5051I0202 
12:52:47.597883  3348 slave.cpp:263] Flags at startup: 
--appc_simple_discovery_uri_prefix="http://; 
--appc_store_dir="C:\Users\activeit\AppData\Local\Temp\mesos\store\appc" 
--attributes="os:windows" --authenticate_http_readonly="false" 
--authenticate_http_readwrite="false" --authenticatee="crammd5" 
--authentication_backoff_factor="1secs" --authorizer="local" 
--container_disk_watch_interval="15secs" --containerizers="docker,mesos" 
--default_role="*" --disk_watch_interval="1mins" --docker="docker" 
--docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; 
--docker_remove_delay="6hrs" --docker_socket="//./pipe/docker_engine" 
--docker_stop_timeout="0ns" 
--docker_store_dir="C:\Users\activeit\AppData\Local\Temp\mesos\store\docker" 
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins" 
--executor_reregistration_timeout="2secs" 
--executor_shutdown_grace_period="5secs" 
--fetcher_cache_dir="C:\Users\activeit\AppData\Local\Temp\mesos\fetch" 
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" 
--gc_disk_headroom="0.1" --hadoop_home="" --help="false" 
--hostname="10.19.10.206" --hostname_lookup="true" 
--http_command_executor="false" --http_heartbeat_interval="30secs" 
--initialize_driver_logging="true" --ip="10.19.10.206" 
--isolation="windows/cpu,windows/mem" --launcher="windows" 
--launcher_dir="c:\mesos\mesos\build\src" --log_dir="c:\mesos\logs" 
--logbufsecs="0" --logging_level="INFO" --master="zk://10.22.1.94:2181/mesos"