Re: Review Request 52787: Reorganized the work directory and the runtime directory in local mode.

2016-10-15 Thread haosdent huang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52787/
---

(Updated Oct. 16, 2016, 5:03 a.m.)


Review request for mesos, Jie Yu and Kevin Klues.


Bugs: MESOS-6380
https://issues.apache.org/jira/browse/MESOS-6380


Repository: mesos


Description
---

Add a new flag `temp_dir` when running Mesos in local mode. It is the
parent directory to create the work directory and other directories for
master and agents automatically.


Diffs
-

  bin/mesos-local-flags.sh.in 5b4553a808dc9f34a15390e69b2f85e95761ec53 
  src/local/flags.hpp c77eff13e7a63630ff6c9428e57f3f6707e1f30f 
  src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 

Diff: https://reviews.apache.org/r/52787/diff/


Testing (updated)
---

```
$ ./src/mesos-local --num_slaves=2
```

```
$ tree /tmp/mesos/local/
/tmp/mesos/local/
??? agent
?   ??? 0
?   ?   ??? runtime_dir
?   ?   ??? work_dir
?   ?   ??? meta
?   ?   ?   ??? boot_id
?   ?   ?   ??? slaves
?   ?   ?   ??? 9a676b40-8118-47ed-a2a2-d6b3e84b9525-S1
?   ?   ?   ?   ??? slave.info
?   ?   ?   ??? latest -> 
/tmp/mesos/local/agent/0/work_dir/meta/slaves/9a676b40-8118-47ed-a2a2-d6b3e84b9525-S1
?   ?   ??? provisioner
?   ??? 1
?   ??? runtime_dir
?   ??? work_dir
?   ??? meta
?   ?   ??? boot_id
?   ?   ??? slaves
?   ?   ??? 9a676b40-8118-47ed-a2a2-d6b3e84b9525-S0
?   ?   ?   ??? slave.info
?   ?   ??? latest -> 
/tmp/mesos/local/agent/1/work_dir/meta/slaves/9a676b40-8118-47ed-a2a2-d6b3e84b9525-S0
?   ??? provisioner
??? master
??? work_dir
??? replicated_log
??? 04.log
??? CURRENT
??? LOCK
??? LOG
??? MANIFEST-02

20 directories, 9 files
```


Thanks,

haosdent huang

Re: Review Request 52787: Reorganized the work directory and the runtime directory in local mode.

2016-10-15 Thread haosdent huang 


> On Oct. 15, 2016, midnight, Kevin Klues wrote:
> > This will no longer be necessary after https://reviews.apache.org/r/52902 
> > lands.

I think we still need this because when launch multiple agents in local mode, 
their runtime directories would conflict.


- haosdent


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52787/#review152763
---


On Oct. 16, 2016, 4:59 a.m., haosdent huang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52787/
> ---
> 
> (Updated Oct. 16, 2016, 4:59 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Kevin Klues.
> 
> 
> Bugs: MESOS-6380
> https://issues.apache.org/jira/browse/MESOS-6380
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Add a new flag `temp_dir` when running Mesos in local mode. It is the
> parent directory to create the work directory and other directories for
> master and agents automatically.
> 
> 
> Diffs
> -
> 
>   bin/mesos-local-flags.sh.in 5b4553a808dc9f34a15390e69b2f85e95761ec53 
>   src/local/flags.hpp c77eff13e7a63630ff6c9428e57f3f6707e1f30f 
>   src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 
> 
> Diff: https://reviews.apache.org/r/52787/diff/
> 
> 
> Testing
> ---
> 
> To void this error when launch `mesos-local` without `sudo`
> 
> ```
>   message: 'Failed to launch container: Failed to make the containerizer 
> runtime directory 
> '/var/run/mesos/containers/f2d6947f-2916-4f1a-90dc-3d137b360b9c': Permission 
> denied; Abnormal executor termination: unknown container'
> ```
> 
> Test produce 
> 
> ```
> # Run with normal user.
> $ ./bin/mesos-local.sh
> 
> # Open a new session
> $ ./src/mesos-execute --master=127.0.0.1:5050 --name="test-single-1" 
> --command="sleep 2000"
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> W1014 00:05:22.457461 36489 scheduler.cpp:161]
> **
> Scheduler driver bound to loopback interface! Cannot communicate with remote 
> master(s). You might want to set 'LIBPROCESS_IP' environment variable to use 
> a routable IP address.
> **
> I1014 00:05:22.457690 36489 scheduler.cpp:176] Version: 1.1.0
> I1014 00:05:22.596534 36497 scheduler.cpp:465] New master detected at 
> master@127.0.0.1:5050
> Subscribed with ID 51e0d33f-d7b2-4c01-9bb3-10731cc7d1ab-
> Submitted task 'test-single-1' to agent 
> '8fba7ecb-b70e-48ac-9541-2070c4d9b1d5-S0'
> Received status update TASK_RUNNING for task 'test-single-1'
>   source: SOURCE_EXECUTOR
> ```
> 
> ```
> $ sudo make install
> $ mesos-local
> ...
> I1014 12:23:47.563256 4284416 slave.cpp:208] Mesos agent started on 
> (1)@127.0.0.1:5050
> I1014 12:23:47.563302 4284416 slave.cpp:209] Flags at startup: 
> --appc_simple_discovery_uri_prefix="http://; 
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" 
> --authenticate_http_readwrite="false" --authenticatee="crammd5" 
> --authentication_backoff_factor="1secs" --authorizer="local" 
> --container_disk_watch_interval="15secs" --containerizers="mesos" 
> --default_role="*" --disk_watch_interval="1mins" --docker="docker" 
> --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; 
> --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" 
> --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" 
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
> --enforce_container_disk_quota="false" 
> --executor_registration_timeout="1mins" 
> --executor_shutdown_grace_period="5secs" 
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" 
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_
 home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" 
--http_command_executor="false" --image_provisioner_backend="copy" 
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" 
--launcher="posix" --launcher_dir="/usr/local/libexec/mesos" --logbufsecs="0" 
--logging_level="INFO" --max_completed_executors_per_framework="150" 
--oversubscribed_resources_interval="15secs" 
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" 
--recovery_timeout="15mins" --registration_backoff_factor="1secs" 
--runtime_dir="/tmp/mesos/local/0" --sandbox_directory="/mnt/mesos/sandbox" 
--strict="true" --switch_user="true" --version="false" 
--work_dir="/tmp/mesos/local/0"
> ...
> 
> $ mesos-execute --master=127.0.0.1:5050 --name="test-single-1" 
> --command="sleep 2000"
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> W1014 12:25:39.922174 3211264 scheduler.cpp:161]
>

Re: Review Request 52787: Reorganized the work directory and the runtime directory in local mode.

2016-10-15 Thread haosdent huang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52787/
---

(Updated Oct. 16, 2016, 4:59 a.m.)


Review request for mesos, Jie Yu and Kevin Klues.


Changes
---

Address @klueska's comments.


Summary (updated)
-

Reorganized the work directory and the runtime directory in local mode.


Bugs: MESOS-6380
https://issues.apache.org/jira/browse/MESOS-6380


Repository: mesos


Description (updated)
---

Add a new flag `temp_dir` when running Mesos in local mode. It is the
parent directory to create the work directory and other directories for
master and agents automatically.


Diffs (updated)
-

  bin/mesos-local-flags.sh.in 5b4553a808dc9f34a15390e69b2f85e95761ec53 
  src/local/flags.hpp c77eff13e7a63630ff6c9428e57f3f6707e1f30f 
  src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 

Diff: https://reviews.apache.org/r/52787/diff/


Testing
---

To void this error when launch `mesos-local` without `sudo`

```
  message: 'Failed to launch container: Failed to make the containerizer 
runtime directory 
'/var/run/mesos/containers/f2d6947f-2916-4f1a-90dc-3d137b360b9c': Permission 
denied; Abnormal executor termination: unknown container'
```

Test produce 

```
# Run with normal user.
$ ./bin/mesos-local.sh

# Open a new session
$ ./src/mesos-execute --master=127.0.0.1:5050 --name="test-single-1" 
--command="sleep 2000"
WARNING: Logging before InitGoogleLogging() is written to STDERR
W1014 00:05:22.457461 36489 scheduler.cpp:161]
**
Scheduler driver bound to loopback interface! Cannot communicate with remote 
master(s). You might want to set 'LIBPROCESS_IP' environment variable to use a 
routable IP address.
**
I1014 00:05:22.457690 36489 scheduler.cpp:176] Version: 1.1.0
I1014 00:05:22.596534 36497 scheduler.cpp:465] New master detected at 
master@127.0.0.1:5050
Subscribed with ID 51e0d33f-d7b2-4c01-9bb3-10731cc7d1ab-
Submitted task 'test-single-1' to agent 
'8fba7ecb-b70e-48ac-9541-2070c4d9b1d5-S0'
Received status update TASK_RUNNING for task 'test-single-1'
  source: SOURCE_EXECUTOR
```

```
$ sudo make install
$ mesos-local
...
I1014 12:23:47.563256 4284416 slave.cpp:208] Mesos agent started on 
(1)@127.0.0.1:5050
I1014 12:23:47.563302 4284416 slave.cpp:209] Flags at startup: 
--appc_simple_discovery_uri_prefix="http://; 
--appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" 
--authenticate_http_readwrite="false" --authenticatee="crammd5" 
--authentication_backoff_factor="1secs" --authorizer="local" 
--container_disk_watch_interval="15secs" --containerizers="mesos" 
--default_role="*" --disk_watch_interval="1mins" --docker="docker" 
--docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; 
--docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" 
--docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" 
--docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
--enforce_container_disk_quota="false" --executor_registration_timeout="1mins" 
--executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" 
--fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" 
--gc_disk_headroom="0.1" --hadoop_ho
 me="" --help="false" --hostname_lookup="true" --http_authenticators="basic" 
--http_command_executor="false" --image_provisioner_backend="copy" 
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" 
--launcher="posix" --launcher_dir="/usr/local/libexec/mesos" --logbufsecs="0" 
--logging_level="INFO" --max_completed_executors_per_framework="150" 
--oversubscribed_resources_interval="15secs" 
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" 
--recovery_timeout="15mins" --registration_backoff_factor="1secs" 
--runtime_dir="/tmp/mesos/local/0" --sandbox_directory="/mnt/mesos/sandbox" 
--strict="true" --switch_user="true" --version="false" 
--work_dir="/tmp/mesos/local/0"
...

$ mesos-execute --master=127.0.0.1:5050 --name="test-single-1" --command="sleep 
2000"
WARNING: Logging before InitGoogleLogging() is written to STDERR
W1014 12:25:39.922174 3211264 scheduler.cpp:161]
**
Scheduler driver bound to loopback interface! Cannot communicate with remote 
master(s). You might want to set 'LIBPROCESS_IP' environment variable to use a 
routable IP address.
**
I1014 12:25:39.922353 3211264 scheduler.cpp:176] Version: 1.1.0
I1014 12:25:39.927863 3211264 scheduler.cpp:465] New master detected at 
master@127.0.0.1:5050
Subscribed with ID b3af099b-8ee5-4fbe-b27b-fe4832b141c2-0001
Submitted task 'test-single-1' to agent 
'ed8e545b-d1e5-4d22-bc88-517be5632779-S0'
Received status update

Re: Review Request 52856: Reverted incorrect changes in 1c2ee5c.

2016-10-15 Thread haosdent huang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52856/
---

(Updated Oct. 16, 2016, 4:59 a.m.)


Review request for mesos, Jie Yu, Kevin Klues, and Vinod Kone.


Changes
---

Rebase.


Bugs: MESOS-6380
https://issues.apache.org/jira/browse/MESOS-6380


Repository: mesos


Description
---

Reverted incorrect changes in 1c2ee5c.


Diffs (updated)
-

  src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 

Diff: https://reviews.apache.org/r/52856/diff/


Testing
---


Thanks,

haosdent huang

Re: Review Request 52856: Reverted incorrect changes in 1c2ee5c.

2016-10-15 Thread haosdent huang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52856/
---

(Updated Oct. 16, 2016, 4:57 a.m.)


Review request for mesos, Jie Yu, Kevin Klues, and Vinod Kone.


Changes
---

Address @vinodkone's comments.


Bugs: MESOS-6380
https://issues.apache.org/jira/browse/MESOS-6380


Repository: mesos


Description
---

Reverted incorrect changes in 1c2ee5c.


Diffs (updated)
-

  src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 

Diff: https://reviews.apache.org/r/52856/diff/


Testing
---


Thanks,

haosdent huang

Re: Review Request 51617: Added the logic for installing and removing DNAT rules.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51617/#review152796
---



Patch looks great!

Reviews applied: [52671, 52771, 52772, 52773, 51617]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 16, 2016, 2:06 a.m., Avinash sridharan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51617/
> ---
> 
> (Updated Oct. 16, 2016, 2:06 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Qian Zhang.
> 
> 
> Bugs: MESOS-6023
> https://issues.apache.org/jira/browse/MESOS-6023
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added the logic for installing and removing DNAT rules.
> 
> 
> Diffs
> -
> 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.hpp
>  7fad707a240234e35828917aea1bc79f42fe130e 
>   
> src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
>  2ff8b0e76a11b6f6c98b839d3ac91a81e41285f5 
> 
> Diff: https://reviews.apache.org/r/51617/diff/
> 
> 
> Testing
> ---
> 
> Ran the CNI plugin against a network namespace with the following JSON input:
> ```
> {
> "name": "mynet",
> "type": "port-mapper",
> "chain": "MESOS-TEST",
> "excludeDevices": ["mesos-cni0"],
> "delegate": {
>   "type" : "bridge",
>   "bridge": "cni0",
>   "isGateway": true,
>   "ipMasq": true,
>   "ipam": {
>   "type": "host-local",
>   "subnet": "192.168.37.0/24",
>   "routes": [
> { "dst": "0.0.0.0/0" }
>   ]
>   }
> },
> "args" : {
>   "org.apache.mesos" : {
> "network_info" : {
>   "port_mappings": {
> "host_port" : 8080,
> "container_port" : 9000
>   }
> }
>   }
> }
> }
> ```
> 
> Used the ADD command to test that the CNI plugin correctly invokes the 
> delegate plugin (a CNI bridge plugin in this case) and also inserts the 
> correct iptable entries for the given port mapping. After running this 
> plugin, this was the output of the `iptables -t nat -S MESOS-TEST` command:
> ```
> sudo iptables -t nat -S MESOS-TEST
> -N MESOS-TEST
> -A MESOS-TEST ! -i mesos-cni0 -p tcp -m tcp --dport 8080 -j DNAT 
> --to-destination 192.168.37.21:9000
> ```
> 
> Ran a python HTTP server in this network namespace and verified that DNAT 
> works from outside the box. Was able to connect to port 9000 of this server, 
> by connecting to port 8080 on the host.
> 
> Used the DEL command to test the CNI plugin correctly deletes the DNAT rule 
> and chain, if there are no DNAT rules exist in the chain. After running the 
> DEL command (by injecting `NetworkInfo` into the above JSON schema) verified 
> the chain and the DNAT rule is deleted from iptables.
> 
> 
> Apart from these tests ran a single node cluster and did an end-to-end test 
> with a modified `mesos-execute` binary that can setup port-mapping.
> 
> 
> Thanks,
> 
> Avinash sridharan
> 
>

Re: Review Request 51617: Added the logic for installing and removing DNAT rules.

2016-10-15 Thread Avinash sridharan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51617/
---

(Updated Oct. 16, 2016, 2:06 a.m.)


Review request for mesos, Jie Yu and Qian Zhang.


Changes
---

Re-factored the code to use `comments` as the key to identify the iptable rules.


Summary (updated)
-

Added the logic for installing and removing DNAT rules.


Bugs: MESOS-6023
https://issues.apache.org/jira/browse/MESOS-6023


Repository: mesos


Description (updated)
---

Added the logic for installing and removing DNAT rules.


Diffs (updated)
-

  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.hpp
 7fad707a240234e35828917aea1bc79f42fe130e 
  
src/slave/containerizer/mesos/isolators/network/cni/plugins/port_mapper/port_mapper.cpp
 2ff8b0e76a11b6f6c98b839d3ac91a81e41285f5 

Diff: https://reviews.apache.org/r/51617/diff/


Testing (updated)
---

Ran the CNI plugin against a network namespace with the following JSON input:
```
{
"name": "mynet",
"type": "port-mapper",
"chain": "MESOS-TEST",
"excludeDevices": ["mesos-cni0"],
"delegate": {
  "type" : "bridge",
  "bridge": "cni0",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
  "type": "host-local",
  "subnet": "192.168.37.0/24",
  "routes": [
{ "dst": "0.0.0.0/0" }
  ]
  }
},
"args" : {
  "org.apache.mesos" : {
"network_info" : {
  "port_mappings": {
"host_port" : 8080,
"container_port" : 9000
  }
}
  }
}
}
```

Used the ADD command to test that the CNI plugin correctly invokes the delegate 
plugin (a CNI bridge plugin in this case) and also inserts the correct iptable 
entries for the given port mapping. After running this plugin, this was the 
output of the `iptables -t nat -S MESOS-TEST` command:
```
sudo iptables -t nat -S MESOS-TEST
-N MESOS-TEST
-A MESOS-TEST ! -i mesos-cni0 -p tcp -m tcp --dport 8080 -j DNAT 
--to-destination 192.168.37.21:9000
```

Ran a python HTTP server in this network namespace and verified that DNAT works 
from outside the box. Was able to connect to port 9000 of this server, by 
connecting to port 8080 on the host.

Used the DEL command to test the CNI plugin correctly deletes the DNAT rule and 
chain, if there are no DNAT rules exist in the chain. After running the DEL 
command (by injecting `NetworkInfo` into the above JSON schema) verified the 
chain and the DNAT rule is deleted from iptables.


Apart from these tests ran a single node cluster and did an end-to-end test 
with a modified `mesos-execute` binary that can setup port-mapping.


Thanks,

Avinash sridharan

Re: Review Request 52827: Added backend suffix to image layer rootfs path.

2016-10-15 Thread Qian Zhang 


> On Oct. 14, 2016, 7:34 a.m., Jie Yu wrote:
> > src/slave/containerizer/mesos/provisioner/docker/paths.cpp, lines 61-64
> > 
> >
> > In fact, i think we should use aufs style whiteout by default because 
> > that's also the OCI standard. So the logic here should be:
> > ```
> > if (backend == "overlay") {
> >   return path::join(layerPath, "rootfs." + backend);
> > }
> > 
> > return path::join(layerPath, "rootfs");
> > ```
> > 
> > Also, I think we should define constants for "overlay" here as it has 
> > been referenced multiple times.
> 
> Qian Zhang wrote:
> Thanks for the comments! And I will address the constants in a separate 
> patch later.

Here is the patch for the constants: https://reviews.apache.org/r/52910/


- Qian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52827/#review152585
---


On Oct. 14, 2016, 11:36 p.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52827/
> ---
> 
> (Updated Oct. 14, 2016, 11:36 p.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Bugs: MESOS-6360
> https://issues.apache.org/jira/browse/MESOS-6360
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Previously image layer rootfs path is in the format below regardless
> of which backend is used.
>   /layers//rootfs
> This introduced an issue: when agent is restarted with a different
> backend, we will wrongly handle the whiteout files since different
> backends(e.g., aufs and overlay) have different whiteout standard.
> 
> In this commit, we added backend suffix to image layer rootfs path
> for overlay backend like below.
>   /layers//rootfs.overlay
> For non-overlay backends, it is still in the previous format, this
> is because they share the same whiteout standard (aufs standard),
> and also used to handle backward compatibility.
> 
> So the expected result of this commit is:
> 1. If user switches backend from overlay to a non-overlay (or vice
> versa) when restarting agent, all the image layers of the previous
> backend in the store will just be ignored.
> 2. In the upgrade case, if user starts a new version of Mesos agent
> with overlay backend, then all the image layers in the store pulled
> by the old agent will just be ignored, but if user starts the new
> agent with a non-overlay backend, then all the image layers in the
> store can still be used.
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/mesos/provisioner/docker/local_puller.cpp 
> 9b09dca5cd8f9e60a90cf574300b250eb18ede35 
>   src/slave/containerizer/mesos/provisioner/docker/metadata_manager.cpp 
> 6456d90220a4026696a04f9969763cf682464353 
>   src/slave/containerizer/mesos/provisioner/docker/paths.hpp 
> 949e0c16a361f22b00e267fcf81093690327041f 
>   src/slave/containerizer/mesos/provisioner/docker/paths.cpp 
> a5cc952072274c61e8c515e8b1b7ea563344e950 
>   src/slave/containerizer/mesos/provisioner/docker/registry_puller.cpp 
> 1e2cc2dda3518d99ef1ad06e2b8ea7f78a4dcf72 
>   src/slave/containerizer/mesos/provisioner/docker/store.cpp 
> 52b9ea934a1dbe3312b8f08f5da8085e9e306de5 
>   src/tests/containerizer/provisioner_docker_tests.cpp 
> 10fbc4149ac2e7503ffe7f2746fbd0e14a2365b4 
> 
> Diff: https://reviews.apache.org/r/52827/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>

Re: Review Request 52887: Added a CHANGELOG description for partition-aware frameworks.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52887/#review152788
---



Patch looks great!

Reviews applied: [52887]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 15, 2016, 2:34 p.m., Neil Conway wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52887/
> ---
> 
> (Updated Oct. 15, 2016, 2:34 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov and Vinod Kone.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added a CHANGELOG description for partition-aware frameworks.
> 
> 
> Diffs
> -
> 
>   CHANGELOG b649a5e415a56ac30aa4bf511bf180777e8b2723 
> 
> Diff: https://reviews.apache.org/r/52887/diff/
> 
> 
> Testing
> ---
> 
> Visual inspection.
> 
> 
> Thanks,
> 
> Neil Conway
> 
>

Re: Review Request 52910: Added a constants.hpp for provisioner.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52910/#review152785
---



Patch looks great!

Reviews applied: [52910]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 15, 2016, 1:39 p.m., Qian Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52910/
> ---
> 
> (Updated Oct. 15, 2016, 1:39 p.m.)
> 
> 
> Review request for mesos and Jie Yu.
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added a constants.hpp for provisioner.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 3bcc0f2dfc2c4f71841bd6d161f39e0e919fc0d7 
>   src/slave/containerizer/mesos/provisioner/backend.cpp 
> 9c258ed2cad82c2c1b1f707acf527239a1f01f95 
>   src/slave/containerizer/mesos/provisioner/constants.hpp PRE-CREATION 
>   src/slave/containerizer/mesos/provisioner/docker/paths.cpp 
> dea37566cbe0313f84f2bac27e29752a980b4394 
>   src/slave/flags.cpp 87d9e4632321134192bb0a67f1b91db7d89f539b 
>   src/tests/containerizer/provisioner_appc_tests.cpp 
> a999fc7991da805dfbcdf5659fcfb762aee5b2b9 
>   src/tests/containerizer/provisioner_backend_tests.cpp 
> 661977b61c0711c450922a64ce0e2895385bf78d 
> 
> Diff: https://reviews.apache.org/r/52910/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>

Re: Review Request 52887: Added a CHANGELOG description for partition-aware frameworks.

2016-10-15 Thread Neil Conway 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52887/
---

(Updated Oct. 15, 2016, 2:34 p.m.)


Review request for mesos, Alexander Rukletsov and Vinod Kone.


Changes
---

Rebase.


Repository: mesos


Description
---

Added a CHANGELOG description for partition-aware frameworks.


Diffs (updated)
-

  CHANGELOG b649a5e415a56ac30aa4bf511bf180777e8b2723 

Diff: https://reviews.apache.org/r/52887/diff/


Testing
---

Visual inspection.


Thanks,

Neil Conway

Review Request 52910: Added a constants.hpp for provisioner.

2016-10-15 Thread Qian Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52910/
---

Review request for mesos and Jie Yu.


Repository: mesos


Description
---

Added a constants.hpp for provisioner.


Diffs
-

  src/Makefile.am 3bcc0f2dfc2c4f71841bd6d161f39e0e919fc0d7 
  src/slave/containerizer/mesos/provisioner/backend.cpp 
9c258ed2cad82c2c1b1f707acf527239a1f01f95 
  src/slave/containerizer/mesos/provisioner/constants.hpp PRE-CREATION 
  src/slave/containerizer/mesos/provisioner/docker/paths.cpp 
dea37566cbe0313f84f2bac27e29752a980b4394 
  src/slave/flags.cpp 87d9e4632321134192bb0a67f1b91db7d89f539b 
  src/tests/containerizer/provisioner_appc_tests.cpp 
a999fc7991da805dfbcdf5659fcfb762aee5b2b9 
  src/tests/containerizer/provisioner_backend_tests.cpp 
661977b61c0711c450922a64ce0e2895385bf78d 

Diff: https://reviews.apache.org/r/52910/diff/


Testing
---

make check


Thanks,

Qian Zhang

Re: Review Request 52906: Simplify the comparison logic for `ExecutorInfo`.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52906/#review152778
---



Patch looks great!

Reviews applied: [52906]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 15, 2016, 5:06 a.m., haosdent huang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52906/
> ---
> 
> (Updated Oct. 15, 2016, 5:06 a.m.)
> 
> 
> Review request for mesos, Anand Mazumdar and Vinod Kone.
> 
> 
> Bugs: MESOS-6397
> https://issues.apache.org/jira/browse/MESOS-6397
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> This is a follow up fix of https://reviews.apache.org/r/52817/.
> 
> 
> Diffs
> -
> 
>   src/common/type_utils.cpp c6cf4f1d4acd040d21434221e5469ca911404a39 
> 
> Diff: https://reviews.apache.org/r/52906/diff/
> 
> 
> Testing
> ---
> 
> ```
> sudo GLOG_v=1 ./bin/mesos-tests.sh --gtest_filter="*TaskUsesExecutor*" 
> --verbose
> sudo GLOG_v=1 ./bin/mesos-tests.sh 
> --gtest_filter="*TaskGroupValidationTest.TaskUsesDifferentExecutor*" --verbose
> ```
> 
> 
> Thanks,
> 
> haosdent huang
> 
>

Re: Review Request 52787: Set `runtime_dir` to a temporary folder in `mesos-local`.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52787/#review152777
---



Patch looks great!

Reviews applied: [52856, 52787]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 14, 2016, 4:26 a.m., haosdent huang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52787/
> ---
> 
> (Updated Oct. 14, 2016, 4:26 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Kevin Klues.
> 
> 
> Bugs: MESOS-6380
> https://issues.apache.org/jira/browse/MESOS-6380
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Set `runtime_dir` to a temporary folder in `mesos-local`.
> 
> 
> Diffs
> -
> 
>   bin/mesos-local-flags.sh.in 5b4553a808dc9f34a15390e69b2f85e95761ec53 
>   src/local/flags.hpp c77eff13e7a63630ff6c9428e57f3f6707e1f30f 
>   src/local/local.cpp 257179443827ffdfc946ed655a6840fcea70d454 
> 
> Diff: https://reviews.apache.org/r/52787/diff/
> 
> 
> Testing
> ---
> 
> To void this error when launch `mesos-local` without `sudo`
> 
> ```
>   message: 'Failed to launch container: Failed to make the containerizer 
> runtime directory 
> '/var/run/mesos/containers/f2d6947f-2916-4f1a-90dc-3d137b360b9c': Permission 
> denied; Abnormal executor termination: unknown container'
> ```
> 
> Test produce 
> 
> ```
> # Run with normal user.
> $ ./bin/mesos-local.sh
> 
> # Open a new session
> $ ./src/mesos-execute --master=127.0.0.1:5050 --name="test-single-1" 
> --command="sleep 2000"
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> W1014 00:05:22.457461 36489 scheduler.cpp:161]
> **
> Scheduler driver bound to loopback interface! Cannot communicate with remote 
> master(s). You might want to set 'LIBPROCESS_IP' environment variable to use 
> a routable IP address.
> **
> I1014 00:05:22.457690 36489 scheduler.cpp:176] Version: 1.1.0
> I1014 00:05:22.596534 36497 scheduler.cpp:465] New master detected at 
> master@127.0.0.1:5050
> Subscribed with ID 51e0d33f-d7b2-4c01-9bb3-10731cc7d1ab-
> Submitted task 'test-single-1' to agent 
> '8fba7ecb-b70e-48ac-9541-2070c4d9b1d5-S0'
> Received status update TASK_RUNNING for task 'test-single-1'
>   source: SOURCE_EXECUTOR
> ```
> 
> ```
> $ sudo make install
> $ mesos-local
> ...
> I1014 12:23:47.563256 4284416 slave.cpp:208] Mesos agent started on 
> (1)@127.0.0.1:5050
> I1014 12:23:47.563302 4284416 slave.cpp:209] Flags at startup: 
> --appc_simple_discovery_uri_prefix="http://; 
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" 
> --authenticate_http_readwrite="false" --authenticatee="crammd5" 
> --authentication_backoff_factor="1secs" --authorizer="local" 
> --container_disk_watch_interval="15secs" --containerizers="mesos" 
> --default_role="*" --disk_watch_interval="1mins" --docker="docker" 
> --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; 
> --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" 
> --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" 
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" 
> --enforce_container_disk_quota="false" 
> --executor_registration_timeout="1mins" 
> --executor_shutdown_grace_period="5secs" 
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" 
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_
 home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" 
--http_command_executor="false" --image_provisioner_backend="copy" 
--initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" 
--launcher="posix" --launcher_dir="/usr/local/libexec/mesos" --logbufsecs="0" 
--logging_level="INFO" --max_completed_executors_per_framework="150" 
--oversubscribed_resources_interval="15secs" 
--qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" 
--recovery_timeout="15mins" --registration_backoff_factor="1secs" 
--runtime_dir="/tmp/mesos/local/0" --sandbox_directory="/mnt/mesos/sandbox" 
--strict="true" --switch_user="true" --version="false" 
--work_dir="/tmp/mesos/local/0"
> ...
> 
> $ mesos-execute --master=127.0.0.1:5050 --name="test-single-1" 
> --command="sleep 2000"
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> W1014 12:25:39.922174 3211264 scheduler.cpp:161]
> **
> Scheduler driver bound to loopback interface! Cannot communicate with remote 
> master(s). You might want to set

Re: Review Request 52880: Added "launcher_dir" to the default executor flags.

2016-10-15 Thread Mesos ReviewBot 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52880/#review152775
---



Patch looks great!

Reviews applied: [52878, 52879, 52880]

Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' 
CONFIGURATION='--verbose' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; 
./support/docker_build.sh

- Mesos ReviewBot


On Oct. 14, 2016, 5:18 p.m., Gastón Kleiman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52880/
> ---
> 
> (Updated Oct. 14, 2016, 5:18 p.m.)
> 
> 
> Review request for mesos, Alexander Rukletsov, Anand Mazumdar, haosdent 
> huang, Jie Yu, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6288
> https://issues.apache.org/jira/browse/MESOS-6288
> 
> 
> Repository: mesos
> 
> 
> Description
> ---
> 
> Added "launcher_dir" to the default executor flags.
> 
> 
> Diffs
> -
> 
>   src/launcher/default_executor.cpp af4a97f7de5f2157aa65fdab742455b0683c40a4 
>   src/slave/slave.cpp 6bd9b49c3bbdb973a0d03552ae8fe55b33371083 
> 
> Diff: https://reviews.apache.org/r/52880/diff/
> 
> 
> Testing
> ---
> 
> `make check`
> 
> 
> Thanks,
> 
> Gastón Kleiman
> 
>