Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60131
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 6, 2014, 4:12 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 4:12 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2254

[Vinod Kone]

Aha. Thanks Ian. 

@vinodkone

> On Nov 5, 2014, at 9:00 PM, Ian Downes  wrote:
> 
> Appears this was from that commit which introduced glog into the fetcher and 
> changed some expected output to include the initial glog message:
> 
> Value of: (os::read(path::join(directory, "stderr"))).get()
>   Actual: "I1106 02:30:53.472061 21707 logging.cpp:177] Logging to 
> STDERR\nthis is stderr\n"
> Expected: errMsg + "\n"
> Which is: "this is stderr\n"
> 
> I'll file a ticket in the morning.
> 
> Ian
> 
> 
> 
>> On Wed, Nov 5, 2014 at 8:48 PM, Vinod Kone  wrote:
>> @ian: can you triage this?
>> 
>>> On Wed, Nov 5, 2014 at 6:37 PM, Apache Jenkins Server 
>>>  wrote:
>>> See 
>>> 
>>> 
>>> Changes:
>>> 
>>> [vinodkone] Fetcher uses Hadoop to fetch URIs with unknown schemes.
>>> 
>>> --
>>> [...truncated 70150 lines...]
>>> I1106 02:35:19.677485 21521 master.cpp:1337] Authorizing framework 
>>> principal 'test-principal' to receive offers for role '*'
>>> I1106 02:35:19.678004 21530 master.cpp:1442] Registering framework 
>>> 20141106-023518-3125920579-59951-21504- (default) at 
>>> scheduler-f40b9448-6469-442b-abe8-89a9a1b215b0@67.195.81.186:59951
>>> I1106 02:35:19.678170 21526 group.cpp:717] Found non-sequence node 
>>> 'log_replicas' at '/znode' in ZooKeeper
>>> I1106 02:35:19.678387 21520 hierarchical_allocator_process.hpp:329] Added 
>>> framework 20141106-023518-3125920579-59951-21504-
>>> I1106 02:35:19.678413 21533 detector.cpp:138] Detected a new leader: 
>>> (id='1')
>>> I1106 02:35:19.678422 21520 hierarchical_allocator_process.hpp:697] No 
>>> resources available to allocate!
>>> I1106 02:35:19.678447 21520 hierarchical_allocator_process.hpp:659] 
>>> Performed allocation for 0 slaves in 24461ns
>>> I1106 02:35:19.678532 21531 sched.cpp:407] Framework registered with 
>>> 20141106-023518-3125920579-59951-21504-
>>> I1106 02:35:19.678622 21531 sched.cpp:421] Scheduler::registered took 
>>> 56953ns
>>> I1106 02:35:19.678834 21523 group.cpp:659] Trying to get 
>>> '/znode/info_01' in ZooKeeper
>>> I1106 02:35:19.680326 21528 detector.cpp:433] A new leading master 
>>> (UPID=master@67.195.81.186:59951) is detected
>>> I1106 02:35:19.680634 21531 status_update_manager.cpp:171] Pausing sending 
>>> status updates
>>> I1106 02:35:19.680631 21526 slave.cpp:602] New master detected at 
>>> master@67.195.81.186:59951
>>> I1106 02:35:19.680694 21526 slave.cpp:665] Authenticating with master 
>>> master@67.195.81.186:59951
>>> I1106 02:35:19.680857 21526 slave.cpp:638] Detecting new master
>>> I1106 02:35:19.680910 21529 authenticatee.hpp:133] Creating new client SASL 
>>> connection
>>> I1106 02:35:19.681162 21529 master.cpp:3896] Authenticating 
>>> slave(223)@67.195.81.186:59951
>>> I1106 02:35:19.681206 21529 master.cpp:3907] Using default CRAM-MD5 
>>> authenticator
>>> I1106 02:35:19.681447 21533 authenticator.hpp:169] Creating new server SASL 
>>> connection
>>> I1106 02:35:19.681614 21533 authenticatee.hpp:224] Received SASL 
>>> authentication mechanisms: CRAM-MD5
>>> I1106 02:35:19.681641 21533 authenticatee.hpp:250] Attempting to 
>>> authenticate with mechanism 'CRAM-MD5'
>>> I1106 02:35:19.681825 21525 authenticator.hpp:275] Received SASL 
>>> authentication start
>>> I1106 02:35:19.681977 21525 authenticator.hpp:397] Authentication requires 
>>> more steps
>>> I1106 02:35:19.682062 21525 authenticatee.hpp:270] Received SASL 
>>> authentication step
>>> I1106 02:35:19.682153 21525 authenticator.hpp:303] Received SASL 
>>> authentication step
>>> I1106 02:35:19.682185 21525 auxprop.cpp:99] Request to lookup properties 
>>> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: 
>>> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
>>> I1106 02:35:19.682199 21525 auxprop.cpp:171] Looking up auxiliary property 
>>> '*userPassword'
>>> I1106 02:35:19.682230 21525 auxprop.cpp:171] Looking up auxiliary property 
>>> '*cmusaslsecretCRAM-MD5'
>>> I1106 02:35:19.682255 21525 auxprop.cpp:99] Request to lookup properties 
>>> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: 
>>> 'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
>>> I1106 02:35:19.682266 21525 auxprop.cpp:121] Skipping auxiliary property 
>>> '*userPassword' since SASL_AUXPROP_AUTHZID == true
>>> I1106 02:35:19.682276 21525 auxprop.cpp:121] Skipping auxiliary property 
>>> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
>>> I1106 02:35:19.682291 21525 authenticator.hpp:389] Authentication success
>>> I1106 02:35:19.682391 21529 authenticatee.hpp:310] Authentication success
>>> I1106 02:35:19.682451 21525 master.cpp:3957] Successfully authenticated 
>>> principal 'test-principal' at slave(223)@67.195.81.186:59951
>

Re: Review Request 27670: Bug fix: Check for non-zero status code and hadoop client not found

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27670/
---

(Updated Nov. 6, 2014, 7:07 a.m.)


Review request for mesos, Benjamin Hindman and Vinod Kone.


Repository: mesos-git


Description
---

@benh noticed that the fetcher would detect the condition where hadoop returns 
a success in case of a success non zero value


Diffs
-

  src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 

Diff: https://reviews.apache.org/r/27670/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Review Request 27660: Made ProcessBase::route hard-fail rather than soft fail

[Adam B]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27660/#review60129
---


I see that none of the processes (master, slave, registrar, etc.) actually 
check the return value, so they wouldn't know if it failed; and I agree that 
not including '/' at the start of a route should hard fail. The code changes 
look shippable to me too.
But I'd like to loop in BenH for the final approval, since he originally wrote 
all this libprocess route code.

- Adam B


On Nov. 5, 2014, 6:24 p.m., Cody Maloney wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27660/
> ---
> 
> (Updated Nov. 5, 2014, 6:24 p.m.)
> 
> 
> Review request for mesos and Adam B.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> It is an API expectation that the path start with '/', and if it doesn't it 
> should be a programmer error.
> 
> No code actually checked the return code. If a developer did get this wrong 
> previously it would silently do nothing and in debugging their endpoint just 
> wouldn't appear. This makes it simple/easy for them to find.
> 
> 
> Diffs
> -
> 
>   3rdparty/libprocess/include/process/process.hpp 
> 81a1f7a3fece78bb7df1e86850572922c8704ea4 
>   3rdparty/libprocess/src/process.cpp 
> 85fb9958342f0bcdde322d9c55333126e6f86668 
> 
> Diff: https://reviews.apache.org/r/27660/diff/
> 
> 
> Testing
> ---
> 
> make distcheck on ubuntu 14.04
> 
> 
> Thanks,
> 
> Cody Maloney
> 
>

Re: Review Request 27670: Bug fix: Check for non-zero status code and hadoop client not found

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27670/#review60130
---


Bad patch!

Reviews applied: [27483, 27670]

Failed command: ./support/apply-review.sh -n -r 27670

Error:
 2014-11-06 06:55:27 URL:https://reviews.apache.org/r/27670/diff/raw/ [632/632] 
-> "27670.patch" [1]
error: patch failed: src/launcher/fetcher.cpp:82
error: src/launcher/fetcher.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Nov. 6, 2014, 6:25 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27670/
> ---
> 
> (Updated Nov. 6, 2014, 6:25 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Vinod Kone.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> @benh noticed that the fetcher would detect the condition where hadoop 
> returns a success in case of a success non zero value
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27670/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27651: Updated CRAMMD5Authentication tests to additionally run against the test authenticatee module.

[Adam B]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27651/#review60128
---

Ship it!


Great job! All of these Authenticatee reviews have been nearly flawless. Only 
trivial nits.


src/tests/cram_md5_authentication_tests.cpp


Not sure why you went from 'T' to 'S' instead of 'U', but it doesn't bother 
me.



src/tests/cram_md5_authentication_tests.cpp


s/credentials/credential/ (singular)



src/tests/cram_md5_authentication_tests.cpp


s/Credentials/Credential/ (singular)


- Adam B


On Nov. 5, 2014, 4:26 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27651/
> ---
> 
> (Updated Nov. 5, 2014, 4:26 p.m.)
> 
> 
> Review request for mesos, Adam B and Vinod Kone.
> 
> 
> Bugs: MESOS-2001
> https://issues.apache.org/jira/browse/MESOS-2001
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CRAMMD5Authentication tests now cover all default and modularized 
> authenticator/authenticatee combinations.
> Also adds a test for missing credentials for the authenticatee.
> 
> 
> Diffs
> -
> 
>   src/authentication/cram_md5/authenticatee.hpp 3088a77 
>   src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
>   src/tests/module.hpp 7970c28 
>   src/tests/module.cpp 482ed22 
> 
> Diff: https://reviews.apache.org/r/27651/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60127
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 6, 2014, 4:12 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 4:12 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27494: Integrated CRAM-MD5 Test Authenticatee module into slave.

[Adam B]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27494/#review60125
---


Code looks great; almost ready to commit. I just want to open up the couple of 
discussions below first.


src/messages/messages.proto


A little more clarity on this change: 'string' and 'bytes' are exactly the 
same on the wire, and both are represented as std::string in C++, so that 
shouldn't be a problem.
https://www.mail-archive.com/protobuf@googlegroups.com/msg01478.html
Java and Python represent UTF-8 strings ('string') differently from byte 
arrays ('bytes'), but the framework scheduler would send this message from the 
C++ libmesos authenticatee.
Some pure language bindings may have to change the datatype used to support 
framework authentication after this change. This may need to be documented in 
upgrade notes.



src/slave/slave.cpp


This feels a little awkward. Now, we try to authenticate if --authenticatee 
is set (to non-default) or --credential is set; otherwise register directly?
Maybe it's time to have an explicit --authenticate flag, so --credential 
isn't both a path to a credential file and a switch to enable authentication. 
Thoughts?

Or maybe just check to see if there's a viable authenticatee, and if not, 
try to register without authenticating. The master/authenticator would reject 
an unauthenticated slave/framework if the master has enabled authn, so the 
authenticatee doesn't have to be so strict about exiting instead of registering 
without authentication.


- Adam B


On Nov. 5, 2014, 4:19 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27494/
> ---
> 
> (Updated Nov. 5, 2014, 4:19 p.m.)
> 
> 
> Review request for mesos, Adam B and Vinod Kone.
> 
> 
> Bugs: MESOS-2001
> https://issues.apache.org/jira/browse/MESOS-2001
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Also fixes messages.proto to use a raw bytestream instead of a string for 
> AuthenticationStartMessage as non CRAM-MD5 authentication may transmit binary 
> data.
> 
> Note that this patch covers modularized slave authentication only. Framework 
> authentication currently is currently covered by the default (built-in) 
> implementation. There will be a subsequent patch for modularized framework 
> authentication. 
> 
> 
> Diffs
> -
> 
>   src/messages/messages.proto de0e2a2 
>   src/sched/sched.cpp e5f828d 
>   src/scheduler/scheduler.cpp c74187c 
>   src/slave/constants.hpp 701dd89 
>   src/slave/constants.cpp d6ad78c 
>   src/slave/flags.hpp efbd35d 
>   src/slave/slave.hpp 5b082fc 
>   src/slave/slave.cpp dbfd1a8 
> 
> Diff: https://reviews.apache.org/r/27494/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> NOTE all three CRAM-MD5 authenticatee module related RRs need to get applied 
> before running make check.
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 27670: Bug fix: Check for non-zero status code and hadoop client not found

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27670/
---

(Updated Nov. 6, 2014, 6:25 a.m.)


Review request for mesos, Benjamin Hindman and Vinod Kone.


Repository: mesos-git


Description (updated)
---

@benh noticed that the fetcher would detect the condition where hadoop returns 
a success in case of a success non zero value


Diffs
-

  src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 

Diff: https://reviews.apache.org/r/27670/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Review Request 27670: Bug fix: Check for non-zero status code and hadoop client not found

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27670/
---

Review request for mesos and Vinod Kone.


Repository: mesos-git


Description
---

Bug fix: Check for non-zero status code and hadoop client not found


Diffs
-

  src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 

Diff: https://reviews.apache.org/r/27670/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Review Request 27493: Added CRAM-MD5 Test Authenticatee Module.

[Adam B]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27493/#review60124
---

Ship it!


Looks great! Besides renaming the process to "crammd5_authenticatee", I could 
only find a couple of nits.


src/authentication/cram_md5/authenticatee.hpp


Why the change in wrapping? I think it would still fit the other way, and 
if you are bringing credential down to the next line, it might be better if you 
give the UPID its own line too.



src/authentication/cram_md5/authenticatee.hpp


s/authenticatee/crammd5_authenticatee/



src/authentication/cram_md5/authenticatee.hpp


"requires a credential" (singular)?



src/examples/test_authentication_module.cpp


Maybe this file should be "test_authentication_modules.cpp" (plural) since 
it contains multiple modules?


- Adam B


On Nov. 5, 2014, 8:24 a.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27493/
> ---
> 
> (Updated Nov. 5, 2014, 8:24 a.m.)
> 
> 
> Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.
> 
> 
> Bugs: MESOS-2001
> https://issues.apache.org/jira/browse/MESOS-2001
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> see summary.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am 2d72a70 
>   src/authentication/authenticatee.hpp PRE-CREATION 
>   src/authentication/cram_md5/authenticatee.hpp PRE-CREATION 
>   src/examples/test_authentication_module.cpp PRE-CREATION 
>   src/examples/test_authenticator_module.cpp PRE-CREATION 
>   src/module/authenticatee.hpp PRE-CREATION 
>   src/module/manager.cpp 7a6c884 
> 
> Diff: https://reviews.apache.org/r/27493/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> Note: All three CRAM-MD5 Authenticatee patches need to get applied before 
> running make check! 
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

[GitHub] mesos pull request: Added Daemon

[farukd]

GitHub user farukd opened a pull request:

https://github.com/apache/mesos/pull/29

Added Daemon



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/farukd/mesos patch-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/mesos/pull/29.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #29


commit b16bfaaa16174767b28a9f0eb3e022dfbd610265
Author: farukd 
Date:   2014-11-06T05:36:49Z

Added Daemon




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60123
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 6, 2014, 4:12 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 4:12 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2254

[Ian Downes]

Appears this was from that commit which introduced glog into the fetcher
and changed some expected output to include the initial glog message:

Value of: (os::read(path::join(directory, "stderr"))).get()
  Actual: "I1106 02:30:53.472061 21707 logging.cpp:177] Logging to
STDERR\nthis is stderr\n"
Expected: errMsg + "\n"
Which is: "this is stderr\n"


I'll file a ticket in the morning.

Ian




On Wed, Nov 5, 2014 at 8:48 PM, Vinod Kone  wrote:

> @ian: can you triage this?
>
> On Wed, Nov 5, 2014 at 6:37 PM, Apache Jenkins Server <
> jenk...@builds.apache.org> wrote:
>
>> See <
>> https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/2254/changes
>> >
>>
>> Changes:
>>
>> [vinodkone] Fetcher uses Hadoop to fetch URIs with unknown schemes.
>>
>> --
>> [...truncated 70150 lines...]
>> I1106 02:35:19.677485 21521 master.cpp:1337] Authorizing framework
>> principal 'test-principal' to receive offers for role '*'
>> I1106 02:35:19.678004 21530 master.cpp:1442] Registering framework
>> 20141106-023518-3125920579-59951-21504- (default) at
>> scheduler-f40b9448-6469-442b-abe8-89a9a1b215b0@67.195.81.186:59951
>> I1106 02:35:19.678170 21526 group.cpp:717] Found non-sequence node
>> 'log_replicas' at '/znode' in ZooKeeper
>> I1106 02:35:19.678387 21520 hierarchical_allocator_process.hpp:329] Added
>> framework 20141106-023518-3125920579-59951-21504-
>> I1106 02:35:19.678413 21533 detector.cpp:138] Detected a new leader:
>> (id='1')
>> I1106 02:35:19.678422 21520 hierarchical_allocator_process.hpp:697] No
>> resources available to allocate!
>> I1106 02:35:19.678447 21520 hierarchical_allocator_process.hpp:659]
>> Performed allocation for 0 slaves in 24461ns
>> I1106 02:35:19.678532 21531 sched.cpp:407] Framework registered with
>> 20141106-023518-3125920579-59951-21504-
>> I1106 02:35:19.678622 21531 sched.cpp:421] Scheduler::registered took
>> 56953ns
>> I1106 02:35:19.678834 21523 group.cpp:659] Trying to get
>> '/znode/info_01' in ZooKeeper
>> I1106 02:35:19.680326 21528 detector.cpp:433] A new leading master (UPID=
>> master@67.195.81.186:59951) is detected
>> I1106 02:35:19.680634 21531 status_update_manager.cpp:171] Pausing
>> sending status updates
>> I1106 02:35:19.680631 21526 slave.cpp:602] New master detected at
>> master@67.195.81.186:59951
>> I1106 02:35:19.680694 21526 slave.cpp:665] Authenticating with master
>> master@67.195.81.186:59951
>> I1106 02:35:19.680857 21526 slave.cpp:638] Detecting new master
>> I1106 02:35:19.680910 21529 authenticatee.hpp:133] Creating new client
>> SASL connection
>> I1106 02:35:19.681162 21529 master.cpp:3896] Authenticating slave(223)@
>> 67.195.81.186:59951
>> I1106 02:35:19.681206 21529 master.cpp:3907] Using default CRAM-MD5
>> authenticator
>> I1106 02:35:19.681447 21533 authenticator.hpp:169] Creating new server
>> SASL connection
>> I1106 02:35:19.681614 21533 authenticatee.hpp:224] Received SASL
>> authentication mechanisms: CRAM-MD5
>> I1106 02:35:19.681641 21533 authenticatee.hpp:250] Attempting to
>> authenticate with mechanism 'CRAM-MD5'
>> I1106 02:35:19.681825 21525 authenticator.hpp:275] Received SASL
>> authentication start
>> I1106 02:35:19.681977 21525 authenticator.hpp:397] Authentication
>> requires more steps
>> I1106 02:35:19.682062 21525 authenticatee.hpp:270] Received SASL
>> authentication step
>> I1106 02:35:19.682153 21525 authenticator.hpp:303] Received SASL
>> authentication step
>> I1106 02:35:19.682185 21525 auxprop.cpp:99] Request to lookup properties
>> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: '
>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
>> I1106 02:35:19.682199 21525 auxprop.cpp:171] Looking up auxiliary
>> property '*userPassword'
>> I1106 02:35:19.682230 21525 auxprop.cpp:171] Looking up auxiliary
>> property '*cmusaslsecretCRAM-MD5'
>> I1106 02:35:19.682255 21525 auxprop.cpp:99] Request to lookup properties
>> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: '
>> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
>> I1106 02:35:19.682266 21525 auxprop.cpp:121] Skipping auxiliary property
>> '*userPassword' since SASL_AUXPROP_AUTHZID == true
>> I1106 02:35:19.682276 21525 auxprop.cpp:121] Skipping auxiliary property
>> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
>> I1106 02:35:19.682291 21525 authenticator.hpp:389] Authentication success
>> I1106 02:35:19.682391 21529 authenticatee.hpp:310] Authentication success
>> I1106 02:35:19.682451 21525 master.cpp:3957] Successfully authenticated
>> principal 'test-principal' at slave(223)@67.195.81.186:59951
>> I1106 02:35:19.682914 21522 slave.cpp:722] Successfully authenticated
>> with master master@67.195.81.186:59951
>> I1106 02:35:19.683043 21522 slave.cpp:1052] Will retry registration in
>> 8.637662ms i

Re: Review Request 27622: Implement MESOS-1248: Use JSON instead of our own format for passing URI information to mesos-fetcher

[Benjamin Hindman]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27622/#review60110
---


There are still fetcherEnvironment remnants here, but I didn't comment on them 
here since the dependent review should clean them up.


src/launcher/fetcher.cpp


Let's put newlines in between each of these stanzas so it's easier to read. 
The code starts to get pretty dense without the newlines:

Try parse =
  JSON::parse(os::getenv("MESOS_COMMAND_INFO"));
  
if (parse.isError()) {
  EXIT(1) << "Failed to parse MESOS_COMMAND_INFO: " << parse.error();
}

Try commandInfo = protobuf::parse(parse.get());

if (commandInfo.isError()) {
  EXIT(1) << "Failed to parse CommandInfo: " << commandInfo.error();
}



src/tests/fetcher_tests.cpp


I don't remember why this got added, but I don't think it's necessary, and 
the fact that it's in the test is misleading because it makes it look like 
something needed to get done in order for the test to pass. If it's not 
necessary can we actually kill this in each test please? Thank you! My bad!



src/tests/fetcher_tests.cpp


Unnecessary newlines?


- Benjamin Hindman


On Nov. 5, 2014, 6:30 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27622/
> ---
> 
> (Updated Nov. 5, 2014, 6:30 p.m.)
> 
> 
> Review request for mesos and Benjamin Hindman.
> 
> 
> Bugs: MESOS-1248
> https://issues.apache.org/jira/browse/MESOS-1248
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Rebasing and replacing https://reviews.apache.org/r/21277/. Now dependent on 
> https://reviews.apache.org/r/27516/ which implements MESOS-1316.
> 
> Replaces the ad-hoc format for env var values that act as fetcher program 
> comd line args with a JSON format that is gained by translation from protobuf 
> parsing to a JSON object. 
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/slave/containerizer/containerizer.cpp 
> 0254679508167a390fd6fed855f19794354ac081 
>   src/slave/containerizer/fetcher.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27622/diff/
> 
> 
> Testing
> ---
> 
> Make check.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60116
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 6, 2014, 4:12 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 4:12 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27516: Rebased and re-edited patch for MESOS-1316: "Abstracted out invoking 'mesos-fetcher'".

[Benjamin Hindman]

> On Nov. 6, 2014, 4:44 a.m., Benjamin Hindman wrote:
> > src/tests/fetcher_tests.cpp, lines 64-67
> > 
> >
> > This is not your bug, but would you mind making the old tests 
> > consistent with the tests that were added in this patch by changing the 
> > code to:
> > 
> > CommandInfo::URI* uri = commandInfo.add_uris();
> > uri->set_value(...);
> > uri->set_executable(...);

Nevermind! This gets done in your next review, thanks!


- Benjamin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27516/#review60098
---


On Nov. 3, 2014, 4:36 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27516/
> ---
> 
> (Updated Nov. 3, 2014, 4:36 p.m.)
> 
> 
> Review request for mesos and Benjamin Hindman.
> 
> 
> Bugs: MESOS-1316
> https://issues.apache.org/jira/browse/MESOS-1316
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Manually rebasing and re-editing https://reviews.apache.org/r/21233/, which 
> is supposed to be replaced now by this patch. 
> 
> Original description: "To test the mesos-fetcher (and the setting of the 
> environment) more cleanly I did some refactoring into a 'fetcher' namespace."
> 
> Also moved fetcher environment tests to fetcher test file. Added two fetcher 
> tests.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am e6a07150c10b9fa040143e394b2f913a18eeebc1 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/slave/containerizer/fetcher.hpp PRE-CREATION 
>   src/slave/containerizer/fetcher.cpp PRE-CREATION 
>   src/slave/containerizer/mesos/containerizer.cpp 
> d4b08f54d6feb453f3a9d27ca54c867176e62102 
>   src/tests/containerizer_tests.cpp 2c90d2fc18a3268c55b6dfe98699bfb36d093983 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27516/diff/
> 
> 
> Testing
> ---
> 
> make check on Mac OS 10.10 and Ubuntu 14.4.
> 
> In total, 3 tests fail: ExamplesTest.NoExecutorFramework, 
> ExamplesTest.JavaFramework
> , ExamplesTest.PythonFramework. It is strongly suspected that those are 
> unrelated to this code change and just generally flaky.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2254

[Vinod Kone]

@ian: can you triage this?

On Wed, Nov 5, 2014 at 6:37 PM, Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See <
> https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/2254/changes
> >
>
> Changes:
>
> [vinodkone] Fetcher uses Hadoop to fetch URIs with unknown schemes.
>
> --
> [...truncated 70150 lines...]
> I1106 02:35:19.677485 21521 master.cpp:1337] Authorizing framework
> principal 'test-principal' to receive offers for role '*'
> I1106 02:35:19.678004 21530 master.cpp:1442] Registering framework
> 20141106-023518-3125920579-59951-21504- (default) at
> scheduler-f40b9448-6469-442b-abe8-89a9a1b215b0@67.195.81.186:59951
> I1106 02:35:19.678170 21526 group.cpp:717] Found non-sequence node
> 'log_replicas' at '/znode' in ZooKeeper
> I1106 02:35:19.678387 21520 hierarchical_allocator_process.hpp:329] Added
> framework 20141106-023518-3125920579-59951-21504-
> I1106 02:35:19.678413 21533 detector.cpp:138] Detected a new leader:
> (id='1')
> I1106 02:35:19.678422 21520 hierarchical_allocator_process.hpp:697] No
> resources available to allocate!
> I1106 02:35:19.678447 21520 hierarchical_allocator_process.hpp:659]
> Performed allocation for 0 slaves in 24461ns
> I1106 02:35:19.678532 21531 sched.cpp:407] Framework registered with
> 20141106-023518-3125920579-59951-21504-
> I1106 02:35:19.678622 21531 sched.cpp:421] Scheduler::registered took
> 56953ns
> I1106 02:35:19.678834 21523 group.cpp:659] Trying to get
> '/znode/info_01' in ZooKeeper
> I1106 02:35:19.680326 21528 detector.cpp:433] A new leading master (UPID=
> master@67.195.81.186:59951) is detected
> I1106 02:35:19.680634 21531 status_update_manager.cpp:171] Pausing sending
> status updates
> I1106 02:35:19.680631 21526 slave.cpp:602] New master detected at
> master@67.195.81.186:59951
> I1106 02:35:19.680694 21526 slave.cpp:665] Authenticating with master
> master@67.195.81.186:59951
> I1106 02:35:19.680857 21526 slave.cpp:638] Detecting new master
> I1106 02:35:19.680910 21529 authenticatee.hpp:133] Creating new client
> SASL connection
> I1106 02:35:19.681162 21529 master.cpp:3896] Authenticating slave(223)@
> 67.195.81.186:59951
> I1106 02:35:19.681206 21529 master.cpp:3907] Using default CRAM-MD5
> authenticator
> I1106 02:35:19.681447 21533 authenticator.hpp:169] Creating new server
> SASL connection
> I1106 02:35:19.681614 21533 authenticatee.hpp:224] Received SASL
> authentication mechanisms: CRAM-MD5
> I1106 02:35:19.681641 21533 authenticatee.hpp:250] Attempting to
> authenticate with mechanism 'CRAM-MD5'
> I1106 02:35:19.681825 21525 authenticator.hpp:275] Received SASL
> authentication start
> I1106 02:35:19.681977 21525 authenticator.hpp:397] Authentication requires
> more steps
> I1106 02:35:19.682062 21525 authenticatee.hpp:270] Received SASL
> authentication step
> I1106 02:35:19.682153 21525 authenticator.hpp:303] Received SASL
> authentication step
> I1106 02:35:19.682185 21525 auxprop.cpp:99] Request to lookup properties
> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: '
> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
> I1106 02:35:19.682199 21525 auxprop.cpp:171] Looking up auxiliary property
> '*userPassword'
> I1106 02:35:19.682230 21525 auxprop.cpp:171] Looking up auxiliary property
> '*cmusaslsecretCRAM-MD5'
> I1106 02:35:19.682255 21525 auxprop.cpp:99] Request to lookup properties
> for user: 'test-principal' realm: 'penates.apache.org' server FQDN: '
> penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
> I1106 02:35:19.682266 21525 auxprop.cpp:121] Skipping auxiliary property
> '*userPassword' since SASL_AUXPROP_AUTHZID == true
> I1106 02:35:19.682276 21525 auxprop.cpp:121] Skipping auxiliary property
> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
> I1106 02:35:19.682291 21525 authenticator.hpp:389] Authentication success
> I1106 02:35:19.682391 21529 authenticatee.hpp:310] Authentication success
> I1106 02:35:19.682451 21525 master.cpp:3957] Successfully authenticated
> principal 'test-principal' at slave(223)@67.195.81.186:59951
> I1106 02:35:19.682914 21522 slave.cpp:722] Successfully authenticated with
> master master@67.195.81.186:59951
> I1106 02:35:19.683043 21522 slave.cpp:1052] Will retry registration in
> 8.637662ms if necessary
> I1106 02:35:19.683217 21525 master.cpp:3063] Registering slave at
> slave(223)@67.195.81.186:59951 (penates.apache.org) with id
> 20141106-023518-3125920579-59951-21504-S0
> I1106 02:35:19.683784 21528 registrar.cpp:445] Applied 1 operations in
> 67790ns; attempting to update the 'registry'
> I1106 02:35:19.687147 21525 log.cpp:680] Attempting to append 318 bytes to
> the log
> I1106 02:35:19.687314 21522 coordinator.cpp:340] Coordinator attempting to
> write APPEND action at position 3
> I1106 02:35:19.688436 21519 repl

Re: Review Request 27516: Rebased and re-edited patch for MESOS-1316: "Abstracted out invoking 'mesos-fetcher'".

[Benjamin Hindman]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27516/#review60098
---



src/slave/containerizer/fetcher.hpp


Really minor, but I think we should update this comment with 
s/splice/redirect/ now that we've killed the splicing implementation.



src/slave/containerizer/fetcher.hpp


s/splicing/redirecting/



src/slave/containerizer/fetcher.hpp


Mind replacing '> >' with '>>' everywhere in your patch please? Thanks!



src/slave/containerizer/fetcher.cpp


Please put "" includes after <> includes.



src/slave/containerizer/fetcher.cpp


The function fetcherEnviroment should be getting replaced with the new 
fetcher::environment that is defined above. It looks like fetcherEnvironment in 
containerizer.hpp|cpp has changed so we should update fetcher::environment with 
that implementation and then delete it there.



src/slave/containerizer/fetcher.cpp


You don't need to redirect yourself, you can just use Subprocess::FD(), 
which properly dups the file descriptor, when you do 'subprocess(...)' above:

--

Try fetcher = subprocess(
command,
Subprocess::PIPE(),
stdout.isSome()
  ? Subprocess::FD(stdout.get())
  : Subprocess::PIPE(),
stderr.isSome()
  ? Subprocess::FD(stderr.get())
  : Subprocess::PIPE(),
environment);

if (fetcher.isError()) {
  return Failure("Failed to execute mesos-fetcher: " + fetcher.error());
}

return fetcher.get().status();

--

The advantage to using Subprocess::FD() is that the redirection will keep 
working even if the parent process exits where is io::redirect() only works as 
long as the parent is still alive.



src/slave/containerizer/fetcher.cpp


Even after making the change above, we shouldn't close the passed in file 
descriptors. The comment for this function also says that we dup the file 
descriptors, which implies that we won't be closing them because we're creating 
our own copy so we can control our copy's lifetime.

In general, the semantics that I've seen compose well are always dup'ing 
the descriptors ourselves (if we want them to outlive the function call) and 
then letting the caller close any file descriptors they passed in on their own 
accord. It's much easier to reason about file descriptor lifetimes with these 
semantics.

Even better would be to have an FD abstraction that does reference 
counting, but unfortunately we don't have one of those yet.



src/slave/containerizer/mesos/containerizer.cpp


Why does this need to be a Future? It looks like you're chaining this below 
via a 'then', which means you can just use Option since a 'then' only 
invokes this if the Future is ready. The fact that you're always doing a 
'get()' on the future too implies this precondition.



src/slave/containerizer/mesos/containerizer.cpp


Since we're no longer doing io::redirect here it would be great to update 
this comment to refer to fetcher::run instead. You'll likely need to reword the 
comment slightly, but I think the gist of it is that we could update 
fetcher::run to take paths instead of file descriptors and then use 
Subprocess::PATH() instead of Subprocess::FD(). Let's not actually do that now, 
but just update the comment. ;-)



src/slave/containerizer/mesos/containerizer.cpp


Please add a space between :" on this line. Same below please.



src/slave/containerizer/mesos/containerizer.cpp


Given the code above this would be a double close, which are notoriously 
hard to debug! We should definitely keep the closes here and remove the close 
up in fetcher::run (see the comment there).



src/tests/fetcher_tests.cpp


We separate these with a newline just like we separate process, stout, 
mesos, etc.

Same below with 'slave/containerizer/fetcher.hpp' and 'slave/flags.hpp'. 
Basically, we put 'dirname' things together, so:

a/b.hpp
a/c.hpp

a/d/e.hpp
a/d/f.hpp

a/d/g/h.hpp

a/i/j.hpp



src/tests/fetcher_tests.cpp


This

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60107
---


Hey Ankur. Can you send a new review instead of re-opening this one? This one 
is already submitted.

- Vinod Kone


On Nov. 6, 2014, 4:12 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 4:12 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
---

(Updated Nov. 6, 2014, 4:12 a.m.)


Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.


Changes
---

Resolve merge conflicts


Bugs: MESOS-1711
https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
---

Previously, the fetcher used a hardcoded list of schemes to determine what URIs 
could be fetched by hadoop (if available). This is now changed such that we 
first check if hadoop can fetch them for us and then we fallback to the os::net 
and then a local copy method (same as it used to be). This allows users to 
fetch artifacts from arbitrary filesystems as long as hadoop is correctly 
configured (in core-site.xml).


Diffs (updated)
-

  src/launcher/fetcher.cpp bd95928bc3191970330e839bcf41e343d5142c54 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60105
---


Bad patch!

Reviews applied: [27483]

Failed command: ./support/apply-review.sh -n -r 27483

Error:
 2014-11-06 04:10:49 URL:https://reviews.apache.org/r/27483/diff/raw/ 
[15088/15088] -> "27483.patch" [1]
error: patch failed: src/hdfs/hdfs.hpp:42
error: src/hdfs/hdfs.hpp: patch does not apply
error: patch failed: src/launcher/fetcher.cpp:27
error: src/launcher/fetcher.cpp: patch does not apply
error: patch failed: src/tests/fetcher_tests.cpp:25
error: src/tests/fetcher_tests.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Nov. 6, 2014, 3:18 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 3:18 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60102
---


Bad patch!

Reviews applied: [27483]

Failed command: ./support/apply-review.sh -n -r 27483

Error:
 2014-11-06 03:45:42 URL:https://reviews.apache.org/r/27483/diff/raw/ 
[15088/15088] -> "27483.patch" [1]
error: patch failed: src/hdfs/hdfs.hpp:42
error: src/hdfs/hdfs.hpp: patch does not apply
error: patch failed: src/launcher/fetcher.cpp:27
error: src/launcher/fetcher.cpp: patch does not apply
error: patch failed: src/tests/fetcher_tests.cpp:25
error: src/tests/fetcher_tests.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Nov. 6, 2014, 3:18 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 6, 2014, 3:18 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27447: mesos: Replaced the ip and port pairs from UPID class and process namespace with Node class

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27447/#review60101
---


Patch looks great!

Reviews applied: [27446, 27447]

All tests passed.

- Mesos ReviewBot


On Nov. 6, 2014, 2:49 a.m., Evelina Dumitrescu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27447/
> ---
> 
> (Updated Nov. 6, 2014, 2:49 a.m.)
> 
> 
> Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched 
> to 'mcypark'.
> 
> 
> Bugs: MESOS-2019
> https://issues.apache.org/jira/browse/MESOS-2019
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> At the moment, the Node class is used to keep a mapping from a socket to the 
> ip & port pair in the process namespace.
> I want to propose to extend its use by replacing the ip & port fields from 
> the UPID class and process namespace with this type.
> 
> 
> Diffs
> -
> 
>   src/common/protobuf_utils.cpp 33ce782960e88e07b910f844fbdcae4c0f19d9a7 
>   src/master/master.cpp a860496d9d75ae4e007d35627d778caf10d08428 
>   src/sched/sched.cpp e5f828d0bf9dd03a01920634abfae685a7861b44 
>   src/scheduler/scheduler.cpp c74187cfeb57a748fd57198d3badd4b8773490eb 
>   src/slave/http.cpp 965d78c3e3ea2e3c922f62aabf6fc87c5cf2ab8d 
>   src/slave/slave.cpp dbfd1a8101d78dee8ea3ac19d990a6a7892e59be 
>   src/tests/fetcher_tests.cpp e026e875c7197cd214ef0432d7c40207553c8671 
>   src/tests/files_tests.cpp a696aa22d56b37ee70c6e64c81a849da6d436451 
>   src/tests/gc_tests.cpp f7747e2999cbd64ab0cd0321cca0021489dc3346 
>   src/tests/logging_tests.cpp 58e9b3399c3de28b1c5b3a8ec8fdf85b7a31c6d5 
>   src/tests/master_contender_detector_tests.cpp 
> 8f7fb18f0ca985364565fc4cd50f113b1461fbe7 
>   src/tests/master_tests.cpp a6d1a4a5b7d50a88e5b101c43f62d21300cf 
>   src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d 
> 
> Diff: https://reviews.apache.org/r/27447/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Evelina Dumitrescu
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
---

(Updated Nov. 6, 2014, 3:18 a.m.)


Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.


Changes
---

Bug fix: If Hadoop client is available but a version check returns a non-zero 
status code we should not continue with the fetcher. @ben


Bugs: MESOS-1711
https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
---

Previously, the fetcher used a hardcoded list of schemes to determine what URIs 
could be fetched by hadoop (if available). This is now changed such that we 
first check if hadoop can fetch them for us and then we fallback to the os::net 
and then a local copy method (same as it used to be). This allows users to 
fetch artifacts from arbitrary filesystems as long as hadoop is correctly 
configured (in core-site.xml).


Diffs (updated)
-

  src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Review Request 27446: libprocess: Replaced the ip and port pairs from UPID class and process namespace with Node class

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27446/
---

(Updated Nov. 6, 2014, 3:07 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Changes
---

Changed commit message.


Summary (updated)
-

libprocess: Replaced the ip and port pairs from UPID class and process 
namespace with Node class


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs (updated)
-

  3rdparty/libprocess/include/process/node.hpp 
7a96894456683be60595a8f96f85df37947ff5b6 
  3rdparty/libprocess/include/process/pid.hpp 
23453229c2a868a62b143416de33cf630268a64e 
  3rdparty/libprocess/include/process/process.hpp 
81a1f7a3fece78bb7df1e86850572922c8704ea4 
  3rdparty/libprocess/src/http.cpp 4ef00d11de4ff4e7eb4e44aac204147799f85ac2 
  3rdparty/libprocess/src/pid.cpp 20ff25c0959c247687955535f02e0397653af52a 
  3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 
  3rdparty/libprocess/src/tests/benchmarks.cpp 
3177a8ecac6adf8b52c3160587dc3a4fe2707d26 
  3rdparty/libprocess/src/tests/http_tests.cpp 
a1c3685c88d268c7ecdc2dad22608198dc0b295d 
  3rdparty/libprocess/src/tests/metrics_tests.cpp 
33539e45bab8f79afa773a325465016f01fd3b75 
  3rdparty/libprocess/src/tests/process_tests.cpp 
b985fb77ea05fae5c0b144ea48814acc7bb5135b 

Diff: https://reviews.apache.org/r/27446/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Re: Review Request 27446: libprocess: Replaced the ip and port pairs from UPID class and process namespace with Node class.

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27446/
---

(Updated Nov. 6, 2014, 2:51 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Summary (updated)
-

libprocess: Replaced the ip and port pairs from UPID class and process 
namespace with Node class.


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs
-

  3rdparty/libprocess/include/process/node.hpp 
7a96894456683be60595a8f96f85df37947ff5b6 
  3rdparty/libprocess/include/process/pid.hpp 
23453229c2a868a62b143416de33cf630268a64e 
  3rdparty/libprocess/include/process/process.hpp 
81a1f7a3fece78bb7df1e86850572922c8704ea4 
  3rdparty/libprocess/src/http.cpp 4ef00d11de4ff4e7eb4e44aac204147799f85ac2 
  3rdparty/libprocess/src/pid.cpp 20ff25c0959c247687955535f02e0397653af52a 
  3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 
  3rdparty/libprocess/src/tests/benchmarks.cpp 
3177a8ecac6adf8b52c3160587dc3a4fe2707d26 
  3rdparty/libprocess/src/tests/http_tests.cpp 
a1c3685c88d268c7ecdc2dad22608198dc0b295d 
  3rdparty/libprocess/src/tests/metrics_tests.cpp 
33539e45bab8f79afa773a325465016f01fd3b75 
  3rdparty/libprocess/src/tests/process_tests.cpp 
b985fb77ea05fae5c0b144ea48814acc7bb5135b 

Diff: https://reviews.apache.org/r/27446/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Re: Review Request 27447: mesos: Replaced the ip and port pairs from UPID class and process namespace with Node class

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27447/
---

(Updated Nov. 6, 2014, 2:49 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Changes
---

Last patch failed compilation.


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs (updated)
-

  src/common/protobuf_utils.cpp 33ce782960e88e07b910f844fbdcae4c0f19d9a7 
  src/master/master.cpp a860496d9d75ae4e007d35627d778caf10d08428 
  src/sched/sched.cpp e5f828d0bf9dd03a01920634abfae685a7861b44 
  src/scheduler/scheduler.cpp c74187cfeb57a748fd57198d3badd4b8773490eb 
  src/slave/http.cpp 965d78c3e3ea2e3c922f62aabf6fc87c5cf2ab8d 
  src/slave/slave.cpp dbfd1a8101d78dee8ea3ac19d990a6a7892e59be 
  src/tests/fetcher_tests.cpp e026e875c7197cd214ef0432d7c40207553c8671 
  src/tests/files_tests.cpp a696aa22d56b37ee70c6e64c81a849da6d436451 
  src/tests/gc_tests.cpp f7747e2999cbd64ab0cd0321cca0021489dc3346 
  src/tests/logging_tests.cpp 58e9b3399c3de28b1c5b3a8ec8fdf85b7a31c6d5 
  src/tests/master_contender_detector_tests.cpp 
8f7fb18f0ca985364565fc4cd50f113b1461fbe7 
  src/tests/master_tests.cpp a6d1a4a5b7d50a88e5b101c43f62d21300cf 
  src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d 

Diff: https://reviews.apache.org/r/27447/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Re: Review Request 27447: mesos: Replaced the ip and port pairs from UPID class and process namespace with Node class

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27447/
---

(Updated Nov. 6, 2014, 2:44 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Summary (updated)
-

mesos:  Replaced the ip and port pairs from UPID class and process namespace 
with Node class


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs (updated)
-

  src/common/protobuf_utils.cpp 33ce782960e88e07b910f844fbdcae4c0f19d9a7 
  src/master/master.cpp a860496d9d75ae4e007d35627d778caf10d08428 
  src/sched/sched.cpp e5f828d0bf9dd03a01920634abfae685a7861b44 
  src/scheduler/scheduler.cpp c74187cfeb57a748fd57198d3badd4b8773490eb 
  src/slave/http.cpp 965d78c3e3ea2e3c922f62aabf6fc87c5cf2ab8d 
  src/slave/slave.cpp dbfd1a8101d78dee8ea3ac19d990a6a7892e59be 
  src/tests/fetcher_tests.cpp e026e875c7197cd214ef0432d7c40207553c8671 
  src/tests/files_tests.cpp a696aa22d56b37ee70c6e64c81a849da6d436451 
  src/tests/gc_tests.cpp f7747e2999cbd64ab0cd0321cca0021489dc3346 
  src/tests/logging_tests.cpp 58e9b3399c3de28b1c5b3a8ec8fdf85b7a31c6d5 
  src/tests/master_contender_detector_tests.cpp 
8f7fb18f0ca985364565fc4cd50f113b1461fbe7 
  src/tests/master_tests.cpp a6d1a4a5b7d50a88e5b101c43f62d21300cf 
  src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d 

Diff: https://reviews.apache.org/r/27447/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2254

[Apache Jenkins Server]

See 


Changes:

[vinodkone] Fetcher uses Hadoop to fetch URIs with unknown schemes.

--
[...truncated 70150 lines...]
I1106 02:35:19.677485 21521 master.cpp:1337] Authorizing framework principal 
'test-principal' to receive offers for role '*'
I1106 02:35:19.678004 21530 master.cpp:1442] Registering framework 
20141106-023518-3125920579-59951-21504- (default) at 
scheduler-f40b9448-6469-442b-abe8-89a9a1b215b0@67.195.81.186:59951
I1106 02:35:19.678170 21526 group.cpp:717] Found non-sequence node 
'log_replicas' at '/znode' in ZooKeeper
I1106 02:35:19.678387 21520 hierarchical_allocator_process.hpp:329] Added 
framework 20141106-023518-3125920579-59951-21504-
I1106 02:35:19.678413 21533 detector.cpp:138] Detected a new leader: (id='1')
I1106 02:35:19.678422 21520 hierarchical_allocator_process.hpp:697] No 
resources available to allocate!
I1106 02:35:19.678447 21520 hierarchical_allocator_process.hpp:659] Performed 
allocation for 0 slaves in 24461ns
I1106 02:35:19.678532 21531 sched.cpp:407] Framework registered with 
20141106-023518-3125920579-59951-21504-
I1106 02:35:19.678622 21531 sched.cpp:421] Scheduler::registered took 56953ns
I1106 02:35:19.678834 21523 group.cpp:659] Trying to get 
'/znode/info_01' in ZooKeeper
I1106 02:35:19.680326 21528 detector.cpp:433] A new leading master 
(UPID=master@67.195.81.186:59951) is detected
I1106 02:35:19.680634 21531 status_update_manager.cpp:171] Pausing sending 
status updates
I1106 02:35:19.680631 21526 slave.cpp:602] New master detected at 
master@67.195.81.186:59951
I1106 02:35:19.680694 21526 slave.cpp:665] Authenticating with master 
master@67.195.81.186:59951
I1106 02:35:19.680857 21526 slave.cpp:638] Detecting new master
I1106 02:35:19.680910 21529 authenticatee.hpp:133] Creating new client SASL 
connection
I1106 02:35:19.681162 21529 master.cpp:3896] Authenticating 
slave(223)@67.195.81.186:59951
I1106 02:35:19.681206 21529 master.cpp:3907] Using default CRAM-MD5 
authenticator
I1106 02:35:19.681447 21533 authenticator.hpp:169] Creating new server SASL 
connection
I1106 02:35:19.681614 21533 authenticatee.hpp:224] Received SASL authentication 
mechanisms: CRAM-MD5
I1106 02:35:19.681641 21533 authenticatee.hpp:250] Attempting to authenticate 
with mechanism 'CRAM-MD5'
I1106 02:35:19.681825 21525 authenticator.hpp:275] Received SASL authentication 
start
I1106 02:35:19.681977 21525 authenticator.hpp:397] Authentication requires more 
steps
I1106 02:35:19.682062 21525 authenticatee.hpp:270] Received SASL authentication 
step
I1106 02:35:19.682153 21525 authenticator.hpp:303] Received SASL authentication 
step
I1106 02:35:19.682185 21525 auxprop.cpp:99] Request to lookup properties for 
user: 'test-principal' realm: 'penates.apache.org' server FQDN: 
'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 
I1106 02:35:19.682199 21525 auxprop.cpp:171] Looking up auxiliary property 
'*userPassword'
I1106 02:35:19.682230 21525 auxprop.cpp:171] Looking up auxiliary property 
'*cmusaslsecretCRAM-MD5'
I1106 02:35:19.682255 21525 auxprop.cpp:99] Request to lookup properties for 
user: 'test-principal' realm: 'penates.apache.org' server FQDN: 
'penates.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 
I1106 02:35:19.682266 21525 auxprop.cpp:121] Skipping auxiliary property 
'*userPassword' since SASL_AUXPROP_AUTHZID == true
I1106 02:35:19.682276 21525 auxprop.cpp:121] Skipping auxiliary property 
'*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1106 02:35:19.682291 21525 authenticator.hpp:389] Authentication success
I1106 02:35:19.682391 21529 authenticatee.hpp:310] Authentication success
I1106 02:35:19.682451 21525 master.cpp:3957] Successfully authenticated 
principal 'test-principal' at slave(223)@67.195.81.186:59951
I1106 02:35:19.682914 21522 slave.cpp:722] Successfully authenticated with 
master master@67.195.81.186:59951
I1106 02:35:19.683043 21522 slave.cpp:1052] Will retry registration in 
8.637662ms if necessary
I1106 02:35:19.683217 21525 master.cpp:3063] Registering slave at 
slave(223)@67.195.81.186:59951 (penates.apache.org) with id 
20141106-023518-3125920579-59951-21504-S0
I1106 02:35:19.683784 21528 registrar.cpp:445] Applied 1 operations in 67790ns; 
attempting to update the 'registry'
I1106 02:35:19.687147 21525 log.cpp:680] Attempting to append 318 bytes to the 
log
I1106 02:35:19.687314 21522 coordinator.cpp:340] Coordinator attempting to 
write APPEND action at position 3
I1106 02:35:19.688436 21519 replica.cpp:508] Replica received write request for 
position 3
I1106 02:35:19.689483 21519 leveldb.cpp:343] Persisting action (337 bytes) to 
leveldb took 1.009715ms
I1106 02:35:19.801269 21519 replica.cpp:676] Persisted action at 3
I1106 02:35:19.692602 21527 slave.cp

Review Request 27660: Made ProcessBase::route hard-fail rather than soft fail

[Cody Maloney]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27660/
---

Review request for mesos and Adam B.


Repository: mesos-git


Description
---

It is an API expectation that the path start with '/', and if it doesn't it 
should be a programmer error.

No code actually checked the return code. If a developer did get this wrong 
previously it would silently do nothing and in debugging their endpoint just 
wouldn't appear. This makes it simple/easy for them to find.


Diffs
-

  3rdparty/libprocess/include/process/process.hpp 
81a1f7a3fece78bb7df1e86850572922c8704ea4 
  3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 

Diff: https://reviews.apache.org/r/27660/diff/


Testing
---

make distcheck on ubuntu 14.04


Thanks,

Cody Maloney

Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui #2532

[Apache Jenkins Server]

See 


Changes:

[vinodkone] Fetcher uses Hadoop to fetch URIs with unknown schemes.

--
[...truncated 64613 lines...]
I1106 02:22:41.260062  8959 authenticator.hpp:275] Received SASL authentication 
start
I1106 02:22:41.260125  8959 authenticator.hpp:397] Authentication requires more 
steps
I1106 02:22:41.263913  8938 sched.cpp:137] Version: 0.22.0
I1106 02:22:41.264335  8965 sched.cpp:233] New master detected at 
master@67.195.81.190:35338
I1106 02:22:41.264369  8965 sched.cpp:283] Authenticating with master 
master@67.195.81.190:35338
I1106 02:22:41.264536  8967 authenticatee.hpp:133] Creating new client SASL 
connection
I1106 02:22:41.264746  8957 master.cpp:3896] Authenticating 
scheduler-52efb44b-4d86-4034-8047-febe66afcacc@67.195.81.190:35338
I1106 02:22:41.264781  8957 master.cpp:3907] Using default CRAM-MD5 
authenticator
I1106 02:22:41.264982  8967 authenticator.hpp:169] Creating new server SASL 
connection
I1106 02:22:41.265151  8964 authenticatee.hpp:224] Received SASL authentication 
mechanisms: CRAM-MD5
I1106 02:22:41.265183  8964 authenticatee.hpp:250] Attempting to authenticate 
with mechanism 'CRAM-MD5'
I1106 02:22:41.265353  8955 authenticator.hpp:275] Received SASL authentication 
start
I1106 02:22:41.265398  8955 authenticator.hpp:397] Authentication requires more 
steps
I1106 02:22:41.265486  8955 authenticatee.hpp:270] Received SASL authentication 
step
I1106 02:22:41.265612  8955 authenticator.hpp:303] Received SASL authentication 
step
I1106 02:22:41.265640  8955 auxprop.cpp:99] Request to lookup properties for 
user: 'test-principal' realm: 'pietas.apache.org' server FQDN: 
'pietas.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false 
I1106 02:22:41.265651  8955 auxprop.cpp:171] Looking up auxiliary property 
'*userPassword'
I1106 02:22:41.265676  8955 auxprop.cpp:171] Looking up auxiliary property 
'*cmusaslsecretCRAM-MD5'
I1106 02:22:41.265704  8955 auxprop.cpp:99] Request to lookup properties for 
user: 'test-principal' realm: 'pietas.apache.org' server FQDN: 
'pietas.apache.org' SASL_AUXPROP_VERIFY_AGAINST_HASH: false 
SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true 
I1106 02:22:41.265720  8955 auxprop.cpp:121] Skipping auxiliary property 
'*userPassword' since SASL_AUXPROP_AUTHZID == true
I1106 02:22:41.265730  8955 auxprop.cpp:121] Skipping auxiliary property 
'*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1106 02:22:41.265750  8955 authenticator.hpp:389] Authentication success
I1106 02:22:41.265872  8956 authenticatee.hpp:310] Authentication success
I1106 02:22:41.265893  8955 master.cpp:3957] Successfully authenticated 
principal 'test-principal' at 
scheduler-52efb44b-4d86-4034-8047-febe66afcacc@67.195.81.190:35338
I1106 02:22:41.266150  8963 sched.cpp:357] Successfully authenticated with 
master master@67.195.81.190:35338
I1106 02:22:41.266181  8963 sched.cpp:476] Sending registration request to 
master@67.195.81.190:35338
I1106 02:22:41.266322  8966 master.cpp:1515] Received re-registration request 
from framework 20141106-022241-3193029443-35338-8938- (default) at 
scheduler-52efb44b-4d86-4034-8047-febe66afcacc@67.195.81.190:35338
I1106 02:22:41.266535  8966 master.cpp:1537] Refusing re-registration of 
framework 20141106-022241-3193029443-35338-8938- (default)  at 
scheduler-52efb44b-4d86-4034-8047-febe66afcacc@67.195.81.190:35338: Framework 
principal 'mismatched-principal' does not match authenticated principal 
'test-principal'
I1106 02:22:41.266685  8967 sched.cpp:729] Got error 'Framework principal 
'mismatched-principal' does not match authenticated principal 'test-principal''
I1106 02:22:41.266762  8967 sched.cpp:740] Scheduler::error took 38897ns
I1106 02:22:41.266809  8967 sched.cpp:770] Aborting framework 
'20141106-022241-3193029443-35338-8938-'
I1106 02:22:41.266829  8967 sched.cpp:775] Not sending a deactivate message as 
master is disconnected
I1106 02:22:41.267096  8938 master.cpp:693] Master terminating
I1106 02:22:41.267168  8957 sched.cpp:745] Stopping framework 
'20141106-022241-3193029443-35338-8938-'
I1106 02:22:41.267202  8964 sched.cpp:745] Stopping framework 
'20141106-022241-3193029443-35338-8938-'
[   OK ] AuthenticationTest.RejectedSchedulerFailover (67 ms)
[ RUN  ] AuthenticationTest.AuthenticatedFramework
Using temporary directory 
'/tmp/AuthenticationTest_AuthenticatedFramework_yr9Ujq'
I1106 02:22:41.276065  8938 leveldb.cpp:176] Opened db in 2.562537ms
I1106 02:22:41.277259  8938 leveldb.cpp:183] Compacted db in 1.164334ms
I1106 02:22:41.277310  8938 leveldb.cpp:198] Created db iterator in 19591ns
I1106 02:22:41.277326  8938 leveldb.cpp:204] Seeked to beginning of db in 2309ns
I1106 02:22:41.277333  8938 leveldb.cpp:273] Iterated through 0 keys in the db 
in 539ns
I1106 02:2

Re: Review Request 27555: Refactored the C++ Resources class to support persistent disk resources.

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27555/#review59882
---


Can you split this review? It's really hard to follow all the changes in 
resources.

You have 5 bullet points in your description. Maybe split according to those?

Also, attach MESOS-1714 to it.


include/mesos/resources.hpp


s/"resources"/these Resources/



include/mesos/resources.hpp


Not sure if "flatten" is the right word because the Resources are already 
stored as flat.



include/mesos/resources.hpp


s/flattened// aren't Resources always flattened.

s/"resources"/these Resources/



include/mesos/resources.hpp


what does "appropriate roles" mean?



include/mesos/resources.hpp


s/targets/target/ ?



include/mesos/resources.hpp


Are these across all roles?



src/common/resources.cpp


why equals instead of the "==" operator? here and below.



src/common/resources.cpp


s/object/objects/



src/common/resources.cpp


You probably want to explain what you mean by "named" Resource objects.



src/common/resources.cpp


s/make/makes/



src/common/resources.cpp


Maybe include the resource name to help in debugging?



src/common/resources.cpp


s/we/We/



src/tests/allocator_tests.cpp


no need for "flags.enabled_default_resources=false" here and below in this 
test?



src/tests/resources_tests.cpp


Why is this killed?



src/tests/resources_tests.cpp


Why not just use .cpus() and .mem(), here and everywhere else?


- Vinod Kone


On Nov. 4, 2014, 10:01 p.m., Jie Yu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27555/
> ---
> 
> (Updated Nov. 4, 2014, 10:01 p.m.)
> 
> 
> Review request for mesos, Ben Mahler and Vinod Kone.
> 
> 
> Bugs: MESOS-1974
> https://issues.apache.org/jira/browse/MESOS-1974
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> The purpose of the refactor is to support persistent disk resources.
> 
> Here are the main things I've done in this refactor:
> 1) Resource objects in Resources are stored in minimal format 
> (validated/non-zero). That allows us to kill isAllocatable, allocatable, 
> isZero, etc.
> 2) 'matches' needs to be split into two pieces: one for combining and one for 
> removing, in order to support persitent disk resource. For example, one 
> cannot combine two Resource object with DiskInfo (it's like two disks), 
> however, you can do removal if they are identical.
> 3) Some of the interfaces are not intuitive (e.g., <=, see details in the 
> ticket). I removed them in favor of more explicit interfaces.
> 4) Unified all the validation code.
> 5) Adjusted the tests accordingly.
> 
> 
> Diffs
> -
> 
>   include/mesos/resources.hpp 0e37170 
>   src/cli/execute.cpp ddaa20d 
>   src/common/resources.cpp e9a0c85 
>   src/examples/low_level_scheduler_libprocess.cpp 7ef5ea7 
>   src/examples/low_level_scheduler_pthread.cpp 6e233a1 
>   src/examples/no_executor_framework.cpp f98a073 
>   src/examples/test_framework.cpp 187a611 
>   src/master/drf_sorter.cpp 5464900 
>   src/master/hierarchical_allocator_process.hpp 31dfb2c 
>   src/master/http.cpp 3189933 
>   src/master/master.cpp d914786 
>   src/tests/allocator_tests.cpp 58e15aa 
>   src/tests/gc_tests.cpp f7747e2 
>   src/tests/master_tests.cpp 2e52574 
>   src/tests/mesos.hpp c1d64a7 
>   src/tests/resource_offers_tests.cpp fe66432 
>   src/tests/resources_tests.cpp 3e50889 
>   src/tests/slave_recovery_tests.cpp 98e059f 
>   src/tests/sorter_tests.cpp 0516ab5 
> 
> Diff: https://reviews.apache.org/r/27555/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Benjamin Hindman]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60091
---



src/launcher/fetcher.cpp


This is a bug: if HDFS::available() returns false we'll assume HDFS is 
available when it is not!


- Benjamin Hindman


On Nov. 5, 2014, 10:07 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 10:07 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27447: mesos: Renamed the Node class in Endpoint and replaced the ip and port pairs from UPID class and process namespace with this concept

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27447/
---

(Updated Nov. 6, 2014, 1:23 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Changes
---

Switched back from Endpoint to Node


Summary (updated)
-

 mesos: Renamed the Node class in Endpoint and replaced the ip and port pairs 
from UPID class and process namespace with this concept


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description (updated)
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs (updated)
-

  src/common/protobuf_utils.cpp 33ce782960e88e07b910f844fbdcae4c0f19d9a7 
  src/master/master.cpp a860496d9d75ae4e007d35627d778caf10d08428 
  src/sched/sched.cpp e5f828d0bf9dd03a01920634abfae685a7861b44 
  src/scheduler/scheduler.cpp c74187cfeb57a748fd57198d3badd4b8773490eb 
  src/slave/http.cpp 965d78c3e3ea2e3c922f62aabf6fc87c5cf2ab8d 
  src/slave/slave.cpp dbfd1a8101d78dee8ea3ac19d990a6a7892e59be 
  src/tests/files_tests.cpp a696aa22d56b37ee70c6e64c81a849da6d436451 
  src/tests/gc_tests.cpp f7747e2999cbd64ab0cd0321cca0021489dc3346 
  src/tests/logging_tests.cpp 58e9b3399c3de28b1c5b3a8ec8fdf85b7a31c6d5 
  src/tests/master_contender_detector_tests.cpp 
8f7fb18f0ca985364565fc4cd50f113b1461fbe7 
  src/tests/master_tests.cpp a6d1a4a5b7d50a88e5b101c43f62d21300cf 
  src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d 

Diff: https://reviews.apache.org/r/27447/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Re: Review Request 27446: libprocess: Renamed the Node class in Endpoint and replaced the ip and port pairs from UPID class and process namespace with this concept

[Evelina Dumitrescu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27446/
---

(Updated Nov. 6, 2014, 1:23 a.m.)


Review request for mesos, Dominic Hamon, Joris Van Remoortere, and switched to 
'mcypark'.


Changes
---

Switched back from Endpoint to Node.


Summary (updated)
-

libprocess: Renamed the Node class in Endpoint and replaced the ip and port 
pairs from UPID class and process namespace with this concept


Bugs: MESOS-2019
https://issues.apache.org/jira/browse/MESOS-2019


Repository: mesos-git


Description (updated)
---

At the moment, the Node class is used to keep a mapping from a socket to the ip 
& port pair in the process namespace.
I want to propose to extend its use by replacing the ip & port fields from the 
UPID class and process namespace with this type.


Diffs (updated)
-

  3rdparty/libprocess/include/process/node.hpp 
7a96894456683be60595a8f96f85df37947ff5b6 
  3rdparty/libprocess/include/process/pid.hpp 
23453229c2a868a62b143416de33cf630268a64e 
  3rdparty/libprocess/include/process/process.hpp 
81a1f7a3fece78bb7df1e86850572922c8704ea4 
  3rdparty/libprocess/src/http.cpp 4ef00d11de4ff4e7eb4e44aac204147799f85ac2 
  3rdparty/libprocess/src/pid.cpp 20ff25c0959c247687955535f02e0397653af52a 
  3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 
  3rdparty/libprocess/src/tests/benchmarks.cpp 
3177a8ecac6adf8b52c3160587dc3a4fe2707d26 
  3rdparty/libprocess/src/tests/http_tests.cpp 
a1c3685c88d268c7ecdc2dad22608198dc0b295d 
  3rdparty/libprocess/src/tests/metrics_tests.cpp 
33539e45bab8f79afa773a325465016f01fd3b75 
  3rdparty/libprocess/src/tests/process_tests.cpp 
b985fb77ea05fae5c0b144ea48814acc7bb5135b 

Diff: https://reviews.apache.org/r/27446/diff/


Testing
---

make check


Thanks,

Evelina Dumitrescu

Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui #2531

[Apache Jenkins Server]

See 


Changes:

[idownes] Update configure.ac version to 0.22.0

--
[...truncated 51705 lines...]
I1106 01:16:23.578289  4825 replica.cpp:508] Replica received write request for 
position 0
I1106 01:16:23.578351  4825 leveldb.cpp:438] Reading position from leveldb took 
30534ns
I1106 01:16:23.579139  4825 leveldb.cpp:343] Persisting action (14 bytes) to 
leveldb took 749601ns
I1106 01:16:23.579170  4825 replica.cpp:676] Persisted action at 0
I1106 01:16:23.579807  4830 replica.cpp:655] Replica received learned notice 
for position 0
I1106 01:16:23.580521  4830 leveldb.cpp:343] Persisting action (16 bytes) to 
leveldb took 669694ns
I1106 01:16:23.580548  4830 replica.cpp:676] Persisted action at 0
I1106 01:16:23.580569  4830 replica.cpp:661] Replica learned NOP action at 
position 0
I1106 01:16:23.581079  4834 log.cpp:672] Writer started with ending position 0
I1106 01:16:23.582252  4833 leveldb.cpp:438] Reading position from leveldb took 
28002ns
I1106 01:16:23.585305  4831 registrar.cpp:346] Successfully fetched the 
registry (0B) in 14.815232ms
I1106 01:16:23.585420  4831 registrar.cpp:445] Applied 1 operations in 17179ns; 
attempting to update the 'registry'
I1106 01:16:23.588289  4828 log.cpp:680] Attempting to append 139 bytes to the 
log
I1106 01:16:23.588444  4835 coordinator.cpp:340] Coordinator attempting to 
write APPEND action at position 1
I1106 01:16:23.589347  4825 replica.cpp:508] Replica received write request for 
position 1
I1106 01:16:23.590143  4825 leveldb.cpp:343] Persisting action (158 bytes) to 
leveldb took 761893ns
I1106 01:16:23.590173  4825 replica.cpp:676] Persisted action at 1
I1106 01:16:23.590919  4834 replica.cpp:655] Replica received learned notice 
for position 1
I1106 01:16:23.591642  4834 leveldb.cpp:343] Persisting action (160 bytes) to 
leveldb took 689963ns
I1106 01:16:23.591668  4834 replica.cpp:676] Persisted action at 1
I1106 01:16:23.591689  4834 replica.cpp:661] Replica learned APPEND action at 
position 1
I1106 01:16:23.592576  4830 registrar.cpp:490] Successfully updated the 
'registry' in 7.100928ms
I1106 01:16:23.592712  4830 registrar.cpp:376] Successfully recovered registrar
I1106 01:16:23.592919  4833 log.cpp:699] Attempting to truncate the log to 1
I1106 01:16:23.592969  4822 master.cpp:1116] Recovered 0 slaves from the 
Registry (101B) ; allowing 10mins for slaves to re-register
I1106 01:16:23.593080  4823 coordinator.cpp:340] Coordinator attempting to 
write TRUNCATE action at position 2
I1106 01:16:23.593986  4826 replica.cpp:508] Replica received write request for 
position 2
I1106 01:16:23.594663  4826 leveldb.cpp:343] Persisting action (16 bytes) to 
leveldb took 646652ns
I1106 01:16:23.594689  4826 replica.cpp:676] Persisted action at 2
I1106 01:16:23.595332  4835 replica.cpp:655] Replica received learned notice 
for position 2
I1106 01:16:23.596163  4835 leveldb.cpp:343] Persisting action (18 bytes) to 
leveldb took 803264ns
I1106 01:16:23.596233  4835 leveldb.cpp:401] Deleting ~1 keys from leveldb took 
41156ns
I1106 01:16:23.596256  4835 replica.cpp:676] Persisted action at 2
I1106 01:16:23.596278  4835 replica.cpp:661] Replica learned TRUNCATE action at 
position 2
I1106 01:16:23.608346  4830 slave.cpp:169] Slave started on 
173)@67.195.81.186:45765
I1106 01:16:23.608391  4830 credentials.hpp:84] Loading credential for 
authentication from 
'/tmp/GarbageCollectorIntegrationTest_DiskUsage_UrQSGU/credential'
I1106 01:16:23.608620  4830 slave.cpp:276] Slave using credential for: 
test-principal
I1106 01:16:23.608861  4830 slave.cpp:289] Slave resources: cpus(*):2; 
mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1106 01:16:23.608960  4830 slave.cpp:318] Slave hostname: penates.apache.org
I1106 01:16:23.608978  4830 slave.cpp:319] Slave checkpoint: false
W1106 01:16:23.608989  4830 slave.cpp:321] Disabling checkpointing is 
deprecated and the --checkpoint flag will be removed in a future release. 
Please avoid using this flag
I1106 01:16:23.609730  4832 state.cpp:33] Recovering state from 
'/tmp/GarbageCollectorIntegrationTest_DiskUsage_UrQSGU/meta'
I1106 01:16:23.610039  4824 status_update_manager.cpp:197] Recovering status 
update manager
I1106 01:16:23.610363  4832 slave.cpp:3466] Finished recovery
I1106 01:16:23.610904  4822 status_update_manager.cpp:171] Pausing sending 
status updates
I1106 01:16:23.610908  4832 slave.cpp:602] New master detected at 
master@67.195.81.186:45765
I1106 01:16:23.610975  4832 slave.cpp:665] Authenticating with master 
master@67.195.81.186:45765
I1106 01:16:23.611120  4832 slave.cpp:638] Detecting new master
I1106 01:16:23.611183  4834 authenticatee.hpp:133] Creating new client SASL 
connection
I1106 01:16:23.611356  4828 master.cpp:3896] Authenticating 
slave(173)@67.195.81.186:45765
I1106 01:16:23.611397  4828 master.cpp:3907] Using default CRAM-MD5

Re: Cutting 0.21.0.

[Vinod Kone]

The freeze is over. You can commit now. Thanks!

On Wed, Nov 5, 2014 at 1:59 PM, Vinod Kone  wrote:

> This is happening now. Please avoid making any more commits upstream while
> this is in progress.
>
> On Mon, Nov 3, 2014 at 10:18 AM, Ian Downes 
> wrote:
>
>> Hello all,
>> We were waiting to get a number of fixes in on Friday; thanks for your
>> patience. They've been committed so I'm tagging the release candidate
>> today. I'll follow up shortly once the release candidate has been built.
>>
>> Thanks,
>> Ian
>>
>> On Sat, Nov 1, 2014 at 9:14 AM, Tom Arnfeld  wrote:
>>
>> > Has there been any further discussion on getting a release candidate for
>> > 0.21.0 cut?
>> >
>> > On 22 October 2014 22:12, Ian Downes 
>> wrote:
>> >
>> > > Please note that we're targeting to cut this release next Wednesday,
>> 29
>> > > October.
>> > >
>> > > Ian
>> > >
>> > > On Wed, Oct 22, 2014 at 1:33 PM, Vinod Kone 
>> wrote:
>> > >
>> > > > Can everyone who has ticket(s) that they absolutely want to get in
>> > 0.21.0
>> > > > please mark them with "target version" as 0.21.0? That will make it
>> > easy
>> > > to
>> > > > track the blockers.
>> > > >
>> > > > On Wed, Oct 22, 2014 at 11:44 AM, Adam Bordelon > >
>> > > > wrote:
>> > > >
>> > > > > I'd also like to see more of the modules work land in 0.21,
>> > especially
>> > > > the
>> > > > > Authenticator module (MESOS-1889).
>> > > > > I expect it to land in less than a week, but I don't know what
>> your
>> > > > > timeframe is for 0.21.
>> > > > >
>> > > > > On Wed, Oct 22, 2014 at 11:22 AM, Ian Downes
>> > > > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Can someone please volunteer to shepherd this work and comment
>> on
>> > the
>> > > > > state
>> > > > > > of the review?
>> > > > > >
>> > > > > > On Tue, Oct 21, 2014 at 9:01 PM, R.B. Boyer <
>> > are...@nexusvector.net>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Can someone see if MESOS-1873
>> > > > > > >  is
>> suitable
>> > for
>> > > > > > 0.21.0?
>> > > > > > >
>> > > > > > > The patch is super simple <
>> https://reviews.apache.org/r/26622/>
>> > > and
>> > > > > > fixes
>> > > > > > > a
>> > > > > > > showstopper bug in the command executor.
>> > > > > > >
>> > > > > > > On Tue, Oct 21, 2014 at 10:52 PM, Benjamin Hindman <
>> > > > > > b...@eecs.berkeley.edu
>> > > > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Awesome, thanks Ben/Ian!
>> > > > > > > >
>> > > > > > > > We've got some Docker updates that we want to land in
>> 0.21.0.
>> > My
>> > > > > > estimate
>> > > > > > > > is it will land sometime this week, or early next week.
>> > > > > > > >
>> > > > > > > > On Tue, Oct 21, 2014 at 6:51 PM, Benjamin Mahler <
>> > > > > > > > benjamin.mah...@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi all,
>> > > > > > > > >
>> > > > > > > > > We would like to cut 0.21.0 very soon to release the task
>> > > > > > > reconciliation
>> > > > > > > > > work that has been completed recently. I spoke with Ian
>> > Downes
>> > > > who
>> > > > > > was
>> > > > > > > > > willing to be the release manager.
>> > > > > > > > >
>> > > > > > > > > I will let him reply here with a ticket and with other
>> > features
>> > > > > that
>> > > > > > > have
>> > > > > > > > > made it in the 0.21.0 release.
>> > > > > > > > >
>> > > > > > > > > Please reply to this thread if you have anything that you
>> > think
>> > > > > needs
>> > > > > > > to
>> > > > > > > > > land in 0.21.0!
>> > > > > > > > >
>> > > > > > > > > Ben
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Re: Review Request 27651: Updated CRAMMD5Authentication tests to additionally run against the test authenticatee module.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27651/#review60071
---


Bad patch!

Reviews applied: [27619, 27493, 27494, 27495, 27651]

Failed command: ./support/apply-review.sh -n -r 27651

Error:
 2014-11-06 00:29:58 URL:https://reviews.apache.org/r/27651/diff/raw/ 
[10571/10571] -> "27651.patch" [1]
error: patch failed: src/tests/module.cpp:70
error: src/tests/module.cpp: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Nov. 6, 2014, 12:26 a.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27651/
> ---
> 
> (Updated Nov. 6, 2014, 12:26 a.m.)
> 
> 
> Review request for mesos, Adam B and Vinod Kone.
> 
> 
> Bugs: MESOS-2001
> https://issues.apache.org/jira/browse/MESOS-2001
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CRAMMD5Authentication tests now cover all default and modularized 
> authenticator/authenticatee combinations.
> Also adds a test for missing credentials for the authenticatee.
> 
> 
> Diffs
> -
> 
>   src/authentication/cram_md5/authenticatee.hpp 3088a77 
>   src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
>   src/tests/module.hpp 7970c28 
>   src/tests/module.cpp 482ed22 
> 
> Diff: https://reviews.apache.org/r/27651/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Review Request 27651: Updated CRAMMD5Authentication tests to additionally run against the test authenticatee module

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27651/
---

Review request for mesos, Adam B and Vinod Kone.


Bugs: MESOS-2001
https://issues.apache.org/jira/browse/MESOS-2001


Repository: mesos-git


Description
---

CRAMMD5Authentication tests now cover all default and modularized 
authenticator/authenticatee combinations.
Also adds a test for missing credentials for the authenticatee.


Diffs
-

  src/authentication/cram_md5/authenticatee.hpp 3088a77 
  src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
  src/tests/module.hpp 7970c28 
  src/tests/module.cpp 482ed22 

Diff: https://reviews.apache.org/r/27651/diff/


Testing
---

make check


Thanks,

Till Toenshoff

Re: Review Request 27651: Updated CRAMMD5Authentication tests to additionally run against the test authenticatee module.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27651/
---

(Updated Nov. 6, 2014, 12:26 a.m.)


Review request for mesos, Adam B and Vinod Kone.


Summary (updated)
-

Updated CRAMMD5Authentication tests to additionally run against the test 
authenticatee module.


Bugs: MESOS-2001
https://issues.apache.org/jira/browse/MESOS-2001


Repository: mesos-git


Description
---

CRAMMD5Authentication tests now cover all default and modularized 
authenticator/authenticatee combinations.
Also adds a test for missing credentials for the authenticatee.


Diffs
-

  src/authentication/cram_md5/authenticatee.hpp 3088a77 
  src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
  src/tests/module.hpp 7970c28 
  src/tests/module.cpp 482ed22 

Diff: https://reviews.apache.org/r/27651/diff/


Testing
---

make check


Thanks,

Till Toenshoff

Re: Review Request 26736: Ensure docker calls stop if a docker_stop_timeout is provided Added the flag docker_stop_timeout that defaults to 0, if the timeout is 0 the a docker kill will be run, otherw

[Ryan Thomas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26736/
---

(Updated Nov. 6, 2014, 12:19 a.m.)


Review request for mesos, Benjamin Hindman and Timothy Chen.


Bugs: MESOS-1925
https://issues.apache.org/jira/browse/MESOS-1925


Repository: mesos-git


Description
---

Ensure docker calls stop if a docker_stop_timeout is provided Added the flag 
docker_stop_timeout that defaults to 0, if the timeout is 0 the a docker kill 
will be run, otherwise a docker stop with the timeout in seconds is used. A 
negative value will result in a Failure.


Diffs
-

  src/docker/docker.hpp 2dc692c51d6f9d45d8d7c59f8678991fd80c6e89 
  src/docker/docker.cpp 0c0a1bfd3c1260ebdf342d0a6e0e04d7b6a111a4 
  src/slave/containerizer/docker.cpp a6689203adbdcb0ad12583389eaeb83329e4ef6b 
  src/slave/flags.hpp 319c002c66212ffb11759dbcc39f7661203a3e4e 
  src/tests/docker_containerizer_tests.cpp 
9d4ccc57f58d61c62aab5cdc79a129e987920bf6 
  src/tests/docker_tests.cpp ff06a01d7f20656c05bcf5afd3bc10059dc5378d 

Diff: https://reviews.apache.org/r/26736/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --gtest_filter="DOCKER" --verbose 
support/mesos-style.py


Thanks,

Ryan Thomas

Re: Review Request 26736: Ensure docker calls stop if a docker_stop_timeout is provided Added the flag docker_stop_timeout that defaults to 0, if the timeout is 0 the a docker kill will be run, otherw

[Timothy Chen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26736/#review60070
---

Ship it!


Ship It!

- Timothy Chen


On Nov. 4, 2014, 9:28 p.m., Ryan Thomas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26736/
> ---
> 
> (Updated Nov. 4, 2014, 9:28 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Bugs: MESOS-1925
> https://issues.apache.org/jira/browse/MESOS-1925
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Ensure docker calls stop if a docker_stop_timeout is provided Added the flag 
> docker_stop_timeout that defaults to 0, if the timeout is 0 the a docker kill 
> will be run, otherwise a docker stop with the timeout in seconds is used. A 
> negative value will result in a Failure.
> 
> 
> Diffs
> -
> 
>   src/docker/docker.hpp 2dc692c51d6f9d45d8d7c59f8678991fd80c6e89 
>   src/docker/docker.cpp 0c0a1bfd3c1260ebdf342d0a6e0e04d7b6a111a4 
>   src/slave/containerizer/docker.cpp a6689203adbdcb0ad12583389eaeb83329e4ef6b 
>   src/slave/flags.hpp 319c002c66212ffb11759dbcc39f7661203a3e4e 
>   src/tests/docker_containerizer_tests.cpp 
> 9d4ccc57f58d61c62aab5cdc79a129e987920bf6 
>   src/tests/docker_tests.cpp ff06a01d7f20656c05bcf5afd3bc10059dc5378d 
> 
> Diff: https://reviews.apache.org/r/26736/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --gtest_filter="DOCKER" --verbose 
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ryan Thomas
> 
>

Re: Review Request 27494: Integrated CRAM-MD5 Test Authenticatee module into slave.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27494/
---

(Updated Nov. 6, 2014, 12:19 a.m.)


Review request for mesos, Adam B and Vinod Kone.


Changes
---

rebased


Bugs: MESOS-2001
https://issues.apache.org/jira/browse/MESOS-2001


Repository: mesos-git


Description
---

Also fixes messages.proto to use a raw bytestream instead of a string for 
AuthenticationStartMessage as non CRAM-MD5 authentication may transmit binary 
data.

Note that this patch covers modularized slave authentication only. Framework 
authentication currently is currently covered by the default (built-in) 
implementation. There will be a subsequent patch for modularized framework 
authentication. 


Diffs (updated)
-

  src/messages/messages.proto de0e2a2 
  src/sched/sched.cpp e5f828d 
  src/scheduler/scheduler.cpp c74187c 
  src/slave/constants.hpp 701dd89 
  src/slave/constants.cpp d6ad78c 
  src/slave/flags.hpp efbd35d 
  src/slave/slave.hpp 5b082fc 
  src/slave/slave.cpp dbfd1a8 

Diff: https://reviews.apache.org/r/27494/diff/


Testing
---

make check

NOTE all three CRAM-MD5 authenticatee module related RRs need to get applied 
before running make check.


Thanks,

Till Toenshoff

Re: Review Request 27648: Update configure.ac to 0.22.0

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27648/#review60069
---


Patch looks great!

Reviews applied: [27648]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 11:39 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27648/
> ---
> 
> (Updated Nov. 5, 2014, 11:39 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Update configure.ac to 0.22.0
> 
> 
> Diffs
> -
> 
>   configure.ac 342e762a09b17f50467dc1f9c6742971b7d5e561 
> 
> Diff: https://reviews.apache.org/r/27648/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27619: Updated CRAMMD5Authentication tests to additionally run against the test authenticator module.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27619/
---

(Updated Nov. 6, 2014, 12:05 a.m.)


Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.


Changes
---

Fixed comment.


Bugs: MESOS-1894
https://issues.apache.org/jira/browse/MESOS-1894


Repository: mesos-git


Description
---

see summary.


Diffs (updated)
-

  src/authentication/cram_md5/authenticator.hpp 601248d 
  src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
  src/tests/module.hpp 7970c28 
  src/tests/module.cpp 482ed22 

Diff: https://reviews.apache.org/r/27619/diff/


Testing
---

make check


Thanks,

Till Toenshoff

Re: Review Request 27648: Update configure.ac to 0.22.0

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27648/#review60065
---

Ship it!


Ship It!

- Vinod Kone


On Nov. 5, 2014, 11:39 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27648/
> ---
> 
> (Updated Nov. 5, 2014, 11:39 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Update configure.ac to 0.22.0
> 
> 
> Diffs
> -
> 
>   configure.ac 342e762a09b17f50467dc1f9c6742971b7d5e561 
> 
> Diff: https://reviews.apache.org/r/27648/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

> On Nov. 5, 2014, 10:12 p.m., Vinod Kone wrote:
> > Thanks! I'll get this committed once 0.21.0 is cut.
> 
> Ankur Chauhan wrote:
> Thanks! And really appreciate all the input, help and hand holding. It's 
> been a long time since I touched C++. Future patches would probably require 
> less handholding.
> 
> Thanks everyone.. @vinod, @tnachen, @tstclair

You are welcome. We love patches, especically those that come with tests :)


- Vinod


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60049
---


On Nov. 5, 2014, 10:07 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 10:07 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27619: Updated CRAMMD5Authentication tests to additionally run against the test authenticator module.

[Till Toenshoff]

> On Nov. 5, 2014, 9:06 p.m., Adam B wrote:
> > src/tests/cram_md5_authentication_tests.cpp, line 211
> > 
> >
> > Why was this test explicitly creating new CRAMMD5Authenticator instead 
> > of using the default initializer with "CRAMMD5Authenticator authenticator;" 
> > like the previous tests?
> > Doesn't seem like it did anything special with the pointer.

IIUC typed tests demand a factory method.


> On Nov. 5, 2014, 9:06 p.m., Adam B wrote:
> > src/tests/module.cpp, line 81
> > 
> >
> > Fix the comment: "Now add our test Authenticator module"

whoops, will update this in a jiffy.


- Till


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27619/#review60029
---


On Nov. 5, 2014, 3:27 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27619/
> ---
> 
> (Updated Nov. 5, 2014, 3:27 p.m.)
> 
> 
> Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.
> 
> 
> Bugs: MESOS-1894
> https://issues.apache.org/jira/browse/MESOS-1894
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> see summary.
> 
> 
> Diffs
> -
> 
>   src/authentication/cram_md5/authenticator.hpp 601248d 
>   src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
>   src/tests/module.hpp 7970c28 
>   src/tests/module.cpp 482ed22 
> 
> Diff: https://reviews.apache.org/r/27619/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Review Request 27648: Update configure.ac to 0.22.0

[Ian Downes]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27648/
---

Review request for mesos and Vinod Kone.


Repository: mesos-git


Description
---

Update configure.ac to 0.22.0


Diffs
-

  configure.ac 342e762a09b17f50467dc1f9c6742971b7d5e561 

Diff: https://reviews.apache.org/r/27648/diff/


Testing
---


Thanks,

Ian Downes

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

> On Nov. 5, 2014, 10:12 p.m., Vinod Kone wrote:
> > Thanks! I'll get this committed once 0.21.0 is cut.

Thanks! And really appreciate all the input, help and hand holding. It's been a 
long time since I touched C++. Future patches would probably require less 
handholding.

Thanks everyone.. @vinod, @tnachen, @tstclair


- Ankur


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60049
---


On Nov. 5, 2014, 10:07 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 10:07 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

[VOTE] Release Apache Mesos 0.21.0 (rc1)

[Ian Downes]

Hi all,

Please vote on releasing the following candidate as Apache Mesos 0.21.0.


0.21.0 includes the following:

State reconciliation for frameworks
Support for Mesos modules
Task status now includes source and reason
A shared filesystem isolator
A pid namespace isolator

The CHANGELOG for the release is available at:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.21.0-rc1


The candidate for Mesos 0.21.0 release is available at:
https://dist.apache.org/repos/dist/dev/mesos/0.21.0-rc1/mesos-0.21.0.tar.gz

The tag to be voted on is 0.21.0-rc1:
https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.21.0-rc1

The MD5 checksum of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.21.0-rc1/mesos-0.21.0.tar.gz.md5

The signature of the tarball can be found at:
https://dist.apache.org/repos/dist/dev/mesos/0.21.0-rc1/mesos-0.21.0.tar.gz.asc

The PGP key used to sign the release is here:
https://dist.apache.org/repos/dist/release/mesos/KEYS

The JAR is up in Maven in a staging repository here:
https://repository.apache.org/content/repositories/orgapachemesos-1038

Please vote on releasing this package as Apache Mesos 0.21.0!

The vote is open until Sat Nov  8 15:09:48 PST 2014 and passes if a
majority of at least 3 +1 PMC votes are cast.

[ ] +1 Release this package as Apache Mesos 0.21.0
[ ] -1 Do not release this package because ...

Thanks,

Ian Downes

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60054
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 10:07 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 10:07 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60049
---

Ship it!


Thanks! I'll get this committed once 0.21.0 is cut.

- Vinod Kone


On Nov. 5, 2014, 10:07 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 10:07 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
---

(Updated Nov. 5, 2014, 10:07 p.m.)


Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.


Changes
---

Minor style fixes and removed unused code


Bugs: MESOS-1711
https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
---

Previously, the fetcher used a hardcoded list of schemes to determine what URIs 
could be fetched by hadoop (if available). This is now changed such that we 
first check if hadoop can fetch them for us and then we fallback to the os::net 
and then a local copy method (same as it used to be). This allows users to 
fetch artifacts from arbitrary filesystems as long as hadoop is correctly 
configured (in core-site.xml).


Diffs (updated)
-

  src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Cutting 0.21.0.

[Vinod Kone]

This is happening now. Please avoid making any more commits upstream while
this is in progress.

On Mon, Nov 3, 2014 at 10:18 AM, Ian Downes 
wrote:

> Hello all,
> We were waiting to get a number of fixes in on Friday; thanks for your
> patience. They've been committed so I'm tagging the release candidate
> today. I'll follow up shortly once the release candidate has been built.
>
> Thanks,
> Ian
>
> On Sat, Nov 1, 2014 at 9:14 AM, Tom Arnfeld  wrote:
>
> > Has there been any further discussion on getting a release candidate for
> > 0.21.0 cut?
> >
> > On 22 October 2014 22:12, Ian Downes 
> wrote:
> >
> > > Please note that we're targeting to cut this release next Wednesday, 29
> > > October.
> > >
> > > Ian
> > >
> > > On Wed, Oct 22, 2014 at 1:33 PM, Vinod Kone 
> wrote:
> > >
> > > > Can everyone who has ticket(s) that they absolutely want to get in
> > 0.21.0
> > > > please mark them with "target version" as 0.21.0? That will make it
> > easy
> > > to
> > > > track the blockers.
> > > >
> > > > On Wed, Oct 22, 2014 at 11:44 AM, Adam Bordelon 
> > > > wrote:
> > > >
> > > > > I'd also like to see more of the modules work land in 0.21,
> > especially
> > > > the
> > > > > Authenticator module (MESOS-1889).
> > > > > I expect it to land in less than a week, but I don't know what your
> > > > > timeframe is for 0.21.
> > > > >
> > > > > On Wed, Oct 22, 2014 at 11:22 AM, Ian Downes
> > >  > > > >
> > > > > wrote:
> > > > >
> > > > > > Can someone please volunteer to shepherd this work and comment on
> > the
> > > > > state
> > > > > > of the review?
> > > > > >
> > > > > > On Tue, Oct 21, 2014 at 9:01 PM, R.B. Boyer <
> > are...@nexusvector.net>
> > > > > > wrote:
> > > > > >
> > > > > > > Can someone see if MESOS-1873
> > > > > > >  is suitable
> > for
> > > > > > 0.21.0?
> > > > > > >
> > > > > > > The patch is super simple  >
> > > and
> > > > > > fixes
> > > > > > > a
> > > > > > > showstopper bug in the command executor.
> > > > > > >
> > > > > > > On Tue, Oct 21, 2014 at 10:52 PM, Benjamin Hindman <
> > > > > > b...@eecs.berkeley.edu
> > > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Awesome, thanks Ben/Ian!
> > > > > > > >
> > > > > > > > We've got some Docker updates that we want to land in 0.21.0.
> > My
> > > > > > estimate
> > > > > > > > is it will land sometime this week, or early next week.
> > > > > > > >
> > > > > > > > On Tue, Oct 21, 2014 at 6:51 PM, Benjamin Mahler <
> > > > > > > > benjamin.mah...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > We would like to cut 0.21.0 very soon to release the task
> > > > > > > reconciliation
> > > > > > > > > work that has been completed recently. I spoke with Ian
> > Downes
> > > > who
> > > > > > was
> > > > > > > > > willing to be the release manager.
> > > > > > > > >
> > > > > > > > > I will let him reply here with a ticket and with other
> > features
> > > > > that
> > > > > > > have
> > > > > > > > > made it in the 0.21.0 release.
> > > > > > > > >
> > > > > > > > > Please reply to this thread if you have anything that you
> > think
> > > > > needs
> > > > > > > to
> > > > > > > > > land in 0.21.0!
> > > > > > > > >
> > > > > > > > > Ben
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review60045
---


Patch looks great!

Reviews applied: [27633]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 8:56 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 8:56 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 05fcb4da92de8a4d75dc936e3d92d8f10675e244 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60040
---


looking goood. minor fixes and i'll get this committed.


src/launcher/fetcher.cpp


s/uri/URI/



src/launcher/fetcher.cpp


s/uri/URI/



src/launcher/fetcher.cpp


s/uris/URIs/



src/launcher/fetcher.cpp


fetcher is not supposed to be run manually by users. so i would kill this.



src/launcher/fetcher.cpp


kill these and the usage(), printVersion() functions.



src/tests/fetcher_tests.cpp


pull this down to where it is used (#99).



src/tests/fetcher_tests.cpp


2 blank lines.


- Vinod Kone


On Nov. 5, 2014, 9:24 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 9:24 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review60038
---

Ship it!



CHANGELOG


Sorry, missed this last time. This should be called out as an API change at 
the top.

* API Changes:
  * [MESOS-1461] - Add task reconciliation to the Python API.


- Vinod Kone


On Nov. 5, 2014, 8:56 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 8:56 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 05fcb4da92de8a4d75dc936e3d92d8f10675e244 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
---

(Updated Nov. 5, 2014, 9:24 p.m.)


Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.


Changes
---

Add locally hosted http server for os::net tests


Bugs: MESOS-1711
https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
---

Previously, the fetcher used a hardcoded list of schemes to determine what URIs 
could be fetched by hadoop (if available). This is now changed such that we 
first check if hadoop can fetch them for us and then we fallback to the os::net 
and then a local copy method (same as it used to be). This allows users to 
fetch artifacts from arbitrary filesystems as long as hadoop is correctly 
configured (in core-site.xml).


Diffs (updated)
-

  src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60032
---


Patch looks great!

Reviews applied: [27483]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 7:57 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 7:57 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27619: Updated CRAMMD5Authentication tests to additionally run against the test authenticator module.

[Adam B]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27619/#review60029
---

Ship it!


Excellent. Elegant and clean. Just update the comment and then we can commit it.


src/tests/cram_md5_authentication_tests.cpp


Why was this test explicitly creating new CRAMMD5Authenticator instead of 
using the default initializer with "CRAMMD5Authenticator authenticator;" like 
the previous tests?
Doesn't seem like it did anything special with the pointer.



src/tests/module.cpp


Fix the comment: "Now add our test Authenticator module"


- Adam B


On Nov. 5, 2014, 7:27 a.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27619/
> ---
> 
> (Updated Nov. 5, 2014, 7:27 a.m.)
> 
> 
> Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.
> 
> 
> Bugs: MESOS-1894
> https://issues.apache.org/jira/browse/MESOS-1894
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> see summary.
> 
> 
> Diffs
> -
> 
>   src/authentication/cram_md5/authenticator.hpp 601248d 
>   src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
>   src/tests/module.hpp 7970c28 
>   src/tests/module.cpp 482ed22 
> 
> Diff: https://reviews.apache.org/r/27619/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Kapil Arya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review60030
---



CHANGELOG


We need to mention the support for Mesos modules for master, slave and 
tests using the --modules flag.


- Kapil Arya


On Nov. 5, 2014, 3:56 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 3:56 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 05fcb4da92de8a4d75dc936e3d92d8f10675e244 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Ian Downes]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/
---

(Updated Nov. 5, 2014, 12:56 p.m.)


Review request for mesos and Vinod Kone.


Bugs: MESOS-1964
https://issues.apache.org/jira/browse/MESOS-1964


Repository: mesos-git


Description
---

CHANGELOG for 0.21.0


Diffs (updated)
-

  CHANGELOG 05fcb4da92de8a4d75dc936e3d92d8f10675e244 
  docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 

Diff: https://reviews.apache.org/r/27633/diff/


Testing
---


Thanks,

Ian Downes

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

> On Nov. 5, 2014, 7:29 a.m., Timothy Chen wrote:
> > src/tests/fetcher_tests.cpp, line 121
> > 
> >
> > How does this test work with a bogus url?
> 
> Ankur Chauhan wrote:
> I used it because it was mentioned in 
> [RFC-2606](http://tools.ietf.org/html/rfc2606) and seemed like a reliable 
> resource. But then maybe reaching out into the internet is not the best thing 
> to do either but it correctly exercises the code path.
> 
> ```
> $ curl http://www.example.com/index.html
> 
> 
> 
> Example Domain
> 
> 
> 
> 
> 
> body {
> background-color: #f0f0f2;
> margin: 0;
> padding: 0;
> font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, 
> sans-serif;
> 
> }
> div {
> width: 600px;
> margin: 5em auto;
> padding: 50px;
> background-color: #fff;
> border-radius: 1em;
> }
> a:link, a:visited {
> color: #38488f;
> text-decoration: none;
> }
> @media (max-width: 700px) {
> body {
> background-color: #fff;
> }
> div {
> width: auto;
> margin: 0 auto;
> border-radius: 0;
> padding: 1em;
> }
> }
> 
> 
> 
> 
> 
> Example Domain
> This domain is established to be used for illustrative examples in 
> documents. You may use this
> domain in examples without prior coordination or asking for 
> permission.
> http://www.iana.org/domains/example";>More 
> information...
> 
> 
> 
> ```
> 
> Vinod Kone wrote:
> AFAIK, this would be the first test that require Internet access, which 
> is unfortunate.
> 
> 2 options:
> 
> --> Setup a simple HTTP server that serves a file/string via an endpoint. 
> See 3rdparty/libprocess/src/tests/http_tests.cpp for examples.
> 
> --> Hit :/help. You can get the 
> libprocess ip and port by doing process::ip() and process::port(). I think 
> that should give a valid response.

I'll have a look and update it.


- Ankur


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review59938
---


On Nov. 5, 2014, 7:57 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 7:57 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60020
---



src/launcher/fetcher.cpp


This change makes fetcher in line with slave/main.cpp or master/main.cpp 
and initialized the logging.


- Ankur Chauhan


On Nov. 5, 2014, 7:57 p.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 7:57 p.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/
---

(Updated Nov. 5, 2014, 7:57 p.m.)


Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.


Changes
---

Initialize logging and add usual flags handling (same as per other parts of 
mesos), fix comments


Bugs: MESOS-1711
https://issues.apache.org/jira/browse/MESOS-1711


Repository: mesos-git


Description
---

Previously, the fetcher used a hardcoded list of schemes to determine what URIs 
could be fetched by hadoop (if available). This is now changed such that we 
first check if hadoop can fetch them for us and then we fallback to the os::net 
and then a local copy method (same as it used to be). This allows users to 
fetch artifacts from arbitrary filesystems as long as hadoop is correctly 
configured (in core-site.xml).


Diffs (updated)
-

  src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27483/diff/


Testing
---

make check
sudo bin/mesos-tests.sh --verbose
support/mesos-style.py


Thanks,

Ankur Chauhan

Re: Logging warning in mesos fetcher

[Ankur Chauhan]

Will do!

-- Ankur

> On 5 Nov 2014, at 10:56, Vinod Kone  wrote:
> 
> Good catch. You want to add the log initialization to the main() in
> fetcher.cpp. See master/main.cpp or slave/main.cpp for examples.
> 
> On Wed, Nov 5, 2014 at 10:32 AM, Ankur Chauhan  wrote:
> 
>> I noticed in a run of bin/mesos-tests.sh --gtest_filter=FetcherTest.*:
>> 
>> $ ./bin/mesos-tests.sh --gtest_filter=FetcherTest.*
>> Source directory: /Users/achauhan/Projects/mesos
>> Build directory: /Users/achauhan/Projects/mesos/build
>> -
>> We cannot run any Docker tests because:
>> Docker tests not supported on non-Linux systems
>> -
>> Note: Google Test filter =
>> FetcherTest.*-DockerContainerizerTest.ROOT_DOCKER_Launch:DockerContainerizerTest.ROOT_DOCKER_Kill:DockerContainerizerTest.ROOT_DOCKER_Usage:DockerContainerizerTest.DISABLED_ROOT_DOCKER_Recover:DockerContainerizerTest.ROOT_DOCKER_Logs:DockerContainerizerTest.ROOT_DOCKER_Default_CMD:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Override:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Args:DockerContainerizerTest.ROOT_DOCKER_SlaveRecoveryTaskContainer:DockerContainerizerTest.DISABLED_ROOT_DOCKER_SlaveRecoveryExecutorContainer:DockerContainerizerTest.ROOT_DOCKER_PortMapping:DockerContainerizerTest.ROOT_DOCKER_LaunchSandboxWithColon:DockerTest.ROOT_DOCKER_interface:DockerTest.ROOT_DOCKER_CheckCommandWithShell:DockerTest.ROOT_DOCKER_CheckPortResource:DockerTest.ROOT_DOCKER_CancelPull:SlaveTest.ROOT_RunTaskWithCommandInfoWithoutUser:SlaveTest.DISABLED_ROOT_RunTaskWithCommandInfoWithUser:SlaveCount/Registrar_BENCHMARK_Test.performance/0:SlaveCount/Registrar_BENCHMARK_Test.performance/1:SlaveCount/Registrar_BENCHMARK_Test.performance/2:SlaveCount/Registrar_BENCHMARK_Test.performance/3
>> [==] Running 4 tests from 1 test case.
>> [--] Global test environment set-up.
>> [--] 4 tests from FetcherTest
>> [ RUN  ] FetcherTest.FileURI
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>> I1104 22:43:38.412901 2068996864 fetcher.cpp:197] Fetching URI
>> 'file:///private/tmp/FetcherTest_FileURI_ipAYlv/from/test'
>> I1104 22:43:38.413653 2068996864 fetcher.cpp:178] Copying resource from
>> '/private/tmp/FetcherTest_FileURI_ipAYlv/from/test' to
>> '/private/tmp/FetcherTest_FileURI_ipAYlv'
>> I1104 22:43:38.419317 2068996864 fetcher.cpp:300] Skipped extracting path
>> '/private/tmp/FetcherTest_FileURI_ipAYlv/test'
>> [   OK ] FetcherTest.FileURI (110 ms)
>> [ RUN  ] FetcherTest.FilePath
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>> I1104 22:43:38.507416 2068996864 fetcher.cpp:197] Fetching URI
>> '/private/tmp/FetcherTest_FilePath_lAzOZG/from/test'
>> I1104 22:43:38.508020 2068996864 fetcher.cpp:178] Copying resource from
>> '/private/tmp/FetcherTest_FilePath_lAzOZG/from/test' to
>> '/private/tmp/FetcherTest_FilePath_lAzOZG'
>> I1104 22:43:38.512864 2068996864 fetcher.cpp:300] Skipped extracting path
>> '/private/tmp/FetcherTest_FilePath_lAzOZG/test'
>> [   OK ] FetcherTest.FilePath (100 ms)
>> [ RUN  ] FetcherTest.OSNetUriTest
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>> I1104 22:43:38.608177 2068996864 fetcher.cpp:197] Fetching URI '
>> http://www.example.com/index.html'
>> I1104 22:43:38.608901 2068996864 fetcher.cpp:109] Fetching URI '
>> http://www.example.com/index.html' with os::net
>> I1104 22:43:38.608924 2068996864 fetcher.cpp:119] Downloading '
>> http://www.example.com/index.html' to
>> '/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
>> I1104 22:43:38.619645 2068996864 fetcher.cpp:300] Skipped extracting path
>> '/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
>> [   OK ] FetcherTest.OSNetUriTest (105 ms)
>> [ RUN  ] FetcherTest.FileLocalhostURI
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>> I1104 22:43:38.712461 2068996864 fetcher.cpp:197] Fetching URI
>> 'file://localhost/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test'
>> I1104 22:43:38.713353 2068996864 fetcher.cpp:178] Copying resource from
>> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test' to
>> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH'
>> I1104 22:43:38.718541 2068996864 fetcher.cpp:300] Skipped extracting path
>> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/test'
>> [   OK ] FetcherTest.FileLocalhostURI (103 ms)
>> [--] 4 tests from FetcherTest (419 ms total)
>> 
>> [--] Global test environment tear-down
>> [==] 4 tests from 1 test case ran. (434 ms total)
>> [  PASSED  ] 4 tests.
>> 
>>  YOU HAVE 5 DISABLED TESTS
>> 
>> There are warnings in the output:
>> 
>> WARNING: Logging before InitGoogleLogging() is written to STDERR
>> And looking at the code, I don't see a glog initialization codeblock.
>> Should there be google::InitGoogleLogging(argv[0]); in the main method? I
>> al

Re: Review Request 27567: MESOS-2038: removed dead code from _runTask()

[Ben Mahler]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27567/#review60016
---



src/slave/slave.cpp


A comment here as to why we don't need to send TASK_LOST would be much 
appreciated! It's not obvious so someone might come along and add a TASK_LOST 
to make sure we're not dropping the task on the floor, so context here would be 
great!


- Ben Mahler


On Nov. 4, 2014, 11:46 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27567/
> ---
> 
> (Updated Nov. 4, 2014, 11:46 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-2038
> https://issues.apache.org/jira/browse/MESOS-2038
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Removed a few lines of dead code that coverty discovered.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 5e9b0e4f93a5100a91340e1f6fb1fe160b2eea4b 
> 
> Diff: https://reviews.apache.org/r/27567/diff/
> 
> 
> Testing
> ---
> 
> none.
> expecting/waiting for review bot to report no problem.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

> On Nov. 5, 2014, 7:29 a.m., Timothy Chen wrote:
> > src/tests/fetcher_tests.cpp, line 121
> > 
> >
> > How does this test work with a bogus url?
> 
> Ankur Chauhan wrote:
> I used it because it was mentioned in 
> [RFC-2606](http://tools.ietf.org/html/rfc2606) and seemed like a reliable 
> resource. But then maybe reaching out into the internet is not the best thing 
> to do either but it correctly exercises the code path.
> 
> ```
> $ curl http://www.example.com/index.html
> 
> 
> 
> Example Domain
> 
> 
> 
> 
> 
> body {
> background-color: #f0f0f2;
> margin: 0;
> padding: 0;
> font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, 
> sans-serif;
> 
> }
> div {
> width: 600px;
> margin: 5em auto;
> padding: 50px;
> background-color: #fff;
> border-radius: 1em;
> }
> a:link, a:visited {
> color: #38488f;
> text-decoration: none;
> }
> @media (max-width: 700px) {
> body {
> background-color: #fff;
> }
> div {
> width: auto;
> margin: 0 auto;
> border-radius: 0;
> padding: 1em;
> }
> }
> 
> 
> 
> 
> 
> Example Domain
> This domain is established to be used for illustrative examples in 
> documents. You may use this
> domain in examples without prior coordination or asking for 
> permission.
> http://www.iana.org/domains/example";>More 
> information...
> 
> 
> 
> ```

AFAIK, this would be the first test that require Internet access, which is 
unfortunate.

2 options:

--> Setup a simple HTTP server that serves a file/string via an endpoint. See 
3rdparty/libprocess/src/tests/http_tests.cpp for examples.

--> Hit :/help. You can get the libprocess ip 
and port by doing process::ip() and process::port(). I think that should give a 
valid response.


- Vinod


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review59938
---


On Nov. 5, 2014, 6:39 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 6:39 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review60008
---



src/launcher/fetcher.cpp


this statement needs rephrasing because if it cannot be fetched by 
os::net/libcurl we just error on #221.


- Vinod Kone


On Nov. 5, 2014, 6:39 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 6:39 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Dominic Hamon]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review60007
---


I think you're missing the change i made last night?

- Dominic Hamon


On Nov. 5, 2014, 10:32 a.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 10:32 a.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 55d8ff4af8bda37a47d31ee62bc424ef496311c4 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review60006
---


Bad patch!

Reviews applied: [27633]

Failed command: ./support/apply-review.sh -n -r 27633

Error:
 2014-11-05 19:27:04 URL:https://reviews.apache.org/r/27633/diff/raw/ 
[9214/9214] -> "27633.patch" [1]
error: patch failed: CHANGELOG:1
error: CHANGELOG: patch does not apply
Failed to apply patch

- Mesos ReviewBot


On Nov. 5, 2014, 6:32 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 6:32 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 55d8ff4af8bda37a47d31ee62bc424ef496311c4 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Re: Review Request 27567: MESOS-2038: removed dead code from _runTask()

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27567/#review60002
---

Ship it!


Ship It!

- Vinod Kone


On Nov. 4, 2014, 11:46 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27567/
> ---
> 
> (Updated Nov. 4, 2014, 11:46 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-2038
> https://issues.apache.org/jira/browse/MESOS-2038
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Removed a few lines of dead code that coverty discovered.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 5e9b0e4f93a5100a91340e1f6fb1fe160b2eea4b 
> 
> Diff: https://reviews.apache.org/r/27567/diff/
> 
> 
> Testing
> ---
> 
> none.
> expecting/waiting for review bot to report no problem.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Review Request 27622: Implement MESOS-1248: Use JSON instead of our own format for passing URI information to mesos-fetcher

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27622/#review59998
---


Patch looks great!

Reviews applied: [27516, 27622]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 6:30 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27622/
> ---
> 
> (Updated Nov. 5, 2014, 6:30 p.m.)
> 
> 
> Review request for mesos and Benjamin Hindman.
> 
> 
> Bugs: MESOS-1248
> https://issues.apache.org/jira/browse/MESOS-1248
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Rebasing and replacing https://reviews.apache.org/r/21277/. Now dependent on 
> https://reviews.apache.org/r/27516/ which implements MESOS-1316.
> 
> Replaces the ad-hoc format for env var values that act as fetcher program 
> comd line args with a JSON format that is gained by translation from protobuf 
> parsing to a JSON object. 
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/slave/containerizer/containerizer.cpp 
> 0254679508167a390fd6fed855f19794354ac081 
>   src/slave/containerizer/fetcher.cpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27622/diff/
> 
> 
> Testing
> ---
> 
> Make check.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Logging warning in mesos fetcher

[Vinod Kone]

Good catch. You want to add the log initialization to the main() in
fetcher.cpp. See master/main.cpp or slave/main.cpp for examples.

On Wed, Nov 5, 2014 at 10:32 AM, Ankur Chauhan  wrote:

> I noticed in a run of bin/mesos-tests.sh --gtest_filter=FetcherTest.*:
>
> $ ./bin/mesos-tests.sh --gtest_filter=FetcherTest.*
> Source directory: /Users/achauhan/Projects/mesos
> Build directory: /Users/achauhan/Projects/mesos/build
> -
> We cannot run any Docker tests because:
> Docker tests not supported on non-Linux systems
> -
> Note: Google Test filter =
> FetcherTest.*-DockerContainerizerTest.ROOT_DOCKER_Launch:DockerContainerizerTest.ROOT_DOCKER_Kill:DockerContainerizerTest.ROOT_DOCKER_Usage:DockerContainerizerTest.DISABLED_ROOT_DOCKER_Recover:DockerContainerizerTest.ROOT_DOCKER_Logs:DockerContainerizerTest.ROOT_DOCKER_Default_CMD:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Override:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Args:DockerContainerizerTest.ROOT_DOCKER_SlaveRecoveryTaskContainer:DockerContainerizerTest.DISABLED_ROOT_DOCKER_SlaveRecoveryExecutorContainer:DockerContainerizerTest.ROOT_DOCKER_PortMapping:DockerContainerizerTest.ROOT_DOCKER_LaunchSandboxWithColon:DockerTest.ROOT_DOCKER_interface:DockerTest.ROOT_DOCKER_CheckCommandWithShell:DockerTest.ROOT_DOCKER_CheckPortResource:DockerTest.ROOT_DOCKER_CancelPull:SlaveTest.ROOT_RunTaskWithCommandInfoWithoutUser:SlaveTest.DISABLED_ROOT_RunTaskWithCommandInfoWithUser:SlaveCount/Registrar_BENCHMARK_Test.performance/0:SlaveCount/Registrar_BENCHMARK_Test.performance/1:SlaveCount/Registrar_BENCHMARK_Test.performance/2:SlaveCount/Registrar_BENCHMARK_Test.performance/3
> [==] Running 4 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 4 tests from FetcherTest
> [ RUN  ] FetcherTest.FileURI
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I1104 22:43:38.412901 2068996864 fetcher.cpp:197] Fetching URI
> 'file:///private/tmp/FetcherTest_FileURI_ipAYlv/from/test'
> I1104 22:43:38.413653 2068996864 fetcher.cpp:178] Copying resource from
> '/private/tmp/FetcherTest_FileURI_ipAYlv/from/test' to
> '/private/tmp/FetcherTest_FileURI_ipAYlv'
> I1104 22:43:38.419317 2068996864 fetcher.cpp:300] Skipped extracting path
> '/private/tmp/FetcherTest_FileURI_ipAYlv/test'
> [   OK ] FetcherTest.FileURI (110 ms)
> [ RUN  ] FetcherTest.FilePath
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I1104 22:43:38.507416 2068996864 fetcher.cpp:197] Fetching URI
> '/private/tmp/FetcherTest_FilePath_lAzOZG/from/test'
> I1104 22:43:38.508020 2068996864 fetcher.cpp:178] Copying resource from
> '/private/tmp/FetcherTest_FilePath_lAzOZG/from/test' to
> '/private/tmp/FetcherTest_FilePath_lAzOZG'
> I1104 22:43:38.512864 2068996864 fetcher.cpp:300] Skipped extracting path
> '/private/tmp/FetcherTest_FilePath_lAzOZG/test'
> [   OK ] FetcherTest.FilePath (100 ms)
> [ RUN  ] FetcherTest.OSNetUriTest
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I1104 22:43:38.608177 2068996864 fetcher.cpp:197] Fetching URI '
> http://www.example.com/index.html'
> I1104 22:43:38.608901 2068996864 fetcher.cpp:109] Fetching URI '
> http://www.example.com/index.html' with os::net
> I1104 22:43:38.608924 2068996864 fetcher.cpp:119] Downloading '
> http://www.example.com/index.html' to
> '/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
> I1104 22:43:38.619645 2068996864 fetcher.cpp:300] Skipped extracting path
> '/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
> [   OK ] FetcherTest.OSNetUriTest (105 ms)
> [ RUN  ] FetcherTest.FileLocalhostURI
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I1104 22:43:38.712461 2068996864 fetcher.cpp:197] Fetching URI
> 'file://localhost/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test'
> I1104 22:43:38.713353 2068996864 fetcher.cpp:178] Copying resource from
> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test' to
> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH'
> I1104 22:43:38.718541 2068996864 fetcher.cpp:300] Skipped extracting path
> '/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/test'
> [   OK ] FetcherTest.FileLocalhostURI (103 ms)
> [--] 4 tests from FetcherTest (419 ms total)
>
> [--] Global test environment tear-down
> [==] 4 tests from 1 test case ran. (434 ms total)
> [  PASSED  ] 4 tests.
>
>   YOU HAVE 5 DISABLED TESTS
>
> There are warnings in the output:
>
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> And looking at the code, I don't see a glog initialization codeblock.
> Should there be google::InitGoogleLogging(argv[0]); in the main method? I
> also cannot figure out how logging is configured in general and/or where
> mesos-fetcher is invoked. Can someone chime in?
>
>
> -- Ankur
>

Re: Review Request 27633: CHANGELOG for 0.21.0

[Vinod Kone]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/#review59993
---



CHANGELOG


Allows frameworks to reconcile the states of the tasks.



CHANGELOG


I think you need to do a pull on master. I don't see Dominic's changes to 
the CHANGELOG

* Task status now includes source and reason:
  * [MESOS-343] - Expose TASK_FAILED reason to Frameworks.
  * [MESOS-1143] - Add a TASK_ERROR task status.



CHANGELOG


For consistency, we just say

"This release also includes several bug fixes and stability improvements."

and pull this up above "Sub-task".



CHANGELOG


Move this below before "Task" section. 

Here, we can something like

* Build changes:
  * [MESOS-1044] - Require C++11 compiler support.

General rule is that everything below

"This release includes..." is a copy paste (properly formatted) from JIRA.

the top section just talks about high level important issues.



CHANGELOG


Can you update this to "Task" in the JIRA and here? Also, we don't need 
this at the top because it is already under "Deprecations". Just move this to 
"Task" section below.



CHANGELOG


"Documentation" should be on a new line.



CHANGELOG


"Task" at the end should be on a new line.


- Vinod Kone


On Nov. 5, 2014, 6:32 p.m., Ian Downes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27633/
> ---
> 
> (Updated Nov. 5, 2014, 6:32 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1964
> https://issues.apache.org/jira/browse/MESOS-1964
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> CHANGELOG for 0.21.0
> 
> 
> Diffs
> -
> 
>   CHANGELOG 55d8ff4af8bda37a47d31ee62bc424ef496311c4 
>   docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 
> 
> Diff: https://reviews.apache.org/r/27633/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Ian Downes
> 
>

Logging warning in mesos fetcher

[Ankur Chauhan]

I noticed in a run of bin/mesos-tests.sh --gtest_filter=FetcherTest.*:

$ ./bin/mesos-tests.sh --gtest_filter=FetcherTest.*
Source directory: /Users/achauhan/Projects/mesos
Build directory: /Users/achauhan/Projects/mesos/build
-
We cannot run any Docker tests because:
Docker tests not supported on non-Linux systems
-
Note: Google Test filter = 
FetcherTest.*-DockerContainerizerTest.ROOT_DOCKER_Launch:DockerContainerizerTest.ROOT_DOCKER_Kill:DockerContainerizerTest.ROOT_DOCKER_Usage:DockerContainerizerTest.DISABLED_ROOT_DOCKER_Recover:DockerContainerizerTest.ROOT_DOCKER_Logs:DockerContainerizerTest.ROOT_DOCKER_Default_CMD:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Override:DockerContainerizerTest.ROOT_DOCKER_Default_CMD_Args:DockerContainerizerTest.ROOT_DOCKER_SlaveRecoveryTaskContainer:DockerContainerizerTest.DISABLED_ROOT_DOCKER_SlaveRecoveryExecutorContainer:DockerContainerizerTest.ROOT_DOCKER_PortMapping:DockerContainerizerTest.ROOT_DOCKER_LaunchSandboxWithColon:DockerTest.ROOT_DOCKER_interface:DockerTest.ROOT_DOCKER_CheckCommandWithShell:DockerTest.ROOT_DOCKER_CheckPortResource:DockerTest.ROOT_DOCKER_CancelPull:SlaveTest.ROOT_RunTaskWithCommandInfoWithoutUser:SlaveTest.DISABLED_ROOT_RunTaskWithCommandInfoWithUser:SlaveCount/Registrar_BENCHMARK_Test.performance/0:SlaveCount/Registrar_BENCHMARK_Test.performance/1:SlaveCount/Registrar_BENCHMARK_Test.performance/2:SlaveCount/Registrar_BENCHMARK_Test.performance/3
[==] Running 4 tests from 1 test case.
[--] Global test environment set-up.
[--] 4 tests from FetcherTest
[ RUN  ] FetcherTest.FileURI
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1104 22:43:38.412901 2068996864 fetcher.cpp:197] Fetching URI 
'file:///private/tmp/FetcherTest_FileURI_ipAYlv/from/test'
I1104 22:43:38.413653 2068996864 fetcher.cpp:178] Copying resource from 
'/private/tmp/FetcherTest_FileURI_ipAYlv/from/test' to 
'/private/tmp/FetcherTest_FileURI_ipAYlv'
I1104 22:43:38.419317 2068996864 fetcher.cpp:300] Skipped extracting path 
'/private/tmp/FetcherTest_FileURI_ipAYlv/test'
[   OK ] FetcherTest.FileURI (110 ms)
[ RUN  ] FetcherTest.FilePath
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1104 22:43:38.507416 2068996864 fetcher.cpp:197] Fetching URI 
'/private/tmp/FetcherTest_FilePath_lAzOZG/from/test'
I1104 22:43:38.508020 2068996864 fetcher.cpp:178] Copying resource from 
'/private/tmp/FetcherTest_FilePath_lAzOZG/from/test' to 
'/private/tmp/FetcherTest_FilePath_lAzOZG'
I1104 22:43:38.512864 2068996864 fetcher.cpp:300] Skipped extracting path 
'/private/tmp/FetcherTest_FilePath_lAzOZG/test'
[   OK ] FetcherTest.FilePath (100 ms)
[ RUN  ] FetcherTest.OSNetUriTest
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1104 22:43:38.608177 2068996864 fetcher.cpp:197] Fetching URI 
'http://www.example.com/index.html'
I1104 22:43:38.608901 2068996864 fetcher.cpp:109] Fetching URI 
'http://www.example.com/index.html' with os::net
I1104 22:43:38.608924 2068996864 fetcher.cpp:119] Downloading 
'http://www.example.com/index.html' to 
'/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
I1104 22:43:38.619645 2068996864 fetcher.cpp:300] Skipped extracting path 
'/private/tmp/FetcherTest_OSNetUriTest_ZB4nAg/index.html'
[   OK ] FetcherTest.OSNetUriTest (105 ms)
[ RUN  ] FetcherTest.FileLocalhostURI
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1104 22:43:38.712461 2068996864 fetcher.cpp:197] Fetching URI 
'file://localhost/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test'
I1104 22:43:38.713353 2068996864 fetcher.cpp:178] Copying resource from 
'/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/from/test' to 
'/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH'
I1104 22:43:38.718541 2068996864 fetcher.cpp:300] Skipped extracting path 
'/private/tmp/FetcherTest_FileLocalhostURI_m1wYnH/test'
[   OK ] FetcherTest.FileLocalhostURI (103 ms)
[--] 4 tests from FetcherTest (419 ms total)
 
[--] Global test environment tear-down
[==] 4 tests from 1 test case ran. (434 ms total)
[  PASSED  ] 4 tests.
 
  YOU HAVE 5 DISABLED TESTS

There are warnings in the output:

WARNING: Logging before InitGoogleLogging() is written to STDERR
And looking at the code, I don't see a glog initialization codeblock. Should 
there be google::InitGoogleLogging(argv[0]); in the main method? I also cannot 
figure out how logging is configured in general and/or where mesos-fetcher is 
invoked. Can someone chime in?


-- Ankur

Re: Review Request 27633: CHANGELOG for 0.21.0

[Ian Downes]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/
---

(Updated Nov. 5, 2014, 10:32 a.m.)


Review request for mesos and Vinod Kone.


Changes
---

Added to docs/upgrades.md


Bugs: MESOS-1964
https://issues.apache.org/jira/browse/MESOS-1964


Repository: mesos-git


Description
---

CHANGELOG for 0.21.0


Diffs (updated)
-

  CHANGELOG 55d8ff4af8bda37a47d31ee62bc424ef496311c4 
  docs/upgrades.md 175eb1308be4548f538ea82e3a238ced2147a64e 

Diff: https://reviews.apache.org/r/27633/diff/


Testing
---


Thanks,

Ian Downes

Re: Review Request 27622: Implement MESOS-1248: Use JSON instead of our own format for passing URI information to mesos-fetcher

[Bernd Mathiske]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27622/
---

(Updated Nov. 5, 2014, 10:30 a.m.)


Review request for mesos and Benjamin Hindman.


Changes
---

Adding missing changes to src/slave/containerizer/fetcher.cpp


Bugs: MESOS-1248
https://issues.apache.org/jira/browse/MESOS-1248


Repository: mesos-git


Description
---

Rebasing and replacing https://reviews.apache.org/r/21277/. Now dependent on 
https://reviews.apache.org/r/27516/ which implements MESOS-1316.

Replaces the ad-hoc format for env var values that act as fetcher program comd 
line args with a JSON format that is gained by translation from protobuf 
parsing to a JSON object. 


Diffs (updated)
-

  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/slave/containerizer/containerizer.cpp 
0254679508167a390fd6fed855f19794354ac081 
  src/slave/containerizer/fetcher.cpp PRE-CREATION 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27622/diff/


Testing
---

Make check.


Thanks,

Bernd Mathiske

Review Request 27633: CHANGELOG for 0.21.0

[Ian Downes]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27633/
---

Review request for mesos and Vinod Kone.


Bugs: MESOS-1964
https://issues.apache.org/jira/browse/MESOS-1964


Repository: mesos-git


Description
---

CHANGELOG for 0.21.0


Diffs
-

  CHANGELOG 55d8ff4af8bda37a47d31ee62bc424ef496311c4 

Diff: https://reviews.apache.org/r/27633/diff/


Testing
---


Thanks,

Ian Downes

Re: Review Request 21277: Passed CommandInfo to mesos-fetcher as JSON.

[Bernd Mathiske]

> On Nov. 5, 2014, 4:29 a.m., Tom Arnfeld wrote:
> > src/launcher/fetcher.cpp, line 183
> > 
> >
> > Do you think it would be OK if we could keep support for the 
> > MESOS_EXECUTOR_URIS environment variable for now, at least for the next 
> > release or two?
> > 
> > Mainly because the mesos-fetcher tool is actually being uesd by other 
> > things outside of the mesos slave itself, e.g external containerizers.
> > 
> > Any thoughts?

We could do this but it would increase the overall engineering cost just to 
postpone some changes that need to happen eventually anyway. JSON seems to be 
the agreed-upon format going forward. In fact, the general idea behind 
switching to JSON was that we could be backward-compatible without such 
problems AFTER MESOS-1248. Anything besides external containerizers that is 
affected?


- Bernd


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21277/#review59949
---


On May 9, 2014, 12:05 p.m., Benjamin Hindman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21277/
> ---
> 
> (Updated May 9, 2014, 12:05 p.m.)
> 
> 
> Review request for mesos, Ben Mahler, Dominic Hamon, and Tom Arnfeld.
> 
> 
> Bugs: MESOS-1248
> https://issues.apache.org/jira/browse/MESOS-1248
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> See summary (and bug).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp 8c9e20da8f39eb5e90403a5093cbea7fb2680468 
>   src/slave/fetcher.hpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/21277/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Hindman
> 
>

Review Request 27622: Implement MESOS-1248: Use JSON instead of our own format for passing URI information to mesos-fetcher

[Bernd Mathiske]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27622/
---

Review request for mesos and Benjamin Hindman.


Bugs: MESOS-1248
https://issues.apache.org/jira/browse/MESOS-1248


Repository: mesos-git


Description
---

Rebasing and replacing https://reviews.apache.org/r/21277/. Now dependent on 
https://reviews.apache.org/r/27516/ which implements MESOS-1316.

Replaces the ad-hoc format for env var values that act as fetcher program comd 
line args with a JSON format that is gained by translation from protobuf 
parsing to a JSON object. 


Diffs
-

  src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
  src/slave/containerizer/containerizer.cpp 
0254679508167a390fd6fed855f19794354ac081 
  src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 

Diff: https://reviews.apache.org/r/27622/diff/


Testing
---

Make check.


Thanks,

Bernd Mathiske

Re: Review Request 27494: Integrated CRAM-MD5 Test Authenticatee module into slave.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27494/
---

(Updated Nov. 5, 2014, 4:30 p.m.)


Review request for mesos, Adam B and Vinod Kone.


Summary (updated)
-

Integrated CRAM-MD5 Test Authenticatee module into slave.


Bugs: MESOS-2001
https://issues.apache.org/jira/browse/MESOS-2001


Repository: mesos-git


Description (updated)
---

Also fixes messages.proto to use a raw bytestream instead of a string for 
AuthenticationStartMessage as non CRAM-MD5 authentication may transmit binary 
data.

Note that this patch covers modularized slave authentication only. Framework 
authentication currently is currently covered by the default (built-in) 
implementation. There will be a subsequent patch for modularized framework 
authentication. 


Diffs
-

  src/messages/messages.proto 76e3980 
  src/sched/sched.cpp 0fb8c7b 
  src/scheduler/scheduler.cpp fb88a3e 
  src/slave/constants.hpp 9030871 
  src/slave/constants.cpp e1da5c0 
  src/slave/flags.hpp f7a8cde 
  src/slave/slave.hpp eb5de73 
  src/slave/slave.cpp 96fb5f7 

Diff: https://reviews.apache.org/r/27494/diff/


Testing
---

make check

NOTE all three CRAM-MD5 authenticatee module related RRs need to get applied 
before running make check.


Thanks,

Till Toenshoff

Re: Review Request 27493: Added CRAM-MD5 Test Authenticatee Module.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27493/
---

(Updated Nov. 5, 2014, 4:24 p.m.)


Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.


Changes
---

made public, hoping to get reviews


Bugs: MESOS-2001
https://issues.apache.org/jira/browse/MESOS-2001


Repository: mesos-git


Description
---

see summary.


Diffs
-

  src/Makefile.am 2d72a70 
  src/authentication/authenticatee.hpp PRE-CREATION 
  src/authentication/cram_md5/authenticatee.hpp PRE-CREATION 
  src/examples/test_authentication_module.cpp PRE-CREATION 
  src/examples/test_authenticator_module.cpp PRE-CREATION 
  src/module/authenticatee.hpp PRE-CREATION 
  src/module/manager.cpp 7a6c884 

Diff: https://reviews.apache.org/r/27493/diff/


Testing
---

make check

Note: All three CRAM-MD5 Authenticatee patches need to get applied before 
running make check! 


Thanks,

Till Toenshoff

Re: Review Request 27619: Updated CRAMMD5Authentication tests to additionally run against the test authenticator module.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27619/#review59965
---


Patch looks great!

Reviews applied: [27619]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 3:27 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27619/
> ---
> 
> (Updated Nov. 5, 2014, 3:27 p.m.)
> 
> 
> Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.
> 
> 
> Bugs: MESOS-1894
> https://issues.apache.org/jira/browse/MESOS-1894
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> see summary.
> 
> 
> Diffs
> -
> 
>   src/authentication/cram_md5/authenticator.hpp 601248d 
>   src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
>   src/tests/module.hpp 7970c28 
>   src/tests/module.cpp 482ed22 
> 
> Diff: https://reviews.apache.org/r/27619/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 22169: Added External Containerizer documentation.

[Till Toenshoff]

> On Nov. 5, 2014, 12:35 p.m., Tom Arnfeld wrote:
> > LGTM! Yeah, relative URLs would be a good idea.

Aye, fixed that.


- Till


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22169/#review59951
---


On Nov. 5, 2014, 4 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22169/
> ---
> 
> (Updated Nov. 5, 2014, 4 p.m.)
> 
> 
> Review request for mesos and Tom Arnfeld.
> 
> 
> Bugs: MESOS-2045
> https://issues.apache.org/jira/browse/MESOS-2045
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Adds a markdown document describing the ExternalContainerizer.
> 
> 
> Diffs
> -
> 
>   docs/external-containerizer.md PRE-CREATION 
>   docs/home.md 7afc4a3 
>   docs/images/ec_kill_seqdiag.png PRE-CREATION 
>   docs/images/ec_launch_seqdiag.png PRE-CREATION 
>   docs/images/ec_lifecycle_seqdiag.png PRE-CREATION 
>   docs/images/ec_orphan_seqdiag.png PRE-CREATION 
>   docs/images/ec_recover_seqdiag.png PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22169/diff/
> 
> 
> Testing
> ---
> 
> see https://github.com/tillt/mesos/blob/master/docs/external-containerizer.md
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 22169: Added External Containerizer documentation.

[Till Toenshoff]

> On Sept. 30, 2014, 11:51 p.m., Niklas Nielsen wrote:
> > Fly by review - it is much better than the current (non-existing) docs on 
> > the EC. So I think we should land this asap :-) Tom, do you have cycles to 
> > review?

Will try to ping Tom directly on that.


> On Sept. 30, 2014, 11:51 p.m., Niklas Nielsen wrote:
> > docs/external-containerizer.md, line 139
> > 
> >
> > Worth making the paths relative?

Fixed. Will commit this very soon.


- Till


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22169/#review55024
---


On Nov. 5, 2014, 4 p.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22169/
> ---
> 
> (Updated Nov. 5, 2014, 4 p.m.)
> 
> 
> Review request for mesos and Tom Arnfeld.
> 
> 
> Bugs: MESOS-2045
> https://issues.apache.org/jira/browse/MESOS-2045
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Adds a markdown document describing the ExternalContainerizer.
> 
> 
> Diffs
> -
> 
>   docs/external-containerizer.md PRE-CREATION 
>   docs/home.md 7afc4a3 
>   docs/images/ec_kill_seqdiag.png PRE-CREATION 
>   docs/images/ec_launch_seqdiag.png PRE-CREATION 
>   docs/images/ec_lifecycle_seqdiag.png PRE-CREATION 
>   docs/images/ec_orphan_seqdiag.png PRE-CREATION 
>   docs/images/ec_recover_seqdiag.png PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22169/diff/
> 
> 
> Testing
> ---
> 
> see https://github.com/tillt/mesos/blob/master/docs/external-containerizer.md
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 22169: Added External Containerizer documentation.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22169/
---

(Updated Nov. 5, 2014, 4 p.m.)


Review request for mesos and Tom Arnfeld.


Bugs: MESOS-2045
https://issues.apache.org/jira/browse/MESOS-2045


Repository: mesos-git


Description
---

Adds a markdown document describing the ExternalContainerizer.


Diffs
-

  docs/external-containerizer.md PRE-CREATION 
  docs/home.md 7afc4a3 
  docs/images/ec_kill_seqdiag.png PRE-CREATION 
  docs/images/ec_launch_seqdiag.png PRE-CREATION 
  docs/images/ec_lifecycle_seqdiag.png PRE-CREATION 
  docs/images/ec_orphan_seqdiag.png PRE-CREATION 
  docs/images/ec_recover_seqdiag.png PRE-CREATION 

Diff: https://reviews.apache.org/r/22169/diff/


Testing (updated)
---

see https://github.com/tillt/mesos/blob/master/docs/external-containerizer.md


Thanks,

Till Toenshoff

Re: Review Request 22169: Added External Containerizer documentation.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22169/
---

(Updated Nov. 5, 2014, 3:58 p.m.)


Review request for mesos and Tom Arnfeld.


Changes
---

Updated towards relative URLs and included a missing image.


Bugs: MESOS-2045
https://issues.apache.org/jira/browse/MESOS-2045


Repository: mesos-git


Description
---

Adds a markdown document describing the ExternalContainerizer.


Diffs (updated)
-

  docs/external-containerizer.md PRE-CREATION 
  docs/home.md 7afc4a3 
  docs/images/ec_kill_seqdiag.png PRE-CREATION 
  docs/images/ec_launch_seqdiag.png PRE-CREATION 
  docs/images/ec_lifecycle_seqdiag.png PRE-CREATION 
  docs/images/ec_orphan_seqdiag.png PRE-CREATION 
  docs/images/ec_recover_seqdiag.png PRE-CREATION 

Diff: https://reviews.apache.org/r/22169/diff/


Testing
---


Thanks,

Till Toenshoff

Review Request 27619: Updated CRAMMD5Authentication tests to additionally run against the test authenticator module.

[Till Toenshoff]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27619/
---

Review request for mesos, Adam B, Kapil Arya, and Vinod Kone.


Bugs: MESOS-1894
https://issues.apache.org/jira/browse/MESOS-1894


Repository: mesos-git


Description
---

see summary.


Diffs
-

  src/authentication/cram_md5/authenticator.hpp 601248d 
  src/tests/cram_md5_authentication_tests.cpp 74ea2ad 
  src/tests/module.hpp 7970c28 
  src/tests/module.cpp 482ed22 

Diff: https://reviews.apache.org/r/27619/diff/


Testing
---

make check


Thanks,

Till Toenshoff

Re: Review Request 22169: Added External Containerizer documentation.

[Tom Arnfeld]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22169/#review59951
---

Ship it!


LGTM! Yeah, relative URLs would be a good idea.

- Tom Arnfeld


On Sept. 5, 2014, 8:48 a.m., Till Toenshoff wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22169/
> ---
> 
> (Updated Sept. 5, 2014, 8:48 a.m.)
> 
> 
> Review request for mesos and Tom Arnfeld.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Adds a markdown document describing the ExternalContainerizer.
> 
> 
> Diffs
> -
> 
>   docs/external-containerizer.md PRE-CREATION 
>   docs/home.md 179a164 
>   docs/images/ec_kill_seqdiag.png PRE-CREATION 
>   docs/images/ec_launch_seqdiag.png PRE-CREATION 
>   docs/images/ec_lifecycle_seqdiag.png PRE-CREATION 
>   docs/images/ec_orphan_seqdiag.png PRE-CREATION 
>   docs/images/ec_recover_seqdiag.png PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/22169/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Till Toenshoff
> 
>

Re: Review Request 27516: Rebased and re-edited patch for MESOS-1316: "Abstracted out invoking 'mesos-fetcher'".

[Tom Arnfeld]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27516/#review59950
---

Ship it!


- Tom Arnfeld


On Nov. 3, 2014, 4:36 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27516/
> ---
> 
> (Updated Nov. 3, 2014, 4:36 p.m.)
> 
> 
> Review request for mesos and Benjamin Hindman.
> 
> 
> Bugs: MESOS-1316
> https://issues.apache.org/jira/browse/MESOS-1316
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Manually rebasing and re-editing https://reviews.apache.org/r/21233/, which 
> is supposed to be replaced now by this patch. 
> 
> Original description: "To test the mesos-fetcher (and the setting of the 
> environment) more cleanly I did some refactoring into a 'fetcher' namespace."
> 
> Also moved fetcher environment tests to fetcher test file. Added two fetcher 
> tests.
> 
> 
> Diffs
> -
> 
>   src/Makefile.am e6a07150c10b9fa040143e394b2f913a18eeebc1 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/slave/containerizer/fetcher.hpp PRE-CREATION 
>   src/slave/containerizer/fetcher.cpp PRE-CREATION 
>   src/slave/containerizer/mesos/containerizer.cpp 
> d4b08f54d6feb453f3a9d27ca54c867176e62102 
>   src/tests/containerizer_tests.cpp 2c90d2fc18a3268c55b6dfe98699bfb36d093983 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27516/diff/
> 
> 
> Testing
> ---
> 
> make check on Mac OS 10.10 and Ubuntu 14.4.
> 
> In total, 3 tests fail: ExamplesTest.NoExecutorFramework, 
> ExamplesTest.JavaFramework
> , ExamplesTest.PythonFramework. It is strongly suspected that those are 
> unrelated to this code change and just generally flaky.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Review Request 21277: Passed CommandInfo to mesos-fetcher as JSON.

[Tom Arnfeld]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21277/#review59949
---



src/launcher/fetcher.cpp


Do you think it would be OK if we could keep support for the 
MESOS_EXECUTOR_URIS environment variable for now, at least for the next release 
or two?

Mainly because the mesos-fetcher tool is actually being uesd by other 
things outside of the mesos slave itself, e.g external containerizers.

Any thoughts?


- Tom Arnfeld


On May 9, 2014, 7:05 p.m., Benjamin Hindman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21277/
> ---
> 
> (Updated May 9, 2014, 7:05 p.m.)
> 
> 
> Review request for mesos, Ben Mahler, Dominic Hamon, and Tom Arnfeld.
> 
> 
> Bugs: MESOS-1248
> https://issues.apache.org/jira/browse/MESOS-1248
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> See summary (and bug).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp 8c9e20da8f39eb5e90403a5093cbea7fb2680468 
>   src/slave/fetcher.hpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/21277/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Hindman
> 
>

Re: Review Request 21277: Passed CommandInfo to mesos-fetcher as JSON.

[Bernd Mathiske]

> On Oct. 14, 2014, 10:10 a.m., Niklas Nielsen wrote:
> > LGTM but needs rebase. Would be sweet to get in: much more robust than 
> > magic encoding :-)
> 
> Bernd Mathiske wrote:
> I will work on it.

I have a patch. It depends on https://reviews.apache.org/r/27516/, which has 
not been reviewed yet. Trying to figure out how I can post a dependent patch...


- Bernd


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21277/#review56549
---


On May 9, 2014, 12:05 p.m., Benjamin Hindman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21277/
> ---
> 
> (Updated May 9, 2014, 12:05 p.m.)
> 
> 
> Review request for mesos, Ben Mahler, Dominic Hamon, and Tom Arnfeld.
> 
> 
> Bugs: MESOS-1248
> https://issues.apache.org/jira/browse/MESOS-1248
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> See summary (and bug).
> 
> 
> Diffs
> -
> 
>   src/launcher/fetcher.cpp 8c9e20da8f39eb5e90403a5093cbea7fb2680468 
>   src/slave/fetcher.hpp PRE-CREATION 
>   src/tests/fetcher_tests.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/21277/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Hindman
> 
>

Re: Review Request 27567: MESOS-2038: removed dead code from _runTask()

[Bernd Mathiske]

> On Nov. 4, 2014, 3:09 a.m., Adam B wrote:
> > Ship It!
> 
> Adam B wrote:
> Following the logic with my internal C++ compiler/interpreter (aka my 
> brain), it's pretty clear that we would have returned earlier in this 
> function if framework were in fact NULL, so the null-check branch is indeed 
> unreachable code.
> 
> Ben Mahler wrote:
> Definitely unreachable, I just took a glance and couldn't easily figure 
> out when this case occurs. I also couldn't quickly figure out whether the 
> creation of the framework was something we want to do in this case vs. log a 
> warning and bail. If we bail when the framework is NULL and this is something 
> we expect to occur, do we need to send task lost?
> 
> With these questions, I realized I should just let vinod review this 
> given he had more context around this.

Vinod and I discussed at some length in https://reviews.apache.org/r/23912 
which style of dealing with the framework life cycle to pursue. The code in 
question here is an artefact of the style I initially opted for: relatively 
late destruction of the framework. You would then have to check if it still 
exists as done by the now dead code. With Vinod's now implemented approach, the 
framework gets discarded at the earliest opportunity. This enabled the test at 
the top that made the code in question dead.

This larger problem is that the original code hinted at BOTH styles. I 
conjecture that this is not an isolated incident of such latent confusion.


- Bernd


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27567/#review59760
---


On Nov. 4, 2014, 3:46 p.m., Bernd Mathiske wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27567/
> ---
> 
> (Updated Nov. 4, 2014, 3:46 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-2038
> https://issues.apache.org/jira/browse/MESOS-2038
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Removed a few lines of dead code that coverty discovered.
> 
> 
> Diffs
> -
> 
>   src/slave/slave.cpp 5e9b0e4f93a5100a91340e1f6fb1fe160b2eea4b 
> 
> Diff: https://reviews.apache.org/r/27567/diff/
> 
> 
> Testing
> ---
> 
> none.
> expecting/waiting for review bot to report no problem.
> 
> 
> Thanks,
> 
> Bernd Mathiske
> 
>

Re: Review Request 27483: Fetcher uses hadoop to fetch URIs regardless of the url scheme.

[Ankur Chauhan]

> On Nov. 5, 2014, 7:29 a.m., Timothy Chen wrote:
> > src/tests/fetcher_tests.cpp, line 121
> > 
> >
> > How does this test work with a bogus url?

I used it because it was mentioned in 
[RFC-2606](http://tools.ietf.org/html/rfc2606) and seemed like a reliable 
resource. But then maybe reaching out into the internet is not the best thing 
to do either but it correctly exercises the code path.

```
$ curl http://www.example.com/index.html



Example Domain





body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, 
sans-serif;

}
div {
width: 600px;
margin: 5em auto;
padding: 50px;
background-color: #fff;
border-radius: 1em;
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
body {
background-color: #fff;
}
div {
width: auto;
margin: 0 auto;
border-radius: 0;
padding: 1em;
}
}





Example Domain
This domain is established to be used for illustrative examples in 
documents. You may use this
domain in examples without prior coordination or asking for permission.
http://www.iana.org/domains/example";>More information...



```


- Ankur


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27483/#review59938
---


On Nov. 5, 2014, 6:39 a.m., Ankur Chauhan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27483/
> ---
> 
> (Updated Nov. 5, 2014, 6:39 a.m.)
> 
> 
> Review request for mesos, Timothy Chen, Timothy St. Clair, and Vinod Kone.
> 
> 
> Bugs: MESOS-1711
> https://issues.apache.org/jira/browse/MESOS-1711
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Previously, the fetcher used a hardcoded list of schemes to determine what 
> URIs could be fetched by hadoop (if available). This is now changed such that 
> we first check if hadoop can fetch them for us and then we fallback to the 
> os::net and then a local copy method (same as it used to be). This allows 
> users to fetch artifacts from arbitrary filesystems as long as hadoop is 
> correctly configured (in core-site.xml).
> 
> 
> Diffs
> -
> 
>   src/hdfs/hdfs.hpp bbfeddef106c598d8379ced085ef0605c4b2f380 
>   src/launcher/fetcher.cpp 9323c28237010fa065ef34d74435c151ded530a8 
>   src/tests/fetcher_tests.cpp d7754009a59fedb43e3422c56b3a786ce80164aa 
> 
> Diff: https://reviews.apache.org/r/27483/diff/
> 
> 
> Testing
> ---
> 
> make check
> sudo bin/mesos-tests.sh --verbose
> support/mesos-style.py
> 
> 
> Thanks,
> 
> Ankur Chauhan
> 
>

Re: Review Request 24776: Add destroy tests for docker containerizer.

[Mesos ReviewBot]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24776/#review59939
---


Patch looks great!

Reviews applied: [24776]

All tests passed.

- Mesos ReviewBot


On Nov. 5, 2014, 7:26 a.m., Timothy Chen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24776/
> ---
> 
> (Updated Nov. 5, 2014, 7:26 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Jie Yu.
> 
> 
> Repository: mesos-git
> 
> 
> Description
> ---
> 
> Review: https://reviews.apache.org/r/24776
> 
> 
> Diffs
> -
> 
>   src/slave/containerizer/docker.hpp ec6b9cd308e9a16e05f016e8aeadbe77646d1621 
>   src/slave/containerizer/docker.cpp a6689203adbdcb0ad12583389eaeb83329e4ef6b 
>   src/tests/docker_containerizer_tests.cpp 
> 9d4ccc57f58d61c62aab5cdc79a129e987920bf6 
> 
> Diff: https://reviews.apache.org/r/24776/diff/
> 
> 
> Testing
> ---
> 
> make check
> 
> 
> Thanks,
> 
> Timothy Chen
> 
>