Re: Beware of ASSERT_* Placement

2015-07-27 Thread Paul Brett
Michael

I think Ben's suggestion of using Try<> is just what we want for common
functions.

In regards to ASSERTs, they can cause tests to be skipped within
instantiations of the fixtures or test case as expected.

For example, If you look at tests such as
SlaveRecoveryTest::ReconnectExecutor, it has 9 ASSERTs in a single test
case.  The first 5 are in setup code and seem pretty reasonable but the
last 4 are:

 489   // Executor should inform about the unacknowledged update.
 490   ASSERT_EQ(1, reregister.updates_size());
 491   const StatusUpdate& update = reregister.updates(0);
 492   ASSERT_EQ(task.task_id(), update.status().task_id());
 493   ASSERT_EQ(TASK_RUNNING, update.status().state());
 494
 495   // Scheduler should receive the recovered update.
 496   AWAIT_READY(status);
 497   ASSERT_EQ(TASK_RUNNING, status.get().state());

So looking at this code, I suspect that lines 492 and 493 might be better
as EXPECT?  What about 497?  What follows afterwards is only cleanup code,
so either it is not necessary and we can omit it or 497 should be an
expect.

Looking through the tests directory, this appears to be a common pattern.
Of course, it is all harmless while the code is passing the tests but when
a change breaks things, the scope of the breakage can be obscured because
of the skipped test conditions.

Given the restrictions you point out on the use of ASSERT combined with the
ability to hide failing tests, I believe we should have a strong preference
for EXPECT over ASSERT unless it is clear that every subsequent test in the
test cast is dependent on the result of this test.

Just my 5c worth

@paul_b

On Mon, Jul 27, 2015 at 7:34 PM, Michael Park  wrote:

> Paul,
>
> With ASSERT, I completely agree with you about the perils of using ASSERT
> > that you list above, but additionally we have cases where ASSERT exits a
> > test fixture skipping later tests that might or might not have failed.
>
>
> We should only be using *ASSERT_** in cases where it doesn't make sense to
> proceed with the rest of the test if the condition fails, so exiting the
> test case seems like it's exactly what we would want. If you're saying that
> we currently use it incorrectly, then yeah, we should perhaps write a guide
> to help with how to use it correctly. But it sounds like you're saying we
> shouldn't use it at all?
>
> Since the CHECK macro aborts the test harness, a single test failure
> > prevents tests from running on all the remaining tests.  Particularly
> > annoying for anyone running automated regression tests.
>
>
> Perhaps my suggestion of resorting to *CHECK_** was not a good one, but I
> still don't think *EXPECT_** is what we want. If we have a condition in
> which it doesn't make sense to proceed with the rest of the test, we should
> stop. Perhaps the helper function should return a *Try* as Ben suggested,
> proceeded by an *ASSERT_** of the result within the test case or something
> like that.
>
> I mainly wanted to inform folks of this limitation and the corresponding
> confusing error message that follows.
>
> On 27 July 2015 at 18:42, Benjamin Mahler 
> wrote:
>
> > Michael, note that we've avoided having ASSERT_ or EXPECT_ inside test
> > helper methods because they print out the line number of the helper
> method,
> > rather than the line number where the helper method was called from the
> > test. This is why we've been pretty careful when adding helpers and have
> > tried to push assertions into the test itself (e.g. helper returns a Try
> > instead of internally asserting).
> >
> > Paul, are you saying that ASSERT within one case in a fixture will stop
> > running all other cases for the fixture? Do you have a pointer to this?
> > Sounds surprising.
> >
> > On Mon, Jul 27, 2015 at 3:04 PM, Paul Brett 
> > wrote:
> >
> > > Mike
> > >
> > > I would suggest that we want to avoid both ASSERT and CHECK macros in
> > > tests.
> > >
> > > With ASSERT, I completely agree with you about the perils of using
> ASSERT
> > > that you list above, but additionally we have cases where ASSERT exits
> a
> > > test fixture skipping later tests that might or might not have failed.
> > >
> > > Since the CHECK macro aborts the test harness, a single test failure
> > > prevents tests from running on all the remaining tests.  Particularly
> > > annoying for anyone running automated regression tests.
> > >
> > > We should add this to either the style guide or mesos-testing-patterns.
> > >
> > > -- @paul_b
> > >
> > > On Mon, Jul 27, 2015 at 2:28 PM, Michael Park 
> wrote:
> > >
> > > > I've had at least 3 individuals who ran into the issue of *ASSERT_**
> > > macro
> > > > placement and since the generated error message is less than
> useless, I
> > > > would like to share with you what the issue is.
> > > >
> > > > The source of the issue is that *ASSERT_** macros from *gtest* can
> only
> > > be
> > > > placed in functions that return *void* as described in Assertion
> > > Placement
> > > > <
> > > >
> > >
> >
> h

Re: Beware of ASSERT_* Placement

2015-07-27 Thread Michael Park
Paul,

With ASSERT, I completely agree with you about the perils of using ASSERT
> that you list above, but additionally we have cases where ASSERT exits a
> test fixture skipping later tests that might or might not have failed.


We should only be using *ASSERT_** in cases where it doesn't make sense to
proceed with the rest of the test if the condition fails, so exiting the
test case seems like it's exactly what we would want. If you're saying that
we currently use it incorrectly, then yeah, we should perhaps write a guide
to help with how to use it correctly. But it sounds like you're saying we
shouldn't use it at all?

Since the CHECK macro aborts the test harness, a single test failure
> prevents tests from running on all the remaining tests.  Particularly
> annoying for anyone running automated regression tests.


Perhaps my suggestion of resorting to *CHECK_** was not a good one, but I
still don't think *EXPECT_** is what we want. If we have a condition in
which it doesn't make sense to proceed with the rest of the test, we should
stop. Perhaps the helper function should return a *Try* as Ben suggested,
proceeded by an *ASSERT_** of the result within the test case or something
like that.

I mainly wanted to inform folks of this limitation and the corresponding
confusing error message that follows.

On 27 July 2015 at 18:42, Benjamin Mahler  wrote:

> Michael, note that we've avoided having ASSERT_ or EXPECT_ inside test
> helper methods because they print out the line number of the helper method,
> rather than the line number where the helper method was called from the
> test. This is why we've been pretty careful when adding helpers and have
> tried to push assertions into the test itself (e.g. helper returns a Try
> instead of internally asserting).
>
> Paul, are you saying that ASSERT within one case in a fixture will stop
> running all other cases for the fixture? Do you have a pointer to this?
> Sounds surprising.
>
> On Mon, Jul 27, 2015 at 3:04 PM, Paul Brett 
> wrote:
>
> > Mike
> >
> > I would suggest that we want to avoid both ASSERT and CHECK macros in
> > tests.
> >
> > With ASSERT, I completely agree with you about the perils of using ASSERT
> > that you list above, but additionally we have cases where ASSERT exits a
> > test fixture skipping later tests that might or might not have failed.
> >
> > Since the CHECK macro aborts the test harness, a single test failure
> > prevents tests from running on all the remaining tests.  Particularly
> > annoying for anyone running automated regression tests.
> >
> > We should add this to either the style guide or mesos-testing-patterns.
> >
> > -- @paul_b
> >
> > On Mon, Jul 27, 2015 at 2:28 PM, Michael Park  wrote:
> >
> > > I've had at least 3 individuals who ran into the issue of *ASSERT_**
> > macro
> > > placement and since the generated error message is less than useless, I
> > > would like to share with you what the issue is.
> > >
> > > The source of the issue is that *ASSERT_** macros from *gtest* can only
> > be
> > > placed in functions that return *void* as described in Assertion
> > Placement
> > > <
> > >
> >
> https://code.google.com/p/googletest/wiki/AdvancedGuide#Assertion_Placement
> > > >
> > > .
> > >
> > > By placing it in a non-void function, you get useless error messages
> like
> > > this:
> > >
> > > From *GCC*: "*error: void value not ignored as it ought to be*"
> > > From *Clang*: "*error: could not convert
> > > ‘testing::internal::AssertHelper((testing::TestPartResult::Type)2u,
> > ((const
> > > char*)"../../src/tests/containerizer/port_mapping_tests.cpp"), 320,
> > >
> > >
> >
> gtest_ar.testing::AssertionResult::failure_message()).testing::internal::AssertHelper::operator=(testing::Message())’
> > > from ‘void’ to ‘int’*"
> > >
> > > I think the following are typical situations that result in this mess:
> > >
> > >- Using them in *constructors/destructors*. For example, it would be
> > >really confusing if you're simply translating a *SetUp*/*TearDown*
> of
> > a
> > >test fixture to be *constructor/destructor* instead.
> > >- Using them in *main*, since it returns an *int*.
> > >- Refactoring a chunk of logic from a test case into a helper
> function
> > >that doesn't return *void*. For example, if we were factor out the
> > >following code inside of a test case:
> > >
> > >
> > >
> > >
> > > *AWAIT_READY(offers); ASSERT_EQ(1u, offers.get().size());
>  offer =
> > >offers.get()[0]; *
> > >into a function like this:
> > >
> > >
> > >
> > >
> > >
> > > *Offer getOffer(Future> &offers) {
> > >AWAIT_READY(offers);   ASSERT_EQ(1u, offers.get().size());
> > >  return
> > >offers.get()[0]; }*
> > >
> > >this innocent-looking transformation would trigger the obscure error
> > >message and you'll be upset once you figure out why.
> > >
> > > If you encounter this case, prefer to resort to *CHECK_** from *glog*,
> > > rather
> > > than *EXPECT_**, since *CHEC

RE: Where's Meses design documents

2015-07-27 Thread Klaus Ma
Wiki seems out of date :). And a central place for links of all documents in 
google doc is fine; it'll also save owner effort on translating from google doc 
to wiki.

Regards,

Klaus Ma (马达), PMP® | http://www.cguru.net

-Original Message-
From: Benjamin Mahler [mailto:benjamin.mah...@gmail.com] 
Sent: 2015年7月28日 5:28
To: dev
Subject: Re: Where's Meses design documents

Vinod, it would be nice to have them within the repo, can't we have an area in 
docs/ for design documents? Even if that just links to active google docs, 
seems nicer than using cwiki?

On Mon, Jul 27, 2015 at 2:05 PM, Vinod Kone  wrote:

> Ideally, all design docs that are finalized should go to Mesos's cwiki 
> space .  We 
> have some there already.
>
> @all: if you've finalized (i.e., corresponding code has been 
> committed) design docs please move them to the cwiki space. Let me 
> know if you need perms.
>
> On Mon, Jul 27, 2015 at 1:45 PM, Michael Park  wrote:
>
> > Hi Klaus,
> >
> > There's no central place where all Mesos design documents live currently.
> > They're mostly google docs which are linked from the JIRA epics. 
> > Perhaps it's a good idea to have a central place to store the design 
> > documents however, so that they can be discovered easier.
> >
> > MPark.
> >
> > On Sat, Jul 25, 2015 at 11:43 PM Klaus Ma  wrote:
> >
> > > Hi Yong/Alex,
> > >
> > > Thanks very much for those info :). I have went through those 
> > > documents
> > to
> > > learn the overall design of Mesos.
> > > I think I have to check JIRA for design document of special feature.
> > >
> > > Regards,
> > > 
> > > Klaus Ma (马达), PMP® | http://www.cguru.net
> > >
> > > > On Jul 26, 2015, at 10:07, Yong Feng  wrote:
> > > >
> > > > Hi Klaus
> > > >
> > > > I assume you would like to learn how Mesos is designed and
> implemented
> > > > feature by feature through design doc of Epic. If that is that 
> > > > case,
> > you
> > > > could start with 
> > > > https://github.com/apache/mesos/tree/master/docs
> > which
> > > > describes Mesos overall architecture, func spec of main 
> > > > features,
> > release
> > > > history, roadmap plan and so on. It actually also cover what 
> > > > Alex
> > cited.
> > > >
> > > > After that, if you have interest on specific topic/feature, you 
> > > > could search JIRA for related ticket to check the design idea 
> > > > and related discussion.
> > > >
> > > > Hope it will help.
> > > >
> > > > Thanks,
> > > >
> > > > Yong
> > > >
> > > > On Sat, Jul 25, 2015 at 9:09 PM, Klaus Ma  wrote:
> > > >
> > > >> I'm asking for the design document of each epic, is there a 
> > > >> central
> > > place
> > > >> to download them?Just go through the design document of 
> > > >> maintenance,
> > > it's
> > > >> helpful; so I'd like to also learn other design documents.
> > > >>
> > > >> Regards,Klaus Ma (马达), PMP® | http://www.cguru.net CallSend 
> > > >> SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree
> > via
> > > >> SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> > > CreditFree
> > > >> via Skype
> > > >>
> > > >>> Date: Sat, 25 Jul 2015 18:01:06 -0700
> > > >>> Subject: Re: Where's Meses design documents
> > > >>> From: clemmer.alexan...@gmail.com
> > > >>> To: dev@mesos.apache.org
> > > >>>
> > > >>> Are you asking about the standard documentation (which is 
> > > >>> here[1]),
> > or
> > > >>> are you asking about the design documents for every epic that 
> > > >>> has
> > been
> > > >>> added historically?
> > > >>>
> > > >>> [1] http://mesos.apache.org/documentation/latest/
> > > >>>
> > > >>> On Sat, Jul 25, 2015 at 5:53 PM, Klaus Ma  wrote:
> > >  Hi team,
> > >  I can get design documents from each EPIC/Story, but where 
> > >  can I
> get
> > > >> ALL design document of Mesos?
> > > 
> > >  Regards,Klaus Ma (马达), PMP® | http://www.cguru.net 
> > >  CallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> CreditFree
> > > >> via SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need 
> > > >> Skype CreditFree via Skype
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Alex
> > > >>>
> > > >>> Theory is the first term in the Taylor series of practice. --
> Thomas
> > M
> > > >>> Cover (1992)
> > > >>
> > > >>
> > >
> > >
> >
>



Re: Beware of ASSERT_* Placement

2015-07-27 Thread Benjamin Mahler
Michael, note that we've avoided having ASSERT_ or EXPECT_ inside test
helper methods because they print out the line number of the helper method,
rather than the line number where the helper method was called from the
test. This is why we've been pretty careful when adding helpers and have
tried to push assertions into the test itself (e.g. helper returns a Try
instead of internally asserting).

Paul, are you saying that ASSERT within one case in a fixture will stop
running all other cases for the fixture? Do you have a pointer to this?
Sounds surprising.

On Mon, Jul 27, 2015 at 3:04 PM, Paul Brett 
wrote:

> Mike
>
> I would suggest that we want to avoid both ASSERT and CHECK macros in
> tests.
>
> With ASSERT, I completely agree with you about the perils of using ASSERT
> that you list above, but additionally we have cases where ASSERT exits a
> test fixture skipping later tests that might or might not have failed.
>
> Since the CHECK macro aborts the test harness, a single test failure
> prevents tests from running on all the remaining tests.  Particularly
> annoying for anyone running automated regression tests.
>
> We should add this to either the style guide or mesos-testing-patterns.
>
> -- @paul_b
>
> On Mon, Jul 27, 2015 at 2:28 PM, Michael Park  wrote:
>
> > I've had at least 3 individuals who ran into the issue of *ASSERT_**
> macro
> > placement and since the generated error message is less than useless, I
> > would like to share with you what the issue is.
> >
> > The source of the issue is that *ASSERT_** macros from *gtest* can only
> be
> > placed in functions that return *void* as described in Assertion
> Placement
> > <
> >
> https://code.google.com/p/googletest/wiki/AdvancedGuide#Assertion_Placement
> > >
> > .
> >
> > By placing it in a non-void function, you get useless error messages like
> > this:
> >
> > From *GCC*: "*error: void value not ignored as it ought to be*"
> > From *Clang*: "*error: could not convert
> > ‘testing::internal::AssertHelper((testing::TestPartResult::Type)2u,
> ((const
> > char*)"../../src/tests/containerizer/port_mapping_tests.cpp"), 320,
> >
> >
> gtest_ar.testing::AssertionResult::failure_message()).testing::internal::AssertHelper::operator=(testing::Message())’
> > from ‘void’ to ‘int’*"
> >
> > I think the following are typical situations that result in this mess:
> >
> >- Using them in *constructors/destructors*. For example, it would be
> >really confusing if you're simply translating a *SetUp*/*TearDown* of
> a
> >test fixture to be *constructor/destructor* instead.
> >- Using them in *main*, since it returns an *int*.
> >- Refactoring a chunk of logic from a test case into a helper function
> >that doesn't return *void*. For example, if we were factor out the
> >following code inside of a test case:
> >
> >
> >
> >
> > *AWAIT_READY(offers); ASSERT_EQ(1u, offers.get().size()); offer =
> >offers.get()[0]; *
> >into a function like this:
> >
> >
> >
> >
> >
> > *Offer getOffer(Future> &offers) {
> >AWAIT_READY(offers);   ASSERT_EQ(1u, offers.get().size());
> >  return
> >offers.get()[0]; }*
> >
> >this innocent-looking transformation would trigger the obscure error
> >message and you'll be upset once you figure out why.
> >
> > If you encounter this case, prefer to resort to *CHECK_** from *glog*,
> > rather
> > than *EXPECT_**, since *CHECK_** has closer semantics.
> >
> > I hope this will help folks save time and also reduce the amount of head
> > banging!
> >
> > Thanks,
> >
> > MPark.
> >
>
>
>
> --
> -- Paul Brett
>


Re: Beware of ASSERT_* Placement

2015-07-27 Thread Paul Brett
Mike

I would suggest that we want to avoid both ASSERT and CHECK macros in tests.

With ASSERT, I completely agree with you about the perils of using ASSERT
that you list above, but additionally we have cases where ASSERT exits a
test fixture skipping later tests that might or might not have failed.

Since the CHECK macro aborts the test harness, a single test failure
prevents tests from running on all the remaining tests.  Particularly
annoying for anyone running automated regression tests.

We should add this to either the style guide or mesos-testing-patterns.

-- @paul_b

On Mon, Jul 27, 2015 at 2:28 PM, Michael Park  wrote:

> I've had at least 3 individuals who ran into the issue of *ASSERT_** macro
> placement and since the generated error message is less than useless, I
> would like to share with you what the issue is.
>
> The source of the issue is that *ASSERT_** macros from *gtest* can only be
> placed in functions that return *void* as described in Assertion Placement
> <
> https://code.google.com/p/googletest/wiki/AdvancedGuide#Assertion_Placement
> >
> .
>
> By placing it in a non-void function, you get useless error messages like
> this:
>
> From *GCC*: "*error: void value not ignored as it ought to be*"
> From *Clang*: "*error: could not convert
> ‘testing::internal::AssertHelper((testing::TestPartResult::Type)2u, ((const
> char*)"../../src/tests/containerizer/port_mapping_tests.cpp"), 320,
>
> gtest_ar.testing::AssertionResult::failure_message()).testing::internal::AssertHelper::operator=(testing::Message())’
> from ‘void’ to ‘int’*"
>
> I think the following are typical situations that result in this mess:
>
>- Using them in *constructors/destructors*. For example, it would be
>really confusing if you're simply translating a *SetUp*/*TearDown* of a
>test fixture to be *constructor/destructor* instead.
>- Using them in *main*, since it returns an *int*.
>- Refactoring a chunk of logic from a test case into a helper function
>that doesn't return *void*. For example, if we were factor out the
>following code inside of a test case:
>
>
>
>
> *AWAIT_READY(offers); ASSERT_EQ(1u, offers.get().size()); offer =
>offers.get()[0]; *
>into a function like this:
>
>
>
>
>
> *Offer getOffer(Future> &offers) {
>AWAIT_READY(offers);   ASSERT_EQ(1u, offers.get().size());
>  return
>offers.get()[0]; }*
>
>this innocent-looking transformation would trigger the obscure error
>message and you'll be upset once you figure out why.
>
> If you encounter this case, prefer to resort to *CHECK_** from *glog*,
> rather
> than *EXPECT_**, since *CHECK_** has closer semantics.
>
> I hope this will help folks save time and also reduce the amount of head
> banging!
>
> Thanks,
>
> MPark.
>



-- 
-- Paul Brett


Re: RFC: batch-only allocations

2015-07-27 Thread James Peach

> On Jul 27, 2015, at 2:26 PM, Vinod Kone  wrote:
> 
> Never mind. Just saw MESOS-3157

OK :)

> 
> On Mon, Jul 27, 2015 at 2:22 PM, Vinod Kone  wrote:
> 
>> Can I ask why you want to do this change in the first place?
>> 
>> On Mon, Jul 27, 2015 at 2:11 PM, James Peach  wrote:
>> 
>>> Hi all,
>>> 
>>> See MESOS-3157 for more details. I'd like mesos-master to only do
>>> resource allocation from the periodic batch callback, never from
>>> reviveOffers, addFramework, etc. Can anyone recommend whether this needs to
>>> be a configuration option, or whether I can just remove the event-triggered
>>> allocation cycles?
>>> 
>>> also ... who can shepherd this? :)
>>> 
>>> thanks,
>>> James
>> 
>> 
>> 



Beware of ASSERT_* Placement

2015-07-27 Thread Michael Park
I've had at least 3 individuals who ran into the issue of *ASSERT_** macro
placement and since the generated error message is less than useless, I
would like to share with you what the issue is.

The source of the issue is that *ASSERT_** macros from *gtest* can only be
placed in functions that return *void* as described in Assertion Placement

.

By placing it in a non-void function, you get useless error messages like
this:

>From *GCC*: "*error: void value not ignored as it ought to be*"
>From *Clang*: "*error: could not convert
‘testing::internal::AssertHelper((testing::TestPartResult::Type)2u, ((const
char*)"../../src/tests/containerizer/port_mapping_tests.cpp"), 320,
gtest_ar.testing::AssertionResult::failure_message()).testing::internal::AssertHelper::operator=(testing::Message())’
from ‘void’ to ‘int’*"

I think the following are typical situations that result in this mess:

   - Using them in *constructors/destructors*. For example, it would be
   really confusing if you're simply translating a *SetUp*/*TearDown* of a
   test fixture to be *constructor/destructor* instead.
   - Using them in *main*, since it returns an *int*.
   - Refactoring a chunk of logic from a test case into a helper function
   that doesn't return *void*. For example, if we were factor out the
   following code inside of a test case:




*AWAIT_READY(offers); ASSERT_EQ(1u, offers.get().size()); offer =
   offers.get()[0]; *
   into a function like this:





*Offer getOffer(Future> &offers) {
   AWAIT_READY(offers);   ASSERT_EQ(1u, offers.get().size());   return
   offers.get()[0]; }*

   this innocent-looking transformation would trigger the obscure error
   message and you'll be upset once you figure out why.

If you encounter this case, prefer to resort to *CHECK_** from *glog*, rather
than *EXPECT_**, since *CHECK_** has closer semantics.

I hope this will help folks save time and also reduce the amount of head
banging!

Thanks,

MPark.


Re: Where's Meses design documents

2015-07-27 Thread Benjamin Mahler
Vinod, it would be nice to have them within the repo, can't we have an area
in docs/ for design documents? Even if that just links to active google
docs, seems nicer than using cwiki?

On Mon, Jul 27, 2015 at 2:05 PM, Vinod Kone  wrote:

> Ideally, all design docs that are finalized should go to Mesos's cwiki
> space
> .  We have some
> there already.
>
> @all: if you've finalized (i.e., corresponding code has been committed)
> design docs please move them to the cwiki space. Let me know if you need
> perms.
>
> On Mon, Jul 27, 2015 at 1:45 PM, Michael Park  wrote:
>
> > Hi Klaus,
> >
> > There's no central place where all Mesos design documents live currently.
> > They're mostly google docs which are linked from the JIRA epics. Perhaps
> > it's a good idea to have a central place to store the design documents
> > however, so that they can be discovered easier.
> >
> > MPark.
> >
> > On Sat, Jul 25, 2015 at 11:43 PM Klaus Ma  wrote:
> >
> > > Hi Yong/Alex,
> > >
> > > Thanks very much for those info :). I have went through those documents
> > to
> > > learn the overall design of Mesos.
> > > I think I have to check JIRA for design document of special feature.
> > >
> > > Regards,
> > > 
> > > Klaus Ma (马达), PMP® | http://www.cguru.net
> > >
> > > > On Jul 26, 2015, at 10:07, Yong Feng  wrote:
> > > >
> > > > Hi Klaus
> > > >
> > > > I assume you would like to learn how Mesos is designed and
> implemented
> > > > feature by feature through design doc of Epic. If that is that case,
> > you
> > > > could start with https://github.com/apache/mesos/tree/master/docs
> > which
> > > > describes Mesos overall architecture, func spec of main features,
> > release
> > > > history, roadmap plan and so on. It actually also cover what Alex
> > cited.
> > > >
> > > > After that, if you have interest on specific topic/feature, you could
> > > > search JIRA for related ticket to check the design idea and related
> > > > discussion.
> > > >
> > > > Hope it will help.
> > > >
> > > > Thanks,
> > > >
> > > > Yong
> > > >
> > > > On Sat, Jul 25, 2015 at 9:09 PM, Klaus Ma  wrote:
> > > >
> > > >> I'm asking for the design document of each epic, is there a central
> > > place
> > > >> to download them?Just go through the design document of maintenance,
> > > it's
> > > >> helpful; so I'd like to also learn other design documents.
> > > >>
> > > >> Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
> > > >> CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree
> > via
> > > >> SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> > > CreditFree
> > > >> via Skype
> > > >>
> > > >>> Date: Sat, 25 Jul 2015 18:01:06 -0700
> > > >>> Subject: Re: Where's Meses design documents
> > > >>> From: clemmer.alexan...@gmail.com
> > > >>> To: dev@mesos.apache.org
> > > >>>
> > > >>> Are you asking about the standard documentation (which is here[1]),
> > or
> > > >>> are you asking about the design documents for every epic that has
> > been
> > > >>> added historically?
> > > >>>
> > > >>> [1] http://mesos.apache.org/documentation/latest/
> > > >>>
> > > >>> On Sat, Jul 25, 2015 at 5:53 PM, Klaus Ma  wrote:
> > >  Hi team,
> > >  I can get design documents from each EPIC/Story, but where can I
> get
> > > >> ALL design document of Mesos?
> > > 
> > >  Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
> > >  CallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> CreditFree
> > > >> via SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> > > >> CreditFree via Skype
> > > >>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Alex
> > > >>>
> > > >>> Theory is the first term in the Taylor series of practice. --
> Thomas
> > M
> > > >>> Cover (1992)
> > > >>
> > > >>
> > >
> > >
> >
>


Re: RFC: batch-only allocations

2015-07-27 Thread Vinod Kone
Never mind. Just saw MESOS-3157

On Mon, Jul 27, 2015 at 2:22 PM, Vinod Kone  wrote:

> Can I ask why you want to do this change in the first place?
>
> On Mon, Jul 27, 2015 at 2:11 PM, James Peach  wrote:
>
>> Hi all,
>>
>> See MESOS-3157 for more details. I'd like mesos-master to only do
>> resource allocation from the periodic batch callback, never from
>> reviveOffers, addFramework, etc. Can anyone recommend whether this needs to
>> be a configuration option, or whether I can just remove the event-triggered
>> allocation cycles?
>>
>> also ... who can shepherd this? :)
>>
>> thanks,
>> James
>
>
>


Re: RFC: batch-only allocations

2015-07-27 Thread Vinod Kone
Can I ask why you want to do this change in the first place?

On Mon, Jul 27, 2015 at 2:11 PM, James Peach  wrote:

> Hi all,
>
> See MESOS-3157 for more details. I'd like mesos-master to only do resource
> allocation from the periodic batch callback, never from reviveOffers,
> addFramework, etc. Can anyone recommend whether this needs to be a
> configuration option, or whether I can just remove the event-triggered
> allocation cycles?
>
> also ... who can shepherd this? :)
>
> thanks,
> James


RFC: batch-only allocations

2015-07-27 Thread James Peach
Hi all,

See MESOS-3157 for more details. I'd like mesos-master to only do resource 
allocation from the periodic batch callback, never from reviveOffers, 
addFramework, etc. Can anyone recommend whether this needs to be a 
configuration option, or whether I can just remove the event-triggered 
allocation cycles?

also ... who can shepherd this? :)

thanks,
James

Re: Where's Meses design documents

2015-07-27 Thread Vinod Kone
Ideally, all design docs that are finalized should go to Mesos's cwiki space
.  We have some
there already.

@all: if you've finalized (i.e., corresponding code has been committed)
design docs please move them to the cwiki space. Let me know if you need
perms.

On Mon, Jul 27, 2015 at 1:45 PM, Michael Park  wrote:

> Hi Klaus,
>
> There's no central place where all Mesos design documents live currently.
> They're mostly google docs which are linked from the JIRA epics. Perhaps
> it's a good idea to have a central place to store the design documents
> however, so that they can be discovered easier.
>
> MPark.
>
> On Sat, Jul 25, 2015 at 11:43 PM Klaus Ma  wrote:
>
> > Hi Yong/Alex,
> >
> > Thanks very much for those info :). I have went through those documents
> to
> > learn the overall design of Mesos.
> > I think I have to check JIRA for design document of special feature.
> >
> > Regards,
> > 
> > Klaus Ma (马达), PMP® | http://www.cguru.net
> >
> > > On Jul 26, 2015, at 10:07, Yong Feng  wrote:
> > >
> > > Hi Klaus
> > >
> > > I assume you would like to learn how Mesos is designed and implemented
> > > feature by feature through design doc of Epic. If that is that case,
> you
> > > could start with https://github.com/apache/mesos/tree/master/docs
> which
> > > describes Mesos overall architecture, func spec of main features,
> release
> > > history, roadmap plan and so on. It actually also cover what Alex
> cited.
> > >
> > > After that, if you have interest on specific topic/feature, you could
> > > search JIRA for related ticket to check the design idea and related
> > > discussion.
> > >
> > > Hope it will help.
> > >
> > > Thanks,
> > >
> > > Yong
> > >
> > > On Sat, Jul 25, 2015 at 9:09 PM, Klaus Ma  wrote:
> > >
> > >> I'm asking for the design document of each epic, is there a central
> > place
> > >> to download them?Just go through the design document of maintenance,
> > it's
> > >> helpful; so I'd like to also learn other design documents.
> > >>
> > >> Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
> > >> CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree
> via
> > >> SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> > CreditFree
> > >> via Skype
> > >>
> > >>> Date: Sat, 25 Jul 2015 18:01:06 -0700
> > >>> Subject: Re: Where's Meses design documents
> > >>> From: clemmer.alexan...@gmail.com
> > >>> To: dev@mesos.apache.org
> > >>>
> > >>> Are you asking about the standard documentation (which is here[1]),
> or
> > >>> are you asking about the design documents for every epic that has
> been
> > >>> added historically?
> > >>>
> > >>> [1] http://mesos.apache.org/documentation/latest/
> > >>>
> > >>> On Sat, Jul 25, 2015 at 5:53 PM, Klaus Ma  wrote:
> >  Hi team,
> >  I can get design documents from each EPIC/Story, but where can I get
> > >> ALL design document of Mesos?
> > 
> >  Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
> >  CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree
> > >> via SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> > >> CreditFree via Skype
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Alex
> > >>>
> > >>> Theory is the first term in the Taylor series of practice. -- Thomas
> M
> > >>> Cover (1992)
> > >>
> > >>
> >
> >
>


Re: Where's Meses design documents

2015-07-27 Thread Michael Park
Hi Klaus,

There's no central place where all Mesos design documents live currently.
They're mostly google docs which are linked from the JIRA epics. Perhaps
it's a good idea to have a central place to store the design documents
however, so that they can be discovered easier.

MPark.

On Sat, Jul 25, 2015 at 11:43 PM Klaus Ma  wrote:

> Hi Yong/Alex,
>
> Thanks very much for those info :). I have went through those documents to
> learn the overall design of Mesos.
> I think I have to check JIRA for design document of special feature.
>
> Regards,
> 
> Klaus Ma (马达), PMP® | http://www.cguru.net
>
> > On Jul 26, 2015, at 10:07, Yong Feng  wrote:
> >
> > Hi Klaus
> >
> > I assume you would like to learn how Mesos is designed and implemented
> > feature by feature through design doc of Epic. If that is that case, you
> > could start with https://github.com/apache/mesos/tree/master/docs which
> > describes Mesos overall architecture, func spec of main features, release
> > history, roadmap plan and so on. It actually also cover what Alex cited.
> >
> > After that, if you have interest on specific topic/feature, you could
> > search JIRA for related ticket to check the design idea and related
> > discussion.
> >
> > Hope it will help.
> >
> > Thanks,
> >
> > Yong
> >
> > On Sat, Jul 25, 2015 at 9:09 PM, Klaus Ma  wrote:
> >
> >> I'm asking for the design document of each epic, is there a central
> place
> >> to download them?Just go through the design document of maintenance,
> it's
> >> helpful; so I'd like to also learn other design documents.
> >>
> >> Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
> >> CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree via
> >> SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> CreditFree
> >> via Skype
> >>
> >>> Date: Sat, 25 Jul 2015 18:01:06 -0700
> >>> Subject: Re: Where's Meses design documents
> >>> From: clemmer.alexan...@gmail.com
> >>> To: dev@mesos.apache.org
> >>>
> >>> Are you asking about the standard documentation (which is here[1]), or
> >>> are you asking about the design documents for every epic that has been
> >>> added historically?
> >>>
> >>> [1] http://mesos.apache.org/documentation/latest/
> >>>
> >>> On Sat, Jul 25, 2015 at 5:53 PM, Klaus Ma  wrote:
>  Hi team,
>  I can get design documents from each EPIC/Story, but where can I get
> >> ALL design document of Mesos?
> 
>  Regards,Klaus Ma (马达), PMP® | http://www.cguru.net
>  CallSend SMSCall from mobileAdd to SkypeYou'll need Skype CreditFree
> >> via SkypeCallSend SMSCall from mobileAdd to SkypeYou'll need Skype
> >> CreditFree via Skype
> >>>
> >>>
> >>>
> >>> --
> >>> Alex
> >>>
> >>> Theory is the first term in the Taylor series of practice. -- Thomas M
> >>> Cover (1992)
> >>
> >>
>
>


Re: Introducing a CMake-based build system for Mesos

2015-07-27 Thread Alex Clemmer
Yes, CMake is currently checking if the -std=c++11 flag exists. CMake
3 supports the autotools-style "feature check" style (which would be
more appropriate on platforms like Windows, anyway) but it's not clear
how far back we want to support just yet -- right now we support back
to 2.8.

One thing to consider is that the "default" install of CMake for
apt-get on recent versions of Ubuntu is v2.8. In general, we'd like to
have as smooth an install experience as possible for as many platforms
as we can; I'm willing to be convinced that this isn't worth it, but
to be safe we've developed with 2.8 in mind, because it's easier to go
from 2.8 to 3 than the reverse.

On Mon, Jul 27, 2015 at 9:37 AM, James Peach  wrote:
>
>> On Jul 23, 2015, at 12:40 PM, Alex Clemmer  
>> wrote:
>>
>> A fix is up for review here[1]. Thanks again for your feedback, this
>> is very valuable!
>>
>> [1] https://reviews.apache.org/r/36743/
>
> AFAICT this just checks whether the -std=c++11 compiler option is accepted. 
> The equivalent autoconf macro checks that various C++11 features compile, and 
> people have added more over time ... 
> 9eda4331dd23c3646aba1ec710e0dd3190e579ab, 
> 623d6a0d0f0eb90be80b7e95c91ece89de513367, 
> b930d5ce32b60b7c126844a3ef6ae119d36bc8d0, etc.
>
> Am I reading the cmake right?
>
>>
>> On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone  wrote:
>>> yup.
>>>
>>> checking for C++ compiler version... 4.1.2
>>>
>>> checking for C++ compiler vendor... (cached) gnu
>>>
>>> configure: error: GCC 4.8 or higher required (found 4.1.2)
>>>
>>> [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?
>>>
>>> 1
>>>
>>> On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer 
>>> wrote:
>>>
 We can easily change that to be a FATAL_ERROR or a WARNING. I
 recommend being at parity with autotools -- am I correct in assuming
 that it errors out?

 On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone  wrote:
> The one thing I found odd while testing was that some errors when running
> 'cmake' do not result in a non-zero exit status.
> For example, when I tested with an older version of GCC it gave a warning
> about C++11 not being supported but went ahead otherwise.
>
> -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed
>
> *--
>
 Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*
>
> -- Looking for include file pthread.h
>
> -- Looking for include file pthread.h - found
>
> -- Looking for pthread_create
>
> -- Looking for pthread_create - not found
>
> -- Looking for pthread_create in pthreads
>
> -- Looking for pthread_create in pthreads - not found
>
> -- Looking for pthread_create in pthread
>
> -- Looking for pthread_create in pthread - found
>
> -- Found Threads: TRUE
>
> -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.3")
>
> -- Found APR headers: /usr/include/apr-1
>
> -- Found APR library: /usr/lib64/libapr-1.so
>
> -- Found APRUTIL headers: /usr/include/apr-1
>
> -- Found APRUTIL library: /usr/lib64/libaprutil-1.so
>
> -- Found SVN lib: /usr/lib64/libsvn_client-1.so
>
> -- Found SVN lib: /usr/lib64/libsvn_delta-1.so
>
> -- Found SVN lib: /usr/lib64/libsvn_diff-1.so
>
> -- Found SVN lib: /usr/lib64/libsvn_fs-1.so
>
> -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so
>
>
>
>
> On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer <
 clemmer.alexan...@gmail.com>
> wrote:
>
>> I've put up a pair of fixes, tested on OS X 10.10. They are here:
>>
>> (1) https://reviews.apache.org/r/36740/
>> (2) https://reviews.apache.org/r/36741/
>>
>> This should resolve the issues, and thanks again for the bug report.
>>
>> On Thu, Jul 23, 2015 at 3:32 AM, haosdent  wrote:
>>> Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when
>> the
>>> operate system is OSX?
>>>
>>> On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer <
>> clemmer.alexan...@gmail.com>
>>> wrote:
>>>
 Thanks for reporting the issue! I appreciate it.

 This code is trying to find librt, which provides the POSIX.1b
 Realtime Extension (i.e., things like message passing, async I/O,
 mmap'd files, etc.). Assuming you're running some flavor of Linux,
 this _should_ exist on your system already, and `find_library` is the
 CMake-standard function to find it, so it is not immediately clear to
 me what went wrong here.

 Do you mind if I ask what system you are running?

 On Thu, Jul 23, 2015 at 1:16 AM, haosdent 
 wrote:
> Hi, @Alex Clemmer  I try to build it on OS X 10.10
>
> ```
> mkdir build-cmake
> cmake ..
> make
> ```
>
> But have this error:
> ```
> CMake Error: The 

Re: Introducing a CMake-based build system for Mesos

2015-07-27 Thread James Peach

> On Jul 23, 2015, at 12:40 PM, Alex Clemmer  
> wrote:
> 
> A fix is up for review here[1]. Thanks again for your feedback, this
> is very valuable!
> 
> [1] https://reviews.apache.org/r/36743/

AFAICT this just checks whether the -std=c++11 compiler option is accepted. The 
equivalent autoconf macro checks that various C++11 features compile, and 
people have added more over time ... 9eda4331dd23c3646aba1ec710e0dd3190e579ab, 
623d6a0d0f0eb90be80b7e95c91ece89de513367, 
b930d5ce32b60b7c126844a3ef6ae119d36bc8d0, etc.

Am I reading the cmake right? 

> 
> On Thu, Jul 23, 2015 at 12:18 PM, Vinod Kone  wrote:
>> yup.
>> 
>> checking for C++ compiler version... 4.1.2
>> 
>> checking for C++ compiler vendor... (cached) gnu
>> 
>> configure: error: GCC 4.8 or higher required (found 4.1.2)
>> 
>> [vinod@smfd-atr-11-sr1 build-cmake]$ echo $?
>> 
>> 1
>> 
>> On Thu, Jul 23, 2015 at 12:17 PM, Alex Clemmer 
>> wrote:
>> 
>>> We can easily change that to be a FATAL_ERROR or a WARNING. I
>>> recommend being at parity with autotools -- am I correct in assuming
>>> that it errors out?
>>> 
>>> On Thu, Jul 23, 2015 at 12:12 PM, Vinod Kone  wrote:
 The one thing I found odd while testing was that some errors when running
 'cmake' do not result in a non-zero exit status.
 For example, when I tested with an older version of GCC it gave a warning
 about C++11 not being supported but went ahead otherwise.
 
 -- Performing Test COMPILER_SUPPORTS_CXX11 - Failed
 
 *--
 
>>> Thecompiler/usr/bin/c++doesnotsupportthe`-std=c++11`flag.PleaseuseadifferentC++compiler.*
 
 -- Looking for include file pthread.h
 
 -- Looking for include file pthread.h - found
 
 -- Looking for pthread_create
 
 -- Looking for pthread_create - not found
 
 -- Looking for pthread_create in pthreads
 
 -- Looking for pthread_create in pthreads - not found
 
 -- Looking for pthread_create in pthread
 
 -- Looking for pthread_create in pthread - found
 
 -- Found Threads: TRUE
 
 -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.3")
 
 -- Found APR headers: /usr/include/apr-1
 
 -- Found APR library: /usr/lib64/libapr-1.so
 
 -- Found APRUTIL headers: /usr/include/apr-1
 
 -- Found APRUTIL library: /usr/lib64/libaprutil-1.so
 
 -- Found SVN lib: /usr/lib64/libsvn_client-1.so
 
 -- Found SVN lib: /usr/lib64/libsvn_delta-1.so
 
 -- Found SVN lib: /usr/lib64/libsvn_diff-1.so
 
 -- Found SVN lib: /usr/lib64/libsvn_fs-1.so
 
 -- Found SVN lib: /usr/lib64/libsvn_fs_base-1.so
 
 
 
 
 On Thu, Jul 23, 2015 at 12:07 PM, Alex Clemmer <
>>> clemmer.alexan...@gmail.com>
 wrote:
 
> I've put up a pair of fixes, tested on OS X 10.10. They are here:
> 
> (1) https://reviews.apache.org/r/36740/
> (2) https://reviews.apache.org/r/36741/
> 
> This should resolve the issues, and thanks again for the bug report.
> 
> On Thu, Jul 23, 2015 at 3:32 AM, haosdent  wrote:
>> Sure, I use OS X 10.10. Seems OS X don't have librt, don't add rt when
> the
>> operate system is OSX?
>> 
>> On Thu, Jul 23, 2015 at 6:22 PM, Alex Clemmer <
> clemmer.alexan...@gmail.com>
>> wrote:
>> 
>>> Thanks for reporting the issue! I appreciate it.
>>> 
>>> This code is trying to find librt, which provides the POSIX.1b
>>> Realtime Extension (i.e., things like message passing, async I/O,
>>> mmap'd files, etc.). Assuming you're running some flavor of Linux,
>>> this _should_ exist on your system already, and `find_library` is the
>>> CMake-standard function to find it, so it is not immediately clear to
>>> me what went wrong here.
>>> 
>>> Do you mind if I ask what system you are running?
>>> 
>>> On Thu, Jul 23, 2015 at 1:16 AM, haosdent 
>>> wrote:
 Hi, @Alex Clemmer  I try to build it on OS X 10.10
 
 ```
 mkdir build-cmake
 cmake ..
 make
 ```
 
 But have this error:
 ```
 CMake Error: The following variables are used in this project, but
> they
>>> are
 set to NOTFOUND.
 Please set them or make sure they are set and tested correctly in
>>> the
>>> CMake
 files:
 LIBRT_LIBRARIES
linked by target "tests" in directory
 /Users/haosdent/workspace/cpp/mesos/3rdparty/libprocess/src/tests
 
 -- Configuring incomplete, errors occurred!
 ```
 
 Any steps I wrong here?
 
 On Thu, Jul 23, 2015 at 11:27 AM, Marco Massenzio <
> ma...@mesosphere.io>
 wrote:
 
> This is really cool!
> Eclipse CDT is becoming a bit tiresome to use, but JetLabs' CLion
> only
> support cmake, so I definitely have a stake in this working :)
> 
>