[jira] [Updated] (MESOS-1832) Slave should accept PingSlaveMessage but not PING message.

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-1832:
--
Target Version/s: 0.24.0  (was: 0.23.0)

 Slave should accept PingSlaveMessage but not PING message.
 

 Key: MESOS-1832
 URL: https://issues.apache.org/jira/browse/MESOS-1832
 Project: Mesos
  Issue Type: Task
Reporter: Vinod Kone
Assignee: Adam B
  Labels: mesosphere

 Slave handles both PING message and PingSlaveMessage in until 0.22.0 for 
 backwards compatibility (https://reviews.apache.org/r/25867/).
 In 0.23.0, slave no longer needs handle PING.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-1831) Master should send PingSlaveMessage instead of PING

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-1831:
--
Target Version/s: 0.24.0  (was: 0.23.0)

 Master should send PingSlaveMessage instead of PING
 -

 Key: MESOS-1831
 URL: https://issues.apache.org/jira/browse/MESOS-1831
 Project: Mesos
  Issue Type: Task
Reporter: Vinod Kone
Assignee: Adam B
  Labels: mesosphere

 In 0.21.0 master sends PING message with an embedded PingSlaveMessage for 
 backwards compatibility (https://reviews.apache.org/r/25867/).
 In 0.22.0, master should send PingSlaveMessage directly instead of PING.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-1791) Introduce Master / Offer Resource Reservations aka Quota

2015-06-24 Thread Alexander Rukletsov (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Rukletsov updated MESOS-1791:
---
  Epic Name: Quota  (was: Offer Reservations)
Component/s: replicated log
 master
 allocation
Summary: Introduce Master / Offer Resource Reservations aka Quota  
(was: Introduce Master / Offer Resource Reservations)

 Introduce Master / Offer Resource Reservations aka Quota
 

 Key: MESOS-1791
 URL: https://issues.apache.org/jira/browse/MESOS-1791
 Project: Mesos
  Issue Type: Epic
  Components: allocation, master, replicated log
Reporter: Tom Arnfeld
  Labels: mesosphere

 Currently Mesos supports the ability to reserve resources (for a given role) 
 on a per-slave basis, as introduced in MESOS-505. This allows you to almost 
 statically partition off a set of resources on a set of machines, to 
 guarantee certain types of frameworks get some resources.
 This is very useful, though it is also very useful to be able to control 
 these reservations through the master (instead of per-slave) for when I don't 
 care which nodes I get on, as long as I get X cpu and Y RAM, or Z sets of 
 (X,Y).
 I'm not sure what structure this could take, but apparently it has already 
 been discussed. Would this be a CLI flag? Could there be a (authenticated) 
 web interface to control these reservations?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2774) SIGSEGV received during process::MessageEncoder::encode()

2015-06-24 Thread Klaus Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599149#comment-14599149
 ] 

Klaus Ma commented on MESOS-2774:
-

Where can I download the core file?

 SIGSEGV received during process::MessageEncoder::encode()
 -

 Key: MESOS-2774
 URL: https://issues.apache.org/jira/browse/MESOS-2774
 Project: Mesos
  Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Yan Xu
Priority: Critical

 Observed in production.
 {noformat:title=}
 Program terminated with signal 11, Segmentation fault.
 #0  0x7fd6ded54b2e in _itoa_word () from /lib64/libc.so.6
 (gdb) bt
 #0  0x7fd6ded54b2e in _itoa_word () from /lib64/libc.so.6
 #1  0x7fd6ded57f13 in vfprintf () from /lib64/libc.so.6
 #2  0x7fd6ded75b49 in vsprintf () from /lib64/libc.so.6
 #3  0x7fd6ded5fe38 in sprintf () from /lib64/libc.so.6
 #4  0x7fd6dedf473e in inet_ntop () from /lib64/libc.so.6
 #5  0x7fd6e058aed6 in operator (stream=..., pid=Unhandled dwarf 
 expression opcode 0xf3
 ) at ./3rdparty/stout/include/stout/ip.hpp:225
 #6  operator (stream=..., pid=Unhandled dwarf expression opcode 0xf3
 ) at ./include/process/address.hpp:106
 #7  process::operator (stream=..., pid=Unhandled dwarf expression opcode 
 0xf3
 ) at src/pid.cpp:65
 #8  0x7fd6e05ab29a in process::MessageEncoder::encode 
 (message=0x7fd6b80e2d70) at src/encoder.hpp:123
 #9  0x7fd6e05ad1b5 in process::MessageEncoder::MessageEncoder 
 (this=0x7fd6b813aed0, s=..., _message=0x7fd6b80e2d70) at src/encoder.hpp:98
 #10 0x7fd6e05a60cc in process::SocketManager::send (this=0x124d100, 
 message=0x7fd6b80e2d70) at src/process.cpp:1583
 #11 0x7fd6e05a6750 in process::ProcessBase::send (this=0x7fd6c00097d0, 
 to=..., name=PONG, data=0x0, length=0) at src/process.cpp:2679
 #12 0x7fd6e0155f87 in mesos::internal::slave::Slave::pingOld 
 (this=0x7fd6c0008f20, from=..., body=Unhandled dwarf expression opcode 0xf3
 ) at slave/slave.cpp:2896
 #13 0x7fd6e05a68cb in operator() (this=0x7fd6c00097d0, event=...) at 
 /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/functional:2464
 #14 process::ProcessBase::visit (this=0x7fd6c00097d0, event=...) at 
 src/process.cpp:2688
 #15 0x7fd6e019120d in 
 ProtobufProcessmesos::internal::slave::Slave::visit (this=0x7fd6c0008f20, 
 event=...) at ../3rdparty/libprocess/include/process/protobuf.hpp:94
 #16 0x7fd6e05a009a in process::ProcessManager::resume (this=0x124d3b0, 
 process=0x7fd6c00097d0) at src/process.cpp:2175
 #17 0x7fd6e05a035c in process::schedule (arg=Unhandled dwarf expression 
 opcode 0xf3
 ) at src/process.cpp:613
 #18 0x7fd6df5f583d in start_thread () from /lib64/libpthread.so.0
 #19 0x7fd6dede7fcd in clone () from /lib64/libc.so.6
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2154) Port CFS support to Docker Containerizer

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599049#comment-14599049
 ] 

Adam B commented on MESOS-2154:
---

Pushing this out to 0.24, since nobody seems to be working on it.

 Port CFS support to Docker Containerizer
 

 Key: MESOS-2154
 URL: https://issues.apache.org/jira/browse/MESOS-2154
 Project: Mesos
  Issue Type: Improvement
  Components: docker, isolation
Affects Versions: 0.21.0
 Environment: Linux (Ubuntu 14.04.1)
Reporter: Andrew Ortman
Priority: Minor

 Port the CFS support the Mesos Containerizer has to the Docker Containerizer. 
 Whenever the --cgroup_enable_cfs flag is set, the Docker Containerizer should 
 update the cfs_period_us and cfs_quota_us values to allow hard CPU capping on 
 the container. 
 Current workaround is to pass those values as LXC configuration parameters



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2154) Port CFS support to Docker Containerizer

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-2154:
--
Target Version/s: 0.24.0  (was: 0.23.0)

 Port CFS support to Docker Containerizer
 

 Key: MESOS-2154
 URL: https://issues.apache.org/jira/browse/MESOS-2154
 Project: Mesos
  Issue Type: Improvement
  Components: docker, isolation
Affects Versions: 0.21.0
 Environment: Linux (Ubuntu 14.04.1)
Reporter: Andrew Ortman
Priority: Minor

 Port the CFS support the Mesos Containerizer has to the Docker Containerizer. 
 Whenever the --cgroup_enable_cfs flag is set, the Docker Containerizer should 
 update the cfs_period_us and cfs_quota_us values to allow hard CPU capping on 
 the container. 
 Current workaround is to pass those values as LXC configuration parameters



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2664) Modernize the codebase to C++11

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599114#comment-14599114
 ] 

Adam B commented on MESOS-2664:
---

Retargeting this epic to 0.24, since we won't complete all of the issues in 
time for 0.23. The issues that have been resolved will be listed individually 
in the CHANGELOG, but we'll have to wait at least one more release before we 
can claim to have modernized the codebase to C++11. Great work so far though!

 Modernize the codebase to C++11
 ---

 Key: MESOS-2664
 URL: https://issues.apache.org/jira/browse/MESOS-2664
 Project: Mesos
  Issue Type: Epic
  Components: technical debt
Reporter: Michael Park
Assignee: Michael Park
  Labels: mesosphere

 Since [this 
 commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7],
  we officially require GCC-4.8+ and Clang-3.5+. This means that we now have 
 full C++11 support and therefore can start to modernize our codebase to be 
 more readable, safer and efficient!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-1807) Disallow executors with cpu only or memory only resources

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-1807:
--
Target Version/s: 0.24.0  (was: 0.23.0)

 Disallow executors with cpu only or memory only resources
 -

 Key: MESOS-1807
 URL: https://issues.apache.org/jira/browse/MESOS-1807
 Project: Mesos
  Issue Type: Improvement
Reporter: Vinod Kone
  Labels: newbie

 Currently master allows executors to be launched with either only cpus or 
 only memory but we shouldn't allow that.
 This is because executor is an actual unix process that is launched by the 
 slave. If an executor doesn't specify cpus, what should do the cpu limits be 
 for that executor when there are no tasks running on it? If no cpu limits are 
 set then it might starve other executors/tasks on the slave violating 
 isolation guarantees. Same goes with memory. Moreover, the current 
 containerizer/isolator code will throw failures when using such an executor, 
 e.g., when the last task on the executor finishes and Containerizer::update() 
 is called with 0 cpus or 0 mem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599058#comment-14599058
 ] 

Adam B commented on MESOS-1807:
---

Thanks for the explanation, Vinod. I'm targeting this out of 0.23 since nobody 
is actively working on it.

 Disallow executors with cpu only or memory only resources
 -

 Key: MESOS-1807
 URL: https://issues.apache.org/jira/browse/MESOS-1807
 Project: Mesos
  Issue Type: Improvement
Reporter: Vinod Kone
  Labels: newbie

 Currently master allows executors to be launched with either only cpus or 
 only memory but we shouldn't allow that.
 This is because executor is an actual unix process that is launched by the 
 slave. If an executor doesn't specify cpus, what should do the cpu limits be 
 for that executor when there are no tasks running on it? If no cpu limits are 
 set then it might starve other executors/tasks on the slave violating 
 isolation guarantees. Same goes with memory. Moreover, the current 
 containerizer/isolator code will throw failures when using such an executor, 
 e.g., when the last task on the executor finishes and Containerizer::update() 
 is called with 0 cpus or 0 mem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2664) Modernize the codebase to C++11

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-2664:
--
Target Version/s: 0.24.0  (was: 0.23.0)

 Modernize the codebase to C++11
 ---

 Key: MESOS-2664
 URL: https://issues.apache.org/jira/browse/MESOS-2664
 Project: Mesos
  Issue Type: Epic
  Components: technical debt
Reporter: Michael Park
Assignee: Michael Park
  Labels: mesosphere

 Since [this 
 commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7],
  we officially require GCC-4.8+ and Clang-3.5+. This means that we now have 
 full C++11 support and therefore can start to modernize our codebase to be 
 more readable, safer and efficient!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (MESOS-1153) configure does not check for patch

2015-06-24 Thread Kapil Arya (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kapil Arya closed MESOS-1153.
-
Resolution: Fixed

commit 349884a9159c6dc62d82a9ddd6e0dca11df2e287
Author: haosdent huang haosd...@gmail.com
Date:   Mon Apr 27 01:55:02 2015 +0200

Added check for availability of the patch command in configure.ac.

Review: https://reviews.apache.org/r/33266

 configure does not check for patch
 --

 Key: MESOS-1153
 URL: https://issues.apache.org/jira/browse/MESOS-1153
 Project: Mesos
  Issue Type: Bug
  Components: build
 Environment: Linux
Reporter: Vinson Lee

 The build uses the patch command but configure never checks for it's 
 existence.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2894) web UI shows YYYY for year instead of year

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-2894:
--
Assignee: haosdent

 web UI shows  for year instead of year
 

 Key: MESOS-2894
 URL: https://issues.apache.org/jira/browse/MESOS-2894
 Project: Mesos
  Issue Type: Bug
  Components: webui
Reporter: brian wickman
Assignee: haosdent
Priority: Minor
  Labels: newbie

 The directory listing formats on  when it should probably format on :
 {noformat}
 drwxr-xr-x3   ads twitter 4 KBJun 18 15:27 .pex   
 drwxr-xr-x2   ads twitter 4 KBDec 01   aurora
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2894) web UI shows YYYY for year instead of year

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598994#comment-14598994
 ] 

Adam B commented on MESOS-2894:
---

Review: https://reviews.apache.org/r/35782

 web UI shows  for year instead of year
 

 Key: MESOS-2894
 URL: https://issues.apache.org/jira/browse/MESOS-2894
 Project: Mesos
  Issue Type: Bug
  Components: webui
Reporter: brian wickman
Assignee: haosdent
Priority: Minor
  Labels: newbie

 The directory listing formats on  when it should probably format on :
 {noformat}
 drwxr-xr-x3   ads twitter 4 KBJun 18 15:27 .pex   
 drwxr-xr-x2   ads twitter 4 KBDec 01   aurora
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2670) Update existing lambdas to meet style guide

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599107#comment-14599107
 ] 

Adam B commented on MESOS-2670:
---

[~jvanremoortere] [~benjaminhindman] Now that the above patches have been 
committed, is there anything left to do before we can close this ticket?

 Update existing lambdas to meet style guide
 ---

 Key: MESOS-2670
 URL: https://issues.apache.org/jira/browse/MESOS-2670
 Project: Mesos
  Issue Type: Task
Reporter: Joris Van Remoortere
Assignee: haosdent
  Labels: c++11

 There are already some lambdas in C++11 specific files. Modify these to meet 
 the updated style guide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2632) Remove capture by reference of temporaries in mesos

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599045#comment-14599045
 ] 

Adam B commented on MESOS-2632:
---

[~jvanremoortere] Do you think we'll get this into 0.23, or should we push it 
out to the next release?

 Remove capture by reference of temporaries in mesos
 ---

 Key: MESOS-2632
 URL: https://issues.apache.org/jira/browse/MESOS-2632
 Project: Mesos
  Issue Type: Task
  Components: technical debt
Reporter: Joris Van Remoortere





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (MESOS-2894) web UI shows YYYY for year instead of year

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B resolved MESOS-2894.
---
   Resolution: Fixed
Fix Version/s: 0.23.0

commit d86341b216dc9bb0f97663007c04d147b663f427
Author: haosdent huang haosd...@gmail.com
Date:   Wed Jun 24 01:29:57 2015 -0700

Fixed web UI showing  for year instead of year.

Review: https://reviews.apache.org/r/35782

 web UI shows  for year instead of year
 

 Key: MESOS-2894
 URL: https://issues.apache.org/jira/browse/MESOS-2894
 Project: Mesos
  Issue Type: Bug
  Components: webui
Reporter: brian wickman
Assignee: haosdent
Priority: Minor
  Labels: newbie
 Fix For: 0.23.0

 Attachments: MESOS-2894-after.png, MESOS-2894-before.png


 The directory listing formats on  when it should probably format on :
 {noformat}
 drwxr-xr-x3   ads twitter 4 KBJun 18 15:27 .pex   
 drwxr-xr-x2   ads twitter 4 KBDec 01   aurora
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2572) Add memory statistics tests.

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599051#comment-14599051
 ] 

Adam B commented on MESOS-2572:
---

[~jieyu] [~idownes] Do either of you have time to review these patches? Looks 
like [~chzhcn] updated them recently after Ian's latest reviews. They've been 
sitting around for a while now, and they won't make it into 0.23 without some 
attention.

 Add memory statistics tests.
 

 Key: MESOS-2572
 URL: https://issues.apache.org/jira/browse/MESOS-2572
 Project: Mesos
  Issue Type: Task
Reporter: Chi Zhang
Assignee: Chi Zhang
  Labels: twitter





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2757) Can't Use - operator with OptionT

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599102#comment-14599102
 ] 

Adam B commented on MESOS-2757:
---

[~jvanremoortere] This review was discarded (no reason given). Could you please 
explain why and update this ticket if it's no longer reviewable? Is it still 
an issue?

 Can't Use - operator with OptionT
 

 Key: MESOS-2757
 URL: https://issues.apache.org/jira/browse/MESOS-2757
 Project: Mesos
  Issue Type: Improvement
  Components: stout
Reporter: Joris Van Remoortere
Assignee: Joris Van Remoortere
  Labels: c++11, mesosphere, option, stout

 Let's add operator overloads to OptionT to allow access to the underlying T 
 using the `-` operator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2673) Follow Google Style Guide for header file include order completely.

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599195#comment-14599195
 ] 

Adam B commented on MESOS-2673:
---

[~js84] If, as described in the gist, this Epic describes not just updating the 
style guide, but also making the changes in stout/libprocess/mesos, then I 
highly doubt that we will complete this Epic in time for 0.23. Could you please 
retarget to 0.24 if appropriate? Thanks!

 Follow Google Style Guide for header file include order completely.
 ---

 Key: MESOS-2673
 URL: https://issues.apache.org/jira/browse/MESOS-2673
 Project: Mesos
  Issue Type: Improvement
Reporter: Joerg Schad
Assignee: Joerg Schad
Priority: Minor
  Labels: mesosphere
 Fix For: 0.23.0


 The header include order for Mesos actually follows the Google Styleguide but 
 omits step 1 without mentioning this exception in the Mesos styleguide. This 
 proposal suggests to adapt to the include order explained in the Google 
 Styleguide i.e. include the direct headers first in the .cpp files 
 implementing them.
 A gist of the proposal can be found here: 
 https://gist.github.com/joerg84/65cb9611d24b2e35b69b
 The corresponding Review Board review can be found here:
 https://reviews.apache.org/r/33646/ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2888) Add SSL socket tests

2015-06-24 Thread Adam B (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam B updated MESOS-2888:
--
Target Version/s: 0.23.0

 Add SSL socket tests
 

 Key: MESOS-2888
 URL: https://issues.apache.org/jira/browse/MESOS-2888
 Project: Mesos
  Issue Type: Improvement
  Components: libprocess
Reporter: Joris Van Remoortere
Assignee: Joris Van Remoortere
  Labels: libprocess, mesosphere, ssl, tests





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2884) Allow isolators to specify required namespaces

2015-06-24 Thread Adam B (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599202#comment-14599202
 ] 

Adam B commented on MESOS-2884:
---

[~karya] These RRs have been committed. Is there anything else left to do for 
this ticket?

commit 1a82a3fb2bc717c468218384190a115b770f88c3
Author: Kapil Arya ka...@mesosphere.io
Date:   Tue Jun 23 12:33:41 2015 -0700

Updated LinuxLauncher to receive list of namespaces.

MesosContainerizer looks up the list of required namespaces by calling
Isolator::namespaces() for all enabled isolators and passes on this
value to LinuxLauncher.

Review: https://reviews.apache.org/r/35586

commit 2143ae0315990ed663bf5810a801adeacff3a986
Author: Kapil Arya ka...@mesosphere.io
Date:   Tue Jun 23 12:32:32 2015 -0700

Updated Isolator to return required namespaces.

This would enable the MesosContainerizer to pass on a list of namespaces
to LinuxLauncher instead of having LinuxLauncher guess it from the
isolation flags.

Review: https://reviews.apache.org/r/35585

 Allow isolators to specify required namespaces
 --

 Key: MESOS-2884
 URL: https://issues.apache.org/jira/browse/MESOS-2884
 Project: Mesos
  Issue Type: Task
  Components: isolation
Reporter: Kapil Arya
Assignee: Kapil Arya
  Labels: mesosphere

 Currently, the LinuxLauncher looks into SlaveFlags to compute the namespaces 
 that should be enabled when launching the executor. This means that a custom 
 Isolator module doesn't have any way to specify dependency on a set of 
 namespaces.
 The proposed solution is to extend the Isolator interface to also export the 
 namespaces dependency. This way the MesosContainerizer can directly query all 
 loaded Isolators (inbuilt and custom modules) to compute the set of 
 namespaces required by the executor. This set of namespaces is then passed on 
 to the LinuxLauncher.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code

2015-06-24 Thread Colin Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600087#comment-14600087
 ] 

Colin Williams commented on MESOS-2637:
---

I'll do that, thanks for your help so far.
On Jun 24, 2015 2:19 PM, Niklas Quarfot Nielsen (JIRA) j...@apache.org



 Consolidate 'foo', 'bar', ... string constants in test and example code
 ---

 Key: MESOS-2637
 URL: https://issues.apache.org/jira/browse/MESOS-2637
 Project: Mesos
  Issue Type: Bug
  Components: technical debt
Reporter: Niklas Quarfot Nielsen
Assignee: Colin Williams

 We are using 'foo', 'bar', ... string constants and pairs in 
 src/tests/master_tests.cpp, src/tests/slave_tests.cpp, 
 src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and 
 hooks tests. These values should be stored in local variables to avoid the 
 possibility of assignment getting out of sync with checking for that same 
 value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-999) Slave should wait() and start executor registration timeout after launch

2015-06-24 Thread Yan Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600134#comment-14600134
 ] 

Yan Xu commented on MESOS-999:
--

So we ended up not taking on this.
Our motivation for addressing this was because with docker and other new 
containerization efforts such as MESOS-2386, some considerable amount of time 
can be spent on pulling and preparing the container images before the executor 
is launched so we don't want the slave to kill it due to a small 
{{--executor_registration_timeout}}.
While implementing this we realized that adding a slave level launch timeout 
flag doesn't solve the fundamental problem of the long image preparation having 
a noticeable impact on the slave and its tasks. We should instead working 
towards a solution that minimizes such impact.
The {{--executor_registration_timeout}} flag was originally introduced to 
account for the time required to fetch the executor so for now giving it a 
large value is the reasonable way for tasks that use container images.
Ultimately only the task knows what the expected preparation time is so such 
timeouts should probably go to ExecutorInfo or TaskInfo.

I feel like this ticket as it is currently phrased could be closed as {{Won't 
Fix}}. Do you agree [~idownes]?

 Slave should wait() and start executor registration timeout after launch 
 -

 Key: MESOS-999
 URL: https://issues.apache.org/jira/browse/MESOS-999
 Project: Mesos
  Issue Type: Bug
  Components: isolation
Affects Versions: 0.18.0
Reporter: Ian Downes
Assignee: Yan Xu
Priority: Minor

 The current code will start launch a container and wait on it before the 
 launch is complete. We should do this only after the container has 
 successfully launched. Likewise for the executor registration timeout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-1187) precision errors with allocation calculations

2015-06-24 Thread Jie Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-1187:
--
Priority: Major  (was: Minor)

 precision errors with allocation calculations
 -

 Key: MESOS-1187
 URL: https://issues.apache.org/jira/browse/MESOS-1187
 Project: Mesos
  Issue Type: Bug
  Components: allocation, master
Reporter: aniruddha sathaye

 As allocations are stored/transmitted as doubles many a times precision 
 errors creep in. 
 we have seen erroneous share calculations happen only because of floating 
 point arithmetic. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (MESOS-2889) Add SSL switch to python configuration

2015-06-24 Thread Artem Harutyunyan (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Harutyunyan reassigned MESOS-2889:


Assignee: Artem Harutyunyan  (was: Joris Van Remoortere)

 Add SSL switch to python configuration
 --

 Key: MESOS-2889
 URL: https://issues.apache.org/jira/browse/MESOS-2889
 Project: Mesos
  Issue Type: Bug
Reporter: Joris Van Remoortere
Assignee: Artem Harutyunyan
  Labels: mesosphere

 The python egg requires explicit dependencies for SSL. Add these to the 
 python configuration if ssl is enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Paul Brett (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600121#comment-14600121
 ] 

Paul Brett commented on MESOS-2925:
---

[~jvanremoortere] - I think the init macro in the initializer list use looks 
much better but the compiler warns against it because the behavior is undefined 
and therefore unsafe.  BTW, I'm using clang on Linux, so I don't know if the 
proposed Apple tweak would help me.  

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-999) Slave should wait() and start executor registration timeout after launch

2015-06-24 Thread Yan Xu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Xu updated MESOS-999:
-
Labels:   (was: twitter)

 Slave should wait() and start executor registration timeout after launch 
 -

 Key: MESOS-999
 URL: https://issues.apache.org/jira/browse/MESOS-999
 Project: Mesos
  Issue Type: Bug
  Components: isolation
Affects Versions: 0.18.0
Reporter: Ian Downes
Priority: Minor

 The current code will start launch a container and wait on it before the 
 launch is complete. We should do this only after the container has 
 successfully launched. Likewise for the executor registration timeout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600254#comment-14600254
 ] 

Alex Clemmer commented on MESOS-898:


[~vinodkone]:
{quote}
-- Looks like you are adding windows support too (I'm guessing that's your 
main motivation to work on this? Stoked to see this btw!). It's probably better 
to work on WIN support in a separate patch.
{quote}

First, yes, I'm attempting to contribute Windows (and Windows container) 
support to the Mesos project, and yes, this is the first big step. But, please 
don't spread it around to too many people just yet -- to the extent possible, I 
am hoping to avoid impacting Mesosphere's ability to effectively launch the 
feature. :)

To your suggestion that we split Windows support into a different patch: it 
seems like you're saying it might be better to first contribute a baseline 
CMake-based build system, and then extend that to support Windows later. (That 
is, I assume you're not talking about how to package up the changes to the C++ 
code that we will need in order to support Windows, which I assume we all agree 
belong in a different patch.) Is that all right?

I'm actually hoping to make x-plat support out of the box a core goal of this 
new build system. My rationale is basically twofold: (1) I am basically certain 
that I will end up developing a very different build system if we don't make 
x-plat support a priority right out of the gate (right now, I'm testing that 
each iteration works on both Linux and Windows, which definitely guides the 
design a lot), and (2) I've taken care to clearly mark off the Windows-specific 
stuff so that we can pull it out easily later if this Windows stuff doesn't 
work out. More particularly, I am suggesting this course of action because I 
think it will result in the least aggregate work in expectation.

That all said, I understand the hesitation to put stuff into the codebase if it 
isn't used, and I'm totally willing to be convinced that I'm wrong. Let me know 
if you see something I don't.

To your other issues, that boost issue that you and [~xujyan] is good to know, 
so thanks for pointing it out. And, I was actually hoping that one of you had 
good knowledge of CMake idioms. :) If we don't have someone with that expertise 
on deck, I agree we need to go find it before merge.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Cody Maloney (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600263#comment-14600263
 ] 

Cody Maloney commented on MESOS-898:


I would suggest that with the move to CMake we switch to using a raw upstream 
packaged version of boost. There isn't a lot we gain by stripping out some of 
the headers, and it adds a lot more complexity. CMake has a lot of stuff 
ready-made for finding, downloading boost if and only if it isn't present on 
the host machine, isn't of the right version, etc. Forcing rebuilding all of 
that logic/code so that we can remove some files in a tarball which shouldn't 
be embedded inside the repository anyways seems like not the best idea.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Joris Van Remoortere (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600084#comment-14600084
 ] 

Joris Van Remoortere commented on MESOS-2925:
-

Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled 
initializer warning. According to [~tillt] this is something that Apple tends 
to tweak before they provide the stable releases of clang for OSX?

AFAIK the macro expands to either `{false}` or {0}. I don't see why it is 
harmful to keep using this to initialize the locks. I find the code in the 
review request harder to read due to the noise. What do you think?

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Reopened] (MESOS-1187) precision errors with allocation calculations

2015-06-24 Thread Jie Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu reopened MESOS-1187:
---

I was able to reproduce.

0.1 + 0.1 + 0.1 - 0.1 - 0.1 = 0.09892  0.1

That would cause issues if the code relies on the fact that

0.1 + 0.1 + 0.1 - 0.1 - 0.1 == 0.1

 precision errors with allocation calculations
 -

 Key: MESOS-1187
 URL: https://issues.apache.org/jira/browse/MESOS-1187
 Project: Mesos
  Issue Type: Bug
  Components: allocation, master
Reporter: aniruddha sathaye
Priority: Minor

 As allocations are stored/transmitted as doubles many a times precision 
 errors creep in. 
 we have seen erroneous share calculations happen only because of floating 
 point arithmetic. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-1187) precision errors with allocation calculations

2015-06-24 Thread Jie Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600257#comment-14600257
 ] 

Jie Yu commented on MESOS-1187:
---

This is a simple test to reproduce:
https://reviews.apache.org/r/35849/

 precision errors with allocation calculations
 -

 Key: MESOS-1187
 URL: https://issues.apache.org/jira/browse/MESOS-1187
 Project: Mesos
  Issue Type: Bug
  Components: allocation, master
Reporter: aniruddha sathaye

 As allocations are stored/transmitted as doubles many a times precision 
 errors creep in. 
 we have seen erroneous share calculations happen only because of floating 
 point arithmetic. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600293#comment-14600293
 ] 

Alex Clemmer commented on MESOS-898:


[~vi...@twitter.com] I would be very, very disappointed in any Apache project 
that did not have community involvement in a decision like this.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code

2015-06-24 Thread Niklas Quarfot Nielsen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niklas Quarfot Nielsen updated MESOS-2637:
--
Shepherd:   (was: Niklas Quarfot Nielsen)

 Consolidate 'foo', 'bar', ... string constants in test and example code
 ---

 Key: MESOS-2637
 URL: https://issues.apache.org/jira/browse/MESOS-2637
 Project: Mesos
  Issue Type: Bug
  Components: technical debt
Reporter: Niklas Quarfot Nielsen
Assignee: Colin Williams

 We are using 'foo', 'bar', ... string constants and pairs in 
 src/tests/master_tests.cpp, src/tests/slave_tests.cpp, 
 src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and 
 hooks tests. These values should be stored in local variables to avoid the 
 possibility of assignment getting out of sync with checking for that same 
 value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-999) Slave should wait() and start executor registration timeout after launch

2015-06-24 Thread Yan Xu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Xu updated MESOS-999:
-
Assignee: (was: Yan Xu)

 Slave should wait() and start executor registration timeout after launch 
 -

 Key: MESOS-999
 URL: https://issues.apache.org/jira/browse/MESOS-999
 Project: Mesos
  Issue Type: Bug
  Components: isolation
Affects Versions: 0.18.0
Reporter: Ian Downes
Priority: Minor

 The current code will start launch a container and wait on it before the 
 launch is complete. We should do this only after the container has 
 successfully launched. Likewise for the executor registration timeout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Vinod Kone (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600279#comment-14600279
 ] 

Vinod Kone commented on MESOS-898:
--

You know that this JIRA is public right :) Also, since this support is being 
added to Apache Mesos, we expect the community to be in the know during 
development of changes like these.

Regarding cross platform support off the bat, I'll trust your instincts on 
this. From what I saw in your commits, windows support didn't seem to 
complicate things. So, it's ok with me.

Regarding CMake idioms, [~tstclair] (Mesos committer), might be able to help 
you out here.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Joris Van Remoortere (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600084#comment-14600084
 ] 

Joris Van Remoortere edited comment on MESOS-2925 at 6/24/15 8:26 PM:
--

Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled 
initializer warning. According to [~tillt] this is something that Apple tends 
to tweak before they provide the stable releases of clang for OSX?

AFAIK the macro expands to either \{false\} or \{0\}. I don't see why it is 
harmful to keep using this to initialize the locks. I find the code in the 
review request harder to read due to the noise. What do you think?


was (Author: jvanremoortere):
Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled 
initializer warning. According to [~tillt] this is something that Apple tends 
to tweak before they provide the stable releases of clang for OSX?

AFAIK the macro expands to either `{false}` or {0}. I don't see why it is 
harmful to keep using this to initialize the locks. I find the code in the 
review request harder to read due to the noise. What do you think?

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Paul Brett (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600104#comment-14600104
 ] 

Paul Brett commented on MESOS-2925:
---

Up for review at https://reviews.apache.org/r/35841/

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2924) Allow simple construction via initializer list on hashset.

2015-06-24 Thread Till Toenshoff (JIRA)
Till Toenshoff created MESOS-2924:
-

 Summary: Allow simple construction via initializer list on hashset.
 Key: MESOS-2924
 URL: https://issues.apache.org/jira/browse/MESOS-2924
 Project: Mesos
  Issue Type: Improvement
Reporter: Till Toenshoff


{{hashmap}} already has a initializer-list constructor, {{hashset}} should 
offer the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2924) Allow simple construction via initializer list on hashset.

2015-06-24 Thread Till Toenshoff (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Till Toenshoff updated MESOS-2924:
--
Priority: Minor  (was: Major)

 Allow simple construction via initializer list on hashset.
 --

 Key: MESOS-2924
 URL: https://issues.apache.org/jira/browse/MESOS-2924
 Project: Mesos
  Issue Type: Improvement
Reporter: Till Toenshoff
Priority: Minor

 {{hashmap}} already has a initializer-list constructor, {{hashset}} should 
 offer the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2923) fetcher.cpp - problem with certificates..?

2015-06-24 Thread Tomasz Mieszkowski (JIRA)
Tomasz Mieszkowski created MESOS-2923:
-

 Summary: fetcher.cpp - problem with certificates..?
 Key: MESOS-2923
 URL: https://issues.apache.org/jira/browse/MESOS-2923
 Project: Mesos
  Issue Type: Bug
Affects Versions: 0.22.1, 0.22.0
 Environment: Ubuntu 14.04 (build + test)
Reporter: Tomasz Mieszkowski


Mesos 0.22.0/0.22.1 built and installed from sources accordingly to the 
instructions given [here|http://mesos.apache.org/gettingstarted/] has some 
problem with certificates.
Every time I try to deploy something that requires downloading any resource via 
HTTPS (with URI specified via Marathon), such deployment fails and I get this 
message in failed app's sandbox:
{code}
E0617 09:58:44.339409 12380 fetcher.cpp:138] Error downloading resource: 
Problem with the SSL CA cert (path? access rights?)
{code}
Trying to download the same resource on the same slave with {{curl}} or 
{{wget}} works without problems.
Moreover, when I install exactly the same version of Mesos from Mesosphere's 
debs on identical machines (i.e., set up by the same Ansible scripts), 
everything works fine as well.
I guess it must be something related to the way how Mesos is built - maybe some 
missing switch for {{configure}} or {{make}}..?

Any ideas..?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo

2015-06-24 Thread Charles Allen (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600410#comment-14600410
 ] 

Charles Allen commented on MESOS-2340:
--

Is rolling back from 0.24 to 0.23 during a failed rolling upgrade or 
immediately after something that is intended to be supported?

 Publish JSON in ZK instead of serialized MasterInfo
 ---

 Key: MESOS-2340
 URL: https://issues.apache.org/jira/browse/MESOS-2340
 Project: Mesos
  Issue Type: Improvement
  Components: leader election
Reporter: Zameer Manji
Assignee: Marco Massenzio

 Currently to discover the master a client needs the ZK node location and 
 access to the MasterInfo protobuf so it can deserialize the binary blob in 
 the node.
 I think it would be nice to publish JSON (like Twitter's ServerSets) so 
 clients are not tied to protobuf to do service discovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2931) Add explanation of master bootstrap process to Operational Guide

2015-06-24 Thread Daniel Nugent (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Nugent updated MESOS-2931:
-
Description: 
When Mesos starts up, masters come up in an empty bootstrap state. End users 
may find that they experience failures without an apparent cause. The 
documentation for the operational guide should lay out the requirements for the 
bootstrap process and its behavior.

See MESOS-2148 for an example of an issue filed when someone wasn't aware of 
the bootstrap process.

  was:
When Mesos starts up, masters come up in an empty bootstrap state. End users 
may find that they experience failures without an apparent cause. The 
documentation for the operational guide should lay out the requirements for the 
bootstrap process and its behavior.

See MESOS-2148 for an example of an issue filed when someone wasn't aware of 
the problem.


 Add explanation of master bootstrap process to Operational Guide
 

 Key: MESOS-2931
 URL: https://issues.apache.org/jira/browse/MESOS-2931
 Project: Mesos
  Issue Type: Documentation
  Components: documentation
Affects Versions: 0.22.1
Reporter: Daniel Nugent
Priority: Minor

 When Mesos starts up, masters come up in an empty bootstrap state. End users 
 may find that they experience failures without an apparent cause. The 
 documentation for the operational guide should lay out the requirements for 
 the bootstrap process and its behavior.
 See MESOS-2148 for an example of an issue filed when someone wasn't aware of 
 the bootstrap process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo

2015-06-24 Thread Marco Massenzio (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600350#comment-14600350
 ] 

Marco Massenzio commented on MESOS-2340:


Update sent to [the user mailing 
list](http://www.mail-archive.com/user@mesos.apache.org/msg03691.html)

 Publish JSON in ZK instead of serialized MasterInfo
 ---

 Key: MESOS-2340
 URL: https://issues.apache.org/jira/browse/MESOS-2340
 Project: Mesos
  Issue Type: Improvement
  Components: leader election
Reporter: Zameer Manji
Assignee: Marco Massenzio

 Currently to discover the master a client needs the ZK node location and 
 access to the MasterInfo protobuf so it can deserialize the binary blob in 
 the node.
 I think it would be nice to publish JSON (like Twitter's ServerSets) so 
 clients are not tied to protobuf to do service discovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2926) Extend mesos-style.py/cpplint.py to check #include files

2015-06-24 Thread Paul Brett (JIRA)
Paul Brett created MESOS-2926:
-

 Summary: Extend mesos-style.py/cpplint.py to check #include files
 Key: MESOS-2926
 URL: https://issues.apache.org/jira/browse/MESOS-2926
 Project: Mesos
  Issue Type: Bug
Reporter: Paul Brett
Assignee: Paul Brett


cpplint.py provides the capability to enforce the style guide requirements for 
#including everything you use and ordering files based on type but it does not 
work for mesos because we do use #include ... for project files where it 
expects #include   

We should update the style checker to support our include usage and then turn 
it on by default in the commit hook.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2832) Enable configuring Mesos with environment variables without having them leak to tasks launched

2015-06-24 Thread Benjamin Hindman (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Hindman updated MESOS-2832:

Story Points: 8

 Enable configuring Mesos with environment variables without having them leak 
 to tasks launched
 --

 Key: MESOS-2832
 URL: https://issues.apache.org/jira/browse/MESOS-2832
 Project: Mesos
  Issue Type: Wish
Reporter: Cody Maloney
Assignee: Benjamin Hindman
Priority: Critical
  Labels: mesosphere
 Fix For: 0.23.0


 Currently if mesos is configured with environment variables (MESOS_MODULES), 
 those show up in every task which is launched unless the executor explicitly 
 cleans them up. 
 If the task being launched happens to be something libprocess / mesos based, 
 this can often prevent the task from starting up (A scheduler has issues 
 loading a module intended for the slave).
 There are also cases where it would be nice to be able to change what the 
 PATH is that tasks launch with (the host may have more in the path than tasks 
 are supposed to / allowed to depend upon).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2824) Support pre-fetching default container image on slave startup

2015-06-24 Thread Yan Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600356#comment-14600356
 ] 

Yan Xu commented on MESOS-2824:
---

We ([~vinodkone], [~idownes] and myself) discussed the pre-fetching strategies 
but it seemed unacceptable to have either 
- the slave to be blocked for extended period of time (minutes) which delays 
the communication between the executor and scheduler, or 
- the first task that uses this image to be blocked for a long time to wait for 
the container image to be ready.

Pre-fetching, either started automatically or upon explicit requests is a good 
idea but we'll probably not do it on slave startup.

 Support pre-fetching default container image on slave startup
 -

 Key: MESOS-2824
 URL: https://issues.apache.org/jira/browse/MESOS-2824
 Project: Mesos
  Issue Type: Improvement
  Components: isolation
Affects Versions: 0.23.0
Reporter: Ian Downes
Assignee: Yan Xu
Priority: Minor
  Labels: twitter

 Default container images can be specified with the --default_container_info 
 flag to the slave. This may be a large image that will take a long time to 
 initially fetch/hash/extract when the first container is provisioned. Add 
 optional support to start fetching the image when the slave starts and 
 consider not registering until the fetch is complete.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2927) Update mesos to #include headers for symbols we rely on

2015-06-24 Thread Paul Brett (JIRA)
Paul Brett created MESOS-2927:
-

 Summary: Update mesos to #include headers for symbols we rely on
 Key: MESOS-2927
 URL: https://issues.apache.org/jira/browse/MESOS-2927
 Project: Mesos
  Issue Type: Bug
Reporter: Paul Brett
Assignee: Paul Brett


Update mesos to #include headers for symbols we rely on



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2929) Update libprocess to #include headers for symbols we rely on

2015-06-24 Thread Paul Brett (JIRA)
Paul Brett created MESOS-2929:
-

 Summary: Update libprocess to #include headers for symbols we rely 
on
 Key: MESOS-2929
 URL: https://issues.apache.org/jira/browse/MESOS-2929
 Project: Mesos
  Issue Type: Bug
Reporter: Paul Brett
Assignee: Paul Brett






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo

2015-06-24 Thread Vinod Kone (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600428#comment-14600428
 ] 

Vinod Kone commented on MESOS-2340:
---

0.23.0 and 0.24.0 components (scheduler, master) should always be able to talk 
to each other.

 Publish JSON in ZK instead of serialized MasterInfo
 ---

 Key: MESOS-2340
 URL: https://issues.apache.org/jira/browse/MESOS-2340
 Project: Mesos
  Issue Type: Improvement
  Components: leader election
Reporter: Zameer Manji
Assignee: Marco Massenzio

 Currently to discover the master a client needs the ZK node location and 
 access to the MasterInfo protobuf so it can deserialize the binary blob in 
 the node.
 I think it would be nice to publish JSON (like Twitter's ServerSets) so 
 clients are not tied to protobuf to do service discovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (MESOS-2832) Enable configuring Mesos with environment variables without having them leak to tasks launched

2015-06-24 Thread Benjamin Hindman (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Hindman resolved MESOS-2832.
-
   Resolution: Fixed
Fix Version/s: 0.23.0

commit 46dc9979e9cc38d36dc7300db13af39bdfbfd52e
Author: Benjamin Hindman benjamin.hind...@gmail.com
Date:   Mon Jun 15 00:29:20 2015 -0700

Added 'executor_environment_variables' flag to slave.

This new flag, 'executor_environment_variables', let's an operator
specify the environment variables that should be passed to an
executor, and thus, any of it's tasks. By default, an executor will
inherit the environment variables of the slave.

Review: https://reviews.apache.org/r/35567

 Enable configuring Mesos with environment variables without having them leak 
 to tasks launched
 --

 Key: MESOS-2832
 URL: https://issues.apache.org/jira/browse/MESOS-2832
 Project: Mesos
  Issue Type: Wish
Reporter: Cody Maloney
Assignee: Benjamin Hindman
Priority: Critical
  Labels: mesosphere
 Fix For: 0.23.0


 Currently if mesos is configured with environment variables (MESOS_MODULES), 
 those show up in every task which is launched unless the executor explicitly 
 cleans them up. 
 If the task being launched happens to be something libprocess / mesos based, 
 this can often prevent the task from starting up (A scheduler has issues 
 loading a module intended for the slave).
 There are also cases where it would be nice to be able to change what the 
 PATH is that tasks launch with (the host may have more in the path than tasks 
 are supposed to / allowed to depend upon).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2931) Add explanation of master bootstrap process to Operational Guide

2015-06-24 Thread Daniel Nugent (JIRA)
Daniel Nugent created MESOS-2931:


 Summary: Add explanation of master bootstrap process to 
Operational Guide
 Key: MESOS-2931
 URL: https://issues.apache.org/jira/browse/MESOS-2931
 Project: Mesos
  Issue Type: Documentation
  Components: documentation
Affects Versions: 0.22.1
Reporter: Daniel Nugent
Priority: Minor


When Mesos starts up, masters come up in an empty bootstrap state. End users 
may find that they experience failures without an apparent cause. The 
documentation for the operational guide should lay out the requirements for the 
bootstrap process and its behavior.

See MESOS-2148 for an example of an issue filed when someone wasn't aware of 
the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2930) Allow the Resource Estimator to express over-allocation of revocable resources.

2015-06-24 Thread Benjamin Mahler (JIRA)
Benjamin Mahler created MESOS-2930:
--

 Summary: Allow the Resource Estimator to express over-allocation 
of revocable resources.
 Key: MESOS-2930
 URL: https://issues.apache.org/jira/browse/MESOS-2930
 Project: Mesos
  Issue Type: Improvement
  Components: slave
Reporter: Benjamin Mahler


Currently the resource estimator returns the amount of oversubscription 
resources that are available, since resources cannot be negative, this allows 
the resource estimator to express the following:

(1) Return empty resources: We are fully allocated for oversubscription 
resources.

(2) Return non-empty resources: We are under-allocated for oversubscription 
resources. In other words, some are available.

However, there is an additional situation that we cannot express:

(3) Analogous to returning non-empty negative resources: We are 
over-allocated for oversubscription resources. Do not re-offer any of the 
over-allocated oversubscription resources that are recovered.

Without (3), the slave can only shrink the total pool of oversubscription 
resources by returning (1) as resources are recovered, until the pool is shrunk 
to the desired size. However, this approach is only best-effort, it's possible 
for a framework to launch more tasks in the window of time (15 seconds by 
default) that the slave polls the estimator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-2928) Update stout to #include headers for symbols we rely on

2015-06-24 Thread Paul Brett (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600455#comment-14600455
 ] 

Paul Brett edited comment on MESOS-2928 at 6/25/15 12:58 AM:
-

https://reviews.apache.org/r/35861/


was (Author: pbrett):
https://reviews.apache.org/r/35860/

 Update stout to #include headers for symbols we rely on
 ---

 Key: MESOS-2928
 URL: https://issues.apache.org/jira/browse/MESOS-2928
 Project: Mesos
  Issue Type: Bug
Reporter: Paul Brett
Assignee: Paul Brett

 Update mesos to #include headers for symbols we rely on



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2928) Update stout to #include headers for symbols we rely on

2015-06-24 Thread Paul Brett (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600455#comment-14600455
 ] 

Paul Brett commented on MESOS-2928:
---

https://reviews.apache.org/r/35860/

 Update stout to #include headers for symbols we rely on
 ---

 Key: MESOS-2928
 URL: https://issues.apache.org/jira/browse/MESOS-2928
 Project: Mesos
  Issue Type: Bug
Reporter: Paul Brett
Assignee: Paul Brett

 Update mesos to #include headers for symbols we rely on



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Michael Park (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600512#comment-14600512
 ] 

Michael Park commented on MESOS-2925:
-

[~pbrett] I gave it a {{ShipIt}} on the review but how about we use the inline 
member initializer syntax instead?

{code}
class Obj
{
public:

  std::atomic_flag lock = ATOMIC_FLAG_INIT;
};
{code}

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2664) Modernize the codebase to C++11

2015-06-24 Thread Michael Park (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599623#comment-14599623
 ] 

Michael Park commented on MESOS-2664:
-

[~bmahler] I think {{-Wswitch}} _almost_ captures what we want.

Yes, to both (a) and (b). The missing part for {{-Wswitch}} for us is: (c) If 
we covered all the cases *and* have a {{default}}, it would be nice for that to 
also be a warning. When we add a new enum case, I think what we want is a 
warning saying you need to handle this new enum value here! rather than 
silently going into the {{default}} case. An example from our codebase:

{code}
switch (volume.mode()) {
  case Volume::RW: volumeConfig += :rw; break;
  case Volume::RO: volumeConfig += :ro; break;
  default:
LOG(FATAL)  Unknown Volume mode:   volume.mode();
break;
}
{code}

I think we can capture what we want with {{-Wswitch}} + a style guide around 
the use {{default}}. What do you think?

P.S. We currently have {{-Wall}} turned on, which includes {{-Wswitch}} :)

 Modernize the codebase to C++11
 ---

 Key: MESOS-2664
 URL: https://issues.apache.org/jira/browse/MESOS-2664
 Project: Mesos
  Issue Type: Epic
  Components: technical debt
Reporter: Michael Park
Assignee: Michael Park
  Labels: mesosphere

 Since [this 
 commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7],
  we officially require GCC-4.8+ and Clang-3.5+. This means that we now have 
 full C++11 support and therefore can start to modernize our codebase to be 
 more readable, safer and efficient!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2545) Developer guide for libprocess

2015-06-24 Thread Bernd Mathiske (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599613#comment-14599613
 ] 

Bernd Mathiske commented on MESOS-2545:
---

commit 98f85e1f81b86502013fdf1783f9d8ec65eae3d0
Author: Joerg Schad jo...@mesosphere.io
Date:   Wed Jun 24 17:47:30 2015 +0200

Remove (almost all) html from libprocess Developer Guide.

Review: https://reviews.apache.org/r/35568

 Developer guide for libprocess
 --

 Key: MESOS-2545
 URL: https://issues.apache.org/jira/browse/MESOS-2545
 Project: Mesos
  Issue Type: Documentation
  Components: libprocess
Reporter: Bernd Mathiske
Assignee: Joerg Schad
  Labels: documentation, mesosphere

 Create a developer guide for libprocess that explains the philosophy behind 
 it and explains the most important features as well as the prevalent use 
 patterns in Mesos with examples. 
 This could be similar to stout/README.md.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2932) There is a typo in docs/docker-containerizer.md file

2015-06-24 Thread Chen Zhiwei (JIRA)
Chen Zhiwei created MESOS-2932:
--

 Summary: There is a typo in docs/docker-containerizer.md file
 Key: MESOS-2932
 URL: https://issues.apache.org/jira/browse/MESOS-2932
 Project: Mesos
  Issue Type: Bug
  Components: documentation
Affects Versions: 0.22.1
Reporter: Chen Zhiwei
Priority: Minor


A docker image currently supports having a entrypoint and/or a default command.

The `a entrypoint` should be `an entrypoint`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706
 ] 

Alex Clemmer commented on MESOS-898:


I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:39 PM:
-

I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for screwing up the posting of this the first time. I don't 
interact with JIRA much.


was (Author: hausdorff):
I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a 

[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:47 PM:
-

I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.
* This commit spans Mesos, libprocess, and stout (rather than separating them 
out) mainly to make it easy to consume -- for the real solution, I will 
happily split them out appropriately.
* So far, this prototype commit follows [~haosd...@gmail.com]'s supposition 
that we want to build the CMake system incrementally and in parallel (though I 
want to note that stout is a header-only library, and does not need to be 
built).
* I agree that with [~tstclair] that it is probably best that we coalesce 
libprocess and stout into one sort of systems layer, but given the progress 
here so far, I'd like to postpone that discussion for another time. I'm willing 
to rewrite this part of the build system in the event that those changes 
materialize.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for double posting this. I don't interact with JIRA much.


was (Author: hausdorff):
I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.
* This commit spans Mesos, libprocess, and stout (rather than separating them 
out) mainly to make it easy to consume -- for the real solution, I will 
happily split them out appropriately.
* So far, this prototype commit follows [~haosd...@gmail.com]'s supposition 
that we want to build the CMake system incrementally and in parallel (though I 
want to note that stout is a header-only library, and does not need to be 
built).

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for double posting this. I don't interact with JIRA much.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  

[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599705#comment-14599705
 ] 

Alex Clemmer commented on MESOS-898:


I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:40 PM:
-

I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for double posting this. I don't interact with JIRA much.


was (Author: hausdorff):
I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for screwing up the posting of this the first time. I don't 
interact with JIRA much.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've 

[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:43 PM:
-

I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.
* This commit spans Mesos, libprocess, and stout (rather than separating them 
out) mainly to make it easy to consume -- for the real solution, I will 
happily split them out appropriately.
* So far, this prototype commit follows [~haosd...@gmail.com]'s supposition 
that we want to build the CMake system incrementally and in parallel (though I 
want to note that stout is a header-only library, and does not need to be 
built).

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for double posting this. I don't interact with JIRA much.


was (Author: hausdorff):
I'd like to start this conversation with (what I think is) the simplest step 
forward.

I've put together a prototype CMake-based build system that covers libprocess 
and the third-party libraries, which successfully builds on several flavors of 
Linux. It is the [last commit of this 
branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm 
hoping that this will help focus the discussion around concrete things that I 
need to fix or do less stupidly, to accomplish our goals in this project.

Some notes:

* I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something 
silly, it's because I am!
* It's pretty well-commented. I'm hoping it's not too hard to navigate.
* It is build to be x-plat at the outset. So, you should be able to take this 
file and generate makefiles or, like, nmake files.

Some things on the roadmap if you all like the progress so far:

[x] Third-party dependencies are downloaded, configured, and built when you run 
`make` (or whatever).
[ ] Support for system installations of the third-party dependencies (i.e., we 
should allow users to use glog if it's already installed on their machine)
[x] Tests build and run on multiple flavors of Linux
[ ] Benchmarks build and run on multiple flavors of Linux

n.b., for issue 2 I have not decided what the right way of communicating the 
system dependencies to CMake is. In the autoconf solution this comes from 
`configure`, but CMake will not have a configure step because it needs to run 
not only on POSIX machines.


EDIT: sorry, folks, for double posting this. I don't interact with JIRA much.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where 

[jira] [Assigned] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Paul Brett (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Brett reassigned MESOS-2925:
-

Assignee: Paul Brett

 Invalid usage of ATOMIC_FLAG_INIT in member initialization
 --

 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett
Assignee: Paul Brett

 The C++ specification states:
 The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
 to initialize an object of type atomic_flag to the clear state. The macro can 
 be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is 
 unspecified whether the macro can be used in other initialization contexts. 
 Clang catches this (although reports it erroneously as a braced scaled init 
 issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2855) Update operational guide to include growing from standalone to high availability

2015-06-24 Thread Michael Schenck (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599603#comment-14599603
 ] 

Michael Schenck commented on MESOS-2855:


https://reviews.apache.org/r/35361/

 Update operational guide to include growing from standalone to high 
 availability
 

 Key: MESOS-2855
 URL: https://issues.apache.org/jira/browse/MESOS-2855
 Project: Mesos
  Issue Type: Documentation
Reporter: Michael Schenck
Assignee: Michael Schenck
  Labels: documentation

 The [Operational 
 Guide|http://mesos.apache.org/documentation/latest/operational-guide/] covers 
 increasing quorum size from {{--quorum=2}}, but does not cover how to move 
 from a _standalone_ master to a high availability configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599866#comment-14599866
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 6:13 PM:
-

Only a few people have given a quick once-over look to the prototype 
[CMake-based build system for 
libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified],
 but they have been pretty positive, so I would like to target EOD tomorrow 
(i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to 
let the commit hang too long, but I am also new to the community, and I want to 
give people a chance to build consensus that this commit is at least 
directionally correct. :)


was (Author: hausdorff):
Only a few people have given a quick once-over look to the prototype 
[CMake-based build system for 
libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified],
 but they have been pretty positive, so I would like to target EOD tomorrow 
(i.e., Th, June 24th) for posting something to ReviewBoard. This slight delay 
is mainly because I am new to the community, and I want to give people a chance 
to build consensus that this commit is at least directionally correct. :)

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code

2015-06-24 Thread Niklas Quarfot Nielsen (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599886#comment-14599886
 ] 

Niklas Quarfot Nielsen commented on MESOS-2637:
---

Hi [~lackita] - Did you get to look at this?

I will be out for the next two weeks from tomorrow.
If you are planning on getting reviews/feedback in the mean time - can you ping 
one of the other committers?

Thanks!
Niklas

 Consolidate 'foo', 'bar', ... string constants in test and example code
 ---

 Key: MESOS-2637
 URL: https://issues.apache.org/jira/browse/MESOS-2637
 Project: Mesos
  Issue Type: Bug
  Components: technical debt
Reporter: Niklas Quarfot Nielsen
Assignee: Colin Williams

 We are using 'foo', 'bar', ... string constants and pairs in 
 src/tests/master_tests.cpp, src/tests/slave_tests.cpp, 
 src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and 
 hooks tests. These values should be stored in local variables to avoid the 
 possibility of assignment getting out of sync with checking for that same 
 value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2919) Framework can overcommit oversubscribable resources during master failover.

2015-06-24 Thread Jie Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599902#comment-14599902
 ] 

Jie Yu commented on MESOS-2919:
---

https://reviews.apache.org/r/35836/

 Framework can overcommit oversubscribable resources during master failover.
 ---

 Key: MESOS-2919
 URL: https://issues.apache.org/jira/browse/MESOS-2919
 Project: Mesos
  Issue Type: Bug
Reporter: Jie Yu
Assignee: Jie Yu
Priority: Critical
  Labels: twitter

 This is due to a bug in the hierarchical allocator. Here is the sequence of 
 events:
 1) slave uses a fixed resource estimator which advertise 4 revocable cpus
 2) a framework A launches a task that uses all the 4 revocable cpus
 3) master fails over
 4) slave re-registers with the new master, and sends UpdateSlaveMessage with 
 4 revocable cpus as oversubscribed resources
 5) framework A hasn't registered yet, therefore, the slave's available 
 resources will be 4 revocable cpus
 6) framework A registered and will receive an additional 4 revocable cpus. So 
 it can launch another task with 4 revocable cpus (that means 8 total!)
 The problem is due to the way we calculate 'allocated' resource in allocator 
 when 'updateSlave'. If the framework is not registered, the 'allocation' 
 below is not accurate (check that if block in 'addSlave').
 {code}
 template class RoleSorter, class FrameworkSorter
 void
 HierarchicalAllocatorProcessRoleSorter, FrameworkSorter::updateSlave(
 const SlaveID slaveId,
 const Resources oversubscribed)
 {
   CHECK(initialized);
   CHECK(slaves.contains(slaveId));
   // Check that all the oversubscribed resources are revocable.
   CHECK_EQ(oversubscribed, oversubscribed.revocable());
   // Update the total resources.
   // First remove the old oversubscribed resources from the total.
   slaves[slaveId].total -= slaves[slaveId].total.revocable();
   // Now add the new estimate of oversubscribed resources.
   slaves[slaveId].total += oversubscribed;
   // Now, update the total resources in the role sorter.
   roleSorter-update(
   slaveId,
   slaves[slaveId].total.unreserved());
   // Calculate the current allocation of oversubscribed resources.
   Resources allocation;
   foreachkey (const std::string role, roles) {
 allocation += roleSorter-allocation(role, slaveId).revocable();
   }
   // Update the available resources.
   // First remove the old oversubscribed resources from available.
   slaves[slaveId].available -= slaves[slaveId].available.revocable();
   // Now add the new estimate of available oversubscribed resources.
   slaves[slaveId].available += oversubscribed - allocation;
   LOG(INFO)  Slave   slaveId   (  slaves[slaveId].hostname
  ) updated with oversubscribed resources   oversubscribed
   (total:   slaves[slaveId].total
  , available:   slaves[slaveId].available  );
   allocate(slaveId);
 }
 template class RoleSorter, class FrameworkSorter
 void
 HierarchicalAllocatorProcessRoleSorter, FrameworkSorter::addSlave(
 const SlaveID slaveId,
 const SlaveInfo slaveInfo,
 const Resources total,
 const hashmapFrameworkID, Resources used)
 {
   CHECK(initialized);
   CHECK(!slaves.contains(slaveId));
   roleSorter-add(slaveId, total.unreserved());
   foreachpair (const FrameworkID frameworkId,
const Resources allocated,
used) {
 if (frameworks.contains(frameworkId)) {
   const std::string role = frameworks[frameworkId].role;
   // TODO(bmahler): Validate that the reserved resources have the
   // framework's role.
   roleSorter-allocated(role, slaveId, allocated.unreserved());
   frameworkSorters[role]-add(slaveId, allocated);
   frameworkSorters[role]-allocated(
   frameworkId.value(), slaveId, allocated);
 }
   }
   ...
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599866#comment-14599866
 ] 

Alex Clemmer commented on MESOS-898:


Only a few people have given a quick once-over look to the prototype 
[CMake-based build system for 
libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified],
 but they have been pretty positive, so I would like to target EOD tomorrow 
(i.e., Th, June 24th) for posting something to ReviewBoard. This slight delay 
is mainly because I am new to the community, and I want to give people a chance 
to build consensus that this commit is at least directionally correct. :)

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Alex Clemmer (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599866#comment-14599866
 ] 

Alex Clemmer edited comment on MESOS-898 at 6/24/15 6:17 PM:
-

Only a few people have given a quick once-over look to the [prototype 
CMake-based build system for 
libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified],
 but they have been pretty positive, so I would like to target EOD tomorrow 
(i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to 
let the commit hang too long, but I am also new to the community, and I want to 
give people a chance to build consensus that this commit is at least 
directionally correct. :)


was (Author: hausdorff):
Only a few people have given a quick once-over look to the prototype 
[CMake-based build system for 
libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified],
 but they have been pretty positive, so I would like to target EOD tomorrow 
(i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to 
let the commit hang too long, but I am also new to the community, and I want to 
give people a chance to build consensus that this commit is at least 
directionally correct. :)

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2919) Framework can overcommit oversubscribable resources during master failover.

2015-06-24 Thread Jie Yu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jie Yu updated MESOS-2919:
--
Story Points: 3

 Framework can overcommit oversubscribable resources during master failover.
 ---

 Key: MESOS-2919
 URL: https://issues.apache.org/jira/browse/MESOS-2919
 Project: Mesos
  Issue Type: Bug
Reporter: Jie Yu
Assignee: Jie Yu
Priority: Critical
  Labels: twitter

 This is due to a bug in the hierarchical allocator. Here is the sequence of 
 events:
 1) slave uses a fixed resource estimator which advertise 4 revocable cpus
 2) a framework A launches a task that uses all the 4 revocable cpus
 3) master fails over
 4) slave re-registers with the new master, and sends UpdateSlaveMessage with 
 4 revocable cpus as oversubscribed resources
 5) framework A hasn't registered yet, therefore, the slave's available 
 resources will be 4 revocable cpus
 6) framework A registered and will receive an additional 4 revocable cpus. So 
 it can launch another task with 4 revocable cpus (that means 8 total!)
 The problem is due to the way we calculate 'allocated' resource in allocator 
 when 'updateSlave'. If the framework is not registered, the 'allocation' 
 below is not accurate (check that if block in 'addSlave').
 {code}
 template class RoleSorter, class FrameworkSorter
 void
 HierarchicalAllocatorProcessRoleSorter, FrameworkSorter::updateSlave(
 const SlaveID slaveId,
 const Resources oversubscribed)
 {
   CHECK(initialized);
   CHECK(slaves.contains(slaveId));
   // Check that all the oversubscribed resources are revocable.
   CHECK_EQ(oversubscribed, oversubscribed.revocable());
   // Update the total resources.
   // First remove the old oversubscribed resources from the total.
   slaves[slaveId].total -= slaves[slaveId].total.revocable();
   // Now add the new estimate of oversubscribed resources.
   slaves[slaveId].total += oversubscribed;
   // Now, update the total resources in the role sorter.
   roleSorter-update(
   slaveId,
   slaves[slaveId].total.unreserved());
   // Calculate the current allocation of oversubscribed resources.
   Resources allocation;
   foreachkey (const std::string role, roles) {
 allocation += roleSorter-allocation(role, slaveId).revocable();
   }
   // Update the available resources.
   // First remove the old oversubscribed resources from available.
   slaves[slaveId].available -= slaves[slaveId].available.revocable();
   // Now add the new estimate of available oversubscribed resources.
   slaves[slaveId].available += oversubscribed - allocation;
   LOG(INFO)  Slave   slaveId   (  slaves[slaveId].hostname
  ) updated with oversubscribed resources   oversubscribed
   (total:   slaves[slaveId].total
  , available:   slaves[slaveId].available  );
   allocate(slaveId);
 }
 template class RoleSorter, class FrameworkSorter
 void
 HierarchicalAllocatorProcessRoleSorter, FrameworkSorter::addSlave(
 const SlaveID slaveId,
 const SlaveInfo slaveInfo,
 const Resources total,
 const hashmapFrameworkID, Resources used)
 {
   CHECK(initialized);
   CHECK(!slaves.contains(slaveId));
   roleSorter-add(slaveId, total.unreserved());
   foreachpair (const FrameworkID frameworkId,
const Resources allocated,
used) {
 if (frameworks.contains(frameworkId)) {
   const std::string role = frameworks[frameworkId].role;
   // TODO(bmahler): Validate that the reserved resources have the
   // framework's role.
   roleSorter-allocated(role, slaveId, allocated.unreserved());
   frameworkSorters[role]-add(slaveId, allocated);
   frameworkSorters[role]-allocated(
   frameworkId.value(), slaveId, allocated);
 }
   }
   ...
 }
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2795) Introduce filesystem provisioner abstraction to Mesos containerizer

2015-06-24 Thread Ian Downes (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599834#comment-14599834
 ] 

Ian Downes commented on MESOS-2795:
---

ContainerImage protobuf: 
https://reviews.apache.org/r/34136

Add support for container image provisioners. This includes the Provisioner 
interface and support for provisioners in the MesosContainerizer, including 
checkpointing and recovery of the optional container rootfs.
https://reviews.apache.org/r/34137



 Introduce filesystem provisioner abstraction to Mesos containerizer
 ---

 Key: MESOS-2795
 URL: https://issues.apache.org/jira/browse/MESOS-2795
 Project: Mesos
  Issue Type: Improvement
  Components: isolation
Affects Versions: 0.22.1
Reporter: Ian Downes
Assignee: Ian Downes
  Labels: twitter

 Optional filesystem provisioner component for the Mesos containerizer that 
 can provision per-container filesystems.
 This is different to a filesystem isolators because it just provisions a root 
 filesystem for a container and doesn't actually do any isolation (e.g., 
 through a mount namespace + pivot or chroot).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization

2015-06-24 Thread Paul Brett (JIRA)
Paul Brett created MESOS-2925:
-

 Summary: Invalid usage of ATOMIC_FLAG_INIT in member initialization
 Key: MESOS-2925
 URL: https://issues.apache.org/jira/browse/MESOS-2925
 Project: Mesos
  Issue Type: Bug
  Components: libprocess
Affects Versions: 0.23.0
Reporter: Paul Brett


The C++ specification states:

The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used 
to initialize an object of type atomic_flag to the clear state. The macro can 
be used in the form: atomic_flag guard = ATOMIC_FLAG_INIT; It is unspecified 
whether the macro can be used in other initialization contexts. 

Clang catches this (although reports it erroneously as a braced scaled init 
issue) and refuses to compile libprocess.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.

2015-06-24 Thread Yan Xu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599955#comment-14599955
 ] 

Yan Xu commented on MESOS-898:
--

{quote}
-- Looks like you are pulling in the whole of boost? While it's not clear, we 
bundle the boost ourselves. It basically includes only the boost headers that 
we need. There should be a README somewhere that explains the process.
{quote}
Yeah it's super unclear. The README is hidden within the tarball. 
https://github.com/apache/mesos/blob/master/3rdparty/libprocess/3rdparty/boost-1.53.0.tar.gz
 
We should move it out.

 Introduce CMake as an alternative build system.
 ---

 Key: MESOS-898
 URL: https://issues.apache.org/jira/browse/MESOS-898
 Project: Mesos
  Issue Type: Epic
  Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
  Labels: build

 This is a rather substantial undertaking, so I would want upstream 
 debate+buy-in prior to full commitment.  The basic premise is: upstream 
 rebundles several of its dependencies in part to tightly control its stack.  
 This is not out of the norm, but in order to be picked up by distribution 
 channels it needs to built against system dependencies, and rebundling is 
 strictly forbidden.  Given that the mesos primary target platform are 
 data-center distributions such as RHEL/CENTOS/SL it makes sense to still have 
 bundling support for those who do not have dependencies in their channels 
 yet.  This is where cmake can be win with it's uber macros 
 (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).  
 I do not know of any equivalent in the autotools world, other then to brew 
 your own solution.   I've done this type of work in the past, and completely 
 transformed condor and would leverage a lot of the work that was done there. 
 I currently have a tracking branch where I've started this work, but before I 
 go off into the woods, it makes sense to have a debate in public. 
 The primary benefits are: 
 1. Enable downstream channels to easily distro without carrying a large patch 
 sets. 
 2. Still support existing non-proper distribution methods. 
 3. Harden / future proof dependent interfaces. 
 Side Benefits: 
 Audit current build mechanics.  
  - Presently the language specific binding are not installed.  (.py  .jar)
  - make -jX currently fails 
  - optionally look in arm support. 
 Costs:
 1. Time
 2. Potential temporary destabilization
 3. Infrastructure around build+test may need to change.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-2551) C++ Scheduler library should send Call messages to Master

2015-06-24 Thread Marco Massenzio (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599969#comment-14599969
 ] 

Marco Massenzio commented on MESOS-2551:


Thanks, [~vi...@twitter.com] for taking this up.

Just for historical records purposes, [this 
review](https://reviews.apache.org/r/35806/) was originally posted.

[~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on 
progress: 0.23RC is coming up fast and we'd like to have this in too, if we 
possibly can. 

 C++ Scheduler library should send Call messages to Master
 -

 Key: MESOS-2551
 URL: https://issues.apache.org/jira/browse/MESOS-2551
 Project: Mesos
  Issue Type: Story
Reporter: Vinod Kone
Assignee: Vinod Kone
  Labels: tech-debt

 Currently, the C++ library sends different messages to Master instead of a 
 single Call message. To vet the new Call API it should send Call messages. 
 Master should be updated to handle all types of Calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (MESOS-2621) Create documentation for observability metrics

2015-06-24 Thread Vinod Kone (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kone resolved MESOS-2621.
---
   Resolution: Fixed
 Assignee: Ricardo Cervera-Navarro
Fix Version/s: 0.23.0

commit f16d73852623ee05cc13d2757115f7815e608964
Author: Ricardo Cervera-Navarro rcerveranava...@twopensource.com
Date:   Wed Jun 24 12:01:39 2015 -0700

Added documentation on monitoring metrics and alerts.

Review: https://reviews.apache.org/r/33241


 Create documentation for observability metrics
 --

 Key: MESOS-2621
 URL: https://issues.apache.org/jira/browse/MESOS-2621
 Project: Mesos
  Issue Type: Documentation
Reporter: Ricardo Cervera-Navarro
Assignee: Ricardo Cervera-Navarro
Priority: Minor
 Fix For: 0.23.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-2551) C++ Scheduler library should send Call messages to Master

2015-06-24 Thread Marco Massenzio (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marco Massenzio updated MESOS-2551:
---
Labels: tech-debt twitter  (was: tech-debt)

 C++ Scheduler library should send Call messages to Master
 -

 Key: MESOS-2551
 URL: https://issues.apache.org/jira/browse/MESOS-2551
 Project: Mesos
  Issue Type: Story
Reporter: Vinod Kone
Assignee: Vinod Kone
  Labels: tech-debt, twitter

 Currently, the C++ library sends different messages to Master instead of a 
 single Call message. To vet the new Call API it should send Call messages. 
 Master should be updated to handle all types of Calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (MESOS-2551) C++ Scheduler library should send Call messages to Master

2015-06-24 Thread Marco Massenzio (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599969#comment-14599969
 ] 

Marco Massenzio edited comment on MESOS-2551 at 6/24/15 7:08 PM:
-

Thanks, [~vi...@twitter.com] for taking this up.

I've marked it in progress, as I'm assuming you're working on it?

[~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on 
progress: 0.23RC is coming up fast and we'd like to have this in too, if we 
possibly can. 

(Just for historical records purposes, [this 
review](https://reviews.apache.org/r/35806/) was originally posted by 
[~ijimenez])


was (Author: marco-mesos):
Thanks, [~vi...@twitter.com] for taking this up.

Just for historical records purposes, [this 
review](https://reviews.apache.org/r/35806/) was originally posted.

[~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on 
progress: 0.23RC is coming up fast and we'd like to have this in too, if we 
possibly can. 

 C++ Scheduler library should send Call messages to Master
 -

 Key: MESOS-2551
 URL: https://issues.apache.org/jira/browse/MESOS-2551
 Project: Mesos
  Issue Type: Story
Reporter: Vinod Kone
Assignee: Vinod Kone
  Labels: tech-debt, twitter

 Currently, the C++ library sends different messages to Master instead of a 
 single Call message. To vet the new Call API it should send Call messages. 
 Master should be updated to handle all types of Calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)