[jira] [Commented] (MESOS-6904) Track resource allocation candidates and batch allocation work

2017-01-10 Thread Jacob Janco (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817026#comment-15817026
 ] 

Jacob Janco commented on MESOS-6904:


Reviews currently in progress: 
https://reviews.apache.org/r/51027/
https://reviews.apache.org/r/51028/
https://reviews.apache.org/r/52534/
WIP from [~gyliu]
https://reviews.apache.org/r/51621/

> Track resource allocation candidates and batch allocation work
> --
>
> Key: MESOS-6904
> URL: https://issues.apache.org/jira/browse/MESOS-6904
> Project: Mesos
>  Issue Type: Bug
>  Components: allocation
>Reporter: Jacob Janco
>Assignee: Jacob Janco
>  Labels: allocator
>
> "Our deployment environments have a lot of churn, with many short-live 
> frameworks that often revive offers. Running the allocator takes a long time 
> (from seconds up to minutes).
> In this situation, event-triggered allocation causes the event queue in the 
> allocator process to get very long, and the allocator effectively becomes 
> unresponsive (eg. a revive offers message takes too long to come to the head 
> of the queue)." - MESOS-3157 
> To remedy the above scenario, it is proposed to track allocation candidates 
> and only dispatch allocation work if there is no pending allocation in the 
> allocator queue. When an enqueued allocation is processed, the tracked set of 
> candidates is cleared. 
> Current behavior will trigger allocation work on cluster events (e.g. 
> `addSlave()`, `addFramework()`, etc) as well as during the periodic batched 
> allocation running at a defined time interval. 
> This ticket tracks the new direction the work has taken since discussion in 
> MESOS-3157 where a previous solution by [~jamespeach] introduced batched 
> allocation only (which we currently run) as well as an approach to reduce 
> redundancy of work in the queue. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-6904) Track resource allocation candidates and batch allocation work

2017-01-10 Thread Jacob Janco (JIRA)
Jacob Janco created MESOS-6904:
--

 Summary: Track resource allocation candidates and batch allocation 
work
 Key: MESOS-6904
 URL: https://issues.apache.org/jira/browse/MESOS-6904
 Project: Mesos
  Issue Type: Bug
  Components: allocation
Reporter: Jacob Janco
Assignee: Jacob Janco


"Our deployment environments have a lot of churn, with many short-live 
frameworks that often revive offers. Running the allocator takes a long time 
(from seconds up to minutes).
In this situation, event-triggered allocation causes the event queue in the 
allocator process to get very long, and the allocator effectively becomes 
unresponsive (eg. a revive offers message takes too long to come to the head of 
the queue)." - MESOS-3157 

To remedy the above scenario, it is proposed to track allocation candidates and 
only dispatch allocation work if there is no pending allocation in the 
allocator queue. When an enqueued allocation is processed, the tracked set of 
candidates is cleared. 

Current behavior will trigger allocation work on cluster events (e.g. 
`addSlave()`, `addFramework()`, etc) as well as during the periodic batched 
allocation running at a defined time interval. 

This ticket tracks the new direction the work has taken since discussion in 
MESOS-3157 where a previous solution by [~jamespeach] introduced batched 
allocation only (which we currently run) as well as an approach to reduce 
redundancy of work in the queue. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6824) mesos-this-capture clang-tidy check has false positives

2017-01-10 Thread Michael Park (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817005#comment-15817005
 ] 

Michael Park commented on MESOS-6824:
-

Committed here: 
https://github.com/mesos/clang-tools-extra/commit/d00b4b380cece7733bf6bf42935ea2a5b419

> mesos-this-capture clang-tidy check has false positives
> ---
>
> Key: MESOS-6824
> URL: https://issues.apache.org/jira/browse/MESOS-6824
> Project: Mesos
>  Issue Type: Bug
>Reporter: Benjamin Bannier
>Assignee: Benjamin Bannier
>  Labels: clang-tidy
>
> The {{mesos-this-capture}} clang-tidy checks incorrectly triggers on the code 
> here,
>   
> https://github.com/apache/mesos/blob/d2117362349ab4c383045720f77d42b2d9fd6871/src/slave/containerizer/mesos/io/switchboard.cpp#L1487
> We should tighten the matcher to avoid triggering on such constructs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread Kevin Klues (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816946#comment-15816946
 ] 

Kevin Klues commented on MESOS-6901:


Great. If [~frankscholten] can confirm the issue is resolved, I will close it.

> Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04
> 
>
> Key: MESOS-6901
> URL: https://issues.apache.org/jira/browse/MESOS-6901
> Project: Mesos
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.1.0
>Reporter: Frank Scholten
>Assignee: Kevin Klues
>
> When building with g++ 4.8.5 the configure script fails:
> {code}
> checking whether we can build usable Python eggs... cc1plus: warning: command 
> line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ 
> [enabled by default]
> g++: error: unrecognized command line option '-Wdate-time'
> g++: error: unrecognized command line option '-fstack-protector-strong'
> error: command 'g++' failed with exit status 1
> configure: error: no
> ---
> It appears we were unable to build a usable native Python egg.
> There are two possible workarounds for this issue:
> 1. Disable python bindings by configuring with --disable-python.
> 2. Use an alternative Python installation that was built using
>the same build setup as Mesos.
> ---
> {code}
> When building with g++ 4.9.4 the build fails with a segmentation fault:
> {code}
> mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
> slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
> /bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
> -DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
> -DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
> -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
> -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
> -DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
> -DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
> -I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
> -D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
> -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
> -I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
> -I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
> -I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
> -pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong 
> -fPIC -fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized 
> -std=c++11 -MT 
> slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP 
> -MF 
> slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
> -o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test 
> -f 'slave/containerizer/mesos/io/switchboard.cpp' || echo 
> '../../src/'`slave/containerizer/mesos/io/switchboard.cpp
> libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
> -DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
> -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
> -DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
> -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
> 

[jira] [Commented] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread haosdent (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816926#comment-15816926
 ] 

haosdent commented on MESOS-6901:
-

[~klueska] We have discussions in slack before. This should be caused by 
[~frankscholten] didn't perform {{./bootstrap}} again after he change compiler. 
[~frankscholten] May you mind close this ticket if it works for you now?

> Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04
> 
>
> Key: MESOS-6901
> URL: https://issues.apache.org/jira/browse/MESOS-6901
> Project: Mesos
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.1.0
>Reporter: Frank Scholten
>Assignee: Kevin Klues
>
> When building with g++ 4.8.5 the configure script fails:
> {code}
> checking whether we can build usable Python eggs... cc1plus: warning: command 
> line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ 
> [enabled by default]
> g++: error: unrecognized command line option '-Wdate-time'
> g++: error: unrecognized command line option '-fstack-protector-strong'
> error: command 'g++' failed with exit status 1
> configure: error: no
> ---
> It appears we were unable to build a usable native Python egg.
> There are two possible workarounds for this issue:
> 1. Disable python bindings by configuring with --disable-python.
> 2. Use an alternative Python installation that was built using
>the same build setup as Mesos.
> ---
> {code}
> When building with g++ 4.9.4 the build fails with a segmentation fault:
> {code}
> mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
> slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
> /bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
> -DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
> -DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
> -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
> -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
> -DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
> -DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
> -I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
> -D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
> -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
> -I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
> -I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
> -I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
> -pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong 
> -fPIC -fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized 
> -std=c++11 -MT 
> slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP 
> -MF 
> slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
> -o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test 
> -f 'slave/containerizer/mesos/io/switchboard.cpp' || echo 
> '../../src/'`slave/containerizer/mesos/io/switchboard.cpp
> libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
> -DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
> -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
> -DMESOS_HAS_PYTHON=1 -I. -I../../src 

[jira] [Commented] (MESOS-6802) SSL socket can lose bytes in the case of EOF

2017-01-10 Thread Benjamin Hindman (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816895#comment-15816895
 ] 

Benjamin Hindman commented on MESOS-6802:
-

Committed https://reviews.apache.org/r/53802:

{code}
commit d4185d4a117d9cba32d80993d14fcefb566cd9f9
Author: Greg Mann 
Date:   Tue Jan 10 18:23:45 2017 -0800

Eliminated an EOF race condition in libprocess SSL socket.

Previously, it was possible for an SSL socket to either:
1) Fail to receive an EOF if the EOF event was received when
   there was no pending recv() request.
2) Fail to receive all data sent on the sending side if an
   EOF event was received before all sent data was read.

This patch eliminates these race conditions to ensure reliable
receipt of both sent data and EOFs.

Review: https://reviews.apache.org/r/53802/
{code}

> SSL socket can lose bytes in the case of EOF
> 
>
> Key: MESOS-6802
> URL: https://issues.apache.org/jira/browse/MESOS-6802
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Reporter: Greg Mann
>Assignee: Greg Mann
>  Labels: libevent, libprocess, ssl
>
> During recent work on SSL-enabled tests in libprocess (MESOS-5966), we 
> discovered a bug in {{LibeventSSLSocketImpl}}, wherein the socket can either 
> fail to receive an EOF, or lose data when an EOF is received.
> The {{LibeventSSLSocketImpl::event_callback(short events)}} method 
> immediately sets any pending {{RecvRequest}}'s promise to zero upon receipt 
> of an EOF. However, at the time the promise is set, there may actually be 
> data waiting to be read by libevent. Upon receipt of an EOF, we should 
> attempt to read the socket's bufferevent first to ensure that we aren't 
> losing any data previously received by the socket.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6866) Mesos agent not checking IDs before using them as part of the paths

2017-01-10 Thread Joseph Wu (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816707#comment-15816707
 ] 

Joseph Wu commented on MESOS-6866:
--

I noticed that the framework can pass an empty string as a "valid" 
{{ExecutorID}}.  This results in slightly malformed sandbox paths:
i.e. {{.../slaves/.../frameworks/.../executors/runs/...}}
Instead of {{.../slaves/.../frameworks/.../executors/.../runs/...}}

I think the agent can handle this case well enough (it doesn't appear to fall 
over), but we may still want to dis-allow empty {{ExecutorID}}s.

> Mesos agent not checking IDs before using them as part of the paths
> ---
>
> Key: MESOS-6866
> URL: https://issues.apache.org/jira/browse/MESOS-6866
> Project: Mesos
>  Issue Type: Bug
>  Components: security
>Reporter: Yan Xu
>Assignee: Yan Xu
>
> Various IDs are used in Mesos, some assigned by the master (AgentID, 
> FrameworkID, etc) and some created by the frameworks (TaskID, ExecutorID etc).
> The master does sufficient validation on the IDs supplied by the frameworks 
> and the agent currently just trusts that the IDs are valid because they have 
> been validated. 
> The problem is that currently any entity can spoof as the master to inject 
> certain actions on the agent which can be executed as "root" and inflict harm 
> on the system. The "right" long term fix is of course to prevent this from 
> happening but as a short-term defensive measure we can insert some hard 
> CHECKs on the validity of the IDs in the agent code paths.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-6903) Master <-> Agent Internal HTTP v1 API

2017-01-10 Thread Yan Xu (JIRA)
Yan Xu created MESOS-6903:
-

 Summary: Master <-> Agent Internal HTTP v1 API
 Key: MESOS-6903
 URL: https://issues.apache.org/jira/browse/MESOS-6903
 Project: Mesos
  Issue Type: Epic
Reporter: Yan Xu


Similar to the framework and operator HTTP v1 APIs (MESOS-2288, MESOS-4791, 
MESOS-4793), we should implement the internal HTTP v1 API. 

The motivations for the scheduler/operator API includes improvements on 
usability as well as connectivity. The latter is shared by the internal APIs 
(See MESOS-304).

Aside from that, HTTP v1 API helps with security as all requests from the 
agents are authenticated and the agents can be sure they only process events 
from the master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-304) Master should register a slave only after it confirms it can talk to the slave

2017-01-10 Thread Yan Xu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yan Xu updated MESOS-304:
-
Issue Type: Bug  (was: Epic)

> Master should register a slave only after it confirms it can talk to the slave
> --
>
> Key: MESOS-304
> URL: https://issues.apache.org/jira/browse/MESOS-304
> Project: Mesos
>  Issue Type: Bug
>Reporter: Vinod Kone
>
> We have seen this issue from users running on EC2 and also at Twitter.
> The crux of the issue is that, the master starts offering the resources of a 
> slave as soon as it gets a Register message. If for some reason the master 
> --> slave connection is not viable (e.g. slave used its private ip address, 
> DNS failures), we end up in a loop as follows:
> --> Slave sends Register message to master
> --> Master accepts it and offers resources to the framework
> --> The slave health checks to the slave keeps failing
> --> Framework launches tasks on this slave, which would be dropped on the 
> floor
> --> After health check timeout (>60s), master disconnects the slave
> --> Slave sends a Register message again.
> --> Repeat
> One way to solve this problem is to do a 3-way handshake for registration.
> This should also be done for framework registration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-6848) The default executor does not exit if a single task pod fails.

2017-01-10 Thread Anand Mazumdar (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anand Mazumdar updated MESOS-6848:
--
Fix Version/s: 1.1.1

> The default executor does not exit if a single task pod fails.
> --
>
> Key: MESOS-6848
> URL: https://issues.apache.org/jira/browse/MESOS-6848
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
>Priority: Blocker
> Fix For: 1.1.1, 1.2.0
>
>
> If a task group has a single task and it exits with a non-zero exit code, the 
> default executor does not commit suicide.
> This mostly happens due to the fact that we invoke {{shutdown()}} in 
> {{waited()}} when we notice the termination of a single container here: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L666
> but then we return early here after executing all the kill calls: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L751
> However, when there is just one task in the task group, this won't result in 
> {{__shutdown}} being called ever leading to the executor committing suicide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6848) The default executor does not exit if a single task pod fails.

2017-01-10 Thread Anand Mazumdar (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816362#comment-15816362
 ] 

Anand Mazumdar commented on MESOS-6848:
---

1.1.x backport
{noformat}
commit 6798ba84a766aeaae0c3f2df455e200c57ce2b28
Author: Anand Mazumdar 
Date:   Tue Jan 10 13:52:37 2017 -0800

Added MESOS-6848 to CHANGELOG for 1.1.1.

commit 17d44f460a553d99edcfa3ff04aefde9e72ae8ea
Author: Anand Mazumdar 
Date:   Tue Jan 10 13:08:03 2017 -0800

Fixed a bug in the default executor around not committing suicide.

This bug is only observed when the task group contains a single task.
The default executor was not committing suicide when this single task
used to exit with a non-zero status code as per the default restart
policy.

Review: https://reviews.apache.org/r/55157/
{noformat}

> The default executor does not exit if a single task pod fails.
> --
>
> Key: MESOS-6848
> URL: https://issues.apache.org/jira/browse/MESOS-6848
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
>Priority: Blocker
> Fix For: 1.1.1, 1.2.0
>
>
> If a task group has a single task and it exits with a non-zero exit code, the 
> default executor does not commit suicide.
> This mostly happens due to the fact that we invoke {{shutdown()}} in 
> {{waited()}} when we notice the termination of a single container here: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L666
> but then we return early here after executing all the kill calls: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L751
> However, when there is just one task in the task group, this won't result in 
> {{__shutdown}} being called ever leading to the executor committing suicide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6848) The default executor does not exit if a single task pod fails.

2017-01-10 Thread Anand Mazumdar (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816312#comment-15816312
 ] 

Anand Mazumdar commented on MESOS-6848:
---

aha, Thanks for the tip.

> The default executor does not exit if a single task pod fails.
> --
>
> Key: MESOS-6848
> URL: https://issues.apache.org/jira/browse/MESOS-6848
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
>Priority: Blocker
> Fix For: 1.2.0
>
>
> If a task group has a single task and it exits with a non-zero exit code, the 
> default executor does not commit suicide.
> This mostly happens due to the fact that we invoke {{shutdown()}} in 
> {{waited()}} when we notice the termination of a single container here: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L666
> but then we return early here after executing all the kill calls: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L751
> However, when there is just one task in the task group, this won't result in 
> {{__shutdown}} being called ever leading to the executor committing suicide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6796) IOSwitchboard server should discard the accept when terminates.

2017-01-10 Thread Greg Mann (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816307#comment-15816307
 ] 

Greg Mann commented on MESOS-6796:
--

{code}
Branch: refs/heads/master
Commit: 87d7230171ec7a3ad2d232fbffc5bfc94f9cd68c
Parents: b492d44
Author: Greg Mann 
Authored: Mon Jan 9 16:06:11 2017 -0800
Committer: Jie Yu 
Committed: Mon Jan 9 16:06:11 2017 -0800

Fixed an FD leak in the IO switchboard.

Previously, the IO switchboard could leak file descriptors
because it held a reference to its server socket within the
socket's accept loop. This patch explicitly discards the
future containing this reference to eliminate the leak.

Review: https://reviews.apache.org/r/55355/
{code}

> IOSwitchboard server should discard the accept when terminates.
> ---
>
> Key: MESOS-6796
> URL: https://issues.apache.org/jira/browse/MESOS-6796
> Project: Mesos
>  Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Greg Mann
>
> Otherwise, it might leak an fd (mainly in tests).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6848) The default executor does not exit if a single task pod fails.

2017-01-10 Thread Vinod Kone (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816306#comment-15816306
 ] 

Vinod Kone commented on MESOS-6848:
---

FWIW, you don't need to keep the issue open. That is the advantage of having 
separate Target and Fixed Versions.

> The default executor does not exit if a single task pod fails.
> --
>
> Key: MESOS-6848
> URL: https://issues.apache.org/jira/browse/MESOS-6848
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
>Priority: Blocker
>
> If a task group has a single task and it exits with a non-zero exit code, the 
> default executor does not commit suicide.
> This mostly happens due to the fact that we invoke {{shutdown()}} in 
> {{waited()}} when we notice the termination of a single container here: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L666
> but then we return early here after executing all the kill calls: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L751
> However, when there is just one task in the task group, this won't result in 
> {{__shutdown}} being called ever leading to the executor committing suicide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-5346) Some endpoints do not specify their allowed request methods.

2017-01-10 Thread Vinod Kone (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816299#comment-15816299
 ] 

Vinod Kone commented on MESOS-5346:
---

Note that the REST endpoints will be deprecated soon in favor of the new RPC 
style /v1 endpoint and associated calls. 

> Some endpoints do not specify their allowed request methods.
> 
>
> Key: MESOS-5346
> URL: https://issues.apache.org/jira/browse/MESOS-5346
> Project: Mesos
>  Issue Type: Bug
>  Components: security, technical debt
>Reporter: Jan Schlicht
>  Labels: http, mesosphere, security, tech-debt
>
> Some HTTP endpoints (for example "/flags" or "/state") create a response 
> regardless of what the request method is. For example an HTTP POST to the 
> "/state" endpoint will create the same response as an HTTP GET.
> While this inconsistency isn't harmful at the moment, it will get problematic 
> when authorization is implemented, using separate ACLs for endpoints that can 
> be GETed and endpoints that can be POSTed to.
> Validation of the request method should be added to all endpoints, e.g. 
> "/state" should return a 405 (Method Not Allowed) when POSTed to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-6796) IOSwitchboard server should discard the accept when terminates.

2017-01-10 Thread Kevin Klues (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Klues updated MESOS-6796:
---
Assignee: Greg Mann

> IOSwitchboard server should discard the accept when terminates.
> ---
>
> Key: MESOS-6796
> URL: https://issues.apache.org/jira/browse/MESOS-6796
> Project: Mesos
>  Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Greg Mann
>
> Otherwise, it might leak an fd (mainly in tests).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6848) The default executor does not exit if a single task pod fails.

2017-01-10 Thread Anand Mazumdar (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816257#comment-15816257
 ] 

Anand Mazumdar commented on MESOS-6848:
---

Keeping the issue open to backport to 1.1.x branch.

{noformat}
commit 3efcd33f440c7e56c137bfb7cd953ee35e4b3aa5
Author: Anand Mazumdar 
Date:   Tue Jan 10 13:08:03 2017 -0800

Fixed a bug in the default executor around not committing suicide.

This bug is only observed when the task group contains a single task.
The default executor was not committing suicide when this single task
used to exit with a non-zero status code as per the default restart
policy.

Review: https://reviews.apache.org/r/55157/
{noformat}

> The default executor does not exit if a single task pod fails.
> --
>
> Key: MESOS-6848
> URL: https://issues.apache.org/jira/browse/MESOS-6848
> Project: Mesos
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Anand Mazumdar
>Assignee: Anand Mazumdar
>Priority: Blocker
>
> If a task group has a single task and it exits with a non-zero exit code, the 
> default executor does not commit suicide.
> This mostly happens due to the fact that we invoke {{shutdown()}} in 
> {{waited()}} when we notice the termination of a single container here: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L666
> but then we return early here after executing all the kill calls: 
> https://github.com/apache/mesos/blob/master/src/launcher/default_executor.cpp#L751
> However, when there is just one task in the task group, this won't result in 
> {{__shutdown}} being called ever leading to the executor committing suicide.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread Kevin Klues (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816077#comment-15816077
 ] 

Kevin Klues commented on MESOS-6901:


Thanks Frank. I will look into this in the coming days.

> Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04
> 
>
> Key: MESOS-6901
> URL: https://issues.apache.org/jira/browse/MESOS-6901
> Project: Mesos
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.1.0
>Reporter: Frank Scholten
>
> When building with g++ 4.8.5 the configure script fails:
> {code}
> checking whether we can build usable Python eggs... cc1plus: warning: command 
> line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ 
> [enabled by default]
> g++: error: unrecognized command line option '-Wdate-time'
> g++: error: unrecognized command line option '-fstack-protector-strong'
> error: command 'g++' failed with exit status 1
> configure: error: no
> ---
> It appears we were unable to build a usable native Python egg.
> There are two possible workarounds for this issue:
> 1. Disable python bindings by configuring with --disable-python.
> 2. Use an alternative Python installation that was built using
>the same build setup as Mesos.
> ---
> {code}
> When building with g++ 4.9.4 the build fails with a segmentation fault:
> {code}
> mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
> slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
> /bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
> -DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
> -DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
> -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
> -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
> -DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
> -DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
> -I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
> -D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
> -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
> -I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
> -I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
> -I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
> -pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong 
> -fPIC -fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized 
> -std=c++11 -MT 
> slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP 
> -MF 
> slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
> -o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test 
> -f 'slave/containerizer/mesos/io/switchboard.cpp' || echo 
> '../../src/'`slave/containerizer/mesos/io/switchboard.cpp
> libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
> -DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
> -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
> -DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
> -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
> -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
> 

[jira] [Assigned] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread Kevin Klues (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Klues reassigned MESOS-6901:
--

Assignee: Kevin Klues

> Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04
> 
>
> Key: MESOS-6901
> URL: https://issues.apache.org/jira/browse/MESOS-6901
> Project: Mesos
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.1.0
>Reporter: Frank Scholten
>Assignee: Kevin Klues
>
> When building with g++ 4.8.5 the configure script fails:
> {code}
> checking whether we can build usable Python eggs... cc1plus: warning: command 
> line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ 
> [enabled by default]
> g++: error: unrecognized command line option '-Wdate-time'
> g++: error: unrecognized command line option '-fstack-protector-strong'
> error: command 'g++' failed with exit status 1
> configure: error: no
> ---
> It appears we were unable to build a usable native Python egg.
> There are two possible workarounds for this issue:
> 1. Disable python bindings by configuring with --disable-python.
> 2. Use an alternative Python installation that was built using
>the same build setup as Mesos.
> ---
> {code}
> When building with g++ 4.9.4 the build fails with a segmentation fault:
> {code}
> mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
> slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
> /bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
> -DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
> -DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
> -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
> -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
> -DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
> -DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
> -I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
> -D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
> -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
> -I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
> -I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
> -I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
> -pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong 
> -fPIC -fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized 
> -std=c++11 -MT 
> slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP 
> -MF 
> slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
> -o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test 
> -f 'slave/containerizer/mesos/io/switchboard.cpp' || echo 
> '../../src/'`slave/containerizer/mesos/io/switchboard.cpp
> libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
> -DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
> -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
> -DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
> -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
> -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
> -I../include/mesos -DPICOJSON_USE_INT64 

[jira] [Updated] (MESOS-6871) Scheme parsing is incorrect in libprocess URL::parse().

2017-01-10 Thread Benjamin Mahler (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Mahler updated MESOS-6871:
---
Summary: Scheme parsing is incorrect in libprocess URL::parse().  (was: 
Scheme handling in libprocess URL::parse())

> Scheme parsing is incorrect in libprocess URL::parse().
> ---
>
> Key: MESOS-6871
> URL: https://issues.apache.org/jira/browse/MESOS-6871
> Project: Mesos
>  Issue Type: Bug
>  Components: libprocess
>Affects Versions: 1.1.0
>Reporter: Ilya Pronin
>Assignee: Ilya Pronin
> Fix For: 1.2.0
>
>
> {{process::http::URL::parse()}} can mistake the host part for the scheme 
> because of unsuitable use of {{std::string::find_first_of()}} method which 
> looks for the first character equal to one of the characters in the given 
> sequence.
> E.g. {{URL::parse("http/abcdef")}} will construct a {{URL}} that represents 
> {{http://cdef/}}.
> Review request: https://reviews.apache.org/r/55177/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-6804) Running 'tty' inside a debug container that has a tty reports "Not a tty"

2017-01-10 Thread Kevin Klues (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Klues updated MESOS-6804:
---
Story Points: 2  (was: 1)

> Running 'tty' inside a debug container that has a tty reports "Not a tty"
> -
>
> Key: MESOS-6804
> URL: https://issues.apache.org/jira/browse/MESOS-6804
> Project: Mesos
>  Issue Type: Bug
>Reporter: Kevin Klues
>Assignee: Kevin Klues
>Priority: Blocker
>  Labels: debugging, mesosphere
>
> We need to inject `/dev/console` into the container and map it to the slave 
> end of the TTY we are attached to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-6902) Add support for agent capabilities

2017-01-10 Thread Neil Conway (JIRA)
Neil Conway created MESOS-6902:
--

 Summary: Add support for agent capabilities
 Key: MESOS-6902
 URL: https://issues.apache.org/jira/browse/MESOS-6902
 Project: Mesos
  Issue Type: Improvement
  Components: agent
Reporter: Neil Conway


Similarly to how we might add support for master capabilities (MESOS-5675), 
agent capabilities would also make sense: in a mixed cluster, the master might 
have support for features that are not present on certain agents, and vice 
versa.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-5675) Add support for master capabilities

2017-01-10 Thread Neil Conway (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815847#comment-15815847
 ] 

Neil Conway commented on MESOS-5675:


Not AFAIK -- makes sense to consider adding that as well. I'll create a JIRA.

> Add support for master capabilities
> ---
>
> Key: MESOS-5675
> URL: https://issues.apache.org/jira/browse/MESOS-5675
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Neil Conway
>  Labels: mesosphere
>
> Right now, frameworks can advertise their capabilities to the master via the 
> {{FrameworkInfo}} they use for registration/re-registration. This allows 
> masters to provide backward compatibility for old frameworks that don't 
> support new functionality.
> To allow new frameworks to support backward compatibility with old masters, 
> the inverse concept would be useful: masters would tell frameworks which 
> capabilities are supported by the master, which the frameworks could then use 
> to decide whether to use features only supported by more recent versions of 
> the master.
> For now, frameworks can workaround this by looking at the master's version 
> number, but that seems a bit fragile and hacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-5675) Add support for master capabilities

2017-01-10 Thread Benjamin Mahler (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815829#comment-15815829
 ] 

Benjamin Mahler commented on MESOS-5675:


Is there a ticket for agent capabilities as well? E.g. the master was upgraded 
to support feature X but only some of the agents were upgraded.

> Add support for master capabilities
> ---
>
> Key: MESOS-5675
> URL: https://issues.apache.org/jira/browse/MESOS-5675
> Project: Mesos
>  Issue Type: Improvement
>  Components: master
>Reporter: Neil Conway
>  Labels: mesosphere
>
> Right now, frameworks can advertise their capabilities to the master via the 
> {{FrameworkInfo}} they use for registration/re-registration. This allows 
> masters to provide backward compatibility for old frameworks that don't 
> support new functionality.
> To allow new frameworks to support backward compatibility with old masters, 
> the inverse concept would be useful: masters would tell frameworks which 
> capabilities are supported by the master, which the frameworks could then use 
> to decide whether to use features only supported by more recent versions of 
> the master.
> For now, frameworks can workaround this by looking at the master's version 
> number, but that seems a bit fragile and hacky.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread Frank Scholten (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Scholten updated MESOS-6901:
--
Description: 
When building with g++ 4.8.5 the configure script fails:

{code}
checking whether we can build usable Python eggs... cc1plus: warning: command 
line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled 
by default]
g++: error: unrecognized command line option '-Wdate-time'
g++: error: unrecognized command line option '-fstack-protector-strong'
error: command 'g++' failed with exit status 1
configure: error: no
---
It appears we were unable to build a usable native Python egg.

There are two possible workarounds for this issue:
1. Disable python bindings by configuring with --disable-python.
2. Use an alternative Python installation that was built using
   the same build setup as Mesos.
---
{code}

When building with g++ 4.9.4 the build fails with a segmentation fault:

{code}
mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
-DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
-DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
-DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
-DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
-DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
-DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
-DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
-DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
-DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
-DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
-DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
-DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
-I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
-D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
-I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -std=c++11 -MT 
slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP -MF 
slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
-o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test -f 
'slave/containerizer/mesos/io/switchboard.cpp' || echo 
'../../src/'`slave/containerizer/mesos/io/switchboard.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
-DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
-DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
-DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
-DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
-DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
-DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
-DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
-DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
-DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
-I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -isystem 
../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-O2 -Wno-unused-local-typedefs 

[jira] [Updated] (MESOS-6901) Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04

2017-01-10 Thread Frank Scholten (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Scholten updated MESOS-6901:
--
Summary: Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on 
Ubuntu 16.04  (was: Master build fails with g++ 4.8.5 and g++ 4.9.4 on Ubuntu 
16.04)

> Master build fails with g++ 4.8.5, g++ 4.9.4 and clang 3.8.0 on Ubuntu 16.04
> 
>
> Key: MESOS-6901
> URL: https://issues.apache.org/jira/browse/MESOS-6901
> Project: Mesos
>  Issue Type: Bug
>  Components: build
>Affects Versions: 1.1.0
>Reporter: Frank Scholten
>
> When building with g++ 4.8.5 the configure script fails:
> {code}
> checking whether we can build usable Python eggs... cc1plus: warning: command 
> line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ 
> [enabled by default]
> g++: error: unrecognized command line option '-Wdate-time'
> g++: error: unrecognized command line option '-fstack-protector-strong'
> error: command 'g++' failed with exit status 1
> configure: error: no
> ---
> It appears we were unable to build a usable native Python egg.
> There are two possible workarounds for this issue:
> 1. Disable python bindings by configuring with --disable-python.
> 2. Use an alternative Python installation that was built using
>the same build setup as Mesos.
> ---
> {code}
> When building with g++ 4.9.4 the build fails with a segmentation fault:
> {code}
> mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
> slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
> /bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
> -DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
> -DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
> -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
> -DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
> -DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
> -DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
> -I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
> -D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
> -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
> -I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
> -I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
> -I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/include 
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
> -pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong 
> -fPIC -fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized 
> -std=c++11 -MT 
> slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP 
> -MF 
> slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
> -o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test 
> -f 'slave/containerizer/mesos/io/switchboard.cpp' || echo 
> '../../src/'`slave/containerizer/mesos/io/switchboard.cpp
> libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
> -DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
> -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
> -DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
> -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
> 

[jira] [Updated] (MESOS-6901) Master build fails with g++ 4.8.5 and g++ 4.9.4 on Ubuntu 16.04

2017-01-10 Thread Frank Scholten (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Scholten updated MESOS-6901:
--
Description: 
When building with g++ 4.8.5 the configure script fails:

{code}
checking whether we can build usable Python eggs... cc1plus: warning: command 
line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled 
by default]
g++: error: unrecognized command line option '-Wdate-time'
g++: error: unrecognized command line option '-fstack-protector-strong'
error: command 'g++' failed with exit status 1
configure: error: no
---
It appears we were unable to build a usable native Python egg.

There are two possible workarounds for this issue:
1. Disable python bindings by configuring with --disable-python.
2. Use an alternative Python installation that was built using
   the same build setup as Mesos.
---
{code}

When building with g++ 4.9.4 the build fails with a segmentation fault:

{code}
mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
-DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
-DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
-DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
-DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
-DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
-DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
-DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
-DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
-DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
-DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
-DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
-DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
-I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
-D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
-I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -std=c++11 -MT 
slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP -MF 
slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
-o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test -f 
'slave/containerizer/mesos/io/switchboard.cpp' || echo 
'../../src/'`slave/containerizer/mesos/io/switchboard.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
-DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
-DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
-DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
-DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
-DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
-DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
-DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
-DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
-DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
-I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -isystem 
../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-O2 -Wno-unused-local-typedefs 

[jira] [Updated] (MESOS-6901) Master build fails with g++ 4.8.5 and g++ 4.9.4 on Ubuntu 16.04

2017-01-10 Thread Frank Scholten (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Scholten updated MESOS-6901:
--
Description: 
When building with g++ 4.8.5 the configure script fails:

{code}
checking whether we can build usable Python eggs... cc1plus: warning: command 
line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled 
by default]
g++: error: unrecognized command line option '-Wdate-time'
g++: error: unrecognized command line option '-fstack-protector-strong'
error: command 'g++' failed with exit status 1
configure: error: no
---
It appears we were unable to build a usable native Python egg.

There are two possible workarounds for this issue:
1. Disable python bindings by configuring with --disable-python.
2. Use an alternative Python installation that was built using
   the same build setup as Mesos.
---
{code}

When building with g++ 4.9.4 the build fails with a segmentation fault:

{code}
mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
-DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
-DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
-DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
-DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
-DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
-DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
-DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
-DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
-DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
-DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
-DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
-DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
-I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
-D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
-I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -std=c++11 -MT 
slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP -MF 
slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
-o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test -f 
'slave/containerizer/mesos/io/switchboard.cpp' || echo 
'../../src/'`slave/containerizer/mesos/io/switchboard.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
-DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
-DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
-DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
-DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
-DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
-DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
-DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
-DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
-DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
-I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -isystem 
../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-O2 -Wno-unused-local-typedefs 

[jira] [Updated] (MESOS-6901) Master build fails with g++ 4.8.5 and g++ 4.9.4 on Ubuntu 16.04

2017-01-10 Thread Frank Scholten (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frank Scholten updated MESOS-6901:
--
Description: 
When building with g++ 4.8.5 the configure script fails:

{code}
checking whether we can build usable Python eggs... cc1plus: warning: command 
line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++ [enabled 
by default]
g++: error: unrecognized command line option '-Wdate-time'
g++: error: unrecognized command line option '-fstack-protector-strong'
error: command 'g++' failed with exit status 1
configure: error: no
---
It appears we were unable to build a usable native Python egg.

There are two possible workarounds for this issue:
1. Disable python bindings by configuring with --disable-python.
2. Use an alternative Python installation that was built using
   the same build setup as Mesos.
---
{code}

When building with g++ 4.9.4 the build fails with a segmentation fault:

```
mv -f slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Tpo 
slave/containerizer/mesos/.deps/libmesos_no_3rdparty_la-utils.Plo
/bin/bash ../libtool  --tag=CXX   --mode=compile g++ -DPACKAGE_NAME=\"mesos\" 
-DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.2.0\" 
-DPACKAGE_STRING=\"mesos\ 1.2.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" 
-DPACKAGE=\"mesos\" -DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 
-DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 
-DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 
-DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 
-DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 
-DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 
-DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 
-DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 
-DHAVE_PYTHON=\"2.7\" -DMESOS_HAS_PYTHON=1 -I. -I../../src   -Werror 
-DLIBDIR=\"/usr/lib\" -DPKGLIBEXECDIR=\"/usr/libexec/mesos\" 
-DPKGDATADIR=\"/usr/share/mesos\" -DPKGMODULEDIR=\"/usr/lib/mesos/modules\" 
-I../../include -I../include -I../include/mesos -DPICOJSON_USE_INT64 
-D__STDC_FORMAT_MACROS -isystem ../3rdparty/boost-1.53.0 
-I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0  
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-fPIE -O2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -std=c++11 -MT 
slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo -MD -MP -MF 
slave/containerizer/mesos/io/.deps/libmesos_no_3rdparty_la-switchboard.Tpo -c 
-o slave/containerizer/mesos/io/libmesos_no_3rdparty_la-switchboard.lo `test -f 
'slave/containerizer/mesos/io/switchboard.cpp' || echo 
'../../src/'`slave/containerizer/mesos/io/switchboard.cpp
libtool: compile:  g++ -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\" 
-DPACKAGE_VERSION=\"1.2.0\" "-DPACKAGE_STRING=\"mesos 1.2.0\"" 
-DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" 
-DVERSION=\"1.2.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1 
-DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1 
-DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1 
-DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1 
-DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -DHAVE_PYTHON=\"2.7\" 
-DMESOS_HAS_PYTHON=1 -I. -I../../src -Werror -DLIBDIR=\"/usr/lib\" 
-DPKGLIBEXECDIR=\"/usr/libexec/mesos\" -DPKGDATADIR=\"/usr/share/mesos\" 
-DPKGMODULEDIR=\"/usr/lib/mesos/modules\" -I../../include -I../include 
-I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS -isystem 
../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.2 -I../3rdparty/glog-0.3.3/src 
-I../3rdparty/leveldb-1.4/include -I../../3rdparty/libprocess/include 
-I../3rdparty/nvml-352.79 -I../3rdparty/picojson-1.3.0 
-I../3rdparty/protobuf-2.6.1/src -I../../3rdparty/stout/include 
-I../3rdparty/zookeeper-3.4.8/src/c/include 
-I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1 
-I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0 
-pthread -Wall -Wsign-compare -Wformat-security -fstack-protector-strong -fPIC 
-O2 -Wno-unused-local-typedefs 

[jira] [Issue Comment Deleted] (MESOS-5342) CPU pinning/binding support for CgroupsCpushareIsolatorProcess

2017-01-10 Thread Chris (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris updated MESOS-5342:
-
Comment: was deleted

(was: Implemented a mesos module and resource estimator - source has been 
posted to https://github.com/ct-clmsn/mesos-cpusets
Added performance counter enabled tools and mesos executor - source posted to 
https://github.com/ct-clmsn/mesos-papi)

> CPU pinning/binding support for CgroupsCpushareIsolatorProcess
> --
>
> Key: MESOS-5342
> URL: https://issues.apache.org/jira/browse/MESOS-5342
> Project: Mesos
>  Issue Type: Improvement
>  Components: cgroups, containerization
>Affects Versions: 0.28.1
>Reporter: Chris
>  Labels: cgroups, cpu, cpu-usage, gpu, isolation, isolator, 
> mentor, perfomance
>
> The cgroups isolator currently lacks support for binding (also called 
> pinning) containers to a set of cores. The GNU/Linux kernel is known to make 
> sub-optimal core assignments for processes and threads. Poor assignments 
> impact program performance, specifically in terms of cache locality. 
> Applications requiring GPU resources can benefit from this feature by getting 
> access to cores closest to the GPU hardware, which reduces cpu-gpu copy 
> latency.
> Most cluster management systems from the HPC community (SLURM) provide both 
> cgroup isolation and cpu binding. This feature would provide similar 
> capabilities. The current interest in supporting Intel's Cache Allocation 
> Technology, and the advent of Intel's Knights-series processors, will require 
> making choices about where container's are going to run on the mesos-agent's 
> processor(s) cores - this feature is a step toward developing a robust 
> solution.
> The improvement in this JIRA ticket will handle hardware topology detection, 
> track container-to-core utilization in a histogram, and use a mathematical 
> optimization technique to select cores for container assignment based on 
> latency and the container-to-core utilization histogram.
> For GPU tasks, the improvement will prioritize selection of cores based on 
> latency between the GPU and cores in an effort to minimize copy latency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-5342) CPU pinning/binding support for CgroupsCpushareIsolatorProcess

2017-01-10 Thread Chris (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815211#comment-15815211
 ] 

Chris commented on MESOS-5342:
--

Implemented a mesos module and resource estimator - source has been posted to 
https://github.com/ct-clmsn/mesos-cpusets
Added performance counter enabled tools and mesos executor - source posted to 
https://github.com/ct-clmsn/mesos-papi

> CPU pinning/binding support for CgroupsCpushareIsolatorProcess
> --
>
> Key: MESOS-5342
> URL: https://issues.apache.org/jira/browse/MESOS-5342
> Project: Mesos
>  Issue Type: Improvement
>  Components: cgroups, containerization
>Affects Versions: 0.28.1
>Reporter: Chris
>  Labels: cgroups, cpu, cpu-usage, gpu, isolation, isolator, 
> mentor, perfomance
>
> The cgroups isolator currently lacks support for binding (also called 
> pinning) containers to a set of cores. The GNU/Linux kernel is known to make 
> sub-optimal core assignments for processes and threads. Poor assignments 
> impact program performance, specifically in terms of cache locality. 
> Applications requiring GPU resources can benefit from this feature by getting 
> access to cores closest to the GPU hardware, which reduces cpu-gpu copy 
> latency.
> Most cluster management systems from the HPC community (SLURM) provide both 
> cgroup isolation and cpu binding. This feature would provide similar 
> capabilities. The current interest in supporting Intel's Cache Allocation 
> Technology, and the advent of Intel's Knights-series processors, will require 
> making choices about where container's are going to run on the mesos-agent's 
> processor(s) cores - this feature is a step toward developing a robust 
> solution.
> The improvement in this JIRA ticket will handle hardware topology detection, 
> track container-to-core utilization in a histogram, and use a mathematical 
> optimization technique to select cores for container assignment based on 
> latency and the container-to-core utilization histogram.
> For GPU tasks, the improvement will prioritize selection of cores based on 
> latency between the GPU and cores in an effort to minimize copy latency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-6900) Add test for framework upgrading to multi-role capability.

2017-01-10 Thread Benjamin Bannier (JIRA)
Benjamin Bannier created MESOS-6900:
---

 Summary: Add test for framework upgrading to multi-role capability.
 Key: MESOS-6900
 URL: https://issues.apache.org/jira/browse/MESOS-6900
 Project: Mesos
  Issue Type: Bug
  Components: test
Reporter: Benjamin Bannier
Assignee: Benjamin Bannier


Frameworks can upgrade to multi-role capability as long as the framework's role 
remains the same.

We consider the framework roles unchanged if 
* a framework previously didn't specify a {{role}} now has {{roles=()}}, or
* a framework which previously had {{role=A}} and now has {{roles=(A)}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-4826) Test helper function mesos::internal::tests::Metrics has name not following mesos style

2017-01-10 Thread Benjamin Bannier (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Bannier updated MESOS-4826:

Assignee: (was: Benjamin Bannier)

> Test helper function mesos::internal::tests::Metrics has name not following 
> mesos style 
> 
>
> Key: MESOS-4826
> URL: https://issues.apache.org/jira/browse/MESOS-4826
> Project: Mesos
>  Issue Type: Bug
>  Components: test
>Reporter: Benjamin Bannier
>Priority: Trivial
>
> The test helper function {{mesos::internal::tests::Metrics}} has a name not 
> following mesos style. The expected name would have been {{metrics}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MESOS-6899) Document the unified containerizer image pulling `cached` option.

2017-01-10 Thread Gilbert Song (JIRA)

 [ 
https://issues.apache.org/jira/browse/MESOS-6899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gilbert Song updated MESOS-6899:

Description: 
In docker containerizer, there is a `force_pull_image` option to force the 
image being pulled overwriting the existing image in docker daemon.

Correspondingly, in unified containerizer, both docker and appc store support 
this option (named as `cached` in protobuf), which overwrites existing image 
layers.

We should document this option in unified containerizer.

  was:
In docker containerizer, there is a {force_pull_image} option to force the 
image being pulled overwriting the existing image in docker daemon.

Correspondingly, in unified containerizer, both docker and appc store support 
this option (named as {cached} in protobuf), which overwrites existing image 
layers.

We should document this option in unified containerizer.


> Document the unified containerizer image pulling `cached` option.
> -
>
> Key: MESOS-6899
> URL: https://issues.apache.org/jira/browse/MESOS-6899
> Project: Mesos
>  Issue Type: Documentation
>  Components: containerization, documentation
>Reporter: Gilbert Song
>  Labels: containerizer, document
>
> In docker containerizer, there is a `force_pull_image` option to force the 
> image being pulled overwriting the existing image in docker daemon.
> Correspondingly, in unified containerizer, both docker and appc store support 
> this option (named as `cached` in protobuf), which overwrites existing image 
> layers.
> We should document this option in unified containerizer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MESOS-6899) Document the unified containerizer image pulling `cached` option.

2017-01-10 Thread Gilbert Song (JIRA)
Gilbert Song created MESOS-6899:
---

 Summary: Document the unified containerizer image pulling `cached` 
option.
 Key: MESOS-6899
 URL: https://issues.apache.org/jira/browse/MESOS-6899
 Project: Mesos
  Issue Type: Documentation
  Components: containerization, documentation
Reporter: Gilbert Song


In docker containerizer, there is a {force_pull_image} option to force the 
image being pulled overwriting the existing image in docker daemon.

Correspondingly, in unified containerizer, both docker and appc store support 
this option (named as {cached} in protobuf), which overwrites existing image 
layers.

We should document this option in unified containerizer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-5346) Some endpoints do not specify their allowed request methods.

2017-01-10 Thread UENISHI Kota (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-5346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814329#comment-15814329
 ] 

UENISHI Kota commented on MESOS-5346:
-

I'd like to note here that and endpoint {{/files/download}} also has weird 
behavior where agents respond full body against any methods like {{POST}} 
{{DELETE}} or even {{HEAD}}. Also, under {{GET}} request recognizing {{Range}} 
header element to enable smarter download of file contents in a sandbox would 
be very nice.

I consider this important because there may be a case where the size of files 
ranges like up to ~10GB. Although necessary files or files larger than that 
should be saved to more reliable storage like HDFS or S3, depending on result 
of a task, files remaining in the sandbox would be sometimes downloaded. Nobody 
expects full 10GB body of a file on just HEADing it.

> Some endpoints do not specify their allowed request methods.
> 
>
> Key: MESOS-5346
> URL: https://issues.apache.org/jira/browse/MESOS-5346
> Project: Mesos
>  Issue Type: Bug
>  Components: security, technical debt
>Reporter: Jan Schlicht
>  Labels: http, mesosphere, security, tech-debt
>
> Some HTTP endpoints (for example "/flags" or "/state") create a response 
> regardless of what the request method is. For example an HTTP POST to the 
> "/state" endpoint will create the same response as an HTTP GET.
> While this inconsistency isn't harmful at the moment, it will get problematic 
> when authorization is implemented, using separate ACLs for endpoints that can 
> be GETed and endpoints that can be POSTed to.
> Validation of the request method should be added to all endpoints, e.g. 
> "/state" should return a 405 (Method Not Allowed) when POSTed to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6509) Incorrect glyphicons-halflings-regular.woff2 file

2017-01-10 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814269#comment-15814269
 ] 

ASF GitHub Bot commented on MESOS-6509:
---

Github user haosdent closed the pull request at:

https://github.com/apache/mesos/pull/176


> Incorrect glyphicons-halflings-regular.woff2 file
> -
>
> Key: MESOS-6509
> URL: https://issues.apache.org/jira/browse/MESOS-6509
> Project: Mesos
>  Issue Type: Bug
>  Components: webui
>Reporter: haosdent
>Assignee: haosdent
>Priority: Minor
>
> We add this font file to fix the incomplete Bootstrap upgrade in 
> https://reviews.apache.org/r/47699/ .The size of 
> glyphicons-halflings-regular.woff2 is 0 now due to review board does support 
> to upload binary files.  
> {code}
> du -sh src/webui/master/static/fonts/*
>  20Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.eot
> 108Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.svg
>  48Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.ttf
>  24Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.woff
>   0Bsrc/webui/master/static/fonts/glyphicons-halflings-regular.woff2
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MESOS-6509) Incorrect glyphicons-halflings-regular.woff2 file

2017-01-10 Thread haosdent (JIRA)

[ 
https://issues.apache.org/jira/browse/MESOS-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814263#comment-15814263
 ] 

haosdent commented on MESOS-6509:
-

{code}
commit 44717b2a4e14d286c3db6acbfe5402e6c72efac7
Author: Haosdent Huang 
Date:   Sun Oct 30 01:26:34 2016 +0800

Fix incorrect glyphicons-halflings-regular.woff2.
{code}

> Incorrect glyphicons-halflings-regular.woff2 file
> -
>
> Key: MESOS-6509
> URL: https://issues.apache.org/jira/browse/MESOS-6509
> Project: Mesos
>  Issue Type: Bug
>  Components: webui
>Reporter: haosdent
>Assignee: haosdent
>Priority: Minor
>
> We add this font file to fix the incomplete Bootstrap upgrade in 
> https://reviews.apache.org/r/47699/ .The size of 
> glyphicons-halflings-regular.woff2 is 0 now due to review board does support 
> to upload binary files.  
> {code}
> du -sh src/webui/master/static/fonts/*
>  20Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.eot
> 108Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.svg
>  48Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.ttf
>  24Ksrc/webui/master/static/fonts/glyphicons-halflings-regular.woff
>   0Bsrc/webui/master/static/fonts/glyphicons-halflings-regular.woff2
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)