[jira] [Commented] (MESOS-9411) Validation of JWT tokens using HS256 hashing algorithm is not thread safe.
[
https://issues.apache.org/jira/browse/MESOS-9411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694723#comment-16694723
]
Alexander Rojas commented on MESOS-9411:
[r/69412/|https://reviews.apache.org/r/69412/]: Fixed thread safety issue in
jwt signature validation.
> Validation of JWT tokens using HS256 hashing algorithm is not thread safe.
> --
>
> Key: MESOS-9411
> URL: https://issues.apache.org/jira/browse/MESOS-9411
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 1.8.0
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Major
> Labels: mesosphere
>
> from the [OpenSSL
> documentation|https://www.openssl.org/docs/man1.0.2/crypto/hmac.html]:
> {quote}
> It places the result in {{md}} (which must have space for the output of the
> hash function, which is no more than {{EVP_MAX_MD_SIZE}} bytes). If {{md}} is
> {{NULL}}, the digest is placed in a static array. The size of the output is
> placed in {{md_len}}, unless it is {{NULL}}. Note: passing a {{NULL}} value
> for {{md}} to use the static array is not thread safe.
> {quote}
> We are calling {{HMAC()}} as follows:
> {code}
> unsigned int md_len = 0;
> unsigned char* rc = HMAC(
> EVP_sha256(),
> secret.data(),
> secret.size(),
> reinterpret_cast(message.data()),
> message.size(),
> nullptr, // <- This is `md`
> &md_len);
> if (rc == nullptr) {
> return Error(addErrorReason("HMAC failed"));
> }
> return string(reinterpret_cast(rc), md_len);
> {code}
> Given that this code does not run inside a process, race conditions could
> occur.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-9411) Validation of JWT tokens using HS256 hashing algorithm is not thread safe.
Alexander Rojas created MESOS-9411:
--
Summary: Validation of JWT tokens using HS256 hashing algorithm is
not thread safe.
Key: MESOS-9411
URL: https://issues.apache.org/jira/browse/MESOS-9411
Project: Mesos
Issue Type: Bug
Components: libprocess
Affects Versions: 1.8.0
Reporter: Alexander Rojas
Assignee: Alexander Rojas
from the [OpenSSL
documentation|https://www.openssl.org/docs/man1.0.2/crypto/hmac.html]:
{quote}
It places the result in {{md}} (which must have space for the output of the
hash function, which is no more than {{EVP_MAX_MD_SIZE}} bytes). If {{md}} is
{{NULL}}, the digest is placed in a static array. The size of the output is
placed in {{md_len}}, unless it is {{NULL}}. Note: passing a {{NULL}} value for
{{md}} to use the static array is not thread safe.
{quote}
We are calling {{HMAC()}} as follows:
{code}
unsigned int md_len = 0;
unsigned char* rc = HMAC(
EVP_sha256(),
secret.data(),
secret.size(),
reinterpret_cast(message.data()),
message.size(),
nullptr, // <- This is `md`
&md_len);
if (rc == nullptr) {
return Error(addErrorReason("HMAC failed"));
}
return string(reinterpret_cast(rc), md_len);
{code}
Given that this code does not run inside a process, race conditions could occur.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596019#comment-16596019
]
Alexander Rojas commented on MESOS-7076:
[~tillt] if you check the mailing list of the conversation I had with the guys
from libevent, they are willing to help us debug the issue, what they request
is a container with a debug version of libevent and a compiled mesos test that
he can use to debug. I was timeboxed when I worked on this, so I had to stop
but perhaps we can try to get their help again?
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: build, libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Assignee: Till Toenshoff
>Priority: Critical
> Labels: ci
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
> fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8987) Master asks agent to shutdown upon auth errors
[
https://issues.apache.org/jira/browse/MESOS-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522501#comment-16522501
]
Alexander Rojas commented on MESOS-8987:
[~vinodkone] I don't think I have the back of everyone here, from discussions
I've had, perhaps [~gkleiman] and [~greggomann] may mention why we don't want
to go with my proposal?
> Master asks agent to shutdown upon auth errors
> --
>
> Key: MESOS-8987
> URL: https://issues.apache.org/jira/browse/MESOS-8987
> Project: Mesos
> Issue Type: Bug
> Components: master, security
>Affects Versions: 1.4.1, 1.5.1, 1.6.0, 1.7.0
>Reporter: Gastón Kleiman
>Assignee: Gastón Kleiman
>Priority: Blocker
> Labels: mesosphere
>
> The Mesos master sends a {{ShutdownMessage}} to an agent if there is an
> [authentication|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/master.cpp#L6532-L6543]
> or an
> [authorization|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/master.cpp#L6622-L6633]
> error during agent registration.
>
> Upon receipt of this message, the agent kills alls its tasks and commits
> suicide. This means that transient auth errors can lead to whole agents being
> killed along with it's tasks.
> I think the master should stop sending the {{ShutdownMessage}}s in these
> cases, or at least let the agent retry the registration a few times before
> asking it to shutdown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9006) The agent's GET_AGENT leaks resource information when using authorization
[
https://issues.apache.org/jira/browse/MESOS-9006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516298#comment-16516298
]
Alexander Rojas commented on MESOS-9006:
By the way, I accidentallt accept this issue while searching for the link
button.
> The agent's GET_AGENT leaks resource information when using authorization
> -
>
> Key: MESOS-9006
> URL: https://issues.apache.org/jira/browse/MESOS-9006
> Project: Mesos
> Issue Type: Bug
>Reporter: Benjamin Bannier
>Priority: Critical
> Labels: agent, security
>
> While the master's {{GET_AGENTS}} call e.g., filters resources (by using an
> approver with {{VIEW_ROLE}}) so that it does not leak resources the querying
> principal should not be able to see, no such filtering is done in the
> corresponding agent's {{GET_AGENT}} call.
> This call should be authorized as well to not expose information we expect to
> be not visible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-9006) The agent's GET_AGENT leaks resource information when using authorization
[
https://issues.apache.org/jira/browse/MESOS-9006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516296#comment-16516296
]
Alexander Rojas commented on MESOS-9006:
Similar to MESOS-7887
> The agent's GET_AGENT leaks resource information when using authorization
> -
>
> Key: MESOS-9006
> URL: https://issues.apache.org/jira/browse/MESOS-9006
> Project: Mesos
> Issue Type: Bug
>Reporter: Benjamin Bannier
>Priority: Critical
> Labels: agent, security
>
> While the master's {{GET_AGENTS}} call e.g., filters resources (by using an
> approver with {{VIEW_ROLE}}) so that it does not leak resources the querying
> principal should not be able to see, no such filtering is done in the
> corresponding agent's {{GET_AGENT}} call.
> This call should be authorized as well to not expose information we expect to
> be not visible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8991) `ObjectApprover::approved()` signature prevents it from being mockable
Alexander Rojas created MESOS-8991:
--
Summary: `ObjectApprover::approved()` signature prevents it from
being mockable
Key: MESOS-8991
URL: https://issues.apache.org/jira/browse/MESOS-8991
Project: Mesos
Issue Type: Task
Components: modules
Affects Versions: 1.6.0
Reporter: Alexander Rojas
The
[{{ObjectApprover::approved()}}|https://github.com/apache/mesos/blob/8b93fa3/include/mesos/authorizer/authorizer.hpp#L221]
method has the following signature:
{code}
virtual Try approved(
const Option& object) const noexcept = 0;
{code}
This is unfortunate since it is impossible to mock a function in google mock
with [two
qualifiers|https://groups.google.com/forum/#!topic/googlemock/LsbubY26qx4],
which reduces the amount of tests we can perform.
Moreover, the {{noexcept}} is not even needed in Mesos, since it does not use
exception and it is compiled without exception support by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8987) Master asks agent to shutdown upon auth errors
[
https://issues.apache.org/jira/browse/MESOS-8987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510811#comment-16510811
]
Alexander Rojas commented on MESOS-8987:
I think we should distinguish two cases here:
# The authorizer failed to reach a concrete answer which is known because the
future containing the answer fails. In this case the agent should retry.
# The authorizer clearly thinks the agent is not allowed to register, the
future is not failed but its value is {{false}}. In this case returning the
{{ShutdownMessage}} still looks like the right solution.
>From the
>[code|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/master.cpp#L6974-L6998]
> I can see that the two cases are merged into one:
{code}
Option authorizationError = None();
if (authorized.isFailed()) { // FIRST CASE, AGENT SHOULD RETRY IN THIS CASE.
authorizationError = "Authorization failure: " + authorized.failure();
} else if (!authorized.get()) { // SECOND CASE, AGENT SHOULD SHUTDOWN.
authorizationError =
"Not authorized to reregister agent providing resources "
"'" + stringify(Resources(slaveInfo.resources())) + "' " +
(principal.isSome()
? "with principal '" + stringify(principal.get()) + "'"
: "without a principal");
}
if (authorizationError.isSome()) {
LOG(WARNING) << "Refusing re-registration of agent " << slaveInfo.id()
<< " at " << pid << " (" << slaveInfo.hostname() << ")"
<< ": " << authorizationError.get();
ShutdownMessage message;
message.set_message(authorizationError.get());
send(pid, message);
slaves.reregistering.erase(slaveInfo.id());
return;
}
{code}
> Master asks agent to shutdown upon auth errors
> --
>
> Key: MESOS-8987
> URL: https://issues.apache.org/jira/browse/MESOS-8987
> Project: Mesos
> Issue Type: Bug
> Components: master, security
>Affects Versions: 1.4.1, 1.5.1, 1.6.0, 1.7.0
>Reporter: Gastón Kleiman
>Priority: Blocker
> Labels: mesosphere
>
> The Mesos master sends a {{ShutdownMessage}} to an agent if there is an
> [authentication|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/master.cpp#L6532-L6543]
> or an
> [authorization|https://github.com/apache/mesos/blob/d733b1031350e03bce443aa287044eb4eee1053a/src/master/master.cpp#L6622-L6633]
> error during agent registration.
>
> Upon receipt of this message, the agent kills alls its tasks and commits
> suicide. This means that transient auth errors can lead to whole agents being
> killed along with it's tasks.
> I think the master should stop sending the {{ShutdownMessage}}s in these
> cases, or at least let the agent retry the registration a few times before
> asking it to shutdown.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8983) SlaveRecoveryTest/0.PingTimeoutDuringRecovery flaky on windows
Alexander Rojas created MESOS-8983:
--
Summary: SlaveRecoveryTest/0.PingTimeoutDuringRecovery flaky on
windows
Key: MESOS-8983
URL: https://issues.apache.org/jira/browse/MESOS-8983
Project: Mesos
Issue Type: Task
Affects Versions: 1.6.0
Reporter: Alexander Rojas
During an unrelated change in a PR, the apache build bot sent the following
error:
{noformat}
@ 7FF71117D888
std::invoke<,process::Future
>,process::ProcessBase *>
@ 7FF71119257B
lambda::internal::Partial<,process::Future
>,std::_Ph<1>
>::invoke_expand<,std::tuple
>,std::_Ph<1> >,st
@ 7FF7110C08BA ),process::Future
>,std::_Ph<1> >,process::ProcessBase *>
@ 7FF711183EBC
std::invoke,process::Future
>,std::_Ph<1> >,process::ProcessBase *>
@ 7FF7110C9F21
),process::Future
>,std::_Ph<1> >,process::ProcessBase *
@ 7FF711236416 process::ProcessBase
*)>::CallableFn,process::Future
>,std::_Ph<1> > >::operator(
@ 7FF712C1A25D process::ProcessBase *)>::operator(
@ 7FF712ACB2F9 process::ProcessBase::consume
@ 7FF712C738CA process::DispatchEvent::consume
@ 7FF70ECE7B07 process::ProcessBase::serve
@ 7FF712AD93B0 process::ProcessManager::resume
@ 7FF712C07371 ??
@ 7FF712B2B130
std::_Invoker_functor::_Call< >
@ 7FF712B8B8E0 std::invoke<
>
@ 7FF712B4076C
std::_LaunchPad
>,std::default_delete > >
> >::_Execute<0>
@ 7FF712C5A60A
std::_LaunchPad
>,std::default_delete > >
> >::_Run
@ 7FF712C45E78
std::_LaunchPad
>,std::default_delete > >
> >::_Go
@ 7FF712C2C3CD std::_Pad::_Call_func
@ 7FFF9BE53428 _register_onexit_function
@ 7FFF9BE53071 _register_onexit_function
@ 7FFFB6391FE4 BaseThreadInitThunk
@ 7FFFB69FF061 RtlUserThreadStart
ll containerizers
I0606 10:25:26.680230 18356 slave.cpp:7158] Recovering executors
I0606 10:25:26.680230 18356 slave.cpp:7182] Sending reconnect request to
executor '3f11d255-bb7b-4e99-967b-055fef95b595' of framework
62cf792a-dc69-4e3c-b54f-d83f98fb9451- at executor(1)@192.10.1.5:55652
I0606 10:25:26.688225 22560 slave.cpp:4984] Received re-registration message
from executor '3f11d255-bb7b-4e99-967b-055fef95b595' of framework
62cf792a-dc69-4e3c-b54f-d83f98fb9451-
I0606 10:25:26.691216 22888 slave.cpp:5901] No pings from master received
within 75secs
F0606 10:25:26.692219 22888 slave.cpp:1249] Check failed: state == DISCONNECTED
|| state == RUNNING || state == TERMINATING RECOVERING
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8961) Output of tasks gets corrupted if task defines the same environment variables as the executor container
Alexander Rojas created MESOS-8961:
--
Summary: Output of tasks gets corrupted if task defines the same
environment variables as the executor container
Key: MESOS-8961
URL: https://issues.apache.org/jira/browse/MESOS-8961
Project: Mesos
Issue Type: Task
Components: containerization
Affects Versions: 1.6.0
Reporter: Alexander Rojas
Assignee: Alexander Rojas
The issue is easily reproducible if one launches a task group and the taks
nested container defines the same set of environment variables as the executor.
In those circumstances, the following [snippet is
activated|https://github.com/apache/mesos/blob/285d82080748cd69044c226950274c7046048c4b/src/slave/containerizer/mesos/launch.cpp#L1057]:
{code}
if (environment.contains(name) && environment[name] != value) {
cout << "Overwriting environment variable '" << name << "'" << endl;
}
{code}
But this is not the only time that this file writes into {{cout}}.
This may be a bad idea because applications which consume the standard output
of a task may end up being corrupted by the container manager output. In these
cases, writing to {{cerr}} should be the right approach.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8821) stout's `os::permissions()` does not follow symlinks
Alexander Rojas created MESOS-8821:
--
Summary: stout's `os::permissions()` does not follow symlinks
Key: MESOS-8821
URL: https://issues.apache.org/jira/browse/MESOS-8821
Project: Mesos
Issue Type: Task
Components: stout
Affects Versions: 1.5.0
Reporter: Alexander Rojas
The {{os::permissions()}} function is implemented using {{stat()}}:
{code}
inline Try permissions(const std::string& path)
{
struct stat status;
if (::stat(path.c_str(), &status) < 0) {
return ErrnoError();
}
return Permissions(status.st_mode);
}
{code}
This works pretty well except in cases where the file given is a symlink, since
symlinks are created with full 777 permissions but defer all the security
questions to the real file, so probably this function should be implemented
using {{lstat()}} instead, unless not following symlinks was the original
intention.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-8677) FaulToleranceTest.ReregisterCompletedFrameworks crashes on macOS
[
https://issues.apache.org/jira/browse/MESOS-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-8677:
--
Assignee: Alexander Rojas
> FaulToleranceTest.ReregisterCompletedFrameworks crashes on macOS
>
>
> Key: MESOS-8677
> URL: https://issues.apache.org/jira/browse/MESOS-8677
> Project: Mesos
> Issue Type: Bug
> Components: test
> Environment: macOS 10.13.3 with LLVM 6.0.0 as well as with Apple LLVM
> version 9.0.0 (clang-900.0.39.2)
>Reporter: Jan Schlicht
>Assignee: Alexander Rojas
>Priority: Major
>
> Here's a {{GLOG_v=1}} run of the test:
> {noformat}
> [ RUN ] FaultToleranceTest.ReregisterCompletedFrameworks
> I0314 14:30:11.240077 2290090816 cluster.cpp:172] Creating default 'local'
> authorizer
> I0314 14:30:11.241261 55140352 master.cpp:463] Master
> 025f775d-9c75-43f6-9ee6-079a605fbf01 (Jenkinss-Mac-mini.local) started on
> 10.0.49.4:54648
> I0314 14:30:11.241287 55140352 master.cpp:465] Flags at startup: --acls=""
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="true" --authenticate_frameworks="true"
> --authenticate_http_frameworks="true" --authenticate_http_readonly="true"
> --authenticate_http_readwrite="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/credentials"
> --filter_gpu_resources="true" --framework_sorter="drf" --help="false"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_framework_authenticators="basic" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_agent_ping_timeouts="5" --max_completed_frameworks="50"
> --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --recovery_agent_removal_limit="100%" --registry="in_memory"
> --registry_fetch_timeout="1mins" --registry_gc_interval="15mins"
> --registry_max_agent_age="2weeks" --registry_max_agent_count="102400"
> --registry_store_timeout="100secs" --registry_strict="false"
> --require_agent_domain="false" --root_submissions="true" --user_sorter="drf"
> --version="false" --webui_dir="/usr/local/share/mesos/webui"
> --work_dir="/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/master"
> --zk_session_timeout="10secs"
> I0314 14:30:11.241439 55140352 master.cpp:514] Master only allowing
> authenticated frameworks to register
> I0314 14:30:11.241447 55140352 master.cpp:520] Master only allowing
> authenticated agents to register
> I0314 14:30:11.241452 55140352 master.cpp:526] Master only allowing
> authenticated HTTP frameworks to register
> I0314 14:30:11.241461 55140352 credentials.hpp:37] Loading credentials for
> authentication from
> '/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/credentials'
> I0314 14:30:11.241678 55140352 master.cpp:570] Using default 'crammd5'
> authenticator
> I0314 14:30:11.241739 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readonly'
> I0314 14:30:11.241824 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readwrite'
> I0314 14:30:11.241873 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-scheduler'
> I0314 14:30:11.241919 55140352 master.cpp:649] Authorization enabled
> I0314 14:30:11.242066 52457472 whitelist_watcher.cpp:77] No whitelist given
> I0314 14:30:11.242079 51920896 hierarchical.cpp:175] Initialized hierarchical
> allocator process
> I0314 14:30:11.243557 52994048 master.cpp:2119] Elected as the leading master!
> I0314 14:30:11.243574 52994048 master.cpp:1678] Recovering from registrar
> I0314 14:30:11.243640 51920896 registrar.cpp:347] Recovering registrar
> I0314 14:30:11.243852 52457472 registrar.cpp:391] Successfully fetched the
> registry (0B) in 190976ns
> I0314 14:30:11.243928 52457472 registrar.cpp:495] Applied 1 operations in
> 28606ns; attempting to update the registry
> I0314 14:30:11.244163 52457472 registrar.cpp:552] Successfully updated the
> registry in 194816ns
> I0314 14:30:11.244222 52457472 registrar.cpp:424] Successfully recovered
> registrar
> I0314 14:30:11.244408 54067200 master.cpp:1792] Recovered 0 agents from the
> registry (155B); allowing 10mins for agents to reregister
> I0314 14:30:11.23 52994048 hierarchical.cpp:213] Skipping recovery of
> hierarchical allocator: nothing to recover
> W0314 14:30:11.247259 2290090816 process.cpp:2805] Attempted to spawn already
> running process files@10.0.49.4:54648
> I0314 14:30:11.247681 2290090816 cluster.c
[jira] [Commented] (MESOS-8677) FaulToleranceTest.ReregisterCompletedFrameworks crashes on macOS
[
https://issues.apache.org/jira/browse/MESOS-8677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400420#comment-16400420
]
Alexander Rojas commented on MESOS-8677:
The issue seem to be using
[{{hashset}}|https://github.com/apache/mesos/blob/master/src/common/http.cpp#L871]
to ensure only one entry per action while constructing the
{{ObjectApprovers}}. We were relying that different iterations would show the
{{hashset}} contents in the same order, but this was not true in macOS (nor can
we ever rely on that), particularly after the {{hashset}} was copied.
Replacing the {{hashset}} for a {{set}} which does have a strong order
guarantee, solves the issue.
> FaulToleranceTest.ReregisterCompletedFrameworks crashes on macOS
>
>
> Key: MESOS-8677
> URL: https://issues.apache.org/jira/browse/MESOS-8677
> Project: Mesos
> Issue Type: Bug
> Components: test
> Environment: macOS 10.13.3 with LLVM 6.0.0 as well as with Apple LLVM
> version 9.0.0 (clang-900.0.39.2)
>Reporter: Jan Schlicht
>Priority: Major
>
> Here's a {{GLOG_v=1}} run of the test:
> {noformat}
> [ RUN ] FaultToleranceTest.ReregisterCompletedFrameworks
> I0314 14:30:11.240077 2290090816 cluster.cpp:172] Creating default 'local'
> authorizer
> I0314 14:30:11.241261 55140352 master.cpp:463] Master
> 025f775d-9c75-43f6-9ee6-079a605fbf01 (Jenkinss-Mac-mini.local) started on
> 10.0.49.4:54648
> I0314 14:30:11.241287 55140352 master.cpp:465] Flags at startup: --acls=""
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="true" --authenticate_frameworks="true"
> --authenticate_http_frameworks="true" --authenticate_http_readonly="true"
> --authenticate_http_readwrite="true" --authenticators="crammd5"
> --authorizers="local"
> --credentials="/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/credentials"
> --filter_gpu_resources="true" --framework_sorter="drf" --help="false"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_framework_authenticators="basic" --initialize_driver_logging="true"
> --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO"
> --max_agent_ping_timeouts="5" --max_completed_frameworks="50"
> --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --recovery_agent_removal_limit="100%" --registry="in_memory"
> --registry_fetch_timeout="1mins" --registry_gc_interval="15mins"
> --registry_max_agent_age="2weeks" --registry_max_agent_count="102400"
> --registry_store_timeout="100secs" --registry_strict="false"
> --require_agent_domain="false" --root_submissions="true" --user_sorter="drf"
> --version="false" --webui_dir="/usr/local/share/mesos/webui"
> --work_dir="/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/master"
> --zk_session_timeout="10secs"
> I0314 14:30:11.241439 55140352 master.cpp:514] Master only allowing
> authenticated frameworks to register
> I0314 14:30:11.241447 55140352 master.cpp:520] Master only allowing
> authenticated agents to register
> I0314 14:30:11.241452 55140352 master.cpp:526] Master only allowing
> authenticated HTTP frameworks to register
> I0314 14:30:11.241461 55140352 credentials.hpp:37] Loading credentials for
> authentication from
> '/private/var/folders/6w/rw03zh013y38ys6cyn8qppf8gn/T/ZyMWb1/credentials'
> I0314 14:30:11.241678 55140352 master.cpp:570] Using default 'crammd5'
> authenticator
> I0314 14:30:11.241739 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readonly'
> I0314 14:30:11.241824 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-readwrite'
> I0314 14:30:11.241873 55140352 http.cpp:957] Creating default 'basic' HTTP
> authenticator for realm 'mesos-master-scheduler'
> I0314 14:30:11.241919 55140352 master.cpp:649] Authorization enabled
> I0314 14:30:11.242066 52457472 whitelist_watcher.cpp:77] No whitelist given
> I0314 14:30:11.242079 51920896 hierarchical.cpp:175] Initialized hierarchical
> allocator process
> I0314 14:30:11.243557 52994048 master.cpp:2119] Elected as the leading master!
> I0314 14:30:11.243574 52994048 master.cpp:1678] Recovering from registrar
> I0314 14:30:11.243640 51920896 registrar.cpp:347] Recovering registrar
> I0314 14:30:11.243852 52457472 registrar.cpp:391] Successfully fetched the
> registry (0B) in 190976ns
> I0314 14:30:11.243928 52457472 registrar.cpp:495] Applied 1 operations in
> 28606ns; attempting to update the registry
> I0314 14:30:11.244163 52457472 registrar.cpp:552] Successfully updated the
> registry in 194816ns
> I0314 14:30:11.244222 5245
[jira] [Commented] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352427#comment-16352427
]
Alexander Rojas commented on MESOS-7076:
I've started a conversation in the [libevent users mailing
list|http://archives.seul.org/libevent/users/], the messages thread are
[here|http://archives.seul.org/libevent/users/Jan-2018/msg2.html] and
continuation if February
[here|http://archives.seul.org/libevent/users/Feb-2018/msg0.html]
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Priority: Major
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
> fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339073#comment-16339073
]
Alexander Rojas commented on MESOS-8413:
Backport commits.
{noformat}
commit af082dc8e31be696dc857374d13359d98b4f4359
Author: Alexander Rojas
AuthorDate: Thu Jan 18 16:20:18 2018 +0100
Commit: Alexander Rojas
CommitDate: Thu Jan 25 11:04:57 2018 +0100
Added SecurePathOrValue for file flags which need not to leak contents.
The default behavior when serializing a flag, if its value comes from
a file (by using the prefix `file://`) is to show the contents of
the file instead of the path to the file.
A common pattern for flags containing sensitive information is to
store the values in a secure file, however the default behavior
of serializing flags causes these values to be leaked when
serializing a subclass of `FlagsBase`.
This patch introduces the structure `SecurePathOrValue` so that
the default behavior is overriden and the path of the flag will
be shown instead of the flag value itself.
Review: https://reviews.apache.org/r/65226
{noformat}
{noformat}
commit e6269bea534dfc512291e7762e34cfdb216eb311
Author: Alexander Rojas
AuthorDate: Thu Jan 11 10:47:04 2018 +0100
Commit: Alexander Rojas
CommitDate: Thu Jan 25 11:05:09 2018 +0100
Mesos flags related to ZooKeeper use SecurePathOrValue.
Up until now the Mesos master flag `--zk` as well as the Mesos agent
flag `--master` would leak ZooKeeper authentication credentials in
both logs and results for the `/flags` endpoint, if the credentials
were part of the configuration url.
This patch prevents this leakage if a user decides to store the
ZooKeeper url in a file and pass the file as a value to the flags
mentioned above (using the preffix `file://`).
Review: https://reviews.apache.org/r/65090
{noformat}
{noformat}
commit b9b626c9b50e7c8ebd376990ab1f41fc5e1a36a7 (HEAD -> 1.5.x, upstream/1.5.x)
Author: Alexander Rojas
AuthorDate: Thu Jan 25 11:49:47 2018 +0100
Commit: Alexander Rojas
CommitDate: Thu Jan 25 11:49:47 2018 +0100
Added MESOS-8413 to CHANGELOG.
{noformat}
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Major
> Labels: mesosphere, security
> Fix For: 1.5.0, 1.6.0
>
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {n
[jira] [Updated] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8413:
---
Fix Version/s: 1.5.0
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Major
> Labels: mesosphere, security
> Fix For: 1.5.0, 1.6.0
>
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
> "recovery_agent_removal_limit": "100%",
> "registry": "replicated_log",
> "registry_fetch_timeout": "1mins",
> "registry_gc_interval": "15mins",
> "registry_max_agent_age": "2weeks",
> "registry_max_agent_count": "102400",
> "registry_store_timeout": "20secs",
> "registry_strict": "false",
> "require_agent_domain": "false",
> "root_submissions": "true",
> "user_sorter": "drf",
> "version": "false",
> "webui_dir": "/home/user/mesos/build/../src/webui",
> "work_dir": "/tmp/user/mesos/master",
> "zk": "zk://user@passwd:127.0.0.1:2181/mesos",
> "zk_session_timeout": "10secs"
> }
> }
> {noformat}
> W
[jira] [Updated] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8413:
---
Fix Version/s: (was: 1.5.0)
1.6.0
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Major
> Labels: mesosphere, security
> Fix For: 1.6.0
>
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
> "recovery_agent_removal_limit": "100%",
> "registry": "replicated_log",
> "registry_fetch_timeout": "1mins",
> "registry_gc_interval": "15mins",
> "registry_max_agent_age": "2weeks",
> "registry_max_agent_count": "102400",
> "registry_store_timeout": "20secs",
> "registry_strict": "false",
> "require_agent_domain": "false",
> "root_submissions": "true",
> "user_sorter": "drf",
> "version": "false",
> "webui_dir": "/home/user/mesos/build/../src/webui",
> "work_dir": "/tmp/user/mesos/master",
> "zk": "zk://user@passwd:127.0.0.1:2181/mesos",
> "zk_session_timeout": "10secs"
[jira] [Assigned] (MESOS-8434) Cleanup Authorization logic in master and agent
[
https://issues.apache.org/jira/browse/MESOS-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-8434:
--
Assignee: Alexander Rojas
> Cleanup Authorization logic in master and agent
> ---
>
> Key: MESOS-8434
> URL: https://issues.apache.org/jira/browse/MESOS-8434
> Project: Mesos
> Issue Type: Improvement
> Components: agent, master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Major
> Labels: mesosphere, security
>
> During MesosCon EU 2017, [~benjaminhindman] came up with a neat abstraction
> called [{{ObjectApprovers}}|https://reviews.apache.org/r/63258/] which go a
> long way into streamlining and unifying the authorization used within mesos.
> However, these patches became stale afterwards.
> Given the benefits of such logic, we should really make the effort to land
> these patches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-7826) XSS in JSONP parameter
[
https://issues.apache.org/jira/browse/MESOS-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334043#comment-16334043
]
Alexander Rojas commented on MESOS-7826:
We don't have any plans to take into this anytime soon.
> XSS in JSONP parameter
> --
>
> Key: MESOS-7826
> URL: https://issues.apache.org/jira/browse/MESOS-7826
> Project: Mesos
> Issue Type: Improvement
> Components: json api
> Environment: Running as part of DC/OS in a docker container.
>Reporter: Vincent Ruijter
>Priority: Critical
>
> It is possible to inject arbitrary content into a server request. Take into
> account the following url:
> https://xxx.xxx.com/mesos/master/state?jsonp=var+oShell+%3d+new+ActiveXObject("WScript.Shell")%3boShell.Run("calc.exe",+1)%3b
> This will result in the following request:
> {code:html}
> GET
> /mesos/master/state?jsonp=var+oShell+%3d+new+ActiveXObject("WScript.Shell")%3boShell.Run("calc.exe",+1)%3b
> HTTP/1.1
> Host: xxx.xxx.com
> User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101
> Firefox/54.0
> Accept: */*
> Accept-Language: en-US,en;q=0.5
> [...SNIP...]
> {code}
> The server response:
> {code:html}
> HTTP/1.1 200 OK
> Server: openresty/1.9.15.1
> Date: Tue, 25 Jul 2017 09:04:31 GMT
> Content-Type: text/javascript
> Content-Length: 1411637
> Connection: close
> var oShell = new ActiveXObject("WScript.Shell");oShell.Run("calc.exe",
> 1);({"version":"1.2.1","git_sha":"f219b2e4f6265c0b6c4d826a390b67fe9d5e1097","build_date":"2017-06-01
> 19:16:40","build_time":149634
> [...SNIP...]
> {code}
> On Internet Explorer this will trigger a file download, and when executing
> the file (state.js), it will pop-up a calculator. It's my recommendation to
> apply input validation on this parameter, to prevent abuse.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332124#comment-16332124
]
Alexander Rojas edited comment on MESOS-7076 at 1/19/18 11:19 AM:
--
After lots of debugging I found that the issues in both linux and macos are
different.
In linux, the server breaks while [accepting a
connection|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
while the client breaks during the {{connect()}} step. The issues are similar,
the variable {{events}} is marked with both {{BEV_EVENT_ERROR}} and
{{BEV_EVENT_READ}} (which is curious for the client) as mentioned before,
neither libevent nor openssl mention any error having happened, however the
socket never properly transitions from a listening state to an accepted state.
In macos the connection succeeds and the client is able to send data to the
server, however the
[{{recv_callback()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L236]
is never called by the server, indicating that the data is never received.
As a sidenote, the output of the client, which runs as subprocess is always
discarded by the tests, which could cause some confusion for anyone trying to
debug this issue. To get any output, one needs to change the std out from
[{{process::Subprocess::PIPE()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/include/process/ssl/gtest.hpp#L372]
to {{process::subprocess::FD(STDOUT_FILENO)}}.
was (Author: arojas):
After lots of debugging I found that the issues in both linux and macos are
different.
In linux, the server breaks while [accepting a
connection|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
while the client breaks during the {{connect()}} step. The issues are similar,
the variable {{events}} is marked with both `BEV_EVENT_ERROR` and
`BEV_EVENT_READ` (which is curious for the client) as mentioned before,
neither libevent nor openssl mention any error having happened, however the
socket never properly transitions from a listening state to an accepted state.
In macos the connection succeeds and the client is able to send data to the
server, however the
[{{recv_callback()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L236]
is never called by the server, indicating that the data is never received.
As a sidenote, the output of the client, which runs as subprocess is always
discarded by the tests, which could cause some confusion for anyone trying to
debug this issue. To get any output, one needs to change the std out from
[{{process::Subprocess::PIPE()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/include/process/ssl/gtest.hpp#L372]
to {{process::subprocess::FD(STDOUT_FILENO)}}.
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Priority: Major
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
> fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer c
[jira] [Commented] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332124#comment-16332124
]
Alexander Rojas commented on MESOS-7076:
After lots of debugging I found that the issues in both linux and macos are
different.
In linux, the server breaks while [accepting a
connection|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
while the client breaks during the {{connect()}} step. The issues are similar,
the variable {{events}} is marked with both `BEV_EVENT_ERROR` and
`BEV_EVENT_READ` (which is curious for the client) as mentioned before,
neither libevent nor openssl mention any error having happened, however the
socket never properly transitions from a listening state to an accepted state.
In macos the connection succeeds and the client is able to send data to the
server, however the
[{{recv_callback()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L236]
is never called by the server, indicating that the data is never received.
As a sidenote, the output of the client, which runs as subprocess is always
discarded by the tests, which could cause some confusion for anyone trying to
debug this issue. To get any output, one needs to change the std out from
[{{process::Subprocess::PIPE()}}|https://github.com/apache/mesos/blob/19cf3068b81355818a9c4cdb797883f2b73cdbce/3rdparty/libprocess/include/process/ssl/gtest.hpp#L372]
to {{process::subprocess::FD(STDOUT_FILENO)}}.
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Priority: Major
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
> fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7076:
--
Assignee: (was: Alexander Rojas)
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Priority: Major
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
> fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7076:
---
Description:
Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
--enable-ssl}} on an operating system using libevent 2.1.8, SSL related tests
fail like
{noformat}
[ RUN ] SSLTest.SSLSocket
I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
Failed to wait 15secs for Socket(socket.get()).recv()
[ FAILED ] SSLTest.SSLSocket (15196 ms)
{noformat}
Tests failing are
{noformat}
SSLTest.SSLSocket
SSLTest.NoVerifyBadCA
SSLTest.VerifyCertificate
SSLTest.ProtocolMismatch
SSLTest.ECDHESupport
SSLTest.PeerAddress
SSLTest.HTTPSGet
SSLTest.HTTPSPost
SSLTest.SilentSocket
SSLTest.ShutdownThenSend
SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
{noformat}
was:
Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
--enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew, SSL
related tests fail like
{noformat}
[ RUN ] SSLTest.SSLSocket
I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
Failed to wait 15secs for Socket(socket.get()).recv()
[ FAILED ] SSLTest.SSLSocket (15196 ms)
{noformat}
Tests failing are
{noformat}
SSLTest.SSLSocket
SSLTest.NoVerifyBadCA
SSLTest.VerifyCertificate
SSLTest.ProtocolMismatch
SSLTest.ECDHESupport
SSLTest.PeerAddress
SSLTest.HTTPSGet
SSLTest.HTTPSPost
SSLTest.SilentSocket
SSLTest.ShutdownThenSend
SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
{noformat}
> libprocess tests fail when using libevent 2.1
[jira] [Updated] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7076:
---
Description:
Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
--enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew, SSL
related tests fail like
{noformat}
[ RUN ] SSLTest.SSLSocket
I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
Failed to wait 15secs for Socket(socket.get()).recv()
[ FAILED ] SSLTest.SSLSocket (15196 ms)
{noformat}
Tests failing are
{noformat}
SSLTest.SSLSocket
SSLTest.NoVerifyBadCA
SSLTest.VerifyCertificate
SSLTest.ProtocolMismatch
SSLTest.ECDHESupport
SSLTest.PeerAddress
SSLTest.HTTPSGet
SSLTest.HTTPSPost
SSLTest.SilentSocket
SSLTest.ShutdownThenSend
SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
{noformat}
was:
Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
--enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew, SSL
related tests fail like
{noformat}
[ RUN ] SSLTest.SSLSocket
I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is unspecified!
NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
certificate if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
Failed to wait 15secs for Socket(socket.get()).recv()
[ FAILED ] SSLTest.SSLSocket (15196 ms)
{noformat}
Tests failing are {{SSLTest.SSLSocket}}, {{SSLTest.NoVerifyBadCA}},
{{SSLTest.VerifyCertificate}}, {{SSLTest.ProtocolMismatch}},
{{SSLTest.HTTPSGet}}, {{SSLTest.HTTPSPost}}, {{SSLTest.SilentSocket}} (hangs),
{{HTTPTest.Endpoints}}, {{HTTPTest.EndpointsHelp}}, {{HTTPTest.Get}},
{{HTTPTest.NestedGet}}, {{HTTPTest.StreamingGetComplete}},
{{HTTPTest.StreamingGetFailure}}, {{HTTPTest.Post}}, {{HTTPTest.Delete}},
{{HTTPTest.Request}}, {{NetSocketTest.EOFBeforeRecv}},
{{NetSocketTest.EOFAfterRecv}}, {{SSLTest.BasicSameProcess}},
{{SSLTest.BasicSameProcessUnix}}, and {{SSLTest.RequireCertificate}}. It
hasn't been tested if Linux builds are affected as well.
> libprocess tests fail when using libevent 2.1
[jira] [Commented] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322322#comment-16322322
]
Alexander Rojas commented on MESOS-7076:
Initial research shows that in the libevent callback the [event bit error is
marked as
BEV_EVENT_ERROR|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
however neither
{{[EVUTIL_SOCKET_ERROR()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1178]}}
nor
{{[bufferevent_get_openssl_error()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1182]}}
report any failure.
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Assignee: Alexander Rojas
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew,
> SSL related tests fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are {{SSLTest.SSLSocket}}, {{SSLTest.NoVerifyBadCA}},
> {{SSLTest.VerifyCertificate}}, {{SSLTest.ProtocolMismatch}},
> {{SSLTest.HTTPSGet}}, {{SSLTest.HTTPSPost}}, {{SSLTest.SilentSocket}}
> (hangs), {{HTTPTest.Endpoints}}, {{HTTPTest.EndpointsHelp}},
> {{HTTPTest.Get}}, {{HTTPTest.NestedGet}}, {{HTTPTest.StreamingGetComplete}},
> {{HTTPTest.StreamingGetFailure}}, {{HTTPTest.Post}}, {{HTTPTest.Delete}},
> {{HTTPTest.Request}}, {{NetSocketTest.EOFBeforeRecv}},
> {{NetSocketTest.EOFAfterRecv}}, {{SSLTest.BasicSameProcess}},
> {{SSLTest.BasicSameProcessUnix}}, and {{SSLTest.RequireCertificate}}. It
> hasn't been tested if Linux builds are affected as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322321#comment-16322321
]
Alexander Rojas commented on MESOS-7076:
Note that this issue also affects newer versions of linux, see MESOS-8271 for
details.
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Assignee: Alexander Rojas
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew,
> SSL related tests fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are {{SSLTest.SSLSocket}}, {{SSLTest.NoVerifyBadCA}},
> {{SSLTest.VerifyCertificate}}, {{SSLTest.ProtocolMismatch}},
> {{SSLTest.HTTPSGet}}, {{SSLTest.HTTPSPost}}, {{SSLTest.SilentSocket}}
> (hangs), {{HTTPTest.Endpoints}}, {{HTTPTest.EndpointsHelp}},
> {{HTTPTest.Get}}, {{HTTPTest.NestedGet}}, {{HTTPTest.StreamingGetComplete}},
> {{HTTPTest.StreamingGetFailure}}, {{HTTPTest.Post}}, {{HTTPTest.Delete}},
> {{HTTPTest.Request}}, {{NetSocketTest.EOFBeforeRecv}},
> {{NetSocketTest.EOFAfterRecv}}, {{SSLTest.BasicSameProcess}},
> {{SSLTest.BasicSameProcessUnix}}, and {{SSLTest.RequireCertificate}}. It
> hasn't been tested if Linux builds are affected as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (MESOS-7076) libprocess tests fail when using libevent 2.1.8
[
https://issues.apache.org/jira/browse/MESOS-7076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7076:
--
Assignee: Alexander Rojas
> libprocess tests fail when using libevent 2.1.8
> ---
>
> Key: MESOS-7076
> URL: https://issues.apache.org/jira/browse/MESOS-7076
> Project: Mesos
> Issue Type: Bug
> Components: libprocess, test
> Environment: macOS 10.12.3, libevent 2.1.8 (installed via Homebrew)
>Reporter: Jan Schlicht
>Assignee: Alexander Rojas
>
> Running {{libprocess-tests}} on Mesos compiled with {{--enable-libevent
> --enable-ssl}} on macOS with the libevent 2.1.8 installed through homebrew,
> SSL related tests fail like
> {noformat}
> [ RUN ] SSLTest.SSLSocket
> I0207 15:20:46.017881 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.017904 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.017918 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.017923 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0207 15:20:46.033001 2528580544 openssl.cpp:419] CA file path is
> unspecified! NOTE: Set CA file path with LIBPROCESS_SSL_CA_FILE=
> I0207 15:20:46.033179 2528580544 openssl.cpp:424] CA directory path
> unspecified! NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
> I0207 15:20:46.033196 2528580544 openssl.cpp:429] Will not verify peer
> certificate!
> NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
> I0207 15:20:46.033201 2528580544 openssl.cpp:435] Will only verify peer
> certificate if presented!
> NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate
> verification
> ../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:257: Failure
> Failed to wait 15secs for Socket(socket.get()).recv()
> [ FAILED ] SSLTest.SSLSocket (15196 ms)
> {noformat}
> Tests failing are {{SSLTest.SSLSocket}}, {{SSLTest.NoVerifyBadCA}},
> {{SSLTest.VerifyCertificate}}, {{SSLTest.ProtocolMismatch}},
> {{SSLTest.HTTPSGet}}, {{SSLTest.HTTPSPost}}, {{SSLTest.SilentSocket}}
> (hangs), {{HTTPTest.Endpoints}}, {{HTTPTest.EndpointsHelp}},
> {{HTTPTest.Get}}, {{HTTPTest.NestedGet}}, {{HTTPTest.StreamingGetComplete}},
> {{HTTPTest.StreamingGetFailure}}, {{HTTPTest.Post}}, {{HTTPTest.Delete}},
> {{HTTPTest.Request}}, {{NetSocketTest.EOFBeforeRecv}},
> {{NetSocketTest.EOFAfterRecv}}, {{SSLTest.BasicSameProcess}},
> {{SSLTest.BasicSameProcessUnix}}, and {{SSLTest.RequireCertificate}}. It
> hasn't been tested if Linux builds are affected as well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-8434) Cleanup Authorization logic in master and agent
Alexander Rojas created MESOS-8434:
--
Summary: Cleanup Authorization logic in master and agent
Key: MESOS-8434
URL: https://issues.apache.org/jira/browse/MESOS-8434
Project: Mesos
Issue Type: Improvement
Components: agent, master
Affects Versions: 1.4.1
Reporter: Alexander Rojas
During MesosCon EU 2017, [~benjaminhindman] came up with a neat abstraction
called [{{ObjectApprovers}}|https://reviews.apache.org/r/63258/] which go a
long way into streamlining and unifying the authorization used within mesos.
However, these patches became stale afterwards.
Given the benefits of such logic, we should really make the effort to land
these patches.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-8271) Libprocess SSL test fail in Mesos 17.10
[
https://issues.apache.org/jira/browse/MESOS-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322240#comment-16322240
]
Alexander Rojas commented on MESOS-8271:
Closing as duplicate of MESOS-7076
> Libprocess SSL test fail in Mesos 17.10
> ---
>
> Key: MESOS-8271
> URL: https://issues.apache.org/jira/browse/MESOS-8271
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 1.5.0
> Environment: Mesos Configuration:
> {noformat}
> ../configure --disable-java --disable-python --enable-libevent --enable-ssl
> {noformat}
> Ubuntu 17.10
> {noformat}
> $ uname -r
> Linux localhost 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:04:08 UTC 2017
> x86_64 x86_64 x86_64 GNU/Linux
> {noformat}
> {noformat}
> $ sudo apt libevent-dev
> Package: libevent-dev
> Version: 2.1.8-stable-4
> Priority: optional
> Section: libdevel
> Source: libevent
> Origin: Ubuntu
> Maintainer: Ubuntu Developers
> Original-Maintainer: Anibal Monsalve Salazar
> Bugs: https://bugs.launchpad.net/ubuntu/+filebug
> Installed-Size: 1,694 kB
> Depends: libevent-2.1-6 (= 2.1.8-stable-4), libevent-core-2.1-6 (=
> 2.1.8-stable-4), libevent-extra-2.1-6 (= 2.1.8-stable-4),
> libevent-pthreads-2.1-6 (= 2.1.8-stable-4), libevent-openssl-2.1-6 (=
> 2.1.8-stable-4)
> Homepage: http://libevent.org/
> Supported: 9m
> Download-Size: 262 kB
> APT-Manual-Installed: yes
> APT-Sources: http://de.archive.ubuntu.com/ubuntu artful/main amd64 Packages
> Description: Asynchronous event notification library (development files)
> Libevent is an asynchronous event notification library that provides a
> mechanism to execute a callback function when a specific event occurs
> on a file descriptor or after a timeout has been reached.
> .
> This package includes development files for compiling against libevent.
> {noformat}
> {noformat}
> $ sudo apt libssl-dev
> Package: libssl-dev
> Version: 1.0.2g-1ubuntu13.2
> Priority: optional
> Section: libdevel
> Source: openssl
> Origin: Ubuntu
> Maintainer: Ubuntu Developers
> Original-Maintainer: Debian OpenSSL Team
>
> Bugs: https://bugs.launchpad.net/ubuntu/+filebug
> Installed-Size: 7,216 kB
> Depends: libssl1.0.0 (= 1.0.2g-1ubuntu13.2), zlib1g-dev
> Recommends: libssl-doc
> Supported: 9m
> Download-Size: 1,357 kB
> APT-Manual-Installed: yes
> APT-Sources: http://de.archive.ubuntu.com/ubuntu artful-updates/main amd64
> Packages
> Description: Secure Sockets Layer toolkit - development files
> This package is part of the OpenSSL project's implementation of the SSL
> and TLS cryptographic protocols for secure communication over the
> Internet.
> .
> It contains development libraries, header files, and manpages for libssl
> {noformat}
>Reporter: Alexander Rojas
>Assignee: Greg Mann
> Labels: mesosphere
>
> The following tests constantly fail when building Mesos in Ubuntu 17.10:
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
> The interesting [log
> line|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1188]
> is:
> {noformat}
> I1128 10:08:48.235530 26158 libevent_ssl_socket.cpp:1188] Socket error:
> error::lib(0):func(0):reason(0)
> {noformat}
> After investigating some more one can notices that in the libevent callback
> the [event bit error is marked as
> BEV_EVENT_ERROR|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
> however neither
> {{[EVUTIL_SOCKET_ERROR()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1178]}}
> nor
> {{[bufferevent_get_openssl_error()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1182]}}
> report any failure.
> Googling for the issue reports at least someone who suffered the same problem
> some years ago but there are no clues as to how it was fixed.
> Full test log output:
> {noformat}
> WARNING: Logging before InitGoogleLogging() is written to STDER
[jira] [Commented] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322039#comment-16322039
]
Alexander Rojas commented on MESOS-8413:
[r/65090/|https://reviews.apache.org/r/65090/]: Added specific flag loader for
zookeeper urls preventing password leaks.
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, security
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
> "recovery_agent_removal_limit": "100%",
> "registry": "replicated_log",
> "registry_fetch_timeout": "1mins",
> "registry_gc_interval": "15mins",
> "registry_max_agent_age": "2weeks",
> "registry_max_agent_count": "102400",
> "registry_store_timeout": "20secs",
> "registry_strict": "false",
> "require_agent_domain": "false",
> "root_submissions": "true",
> "user_sorter": "drf",
> "version": "false",
> "webui_dir": "/home/user/mesos/build/../src/webui",
> "work_dir": "/tmp/user/mesos/master",
> "zk": "zk://user@passwd:127.
[jira] [Assigned] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-8413:
--
Assignee: Alexander Rojas
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, security
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
> "recovery_agent_removal_limit": "100%",
> "registry": "replicated_log",
> "registry_fetch_timeout": "1mins",
> "registry_gc_interval": "15mins",
> "registry_max_agent_age": "2weeks",
> "registry_max_agent_count": "102400",
> "registry_store_timeout": "20secs",
> "registry_strict": "false",
> "require_agent_domain": "false",
> "root_submissions": "true",
> "user_sorter": "drf",
> "version": "false",
> "webui_dir": "/home/user/mesos/build/../src/webui",
> "work_dir": "/tmp/user/mesos/master",
> "zk": "zk://user@passwd:127.0.0.1:2181/mesos",
> "zk_session_timeout": "10secs"
> }
> }
> {noformat}
> Which leads to having no effective way to prevent the pa
[jira] [Commented] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316503#comment-16316503
]
Alexander Rojas commented on MESOS-8413:
After doing some research, I can conclude this is a bug. The other flags which
store passwords ({{--credentials}} in master and {{--credential}} and
{{--credentials}} in the agent) all of them have the behavior where, if a file
is given, the file path will be shown instead of the contents of the file.
I see no reason why {{--zk}} should behave differently.
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
> Labels: mesosphere, security
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
> "recovery_agent_removal_limit": "100%",
> "registry": "replicated_log",
> "registry_fetch_timeout": "1mins",
> "registry_gc_interval": "15mins",
> "registry_max_agent_age": "2weeks",
> "registry_max_agent_count": "102400",
> "registry_store_timeout": "20secs",
> "registry_strict": "false",
> "require_agent_domain": "false",
> "root_submissions": "true",
>
[jira] [Comment Edited] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
[
https://issues.apache.org/jira/browse/MESOS-8413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316503#comment-16316503
]
Alexander Rojas edited comment on MESOS-8413 at 1/8/18 3:46 PM:
After doing some research, I can conclude this is a bug. The other flags which
store passwords ({{\-\-credentials}} in master and {{\-\-credential}} and
{{\-\-credentials}} in the agent) all of them have the behavior where, if a
file is given, the file path will be shown instead of the contents of the file.
I see no reason why {{\-\-zk}} should behave differently.
was (Author: arojas):
After doing some research, I can conclude this is a bug. The other flags which
store passwords ({{--credentials}} in master and {{--credential}} and
{{--credentials}} in the agent) all of them have the behavior where, if a file
is given, the file path will be shown instead of the contents of the file.
I see no reason why {{--zk}} should behave differently.
> Zookeeper configuration passwords are shown in clear text
> -
>
> Key: MESOS-8413
> URL: https://issues.apache.org/jira/browse/MESOS-8413
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.1
>Reporter: Alexander Rojas
> Labels: mesosphere, security
>
> No matter how one configures mesos, either by passing the ZooKeeper flags in
> the command line or using a file, as follows:
> {noformat}
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log
> --zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
> {noformat}
> {noformat}
> echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
> /tmp/${USER}/mesos/zk_config.txt
> ./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
> --log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
> {noformat}
> both the logs and the results of the {{/flags}} endpoint will resolve to the
> contents of the flags, i.e.:
> {noformat}
> I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
> --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate_agents="false" --authenticate_frameworks="false"
> --authenticate_http_frameworks="false" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticators="crammd5"
> --authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --log_auto_initialize="true"
> --log_dir="/tmp/user/mesos/master/log" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
> --quorum="1" --recovery_agent_removal_limit="100%"
> --registry="replicated_log" --registry_fetch_timeout="1mins"
> --registry_gc_interval="15mins" --registry_max_agent_age="2weeks"
> --registry_max_agent_count="102400" --registry_store_timeout="20secs"
> --registry_strict="false" --require_agent_domain="false"
> --root_submissions="true" --user_sorter="drf" --version="false"
> --webui_dir="/home/user/mesos/build/../src/webui"
> --work_dir="/tmp/user/mesos/master"
> --zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
> {noformat}
> {noformat}
> HTTP/1.1 200 OK
> Content-Encoding: gzip
> Content-Length: 591
> Content-Type: application/json
> Date: Mon, 08 Jan 2018 15:12:53 GMT
> {
> "flags": {
> "agent_ping_timeout": "15secs",
> "agent_reregister_timeout": "10mins",
> "allocation_interval": "1secs",
> "allocator": "HierarchicalDRF",
> "authenticate_agents": "false",
> "authenticate_frameworks": "false",
> "authenticate_http_frameworks": "false",
> "authenticate_http_readonly": "false",
> "authenticate_http_readwrite": "false",
> "authenticators": "crammd5",
> "authorizers": "local",
> "filter_gpu_resources": "true",
> "framework_sorter": "drf",
> "help": "false",
> "hostname_lookup": "true",
> "http_authenticators": "basic",
> "initialize_driver_logging": "true",
> "log_auto_initialize": "true",
> "log_dir": "/tmp/user/mesos/master/log",
> "logbufsecs": "0",
> "logging_level": "INFO",
> "max_agent_ping_timeouts": "5",
> "max_completed_frameworks": "50",
> "max_completed_tasks_per_framework": "1000",
> "max_unreachable_tasks_per_framework": "1000",
> "port": "5050",
> "quiet": "false",
> "quorum": "1",
>
[jira] [Created] (MESOS-8413) Zookeeper configuration passwords are shown in clear text
Alexander Rojas created MESOS-8413:
--
Summary: Zookeeper configuration passwords are shown in clear text
Key: MESOS-8413
URL: https://issues.apache.org/jira/browse/MESOS-8413
Project: Mesos
Issue Type: Bug
Components: master
Affects Versions: 1.4.1
Reporter: Alexander Rojas
No matter how one configures mesos, either by passing the ZooKeeper flags in
the command line or using a file, as follows:
{noformat}
./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
--log_dir=/tmp/$USER/mesos/master/log
--zk=zk://${zk_username}:${zk_password}@${zk_addr}/mesos --quorum=1
{noformat}
{noformat}
echo "zk://${zk_username}:${zk_password}@${zk_addr}/mesos" >
/tmp/${USER}/mesos/zk_config.txt
./bin/mesos-master.sh --work_dir=/tmp/$USER/mesos/master
--log_dir=/tmp/$USER/mesos/master/log --zk=/tmp/${USER}/mesos/zk_config.txt
{noformat}
both the logs and the results of the {{/flags}} endpoint will resolve to the
contents of the flags, i.e.:
{noformat}
I0108 10:12:50.387522 28579 master.cpp:458] Flags at startup:
--agent_ping_timeout="15secs" --agent_reregister_timeout="10mins"
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate_agents="false" --authenticate_frameworks="false"
--authenticate_http_frameworks="false" --authenticate_http_readonly="false"
--authenticate_http_readwrite="false" --authenticators="crammd5"
--authorizers="local" --filter_gpu_resources="true" --framework_sorter="drf"
--help="false" --hostname_lookup="true" --http_authenticators="basic"
--initialize_driver_logging="true" --log_auto_initialize="true"
--log_dir="/tmp/user/mesos/master/log" --logbufsecs="0" --logging_level="INFO"
--max_agent_ping_timeouts="5" --max_completed_frameworks="50"
--max_completed_tasks_per_framework="1000"
--max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false"
--quorum="1" --recovery_agent_removal_limit="100%" --registry="replicated_log"
--registry_fetch_timeout="1mins" --registry_gc_interval="15mins"
--registry_max_agent_age="2weeks" --registry_max_agent_count="102400"
--registry_store_timeout="20secs" --registry_strict="false"
--require_agent_domain="false" --root_submissions="true" --user_sorter="drf"
--version="false" --webui_dir="/home/user/mesos/build/../src/webui"
--work_dir="/tmp/user/mesos/master"
--zk="zk://user@passwd:127.0.0.1:2181/mesos" --zk_session_timeout="10secs"
{noformat}
{noformat}
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Length: 591
Content-Type: application/json
Date: Mon, 08 Jan 2018 15:12:53 GMT
{
"flags": {
"agent_ping_timeout": "15secs",
"agent_reregister_timeout": "10mins",
"allocation_interval": "1secs",
"allocator": "HierarchicalDRF",
"authenticate_agents": "false",
"authenticate_frameworks": "false",
"authenticate_http_frameworks": "false",
"authenticate_http_readonly": "false",
"authenticate_http_readwrite": "false",
"authenticators": "crammd5",
"authorizers": "local",
"filter_gpu_resources": "true",
"framework_sorter": "drf",
"help": "false",
"hostname_lookup": "true",
"http_authenticators": "basic",
"initialize_driver_logging": "true",
"log_auto_initialize": "true",
"log_dir": "/tmp/user/mesos/master/log",
"logbufsecs": "0",
"logging_level": "INFO",
"max_agent_ping_timeouts": "5",
"max_completed_frameworks": "50",
"max_completed_tasks_per_framework": "1000",
"max_unreachable_tasks_per_framework": "1000",
"port": "5050",
"quiet": "false",
"quorum": "1",
"recovery_agent_removal_limit": "100%",
"registry": "replicated_log",
"registry_fetch_timeout": "1mins",
"registry_gc_interval": "15mins",
"registry_max_agent_age": "2weeks",
"registry_max_agent_count": "102400",
"registry_store_timeout": "20secs",
"registry_strict": "false",
"require_agent_domain": "false",
"root_submissions": "true",
"user_sorter": "drf",
"version": "false",
"webui_dir": "/home/user/mesos/build/../src/webui",
"work_dir": "/tmp/user/mesos/master",
"zk": "zk://user@passwd:127.0.0.1:2181/mesos",
"zk_session_timeout": "10secs"
}
}
{noformat}
Which leads to having no effective way to prevent the passwords to be shown if
someone can get the hands in either of the previous methods.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MESOS-8283) Operator API `SUBSCRIBE` call doesn't fully respect acls
[
https://issues.apache.org/jira/browse/MESOS-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8283:
---
Description:
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call,
the initial response is properly authorized (with filter in the state). The
same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a
user has no right to see them he won't receive the updates, {{AGENT_ADDED}}
seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and
{{FRAMEWORK_UPDATED}} events.
I didn't {{AGENT_REMOVED}}
In order to test I used the following acls:
{code:javascript}
{
"permissive": true,
"view_roles" : [
{
"principals" : { "type" : "ANY" },
"roles" : { "values" : ["*"] }
},
{
"principals" : { "values" : ["hal-9000"] },
"roles" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"roles" : { "type" : "NONE" }
}
],
"view_framework": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"view_tasks": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"register_agent": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
]
}
{code}
the following credential files:
{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}
And launch a master and an agent as follows:
{noformat}
./mesos-master.sh \
--work_dir=/tmp/$USER/mesos/master \
--log_dir=/tmp/$USER/mesos/master/log \
--ip=${MASTER_IP} \
--agent_ping_timeout=5secs \
--max_agent_ping_timeouts=2 \
--agent_removal_rate_limit=1/1secs \
--http_framework_authenticators=basic \
--authenticate_http \
--authenticate_http_frameworks \
--acls=$HOME/testing/acls.json \
--credentials=$HOME/testing/credentials.txt
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent \
--log_dir=/tmp/$USER/mesos/agent/log \
--containerizers=mesos,docker \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/Documents/workspace/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
Once the master and agent are running, use the following script (user this
[recordio|https://github.com/dcos/dcos-test-utils/blob/aa22cf1e6ffcf0a375e7e424a02669f51b0adbee/dcos_test_utils/recordio.py]
file):
{code}
#! /usr/bin/env python3
import json
import recordio
import requests
import sys
from contextlib import closing
from typing import Dict
MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'
def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
base_url = 'http://{}:5050'.format(host)
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Connection': 'close',
}
decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('hal-9000', 'hal-9000'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
events = [] # type: ignore
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] != 'HEARTBEAT':
print('got event "' + record['type'])
events.append(record['type'])
# It takes at least 10 mins for an agent to be marked as gone.
if len(events) >= 7:
print('got the following events "' + '", "'.join(events) + '"')
break
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('glados', 'glados'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
print('It wasn\'t supposed to get "' + record['type'] + '"')
def main(args: Dict[str, str]) -> None:
test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
if __name__ == '__main__':
main(sys.argv)
{code}
Finally while the machine is running you do the following:
1. Launch an extra agent:
{noform
[jira] [Updated] (MESOS-8283) Operator API `SUBSCRIBE` call doesn't fully respect acls
[
https://issues.apache.org/jira/browse/MESOS-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8283:
---
Description:
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call,
the initial response is properly authorized (with filter in the state). The
same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a
user has no right to see them he won't receive the updates, {{AGENT_ADDED}}
seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and
{{FRAMEWORK_UPDATED}} events.
I didn't {{AGENT_REMOVED}}
In order to test I used the following acls:
{code:javascript}
{
"permissive": true,
"view_roles" : [
{
"principals" : { "type" : "ANY" },
"roles" : { "values" : ["*"] }
},
{
"principals" : { "values" : ["hal-9000"] },
"roles" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"roles" : { "type" : "NONE" }
}
],
"view_framework": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"view_tasks": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"register_agent": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
]
}
{code}
the following credential files:
{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}
And launch a master and an agent as follows:
{noformat}
./mesos-master.sh \
--work_dir=/tmp/$USER/mesos/master \
--log_dir=/tmp/$USER/mesos/master/log \
--ip=${MASTER_IP} \
--agent_ping_timeout=5secs \
--max_agent_ping_timeouts=2 \
--agent_removal_rate_limit=1/1secs \
--http_framework_authenticators=basic \
--authenticate_http \
--authenticate_http_frameworks \
--acls=$HOME/testing/acls.json \
--credentials=$HOME/testing/credentials.txt
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent \
--log_dir=/tmp/$USER/mesos/agent/log \
--containerizers=mesos,docker \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/Documents/workspace/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
Once the master and agent are running, use the following script:
{code}
#! /usr/bin/env python3
import json
import recordio
import requests
import sys
from contextlib import closing
from typing import Dict
MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'
def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
base_url = 'http://{}:5050'.format(host)
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Connection': 'close',
}
decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('hal-9000', 'hal-9000'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
events = [] # type: ignore
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] != 'HEARTBEAT':
print('got event "' + record['type'])
events.append(record['type'])
# It takes at least 10 mins for an agent to be marked as gone.
if len(events) >= 7:
print('got the following events "' + '", "'.join(events) + '"')
break
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('glados', 'glados'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
print('It wasn\'t supposed to get "' + record['type'] + '"')
def main(args: Dict[str, str]) -> None:
test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
if __name__ == '__main__':
main(sys.argv)
{code}
Finally while the machine is running you do the following:
1. Launch an extra agent:
{noformat}
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent-2 \
--log_dir=/tmp/$USER/mesos/agent-2/log \
--containerizers=mesos
[jira] [Created] (MESOS-8283) Operator API `SUBSCRIBE` call doesn't fully respect acls
Alexander Rojas created MESOS-8283:
--
Summary: Operator API `SUBSCRIBE` call doesn't fully respect acls
Key: MESOS-8283
URL: https://issues.apache.org/jira/browse/MESOS-8283
Project: Mesos
Issue Type: Bug
Components: HTTP API, master
Affects Versions: 1.4.0
Reporter: Alexander Rojas
When an operator subscribes to the event stream using the {{SUBSCRIBE}} call,
the initial response is properly authorized (with filter in the state). The
same holds true for the events {{TASK_ADDED}} and {{TASK_UPDATED}} which, if a
user has no right to see them he won't receive the updates, {{AGENT_ADDED}}
seems to be respected too. However any user will get {{FRAMEWORK_ADDED}} and
{{FRAMEWORK_UPDATED}} events.
I didn't {{AGENT_REMOVED}}
In order to test I used the following acls:
{code:javascript}
{
"permissive": true,
"view_roles" : [
{
"principals" : { "type" : "ANY" },
"roles" : { "values" : ["*"] }
},
{
"principals" : { "values" : ["hal-9000"] },
"roles" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"roles" : { "type" : "NONE" }
}
],
"view_framework": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"view_tasks": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
],
"register_agent": [
{
"principals" : { "values" : ["hal-9000"] },
"users" : { "type" : "ANY" }
},
{
"principals" : { "values" : ["glados"] },
"users" : { "type" : "NONE" }
}
]
}
{code}
the following credential files:
{noformat}
super super
hal-9000 hal-9000
glados glados
{noformat}
And launch a master and an agent as follows:
{noformat}
./mesos-master.sh \
--work_dir=/tmp/$USER/mesos/master \
--log_dir=/tmp/$USER/mesos/master/log \
--ip=${MASTER_IP} \
--agent_ping_timeout=5secs \
--max_agent_ping_timeouts=2 \
--agent_removal_rate_limit=1/1secs \
--http_framework_authenticators=basic \
--authenticate_http \
--authenticate_http_frameworks \
--acls=$HOME/testing/acls.json \
--credentials=$HOME/testing/credentials.txt
sudo ./mesos-agent.sh \
--work_dir=/tmp/$USER/mesos/agent \
--log_dir=/tmp/$USER/mesos/agent/log \
--containerizers=mesos,docker \
--master=${MASTER_IP}:5050 \
--authenticate_http_readwrite \
--http_authenticators=basic \
--acls=$HOME/Documents/workspace/testing/acls.json \
--http_credentials=$HOME/testing/credentials.txt
{noformat}
Once the master and agent are running, use the following script:
{code}
#! /usr/bin/env python3
import json
import recordio
import requests
import sys
from contextlib import closing
from typing import Dict
MACHINE_NAME='ip.to.machine'
MACHINE_IP='ip.to.machine'
def test_mesos_v1_master_subscribe(host: str, ip: str) -> None:
base_url = 'http://{}:5050'.format(host)
headers = {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Connection': 'close',
}
decoder = recordio.Decoder(lambda s: json.loads(s.decode("UTF-8")))
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('hal-9000', 'hal-9000'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
events = [] # type: ignore
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] != 'HEARTBEAT':
print('got event "' + record['type'])
events.append(record['type'])
# It takes at least 10 mins for an agent to be marked as gone.
if len(events) >= 7:
print('got the following events "' + '", "'.join(events) + '"')
break
with closing(requests.post(
base_url + '/api/v1',
json={'type': 'SUBSCRIBE'},
auth=('glados', 'glados'),
headers=headers,
stream=True,
)) as response:
assert response.status_code == 200, response.text
chunks = response.iter_content(chunk_size=None)
for chunk in chunks:
for record in decoder.decode(chunk):
if record['type'] not in ('HEARTBEAT', 'SUBSCRIBED'):
print('It wasn\'t supposed to get "' + record['type'] + '"')
def main(args: Dict[str, str]) -> None:
test_mesos_v1_master_subscribe(MACHINE_NAME, MACHINE_IP)
if __name__ == '__main__':
main(sys.argv)
{code}
Finally while the machine is runnin
[jira] [Updated] (MESOS-8271) Libprocess SSL test fail in Mesos 17.10
[
https://issues.apache.org/jira/browse/MESOS-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8271:
---
Description:
The following tests constantly fail when building Mesos in Ubuntu 17.10:
{noformat}
SSLTest.SSLSocket
SSLTest.NoVerifyBadCA
SSLTest.VerifyCertificate
SSLTest.ProtocolMismatch
SSLTest.ECDHESupport
SSLTest.PeerAddress
SSLTest.HTTPSGet
SSLTest.HTTPSPost
SSLTest.SilentSocket
SSLTest.ShutdownThenSend
SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
{noformat}
The interesting [log
line|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1188]
is:
{noformat}
I1128 10:08:48.235530 26158 libevent_ssl_socket.cpp:1188] Socket error:
error::lib(0):func(0):reason(0)
{noformat}
After investigating some more one can notices that in the libevent callback the
[event bit error is marked as
BEV_EVENT_ERROR|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
however neither
{{[EVUTIL_SOCKET_ERROR()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1178]}}
nor
{{[bufferevent_get_openssl_error()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1182]}}
report any failure.
Googling for the issue reports at least someone who suffered the same problem
some years ago but there are no clues as to how it was fixed.
Full test log output:
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1128 10:08:47.934916 26145 process.cpp:1202] libprocess is initialized on
127.0.1.1:45399 with 12 worker threads
Note: Google Test filter = SSL*.*:Scheme/HTTPTest-
[==] Running 24 tests from 3 test cases.
[--] Global test environment set-up.
[--] 1 test from SSL
[ RUN ] SSL.Disabled
[ OK ] SSL.Disabled (0 ms)
[--] 1 test from SSL (0 ms total)
[--] 17 tests from SSLTest
[ RUN ] SSLTest.SSLSocket
I1128 10:08:48.113163 26145 openssl.cpp:504] CA file path is unspecified! NOTE:
Set CA file path with LIBPROCESS_SSL_CA_FILE=
I1128 10:08:48.113174 26145 openssl.cpp:509] CA directory path unspecified!
NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I1128 10:08:48.113179 26145 openssl.cpp:514] Will not verify peer certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I1128 10:08:48.113183 26145 openssl.cpp:520] Will only verify peer certificate
if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
I1128 10:08:48.113286 26145 openssl.cpp:643] Using ciphers:
AES128-SHA:AES256-SHA:RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1128 10:08:48.119578 26159 openssl.cpp:504] CA file path is unspecified! NOTE:
Set CA file path with LIBPROCESS_SSL_CA_FILE=
I1128 10:08:48.119622 26159 openssl.cpp:509] CA directory path unspecified!
NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I1128 10:08:48.119627 26159 openssl.cpp:514] Will not verify peer certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I1128 10:08:48.119632 26159 openssl.cpp:520] Will only verify peer certificate
if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
I1128 10:08:48.235530 26158 libevent_ssl_socket.cpp:1188] Socket error:
error::lib(0):func(0):reason(0)
../../../3rdparty/libprocess/src/tests/ssl_tests.cpp:254: Failure
(socket).failure(): Failed accept: connection error:
error::lib(0):func(0):reason(0)
[ FAILED ] SSLTest.SSLSocket (301 ms)
[ RUN ] SSLTest.NonSSLSocket
I1128 10:08:48.565058 26145 openssl.cpp:504] CA file path is unspecified! NOTE:
Set CA file path with LIBPROCESS_SSL_CA_FILE=
I1128 10:08:48.565075 26145 openssl.cpp:509] CA directory path unspecified!
NOTE: Set CA directory path with LIBPROCESS_SSL_CA_DIR=
I1128 10:08:48.565081 26145 openssl.cpp:514] Will not verify peer certificate!
NOTE: Set LIBPROCESS_SSL_VERIFY_CERT=1 to enable peer certificate verification
I1128 10:08:48.565085 26145 openssl.cpp:520] Will only verify peer certificate
if presented!
NOTE: Set LIBPROCESS_SSL_REQUIRE_CERT=1 to require peer certificate verification
I1128 10:08:48.565171 26145 openssl.cpp:643] Us
[jira] [Updated] (MESOS-8271) Libprocess SSL test fail in Mesos 17.10
[
https://issues.apache.org/jira/browse/MESOS-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8271:
---
Labels: mesosphere (was: )
> Libprocess SSL test fail in Mesos 17.10
> ---
>
> Key: MESOS-8271
> URL: https://issues.apache.org/jira/browse/MESOS-8271
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 1.5.0
> Environment: Mesos Configuration:
> {noformat}
> ../configure --disable-java --disable-python --enable-libevent --enable-ssl
> {noformat}
> Ubuntu 17.10
> {noformat}
> $ uname -r
> Linux localhost 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:04:08 UTC 2017
> x86_64 x86_64 x86_64 GNU/Linux
> {noformat}
> {noformat}
> $ sudo apt libevent-dev
> Package: libevent-dev
> Version: 2.1.8-stable-4
> Priority: optional
> Section: libdevel
> Source: libevent
> Origin: Ubuntu
> Maintainer: Ubuntu Developers
> Original-Maintainer: Anibal Monsalve Salazar
> Bugs: https://bugs.launchpad.net/ubuntu/+filebug
> Installed-Size: 1,694 kB
> Depends: libevent-2.1-6 (= 2.1.8-stable-4), libevent-core-2.1-6 (=
> 2.1.8-stable-4), libevent-extra-2.1-6 (= 2.1.8-stable-4),
> libevent-pthreads-2.1-6 (= 2.1.8-stable-4), libevent-openssl-2.1-6 (=
> 2.1.8-stable-4)
> Homepage: http://libevent.org/
> Supported: 9m
> Download-Size: 262 kB
> APT-Manual-Installed: yes
> APT-Sources: http://de.archive.ubuntu.com/ubuntu artful/main amd64 Packages
> Description: Asynchronous event notification library (development files)
> Libevent is an asynchronous event notification library that provides a
> mechanism to execute a callback function when a specific event occurs
> on a file descriptor or after a timeout has been reached.
> .
> This package includes development files for compiling against libevent.
> {noformat}
> {noformat}
> $ sudo apt libssl-dev
> Package: libssl-dev
> Version: 1.0.2g-1ubuntu13.2
> Priority: optional
> Section: libdevel
> Source: openssl
> Origin: Ubuntu
> Maintainer: Ubuntu Developers
> Original-Maintainer: Debian OpenSSL Team
>
> Bugs: https://bugs.launchpad.net/ubuntu/+filebug
> Installed-Size: 7,216 kB
> Depends: libssl1.0.0 (= 1.0.2g-1ubuntu13.2), zlib1g-dev
> Recommends: libssl-doc
> Supported: 9m
> Download-Size: 1,357 kB
> APT-Manual-Installed: yes
> APT-Sources: http://de.archive.ubuntu.com/ubuntu artful-updates/main amd64
> Packages
> Description: Secure Sockets Layer toolkit - development files
> This package is part of the OpenSSL project's implementation of the SSL
> and TLS cryptographic protocols for secure communication over the
> Internet.
> .
> It contains development libraries, header files, and manpages for libssl
> {noformat}
>Reporter: Alexander Rojas
> Labels: mesosphere
>
> The following tests constantly fail when building Mesos in Ubuntu 17.10:
> {noformat}
> SSLTest.SSLSocket
> SSLTest.NoVerifyBadCA
> SSLTest.VerifyCertificate
> SSLTest.ProtocolMismatch
> SSLTest.ECDHESupport
> SSLTest.PeerAddress
> SSLTest.HTTPSGet
> SSLTest.HTTPSPost
> SSLTest.SilentSocket
> SSLTest.ShutdownThenSend
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
> SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
> {noformat}
> The interesting [log
> line|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1188]
> is:
> {noformat}
> I1128 10:08:48.235530 26158 libevent_ssl_socket.cpp:1188] Socket error:
> error::lib(0):func(0):reason(0)
> {noformat}
> After investigating some more one can notices that in the libevent callback
> the [event bit error is marked as
> BEV_EVENT_ERROR|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
> however neither
> {{[EVUTIL_SOCKET_ERROR()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1178]}}
> nor
> {{[bufferevent_get_openssl_error()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1182]}}
> report any failure.
> Googling for the issue reports at least someone who suffered the same problem
> some years ago but there are no clues as to how it was fixed.
> Full test log output:
> {noformat}
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I1128 10:08:47.934916 26145 process.cpp:1202] libprocess is initialized on
> 127.0.1.1:45
[jira] [Created] (MESOS-8271) Libprocess SSL test fail in Mesos 17.10
Alexander Rojas created MESOS-8271:
--
Summary: Libprocess SSL test fail in Mesos 17.10
Key: MESOS-8271
URL: https://issues.apache.org/jira/browse/MESOS-8271
Project: Mesos
Issue Type: Bug
Components: libprocess
Affects Versions: 1.5.0
Environment: Mesos Configuration:
{noformat}
../configure --disable-java --disable-python --enable-libevent --enable-ssl
{noformat}
Ubuntu 17.10
{noformat}
$ uname -r
Linux localhost 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:04:08 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux
{noformat}
{noformat}
$ sudo apt libevent-dev
Package: libevent-dev
Version: 2.1.8-stable-4
Priority: optional
Section: libdevel
Source: libevent
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: Anibal Monsalve Salazar
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 1,694 kB
Depends: libevent-2.1-6 (= 2.1.8-stable-4), libevent-core-2.1-6 (=
2.1.8-stable-4), libevent-extra-2.1-6 (= 2.1.8-stable-4),
libevent-pthreads-2.1-6 (= 2.1.8-stable-4), libevent-openssl-2.1-6 (=
2.1.8-stable-4)
Homepage: http://libevent.org/
Supported: 9m
Download-Size: 262 kB
APT-Manual-Installed: yes
APT-Sources: http://de.archive.ubuntu.com/ubuntu artful/main amd64 Packages
Description: Asynchronous event notification library (development files)
Libevent is an asynchronous event notification library that provides a
mechanism to execute a callback function when a specific event occurs
on a file descriptor or after a timeout has been reached.
.
This package includes development files for compiling against libevent.
{noformat}
{noformat}
$ sudo apt libssl-dev
Package: libssl-dev
Version: 1.0.2g-1ubuntu13.2
Priority: optional
Section: libdevel
Source: openssl
Origin: Ubuntu
Maintainer: Ubuntu Developers
Original-Maintainer: Debian OpenSSL Team
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 7,216 kB
Depends: libssl1.0.0 (= 1.0.2g-1ubuntu13.2), zlib1g-dev
Recommends: libssl-doc
Supported: 9m
Download-Size: 1,357 kB
APT-Manual-Installed: yes
APT-Sources: http://de.archive.ubuntu.com/ubuntu artful-updates/main amd64
Packages
Description: Secure Sockets Layer toolkit - development files
This package is part of the OpenSSL project's implementation of the SSL
and TLS cryptographic protocols for secure communication over the
Internet.
.
It contains development libraries, header files, and manpages for libssl
{noformat}
Reporter: Alexander Rojas
The following tests constantly fail when building Mesos in Ubuntu 17.10:
{noformat}
SSLTest.SSLSocket
SSLTest.NoVerifyBadCA
SSLTest.VerifyCertificate
SSLTest.ProtocolMismatch
SSLTest.ECDHESupport
SSLTest.PeerAddress
SSLTest.HTTPSGet
SSLTest.HTTPSPost
SSLTest.SilentSocket
SSLTest.ShutdownThenSend
SSLVerifyIPAdd/SSLTest.BasicSameProcess/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcess/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.BasicSameProcessUnix/1, where GetParam() = "true"
SSLVerifyIPAdd/SSLTest.RequireCertificate/0, where GetParam() = "false"
SSLVerifyIPAdd/SSLTest.RequireCertificate/1, where GetParam() = "true"
{noformat}
The interesting [log
line|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1188]
is:
{noformat}
I1128 10:08:48.235530 26158 libevent_ssl_socket.cpp:1188] Socket error:
error::lib(0):func(0):reason(0)
{noformat}
After investigating some more one can notices that in the libevent callback the
[event bit error is marked as
BEV_EVENT_ERROR|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1176],
however neither
{{[EVUTIL_SOCKET_ERROR()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1178]}}
nor
{{[bufferevent_get_openssl_error()|https://github.com/apache/mesos/blob/f06e6184b0d6cc4a3245a714e5b56f26eb454233/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L1182]}}
report any failure.
Googling for the issue reports at least someone who suffered the same problem
some years ago but there are no clues as to how it was fixed.
Full test log output:
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I1128 10:08:47.934916 26145 process.cpp:1202] libprocess is initialized on
127.0.1.1:45399 with 12 worker threads
Note: Google Test filter = SSL*.*:Scheme/HTTPTest-
[==] Running 24 tests from 3 test cases.
[--] Global test environment set-up.
[--] 1 test from SSL
[ RUN ] SSL.Disabled
[ OK ] SSL.Disabled (0 ms)
[--] 1 test from SSL (0 ms total)
[--] 17 tests from SSLTest
[ RUN ] SSLTest.SSLSocket
I1128 10:08:48.113163 26145 openssl.cpp:504] CA file pa
[jira] [Created] (MESOS-8231) No documentation on hierarchical roles
Alexander Rojas created MESOS-8231:
--
Summary: No documentation on hierarchical roles
Key: MESOS-8231
URL: https://issues.apache.org/jira/browse/MESOS-8231
Project: Mesos
Issue Type: Documentation
Components: documentation
Affects Versions: 1.5.0
Reporter: Alexander Rojas
Hierarchical roles landed a while ago, however, there is no documentation on
how to describe hierarchies or specify subroles. This is particularly important
since hierarchical roles are not straightforward, i.e.:
# {{role}} gives permissions to {{role}} but not its sub-roles.
# {{role/%}} gives permissions to all subroles of {{role}} but not {{role}}
itself.
# {{role/subrole}} gives permission to the subrole {{subrole}}
# {{*}} is not a wildcard character as many users expect.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-5918) Replace jsonp with a more secure alternative
[
https://issues.apache.org/jira/browse/MESOS-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16250447#comment-16250447
]
Alexander Rojas commented on MESOS-5918:
For backwards compatibility i think it will be a while before we can completely
remove the {{jsonp}} parameter from our codebase, however that doesn't mean we
cannot mitigate the problem of the possible attacks by properly treating the
{{jsonp}} parameter.
As it is currently implemented, we just return whatever value was given in the
parameter, e.g.:
{code}
return OK(_flags(), request.url.query.get("jsonp"));
{code}
But we should probably parse that {{jsonp}} is just a JS identifier. Apparently
just Internet Explorer up to version 11 is vulnerable to this attack.
> Replace jsonp with a more secure alternative
>
>
> Key: MESOS-5918
> URL: https://issues.apache.org/jira/browse/MESOS-5918
> Project: Mesos
> Issue Type: Improvement
> Components: webui
>Reporter: Yan Xu
>
> We currently use the {{jsonp}} technique to bypass CORS check. This practice
> has many security concerns (see discussions on MESOS-5911) so we should
> replace it with a better alternative.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7826) XSS in JSONP parameter
[
https://issues.apache.org/jira/browse/MESOS-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16250448#comment-16250448
]
Alexander Rojas commented on MESOS-7826:
For backwards compatibility i think it will be a while before we can completely
remove the {{jsonp}} parameter from our codebase, however that doesn't mean we
cannot mitigate the problem of the possible attacks by properly treating the
{{jsonp}} parameter.
As it is currently implemented, we just return whatever value was given in the
parameter, e.g.:
{code}
return OK(_flags(), request.url.query.get("jsonp"));
{code}
But we should probably parse that {{jsonp}} is just a JS identifier. Apparently
just Internet Explorer up to version 11 is vulnerable to this attack.
> XSS in JSONP parameter
> --
>
> Key: MESOS-7826
> URL: https://issues.apache.org/jira/browse/MESOS-7826
> Project: Mesos
> Issue Type: Bug
> Components: json api
> Environment: Running as part of DC/OS in a docker container.
>Reporter: Vincent Ruijter
>Priority: Critical
>
> It is possible to inject arbitrary content into a server request. Take into
> account the following url:
> https://xxx.xxx.com/mesos/master/state?jsonp=var+oShell+%3d+new+ActiveXObject("WScript.Shell")%3boShell.Run("calc.exe",+1)%3b
> This will result in the following request:
> {code:html}
> GET
> /mesos/master/state?jsonp=var+oShell+%3d+new+ActiveXObject("WScript.Shell")%3boShell.Run("calc.exe",+1)%3b
> HTTP/1.1
> Host: xxx.xxx.com
> User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101
> Firefox/54.0
> Accept: */*
> Accept-Language: en-US,en;q=0.5
> [...SNIP...]
> {code}
> The server response:
> {code:html}
> HTTP/1.1 200 OK
> Server: openresty/1.9.15.1
> Date: Tue, 25 Jul 2017 09:04:31 GMT
> Content-Type: text/javascript
> Content-Length: 1411637
> Connection: close
> var oShell = new ActiveXObject("WScript.Shell");oShell.Run("calc.exe",
> 1);({"version":"1.2.1","git_sha":"f219b2e4f6265c0b6c4d826a390b67fe9d5e1097","build_date":"2017-06-01
> 19:16:40","build_time":149634
> [...SNIP...]
> {code}
> On Internet Explorer this will trigger a file download, and when executing
> the file (state.js), it will pop-up a calculator. It's my recommendation to
> apply input validation on this parameter, to prevent abuse.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (MESOS-7914) Replace usage of `ObjectApprover` with `AuthorizationAcceptor`
[
https://issues.apache.org/jira/browse/MESOS-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7914:
--
Assignee: Greg Mann (was: Alexander Rojas)
> Replace usage of `ObjectApprover` with `AuthorizationAcceptor`
> --
>
> Key: MESOS-7914
> URL: https://issues.apache.org/jira/browse/MESOS-7914
> Project: Mesos
> Issue Type: Improvement
> Components: security
>Affects Versions: 1.4.0
>Reporter: Greg Mann
>Assignee: Greg Mann
> Labels: authorization, mesosphere
>
> Now that the {{AuthorizationAcceptor}} class has been added, we can replace
> all occurrences of {{getObjectApprover}} with
> {{AuthorizationAcceptor::create}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (MESOS-7914) Replace usage of `ObjectApprover` with `AuthorizationAcceptor`
[
https://issues.apache.org/jira/browse/MESOS-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7914:
--
Assignee: Alexander Rojas (was: Greg Mann)
> Replace usage of `ObjectApprover` with `AuthorizationAcceptor`
> --
>
> Key: MESOS-7914
> URL: https://issues.apache.org/jira/browse/MESOS-7914
> Project: Mesos
> Issue Type: Improvement
> Components: security
>Affects Versions: 1.4.0
>Reporter: Greg Mann
>Assignee: Alexander Rojas
> Labels: authorization, mesosphere
>
> Now that the {{AuthorizationAcceptor}} class has been added, we can replace
> all occurrences of {{getObjectApprover}} with
> {{AuthorizationAcceptor::create}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-8175) CMake build fails to build debug mode
Alexander Rojas created MESOS-8175:
--
Summary: CMake build fails to build debug mode
Key: MESOS-8175
URL: https://issues.apache.org/jira/browse/MESOS-8175
Project: Mesos
Issue Type: Bug
Components: cmake
Affects Versions: 1.4.0
Reporter: Alexander Rojas
Trying to build Mesos with the following command:
{noformat}
cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_LIBEVENT=TRUE -DENABLE_SSL=TRUE ..
{noformat}
Yields:
{noformat}
cd /home/alexander/workspace/pmesos/cmake-build && /usr/bin/cmake -E
cmake_depends "Unix Makefiles" /home/alexander/workspace/pmesos
/home/alexander/workspace/pmesos/src
/home/alexander/workspace/pmesos/cmake-build
/home/alexander/workspace/pmesos/cmake-build/src
/home/alexander/workspace/pmesos/cmake-build/src/CMakeFiles/mesos-protobufs.dir/DependInfo.cmake
--color=
make[3]: Leaving directory '/home/alexander/workspace/pmesos/cmake-build'
make -f src/CMakeFiles/mesos-protobufs.dir/build.make
src/CMakeFiles/mesos-protobufs.dir/build
make[3]: Entering directory '/home/alexander/workspace/pmesos/cmake-build'
make[3]: *** No rule to make target
'3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-build/libprotobuf.so', needed by
'src/libmesos-protobufs.so'. Stop.
make[3]: Leaving directory '/home/alexander/workspace/pmesos/cmake-build'
CMakeFiles/Makefile2:1125: recipe for target
'src/CMakeFiles/mesos-protobufs.dir/all' failed
make[2]: *** [src/CMakeFiles/mesos-protobufs.dir/all] Error 2
make[2]: Leaving directory '/home/alexander/workspace/pmesos/cmake-build'
CMakeFiles/Makefile2:188: recipe for target 'CMakeFiles/check.dir/rule' failed
make[1]: *** [CMakeFiles/check.dir/rule] Error 2
make[1]: Leaving directory '/home/alexander/workspace/pmesos/cmake-build'
Makefile:193: recipe for target 'check' failed
make: *** [check] Error 2
{noformat}
Listing the directory for protobuf shows the following:
{noformat}
$ ls -l 3rdparty/protobuf-3.3.0/src/protobuf-3.3.0-build
-rw-rw-r--. 1 user user18485 Nov 7 04:35 CMakeCache.txt
drwxrwxr-x. 10 user user 4096 Nov 7 04:36 CMakeFiles
-rw-rw-r--. 1 user user52241 Nov 7 04:35 cmake_install.cmake
-rwxrwxr-x. 1 user user35496 Nov 7 04:35 js_embed
drwxrwxr-x. 3 user user 26 Nov 7 04:35 lib64
-rwxrwxr-x. 1 user user 21747424 Nov 7 04:35 libprotobufd.so
-rwxrwxr-x. 1 user user 2020752 Nov 7 04:35 libprotobuf-lited.so
-rwxrwxr-x. 1 user user 19788608 Nov 7 04:36 libprotocd.so
-rw-rw-r--. 1 user user 597984 Nov 7 04:35 Makefile
-rwxrwxr-x. 1 user user 190328 Nov 7 04:36 protoc
{noformat}
>From this is easy to infer that protobuf suffixes a {{d}} to the debug version
>of its libraries but when we try to link to them we expect them to not use any
>suffix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7111) HttpFaultToleranceTest.SchedulerFailoverFrameworkToExecutorMessage segfaults
[
https://issues.apache.org/jira/browse/MESOS-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234013#comment-16234013
]
Alexander Rojas commented on MESOS-7111:
Got a similar issue on _Fedora 25 TwentyFive_ with clang 3.9.1:
{noformat}
[==] Running 6 tests from 1 test case.
[--] Global test environment set-up.
[--] 6 tests from HttpFaultToleranceTest
[ RUN ] HttpFaultToleranceTest.SchedulerSubscribeAfterFailoverTimeout
[ OK ] HttpFaultToleranceTest.SchedulerSubscribeAfterFailoverTimeout (297
ms)
[ RUN ] HttpFaultToleranceTest.SchedulerSubscribeAfterTeardown
[ OK ] HttpFaultToleranceTest.SchedulerSubscribeAfterTeardown (305 ms)
[ RUN ] HttpFaultToleranceTest.SchedulerFailoverStatusUpdate
[ OK ] HttpFaultToleranceTest.SchedulerFailoverStatusUpdate (459 ms)
[ RUN ] HttpFaultToleranceTest.SchedulerFailoverExecutorToFrameworkMessage
*** Aborted at 1509538627 (unix time) try "date -d @1509538627" if you are
using GNU date ***
PC: @ 0x55b23f76430e
testing::internal::UntypedFunctionMockerBase::SetOwnerAndName()
*** SIGSEGV (@0xd8) received by PID 8178 (TID 0x7f336958c700) from PID 216;
stack trace: ***
@ 0x7f3375f645d0 (unknown)
@ 0x55b23f76430e
testing::internal::UntypedFunctionMockerBase::SetOwnerAndName()
@ 0x55b23e18f373
mesos::internal::tests::executor::MockHTTPExecutor<>::subscribed()
@ 0x55b23e18ed0d
mesos::internal::tests::executor::MockHTTPExecutor<>::event()
@ 0x55b23e18d844 mesos::internal::tests::executor::TestMesos<>::events()
@ 0x55b23e190fe3
_ZSt13__invoke_implIvRKMN5mesos8internal5tests8executor9TestMesosINS0_2v18executor5MesosENS6_5EventEEEFvSt5queueIS8_St5dequeIS8_SaIS8_RPS9_JRKSE
_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2_
@ 0x55b23e190ef7
_ZSt8__invokeIRKMN5mesos8internal5tests8executor9TestMesosINS0_2v18executor5MesosENS6_5EventEEEFvSt5queueIS8_St5dequeIS8_SaIS8_JRPS9_RKSE_EENSt9
result_ofIFOT_DpOT0_EE4typeESP_SS_
@ 0x55b23e190e05
_ZNKSt12_Mem_fn_baseIMN5mesos8internal5tests8executor9TestMesosINS0_2v18executor5MesosENS6_5EventEEEFvSt5queueIS8_St5dequeIS8_SaIS8_Lb1EEclIJRPS
9_RKSE_EEEDTclsr3stdE8__invokedtdefpT6_M_pmfspclsr3stdE7forwardIT_Efp_EEEDpOSN_
@ 0x55b23e190d25
_ZNSt5_BindIFSt7_Mem_fnIMN5mesos8internal5tests8executor9TestMesosINS1_2v18executor5MesosENS7_5EventEEEFvSt5queueIS9_St5dequeIS9_SaIS9_EPSA_St1$
_PlaceholderILi16__callIvJRKSF_EJLm0ELm1T_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x55b23e190c56
_ZNSt5_BindIFSt7_Mem_fnIMN5mesos8internal5tests8executor9TestMesosINS1_2v18executor5MesosENS7_5EventEEEFvSt5queueIS9_St5dequeIS9_SaIS9_EPSA_St1$
_PlaceholderILi1clIJRKSF_EvEET0_DpOT_
@ 0x55b23e190922 std::_Function_handler<>::_M_invoke()
@ 0x7f3377e47398 std::function<>::operator()()
@ 0x7f3377e46fad process::AsyncExecutorProcess::execute<>()
@ 0x7f3377e4825a
_ZZN7process8dispatchI7NothingNS_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v18executor5EventESt5dequeIS8_SaIS8_ESC_SI_RSC_EENS_$
FutureIT_EERKNS_3PIDIT0_EEMSO_FSL_T1_T2_EOT3_OT4_ENKUlRSG_SJ_PNS_11ProcessBaseEE_clES10_SJ_S12_
@ 0x7f3377e48056
_ZNSt5_BindIFZN7process8dispatchI7NothingNS0_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v18executor5EventESt5dequeIS9_SaIS9_ESD_S
J_RSD_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMSP_FSM_T1_T2_EOT3_OT4_EUlRSH_SK_PNS0_11ProcessBaseEE_SH_SD_St12_PlaceholderILi16__callIvJOS13_EJLm0ELm1ELm2SM_OSt5tupleIJDpT
0_EESt12_Index_tupleIJXspT1_EEE
@ 0x7f3377e47f76
_ZNSt5_BindIFZN7process8dispatchI7NothingNS0_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v18executor5EventESt5dequeIS9_SaIS9_ESD_S
J_RSD_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMSP_FSM_T1_T2_EOT3_OT4_EUlRSH_SK_PNS0_11ProcessBaseEE_SH_SD_St12_PlaceholderILi1clIJS13_EvEESP_DpOT_
@ 0x7f3377e47912
_ZNSt17_Function_handlerIFvPN7process11ProcessBaseEESt5_BindIFZNS0_8dispatchI7NothingNS0_20AsyncExecutorProcessERKSt8functionIFvRKSt5queueIN5mesos2v
18executor5EventESt5dequeISD_SaISD_ESH_SN_RSH_EENS0_6FutureIT_EERKNS0_3PIDIT0_EEMST_FSQ_T1_T2_EOT3_OT4_EUlRSL_SO_S2_E_SL_SH_St12_PlaceholderILi1E9_M_invokeERKSt9_Any
_dataOS2_
@ 0x7f3379c57464 std::function<>::operator()()
@ 0x7f3379c3cac1 process::ProcessBase::visit()
@ 0x7f3379c8e7be process::DispatchEvent::visit()
@ 0x7f3377b13681 process::ProcessBase::serve()
@ 0x7f3379c39f21 process::ProcessManager::resume()
@ 0x7f3379c43a2b
process::ProcessManager::init_threads()::$_9::operator()()
@ 0x7f3379c438d5
_ZNSt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvE3$_9vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
@ 0x7f3379c438a5 std::_Bind_simple<>::operator()()
@ 0x7f3379c43799 std::thread::_State_impl<>::_M_run()
@ 0x7f3
[jira] [Commented] (MESOS-8057) Apply security patches to AngularJS and JQuery in the Mesos UI
[
https://issues.apache.org/jira/browse/MESOS-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16209438#comment-16209438
]
Alexander Rojas commented on MESOS-8057:
Changes landed in Mesos master branch. They will be part of next Mesos bump on
DC/OS
{noformat}
commit b0a660bb1811c0144cba781482b1ce4573e685b3
Author: Alexander Rojas
AuthorDate: Wed Oct 18 12:11:05 2017 +0200
Commit: Alexander Rojas
CommitDate: Wed Oct 18 16:33:19 2017 +0200
Upgrades jQuery used by Mesos WebUI to version 3.2.1.
The version of jQuery distributed with Mesos (1.7.1) was found to have
security issues which have been addressed in latter versions.
Review: https://reviews.apache.org/r/63101
{noformat}
{noformat}
commit 1b5a4e77e55f5c8665526294626a66905569a284 (HEAD -> master,
upstream/master)
Author: Alexander Rojas
AuthorDate: Wed Oct 18 12:11:40 2017 +0200
Commit: Alexander Rojas
CommitDate: Wed Oct 18 16:33:37 2017 +0200
Upgrades AngularJS used by Mesos WebUI to version 1.2.32.
The version of AngularJS distributed with Mesos (1.2.3) was found to
have security issues which have been addressed in latter versions.
Review: https://reviews.apache.org/r/63102
{noformat}
> Apply security patches to AngularJS and JQuery in the Mesos UI
> --
>
> Key: MESOS-8057
> URL: https://issues.apache.org/jira/browse/MESOS-8057
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Affects Versions: 1.4.0
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Blocker
> Labels: mesosphere
> Fix For: 1.5.0
>
>
> Running a security tool returns:
> {noformat}
> Evidence
> Vulnerable libraries were found:
> https://admin.kpn-dsh.com/mesos/static/js/angular-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/angular-route-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/jquery-1.7.1.min.js
> More information about the issues can be found at: -
> https://github.com/angular/angular.js/blob/master/CHANGELOG.md -
> http://bugs.jquery.com/ticket/11290
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Issue Comment Deleted] (MESOS-8057) Apply security patches to AngularJS and JQuery in the Mesos UI
[
https://issues.apache.org/jira/browse/MESOS-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-8057:
---
Comment: was deleted
(was: Changes landed in Mesos master branch. They will be part of next Mesos
bump on DC/OS
{noformat}
commit b0a660bb1811c0144cba781482b1ce4573e685b3
Author: Alexander Rojas
AuthorDate: Wed Oct 18 12:11:05 2017 +0200
Commit: Alexander Rojas
CommitDate: Wed Oct 18 16:33:19 2017 +0200
Upgrades jQuery used by Mesos WebUI to version 3.2.1.
The version of jQuery distributed with Mesos (1.7.1) was found to have
security issues which have been addressed in latter versions.
Review: https://reviews.apache.org/r/63101
{noformat}
{noformat}
commit 1b5a4e77e55f5c8665526294626a66905569a284 (HEAD -> master,
upstream/master)
Author: Alexander Rojas
AuthorDate: Wed Oct 18 12:11:40 2017 +0200
Commit: Alexander Rojas
CommitDate: Wed Oct 18 16:33:37 2017 +0200
Upgrades AngularJS used by Mesos WebUI to version 1.2.32.
The version of AngularJS distributed with Mesos (1.2.3) was found to
have security issues which have been addressed in latter versions.
Review: https://reviews.apache.org/r/63102
{noformat})
> Apply security patches to AngularJS and JQuery in the Mesos UI
> --
>
> Key: MESOS-8057
> URL: https://issues.apache.org/jira/browse/MESOS-8057
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Affects Versions: 1.4.0
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Blocker
> Labels: mesosphere
> Fix For: 1.5.0
>
>
> Running a security tool returns:
> {noformat}
> Evidence
> Vulnerable libraries were found:
> https://admin.kpn-dsh.com/mesos/static/js/angular-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/angular-route-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/jquery-1.7.1.min.js
> More information about the issues can be found at: -
> https://github.com/angular/angular.js/blob/master/CHANGELOG.md -
> http://bugs.jquery.com/ticket/11290
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (MESOS-8057) Apply security patches to AngularJS and JQuery in the Mesos UI
[
https://issues.apache.org/jira/browse/MESOS-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-8057:
--
Assignee: Alexander Rojas
> Apply security patches to AngularJS and JQuery in the Mesos UI
> --
>
> Key: MESOS-8057
> URL: https://issues.apache.org/jira/browse/MESOS-8057
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Affects Versions: 1.4.0
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>Priority: Blocker
> Labels: mesosphere
>
> Running a security tool returns:
> {noformat}
> Evidence
> Vulnerable libraries were found:
> https://admin.kpn-dsh.com/mesos/static/js/angular-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/angular-route-1.2.3.min.js
> https://admin.kpn-dsh.com/mesos/static/js/jquery-1.7.1.min.js
> More information about the issues can be found at: -
> https://github.com/angular/angular.js/blob/master/CHANGELOG.md -
> http://bugs.jquery.com/ticket/11290
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7957) The REGISTER_FRAMEWORK_WITH_ROLE does not use in source code
[
https://issues.apache.org/jira/browse/MESOS-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194402#comment-16194402
]
Alexander Rojas commented on MESOS-7957:
{{REGISTER_FRAMEWORK_WITH_ROLE}} as well as all other actions using the
{{_WITH_*}} suffixes are deprecated since November 2016. That means that our
six months deprecation cycle has finished.
I think we can move forward with this, but not only with the for the action in
particular, but with all the deprecated ones, so probably we need to change
this issue title and description. If necessary, I could shepherd this.
> The REGISTER_FRAMEWORK_WITH_ROLE does not use in source code
>
>
> Key: MESOS-7957
> URL: https://issues.apache.org/jira/browse/MESOS-7957
> Project: Mesos
> Issue Type: Improvement
> Components: security
>Reporter: jackyoh
>Priority: Trivial
>
> Mesos test code has the REGISTER_FRAMEWORK_WITH_ROLE action in
> src/test/authorization_tests.cpp, but the source code does not
> use the REGISTER_FRAMEWORK_WITH_ROLE action.
> Can I remove the REGISTER_FRAMEWORK_WITH_ROLE action in
> src/tests/authorization_tests.cpp?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (MESOS-7957) The REGISTER_FRAMEWORK_WITH_ROLE does not use in source code
[
https://issues.apache.org/jira/browse/MESOS-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194402#comment-16194402
]
Alexander Rojas edited comment on MESOS-7957 at 10/6/17 10:17 AM:
--
{{REGISTER_FRAMEWORK_WITH_ROLE}} as well as all other actions using the
{{\_WITH\_*}} suffixes are deprecated since November 2016. That means that our
six months deprecation cycle has finished.
I think we can move forward with this, but not only with the for the action in
particular, but with all the deprecated ones, so probably we need to change
this issue title and description. If necessary, I could shepherd this.
was (Author: arojas):
{{REGISTER_FRAMEWORK_WITH_ROLE}} as well as all other actions using the
{{_WITH_*}} suffixes are deprecated since November 2016. That means that our
six months deprecation cycle has finished.
I think we can move forward with this, but not only with the for the action in
particular, but with all the deprecated ones, so probably we need to change
this issue title and description. If necessary, I could shepherd this.
> The REGISTER_FRAMEWORK_WITH_ROLE does not use in source code
>
>
> Key: MESOS-7957
> URL: https://issues.apache.org/jira/browse/MESOS-7957
> Project: Mesos
> Issue Type: Improvement
> Components: security
>Reporter: jackyoh
>Priority: Trivial
>
> Mesos test code has the REGISTER_FRAMEWORK_WITH_ROLE action in
> src/test/authorization_tests.cpp, but the source code does not
> use the REGISTER_FRAMEWORK_WITH_ROLE action.
> Can I remove the REGISTER_FRAMEWORK_WITH_ROLE action in
> src/tests/authorization_tests.cpp?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-8057) Apply security patches to AngularJS and JQuery in the Mesos UI
Alexander Rojas created MESOS-8057:
--
Summary: Apply security patches to AngularJS and JQuery in the
Mesos UI
Key: MESOS-8057
URL: https://issues.apache.org/jira/browse/MESOS-8057
Project: Mesos
Issue Type: Bug
Components: webui
Affects Versions: 1.4.0
Reporter: Alexander Rojas
Priority: Blocker
Running a security tool returns:
{noformat}
Evidence
Vulnerable libraries were found:
https://admin.kpn-dsh.com/mesos/static/js/angular-1.2.3.min.js
https://admin.kpn-dsh.com/mesos/static/js/angular-route-1.2.3.min.js
https://admin.kpn-dsh.com/mesos/static/js/jquery-1.7.1.min.js
More information about the issues can be found at: -
https://github.com/angular/angular.js/blob/master/CHANGELOG.md -
http://bugs.jquery.com/ticket/11290
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MESOS-7887) `GET_EXECUTORS` and `/state` is not consistent between master and agent
[
https://issues.apache.org/jira/browse/MESOS-7887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7887:
---
Description:
The master seem not to keep information about the executors since they are not
returned either either by getting the master state (with either v0 and v1
API's) or with the call {{GET_EXECUTORS}}. Creating a cluster as follows:
{noformat}
./bin/mesos-master.sh \
--ip=${MASTER_IP} \
--work_dir=/tmp/mesos/master \
--log_dir=/tmp/mesos/master/log
{noformat}
{noformat}
sudo ./bin/mesos-agent.sh \
--master=${MASTER_IP}:5050 \
--work_dir=/tmp/mesos/agent \
--log_dir=/tmp/mesos/agent/log \
--containerizers=mesos,docker
{noformat}
And launch a couple of frameworks as follows:
{noformat}
./src/mesos-execute \
--master=${MASTER_IP}:5050 \
--task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor":{"executor_id":{"value":"test-custom-command-executor"},"command":{"value":"while
true; do echo \"Hello World\"; sleep 5; done;"}}}'
{noformat}
{noformat}
./src/mesos-execute \
--master=${MASTER_IP}:5050 \
--name=test-command \
--command='while true; do echo "Hello World"; sleep 5; done;' \
--containerizer=docker \
--docker_image=ubuntu:latest
{noformat}
Not using the operator endpoints on the agent:
{noformat}
$ http POST ${AGENT_IP}:5051/api/v1 type=GET_EXECUTORS
{
"get_executors": {
"completed_executors": [
],
"executors": [
{
"executor_info": {
"command": {
"arguments": [
"mesos-executor",
"--launcher_dir=/workspace/mesos/build/src"
],
"shell": false,
"value": "/workspace/mesos/build/src/mesos-executor"
},
"container": {
"docker": {
"image": "ubuntu:latest",
"network": "HOST",
"privileged": false
},
"type": "DOCKER"
},
"executor_id": {
"value": "test-command"
},
"framework_id": {
"value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0001"
},
"name": "Command Executor (Task: test-command) (Command: sh -c 'while
true; ...')",
"resources": [
{
"allocation_info": {
"role": "*"
},
"name": "cpus",
"scalar": {
"value": 0.1
},
"type": "SCALAR"
},
{
"allocation_info": {
"role": "*"
},
"name": "mem",
"scalar": {
"value": 32
},
"type": "SCALAR"
}
],
"source": "test-command"
}
},
{
"executor_info": {
"command": {
"shell": true,
"value": "while true; do echo \"Hello World\"; sleep 5; done;"
},
"executor_id": {
"value": "test-custom-command-executor"
},
"framework_id": {
"value": "87577bcd-093d-4240-a24b-107b4d1d21bd-"
}
}
}
]
},
"type": "GET_EXECUTORS"
}
{noformat}
While the master does
{noformat}
http POST ${MASTER_IP}:5050/api/v1 type=GET_EXECUTORS
{
"get_executors": {},
"type": "GET_EXECUTORS"
}
{noformat}
These results are consistent using the `/state` endpoint on both, agent and
master as well as using the {{GET_STATE}} v1 API call. The agent returns
information about executors, while the master response has none.
was:
The master seem not to keep information about the executors since they are not
returned either either by getting the master state (with either v0 and v1
API's) or with the call `GET_EXECUTORS`. Creating a cluster as follows:
{noformat}
./bin/mesos-master.sh \
--ip=${MASTER_IP} \
--work_dir=/tmp/mesos/master \
--log_dir=/tmp/mesos/master/log
{noformat}
{noformat}
sudo ./bin/mesos-agent.sh \
--master=${MASTER_IP}:5050 \
--work_dir=/tmp/mesos/agent \
--log_dir=/tmp/mesos/agent/log \
--containerizers=mesos,docker
{noformat}
And launch a couple of frameworks as follows:
{noformat}
./src/mesos-execute \
--master=${MASTER_IP}:5050 \
--task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor"
[jira] [Created] (MESOS-7887) `GET_EXECUTORS` and `/state` is not consistent between master and agent
Alexander Rojas created MESOS-7887:
--
Summary: `GET_EXECUTORS` and `/state` is not consistent between
master and agent
Key: MESOS-7887
URL: https://issues.apache.org/jira/browse/MESOS-7887
Project: Mesos
Issue Type: Bug
Components: HTTP API, master
Affects Versions: 1.3.0
Reporter: Alexander Rojas
The master seem not to keep information about the executors since they are not
returned either either by getting the master state (with either v0 and v1
API's) or with the call `GET_EXECUTORS`. Creating a cluster as follows:
{noformat}
./bin/mesos-master.sh \
--ip=${MASTER_IP} \
--work_dir=/tmp/mesos/master \
--log_dir=/tmp/mesos/master/log
{noformat}
{noformat}
sudo ./bin/mesos-agent.sh \
--master=${MASTER_IP}:5050 \
--work_dir=/tmp/mesos/agent \
--log_dir=/tmp/mesos/agent/log \
--containerizers=mesos,docker
{noformat}
And launch a couple of frameworks as follows:
{noformat}
./src/mesos-execute \
--master=${MASTER_IP}:5050 \
--task='{"name":"test-custom-command","task_id":{"value":"test-custom-command-task-1"},"agent_id":{"value":"50f4e551-aa5c-42db-8967-4dc3ee11658f-S0"},"resources":[{"name":"cpus","type":"SCALAR","scalar":{"value":1}},{"name":"mem","type":"SCALAR","scalar":{"value":32}},{"name":"disk","type":"SCALAR","scalar":{"value":32}}],"executor":{"executor_id":{"value":"test-custom-command-executor"},"command":{"value":"while
true; do echo \"Hello World\"; sleep 5; done;"}}}'
{noformat}
{noformat}
./src/mesos-execute \
--master=${MASTER_IP}:5050 \
--name=test-command \
--command='while true; do echo "Hello World"; sleep 5; done;' \
--containerizer=docker \
--docker_image=ubuntu:latest
{noformat}
Not using the operator endpoints on the agent:
{noformat}
$ http POST ${AGENT_IP}:5051/api/v1 type=GET_EXECUTORS
{
"get_executors": {
"completed_executors": [
],
"executors": [
{
"executor_info": {
"command": {
"arguments": [
"mesos-executor",
"--launcher_dir=/workspace/mesos/build/src"
],
"shell": false,
"value": "/workspace/mesos/build/src/mesos-executor"
},
"container": {
"docker": {
"image": "ubuntu:latest",
"network": "HOST",
"privileged": false
},
"type": "DOCKER"
},
"executor_id": {
"value": "test-command"
},
"framework_id": {
"value": "87577bcd-093d-4240-a24b-107b4d1d21bd-0001"
},
"name": "Command Executor (Task: test-command) (Command: sh -c 'while
true; ...')",
"resources": [
{
"allocation_info": {
"role": "*"
},
"name": "cpus",
"scalar": {
"value": 0.1
},
"type": "SCALAR"
},
{
"allocation_info": {
"role": "*"
},
"name": "mem",
"scalar": {
"value": 32
},
"type": "SCALAR"
}
],
"source": "test-command"
}
},
{
"executor_info": {
"command": {
"shell": true,
"value": "while true; do echo \"Hello World\"; sleep 5; done;"
},
"executor_id": {
"value": "test-custom-command-executor"
},
"framework_id": {
"value": "87577bcd-093d-4240-a24b-107b4d1d21bd-"
}
}
}
]
},
"type": "GET_EXECUTORS"
}
{noformat}
While the master does
{noformat}
http POST ${MASTER_IP}:5050/api/v1 type=GET_EXECUTORS
{
"get_executors": {},
"type": "GET_EXECUTORS"
}
{noformat}
These results are consistent using the `/state` endpoint on both, agent and
master as well as using the {{GET_STATE}} v1 API call. The agent returns
information about executors, while the master response has none.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-7864) Agent logs should be accesible with the `/agent/log` path.
Alexander Rojas created MESOS-7864:
--
Summary: Agent logs should be accesible with the `/agent/log` path.
Key: MESOS-7864
URL: https://issues.apache.org/jira/browse/MESOS-7864
Project: Mesos
Issue Type: Improvement
Affects Versions: 1.3.0
Reporter: Alexander Rojas
Assignee: Alexander Rojas
Priority: Trivial
Despite the efforts to alias all elements in Mesos from Slave to Agent, logs in
the agent are still only accessible through the {{/slave/log}} path.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-2092) Make ACLs dynamic
[ https://issues.apache.org/jira/browse/MESOS-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112941#comment-16112941 ] Alexander Rojas commented on MESOS-2092: [~saitejar] [~alexr] I don't think this issue is even necessary. The reason is, that providing dynamic ACLs is relatively easy to do by creating a module that mesos loads, and this kind of functionality is better to be built on top of mesos rather than inside mesos. > Make ACLs dynamic > - > > Key: MESOS-2092 > URL: https://issues.apache.org/jira/browse/MESOS-2092 > Project: Mesos > Issue Type: Task > Components: security >Reporter: Alexander Rukletsov >Assignee: Yongqiao Wang > Labels: mesosphere, newbie > > Master loads ACLs once during its launch and there is no way to update them > in a running master. Making them dynamic will allow for updating ACLs on the > fly, for example granting a new framework necessary rights. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MESOS-7652) Docker image with universal containerizer does not work if WORKDIR is missing in the rootfs.
[
https://issues.apache.org/jira/browse/MESOS-7652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7652:
---
Target Version/s: 1.2.3, 1.3.2, 1.4.0 (was: 1.2.2, 1.3.2, 1.4.0)
> Docker image with universal containerizer does not work if WORKDIR is missing
> in the rootfs.
>
>
> Key: MESOS-7652
> URL: https://issues.apache.org/jira/browse/MESOS-7652
> Project: Mesos
> Issue Type: Bug
> Components: containerization
>Affects Versions: 1.2.1
>Reporter: michael beisiegel
>Assignee: Gilbert Song
>Priority: Critical
> Labels: mesosphere
>
> hello,
> used the following docker image recently
> quay.io/spinnaker/front50:master
> https://quay.io/repository/spinnaker/front50
> Here the link to the Dockerfile
> https://github.com/spinnaker/front50/blob/master/Dockerfile
> and here the source
> {color:blue}FROM java:8
> MAINTAINER delivery-engineer...@netflix.com
> COPY . workdir/
> WORKDIR workdir
> RUN GRADLE_USER_HOME=cache ./gradlew buildDeb -x test && \
> dpkg -i ./front50-web/build/distributions/*.deb && \
> cd .. && \
> rm -rf workdir
> CMD ["/opt/front50/bin/front50"]{color}
> The image works fine with the docker containerizer, but the universal
> containerizer shows the following in stderr.
> "Failed to chdir into current working directory '/workdir': No such file or
> directory"
> The problem comes from the fact that the Dockerfile creates a workdir but
> then later removes the created dir as part of a RUN. The docker containerizer
> has no problem with it if you do
> docker run -ti --rm quay.io/spinnaker/front50:master bash
> you get into the working dir, but the universal containerizer fails with the
> error.
> thanks for your help,
> Michael
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MESOS-7622) Agent can crash if a HTTP executor tries to retry subscription in running state.
[
https://issues.apache.org/jira/browse/MESOS-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7622:
---
Target Version/s: 1.2.3, 1.3.2 (was: 1.2.2, 1.3.2)
> Agent can crash if a HTTP executor tries to retry subscription in running
> state.
>
>
> Key: MESOS-7622
> URL: https://issues.apache.org/jira/browse/MESOS-7622
> Project: Mesos
> Issue Type: Bug
> Components: agent, executor
>Reporter: Aaron Wood
>Assignee: Anand Mazumdar
>Priority: Blocker
>
> It is possible that a running executor might retry its subscribe request.
> This can lead to a crash if it previously had any launched tasks. Note that
> the executor would still be able to subscribe again when the agent process
> restarts and is recovering.
> {code}
> sudo ./mesos-agent --master=10.0.2.15:5050 --work_dir=/tmp/slave
> --isolation=cgroups/cpu,cgroups/mem,disk/du,network/cni,filesystem/linux,docker/runtime
> --image_providers=docker --image_provisioner_backend=overlay
> --containerizers=mesos --launcher_dir=$(pwd)
> --executor_environment_variables='{"LD_LIBRARY_PATH":
> "/home/aaron/Code/src/mesos/build/src/.libs"}'
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0605 14:58:23.748180 10710 main.cpp:323] Build: 2017-06-02 17:09:05 UTC by
> aaron
> I0605 14:58:23.748252 10710 main.cpp:324] Version: 1.4.0
> I0605 14:58:23.755409 10710 systemd.cpp:238] systemd version `232` detected
> I0605 14:58:23.755450 10710 main.cpp:433] Initializing systemd state
> I0605 14:58:23.763049 10710 systemd.cpp:326] Started systemd slice
> `mesos_executors.slice`
> I0605 14:58:23.763777 10710 resolver.cpp:69] Creating default secret resolver
> I0605 14:58:23.764214 10710 containerizer.cpp:230] Using isolation:
> cgroups/cpu,cgroups/mem,disk/du,network/cni,filesystem/linux,docker/runtime,volume/image,environment_secret
> I0605 14:58:23.767192 10710 linux_launcher.cpp:150] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> E0605 14:58:23.770179 10710 shell.hpp:107] Command 'hadoop version 2>&1'
> failed; this is the output:
> sh: 1: hadoop: not found
> I0605 14:58:23.770217 10710 fetcher.cpp:69] Skipping URI fetcher plugin
> 'hadoop' as it could not be created: Failed to create HDFS client: Failed to
> execute 'hadoop version 2>&1'; the command was either not found or exited
> with a non-zero exit status: 127
> I0605 14:58:23.770643 10710 provisioner.cpp:255] Using default backend
> 'overlay'
> I0605 14:58:23.785892 10710 slave.cpp:248] Mesos agent started on
> (1)@127.0.1.1:5051
> I0605 14:58:23.785957 10710 slave.cpp:249] Flags at startup:
> --appc_simple_discovery_uri_prefix="http://";
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticatee="crammd5"
> --authentication_backoff_factor="1secs" --authorizer="local"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins"
> --docker="docker" --docker_kill_orphans="true"
> --docker_registry="https://registry-1.docker.io"; --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
> --enforce_container_disk_quota="false"
> --executor_environment_variables="{"LD_LIBRARY_PATH":"\/home\/aaron\/Code\/src\/mesos\/build\/src\/.libs"}"
> --executor_registration_timeout="1mins"
> --executor_reregistration_timeout="2secs"
> --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_command_executor="false" --http_heartbeat_interval="30secs"
> --image_providers="docker" --image_provisioner_backend="overlay"
> --initialize_driver_logging="true"
> --isolation="cgroups/cpu,cgroups/mem,disk/du,network/cni,filesystem/linux,docker/runtime"
> --launcher="linux" --launcher_dir="/home/aaron/Code/src/mesos/build/src"
> --logbufsecs="0" --logging_level="INFO" --master="10.0.2.15:5050"
> --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true"
[jira] [Commented] (MESOS-7215) Race condition on re-registration of non-partition-aware frameworks
[
https://issues.apache.org/jira/browse/MESOS-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101311#comment-16101311
]
Alexander Rojas commented on MESOS-7215:
Is this still being worked on? it this a blocker for 1.2.2? will it land there?
> Race condition on re-registration of non-partition-aware frameworks
> ---
>
> Key: MESOS-7215
> URL: https://issues.apache.org/jira/browse/MESOS-7215
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Yan Xu
>Assignee: Megha Sharma
>Priority: Critical
>
> Prior to the partition-awareness work MESOS-5344, upon agent reregistration
> after it has been removed, the master only sends ShutdownFrameworkMessages to
> the agent for frameworks that it knows have been torn down.
> With the new logic in MESOS-5344, Mesos is now sending
> {{ShutdownFrameworkMessages}} to the agent for all non-partition-aware
> frameworks (including the ones that are still registered)
> This is problematic. The offer from this agent can still go to the same
> framework which can then launch new tasks. The agent then receives tasks of
> the same framework and ignores them because it thinks the framework is
> shutting down. The framework is not shutting down of course, so from the
> master and the scheduler's perspective the task is pending in STAGING forever
> until the next agent reregistration, which could happen much later.
> This also makes the semantics of `ShutdownFrameworkMessage` ambiguous: the
> agent is assuming the framework to be going away (and act accordingly) when
> it's not.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MESOS-7792) Add support for ECDH ciphers
[ https://issues.apache.org/jira/browse/MESOS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rojas updated MESOS-7792: --- Labels: security (was: ) > Add support for ECDH ciphers > > > Key: MESOS-7792 > URL: https://issues.apache.org/jira/browse/MESOS-7792 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 1.3.0 >Reporter: Alexander Rojas >Assignee: Alexander Rojas > Labels: security > > [Elliptic curve > ciphers|https://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography] are a > family of ciphers supported by OpenSSL. They allow to have smaller keys, but > require an extra configuration parameter, the actual curve to be used, which > can't be done through libprocess as it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-7792) Add support for ECDH ciphers
[ https://issues.apache.org/jira/browse/MESOS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16089970#comment-16089970 ] Alexander Rojas commented on MESOS-7792: [r/60913/|https://reviews.apache.org/r/60913/]: Adds support for OpenSSL's ECDH handshake. > Add support for ECDH ciphers > > > Key: MESOS-7792 > URL: https://issues.apache.org/jira/browse/MESOS-7792 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 1.3.0 >Reporter: Alexander Rojas >Assignee: Alexander Rojas > > [Elliptic curve > ciphers|https://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography] are a > family of ciphers supported by OpenSSL. They allow to have smaller keys, but > require an extra configuration parameter, the actual curve to be used, which > can't be done through libprocess as it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MESOS-7792) Add support for ECDH ciphers
[ https://issues.apache.org/jira/browse/MESOS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rojas updated MESOS-7792: --- Sprint: Mesosphere Sprint 59 > Add support for ECDH ciphers > > > Key: MESOS-7792 > URL: https://issues.apache.org/jira/browse/MESOS-7792 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 1.3.0 >Reporter: Alexander Rojas >Assignee: Alexander Rojas > > [Elliptic curve > ciphers|https://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography] are a > family of ciphers supported by OpenSSL. They allow to have smaller keys, but > require an extra configuration parameter, the actual curve to be used, which > can't be done through libprocess as it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (MESOS-7792) Add support for ECDH ciphers
Alexander Rojas created MESOS-7792: -- Summary: Add support for ECDH ciphers Key: MESOS-7792 URL: https://issues.apache.org/jira/browse/MESOS-7792 Project: Mesos Issue Type: Bug Components: libprocess Affects Versions: 1.3.0 Reporter: Alexander Rojas Assignee: Alexander Rojas [Elliptic curve ciphers|https://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography] are a family of ciphers supported by OpenSSL. They allow to have smaller keys, but require an extra configuration parameter, the actual curve to be used, which can't be done through libprocess as it is. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-7775) Eliminate extra process abort in a subprocess watchdog
[
https://issues.apache.org/jira/browse/MESOS-7775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085583#comment-16085583
]
Alexander Rojas commented on MESOS-7775:
{noformat}
commit 72ebb5e5b5fe6403db0f0b051821d3346ff6da22
Author: Andrei Budnik
AuthorDate: Thu Jul 13 13:35:26 2017 +0200
Commit: Alexander Rojas
CommitDate: Thu Jul 13 13:35:26 2017 +0200
Replaced `abort()` with `_exit()` in `ChildHook::SUPERVISOR`.
Previously, `abort()` was called in `SUPERVISOR` hook when child
process exited with an error code, or `waitpid()` failed, or parent
process exited. All these cases shouldn't lead to abnormal program
termination with coredumps.
Review: https://reviews.apache.org/r/60598/
{noformat}
> Eliminate extra process abort in a subprocess watchdog
> --
>
> Key: MESOS-7775
> URL: https://issues.apache.org/jira/browse/MESOS-7775
> Project: Mesos
> Issue Type: Bug
>Reporter: Andrei Budnik
>Assignee: Andrei Budnik
> Labels: mesosphere, tech-debt
>
> `abort()` is called in `SUPERVISOR` hook when child process exits with an
> error code, or `waitpid()` fails, or parent process exits. All these cases
> shouldn't lead to abnormal program termination with coredumps.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-7779) Consider showing aggregated anonymous data on filtered results.
Alexander Rojas created MESOS-7779: -- Summary: Consider showing aggregated anonymous data on filtered results. Key: MESOS-7779 URL: https://issues.apache.org/jira/browse/MESOS-7779 Project: Mesos Issue Type: Bug Reporter: Alexander Rojas Priority: Minor Some of the endpoints/json results filter data like resources, however this leads to Data not adding up, for example an entry would show the total resources of an agent, another would show the free resources and another the used ones, however, used resources are filtered based in the role creating a situation in which adding the free resources + used resources ≠ total resources. Perhaps an anonymous aggregated field used to show filtered data so that the results add up could do the trick. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
[
https://issues.apache.org/jira/browse/MESOS-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16059310#comment-16059310
]
Alexander Rojas edited comment on MESOS-7414 at 6/22/17 1:05 PM:
-
{noformat}
commit 9ea7fae68477c7cb8e33e04fd32616a44ee9be2b
Author: Alexander Rojas alexan...@mesosphere.io
Date: Fri Jun 16 20:29:50 2017 +0200
Adds authorization for the master's v1 API call SET_LOGGING_LEVEL.
Enables authorization in the v1 API call `SET_LOGGING_LEVEL` following
the existing implementation for the agent call with the same name.
Likewise it reuses the authorizer action `SET_LOG_LEVEL`.
Review: https://reviews.apache.org/r/58955/
{noformat}
was (Author: arojas):
{no-format}
commit 9ea7fae68477c7cb8e33e04fd32616a44ee9be2b
Author: Alexander Rojas alexan...@mesosphere.io
Date: Fri Jun 16 20:29:50 2017 +0200
Adds authorization for the master's v1 API call SET_LOGGING_LEVEL.
Enables authorization in the v1 API call `SET_LOGGING_LEVEL` following
the existing implementation for the agent call with the same name.
Likewise it reuses the authorizer action `SET_LOG_LEVEL`.
Review: https://reviews.apache.org/r/58955/
{no-format}
> Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and
> SET_LOGGING_LEVEL
> -
>
> Key: MESOS-7414
> URL: https://issues.apache.org/jira/browse/MESOS-7414
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, operator, security
> Fix For: 1.4.0
>
>
> The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}} lack
> authorization so any recognized user will be able to change the logging level
> of a given master.
> The v0 endpoint {{/logging/toggle}} has authorization through the
> {{GET_ENDPOINT_WITH_PATH}} action. We need to decide whether it should also
> use additional authorization.
> Note that there are already actions defined for authorization of these
> actions as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (MESOS-7708) Some of the headers/functions defined in `src/common` should probably be in `include`
Alexander Rojas created MESOS-7708:
--
Summary: Some of the headers/functions defined in `src/common`
should probably be in `include`
Key: MESOS-7708
URL: https://issues.apache.org/jira/browse/MESOS-7708
Project: Mesos
Issue Type: Bug
Components: modules
Reporter: Alexander Rojas
Priority: Minor
Some of the functions/methods defined in {{src/common/}} are useful enough to
be part of {{include/mesos/common}}. In particular, functions declared in
{{protobuf_utils.hpp}} are helpful enough to module writers that would be
better located in the public API.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (MESOS-7708) Some of the headers/functions defined in `src/common` should probably be in `include`
[
https://issues.apache.org/jira/browse/MESOS-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7708:
---
Labels: mesosphere (was: )
> Some of the headers/functions defined in `src/common` should probably be in
> `include`
> -
>
> Key: MESOS-7708
> URL: https://issues.apache.org/jira/browse/MESOS-7708
> Project: Mesos
> Issue Type: Bug
> Components: modules
>Reporter: Alexander Rojas
>Priority: Minor
> Labels: mesosphere
>
> Some of the functions/methods defined in {{src/common/}} are useful enough to
> be part of {{include/mesos/common}}. In particular, functions declared in
> {{protobuf_utils.hpp}} are helpful enough to module writers that would be
> better located in the public API.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (MESOS-7583) _scheduler.so segfault when Mesos is built with Xcode 8.3.2
[
https://issues.apache.org/jira/browse/MESOS-7583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029439#comment-16029439
]
Alexander Rojas commented on MESOS-7583:
Could not reproduce as described here, however running
{{./builc/src/mesos-tests --gtest_filter="ExamplesTest.PythonFramework"}} does
crash and yields:
{noformat}
Process: Python [4146]
Path:
/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
Identifier:Python
Version: 2.7.13 (2.7.13)
Code Type: X86-64 (Native)
Parent Process:mesos-tests [4126]
Responsible: Python [4146]
Crashed Thread:6
Exception Type:EXC_CRASH (SIGABRT)
Exception Codes: 0x, 0x
Exception Note:EXC_CORPSE_NOTIFY
Application Specific Information:
abort() called
*** error for object 0x110f68290: pointer being freed was not allocated
Thread 0:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x7fff9d1babf2 __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x7fff9d2a686e _pthread_cond_wait +
712
2 libc++.1.dylib 0x000111a10b60
std::__1::condition_variable::wait(std::__1::unique_lock&) + 18
3 _scheduler.so 0x00011a2e5c2f void
synchronized_wait(std::__1::condition_variable*, std::__1::mutex*) + 175
4 _scheduler.so 0x00011d30988c Gate::arrive(long) +
188
5 _scheduler.so 0x00011d309263
process::ProcessManager::wait(process::UPID const&) + 3411
6 _scheduler.so 0x00011d300a4b
process::wait(process::UPID const&, Duration const&) + 715
7 _scheduler.so 0x00011d1c35f2
process::Latch::await(Duration const&) + 162
8 _scheduler.so 0x00011ac8d497
mesos::MesosSchedulerDriver::join() + 615
9 _scheduler.so 0x00011ac8d86f
mesos::MesosSchedulerDriver::run() + 63
10 _scheduler.so 0x000119cd84de
mesos::python::MesosSchedulerDriverImpl_run(mesos::python::MesosSchedulerDriverImpl*)
+ 94 (mesos_scheduler_driver_impl.cpp:388)
11 org.python.python 0x00010ee47884 PyEval_EvalFrameEx +
26960
12 org.python.python 0x00010ee40d3e PyEval_EvalCodeEx +
1617
13 org.python.python 0x00010ee406e7 PyEval_EvalCode + 48
14 org.python.python 0x00010ee64068 run_mod + 53
15 org.python.python 0x00010ee6410b PyRun_FileExFlags +
133
16 org.python.python 0x00010ee63c5c
PyRun_SimpleFileExFlags + 702
17 org.python.python 0x00010ee75442 Py_Main + 3094
18 libdyld.dylib 0x7fff9d08c235 start + 1
Thread 1:
0 libsystem_kernel.dylib 0x7fff9d1babf2 __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x7fff9d2a686e _pthread_cond_wait +
712
2 libc++.1.dylib 0x000111a10b60
std::__1::condition_variable::wait(std::__1::unique_lock&) + 18
3 _scheduler.so 0x00011a2e5c2f void
synchronized_wait(std::__1::condition_variable*, std::__1::mutex*) + 175
4 _scheduler.so 0x00011d30988c Gate::arrive(long) +
188
5 _scheduler.so 0x00011d43c6a7
process::ProcessManager::init_threads()::$_1::operator()() const + 231
6 _scheduler.so 0x00011d43c2f2 void*
std::__1::__thread_proxy
>(void*) + 386
7 libsystem_pthread.dylib 0x7fff9d2a59af _pthread_body + 180
8 libsystem_pthread.dylib 0x7fff9d2a58fb _pthread_start + 286
9 libsystem_pthread.dylib 0x7fff9d2a5101 thread_start + 13
Thread 2:
0 libsystem_kernel.dylib 0x7fff9d1babf2 __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x7fff9d2a686e _pthread_cond_wait +
712
2 libc++.1.dylib 0x000111a10b60
std::__1::condition_variable::wait(std::__1::unique_lock&) + 18
3 _scheduler.so 0x00011a2e5c2f void
synchronized_wait(std::__1::condition_variable*, std::__1::mutex*) + 175
4 _scheduler.so 0x00011d30988c Gate::arrive(long) +
188
5 _scheduler.so 0x00011d43c6a7
process::ProcessManager::init_threads()::$_1::operator()() const + 231
6 _scheduler.so 0x00011d43c2f2 void*
std::__1::__thread_proxy
>(void*) + 386
7 libsystem_pthread.dylib 0x7fff9d2a59af _pthread_body + 180
8 libsystem_pthread.dylib 0x7fff9d2a58fb _pthread_start + 286
9 libsystem_pthread.dylib 0x7fff9d2a5101
[jira] [Updated] (MESOS-7587) Launching tasks with the Mesos Containerizer after a long time without launching new tasks fails
[
https://issues.apache.org/jira/browse/MESOS-7587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7587:
---
Description:
After having a cluster running without launching new tasks for an extended
period of time, ~1week. When launching a new task using the Mesos
Containerizer, the task fails to launch with the error:
[{{Failed to execute command: No such file or
directory}}|https://github.com/apache/mesos/blob/8245981b889ec3725cc0be4150b15d1fe9d64b86/src/slave/containerizer/mesos/launch.cpp#L778]
The task is launched from Marathon with the app definition:
{code}
{
"container": {
"type": "MESOS",
"docker": {
"forcePullImage": true,
"image": "private.repository.local/updated:fixed",
"privileged": false
}
},
"cpus": 0.1,
"id": "/20150530/mesos9",
"instances": 1,
"minimumHealthCapacity": 1,
"acceptedResourceRoles": ["*"],
"constraints": [["hostname", "UNIQUE"]],
"mem": 128
}
{code}
and {{Dockerfile}}
{code}
FROM private.repository.local/centos:stable
MAINTAINER Sebastian Gerlach "s...@boreus.de"
CMD python -m SimpleHTTPServer 80
{code}
The obtained stdout is:
{noformat}
Executing pre-exec command
'{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/usr\/libexec\/mesos\/mesos-containerizer"}'
Executing pre-exec command
'{"arguments":["mount","-n","--rbind","\/var\/lib\/mesos\/slaves\/562f1892-41c6-4c55-955e-662c63ab3ddf-S8\/frameworks\/562f1892-41c6-4c55-955e-662c63ab3ddf-\/executors\/20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0\/runs\/da1cb066-1354-42a6-82bc-eee688d61b94","\/var\/lib\/mesos\/provisioner\/containers\/da1cb066-1354-42a6-82bc-eee688d61b94\/backends\/overlay\/rootfses\/f74816b3-c115-4a5e-a51c-fbd502efd1fe\/mnt\/mesos\/sandbox"],"shell":false,"value":"mount"}'
Received SUBSCRIBED event
Subscribed executor on bp-mesos8.private.local
Received LAUNCH event
Starting task 20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0
/usr/libexec/mesos/mesos-containerizer launch --help="false"
--launch_info="{"command":{"arguments":["\/bin\/sh","-c","python -m
SimpleHTTPServer
80"],"environment":{"variables":[{"name":"MARATHON_APP_VERSION","value":"2017-05-30T09:06:59.666Z"},{"name":"HOST","value":"bp-mesos8.private.local"},{"name":"MARATHON_APP_RESOURCE_CPUS","value":"0.1"},{"name":"MARATHON_APP_RESOURCE_GPUS","value":"0"},{"name":"MESOS_TASK_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"MARATHON_APP_RESOURCE_MEM","value":"128.0"},{"name":"MARATHON_APP_RESOURCE_DISK","value":"0.0"},{"name":"MARATHON_APP_LABELS","value":""},{"name":"MARATHON_APP_ID","value":"\/20150530\/mesos13"}]},"shell":false,"value":"\/bin\/sh"},"environment":{"variables":[{"name":"HOST","value":"bp-mesos8.private.local"},{"name":"LIBPROCESS_IP","value":"0.0.0.0"},{"name":"LIBPROCESS_PORT","value":"0"},{"name":"MARATHON_APP_ID","value":"\/20150530\/mesos13"},{"name":"MARATHON_APP_LABELS","value":""},{"name":"MARATHON_APP_RESOURCE_CPUS","value":"0.1"},{"name":"MARATHON_APP_RESOURCE_DISK","value":"0.0"},{"name":"MARATHON_APP_RESOURCE_GPUS","value":"0"},{"name":"MARATHON_APP_RESOURCE_MEM","value":"128.0"},{"name":"MARATHON_APP_VERSION","value":"2017-05-30T09:06:59.666Z"},{"name":"MESOS_AGENT_ENDPOINT","value":"10.107.19.26:5051"},{"name":"MESOS_CHECKPOINT","value":"1"},{"name":"MESOS_DIRECTORY","value":"\/var\/lib\/mesos\/slaves\/562f1892-41c6-4c55-955e-662c63ab3ddf-S8\/frameworks\/562f1892-41c6-4c55-955e-662c63ab3ddf-\/executors\/20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0\/runs\/da1cb066-1354-42a6-82bc-eee688d61b94"},{"name":"MESOS_EXECUTOR_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD","value":"5secs"},{"name":"MESOS_FRAMEWORK_ID","value":"562f1892-41c6-4c55-955e-662c63ab3ddf-"},{"name":"MESOS_HTTP_COMMAND_EXECUTOR","value":"0"},{"name":"MESOS_NATIVE_JAVA_LIBRARY","value":"\/usr\/lib\/libmesos-1.2.0.so"},{"name":"MESOS_NATIVE_LIBRARY","value":"\/usr\/lib\/libmesos-1.2.0.so"},{"name":"MESOS_RECOVERY_TIMEOUT","value":"15mins"},{"name":"MESOS_SANDBOX","value":"\/mnt\/mesos\/sandbox"},{"name":"MESOS_SLAVE_ID","value":"562f1892-41c6-4c55-955e-662c63ab3ddf-S8"},{"name":"MESOS_SLAVE_PID","value":"slave(1)@10.107.19.26:5051"},{"name":"MESOS_SUBSCRIPTION_BACKOFF_MAX","value":"2secs"},{"name":"MESOS_TASK_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"PATH","value":"\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin:\/sbin:\/bin"},{"name":"PATH","value":"\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin:\/sbin:\/bin"},{"name":"ftp_proxy","value":"http:\/\/bp-proxy.private.local:3128"},{"name":"http_proxy","value":"http:\/\/bp-proxy.private.local:3128"},{"name":"https_proxy","value":"http:\/\/bp-proxy.
[jira] [Created] (MESOS-7587) Launching tasks with the Mesos Containerizer after a long time without launching new tasks fails
Alexander Rojas created MESOS-7587:
--
Summary: Launching tasks with the Mesos Containerizer after a long
time without launching new tasks fails
Key: MESOS-7587
URL: https://issues.apache.org/jira/browse/MESOS-7587
Project: Mesos
Issue Type: Bug
Components: containerization
Affects Versions: 1.2.0
Reporter: Alexander Rojas
After having a cluster running without launching new tasks for an extended
period of time, ~1week. When launching a new task using the Mesos
Containerizer, the task fails to launch with the error:
[{{Failed to execute command: No such file or
directory}}|https://github.com/apache/mesos/blob/8245981b889ec3725cc0be4150b15d1fe9d64b86/src/slave/containerizer/mesos/launch.cpp#L778]
The task is launched from Marathon with the app definition:
{code}
{
"container": {
"type": "MESOS",
"docker": {
"forcePullImage": true,
"image": "private.repository.local/updated:fixed",
"privileged": false
}
},
"cpus": 0.1,
"id": "/20150530/mesos9",
"instances": 1,
"minimumHealthCapacity": 1,
"acceptedResourceRoles": ["*"],
"constraints": [["hostname", "UNIQUE"]],
"mem": 128
}
{code}
and {{Docerfile}}
{code}
FROM private.repository.local/centos:stable
MAINTAINER Sebastian Gerlach "s...@boreus.de"
CMD python -m SimpleHTTPServer 80
{code}
The obtained stdout is:
{noformat}
Executing pre-exec command
'{"arguments":["mesos-containerizer","mount","--help=false","--operation=make-rslave","--path=\/"],"shell":false,"value":"\/usr\/libexec\/mesos\/mesos-containerizer"}'
Executing pre-exec command
'{"arguments":["mount","-n","--rbind","\/var\/lib\/mesos\/slaves\/562f1892-41c6-4c55-955e-662c63ab3ddf-S8\/frameworks\/562f1892-41c6-4c55-955e-662c63ab3ddf-\/executors\/20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0\/runs\/da1cb066-1354-42a6-82bc-eee688d61b94","\/var\/lib\/mesos\/provisioner\/containers\/da1cb066-1354-42a6-82bc-eee688d61b94\/backends\/overlay\/rootfses\/f74816b3-c115-4a5e-a51c-fbd502efd1fe\/mnt\/mesos\/sandbox"],"shell":false,"value":"mount"}'
Received SUBSCRIBED event
Subscribed executor on bp-mesos8.private.local
Received LAUNCH event
Starting task 20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0
/usr/libexec/mesos/mesos-containerizer launch --help="false"
--launch_info="{"command":{"arguments":["\/bin\/sh","-c","python -m
SimpleHTTPServer
80"],"environment":{"variables":[{"name":"MARATHON_APP_VERSION","value":"2017-05-30T09:06:59.666Z"},{"name":"HOST","value":"bp-mesos8.private.local"},{"name":"MARATHON_APP_RESOURCE_CPUS","value":"0.1"},{"name":"MARATHON_APP_RESOURCE_GPUS","value":"0"},{"name":"MESOS_TASK_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"MARATHON_APP_RESOURCE_MEM","value":"128.0"},{"name":"MARATHON_APP_RESOURCE_DISK","value":"0.0"},{"name":"MARATHON_APP_LABELS","value":""},{"name":"MARATHON_APP_ID","value":"\/20150530\/mesos13"}]},"shell":false,"value":"\/bin\/sh"},"environment":{"variables":[{"name":"HOST","value":"bp-mesos8.private.local"},{"name":"LIBPROCESS_IP","value":"0.0.0.0"},{"name":"LIBPROCESS_PORT","value":"0"},{"name":"MARATHON_APP_ID","value":"\/20150530\/mesos13"},{"name":"MARATHON_APP_LABELS","value":""},{"name":"MARATHON_APP_RESOURCE_CPUS","value":"0.1"},{"name":"MARATHON_APP_RESOURCE_DISK","value":"0.0"},{"name":"MARATHON_APP_RESOURCE_GPUS","value":"0"},{"name":"MARATHON_APP_RESOURCE_MEM","value":"128.0"},{"name":"MARATHON_APP_VERSION","value":"2017-05-30T09:06:59.666Z"},{"name":"MESOS_AGENT_ENDPOINT","value":"10.107.19.26:5051"},{"name":"MESOS_CHECKPOINT","value":"1"},{"name":"MESOS_DIRECTORY","value":"\/var\/lib\/mesos\/slaves\/562f1892-41c6-4c55-955e-662c63ab3ddf-S8\/frameworks\/562f1892-41c6-4c55-955e-662c63ab3ddf-\/executors\/20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0\/runs\/da1cb066-1354-42a6-82bc-eee688d61b94"},{"name":"MESOS_EXECUTOR_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"MESOS_EXECUTOR_SHUTDOWN_GRACE_PERIOD","value":"5secs"},{"name":"MESOS_FRAMEWORK_ID","value":"562f1892-41c6-4c55-955e-662c63ab3ddf-"},{"name":"MESOS_HTTP_COMMAND_EXECUTOR","value":"0"},{"name":"MESOS_NATIVE_JAVA_LIBRARY","value":"\/usr\/lib\/libmesos-1.2.0.so"},{"name":"MESOS_NATIVE_LIBRARY","value":"\/usr\/lib\/libmesos-1.2.0.so"},{"name":"MESOS_RECOVERY_TIMEOUT","value":"15mins"},{"name":"MESOS_SANDBOX","value":"\/mnt\/mesos\/sandbox"},{"name":"MESOS_SLAVE_ID","value":"562f1892-41c6-4c55-955e-662c63ab3ddf-S8"},{"name":"MESOS_SLAVE_PID","value":"slave(1)@10.107.19.26:5051"},{"name":"MESOS_SUBSCRIPTION_BACKOFF_MAX","value":"2secs"},{"name":"MESOS_TASK_ID","value":"20150530_mesos13.5f8c3db1-4517-11e7-8d95-024276be29e0"},{"name":"PATH","value":"\/usr\/local\/sbin:\/usr\/local\/bin:\/usr\/sbin:\/usr\/bin:\/sbin:\/bin"},{"name":"PATH","value":"\/usr\/local\/sbin:
[jira] [Created] (MESOS-7530) HTTP authenticators modules never get the realm they are registered for
Alexander Rojas created MESOS-7530:
--
Summary: HTTP authenticators modules never get the realm they are
registered for
Key: MESOS-7530
URL: https://issues.apache.org/jira/browse/MESOS-7530
Project: Mesos
Issue Type: Bug
Components: modules, security
Affects Versions: 1.3.0
Reporter: Alexander Rojas
When someone creates a module to provide HTTP Authenticator, only in the master
it can be register for one of three realms:
* {{READONLY_HTTP_AUTHENTICATION_REALM}}
* {{READWRITE_HTTP_AUTHENTICATION_REALM}}
* {{DEFAULT_HTTP_FRAMEWORK_AUTHENTICATION_REALM}}
These realms are passed to the HTTP basic authenticator when it is constructed:
{code}
Result createBasicAuthenticator(
const string& realm,
const string& authenticatorName,
const Option& credentials)
{
if (credentials.isNone()) {
return Error(
"No credentials provided for the default '" +
string(internal::DEFAULT_BASIC_HTTP_AUTHENTICATOR) +
"' HTTP authenticator for realm '" + realm + "'");
}
LOG(INFO) << "Creating default '"
<< internal::DEFAULT_BASIC_HTTP_AUTHENTICATOR
<< "' HTTP authenticator for realm '" << realm << "'";
return BasicAuthenticatorFactory::create(realm, credentials.get());
}
{code}
However modules don't get to configure their configured realm at construction
and the API doesn't allow to change that afterwards:
{code}
Result createCustomAuthenticator(
const string& realm,
const string& authenticatorName)
{
if (!modules::ModuleManager::contains<
process::http::authentication::Authenticator>(authenticatorName)) {
return Error(
"HTTP authenticator '" + authenticatorName + "' not found. "
"Check the spelling (compare to '" +
string(internal::DEFAULT_BASIC_HTTP_AUTHENTICATOR) +
"') or verify that the authenticator was loaded "
"successfully (see --modules)");
}
LOG(INFO) << "Creating '" << authenticatorName << "' HTTP authenticator "
<< "for realm '" << realm << "'";
return modules::ModuleManager::create<
process::http::authentication::Authenticator>(authenticatorName);
}
{code}
Since the same authenticator module is used for all the realms, it is
impossible to provide one authenticator per realm if using modules.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7489) Reduce code duplication when requesting an ObjectApprover
Alexander Rojas created MESOS-7489:
--
Summary: Reduce code duplication when requesting an ObjectApprover
Key: MESOS-7489
URL: https://issues.apache.org/jira/browse/MESOS-7489
Project: Mesos
Issue Type: Improvement
Components: agent, master, security
Reporter: Alexander Rojas
Priority: Minor
Every time the code needs to use an authorizer, the following snippet is used:
{code}
Future> somethingApprover;
if (master->authorizer.isSome()) {
Option subject = createSubject(principal);
somethingApprover = master->authorizer.get()->getObjectApprover(
subject, authorization::VIEW_FRAMEWORK);
} else {
somethingApprover = Owned(new AcceptingObjectApprover());
}
return somethingApprover.then(…)
{code}
This can be easily replace my a method:
{code}
Future Master::approver(
const Option &subject,
const authorization::Action &action)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
[
https://issues.apache.org/jira/browse/MESOS-7415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7415:
---
Story Points: 3 (was: 2)
> Add authorization to master's operator maintenance API in v0 and v1
> ---
>
> Key: MESOS-7415
> URL: https://issues.apache.org/jira/browse/MESOS-7415
> Project: Mesos
> Issue Type: Task
> Components: c++ api, HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: authorization, mesosphere, security
>
> None of the maintenance primitives in either API v0 or API v1 have any kind
> of authorization, which allows any user with valid credentials to do things
> such as shutting down a machine, schedule time off on an agent, modify
> maintenance schedule, etc.
> The authorization support needs to be added to the v0 endpoints:
> * {{/master/machine/up}}
> * {{/master/machine/down}}
> * {{/master/maintenance/schedule}}
> * {{/master/maintenance/status}}
> as well as to the v1 calls:
> * {{GET_MAINTENANCE_STATUS}}
> * {{GET_MAINTENANCE_SCHEDULE}}
> * {{UPDATE_MAINTENANCE_SCHEDULE}}
> * {{START_MAINTENANCE}}
> * {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
[
https://issues.apache.org/jira/browse/MESOS-7415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994679#comment-15994679
]
Alexander Rojas commented on MESOS-7415:
h2. Analysis
Each of the API v1 calls can be paired to one of the API v0 endpoints, that
means that after some transformations, the handler of both API's ultimately
call the same method. The pairing works as follows:
* {{GET_MAINTENANCE_SCHEDULE}} pairs with {{/maintenance/schedule}} if the
second's method is {{GET}}.
* {{UPDATE_MAINTENANCE_SCHEDULE}} pairs with {{/maintenance/schedule}} if the
second's method is {{POST}}.
* {{GET_MAINTENANCE_STATUS}} pairs with {{/maintenance/status}} and only
accepts {{GET}}.
* {{START_MAINTENANCE}} pairs with {{/machine/down}} and only accepts {{POST}}.
* {{STOP_MAINTENANCE}} pairs with {{/machine/up}} and only accepts {{POST}}.
h3. Objects
The most likely candidates to be used as authorization objects are the machine
ids, if we resolve that an administrator only has control over a subset of the
machines, likewise one can assume that the operator is a super user across the
whole cluster in which case the object will only be used to define global ACLs
(like {{ANY}} or {{NONE}}).
I also suggest to leave {{GET_MAINTENANCE_SCHEDULE}} without authorization
since it is something relevant for all kinds of users thinking of make use of
resource in certain machines (One can think of a scheduler verifying the
schedule before accepting an offer).
> Add authorization to master's operator maintenance API in v0 and v1
> ---
>
> Key: MESOS-7415
> URL: https://issues.apache.org/jira/browse/MESOS-7415
> Project: Mesos
> Issue Type: Task
> Components: c++ api, HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: authorization, mesosphere, security
>
> None of the maintenance primitives in either API v0 or API v1 have any kind
> of authorization, which allows any user with valid credentials to do things
> such as shutting down a machine, schedule time off on an agent, modify
> maintenance schedule, etc.
> The authorization support needs to be added to the v0 endpoints:
> * {{/master/machine/up}}
> * {{/master/machine/down}}
> * {{/master/maintenance/schedule}}
> * {{/master/maintenance/status}}
> as well as to the v1 calls:
> * {{GET_MAINTENANCE_STATUS}}
> * {{GET_MAINTENANCE_SCHEDULE}}
> * {{UPDATE_MAINTENANCE_SCHEDULE}}
> * {{START_MAINTENANCE}}
> * {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
[
https://issues.apache.org/jira/browse/MESOS-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992977#comment-15992977
]
Alexander Rojas commented on MESOS-7414:
We won't be adding authorization to {{GET_LOGGING_LEVEL}} since it doesn't
affect the behavior of Mesos not it is consider critical to be protected.
{{/logging/toggle}} is a libprocess endpoint and authorization is added at
Mesos level, since it already contains authorization with the v0 API, it is not
a priority to use the same authz as v1, likewise it is not trivial to authorize
both calls.
> Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and
> SET_LOGGING_LEVEL
> -
>
> Key: MESOS-7414
> URL: https://issues.apache.org/jira/browse/MESOS-7414
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, operator, security
>
> The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}} lack
> authorization so any recognized user will be able to change the logging level
> of a given master.
> The v0 endpoint {{/logging/toggle}} has authorization through the
> {{GET_ENDPOINT_WITH_PATH}} action. We need to decide whether it should also
> use additional authorization.
> Note that there are already actions defined for authorization of these
> actions as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
[
https://issues.apache.org/jira/browse/MESOS-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7414:
---
Description:
The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}} lack
authorization so any recognized user will be able to change the logging level
of a given master.
The v0 endpoint {{/logging/toggle}} has authorization through the
{{GET_ENDPOINT_WITH_PATH}} action. We need to decide whether it should also use
additional authorization.
Note that there are already actions defined for authorization of these actions
as they were already implemented in the agent.
was:
The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}}, as
well as the v0 endpoint {{/logging/toggle}} lack authorization so any
recognized user will be able to change the logging level of a given master.
Note that there are already actions defined for authorization of these actions
as they were already implemented in the agent.
> Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and
> SET_LOGGING_LEVEL
> -
>
> Key: MESOS-7414
> URL: https://issues.apache.org/jira/browse/MESOS-7414
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, operator, security
>
> The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}} lack
> authorization so any recognized user will be able to change the logging level
> of a given master.
> The v0 endpoint {{/logging/toggle}} has authorization through the
> {{GET_ENDPOINT_WITH_PATH}} action. We need to decide whether it should also
> use additional authorization.
> Note that there are already actions defined for authorization of these
> actions as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
[
https://issues.apache.org/jira/browse/MESOS-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7414:
---
Shepherd: Adam B
Sprint: Mesosphere Sprint 56
Story Points: 5
> Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and
> SET_LOGGING_LEVEL
> -
>
> Key: MESOS-7414
> URL: https://issues.apache.org/jira/browse/MESOS-7414
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, operator, security
>
> The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}}, as
> well as the v0 endpoint {{/logging/toggle}} lack authorization so any
> recognized user will be able to change the logging level of a given master.
> Note that there are already actions defined for authorization of these
> actions as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
[
https://issues.apache.org/jira/browse/MESOS-7415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7415:
---
Shepherd: Adam B
Sprint: Mesosphere Sprint 56
Story Points: 2
> Add authorization to master's operator maintenance API in v0 and v1
> ---
>
> Key: MESOS-7415
> URL: https://issues.apache.org/jira/browse/MESOS-7415
> Project: Mesos
> Issue Type: Task
> Components: c++ api, HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: authorization, mesosphere, security
>
> None of the maintenance primitives in either API v0 or API v1 have any kind
> of authorization, which allows any user with valid credentials to do things
> such as shutting down a machine, schedule time off on an agent, modify
> maintenance schedule, etc.
> The authorization support needs to be added to the v0 endpoints:
> * {{/master/machine/up}}
> * {{/master/machine/down}}
> * {{/master/maintenance/schedule}}
> * {{/master/maintenance/status}}
> as well as to the v1 calls:
> * {{GET_MAINTENANCE_STATUS}}
> * {{GET_MAINTENANCE_SCHEDULE}}
> * {{UPDATE_MAINTENANCE_SCHEDULE}}
> * {{START_MAINTENANCE}}
> * {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7416) Filter results of `/master/slaves` and the v1 call GET_AGENTS
[
https://issues.apache.org/jira/browse/MESOS-7416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7416:
---
Shepherd: Adam B
Sprint: Mesosphere Sprint 56
Story Points: 2
> Filter results of `/master/slaves` and the v1 call GET_AGENTS
> -
>
> Key: MESOS-7416
> URL: https://issues.apache.org/jira/browse/MESOS-7416
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, security
>
> The results returned by both the endpoint {{/master/slaves}} and the API v1
> {{GET_AGENTS}} return full information about the agent state which probably
> need to be filtered for certain uses, particularly in a multi-tenancy
> scenario.
> The kind of leaked data includes specific role names and their specific
> allocations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MESOS-7260) Authorization for `/role` endpoint should take both VIEW_ROLES and VIEW_FRAMEWORKS into account.
[
https://issues.apache.org/jira/browse/MESOS-7260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992563#comment-15992563
]
Alexander Rojas commented on MESOS-7260:
Sure, I've been already reviewing these patches.
> Authorization for `/role` endpoint should take both VIEW_ROLES and
> VIEW_FRAMEWORKS into account.
>
>
> Key: MESOS-7260
> URL: https://issues.apache.org/jira/browse/MESOS-7260
> Project: Mesos
> Issue Type: Bug
> Components: HTTP API, master
>Reporter: Jay Guo
>Assignee: Jay Guo
>
> Consider following case: both {{framework1}} and {{framework2}} subscribe to
> {{roleX}}, {{principal}} is allowed to view {{roleX}} and {{framework1}}, but
> *NOT* {{framework2}}, therefore, {{/role}} endpoint should only contain
> {{framework1}}, but not both frameworks.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MESOS-7247) HTTP Authenticator modules should be able to redirect users
[ https://issues.apache.org/jira/browse/MESOS-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988681#comment-15988681 ] Alexander Rojas commented on MESOS-7247: The reason the authenticator is not allowed to return any kind of response, is that if it could, anyone could write an authorizer to spoof messages, partially or completely, addressed to mesos. I personally am not even very fond that the authorizer is able to read the whole message, and I would have preferred that it only had access to the headers. At the same time, I don't think you are interested in returned all kinds of returned messages, so adding the option to return a 3XX message should be enoug. > HTTP Authenticator modules should be able to redirect users > --- > > Key: MESOS-7247 > URL: https://issues.apache.org/jira/browse/MESOS-7247 > Project: Mesos > Issue Type: Improvement > Components: agent, libprocess, master >Reporter: Silas Snider >Assignee: Silas Snider > Labels: mesosphere > > RIght now, Autheticator modules can only respond with an Unauthorized HTTP > status code if they need to get auth information from the client. This works > for Basic auth, but not for authentication types like oauth, which expect the > server to redirect the client to the right authorization provider URL. > We should change AuthenticationResult to allow arbitrary http responses to > allow for more flexibility here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (MESOS-7247) HTTP Authenticator modules should be able to redirect users
[ https://issues.apache.org/jira/browse/MESOS-7247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988681#comment-15988681 ] Alexander Rojas edited comment on MESOS-7247 at 4/28/17 11:34 AM: -- The reason the authenticator is not allowed to return any kind of response, is that if it could, anyone could write an authorizer to spoof messages, partially or completely, addressed to mesos. I personally am not even very fond that the authorizer is able to read the whole message, and I would have preferred that it only had access to the headers. At the same time, I don't think you are interested in returned all kinds of returned messages, so adding the option to return a 3XX message should be enough. was (Author: arojas): The reason the authenticator is not allowed to return any kind of response, is that if it could, anyone could write an authorizer to spoof messages, partially or completely, addressed to mesos. I personally am not even very fond that the authorizer is able to read the whole message, and I would have preferred that it only had access to the headers. At the same time, I don't think you are interested in returned all kinds of returned messages, so adding the option to return a 3XX message should be enoug. > HTTP Authenticator modules should be able to redirect users > --- > > Key: MESOS-7247 > URL: https://issues.apache.org/jira/browse/MESOS-7247 > Project: Mesos > Issue Type: Improvement > Components: agent, libprocess, master >Reporter: Silas Snider >Assignee: Silas Snider > Labels: mesosphere > > RIght now, Autheticator modules can only respond with an Unauthorized HTTP > status code if they need to get auth information from the client. This works > for Basic auth, but not for authentication types like oauth, which expect the > server to redirect the client to the right authorization provider URL. > We should change AuthenticationResult to allow arbitrary http responses to > allow for more flexibility here. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7416) Filter results of `/master/slaves` and the v1 call GET_AGENTS
[
https://issues.apache.org/jira/browse/MESOS-7416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7416:
--
Assignee: Alexander Rojas
> Filter results of `/master/slaves` and the v1 call GET_AGENTS
> -
>
> Key: MESOS-7416
> URL: https://issues.apache.org/jira/browse/MESOS-7416
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, security
>
> The results returned by both the endpoint {{/master/slaves}} and the API v1
> {{GET_AGENTS}} return full information about the agent state which probably
> need to be filtered for certain uses, particularly in a multi-tenancy
> scenario.
> The kind of leaked data includes specific role names and their specific
> allocations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
[
https://issues.apache.org/jira/browse/MESOS-7415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7415:
--
Assignee: Alexander Rojas
> Add authorization to master's operator maintenance API in v0 and v1
> ---
>
> Key: MESOS-7415
> URL: https://issues.apache.org/jira/browse/MESOS-7415
> Project: Mesos
> Issue Type: Task
> Components: c++ api, HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: authorization, mesosphere, security
>
> None of the maintenance primitives in either API v0 or API v1 have any kind
> of authorization, which allows any user with valid credentials to do things
> such as shutting down a machine, schedule time off on an agent, modify
> maintenance schedule, etc.
> The authorization support needs to be added to the v0 endpoints:
> * {{/master/machine/up}}
> * {{/master/machine/down}}
> * {{/master/maintenance/schedule}}
> * {{/master/maintenance/status}}
> as well as to the v1 calls:
> * {{GET_MAINTENANCE_STATUS}}
> * {{GET_MAINTENANCE_SCHEDULE}}
> * {{UPDATE_MAINTENANCE_SCHEDULE}}
> * {{START_MAINTENANCE}}
> * {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
[
https://issues.apache.org/jira/browse/MESOS-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7414:
--
Assignee: Alexander Rojas
> Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and
> SET_LOGGING_LEVEL
> -
>
> Key: MESOS-7414
> URL: https://issues.apache.org/jira/browse/MESOS-7414
> Project: Mesos
> Issue Type: Task
> Components: HTTP API, master
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
> Labels: mesosphere, operator, security
>
> The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}}, as
> well as the v0 endpoint {{/logging/toggle}} lack authorization so any
> recognized user will be able to change the logging level of a given master.
> Note that there are already actions defined for authorization of these
> actions as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7416) Filter results of `/master/slaves` and the v1 call GET_AGENTS
Alexander Rojas created MESOS-7416:
--
Summary: Filter results of `/master/slaves` and the v1 call
GET_AGENTS
Key: MESOS-7416
URL: https://issues.apache.org/jira/browse/MESOS-7416
Project: Mesos
Issue Type: Task
Components: HTTP API, master
Reporter: Alexander Rojas
The results returned by both the endpoint {{/master/slaves}} and the API v1
{{GET_AGENTS}} return full information about the agent state which probably
need to be filtered for certain uses, particularly in a multi-tenancy scenario.
The kind of leaked data includes specific role names and their specific
allocations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
[
https://issues.apache.org/jira/browse/MESOS-7415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7415:
---
Labels: authorization mesosphere security (was: )
> Add authorization to master's operator maintenance API in v0 and v1
> ---
>
> Key: MESOS-7415
> URL: https://issues.apache.org/jira/browse/MESOS-7415
> Project: Mesos
> Issue Type: Task
> Components: c++ api, HTTP API, master
>Reporter: Alexander Rojas
> Labels: authorization, mesosphere, security
>
> None of the maintenance primitives in either API v0 or API v1 have any kind
> of authorization, which allows any user with valid credentials to do things
> such as shutting down a machine, schedule time off on an agent, modify
> maintenance schedule, etc.
> The authorization support needs to be added to the v0 endpoints:
> * {{/master/machine/up}}
> * {{/master/machine/down}}
> * {{/master/maintenance/schedule}}
> * {{/master/maintenance/status}}
> as well as to the v1 calls:
> * {{GET_MAINTENANCE_STATUS}}
> * {{GET_MAINTENANCE_SCHEDULE}}
> * {{UPDATE_MAINTENANCE_SCHEDULE}}
> * {{START_MAINTENANCE}}
> * {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7415) Add authorization to master's operator maintenance API in v0 and v1
Alexander Rojas created MESOS-7415:
--
Summary: Add authorization to master's operator maintenance API in
v0 and v1
Key: MESOS-7415
URL: https://issues.apache.org/jira/browse/MESOS-7415
Project: Mesos
Issue Type: Task
Components: c++ api, HTTP API, master
Reporter: Alexander Rojas
None of the maintenance primitives in either API v0 or API v1 have any kind of
authorization, which allows any user with valid credentials to do things such
as shutting down a machine, schedule time off on an agent, modify maintenance
schedule, etc.
The authorization support needs to be added to the v0 endpoints:
* {{/master/machine/up}}
* {{/master/machine/down}}
* {{/master/maintenance/schedule}}
* {{/master/maintenance/status}}
as well as to the v1 calls:
* {{GET_MAINTENANCE_STATUS}}
* {{GET_MAINTENANCE_SCHEDULE}}
* {{UPDATE_MAINTENANCE_SCHEDULE}}
* {{START_MAINTENANCE}}
* {{STOP_MAINTENANCE}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7414) Enable authorization for master's logging API calls: GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
Alexander Rojas created MESOS-7414:
--
Summary: Enable authorization for master's logging API calls:
GET_LOGGING_LEVEL and SET_LOGGING_LEVEL
Key: MESOS-7414
URL: https://issues.apache.org/jira/browse/MESOS-7414
Project: Mesos
Issue Type: Task
Components: HTTP API, master
Reporter: Alexander Rojas
The Operator API calls {{GET_LOGGING_LEVEL}} and {{SET_LOGGING_LEVEL}}, as
well as the v0 endpoint {{/logging/toggle}} lack authorization so any
recognized user will be able to change the logging level of a given master.
Note that there are already actions defined for authorization of these actions
as they were already implemented in the agent.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (MESOS-7012) Add authorization actions for V1 executor calls
[ https://issues.apache.org/jira/browse/MESOS-7012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rojas reassigned MESOS-7012: -- Assignee: Alexander Rojas > Add authorization actions for V1 executor calls > --- > > Key: MESOS-7012 > URL: https://issues.apache.org/jira/browse/MESOS-7012 > Project: Mesos > Issue Type: Task > Components: agent, executor, security >Reporter: Greg Mann >Assignee: Alexander Rojas > Labels: authorization, executor, mesosphere, protobuf, security > > Authorization actions should be added for the V1 executor calls: > * Subscribe > * Update > * Message -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (MESOS-7320) Remove deprecated ACL `ShutdownFramework`
[
https://issues.apache.org/jira/browse/MESOS-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7320:
--
Assignee: Alexander Rojas
> Remove deprecated ACL `ShutdownFramework`
> -
>
> Key: MESOS-7320
> URL: https://issues.apache.org/jira/browse/MESOS-7320
> Project: Mesos
> Issue Type: Task
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>
> The ACLs {{ShutdownFramework}} was marked for deprecation more than six
> months ago, where it was replaced for the more convenient
> {{TeardownFramework}} ACL. The deprecation cycle for this action is finally
> due
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7320) Remove deprecated ACL `ShutdownFramework`
Alexander Rojas created MESOS-7320:
--
Summary: Remove deprecated ACL `ShutdownFramework`
Key: MESOS-7320
URL: https://issues.apache.org/jira/browse/MESOS-7320
Project: Mesos
Issue Type: Task
Reporter: Alexander Rojas
The ACLs {{ShutdownFramework}} was marked for deprecation more than six months
ago, where it was replaced for the more convenient {{TeardownFramework}} ACL.
The deprecation cycle for this action is finally due
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (MESOS-7259) Remove deprecated ACLs `SetQuota` and `RemoveQuota`
[
https://issues.apache.org/jira/browse/MESOS-7259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-7259:
--
Assignee: Alexander Rojas
> Remove deprecated ACLs `SetQuota` and `RemoveQuota`
> ---
>
> Key: MESOS-7259
> URL: https://issues.apache.org/jira/browse/MESOS-7259
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rojas
>Assignee: Alexander Rojas
>
> The ACLs {{SetQuota}} and {{RemoveQuota}} where marked for deprecation more
> than six months ago, where they were replaced for the more convenient
> {{UpdateQuota}} ACL. The deprecation cycle for these actions is finally due
> while at the same time removing will make easier to implement the
> hierarchical role feature in a generic way.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (MESOS-7259) Remove deprecated ACLs `SetQuota` and `RemoveQuota`
Alexander Rojas created MESOS-7259:
--
Summary: Remove deprecated ACLs `SetQuota` and `RemoveQuota`
Key: MESOS-7259
URL: https://issues.apache.org/jira/browse/MESOS-7259
Project: Mesos
Issue Type: Bug
Reporter: Alexander Rojas
The ACLs {{SetQuota}} and {{RemoveQuota}} where marked for deprecation more
than six months ago, where they were replaced for the more convenient
{{UpdateQuota}} ACL. The deprecation cycle for these actions is finally due
while at the same time removing will make easier to implement the hierarchical
role feature in a generic way.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (MESOS-5824) Include disk source information in stringification
[
https://issues.apache.org/jira/browse/MESOS-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas reassigned MESOS-5824:
--
Assignee: (was: Tim Harper)
> Include disk source information in stringification
> --
>
> Key: MESOS-5824
> URL: https://issues.apache.org/jira/browse/MESOS-5824
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.28.2
>Reporter: Tim Harper
>Priority: Minor
> Labels: mesosphere
>
> Some frameworks (like kafka_mesos) ignore the Source field when trying to
> reserve an offered mount or path persistent volume; the resulting error
> message is bewildering:
> {code:none}
> Task uses more resources
> cpus(*):4; mem(*):4096; ports(*):[31000-31000]; disk(kafka,
> kafka)[kafka_0:data]:960679
> than available
> cpus(*):32; mem(*):256819; ports(*):[31000-32000]; disk(kafka,
> kafka)[kafka_0:data]:960679; disk(*):240169;
> {code}
> The stringification of disk resources should include source information.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (MESOS-5824) Include disk source information in stringification
[
https://issues.apache.org/jira/browse/MESOS-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15896945#comment-15896945
]
Alexander Rojas commented on MESOS-5824:
Review is closed due to inactivity.
> Include disk source information in stringification
> --
>
> Key: MESOS-5824
> URL: https://issues.apache.org/jira/browse/MESOS-5824
> Project: Mesos
> Issue Type: Improvement
> Components: stout
>Affects Versions: 0.28.2
>Reporter: Tim Harper
>Priority: Minor
> Labels: mesosphere
>
> Some frameworks (like kafka_mesos) ignore the Source field when trying to
> reserve an offered mount or path persistent volume; the resulting error
> message is bewildering:
> {code:none}
> Task uses more resources
> cpus(*):4; mem(*):4096; ports(*):[31000-31000]; disk(kafka,
> kafka)[kafka_0:data]:960679
> than available
> cpus(*):32; mem(*):256819; ports(*):[31000-32000]; disk(kafka,
> kafka)[kafka_0:data]:960679; disk(*):240169;
> {code}
> The stringification of disk resources should include source information.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Issue Comment Deleted] (MESOS-7146) OSX broken due to wrong configuration of LevelDB after update.
[
https://issues.apache.org/jira/browse/MESOS-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7146:
---
Comment: was deleted
(was: The issue that caused the problem)
> OSX broken due to wrong configuration of LevelDB after update.
> --
>
> Key: MESOS-7146
> URL: https://issues.apache.org/jira/browse/MESOS-7146
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rojas
>Priority: Blocker
> Labels: mesosphere
>
> The commit
> [74878e255bb099029dde2a03e0b1d22fecf16000|https://reviews.apache.org/r/51053/]
> broke the build in OS-X. On a first run it will break with the following
> message:
> {noformat}
> checking if clang supports -c -o file.o... yes
> checking for poll.h... yes
> checking sys/select.h usability... checking if clang supports -fno-rtti
> -fno-exceptions... yes
> checking if clang supports -c -o file.o... yes
> (cached) yes
> checking for clang option to produce PIC... checking whether the clang linker
> (/usr/bin/ld) supports shared libraries... -fno-common -DPIC
> checking if clang PIC flag -fno-common -DPIC works... yes
> checking dynamic linker characteristics... mkdir out-shared
> clang++ -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1
> _TUPLE=1 -DGTEST_LANG_CXX11 -I. -I./include -std=c++0x -DOS_MACOSX
> -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibil
> ity-inlines-hidden -fcolor-diagnostics -Wno-unused-local-typedef -std=c++11
> -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -fPIC -c
> db/db_bench.cc -o out-shared/db/db_bench.o
> yes
> checking sys/select.h presence... error: unable to open output file
> 'out-shared/db/db_bench.o': 'No such file or directory'
> 1 error generated.
> make[4]: *** [out-shared/db/db_bench.o] Error 1
> make[3]: *** [leveldb-1.19/out-static/libleveldb.a] Error 2
> make[3]: *** Waiting for unfinished jobs
> {noformat}
> if one re-runs the make command, then it fails with the following error:
> {noformat}
> clang -I. -I./include -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -c db/c_test.c -o
> out-static/db/c_test.o
> error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
> {noformat}
> This one indicates that the C compiler is using CXXFLAGS instead of CFLAGS or
> that CFLAGS are being wrongly generated. Running a thir time the make command
> throes the following output:
> {noformat}
> clang -I. -I./include -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagno$
> tics -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -c db/c_test.c -o
> out-static/db/c_test.o
> clang++ -L/usr/local/opt/subversion/lib -L/usr/local/opt/openssl/lib
> -L/usr/local/opt/libevent/lib -L/usr/local/opt/apr/libexec/lib
> -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
> -fcolor-diagnostics -stdlib=libc++ -nostdi$
> c++ -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -I$
> -I./include -std=c++0x -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-dia$
> nostics -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC db/corruption_test.cc
> out-static/db/builder.o out-static/db/c.o out-static/db/db_impl.o
> out-static/db/db_iter.o out-static/d$
> /dbformat.o out-static/db/dumpfile.o out-static/db/filename.o
> out-static/db/log_reader.o out-static/db/log_writer.o
> out-static/db/memtable.o out-static/db/repair.o out-static/db/table_cache.o
> out-static/db/version_edit.o out-static/db/ve$
> sion_set.o out-static/db/write_batch.o out-static/table/block.o
> out-static/table/block_builder.o out-static/table/filter_block.o
> out-static/table/f
[jira] [Updated] (MESOS-7146) OSX broken due to wrong configuration of LevelDB after update.
[
https://issues.apache.org/jira/browse/MESOS-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Rojas updated MESOS-7146:
---
Labels: mesosphere (was: )
> OSX broken due to wrong configuration of LevelDB after update.
> --
>
> Key: MESOS-7146
> URL: https://issues.apache.org/jira/browse/MESOS-7146
> Project: Mesos
> Issue Type: Bug
>Reporter: Alexander Rojas
>Priority: Blocker
> Labels: mesosphere
>
> The commit
> [74878e255bb099029dde2a03e0b1d22fecf16000|https://reviews.apache.org/r/51053/]
> broke the build in OS-X. On a first run it will break with the following
> message:
> {noformat}
> checking if clang supports -c -o file.o... yes
> checking for poll.h... yes
> checking sys/select.h usability... checking if clang supports -fno-rtti
> -fno-exceptions... yes
> checking if clang supports -c -o file.o... yes
> (cached) yes
> checking for clang option to produce PIC... checking whether the clang linker
> (/usr/bin/ld) supports shared libraries... -fno-common -DPIC
> checking if clang PIC flag -fno-common -DPIC works... yes
> checking dynamic linker characteristics... mkdir out-shared
> clang++ -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++ -DGTEST_USE_OWN_TR1
> _TUPLE=1 -DGTEST_LANG_CXX11 -I. -I./include -std=c++0x -DOS_MACOSX
> -DLEVELDB_PLATFORM_POSIX -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibil
> ity-inlines-hidden -fcolor-diagnostics -Wno-unused-local-typedef -std=c++11
> -stdlib=libc++ -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -fPIC -c
> db/db_bench.cc -o out-shared/db/db_bench.o
> yes
> checking sys/select.h presence... error: unable to open output file
> 'out-shared/db/db_bench.o': 'No such file or directory'
> 1 error generated.
> make[4]: *** [out-shared/db/db_bench.o] Error 1
> make[3]: *** [leveldb-1.19/out-static/libleveldb.a] Error 2
> make[3]: *** Waiting for unfinished jobs
> {noformat}
> if one re-runs the make command, then it fails with the following error:
> {noformat}
> clang -I. -I./include -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -c db/c_test.c -o
> out-static/db/c_test.o
> error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
> {noformat}
> This one indicates that the C compiler is using CXXFLAGS instead of CFLAGS or
> that CFLAGS are being wrongly generated. Running a thir time the make command
> throes the following output:
> {noformat}
> clang -I. -I./include -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagno$
> tics -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC -c db/c_test.c -o
> out-static/db/c_test.o
> clang++ -L/usr/local/opt/subversion/lib -L/usr/local/opt/openssl/lib
> -L/usr/local/opt/libevent/lib -L/usr/local/opt/apr/libexec/lib
> -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
> -fcolor-diagnostics -stdlib=libc++ -nostdi$
> c++ -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-diagnostics
> -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -I$
> -I./include -std=c++0x -DOS_MACOSX -DLEVELDB_PLATFORM_POSIX
> -DLEVELDB_ATOMIC_PRESENT -stdlib=libc++ -nostdinc++
> -I/usr/local/opt/llvm@3.8/lib/llvm-3.8/include/c++/v1
> -Wno-deprecated-declarations -fvisibility-inlines-hidden -fcolor-dia$
> nostics -Wno-unused-local-typedef -std=c++11 -stdlib=libc++
> -DGTEST_USE_OWN_TR1_TUPLE=1 -DGTEST_LANG_CXX11 -fPIC db/corruption_test.cc
> out-static/db/builder.o out-static/db/c.o out-static/db/db_impl.o
> out-static/db/db_iter.o out-static/d$
> /dbformat.o out-static/db/dumpfile.o out-static/db/filename.o
> out-static/db/log_reader.o out-static/db/log_writer.o
> out-static/db/memtable.o out-static/db/repair.o out-static/db/table_cache.o
> out-static/db/version_edit.o out-static/db/ve$
> sion_set.o out-static/db/write_batch.o out-static/table/block.o
> out-static/table/block_builder.o out-static/table/filter_block.o
> out-static/table/format.o out-static/table/iterator.o
