[jira] [Comment Edited] (MESOS-6113) Offer Quota resources as revocable
[
https://issues.apache.org/jira/browse/MESOS-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460347#comment-15460347
]
Guangya Liu edited comment on MESOS-6113 at 9/3/16 4:35 AM:
Does the section section in MESOS-4392 help? It is saying lend out the un-used
quota to other framework and reclaim them back when needed.
{code}
A greedy analytics batch system wants to use as much of the cluster as possible
to maximize computational throughput. When a competing web service with fixed
task size starts up, there must be sufficient resources to run it immediately.
The operator can reserve these resources by setting quota. However, if these
resources are kept idle until the service is in use, this is wasteful from the
analytics job's point of view. On the other hand, the analytics job should hand
back reserved resources to the service when needed to avoid starvation of the
latter.
{code}
was (Author: gyliu):
Does the section section in MESOS-4392 help? It is saying lend out the un-used
quota to other framework and reclaim them back when needed.
{quota}
A greedy analytics batch system wants to use as much of the cluster as possible
to maximize computational throughput. When a competing web service with fixed
task size starts up, there must be sufficient resources to run it immediately.
The operator can reserve these resources by setting quota. However, if these
resources are kept idle until the service is in use, this is wasteful from the
analytics job's point of view. On the other hand, the analytics job should hand
back reserved resources to the service when needed to avoid starvation of the
latter.
{quota}
> Offer Quota resources as revocable
> --
>
> Key: MESOS-6113
> URL: https://issues.apache.org/jira/browse/MESOS-6113
> Project: Mesos
> Issue Type: Task
> Components: allocation
>Affects Versions: 1.0.1
>Reporter: Michael Gummelt
>
> *Goal:*
> I have high-priority Spark jobs, and best-effort jobs. I need my
> high-priority jobs to pre-empt my best-effort jobs, so I'd like to launch the
> best-effort jobs on revocable resources.
> *Problem:*
> Revocable resources are currently only created via oversubscription, where
> resources allocated to but not used by a framework will be offered to other
> frameworks. This doesn't support the ability for a high-pri framework to
> start up and pre-empty a low-pri framework.
> *Solution:*
> Let's allow quota (and ideally any reserved resources) to be configurable to
> be offered as revocable resources to other frameworks that don't register
> with the role.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-1027) IPv6 support
[ https://issues.apache.org/jira/browse/MESOS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459689#comment-15459689 ] Evgeny Petrov edited comment on MESOS-1027 at 9/2/16 10:01 PM: --- BTW, there is a patch https://github.com/lava/mesos which is done by us, and work for us for a while. But contribution process hangs on "Find a shepherd" step. p.s. Not sure if it solves all ipv6 problems (for example docker-related), probably it solves only some set of subtasks. was (Author: golovasteek): BTW, there is a patch https://github.com/lava/mesos which is done by us, and work for us for a while. But contribution process hangs on "Find a shepherd" step. > IPv6 support > > > Key: MESOS-1027 > URL: https://issues.apache.org/jira/browse/MESOS-1027 > Project: Mesos > Issue Type: Epic > Components: framework, libprocess, master, slave >Reporter: Dominic Hamon > > From the CLI down through the various layers of tech we should support IPv6. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1027) IPv6 support
[ https://issues.apache.org/jira/browse/MESOS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459689#comment-15459689 ] Evgeny Petrov commented on MESOS-1027: -- BTW, there is a patch https://github.com/lava/mesos which is done by us, and work for us for a while. But contribution process hangs on "Find a shepherd" step. > IPv6 support > > > Key: MESOS-1027 > URL: https://issues.apache.org/jira/browse/MESOS-1027 > Project: Mesos > Issue Type: Epic > Components: framework, libprocess, master, slave >Reporter: Dominic Hamon > > From the CLI down through the various layers of tech we should support IPv6. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1027) IPv6 support
[ https://issues.apache.org/jira/browse/MESOS-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459613#comment-15459613 ] Nickolay Sukhanov commented on MESOS-1027: -- Any progress here? It's a huge blocker nowadays because more and more data-centers are becoming IPv6 only. If you can lead me, I can try to help. > IPv6 support > > > Key: MESOS-1027 > URL: https://issues.apache.org/jira/browse/MESOS-1027 > Project: Mesos > Issue Type: Epic > Components: framework, libprocess, master, slave >Reporter: Dominic Hamon > > From the CLI down through the various layers of tech we should support IPv6. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6118) Agent would crash with docker container tasks due to host mount table read.
[ https://issues.apache.org/jira/browse/MESOS-6118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459510#comment-15459510 ] Jie Yu commented on MESOS-6118: --- I committed the above patch to head as well. > Agent would crash with docker container tasks due to host mount table read. > --- > > Key: MESOS-6118 > URL: https://issues.apache.org/jira/browse/MESOS-6118 > Project: Mesos > Issue Type: Bug > Components: slave >Affects Versions: 1.0.1 > Environment: Build: 2016-08-26 23:06:27 by centos > Version: 1.0.1 > Git tag: 1.0.1 > Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3 > systemd version `219` detected > Inializing systemd state > Created systemd slice: `/run/systemd/system/mesos_executors.slice` > Started systemd slice `mesos_executors.slice` > Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni > Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher > Linux ip-10-254-192-40 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu Aug 18 19:05:49 > UTC 2016 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jamie Briant >Assignee: Kevin Klues >Priority: Critical > Labels: linux, slave > Fix For: 1.1.0, 1.0.2 > > Attachments: slave-crash.log > > > I have a framework which schedules thousands of short running (a few seconds > to a few minutes) of tasks, over a period of several minutes. In 1.0.1, the > slave process will crash every few minutes (with systemd restarting it). > Crash is: > Sep 01 20:52:23 ip-10-254-192-99 mesos-slave: F0901 20:52:23.905678 1232 > fs.cpp:140] Check failed: !visitedParents.contains(parentId) > Sep 01 20:52:23 ip-10-254-192-99 mesos-slave: *** Check failure stack trace: > *** > Version 1.0.0 works without this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6121) Unable to make on 1.0.1 version
[
https://issues.apache.org/jira/browse/MESOS-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459474#comment-15459474
]
Benjamin Bannier commented on MESOS-6121:
-
Could you share some more information about your setup, in particular the
configure flags you used? Also, above you mention as environment ubuntu-14.04,
while the build output appears to be from ubuntu-12.04 ({{precise}}). Which one
is it?
> Unable to make on 1.0.1 version
>
>
> Key: MESOS-6121
> URL: https://issues.apache.org/jira/browse/MESOS-6121
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: Ubuntu 14.04
>Reporter: Ankur Verma
>
> Unable to make on 1.0.1 version , getting errors
> vagrant@precise32:~/mesos/build$ make -j 1
> Making all in .
> make[1]: Entering directory `/home/vagrant/mesos/build'
> make[1]: Nothing to be done for `all-am'.
> make[1]: Leaving directory `/home/vagrant/mesos/build'
> Making all in 3rdparty
> make[1]: Entering directory `/home/vagrant/mesos/build/3rdparty'
> make all-recursive
> make[2]: Entering directory `/home/vagrant/mesos/build/3rdparty'
> Making all in .
> make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty'
> make[3]: Nothing to be done for `all-am'.
> make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty'
> Making all in stout
> make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout'
> make all-recursive
> make[4]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout'
> Making all in .
> make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout'
> make[5]: Nothing to be done for `all-am'.
> make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout'
> Making all in include
> make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout/include'
> make[5]: Nothing to be done for `all'.
> make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout/include'
> make[4]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout'
> make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout'
> Making all in libprocess
> make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> make all-recursive
> make[4]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> Making all in .
> make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> make[5]: Nothing to be done for `all-am'.
> make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> Making all in include
> make[5]: Entering directory
> `/home/vagrant/mesos/build/3rdparty/libprocess/include'
> make[5]: Nothing to be done for `all'.
> make[5]: Leaving directory
> `/home/vagrant/mesos/build/3rdparty/libprocess/include'
> make[4]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess'
> make[2]: Leaving directory `/home/vagrant/mesos/build/3rdparty'
> make[1]: Leaving directory `/home/vagrant/mesos/build/3rdparty'
> Making all in src
> make[1]: Entering directory `/home/vagrant/mesos/build/src'
> /usr/lib/jvm/java-7-openjdk-i386/bin/javah -d java/jni
> \
> -classpath
> java/target/mesos-1.1.0.jar:/home/vagrant/mesos/build/src/java/target/protobuf-java-2.6.1.jar
> \
> org.apache.mesos.Log
> /usr/lib/jvm/java-7-openjdk-i386/bin/javah -d java/jni
> \
> -classpath
> java/target/mesos-1.1.0.jar:/home/vagrant/mesos/build/src/java/target/protobuf-java-2.6.1.jar
> \
> org.apache.mesos.MesosExecutorDriver
> Error: Class com.google.protobuf.GeneratedMessage could not be found.
> make[1]: *** [java/jni/org_apache_mesos_MesosExecutorDriver.h] Error 1
> make[1]: Leaving directory `/home/vagrant/mesos/build/src'
> make: *** [all-recursive] Error 1
> vagrant@precise32:~/mesos/build$ cd ..
> vagrant@precise32:~/mesos$ ls
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-6121) Unable to make on 1.0.1 version
Ankur Verma created MESOS-6121: -- Summary: Unable to make on 1.0.1 version Key: MESOS-6121 URL: https://issues.apache.org/jira/browse/MESOS-6121 Project: Mesos Issue Type: Bug Affects Versions: 1.0.1 Environment: Ubuntu 14.04 Reporter: Ankur Verma Unable to make on 1.0.1 version , getting errors vagrant@precise32:~/mesos/build$ make -j 1 Making all in . make[1]: Entering directory `/home/vagrant/mesos/build' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/vagrant/mesos/build' Making all in 3rdparty make[1]: Entering directory `/home/vagrant/mesos/build/3rdparty' make all-recursive make[2]: Entering directory `/home/vagrant/mesos/build/3rdparty' Making all in . make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty' Making all in stout make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout' make all-recursive make[4]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout' Making all in . make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout' Making all in include make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/stout/include' make[5]: Nothing to be done for `all'. make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout/include' make[4]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout' make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty/stout' Making all in libprocess make[3]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess' make all-recursive make[4]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess' Making all in . make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess' make[5]: Nothing to be done for `all-am'. make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess' Making all in include make[5]: Entering directory `/home/vagrant/mesos/build/3rdparty/libprocess/include' make[5]: Nothing to be done for `all'. make[5]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess/include' make[4]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess' make[3]: Leaving directory `/home/vagrant/mesos/build/3rdparty/libprocess' make[2]: Leaving directory `/home/vagrant/mesos/build/3rdparty' make[1]: Leaving directory `/home/vagrant/mesos/build/3rdparty' Making all in src make[1]: Entering directory `/home/vagrant/mesos/build/src' /usr/lib/jvm/java-7-openjdk-i386/bin/javah -d java/jni \ -classpath java/target/mesos-1.1.0.jar:/home/vagrant/mesos/build/src/java/target/protobuf-java-2.6.1.jar \ org.apache.mesos.Log /usr/lib/jvm/java-7-openjdk-i386/bin/javah -d java/jni \ -classpath java/target/mesos-1.1.0.jar:/home/vagrant/mesos/build/src/java/target/protobuf-java-2.6.1.jar \ org.apache.mesos.MesosExecutorDriver Error: Class com.google.protobuf.GeneratedMessage could not be found. make[1]: *** [java/jni/org_apache_mesos_MesosExecutorDriver.h] Error 1 make[1]: Leaving directory `/home/vagrant/mesos/build/src' make: *** [all-recursive] Error 1 vagrant@precise32:~/mesos/build$ cd .. vagrant@precise32:~/mesos$ ls -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6118) Agent would crash with docker container tasks due to host mount table read.
[ https://issues.apache.org/jira/browse/MESOS-6118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459411#comment-15459411 ] Jamie Briant commented on MESOS-6118: - I'm working on setting up a dev environment for mesos. I'll let you know how it goes. > Agent would crash with docker container tasks due to host mount table read. > --- > > Key: MESOS-6118 > URL: https://issues.apache.org/jira/browse/MESOS-6118 > Project: Mesos > Issue Type: Bug > Components: slave >Affects Versions: 1.0.1 > Environment: Build: 2016-08-26 23:06:27 by centos > Version: 1.0.1 > Git tag: 1.0.1 > Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3 > systemd version `219` detected > Inializing systemd state > Created systemd slice: `/run/systemd/system/mesos_executors.slice` > Started systemd slice `mesos_executors.slice` > Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni > Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher > Linux ip-10-254-192-40 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu Aug 18 19:05:49 > UTC 2016 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Jamie Briant >Assignee: Kevin Klues >Priority: Critical > Labels: linux, slave > Fix For: 1.1.0, 1.0.2 > > Attachments: slave-crash.log > > > I have a framework which schedules thousands of short running (a few seconds > to a few minutes) of tasks, over a period of several minutes. In 1.0.1, the > slave process will crash every few minutes (with systemd restarting it). > Crash is: > Sep 01 20:52:23 ip-10-254-192-99 mesos-slave: F0901 20:52:23.905678 1232 > fs.cpp:140] Check failed: !visitedParents.contains(parentId) > Sep 01 20:52:23 ip-10-254-192-99 mesos-slave: *** Check failure stack trace: > *** > Version 1.0.0 works without this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6113) Offer Quota resources as revocable
[ https://issues.apache.org/jira/browse/MESOS-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15459148#comment-15459148 ] Michael Gummelt commented on MESOS-6113: Maybe. It's not clear to me from the title nor the description that MESOS-4392 is proposing to mark quota as revocable. > Offer Quota resources as revocable > -- > > Key: MESOS-6113 > URL: https://issues.apache.org/jira/browse/MESOS-6113 > Project: Mesos > Issue Type: Task > Components: allocation >Affects Versions: 1.0.1 >Reporter: Michael Gummelt > > *Goal:* > I have high-priority Spark jobs, and best-effort jobs. I need my > high-priority jobs to pre-empt my best-effort jobs, so I'd like to launch the > best-effort jobs on revocable resources. > *Problem:* > Revocable resources are currently only created via oversubscription, where > resources allocated to but not used by a framework will be offered to other > frameworks. This doesn't support the ability for a high-pri framework to > start up and pre-empty a low-pri framework. > *Solution:* > Let's allow quota (and ideally any reserved resources) to be configurable to > be offered as revocable resources to other frameworks that don't register > with the role. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-4907) ClangTidy Integration
[
https://issues.apache.org/jira/browse/MESOS-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458864#comment-15458864
]
Michael Park commented on MESOS-4907:
-
{noformat}
commit e6f6357037a7a9d5fda1be4906a5a46deb451626
Author: Benjamin Bannier
Date: Fri Sep 2 10:10:29 2016 +0200
Properly quoted string in shell script.
This was intended to be formatted as code in markdown style, instead
it was interpreted as a command to run. Use single quotes to prevent
interpretion of the string.
Review: https://reviews.apache.org/r/51594/
{noformat}
> ClangTidy Integration
> -
>
> Key: MESOS-4907
> URL: https://issues.apache.org/jira/browse/MESOS-4907
> Project: Mesos
> Issue Type: Epic
> Components: technical debt
>Reporter: Michael Park
> Labels: gsoc, gsoc2016, mentor, mesosphere
>
> While {{cpplint}} has been a useful tool as a C++ linter for quite some time,
> It carries limitations since it does its best without actually parsing C++.
> [ClangTidy|http://clang.llvm.org/extra/clang-tidy/] is a clang tool that is
> based
> off of Clang, and has the advantage that it has access to a full AST.
> There are many checks that come built-in with {{clang-tidy}} which are very
> useful,
> but we can extend it to fit Mesos coding style and patterns as well.
> The initial phase of the project will be to create a basis with which to
> leverage
> the existing checks as applicable to Mesos, then to create a scaffolding to
> add
> custom checks, and ways to integrate the custom checks to infrastructure such
> as Mesos ReviewBot, or Apache CI.
> I've done some preliminary, experimental work for this for a Hackathon project
> and have given a
> [presentation|https://docs.google.com/presentation/d/1z_qGzpY7Mt46TXxuLRW6M5HcCWBLRz6UJfd4bPknYeg/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-3172) Getting Started docs still mentions about v0.22.1
[ https://issues.apache.org/jira/browse/MESOS-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458854#comment-15458854 ] Gavin Baumanis commented on MESOS-3172: --- Hey there, Looks like this needs to get revisited to ensure that latest release is shown. > Getting Started docs still mentions about v0.22.1 > - > > Key: MESOS-3172 > URL: https://issues.apache.org/jira/browse/MESOS-3172 > Project: Mesos > Issue Type: Documentation >Reporter: Ryuichi Okumura >Assignee: Ryuichi Okumura >Priority: Minor > > Mesos v0.23.0 seems to released, but the "Getting Started" docs still > mentions about Mesos v0.22.1. > See: http://mesos.apache.org/gettingstarted/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6120) TCP health checks does not support IPv6.
Alexander Rukletsov created MESOS-6120: -- Summary: TCP health checks does not support IPv6. Key: MESOS-6120 URL: https://issues.apache.org/jira/browse/MESOS-6120 Project: Mesos Issue Type: Bug Reporter: Alexander Rukletsov Though currently Mesos does not officially support IPv6, TCP health check utility should do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-6110) Deprecate using health checks without setting the type
[ https://issues.apache.org/jira/browse/MESOS-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rukletsov updated MESOS-6110: --- Sprint: Mesosphere Sprint 42 (was: Mesosphere Sprint 41) > Deprecate using health checks without setting the type > -- > > Key: MESOS-6110 > URL: https://issues.apache.org/jira/browse/MESOS-6110 > Project: Mesos > Issue Type: Bug > Components: master >Affects Versions: 1.1.0 >Reporter: Silas Snider >Assignee: haosdent >Priority: Blocker > Labels: compatibility, health-check, mesosphere > > When sending a task launch using the 1.0.x protos and the legacy (non-http) > API, tasks with a healthcheck defined are rejected (TASK_ERROR) because the > 'type' field is not set. > This field is marked optional in the proto and is not available before 1.1.0, > so it should not be required in order to keep the mesos v1 api compatibility > promise. > For backwards compatibility temporarily allow the use case when command > health check is set without a type. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-6119) TCP health checks are not portable.
Alexander Rukletsov created MESOS-6119: -- Summary: TCP health checks are not portable. Key: MESOS-6119 URL: https://issues.apache.org/jira/browse/MESOS-6119 Project: Mesos Issue Type: Bug Reporter: Alexander Rukletsov Assignee: Alexander Rukletsov MESOS-3567 introduced a dependency on "bash" for TCP health checks, which is undesirable. We should implement a portable solution for TCP health checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-6013) Use readdir instead of readdir_r
[
https://issues.apache.org/jira/browse/MESOS-6013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458436#comment-15458436
]
Kamil Chmielewski commented on MESOS-6013:
--
Same issue for NixOS https://github.com/NixOS/nixpkgs/issues/18209
> Use readdir instead of readdir_r
>
>
> Key: MESOS-6013
> URL: https://issues.apache.org/jira/browse/MESOS-6013
> Project: Mesos
> Issue Type: Bug
> Components: stout
> Environment: Linux archlinux.vagrant.vm 4.6.4-1-ARCH #1 SMP PREEMPT
> Mon Jul 11 19:12:32 CEST 2016 x86_64 GNU/Linux
>Reporter: Neil Conway
>Assignee: Neil Conway
> Labels: mesosphere
>
> {{readdir_r}} is deprecated in recent versions of glibc
> (https://sourceware.org/ml/libc-alpha/2016-02/msg00093.html). As a result,
> Mesos doesn't build on recent Arch Linux:
> {noformat}
> /bin/sh ../libtool --tag=CXX --mode=compile ccache g++
> -DPACKAGE_NAME=\"mesos\" -DPACKAGE_TARNAME=\"mesos\"
> -DPACKAGE_VERSION=\"1.1.0\" -DPACKAGE_STRING=\"mesos\ 1.1.0\"
> -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\"
> -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1
> -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1
> -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1
> -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1 -DHAVE_PTHREAD_PRIO_INHERIT=1
> -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1 -DHAVE_APR_POOLS_H=1
> -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1 -DHAVE_LIBSASL2=1
> -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1 -DHAVE_SVN_DELTA_H=1
> -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I. -I../../mesos/src -Wall -Werror
> -Wsign-compare -DLIBDIR=\"/usr/local/lib\"
> -DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\"
> -DPKGDATADIR=\"/usr/local/share/mesos\"
> -DPKGMODULEDIR=\"/usr/local/lib/mesos/modules\" -I../../mesos/include
> -I../include -I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS
> -isystem ../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.1
> -I../3rdparty/glog-0.3.3/src -I../3rdparty/leveldb-1.4/include
> -I../../mesos/3rdparty/libprocess/include -I../3rdparty/nvml-352.79
> -I../3rdparty/picojson-1.3.0 -I../3rdparty/protobuf-2.6.1/src
> -I../../mesos/3rdparty/stout/include
> -I../3rdparty/zookeeper-3.4.8/src/c/include
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0
> -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT
> appc/libmesos_no_3rdparty_la-spec.lo -MD -MP -MF
> appc/.deps/libmesos_no_3rdparty_la-spec.Tpo -c -o
> appc/libmesos_no_3rdparty_la-spec.lo `test -f 'appc/spec.cpp' || echo
> '../../mesos/src/'`appc/spec.cpp
> libtool: compile: ccache g++ -DPACKAGE_NAME=\"mesos\"
> -DPACKAGE_TARNAME=\"mesos\" -DPACKAGE_VERSION=\"1.1.0\"
> "-DPACKAGE_STRING=\"mesos 1.1.0\"" -DPACKAGE_BUGREPORT=\"\"
> -DPACKAGE_URL=\"\" -DPACKAGE=\"mesos\" -DVERSION=\"1.1.0\" -DSTDC_HEADERS=1
> -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1
> -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1
> -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_CXX11=1
> -DHAVE_PTHREAD_PRIO_INHERIT=1 -DHAVE_PTHREAD=1 -DHAVE_LIBZ=1 -DHAVE_FTS_H=1
> -DHAVE_APR_POOLS_H=1 -DHAVE_LIBAPR_1=1 -DHAVE_LIBCURL=1 -DMESOS_HAS_JAVA=1
> -DHAVE_LIBSASL2=1 -DHAVE_SVN_VERSION_H=1 -DHAVE_LIBSVN_SUBR_1=1
> -DHAVE_SVN_DELTA_H=1 -DHAVE_LIBSVN_DELTA_1=1 -DHAVE_LIBZ=1 -I.
> -I../../mesos/src -Wall -Werror -Wsign-compare -DLIBDIR=\"/usr/local/lib\"
> -DPKGLIBEXECDIR=\"/usr/local/libexec/mesos\"
> -DPKGDATADIR=\"/usr/local/share/mesos\"
> -DPKGMODULEDIR=\"/usr/local/lib/mesos/modules\" -I../../mesos/include
> -I../include -I../include/mesos -DPICOJSON_USE_INT64 -D__STDC_FORMAT_MACROS
> -isystem ../3rdparty/boost-1.53.0 -I../3rdparty/elfio-3.1
> -I../3rdparty/glog-0.3.3/src -I../3rdparty/leveldb-1.4/include
> -I../../mesos/3rdparty/libprocess/include -I../3rdparty/nvml-352.79
> -I../3rdparty/picojson-1.3.0 -I../3rdparty/protobuf-2.6.1/src
> -I../../mesos/3rdparty/stout/include
> -I../3rdparty/zookeeper-3.4.8/src/c/include
> -I../3rdparty/zookeeper-3.4.8/src/c/generated -DHAS_AUTHENTICATION=1
> -I/usr/include/subversion-1 -I/usr/include/apr-1 -I/usr/include/apr-1.0
> -pthread -g1 -O0 -Wno-unused-local-typedefs -std=c++11 -MT
> appc/libmesos_no_3rdparty_la-spec.lo -MD -MP -MF
> appc/.deps/libmesos_no_3rdparty_la-spec.Tpo -c ../../mesos/src/appc/spec.cpp
> -fPIC -DPIC -o appc/.libs/libmesos_no_3rdparty_la-spec.o
> In file included from ../../mesos/3rdparty/stout/include/stout/os.hpp:52:0,
> from ../../mesos/src/appc/spec.cpp:17:
> ../../mesos/3rdparty/stout/include/stout/os/ls.hpp: In function
>
[jira] [Commented] (MESOS-3333) Allow executors to create nested containers.
[ https://issues.apache.org/jira/browse/MESOS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457721#comment-15457721 ] Vinod Kone commented on MESOS-: --- Yes, executors should be able to (eventually) put tasks into cgroups once MESOS-2449 is completely done. > Allow executors to create nested containers. > > > Key: MESOS- > URL: https://issues.apache.org/jira/browse/MESOS- > Project: Mesos > Issue Type: Epic > Components: containerization, slave >Reporter: Connor Doyle > Labels: mesosphere > > Allow executors to create and manage nested containers. These nested > containers should be isolated and monitored in the same manner as executor > containers are currently managed. > Some high-level thoughts and ideas are captured here: > https://docs.google.com/document/d/1rwDumptgM4jCQQCXWgjeARRHNHkKYRuHJVhwYEFXQRU > CC [~jieyu] [~jdef] [~karya] [~idownes] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-4907) ClangTidy Integration
[
https://issues.apache.org/jira/browse/MESOS-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456320#comment-15456320
]
Michael Park edited comment on MESOS-4907 at 9/2/16 6:52 AM:
-
{noformat}
commit e90752d408713a174630d8b80dff194707a4
Author: Benjamin Bannier
Date: Thu Sep 1 20:43:11 2016 +0200
Added tooling to execute Mesos-specific clang-tidy checks.
This commit adds tooling to both build and execute Mesos-specific
clang-tidy checks. We use a docker container for the execution, and
also provide a script to further streamline the tooling.
Review: https://reviews.apache.org/r/51525/
{noformat}
was (Author: mcypark):
{noformat}
commit b817e82235c65fa7044de60c9ca31fe6e4d60c48
Author: Benjamin Bannier
Date: Thu Sep 1 20:43:11 2016 +0200
Added tooling to execute Mesos-specific clang-tidy checks.
This commit adds tooling to both build and execute Mesos-specific
clang-tidy checks. We use a docker container for the execution, and
also provide a script to further streamline the tooling.
Review: https://reviews.apache.org/r/51525/
{noformat}
> ClangTidy Integration
> -
>
> Key: MESOS-4907
> URL: https://issues.apache.org/jira/browse/MESOS-4907
> Project: Mesos
> Issue Type: Epic
> Components: technical debt
>Reporter: Michael Park
> Labels: gsoc, gsoc2016, mentor, mesosphere
>
> While {{cpplint}} has been a useful tool as a C++ linter for quite some time,
> It carries limitations since it does its best without actually parsing C++.
> [ClangTidy|http://clang.llvm.org/extra/clang-tidy/] is a clang tool that is
> based
> off of Clang, and has the advantage that it has access to a full AST.
> There are many checks that come built-in with {{clang-tidy}} which are very
> useful,
> but we can extend it to fit Mesos coding style and patterns as well.
> The initial phase of the project will be to create a basis with which to
> leverage
> the existing checks as applicable to Mesos, then to create a scaffolding to
> add
> custom checks, and ways to integrate the custom checks to infrastructure such
> as Mesos ReviewBot, or Apache CI.
> I've done some preliminary, experimental work for this for a Hackathon project
> and have given a
> [presentation|https://docs.google.com/presentation/d/1z_qGzpY7Mt46TXxuLRW6M5HcCWBLRz6UJfd4bPknYeg/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6114) ClassNotFoundException shows when loading java class in framework
[ https://issues.apache.org/jira/browse/MESOS-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457673#comment-15457673 ] Sam chen commented on MESOS-6114: - Anyone can help to review this? thanks > ClassNotFoundException shows when loading java class in framework > - > > Key: MESOS-6114 > URL: https://issues.apache.org/jira/browse/MESOS-6114 > Project: Mesos > Issue Type: Bug > Components: framework >Affects Versions: 0.28.1, 0.28.2, 1.0.0, 1.0.1 > Environment: Mesos0.28.1 > Marathon 1.1.1 > os redhat-7.2 (x86-64) > kernel 3.10.0-327 (x86-64) > Java openjdk-1.8.0_65 >Reporter: Sam chen > Labels: patch > Fix For: 0.28.3 > > > 1. When we are trying to develop "scheduler" and "executor" using java > 2. When we use our own java ClassLoader > 3. It throws "ClassNotFoundException" when loading java class > After we investigated Mesos, it created jvm via jni. While > attachcurrentthread, it did not mulipulate context. The below is error log: > I0823 05:54:38.074373 8 logging.cpp:188] INFO level logging started! > I0823 05:54:38.076400 8 exec.cpp:143] Version: 0.28.1 > I0823 05:54:38.08059052 exec.cpp:217] Executor registered on slave > 326a-43cc-42f7-8e55-648bdc8cc9d8-S12 > Exception in thread "Thread-17" java.lang.NoClassDefFoundError: > com/googlecode/aviator/ClassExpression > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:760) > at java.lang.ClassLoader.defineClass(ClassLoader.java:642) > at > com.googlecode.aviator.parser.AviatorClassLoader.defineClass(AviatorClassLoader.java:35) > at > com.googlecode.aviator.code.asm.ASMCodeGenerator.getResult(ASMCodeGenerator.java:664) > at > com.googlecode.aviator.code.OptimizeCodeGenerator.getResult(OptimizeCodeGenerator.java:367) > at > com.googlecode.aviator.parser.ExpressionParser.parse(ExpressionParser.java:681) > at > com.googlecode.aviator.AviatorEvaluator.innerCompile(AviatorEvaluator.java:468) > at > com.googlecode.aviator.AviatorEvaluator.compile(AviatorEvaluator.java:447) > at > com.googlecode.aviator.AviatorEvaluator.compile(AviatorEvaluator.java:495) > at com.cusi.babel.rwsplit.sync.convertor.ExprRule.(ExprRule.java:20) > at > com.cusi.babel.rwsplit.sync.convertor.ConvertRuleFactory.createConvertRule(ConvertRuleFactory.java:15) > at > com.cusi.babel.rwsplit.sync.convertor.ColumnRule.(ColumnRule.java:29) > at com.cusi.babel.rwsplit.sync.convertor.TaskRule.(TaskRule.java:27) > at com.cusi.babel.rwsplit.sync.task.Task.(Task.java:27) > at com.cusi.babel.rwsplit.sync.Engine.startTask(Engine.java:103) > at > com.cusi.babel.rwsplit.sync.mesos.SyncExecutor.launchTask(SyncExecutor.java:82) > Caused by: java.lang.ClassNotFoundException: > com.googlecode.aviator.ClassExpression > at java.lang.ClassLoader.findClass(ClassLoader.java:530) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 17 more > I0823 05:54:38.87330152 exec.cpp:425] Deactivating the executor libprocess -- This message was sent by Atlassian JIRA (v6.3.4#6332)
