[jira] [Assigned] (MESOS-9056) apply-reviews.py messaging and dependency management.
[
https://issues.apache.org/jira/browse/MESOS-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Toenshoff reassigned MESOS-9056:
-
Assignee: Armand Grillet
> apply-reviews.py messaging and dependency management.
> -
>
> Key: MESOS-9056
> URL: https://issues.apache.org/jira/browse/MESOS-9056
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.7.0
>Reporter: Till Toenshoff
>Assignee: Armand Grillet
>Priority: Minor
>
> After running into all sorts of issues connected to the {{-c}} option, at
> some point I tried the {{-k}} option and got the following output;
> {noformat}
> $ support/apply-reviews.py -k -r 67842
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOS_SUPPORT_PYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> 2018-07-07 18:35:07 URL:https://reviews.apache.org/r/67842/diff/raw/
> [5879/5879] -> "67842.patch" [1]
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOS_SUPPORT_PYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> No C++ files to lint
> No JavaScript files to lint
> Checking 1 Python file
> usage: tox [--version] [-h] [--help-ini] [-v] [--showconfig] [-l] [-a]
>[-c CONFIGFILE] [-e envlist] [--notest] [--sdistonly]
>[--installpkg PATH] [--develop] [-i URL] [--pre] [-r]
>[--result-json PATH] [--hashseed SEED] [--force-dep REQ]
>[--sitepackages] [--alwayscopy] [--skip-missing-interpreters]
>[--workdir PATH]
>[args [args ...]]
> tox: error: unrecognized arguments: -qq
> Total errors found: 0
> Congratulations! You have Python 3 installed correctly.
> Please start using the scripts in `support/python3`.
> Please also set the environment variable `MESOS_SUPPORT_PYTHON` to `3`
> so that the Git hooks use the Python 3 scripts.
> [...]
> 1 file changed, 35 insertions(+), 29 deletions(-)
> {noformat}
> It appears that the messaging here is worth reconsidering;
> - instead of "congratulating" three times - how about we warn/error if
> python3 is not installed?
> - apparently {{tox}} is outdated in my virtual-env - maybe we could run
> buildvirtualenv automatically in such cases?
> - "so that the Git hooks use the Python 3 scripts." should not have a forced
> linefeed
> - "No C++ files to lint", "No JavaScript files to lint", "Total errors found:
> 0" seems to have little value for the normal user - how about we make that a
> user activated verbose output?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8770) Use Python3 for Mesos support scripts
[
https://issues.apache.org/jira/browse/MESOS-8770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539864#comment-16539864
]
Benjamin Bannier commented on MESOS-8770:
-
{noformat}
commit 14c68199cb0a35c8f5dc77472ba483a73ae88653
Author: Armand Grillet
Date: Wed Jul 11 12:22:00 2018 +0200
Updated Python virtual environment dependencies.
Fixes the error message when running support/build-virtualenv:
`npm WARN ajv-keywords@3.2.0 requires a peer of ajv@^6.0.0 but none
is installed. You must install peer dependencies yourself.`.
Updates the PyInstaller dependency due to issues with PyInstaller
3.1.1 and Python 3.6.
Review: https://reviews.apache.org/r/67856/
commit 1cc7054d1ae9b6180013353ddb7d5427f01c7cb2
Author: Armand Grillet
Date: Wed Jul 11 12:21:54 2018 +0200
Added virtualenv requirements watcher to Python 3 mesos-style.py.
The Python and JavaScript linters run by mesos-style.py, called
by our Git hooks, require a virtual environment. This virtual
environment needs to be rebuilt by running `build-virtualenv`
each time the pip dependencies of our Python codebase are
updated.
The check `should_build_virtualenv` had not been added to the Python 3
`mesos-style.py` for an unknown reason and this is now fixed.
We pass the Python interpreter used to run `mesos-style.py` when running
`build-virtualenv` in case Python 3 is used to run `mesos-style.py`
but Python 2 is still the default `python`.
Review: https://reviews.apache.org/r/67855/
{noformat}
> Use Python3 for Mesos support scripts
> -
>
> Key: MESOS-8770
> URL: https://issues.apache.org/jira/browse/MESOS-8770
> Project: Mesos
> Issue Type: Task
>Reporter: Benjamin Bannier
>Assignee: Armand Grillet
>Priority: Major
>
> Our Python scripts under {{support/}} currently implicitly assume that
> developers have a python2 environment as their primary Python installation.
> We should consider updating these scripts so that they can be used with a
> python3 installation as well. There exist [some
> resources|http://python-future.org/overview.html#automatic-conversion-to-py2-3-compatible-code]
> on the web documenting best practices and tools for automatic rewrites which
> should get us a long way.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call
[ https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540016#comment-16540016 ] ASF GitHub Bot commented on MESOS-7211: --- Github user thzois closed the pull request at: https://github.com/apache/mesos/pull/301 > Document SUPPRESS HTTP call > --- > > Key: MESOS-7211 > URL: https://issues.apache.org/jira/browse/MESOS-7211 > Project: Mesos > Issue Type: Documentation > Components: documentation >Affects Versions: 1.1.0 >Reporter: Bruce Merry >Priority: Minor > Labels: mesosphere, newbie > > The documentation at > http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list > the SUPPRESS call as one of the call types, but it does seem to be > implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MESOS-8686) Mesos build failed with /permissive- + MSVC on windows
[ https://issues.apache.org/jira/browse/MESOS-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schwartzmeyer reassigned MESOS-8686: --- Assignee: Andrew Schwartzmeyer > Mesos build failed with /permissive- + MSVC on windows > --- > > Key: MESOS-8686 > URL: https://issues.apache.org/jira/browse/MESOS-8686 > Project: Mesos > Issue Type: Bug > Components: build > Environment: VS2017 15.5.7 + windows server 2016 >Reporter: PhoebeHui >Assignee: Andrew Schwartzmeyer >Priority: Major > Labels: windows > > Mesos(master branch) failed with error C2276 when build with permissive- with > MSVC, this should be source issue, the code is trying to use a member > function of a dependent base class. > Noted that this issue only found when compiles with unreleased vctoolset, > that next release of MSVC will have this behavior. > > On line#528 and #529 of > "D:\Mesos\src\3rdparty\libprocess\src\tests\benchmarks.cpp" > dispatch(self(), ::_handler).then( > defer(self(), ::handler, data)); > > Should be > dispatch(this->self(), ::_handler).then( > defer(this->self(), ::handler, data)); > > Failures like > D:\Mesos\src\3rdparty\libprocess\src\tests\benchmarks.cpp(566): error C2276: > '&': illegal operation on bound member function expression > > *Environment:* > VS2017 15.5.7 + windows server 2016 > > *Repro steps:* > # git clone -c core.autocrlf=true [https://github.com/apache/mesos] > D:\mesos\src > # cd d:\mesos\src > # .\bootstrap.bat > # cd.. > # set _CL_=/D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING /permissive- > # mkdir build_x64 && pushd build_x64 > # cmake ..\src -G "Visual Studio 15 2017 Win64" > -DCMAKE_SYSTEM_VERSION=10.0.16299.0 -DENABLE_LIBEVENT=1 > -DHAS_AUTHENTICATION=0 -DPATCHEXE_PATH="C:\gnuwin32\bin" -T host=x64 > # msbuild Mesos.sln /p:Configuration=Debug /p:Platform=x64 /maxcpucount:4 > /t:Rebuild -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-9069) UI page /#/agents/foo lies about agent IP
Marcin Owsiany created MESOS-9069: - Summary: UI page /#/agents/foo lies about agent IP Key: MESOS-9069 URL: https://issues.apache.org/jira/browse/MESOS-9069 Project: Mesos Issue Type: Bug Components: webui Environment: Observed when the master was being slow to respond. Reporter: Marcin Owsiany Attachments: after.png, before.png After navigating from /#/agents/foo to /#/agents/bar, the ID of the agent at the top of the page was already updated (showed agent "bar"s ID), while the value of the "Agent" field was stale (still showed agent "foo"s IP address). See the attachments which show screenshots of the page before and after the correct IP was loaded. Note how the ID at the top stays constant while the IP changed. Unless one is looking closely, such lie can seriously confuse the operator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-7211) Document SUPPRESS HTTP call
[ https://issues.apache.org/jira/browse/MESOS-7211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540620#comment-16540620 ] ASF GitHub Bot commented on MESOS-7211: --- GitHub user thzois reopened a pull request: https://github.com/apache/mesos/pull/301 Document SUPPRESS HTTP call [MESOS-7211] You can merge this pull request into a Git repository by running: $ git pull https://github.com/thzois/mesos master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/mesos/pull/301.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #301 commit 2c9bc7270956c17b055f23b24a24a006748b7725 Author: zois Date: 2018-07-10T13:24:37Z Document SUPPRESS HTTP call [MESOS-7211] commit f9725b0c51db2dc49d6437798b798bb1534659e6 Author: Thodoris Zois Date: 2018-07-10T23:39:55Z Merge branch 'master' of https://github.com/apache/mesos commit 23c8ec9669549917dc2e4722034446e5b33a9b79 Author: Thodoris Zois Date: 2018-07-11T00:02:12Z Changes to documentation of SUPPRESS HTTP call [MESOS-7211] > Document SUPPRESS HTTP call > --- > > Key: MESOS-7211 > URL: https://issues.apache.org/jira/browse/MESOS-7211 > Project: Mesos > Issue Type: Documentation > Components: documentation >Affects Versions: 1.1.0 >Reporter: Bruce Merry >Priority: Minor > Labels: mesosphere, newbie > > The documentation at > http://mesos.apache.org/documentation/latest/scheduler-http-api/ doesn't list > the SUPPRESS call as one of the call types, but it does seem to be > implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MESOS-8238) Add recommended accept/decline/suppress/revive behavior to scheduler docs
[ https://issues.apache.org/jira/browse/MESOS-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Mahler reassigned MESOS-8238: -- Assignee: Benjamin Mahler (was: Greg Mann) > Add recommended accept/decline/suppress/revive behavior to scheduler docs > - > > Key: MESOS-8238 > URL: https://issues.apache.org/jira/browse/MESOS-8238 > Project: Mesos > Issue Type: Task >Reporter: Greg Mann >Assignee: Benjamin Mahler >Priority: Major > Labels: mesosphere > > We should update the scheduler documentation to provide recommendations to > framework authors regarding how to use the ACCEPT / DECLINE / SUPPRESS / > REVIVE calls, in order to maximize resource availability. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-7966) check for maintenance on agent causes fatal error
[
https://issues.apache.org/jira/browse/MESOS-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540703#comment-16540703
]
Greg Mann commented on MESOS-7966:
--
[~kaysoky] ping :)
There is some interest in the community in having this backported, would be
great to make that happen. Let me know if you don't have cycles and would like
some assistance!
> check for maintenance on agent causes fatal error
> -
>
> Key: MESOS-7966
> URL: https://issues.apache.org/jira/browse/MESOS-7966
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.1.3, 1.2.3, 1.3.2, 1.4.1, 1.5.0, 1.6.0
>Reporter: Rob Johnson
>Assignee: Benno Evers
>Priority: Critical
> Labels: mesosphere, reliability
> Fix For: 1.7.0
>
>
> We interact with the maintenance API frequently to orchestrate gracefully
> draining agents of tasks without impacting service availability.
> Occasionally we seem to trigger a fatal error in Mesos when interacting with
> the api. This happens relatively frequently, and impacts us when downstream
> frameworks (marathon) react badly to leader elections.
> Here is the log line that we see when the master dies:
> {code}
> F0911 12:18:49.543401 123748 hierarchical.cpp:872] Check failed:
> slaves[slaveId].maintenance.isSome()
> {code}
> It's quite possibly we're using the maintenance API in the wrong way. We're
> happy to provide any other logs you need - please let me know what would be
> useful for debugging.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-7966) check for maintenance on agent causes fatal error
[
https://issues.apache.org/jira/browse/MESOS-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540704#comment-16540704
]
Benjamin Mahler commented on MESOS-7966:
[~bennoe] [~kaysoky] ping, can this be backported?
> check for maintenance on agent causes fatal error
> -
>
> Key: MESOS-7966
> URL: https://issues.apache.org/jira/browse/MESOS-7966
> Project: Mesos
> Issue Type: Bug
> Components: master
>Affects Versions: 1.1.3, 1.2.3, 1.3.2, 1.4.1, 1.5.0, 1.6.0
>Reporter: Rob Johnson
>Assignee: Benno Evers
>Priority: Critical
> Labels: mesosphere, reliability
> Fix For: 1.7.0
>
>
> We interact with the maintenance API frequently to orchestrate gracefully
> draining agents of tasks without impacting service availability.
> Occasionally we seem to trigger a fatal error in Mesos when interacting with
> the api. This happens relatively frequently, and impacts us when downstream
> frameworks (marathon) react badly to leader elections.
> Here is the log line that we see when the master dies:
> {code}
> F0911 12:18:49.543401 123748 hierarchical.cpp:872] Check failed:
> slaves[slaveId].maintenance.isSome()
> {code}
> It's quite possibly we're using the maintenance API in the wrong way. We're
> happy to provide any other logs you need - please let me know what would be
> useful for debugging.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-2728) Introduce concept of cluster wide resources.
[ https://issues.apache.org/jira/browse/MESOS-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540697#comment-16540697 ] Chun-Hung Hsiao commented on MESOS-2728: Recently we're working on supporting cluster-wide storage resources and here is the ticket for the design doc: https://issues.apache.org/jira/browse/MESOS-8528 > Introduce concept of cluster wide resources. > > > Key: MESOS-2728 > URL: https://issues.apache.org/jira/browse/MESOS-2728 > Project: Mesos > Issue Type: Epic >Reporter: Jörg Schad >Assignee: Jörg Schad >Priority: Major > Labels: external-volumes, mesosphere > > There are resources which are not provided by a single node. Consider for > example a external Network Bandwidth of a cluster. Being a limited resource > it makes sense for Mesos to manage it but still it is not a resource being > offered by a single node. A cluster-wide resource is still consumed by a > task, and when that task completes, the resources are then available to be > allocated to another framework/task. > Use Cases: > 1. Network Bandwidth > 2. IP Addresses > 3. Global Service Ports > 4. Distributed File System Storage > 5. Software Licences > 6. SAN Volumes -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-9071) Improve containerizer logging for isolator prepare/update/cleanup.
Chun-Hung Hsiao created MESOS-9071: -- Summary: Improve containerizer logging for isolator prepare/update/cleanup. Key: MESOS-9071 URL: https://issues.apache.org/jira/browse/MESOS-9071 Project: Mesos Issue Type: Task Components: containerization Reporter: Chun-Hung Hsiao It has been very hard to debug container startup/teardown issues if the containers are stuck in some isolator prepare/update/cleanup steps. We should add more logging in the Mesos containerizer for each isolator calls. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-9070) Support systemd and freezer cgroup subsystems bind mount for container with rootfs.
Gilbert Song created MESOS-9070:
---
Summary: Support systemd and freezer cgroup subsystems bind mount
for container with rootfs.
Key: MESOS-9070
URL: https://issues.apache.org/jira/browse/MESOS-9070
Project: Mesos
Issue Type: Task
Components: containerization
Reporter: Gilbert Song
>From MESOS-8327, cgroup subsystems are bind mounted to the container's rootfs,
>but systemd and freezer cgroup are not bind mounted yet since they are not
>subsystems under the cgroup isolator but from the linux launcher.
Some applications (e.g., dockerd) may check the /proc/self/cgorup for enabled
subsystems and check them at /proc/self/mountinfo to make sure there are those
mounts. Here is an example:
{noformat}
➜ aws dcos task exec --interactive test.bf2fad80-846b-11e8-b5a0-eaa1bec34306
/bin/bash
cat /proc/self/cgroup
11:blkio:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
10:perf_event:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
9:cpuset:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
8:memory:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
7:pids:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
6:devices:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
5:cpu,cpuacct:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
4:freezer:/mesos/87899f08-53e5-47bf-aba3-712c31c33543/mesos/12fde554-5262-473c-a20c-7dd201148b11
3:net_cls,net_prio:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
2:hugetlb:/mesos/87899f08-53e5-47bf-aba3-712c31c33543
1:name=systemd:/mesos/87899f08-53e5-47bf-aba3-712c31c33543/mesos/12fde554-5262-473c-a20c-7dd201148b11
cat /proc/self/mountinfo
388 387 202:9 / / rw,relatime master:1 - ext4 /dev/xvda9
rw,seclabel,data=ordered
389 388 254:0 / /usr ro,relatime master:2 - ext4 /dev/mapper/usr
ro,seclabel,block_validity,delalloc,barrier,user_xattr,acl
390 389 202:6 / /usr/share/oem rw,nodev,relatime master:32 - ext4 /dev/xvda6
rw,seclabel,commit=600,data=ordered
391 388 0:6 / /dev rw,nosuid master:3 - devtmpfs devtmpfs
rw,seclabel,size=8201844k,nr_inodes=2050461,mode=755
392 391 0:19 / /dev/shm rw,nosuid,nodev master:4 - tmpfs tmpfs rw,seclabel
393 391 0:20 / /dev/pts rw,nosuid,noexec,relatime master:5 - devpts devpts
rw,seclabel,gid=5,mode=620,ptmxmode=000
394 391 0:15 / /dev/mqueue rw,relatime master:26 - mqueue mqueue rw,seclabel
395 391 0:37 / /dev/hugepages rw,relatime master:27 - hugetlbfs hugetlbfs
rw,seclabel
396 388 0:4 / /proc rw,nosuid,nodev,noexec,relatime master:6 - proc proc rw
397 396 0:35 / /proc/sys/fs/binfmt_misc rw,relatime master:24 - autofs
systemd-1 rw,fd=23,pgrp=0,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1017
398 396 0:40 / /proc/xen rw,relatime master:31 - xenfs xenfs rw
399 388 0:18 / /sys rw,nosuid,nodev,noexec,relatime master:7 - sysfs sysfs
rw,seclabel
400 399 0:17 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime master:8 -
securityfs securityfs rw
401 399 0:22 / /sys/fs/cgroup ro,nosuid,nodev,noexec master:9 - tmpfs tmpfs
ro,seclabel,mode=755
402 401 0:23 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime master:10
- cgroup cgroup
rw,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
403 401 0:25 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime master:11
- cgroup cgroup rw,hugetlb
404 401 0:26 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime
master:12 - cgroup cgroup rw,net_cls,net_prio
405 401 0:27 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime master:13
- cgroup cgroup rw,freezer
406 401 0:28 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime
master:14 - cgroup cgroup rw,cpu,cpuacct
407 401 0:29 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime master:15
- cgroup cgroup rw,devices
408 401 0:30 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime master:16 -
cgroup cgroup rw,pids
409 401 0:31 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime master:17
- cgroup cgroup rw,memory
410 401 0:32 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime master:18
- cgroup cgroup rw,cpuset
411 401 0:33 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime
master:19 - cgroup cgroup rw,perf_event
412 401 0:34 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime master:20 -
cgroup cgroup rw,blkio
413 399 0:24 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime master:21 -
pstore pstore rw,seclabel
414 399 0:16 / /sys/fs/selinux rw,relatime master:22 - selinuxfs selinuxfs rw
415 399 0:7 / /sys/kernel/debug rw,relatime master:29 - debugfs debugfs
rw,seclabel
416 388 0:21 / /run rw,nosuid,nodev master:23 - tmpfs tmpfs rw,seclabel,mode=755
417 388 0:36 / /boot rw,relatime master:25 - autofs systemd-1
rw,fd=33,pgrp=0,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=10774
418 417 202:1 / /boot rw,relatime master:33 - vfat /dev/xvda1
rw,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro
419 388 0:38 / /media
