[jira] [Commented] (MESOS-8882) Add Python 3 as a dependency and use it for our Python codebase
[
https://issues.apache.org/jira/browse/MESOS-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464396#comment-16464396
]
Armand Grillet commented on MESOS-8882:
---
[~andschwa] Thank you for your feedback and first patch! I have edited the
ticket following your first comment.
Yes, pyenv is still the standard way to go for versioning.
Will take a look at #66964 during the next days, I have read the description
and testing done and I am now looking at how to handle {{pylint}} with the
situation we will have.
> Add Python 3 as a dependency and use it for our Python codebase
> ---
>
> Key: MESOS-8882
> URL: https://issues.apache.org/jira/browse/MESOS-8882
> Project: Mesos
> Issue Type: Epic
> Components: cli, python api
>Reporter: Armand Grillet
>Priority: Major
>
> As mentioned by [~andschwa] [2 weeks
> ago|https://mesos.slack.com/archives/C1LPTK50T/p1524165786000521], nothing
> new should use Python 2. [~cinchurge] and [~ArmandGrillet] are currently
> working on the CLI while being shepherded by [~klueska] and, as we plan to do
> more work on it soon, mentioned again today that it would be a good time to
> start using Python 3 in our codebase.
> We use Python 2 in four areas of the Mesos codebase:
> * In the support scripts.
> * In the new CLI.
> * In the new Mesos Python package.
> * In the Python bindings.
> The support scripts and CLI can be updated independently of the switch if we
> assume that most of our devs have Python 2 and 3 installed on their machines.
> The Python can be updated last as they should not be used anymore, in favor
> of our V1 API.
> This new dependency will require changes in our documentation regarding how
> to build Python. As it will be a change for Mesos devs, we have to express
> the reasons behind this change and check if it could cause any issue on their
> side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8628) Perform copy elimination across master message handlers.
[
https://issues.apache.org/jira/browse/MESOS-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464390#comment-16464390
]
Benjamin Mahler commented on MESOS-8628:
Framework (re-)registration:
{noformat}
commit cbe2edd80e554309c32f7e7d29dbe08fc251da93
Author: Meng Zhu
Date: Fri May 4 14:00:30 2018 -0700
Avoid copying of register framework messages in the master.
Prior to this patch, the register framework message was copied
into a subscribe call. This updates the handler to perform moves
instead.
Review: https://reviews.apache.org/r/66860/
{noformat}
{noformat}
commit cc68152b3d3e83a674c231dc14298b482a189e49
Author: Meng Zhu
Date: Fri May 4 14:00:46 2018 -0700
Avoid copying of re-register framework messages in the master.
Prior to this patch, the re-register framework message was copied
into a subscribe call. This updates the handler to perform moves
instead.
Review: https://reviews.apache.org/r/66900/
{noformat}
> Perform copy elimination across master message handlers.
>
>
> Key: MESOS-8628
> URL: https://issues.apache.org/jira/browse/MESOS-8628
> Project: Mesos
> Issue Type: Improvement
> Components: master
>Reporter: Benjamin Mahler
>Priority: Major
> Labels: performance
>
> Now that we have protobuf move support and move support for libprocess
> {{install}}, we can do a sweep of the master's message handlers and eliminate
> copying.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8882) Add Python 3 as a dependency and use it for our Python codebase
[
https://issues.apache.org/jira/browse/MESOS-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464377#comment-16464377
]
Andrew Schwartzmeyer commented on MESOS-8882:
-
All right, I tried out {{futurize}} and it seems promising to at least make out
support scripts single-source Python 2 and 3 compatible. If it works out well
enough, then we can at least fix MESOS-7658 (which I consider a very rude
issue) without forcing the adoption of Python 3 as a dependency immediately.
WIP patch is [review #66964|https://reviews.apache.org/r/66964/].
> Add Python 3 as a dependency and use it for our Python codebase
> ---
>
> Key: MESOS-8882
> URL: https://issues.apache.org/jira/browse/MESOS-8882
> Project: Mesos
> Issue Type: Epic
> Components: cli, python api
>Reporter: Armand Grillet
>Priority: Major
>
> As mentioned by [~andschwa] [2 weeks
> ago|https://mesos.slack.com/archives/C1LPTK50T/p1524165786000521], nothing
> new should use Python 2. [~cinchurge] and [~ArmandGrillet] are currently
> working on the CLI while being shepherded by [~klueska] and, as we plan to do
> more work on it soon, mentioned again today that it would be a good time to
> start using Python 3 in our codebase.
> We use Python 2 in four areas of the Mesos codebase:
> * In the support scripts.
> * In the new CLI.
> * In the new Mesos Python package.
> * In the Python bindings.
> All these areas need to be updated if we switch to Python 3, except maybe the
> Python bindings that should not be used anymore, in favor of our V1 API.
> This new dependency would also require some changes in our documentation
> regarding how to build Python. As it will be a change for Mesos devs, we have
> to express the reasons behind this change and check if this change could
> cause any issue on their side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-7658) apply-reviews.py fails with Unicode characters
[
https://issues.apache.org/jira/browse/MESOS-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Schwartzmeyer reassigned MESOS-7658:
---
Assignee: Andrew Schwartzmeyer
> apply-reviews.py fails with Unicode characters
> --
>
> Key: MESOS-7658
> URL: https://issues.apache.org/jira/browse/MESOS-7658
> Project: Mesos
> Issue Type: Bug
>Reporter: Andrew Schwartzmeyer
>Assignee: Andrew Schwartzmeyer
>Priority: Major
> Labels: reviewboard
>
> This prevents commits from being applied if the name or message contains
> non-ASCII characters, and so can break the Windows ReviewBot.
> {code}
> > git checkout b2801f0012535e29609f603b4324a9ca693f70cb~
> > python .\support\apply-reviews.py -r 59874
> Traceback (most recent call last):
> File ".\support\apply-reviews.py", line 381, in
> reviewboard()
> File ".\support\apply-reviews.py", line 360, in reviewboard
> apply_review()
> File ".\support\apply-reviews.py", line 126, in apply_review
> commit_patch()
> File ".\support\apply-reviews.py", line 225, in commit_patch
> shell(cmd, options['dry_run'])
> File ".\support\apply-reviews.py", line 111, in shell
> error_code = subprocess.call(command, stderr=subprocess.STDOUT,
> shell=True)
> File "C:\Python27\lib\subprocess.py", line 168, in call
> return Popen(*popenargs, **kwargs).wait()
> File "C:\Python27\lib\subprocess.py", line 390, in __init__
> errread, errwrite)
> File "C:\Python27\lib\subprocess.py", line 610, in _execute_child
> args = '{} /c "{}"'.format (comspec, args)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position
> 25: ordinal not in range(128)
> ~\src\mesos-copy2 |-/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-8649) CSI Spec v0.2 Support
[ https://issues.apache.org/jira/browse/MESOS-8649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun-Hung Hsiao reassigned MESOS-8649: -- Assignee: Chun-Hung Hsiao > CSI Spec v0.2 Support > - > > Key: MESOS-8649 > URL: https://issues.apache.org/jira/browse/MESOS-8649 > Project: Mesos > Issue Type: Epic > Components: storage >Reporter: Chun-Hung Hsiao >Assignee: Chun-Hung Hsiao >Priority: Critical > Labels: mesosphere, storage > Fix For: 1.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-8882) Add Python 3 as a dependency and use it for our Python codebase
[ https://issues.apache.org/jira/browse/MESOS-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464111#comment-16464111 ] Andrew Schwartzmeyer commented on MESOS-8882: - Is [pyenv|https://github.com/pyenv/pyenv] still the preferred Python version manager? It's been a while since I wrote a lot of Python... > Add Python 3 as a dependency and use it for our Python codebase > --- > > Key: MESOS-8882 > URL: https://issues.apache.org/jira/browse/MESOS-8882 > Project: Mesos > Issue Type: Epic > Components: cli, python api >Reporter: Armand Grillet >Priority: Major > > As mentioned by [~andschwa] [2 weeks > ago|https://mesos.slack.com/archives/C1LPTK50T/p1524165786000521], nothing > new should use Python 2. [~cinchurge] and [~ArmandGrillet] are currently > working on the CLI while being shepherded by [~klueska] and, as we plan to do > more work on it soon, mentioned again today that it would be a good time to > start using Python 3 in our codebase. > We use Python 2 in four areas of the Mesos codebase: > * In the support scripts. > * In the new CLI. > * In the new Mesos Python package. > * In the Python bindings. > All these areas need to be updated if we switch to Python 3, except maybe the > Python bindings that should not be used anymore, in favor of our V1 API. > This new dependency would also require some changes in our documentation > regarding how to build Python. As it will be a change for Mesos devs, we have > to express the reasons behind this change and check if this change could > cause any issue on their side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-8882) Add Python 3 as a dependency and use it for our Python codebase
[ https://issues.apache.org/jira/browse/MESOS-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464108#comment-16464108 ] Andrew Schwartzmeyer commented on MESOS-8882: - The reason I want to make clear that the support scripts can be moved to Python 3 separately is that, per MESOS-7658, using Python 2 creates [multiple|https://github.com/apache/mesos/pull/284#issue-182300541] [problems|https://github.com/apache/mesos/pull/284#issuecomment-382868176] where people have had to _change their name_ in order to contribute to Mesos. > Add Python 3 as a dependency and use it for our Python codebase > --- > > Key: MESOS-8882 > URL: https://issues.apache.org/jira/browse/MESOS-8882 > Project: Mesos > Issue Type: Epic > Components: cli, python api >Reporter: Armand Grillet >Priority: Major > > As mentioned by [~andschwa] [2 weeks > ago|https://mesos.slack.com/archives/C1LPTK50T/p1524165786000521], nothing > new should use Python 2. [~cinchurge] and [~ArmandGrillet] are currently > working on the CLI while being shepherded by [~klueska] and, as we plan to do > more work on it soon, mentioned again today that it would be a good time to > start using Python 3 in our codebase. > We use Python 2 in four areas of the Mesos codebase: > * In the support scripts. > * In the new CLI. > * In the new Mesos Python package. > * In the Python bindings. > All these areas need to be updated if we switch to Python 3, except maybe the > Python bindings that should not be used anymore, in favor of our V1 API. > This new dependency would also require some changes in our documentation > regarding how to build Python. As it will be a change for Mesos devs, we have > to express the reasons behind this change and check if this change could > cause any issue on their side. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MESOS-8882) Add Python 3 as a dependency and use it for our Python codebase
[
https://issues.apache.org/jira/browse/MESOS-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464102#comment-16464102
]
Andrew Schwartzmeyer commented on MESOS-8882:
-
I'm not sure I agree with this part:
{quote}All these areas need to be updated if we switch to Python 3, except
maybe the Python bindings that should not be used anymore, in favor of our V1
API.
{quote}
I think both the support scripts and CLI can be updated independently. It's
realistic to assume most people already have {{python2/python3}} on their
machine (or can get it) due to the longstanding split in the Python community.
Moreover, taking a Python 3 dependency for the support scripts does not imply
that _Mesos_ has a Python 3 dependency, as the support scripts are not a
required part of the build. I agree the CLI should also be in Python 3, which
would change the dependency for that component, but just want to point out that
this work can be done separately, and that Mesos can still be built without
Python at all.
> Add Python 3 as a dependency and use it for our Python codebase
> ---
>
> Key: MESOS-8882
> URL: https://issues.apache.org/jira/browse/MESOS-8882
> Project: Mesos
> Issue Type: Epic
> Components: cli, python api
>Reporter: Armand Grillet
>Priority: Major
>
> As mentioned by [~andschwa] [2 weeks
> ago|https://mesos.slack.com/archives/C1LPTK50T/p1524165786000521], nothing
> new should use Python 2. [~cinchurge] and [~ArmandGrillet] are currently
> working on the CLI while being shepherded by [~klueska] and, as we plan to do
> more work on it soon, mentioned again today that it would be a good time to
> start using Python 3 in our codebase.
> We use Python 2 in four areas of the Mesos codebase:
> * In the support scripts.
> * In the new CLI.
> * In the new Mesos Python package.
> * In the Python bindings.
> All these areas need to be updated if we switch to Python 3, except maybe the
> Python bindings that should not be used anymore, in favor of our V1 API.
> This new dependency would also require some changes in our documentation
> regarding how to build Python. As it will be a change for Mesos devs, we have
> to express the reasons behind this change and check if this change could
> cause any issue on their side.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (MESOS-8340) Add a no-enforce isolation option.
[
https://issues.apache.org/jira/browse/MESOS-8340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xudong Ni reassigned MESOS-8340:
Assignee: Xudong Ni
> Add a no-enforce isolation option.
> --
>
> Key: MESOS-8340
> URL: https://issues.apache.org/jira/browse/MESOS-8340
> Project: Mesos
> Issue Type: Improvement
> Components: containerization
>Reporter: James Peach
>Assignee: Xudong Ni
>Priority: Major
>
> Some resource isolators ({{disk/du}}, {{disk/xfs}} and {{network/ports}})
> have the ability to run in a no-enforce mode where they report resource usage
> but do not enforce the allocated resource limit. Rather than a separate flag
> for each possibility, we could add an agent flag named
> {{\-\-noenforce-isolation}} that causes the agent to log any limitation
> raised by the given list of isolators, but would not cause the container to
> be killed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8884) Flaky `DockerContainerizerTest.ROOT_DOCKER_MaxCompletionTime`.
Andrei Budnik created MESOS-8884:
Summary: Flaky
`DockerContainerizerTest.ROOT_DOCKER_MaxCompletionTime`.
Key: MESOS-8884
URL: https://issues.apache.org/jira/browse/MESOS-8884
Project: Mesos
Issue Type: Bug
Environment: master-520b7298
Reporter: Andrei Budnik
Attachments: ROOT_DOCKER_MaxCompletionTime-badrun.txt
This test fails quite often in our internal CI.
{code:java}
../../src/tests/containerizer/docker_containerizer_tests.cpp:663: Failure
termination.get() is NONE
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8883) Send email to dev list concerning the Python 3 update
Armand Grillet created MESOS-8883:
-
Summary: Send email to dev list concerning the Python 3 update
Key: MESOS-8883
URL: https://issues.apache.org/jira/browse/MESOS-8883
Project: Mesos
Issue Type: Task
Affects Versions: 1.6.0
Reporter: Armand Grillet
Assignee: Andrew Schwartzmeyer
Let's prepare an email for the dev community to express our wish and reasons to
add a {{python3}} dependency to the Mesos codebase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-8876) Normal exit of Docker container using rexray volume results in TASK_FAILED
[
https://issues.apache.org/jira/browse/MESOS-8876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463620#comment-16463620
]
Qian Zhang commented on MESOS-8876:
---
https://reviews.apache.org/r/66947/
> Normal exit of Docker container using rexray volume results in TASK_FAILED
> --
>
> Key: MESOS-8876
> URL: https://issues.apache.org/jira/browse/MESOS-8876
> Project: Mesos
> Issue Type: Bug
> Components: docker
>Reporter: Qian Zhang
>Assignee: Qian Zhang
>Priority: Major
> Attachments: task_exit_volume.json
>
>
> In the fix to MESOS-8488, we reap the Docker container process directly in
> Docker executor, and it will wait for `docker run` to return for at most 3
> seconds. However, in some cases, the `docker run` command will indeed need
> more than 3 seconds to return, e.g., the Docker container uses an external
> rexray volume (see the attached task json as an example), for such container,
> there will be about 5 seconds between container process exits and the `docker
> run` returns (I suspect Docker daemon was doing some stuff related to rexray
> volume during this time), so we will reap this container, and send a
> {{TASK_FAILED}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (MESOS-8882) Python 2 is still used for our CLI, support scripts, and bindings. With Python 3 becoming a standard, development starts being hard and we should move to Python 3.
Armand Grillet created MESOS-8882: - Summary: Python 2 is still used for our CLI, support scripts, and bindings. With Python 3 becoming a standard, development starts being hard and we should move to Python 3. Key: MESOS-8882 URL: https://issues.apache.org/jira/browse/MESOS-8882 Project: Mesos Issue Type: Epic Components: cli, python api Reporter: Armand Grillet -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MESOS-8881) libprocess avoid using libevent epoll backend
Teng Su created MESOS-8881: -- Summary: libprocess avoid using libevent epoll backend Key: MESOS-8881 URL: https://issues.apache.org/jira/browse/MESOS-8881 Project: Mesos Issue Type: Improvement Components: libprocess Affects Versions: 1.5.0 Reporter: Teng Su In libprocess libevent.cpp, is avoid to using epoll. These is the code: /home/suteng/code/mesos/3rdparty/libprocess/src/libevent.cpp 206 // TODO(jmlvanre): Allow support for 'epoll' once SSL related 207 // issues are resolved. 208 struct event_config* config = event_config_new(); 209 event_config_avoid_method(config, "epoll"); -- This message was sent by Atlassian JIRA (v7.6.3#76005)
