Package: release.debian.org
Severity: normal
Tags: bullseye
User: release.debian....@packages.debian.org
Usertags: pu
X-Debbugs-Cc: car...@debian.org
[ Reason ]
I attempted to backport the fix for CVE-2021-42343 to
2021.01.01+ds.1-2.1 while trying to test that it was fixed I
discovered there was an import error on Python 3.9 that prevented the
local cluster from launching.
The upstream discussion about the import problem is available here:
https://github.com/dask/distributed/issues/4168
[ Impact ]
Without the update the the local Client object can't run with
python3.9 and if someone patches it to run then they might be subject
to CVE-2021-42343.
[ Tests ]
pass-host-to-local-cluster.path adds the test
test_cluster_host_used_throughout_cluster(host, use_nanny):
to make sure the host argument is passed.
There are 3 tests on ci that fail due to the python3.9-compatibility issue
https://ci.debian.net/data/autopkgtest/stable/amd64/d/dask.distributed/17215513/log.gz
test_nprocs_negative, test_nprocs_negative, test_nprocs_auto
which pass in my autopkgtest runs with the patch applied.
[ Risks ]
The changes are fairly small.
pass-host-to-local-cluster.path is making sure the default is passed
though to the client instatiation.
python3.9-compatibility adds an import to make sure some internal part
of python3.9 is initialized before it's used, avoiding a cyclic import.
[ Checklist ]
[X] *all* changes are documented in the d/changelog
[X] I reviewed all changes and I approve them
[X] attach debdiff against the package in (old)stable
[\] the issue is verified as fixed in unstable.
The security vulnerability patch was applied to unstable and is in
testing.
The python3.9-compatibility patch isn't necessary for unstable because
unstable and testing are now on Python 3.10
[ Changes ]
* Apply pass-host-to-local-cluster.patch. Resolves CVE-2021-42343
* Add python3.9-compatibility.patch. Fixes cannot import name 'Popen'
from partially initialized module 'multiprocessing.popen_spawn_posix'
[ Other info ]
Unfortunately there's still 2 CI test failures in my autopkgtests on bullseye
that I don't know how to fix though.
The changes are also committed to the debian/bullseye branch at salsa.
I haven't done a point release before so I'm not sure what else I need to do,
and if mentioning the CVE id was enough to include the security team.
Hope that's enough.
Diane Trout
diff -Nru dask.distributed-2021.01.0+ds.1/debian/changelog
dask.distributed-2021.01.0+ds.1/debian/changelog
--- dask.distributed-2021.01.0+ds.1/debian/changelog 2021-07-13
09:19:56.000000000 -0700
+++ dask.distributed-2021.01.0+ds.1/debian/changelog 2021-11-27
11:29:20.000000000 -0800
@@ -1,3 +1,11 @@
+dask.distributed (2021.01.0+ds.1-2.1+deb11u1) bullseye; urgency=medium
+
+ * Apply pass-host-to-local-cluster.patch. Resolves CVE-2021-42343
+ * Add python3.9-compatibility.patch. Fixes cannot import name 'Popen'
+ from partially initialized module 'multiprocessing.popen_spawn_posix'
+
+ -- Diane Trout <di...@ghic.org> Sat, 27 Nov 2021 11:29:20 -0800
+
dask.distributed (2021.01.0+ds.1-2.1) unstable; urgency=medium
* Non-maintainer upload.
diff -Nru
dask.distributed-2021.01.0+ds.1/debian/patches/pass-host-to-local-cluster.patch
dask.distributed-2021.01.0+ds.1/debian/patches/pass-host-to-local-cluster.patch
---
dask.distributed-2021.01.0+ds.1/debian/patches/pass-host-to-local-cluster.patch
1969-12-31 16:00:00.000000000 -0800
+++
dask.distributed-2021.01.0+ds.1/debian/patches/pass-host-to-local-cluster.patch
2021-11-27 11:27:50.000000000 -0800
@@ -0,0 +1,55 @@
+From 295bf8f08fdd99f1767b616e6247253b89f47022 Mon Sep 17 00:00:00 2001
+From: Jim Crist-Harif <jcristha...@gmail.com>
+Date: Mon, 4 Oct 2021 10:02:55 -0500
+Subject: [PATCH] Pass `host` through LocalCluster to workers
+
+Previously the `host` parameter to `LocalCluster` would only be
+forwarded to `Scheduler` instances and not `Worker`/`Nanny` instances,
+leading to workers listening on non-localhost in some configurations.
+This fixes that and adds a test.
+---
+ distributed/deploy/local.py | 1 +
+ distributed/deploy/tests/test_local.py | 18 ++++++++++++++++++
+ 2 files changed, 19 insertions(+)
+
+--- a/distributed/deploy/local.py
++++ b/distributed/deploy/local.py
+@@ -189,6 +189,7 @@
+
+ worker_kwargs.update(
+ {
++ "host": host,
+ "nthreads": threads_per_worker,
+ "services": worker_services,
+ "dashboard_address": worker_dashboard_address,
+--- a/distributed/deploy/tests/test_local.py
++++ b/distributed/deploy/tests/test_local.py
+@@ -4,6 +4,7 @@
+ import subprocess
+ import sys
+ from time import sleep
++from urllib.parse import urlparse
+ from threading import Lock
+ import unittest
+ import weakref
+@@ -1045,3 +1046,20 @@
+ n_workers=0, silence_logs=False, dashboard_address=None,
asynchronous=True
+ ) as c:
+ pass
++
++
++@pytest.mark.asyncio
++@pytest.mark.parametrize("host", [None, "127.0.0.1"])
++@pytest.mark.parametrize("use_nanny", [True, False])
++async def test_cluster_host_used_throughout_cluster(host, use_nanny):
++ """Ensure that the `host` kwarg is propagated through scheduler, nanny,
and workers"""
++ async with LocalCluster(host=host, asynchronous=True) as cluster:
++ url = urlparse(cluster.scheduler_address)
++ assert url.hostname == "127.0.0.1"
++ for worker in cluster.workers.values():
++ url = urlparse(worker.address)
++ assert url.hostname == "127.0.0.1"
++
++ if use_nanny:
++ url = urlparse(worker.process.worker_address)
++ assert url.hostname == "127.0.0.1"
diff -Nru
dask.distributed-2021.01.0+ds.1/debian/patches/python3.9-compatibility.patch
dask.distributed-2021.01.0+ds.1/debian/patches/python3.9-compatibility.patch
---
dask.distributed-2021.01.0+ds.1/debian/patches/python3.9-compatibility.patch
1969-12-31 16:00:00.000000000 -0800
+++
dask.distributed-2021.01.0+ds.1/debian/patches/python3.9-compatibility.patch
2021-11-27 11:29:20.000000000 -0800
@@ -0,0 +1,108 @@
+From 2c482276ed39112c650ed886c66d2c7b7d5e3783 Mon Sep 17 00:00:00 2001
+From: Jim Crist-Harif <jcristha...@gmail.com>
+Date: Tue, 10 Nov 2020 16:29:57 -0600
+Subject: [PATCH 1/4] Python 3.9 compatibility
+Bug: https://github.com/dask/distributed/issues/4168
+
+---
+ distributed/utils.py | 4 ++++
+ 1 file changed, 4 insertions(+)
+
+diff --git a/distributed/utils.py b/distributed/utils.py
+index 77487f8cec..c1a0d2caa8 100644
+--- a/distributed/utils.py
++++ b/distributed/utils.py
+@@ -72,6 +72,10 @@
+
+
+ def _initialize_mp_context():
++ if not WINDOWS:
++ # For some reason this is required in python >= 3.9
++ import multiprocessing.popen_spawn_posix
++
+ if WINDOWS or PYPY:
+ return multiprocessing
+ else:
+
+From ef5bd0f8729a2c8c6f63c38dcac29020af01a7c4 Mon Sep 17 00:00:00 2001
+From: Matthew Rocklin <mrock...@gmail.com>
+Date: Mon, 25 Jan 2021 10:51:36 -0800
+Subject: [PATCH 2/4] import multiprocessing
+
+---
+ distributed/utils.py | 4 +++-
+ 1 file changed, 3 insertions(+), 1 deletion(-)
+
+diff --git a/distributed/utils.py b/distributed/utils.py
+index a6af0b8f45..949b39cf52 100644
+--- a/distributed/utils.py
++++ b/distributed/utils.py
+@@ -10,7 +10,7 @@
+ import html
+ import json
+ import logging
+-import multiprocessing
++import multiprocessing # noqa: F401
+ import os
+ import re
+ import shutil
+@@ -71,6 +71,8 @@
+
+
+ def _initialize_mp_context():
++ import multiprocessing # noqa: F401
++
+ if not WINDOWS:
+ # For some reason this is required in python >= 3.9
+ import multiprocessing.popen_spawn_posix
+
+From 60eecf8cf82026e4e48f570e255693bd01c3019b Mon Sep 17 00:00:00 2001
+From: Matthew Rocklin <mrock...@gmail.com>
+Date: Mon, 25 Jan 2021 13:39:18 -0800
+Subject: [PATCH 3/4] Add Python 3.9 to CI
+
+---
+ .github/workflows/ci-windows.yaml | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/.github/workflows/ci-windows.yaml
b/.github/workflows/ci-windows.yaml
+index 6d5923eb18..a807bf187b 100644
+--- a/.github/workflows/ci-windows.yaml
++++ b/.github/workflows/ci-windows.yaml
+@@ -8,7 +8,7 @@ jobs:
+ strategy:
+ fail-fast: false
+ matrix:
+- python-version: ["3.6", "3.7", "3.8"]
++ python-version: ["3.6", "3.7", "3.8", "3.9"]
+
+ steps:
+ - name: Checkout source
+
+From 236f8af2e6ff2dc409f83dd0270620443943624f Mon Sep 17 00:00:00 2001
+From: James Bourbeau <jrbourb...@gmail.com>
+Date: Tue, 26 Jan 2021 10:58:15 -0600
+Subject: [PATCH 4/4] Update tornado install
+
+---
+ .github/workflows/ci-windows.yaml | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/.github/workflows/ci-windows.yaml
b/.github/workflows/ci-windows.yaml
+index a807bf187b..3fff13179f 100644
+--- a/.github/workflows/ci-windows.yaml
++++ b/.github/workflows/ci-windows.yaml
+@@ -33,10 +33,10 @@ jobs:
+ - name: Install tornado
+ shell: bash -l {0}
+ run: |
+- if [[ "${{ matrix.python-version }}" = "3.8" ]]; then
+- conda install -c conda-forge tornado=6
+- else
++ if [[ "${{ matrix.python-version }}" = "3.6" ]]; then
+ conda install -c conda-forge tornado=5
++ else
++ conda install -c conda-forge tornado=6
+ fi
+
+ - name: Install distributed from source
diff -Nru dask.distributed-2021.01.0+ds.1/debian/patches/series
dask.distributed-2021.01.0+ds.1/debian/patches/series
--- dask.distributed-2021.01.0+ds.1/debian/patches/series 2021-07-13
09:19:56.000000000 -0700
+++ dask.distributed-2021.01.0+ds.1/debian/patches/series 2021-11-27
11:29:20.000000000 -0800
@@ -8,3 +8,5 @@
mark-tests-require-installation.patch
fall-back-to-ipv6-localhost.patch
0001-Remove-tests-for-process_time-and-thread_time-4895.patch
+pass-host-to-local-cluster.patch
+python3.9-compatibility.patch
