Package: release.debian.org Severity: normal Tags: bullseye User: release.debian....@packages.debian.org Usertags: pu X-Debbugs-Cc: r...@debian.org
This is a future unblock request before I upload djbdns-1:1.05-13+deb11u1 to fix an RC bug in stable. [ Reason ] See #996807 for more information; in short, the tinydns server program may stop responding to DNS queries because it has reached the process data size limit. The reason is that recent versions of glibc appear to need more memory, so the value of the limit that has been enough for many years now is no longer adequate. Note that this only happens on some Debian architectures. [ Impact ] The authoritative server (tinydns) from the djbdns package will stop responding to queries, thus not fulfilling its basic purpose. [ Tests ] The attached debdiff also backports some improvements to the autopkgtest suite, including a new test that sends a lot of queries to the tinydns server to make sure it keeps responding. [ Risks ] The fix itself should not pose any risk at all, except maybe when running the tinydns server in extremely resource-contstrained environments when the additional 100K now available to the tinydns process may eat into some other processes' memory. However, it is my belief that people running tinydns in such environments will use their own configuration files or at least be mindful of changes in the Debian packages. The changes to the autopkgtest suite should also not pose any risk at all: they are mostly code clean-up and simplification. If, however, it is your opinion that they are too extensive, I can prepare another upload that drops them and only adds the "raise the process data size limit" patch itself. [ Checklist ] [x] *all* changes are documented in the d/changelog [x] I reviewed all changes and I approve them [x] attach debdiff against the package in (old)stable [x] the issue is verified as fixed in unstable [ Changes ] The data limit for the djbdns services is raised by an additional 100,000 bytes to reflect the programs' needs imposed by glibc. The autopkgtest suite's Python code is cleaned up. The autopkgtest suite performs an additional check: run a lot of queries against a tinydns server to make sure it keeps responding. Thanks in advance for looking at this, and keep up the great work! G'luck, Peter
diff -Nru djbdns-1.05/debian/changelog djbdns-1.05/debian/changelog --- djbdns-1.05/debian/changelog 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/changelog 2022-02-26 19:29:35.000000000 +0200 @@ -1,3 +1,18 @@ +djbdns (1:1.05-13+deb11u1) bullseye; urgency=medium + + * Add the 0011-datalimit patch to catch up with recent versions of + glibc generating larger executable files. Closes: #996807 + * Several improvements to the Python tinytest autopkgtest tool: + - use "with subprocess.Popen()" + - simplify the command-line parsing a bit + - minor import statement fixes + - add a tox.ini file to make it easier to run static code checkers + - turn a class into a dataclass + - send a lot of queries to tinydns to make sure that the fix for + #996807 actually works + + -- Peter Pentchev <r...@debian.org> Sat, 26 Feb 2022 19:29:35 +0200 + djbdns (1:1.05-13) unstable; urgency=medium * Do not even consider running tinydns during the package build. diff -Nru djbdns-1.05/debian/patches/0011-datalimit.patch djbdns-1.05/debian/patches/0011-datalimit.patch --- djbdns-1.05/debian/patches/0011-datalimit.patch 1970-01-01 02:00:00.000000000 +0200 +++ djbdns-1.05/debian/patches/0011-datalimit.patch 2022-02-26 19:29:35.000000000 +0200 @@ -0,0 +1,38 @@ +Description: Raise the axfrdns, dnscache, and tinydns data limit. +Bug-Debian: https://bugs.debian.org/996807 +Author: Peter Pentchev <r...@ringlet.net> +Last-Update: 2021-11-13 + +--- a/axfrdns-conf.c ++++ b/axfrdns-conf.c +@@ -50,7 +50,7 @@ + + start("run"); + outs("#!/bin/sh\nexec 2>&1\nexec envdir ./env sh -c '\n exec envuidgid "); outs(user); +- outs(" softlimit -d300000 tcpserver -vDRHl0 -x tcp.cdb -- \"$IP\" 53 "); ++ outs(" softlimit -d400000 tcpserver -vDRHl0 -x tcp.cdb -- \"$IP\" 53 "); + outs(auto_home); outs("/sbin/axfrdns\n'\n"); + finish(); + perm(0755); +--- a/dnscache-conf.c ++++ b/dnscache-conf.c +@@ -118,7 +118,7 @@ + seed_addtime(); perm(0644); + seed_addtime(); start("env/CACHESIZE"); outs("1000000\n"); finish(); + seed_addtime(); perm(0644); +- seed_addtime(); start("env/DATALIMIT"); outs("3000000\n"); finish(); ++ seed_addtime(); start("env/DATALIMIT"); outs("4000000\n"); finish(); + seed_addtime(); perm(0644); + seed_addtime(); start("run"); + outs("#!/bin/sh\nexec 2>&1\nexec <seed\nexec envdir ./env sh -c '\n exec envuidgid "); outs(user); +--- a/tinydns-conf.c ++++ b/tinydns-conf.c +@@ -46,7 +46,7 @@ + + start("run"); + outs("#!/bin/sh\nexec 2>&1\nexec envuidgid "); outs(user); +- outs(" envdir ./env softlimit -d300000 "); ++ outs(" envdir ./env softlimit -d400000 "); + outs(auto_home); outs("/sbin/tinydns\n"); + finish(); + perm(0755); diff -Nru djbdns-1.05/debian/patches/series djbdns-1.05/debian/patches/series --- djbdns-1.05/debian/patches/series 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/patches/series 2022-02-26 19:29:35.000000000 +0200 @@ -7,3 +7,4 @@ 0007-dnscache-merge-similar-outgoing-udp-packets.patch 0008-Cache-SOA-records.patch 0009-usr-sbin.patch +0011-datalimit.patch diff -Nru djbdns-1.05/debian/service/dnscache/conf/DATALIMIT djbdns-1.05/debian/service/dnscache/conf/DATALIMIT --- djbdns-1.05/debian/service/dnscache/conf/DATALIMIT 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/service/dnscache/conf/DATALIMIT 2022-02-26 19:29:35.000000000 +0200 @@ -1 +1 @@ -3000000 +4000000 diff -Nru djbdns-1.05/debian/service/tinydns/conf/DATALIMIT djbdns-1.05/debian/service/tinydns/conf/DATALIMIT --- djbdns-1.05/debian/service/tinydns/conf/DATALIMIT 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/service/tinydns/conf/DATALIMIT 2022-02-26 19:29:35.000000000 +0200 @@ -1 +1 @@ -3000000 +4000000 diff -Nru djbdns-1.05/debian/tests/pyproject.toml djbdns-1.05/debian/tests/pyproject.toml --- djbdns-1.05/debian/tests/pyproject.toml 1970-01-01 02:00:00.000000000 +0200 +++ djbdns-1.05/debian/tests/pyproject.toml 2022-02-26 19:29:35.000000000 +0200 @@ -0,0 +1,2 @@ +[tool.black] +line-length = 79 diff -Nru djbdns-1.05/debian/tests/setup.cfg djbdns-1.05/debian/tests/setup.cfg --- djbdns-1.05/debian/tests/setup.cfg 1970-01-01 02:00:00.000000000 +0200 +++ djbdns-1.05/debian/tests/setup.cfg 2022-02-26 19:29:35.000000000 +0200 @@ -0,0 +1,6 @@ +[flake8] +ignore = E203,W503 + +[mypy] +python_version = 3.7 +strict = True diff -Nru djbdns-1.05/debian/tests/tinytest/defs.py djbdns-1.05/debian/tests/tinytest/defs.py --- djbdns-1.05/debian/tests/tinytest/defs.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/defs.py 2022-02-26 19:29:35.000000000 +0200 @@ -4,7 +4,7 @@ import pathlib import subprocess -from typing import Dict, List, Union +from typing import Dict, List, Union # noqa: H301 @dataclasses.dataclass(frozen=True) diff -Nru djbdns-1.05/debian/tests/tinytest/__main__.py djbdns-1.05/debian/tests/tinytest/__main__.py --- djbdns-1.05/debian/tests/tinytest/__main__.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/__main__.py 2022-02-26 19:29:35.000000000 +0200 @@ -29,14 +29,14 @@ parser.add_argument( "-b", "--bindir", - type=str, + type=pathlib.Path, required=True, help="the path to the djbdns user tools", ) parser.add_argument( "-s", "--sbindir", - type=str, + type=pathlib.Path, required=True, help="the path to the djbdns system tools", ) @@ -60,11 +60,11 @@ subenv["LANGUAGE"] = "en" return Config( - bindir=pathlib.Path(args.bindir), - sbindir=pathlib.Path(args.sbindir), - skip_run_test=bool(args.skip_run_test), + bindir=args.bindir, + sbindir=args.sbindir, + skip_run_test=args.skip_run_test, subenv=subenv, - verbose=bool(args.verbose), + verbose=args.verbose, ) @@ -80,8 +80,9 @@ if not cfg.skip_run_test: test_run.test_tinydns_run(cfg, tempd) + test_run.test_tinydns_run_udp(cfg, tempd) else: - print("\n==== Skipping test_tinydns_run") + print("\n==== Skipping test_tinydns_run*") print("\n==== All fine!") diff -Nru djbdns-1.05/debian/tests/tinytest/test_conf.py djbdns-1.05/debian/tests/tinytest/test_conf.py --- djbdns-1.05/debian/tests/tinytest/test_conf.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/test_conf.py 2022-02-26 19:29:35.000000000 +0200 @@ -5,7 +5,7 @@ import pwd import sys -from typing import List, Union +from typing import List, Union # noqa: H301 from . import defs diff -Nru djbdns-1.05/debian/tests/tinytest/test_edit.py djbdns-1.05/debian/tests/tinytest/test_edit.py --- djbdns-1.05/debian/tests/tinytest/test_edit.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/test_edit.py 2022-02-26 19:29:35.000000000 +0200 @@ -1,10 +1,10 @@ """Test the `tinydns-edit` tool and the generated data file.""" -import pathlib import os +import pathlib import sys -from typing import List, Union +from typing import List, Union # noqa: H301 from . import defs diff -Nru djbdns-1.05/debian/tests/tinytest/test_get.py djbdns-1.05/debian/tests/tinytest/test_get.py --- djbdns-1.05/debian/tests/tinytest/test_get.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/test_get.py 2022-02-26 19:29:35.000000000 +0200 @@ -5,7 +5,7 @@ import re import sys -from typing import List, Union +from typing import List, Union # noqa: H301 from . import defs diff -Nru djbdns-1.05/debian/tests/tinytest/test_run.py djbdns-1.05/debian/tests/tinytest/test_run.py --- djbdns-1.05/debian/tests/tinytest/test_run.py 2020-07-28 00:03:21.000000000 +0300 +++ djbdns-1.05/debian/tests/tinytest/test_run.py 2022-02-26 19:29:35.000000000 +0200 @@ -4,16 +4,31 @@ import enum import os import pathlib +import socket +import struct import subprocess import sys import time -from typing import Callable, Dict, List, Tuple, Union +from typing import ( + Callable, # noqa: H301 + Dict, + List, + Tuple, + Union, + TYPE_CHECKING, +) from . import defs from . import test_get +if TYPE_CHECKING: + PopenStr = subprocess.Popen[str] # pylint: disable=unsubscriptable-object +else: + PopenStr = subprocess.Popen + + class ProcessFDType(str, enum.Enum): """File descriptor types, enough for this very limited test.""" @@ -54,22 +69,15 @@ files: List[ProcessFDInfo] +@dataclasses.dataclass class ParseProcessInfo: """Parse the output of `lsof -F` for our very limited testing needs.""" cfg: defs.Config - cproc: Dict[str, str] - cfile: Dict[str, str] - cfiles: List[ProcessFDInfo] - result: Dict[int, ProcessInfo] - - def __init__(self, cfg: defs.Config) -> None: - """Initialize an empty object.""" - self.cfg = cfg - self.cproc = {} - self.cfile = {} - self.cfiles = [] - self.result = {} + cproc: Dict[str, str] = dataclasses.field(default_factory=dict) + cfile: Dict[str, str] = dataclasses.field(default_factory=dict) + cfiles: List[ProcessFDInfo] = dataclasses.field(default_factory=list) + result: Dict[int, ProcessInfo] = dataclasses.field(default_factory=dict) def finish_file_ipv4(self) -> ProcessFDInfo: """Store the information about a socket.""" @@ -188,6 +196,72 @@ return data.proto == "UDP" and data.address == ipaddr and data.port == 53 +def run_tinydns( + cfg: defs.Config, + tempd: pathlib.Path, + callback: Callable[[PopenStr, str], None], +) -> None: + """Test the `tinydns` and `dnsq` programs.""" + ipaddr = defs.RECORDS[0].address + svcdir = tempd / defs.SVCDIR + if not svcdir.is_dir(): + sys.exit(f"Expected {svcdir} to be a directory") + os.chdir(svcdir) + + with subprocess.Popen( + ["./run"], + shell=False, + env=cfg.subenv, + bufsize=0, + encoding=defs.MINENC, + stdout=subprocess.PIPE, + ) as proc: + try: + print(f"- spawned process {proc.pid}") + assert proc.stdout is not None + print("- waiting for the 'starting tinydns' line") + line = proc.stdout.readline() + print(f"- got line {line!r}") + if line != "starting tinydns\n": + sys.exit(f"Unexpected first line from tinydns: {line!r}") + + lines = cfg.check_output( + [ + "lsof", + "-a", + "-n", + "-P", + "-p", + str(proc.pid), + "-F", + "-i4udp:53", + ] + ).splitlines() + cfg.diag(f"lsof output: {lines!r}") + processes = ParseProcessInfo(cfg).parse(lines) + if not check_single_udp_socket(processes, proc.pid, ipaddr): + sys.exit( + f"tinydns should listen on a single UDP socket: " + f"{processes!r}" + ) + print(f"- tinydns is listening at {ipaddr}:53") + callback(proc, ipaddr) + finally: + print("- we spawned a process, checking if it has exited") + res = proc.poll() + if res is None: + print(f"Terminating process {proc.pid}") + proc.terminate() + time.sleep(0.5) + res = proc.poll() + if res is None: + print(f"Killing process {proc.pid}") + proc.kill() + + res = proc.wait() + print(f"Process {proc.pid}: exit code {res}") + + def test_tinydns_run(cfg: defs.Config, tempd: pathlib.Path) -> None: """Test the `tinydns` and `dnsq` programs.""" print("\n==== test_tinydns_run") @@ -199,40 +273,9 @@ if not dnsq.is_file() or not os.access(dnsq, os.X_OK): sys.exit(f"Not an executable file: {dnsq}") - ipaddr = defs.RECORDS[0].address - svcdir = tempd / defs.SVCDIR - if not svcdir.is_dir(): - sys.exit(f"Expected {svcdir} to be a directory") - os.chdir(svcdir) - - proc = None - try: - proc = subprocess.Popen( - ["./run"], - shell=False, - env=cfg.subenv, - bufsize=0, - encoding=defs.MINENC, - stdout=subprocess.PIPE, - ) - print(f"- spawned process {proc.pid}") + def test_dnsq(proc: PopenStr, ipaddr: str) -> None: + """Run `dnsq` against the specified `tinydns` instance.""" assert proc.stdout is not None - print("- waiting for the 'starting tinydns' line") - line = proc.stdout.readline() - print(f"- got line {line!r}") - if line != "starting tinydns\n": - sys.exit(f"Unexpected first line from tinydns: {line!r}") - - lines = cfg.check_output( - ["lsof", "-a", "-n", "-P", "-p", str(proc.pid), "-F", "-i4udp:53"] - ).splitlines() - cfg.diag(f"lsof output: {lines!r}") - processes = ParseProcessInfo(cfg).parse(lines) - if not check_single_udp_socket(processes, proc.pid, ipaddr): - sys.exit( - f"tinydns should listen on a single UDP socket: {processes!r}" - ) - print(f"- tinydns is listening at {ipaddr}:53") for rec in defs.RECORDS: rdef = defs.TYPES[rec.rtype] @@ -256,17 +299,68 @@ print(" - there should be a single line of output from tinydns") line = proc.stdout.readline() cfg.diag(f"line: {line!r}") - finally: - if proc is not None: - print("- we spawned a process, checking if it has exited") - res = proc.poll() - if res is None: - print(f"Terminating process {proc.pid}") - proc.terminate() - time.sleep(0.5) - res = proc.poll() - if res is None: - print(f"Killing process {proc.pid}") - proc.kill() - res = proc.wait() - print(f"Process {proc.pid}: exit code {res}") + + run_tinydns(cfg, tempd, test_dnsq) + + +def test_tinydns_run_udp(cfg: defs.Config, tempd: pathlib.Path) -> None: + """Test the `tinydns` program by sending UDP packets.""" + print("\n==== test_tinydns_run_udp") + if os.geteuid() != 0: + print("- not running as root, skipped") + return + + def test_dnsq(proc: PopenStr, ipaddr: str) -> None: + """Run `dnsq` against the specified `tinydns` instance.""" + assert proc.stdout is not None + + pkt = struct.pack( + ">HHHHHH8p4p1pHH", + 17, + 0, + 1, + 0, + 0, + 0, + "example".encode("us-ascii"), + "com".encode("us-ascii"), + b"", + 1, + 1, + ) + print(f"- query packet: {pkt!r}") + + sock = socket.socket( + socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP + ) + sock.connect((ipaddr, 53)) + + for idx in range(1000000): + pkt = struct.pack(">H", (idx + 17) % 65536) + pkt[2:] + sent = sock.send(pkt) + if sent != len(pkt): + sys.exit(f"Only sent {sent} of {len(pkt)} bytes") + + line = proc.stdout.readline() + if "+" not in line: + sys.exit( + f"Query {idx + 1}: expected a '+' in " + f"the tinydns log line {line!r}" + ) + + resp = sock.recv(4096) + if resp[:2] != pkt[:2]: + sys.exit( + f"Query {idx + 1}: unexpected ID: " + f"query {pkt!r} response {resp!r}" + ) + if resp[4:8] != b"\x00\x01\x00\x01": + sys.exit( + f"Query {idx + 1}: unexpected query/response count: " + f"query {pkt!r} response {resp!r}" + ) + + if (idx % 10000) == 9999: + print(f"- sent {idx + 1} queries to {ipaddr}") + + run_tinydns(cfg, tempd, test_dnsq) diff -Nru djbdns-1.05/debian/tests/tox.ini djbdns-1.05/debian/tests/tox.ini --- djbdns-1.05/debian/tests/tox.ini 1970-01-01 02:00:00.000000000 +0200 +++ djbdns-1.05/debian/tests/tox.ini 2022-02-26 19:29:35.000000000 +0200 @@ -0,0 +1,55 @@ +[tox] +envlist = + black + pep8 + pep8h + mypy + pylint +skipsdist = True + +[defs] +pyfiles = + tinytest + +[testenv:black] +skipinstall = True +deps = + black >= 21b0, < 22b0 +commands = + black --check {[defs]pyfiles} + +[testenv:black-reformat] +skipinstall = True +deps = + black >= 21b0, < 22b0 +commands = + black {[defs]pyfiles} + +[testenv:pep8] +skipinstall = True +deps = + flake8 +commands = + flake8 {[defs]pyfiles} + +[testenv:pep8h] +skipinstall = True +deps = + flake8 + hacking >= 4 +commands = + flake8 {[defs]pyfiles} + +[testenv:mypy] +skipinstall = True +deps = + mypy +commands = + mypy {[defs]pyfiles} + +[testenv:pylint] +skipinstall = True +deps = + pylint +commands = + pylint {[defs]pyfiles}
signature.asc
Description: PGP signature
