#37106: pylibmc install fails from tests requirements on Python 3.12+
-------------------------------+-----------------------------------------
Reporter: Kiko Barr | Type: Uncategorized
Status: new | Component: Documentation
Version: 6.0 | Severity: Normal
Keywords: test, pylibmc | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
π Hello from the Django sprint at PyCon 2026!
== High Level Overview ==
When following the Unit Tests walkthrough, `python -m pip install -r
tests/requirements/py3.txt` is failing to install pylibmc. This is because
Django `pyproject.toml` requires CPython 3.12+, but wheel for pylimbc
hosted on PyPI only goes up to 3.11. See PyPI:
https://pypi.org/project/pylibmc/#files
Six people at the sprint encountered this same issue. I think this is
important to document because new contributors are encouraged to start by
running tests and then they run into this issue. While many of the tests
can still be run, some will produce errors if pylibmc is not pre-
installed.
There have been several previous Django-related posts discuss removing
pylibmc as a dependency, but have not come to a conclusion. While the
discussion is occurring, I think it would be good to update the test
documentation with some short-term workarounds in the meantime.
Django closed issues:
* https://code.djangoproject.com/ticket/36427
* https://code.djangoproject.com/ticket/36153
Django forum posts:
* https://forum.djangoproject.com/t/error-when-installing-requirements-
for-django-tests-fatal-error-libmemcached-memcached-h-file-not-found-
include-libmemcached-memcached-h/1685
* https://forum.djangoproject.com/t/should-we-stop-recommending-
pylibmc/42993
pylibmc issue:
* https://github.com/lericson/pylibmc/issues/288
== Reproducing the Issue ==
The issue appears while running commands from the "Quickstart" instruction
in the "Unit Tests" documentation, found here:
https://docs.djangoproject.com/en/dev/internals/contributing/writing-code
/unit-tests/#quickstart
Based on the documentation, I created a virtual environment and ran these
commands:
{{{
git clone https://github.com/YourGitHubName/django.git django-repo
# created and activated the venv here and ran the rest of the commands
cd django-repo/tests
python -m pip install -e ..
}}}
However, the next command will fail:
{{{
python -m pip install -r requirements/py3.txt
}}}
This is the error message:
{{{
Building wheels for collected packages: pylibmc
Building wheel for pylibmc (pyproject.toml) ... error
error: subprocess-exited-with-error
Γ Building wheel for pylibmc (pyproject.toml) did not run successfully.
β exit code: 1
β°β> [6 lines of output]
In file included from src/_pylibmcmodule.c:34:
src/_pylibmcmodule.h:42:10: fatal error: 'libmemcached/memcached.h'
file not found
42 | #include <libmemcached/memcached.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a
problem with pip.
ERROR: Failed building wheel for pylibmc
Failed to build pylibmc
error: failed-wheel-build-for-install
Γ Failed to build installable wheels for some pyproject.toml based
projects
β°β> pylibmc
}}}
== Issue Description ==
Issue for those running Python 3.12 with no libmemcached installed:
* When running `python -m pip install -r requirements/py3.txt`, pip reads
`requirements/py3.txt` and sees pylibmc.
* pip queries PyPI for pylibmc distributions, including wheel files
(.whl) and source archive (.tar.gz / .zip, called sdist)
* Since there is no wheel for Python 3.12, pip does not use the wheel
install and instead downloads the sdist to build pylibmc locally.
* The source build compiles C code for pylibmc, which needs libmemcached
headers/libraries. If libmemcached is not installed, then this step fails,
producing the error from above, `src/_pylibmcmodule.h:42:10: fatal error:
'libmemcached/memcached.h' file not found`
Additional issue for some on macOS:
* Even after libmemcached was installed globally, people on macOS
continued to run on the same error. This is because Homebrew, a very
common package manager for macOS, often installs under a non-default
prefix (usually /opt/homebrew). If that prefix isnβt in the
compiler/linker search paths for that shell/build, compile fails even
though libmemcached is installed.
* People on macOS had to run additional commands to direct clang to the
right path using `CPPFLAGS` to tell compiler where headers are and
`LDFLAGS` to tell linker where libs are.
== Short-term Workaround ==
Of the six people at the sprint who encountered this same issue, there
were several different versions of the same workaround.
For four of us on Macs, we needed to first install libmemcached headers
(someone else also had to install memcached, zlib, and pkg-config to get
it to work, which may be worth looking into):
{{{
# install globally outside of venv
brew install libmemcached
# run these inside of the venv
export CPPFLAGS="-I$(brew --prefix)/include"
export LDFLAGS="-L$(brew --prefix)/lib"
python -m pip install -r requirements/py3.txt
}}}
Someone on Linux Omarchy only had to run:
`sudo pacman -S libmemcached-awesome`
Someone else on Linux Debian ran:
`sudo apt install libmemcached-dev`
Listing all these variations for different operating systems is not the
most elegant solution, but it does facilitate a smoother onboarding
experience for new contributors.
--
Ticket URL: <https://code.djangoproject.com/ticket/37106>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019e3d07b121-7ea7c844-635d-479d-bb58-d874f5077718-000000%40eu-central-1.amazonses.com.
