Hello Mercurial developers, I invested a bit of time on Mercurial in the hope of improving its situation for the next years.
Good news: Mercurial on the default branch now (after https://foss.heptapod.net/mercurial/mercurial-devel/-/merge_requests/1131) supports the basic of modern Python packaging (PEP 517), which involves "PEP 517 frontends" (pip, pipx, UV, PDM, ...), "PEP 517 backends" (setuptools in the case of Mercurial) and "isolated builds". There are still things to be fixed in terms of packaging (in particular the Windows wheels), but it seems doable and actually not so difficult. I recently worked a bit on the situation on Windows and learned why native Windows applications are so important and how they can be built. I asked few questions regarding this subject on https://discuss.python.org (in particular https://discuss.python.org/t/how-to-create-a-window-command-line-application-and-its-installer-from-wheels/76136/4) and what I learned could be interesting for Mercurial. I'd like with the email to open a discussion on this subject, share few useful links and ask few questions. ## Why native applications are useful - Some Windows users like installers (download by clicking, execute, everything is installed). Other are more happy with the Microsoft Store. Both solutions require a native Windows application. - The terminal commands on Windows have to be implemented with .exe files (unlike on Unix where Python scripts can be used). .bat files are not a good solution. The .exe files used by solutions like pipx, UV and pixi are not signed (and cannot be signed). - These .exe files have to be signed to please the antivirus software (otherwise there can be big startup delays of the other of hundreds of milliseconds). Note that this is not particularly an issue related to Python. chg/rhg would also suffer from this potential issue. Therefore native applications are useful on Windows... And much less on Unix. ## About creation and distribution of native Windows applications Maintainers have to do few things: 1. Create the application (a binary executable .exe or a directory containing at least one executable file .exe) 2. Sign the binary (not absolutely necessary though, but for Mercurial the point of an app is to distribute a signed executable) 3. Create an installer (from nothing - manual download of the app - to .msi or MSIX installers) 4. Distribute the installer (from just put it on the web to register to the Microsoft Store) Many different tools can be used to help for these steps. ## What is the current situation for Mercurial - TortoiseHg applications are based on py2exe, an old project which creates native Windows applications through a distutils extension. Distutils does not exist anymore in Python >=3.12 so it is actually a setuptools extension. The advantage of this approach is that (if I understand correctly) py2exe tries to automatically remove from the application parts of Python (and other libraries) which are not used. The disadvantage of py2exe is that it is by design (being a distutils/setuptools extension) not up-to-date with the improvements of Python packaging (more on this in the next section). Therefore, Mercurial's setup.py currently contains py2exe related hacks whereas it should be used only to create a source distribution (sdist) and wheels. Note that even from the point of view of py2exe, using setup.py is deprecated (https://github.com/py2exe/py2exe?tab=readme-ov-file#usage). - PyOxidizer has been used to build Mercurial Windows native applications. I don't exactly know why but it does not work anymore with recent Mercurial versions. PyOxidizer produces from a Python project a Rust binary containing the Python interpreter. The advantage of the approach is that you get a one-file application (hg.exe), which cannot be modified after build. The one-file nature is actually a quite weak advantage. All Windows installers know how to handle a .zip file by unzip it in the right place and adding the right path in the PATH. Users don't see that the resulting app is a directory (and most Windows apps are actually installed as directories). The disadvantage of PyOxidizer is that few things do not work like in standard Python on Windows. Moreover, PyOxidizer is complicated and seems unmaintained since its creator declared that he stopped his implication in the project. Moreover, this is not the method proposed by Microsoft engineers working on Python and Python packaging. ## Alternative methods and tools ### What are native Windows Python apps and how to create them? An important point to realize is that native Windows Python apps are conceptually very simple. It is "just" a directory with 1. a launcher (for Mercurial hg.exe) written in a native language (usually C, or Rust if one wants but it is so simple that it does not seem so useful for this application) 2. a directory with a Python interpreter (one should download the official "Windows embeddable package" from https://www.python.org/downloads/windows/) 3. another directory with the Python packages (an important point is that they can be installed here from wheels with just a `pip install`) So it's quite simple to write by hands few scripts to create with few commands a native app. There are also modern tools to help. Few links about this possibility: - https://discuss.python.org/t/how-to-create-a-window-command-line-application-and-its-installer-from-wheels/76136 - https://stevedower.id.au/blog/build-a-python-app - https://github.com/pypa/packaging.python.org/pull/1778 (doc PR, preview result at https://python-packaging-user-guide--1778.org.readthedocs.build/en/1778/) Three modern tools to help: - https://github.com/zooba/pymsbuild - https://beeware.org/project/projects/tools/briefcase/ - https://pynsist.readthedocs.io ### pyapp: another alternative, very simple and potentially usable for Mercurial [pyapp](https://github.com/ofek/pyapp) is an alternative solution to create (with Rust) another type of native application to use a Python CLI (like hg). The principle is quite different because it creates a one-file app (hg.exe in our case) which is both a Mercurial installer (using UV in the background) and a Mercurial caller. This is attractive for a simple Mercurial installer because it is VERY simple and easy to create (once Mercurial works with a console-script Python entry point and is installable with UV on Windows). I think we could have the app and the installer after literally few minutes of work. ### Windows apps and installers based on conda-forge Spyder uses this other approach for its "Standalone installers" (https://docs.spyder-ide.org/current/installation.html#standalone-installers) and it works very well. See https://github.com/spyder-ide/spyder/tree/master/installers-conda. ## Questions for Mercurial/TortoiseHg devs - Are the hg executables in current Windows apps signed? - Does the Mercurial project have what is needed to sign executable? - Did I misunderstand something? ## What's next Windows apps and installers for Mercurial and TortoiseHg have to be produced for recent versions. Since one needs to work on the process to create them, it seems to me that modern methods could be considered, namely an app based on the Windows embeddable Python package for TortoiseHg and maybe an app based on pyapp for Mercurial. However, if the work is done to produce an app from the Windows embeddable Python package for TortoiseHg, it should be very easy to adapt it for Mercurial. For TortoiseHg, a conda-forge based installer similar to the one of Spyder could also be an interesting solution. However, there is the question of how to sign `hg.exe` in this case. An aspect to consider is the ability to use pip, UV or conda (from a new Mercurial command) to manage (install/uninstall/update) Mercurial extensions. Some methods could allow that and others not (in particular I don't think it would be possible with PyOxidizer). Pierre -- Pierre Augier - CR CNRS http://www.legi.grenoble-inp.fr LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels BP53, 38041 Grenoble Cedex, France tel:+33.4.56.52.86.16 _______________________________________________ Mercurial-devel mailing list Mercurial-devel@lists.mercurial-scm.org https://lists.mercurial-scm.org/mailman/listinfo/mercurial-devel
