One of the goals is to make the creation of the distribution tarball
more directly traceable to the git repository. That is why we removed
the "make distprep" step.
Here I want to take another step in that direction, by changing "make
dist" to directly use "git archive", rather than the custom shell script
it currently runs.
The simple summary is that it would run
git archive --format tar.gz --prefix postgresql-17devel/ HEAD -o
postgresql-17devel.tar.gz
(with appropriate version numbers, of course), and that's the tarball we
would ship.
There are analogous commands for other compression formats.
The actual command gets subtly more complicated if you need to run this
in a separate build directory. In my attached patch, the make version
doesn't support vpath at the moment, just so that it's easier to
understand for now. The meson version is a bit hairy.
I have studied and tested this quite a bit, and I have found that the
archives produced this way are deterministic and reproducible, meaning
for a given commit the result file should always be bit-for-bit identical.
The exception is that if you use a git version older than 2.38.0, gzip
records the platform in the archive, so you'd get a different output on
Windows vs. macOS vs. "UNIX" (everything else). In git 2.38.0, this was
changed so that everything is recorded as "UNIX" now. This is just
something to keep in mind. This issue is specific to the gzip format,
it does not affect other compression formats.
Meson has its own distribution building command (meson dist), but opted
against using that. The main problem is that the way they have
implemented it, it is not deterministic in the above sense. (Another
point is of course that we probably want a "make" version for the time
being.)
But the target name "dist" in meson is reserved for that reason, so I
needed to call the custom target "pgdist".
I did take one idea from meson: It runs a check before git archive that
the checkout is clean. That way, you avoid mistakes because of
uncommitted changes. This works well in my "make" implementation. In
the meson implementation, I had to find a workaround, because a
custom_target cannot have a dependency on a run_target. As also
mentioned above, the whole meson implementation is a bit ugly.
Anyway, with the attached patch you can do
make dist
or
meson compile -C build pgdist
and it produces the same set of tarballs as before, except it's done
differently.
The actual build scripts need some fine-tuning, but the general idea is
correct, I think.From 4b128faca90238d0a0bb6949a8050c2501d1bd67 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Sat, 20 Jan 2024 21:54:36 +0100
Subject: [PATCH v0] make dist uses git archive
---
GNUmakefile.in | 34 ++++++++++++----------------------
meson.build | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 50 insertions(+), 22 deletions(-)
diff --git a/GNUmakefile.in b/GNUmakefile.in
index eba569e930e..3e04785ada2 100644
--- a/GNUmakefile.in
+++ b/GNUmakefile.in
@@ -87,29 +87,19 @@ update-unicode: | submake-generated-headers
submake-libpgport
distdir = postgresql-$(VERSION)
dummy = =install=
+GIT = git
+
dist: $(distdir).tar.gz $(distdir).tar.bz2
- rm -rf $(distdir)
-
-$(distdir).tar: distdir
- $(TAR) chf $@ $(distdir)
-
-.INTERMEDIATE: $(distdir).tar
-
-distdir-location:
- @echo $(distdir)
-
-distdir:
- rm -rf $(distdir)* $(dummy)
- for x in `cd $(top_srcdir) && find . \( -name CVS -prune \) -o \( -name
.git -prune \) -o -print`; do \
- file=`expr X$$x : 'X\./\(.*\)'`; \
- if test -d "$(top_srcdir)/$$file" ; then \
- mkdir "$(distdir)/$$file" && chmod 777 "$(distdir)/$$file"; \
- else \
- ln "$(top_srcdir)/$$file" "$(distdir)/$$file" >/dev/null 2>&1 \
- || cp "$(top_srcdir)/$$file" "$(distdir)/$$file"; \
- fi || exit; \
- done
- $(MAKE) -C $(distdir) distclean
+
+.PHONY: check-dirty-index
+check-dirty-index:
+ $(GIT) diff-index --quiet HEAD
+
+$(distdir).tar.gz: check-dirty-index
+ $(GIT) archive --format tar.gz --prefix $(distdir)/ HEAD -o $@
+
+$(distdir).tar.bz2: check-dirty-index
+ $(GIT) -c tar.tar.bz2.command='$(BZIP2) -c' archive --format tar.bz2
--prefix $(distdir)/ HEAD -o $@
distcheck: dist
rm -rf $(dummy)
diff --git a/meson.build b/meson.build
index c317144b6bc..f0d870c5192 100644
--- a/meson.build
+++ b/meson.build
@@ -3347,6 +3347,44 @@ run_target('help',
+###############################################################
+# Distribution archive
+###############################################################
+
+git = find_program('git', required: false, native: true, disabler: true)
+bzip2 = find_program('bzip2', required: false, native: true, disabler: true)
+
+distdir = meson.project_name() + '-' + meson.project_version()
+
+check_dirty_index = run_target('check-dirty-index',
+ command: [git, 'diff-index', '--quiet', 'HEAD'])
+
+tar_gz = custom_target('tar.gz',
+ build_always_stale: true,
+ command: [git, '-C', '@SOURCE_ROOT@', 'archive',
+ '--format', 'tar.gz',
+ '--prefix', distdir + '/',
+ '-o', '@BUILD_ROOT@/@OUTPUT@',
+ 'HEAD', '.'],
+ install: false,
+ output: distdir + '.tar.gz',
+)
+
+tar_bz2 = custom_target('tar.bz2',
+ build_always_stale: true,
+ command: [git, '-C', '@SOURCE_ROOT@', '-c', 'tar.tar.bz2.command=' +
bzip2.path() + ' -c', 'archive',
+ '--format', 'tar.bz2',
+ '--prefix', distdir + '/',
+ '-o', '@BUILD_ROOT@/@OUTPUT@',
+ 'HEAD', '.'],
+ install: false,
+ output: distdir + '.tar.bz2',
+)
+
+alias_target('pgdist', [check_dirty_index, tar_gz, tar_bz2])
+
+
+
###############################################################
# The End, The End, My Friend
###############################################################
--
2.43.0
