Hi, On Mon, Jan 22, 2024 at 3:32 PM Peter Eisentraut <pe...@eisentraut.org> wrote: > > One of the goals is to make the creation of the distribution tarball > more directly traceable to the git repository. That is why we removed > the "make distprep" step. > > Here I want to take another step in that direction, by changing "make > dist" to directly use "git archive", rather than the custom shell script > it currently runs. > > The simple summary is that it would run > > git archive --format tar.gz --prefix postgresql-17devel/ HEAD -o > postgresql-17devel.tar.gz > > (with appropriate version numbers, of course), and that's the tarball we > would ship. > > There are analogous commands for other compression formats. > > The actual command gets subtly more complicated if you need to run this > in a separate build directory. In my attached patch, the make version > doesn't support vpath at the moment, just so that it's easier to > understand for now. The meson version is a bit hairy. > > I have studied and tested this quite a bit, and I have found that the > archives produced this way are deterministic and reproducible, meaning > for a given commit the result file should always be bit-for-bit identical. > > The exception is that if you use a git version older than 2.38.0, gzip > records the platform in the archive, so you'd get a different output on > Windows vs. macOS vs. "UNIX" (everything else). In git 2.38.0, this was > changed so that everything is recorded as "UNIX" now. This is just > something to keep in mind. This issue is specific to the gzip format, > it does not affect other compression formats. > > Meson has its own distribution building command (meson dist), but opted > against using that. The main problem is that the way they have > implemented it, it is not deterministic in the above sense. (Another > point is of course that we probably want a "make" version for the time > being.) > > But the target name "dist" in meson is reserved for that reason, so I > needed to call the custom target "pgdist". > > I did take one idea from meson: It runs a check before git archive that > the checkout is clean. That way, you avoid mistakes because of > uncommitted changes. This works well in my "make" implementation. In > the meson implementation, I had to find a workaround, because a > custom_target cannot have a dependency on a run_target. As also > mentioned above, the whole meson implementation is a bit ugly. > > Anyway, with the attached patch you can do > > make dist > > or > > meson compile -C build pgdist
I played this with meson build on macOS, the packages are generated in source root but not build root, I'm sure if this is by design but I think polluting *working directory* is not good. Another thing I'd like to point out is, should we also introduce *git commit* or maybe *git tag* to package name, something like: git archive --format tar.gz --prefix postgresql-17devel/ HEAD -o postgresql-17devel-`git rev-parse --short HEAD`.tar.gz git archive --format tar.gz --prefix postgresql-17devel/ HEAD -o postgresql-`git describe --tags`.tar.gz > > and it produces the same set of tarballs as before, except it's done > differently. > > The actual build scripts need some fine-tuning, but the general idea is > correct, I think. I think this is a good idea, thanks for working on this. -- Regards Junwang Zhao
