Okay I have flit working <https://github.com/apache/hamilton/pull/1387> - it does not create the `egg`, but does create PKG-INFO (which is allowed). Once merged I will create RC1 for 1.89.0, and then assuming that goes well document the process.
On Thu, Sep 18, 2025 at 1:05 PM Stefan Krawczyk <[email protected]> wrote: > oh cool -- will look at flit. > > On Thu, Sep 18, 2025 at 8:53 AM Jarek Potiuk <[email protected]> wrote: > >> The .egg-info folders are a "side artifact" of building packages - I >> think >> they are side effect of using setuptools by your pyproject.toml >> >> [build-system] >> requires = ["setuptools>=72.1"] >> build-backend = "setuptools.build_meta" >> >> So they can be entirely ignored and not part of the produced archives. >> This >> is where `git archive` is very handy - when paired with .gitignore >> excluding those files because it only archives whatever is actually stored >> in git. If you have proper .gitignore, it will ignore all the artifacts >> that are accidentally generated and should never be committed to the repo. >> Git Archive does exactly this - it only adds to the archive files that are >> **actually** part of the repo and committed there if proper .gitignore is >> used. >> >> BTW. My suggestion is also to explore other build backends - specifically, >> if you have no special customisations, python-only package I strongly >> recommend `flit` : https://flit.pypa.io/en/stable/. It is maintained by >> the >> official Python Packaging group (same that manages pip, pipx, hatch and >> few >> other tools) and it's a kind of minimal implementation of the build >> backend. It has 0 dependencies - it's a fully standalone package, very >> small and modern, and follows quickly all the changes that Python >> Packaging >> team implements (new metadata versions etc.). When you use flit rather >> than >> setuptools backend, the .egg-info artifacts are also not generated, they >> are - I believe setuptools historically produces those. Setuptools also >> has >> some "stability" issues - over the last few years new releases of >> setuptools caused problems (mostly because they are pretty much forced to >> keep backwards compatibility and support invalid metadata because >> otherwise >> they would break many existing - badly configured - packages in PyPI. Flit >> is much more modern and better maintained and follows all the metadata >> versioning and is generally a lot faster as well. >> >> J. >> >> >> J. >> >> On Wed, Sep 17, 2025 at 9:13 PM Stefan Krawczyk < >> [email protected]> >> wrote: >> >> > So is it okay to leave the .egg-info in? Or? >> > >> > On Mon, Sep 15, 2025, 6:55 AM Jarek Potiuk <[email protected]> wrote: >> > >> > > The .egg-info files are added to .gitignore of ours and we are using >> > > generally speaking `git archive` to build our archives - it will skip >> > > .gitignored files >> > > >> > > On Mon, Sep 15, 2025 at 8:50 AM Jarek Potiuk <[email protected]> >> wrote: >> > > >> > > > Generated files do not need to include licence information. If you >> are >> > > > using RAT to check licences, you can add .rat-excludes file to >> exclude >> > > > those from rat check. See https://creadur.apache.org/rat/. The >> future >> > > > Apache Trusted Release tool will check .rat-excludes at the top >> level >> > of >> > > > your source .tar.gz and will use it automatically. >> > > > >> > > > On Mon, Sep 15, 2025 at 12:21 AM Stefan Krawczyk < >> > > > [email protected]> wrote: >> > > > >> > > >> > >> > > >> > source release which is required must have 'incubating' in the >> > name. >> > > >> >> > > >> ✅ >> > > >> >> > > >> Question: we will likely evolve the other packages differently. So >> I'm >> > > >> planning on including subdirectories for the other packages to >> > > distinguish >> > > >> things. I assume that's okay. I see pekka does that. >> > > >> >> > > >> If you want to also release an sf-hamilton - then you can have 2 >> tar >> > gz >> > > >> > files. >> > > >> >> > > >> ✅ actually will add a third, a wheel -- since airflow-core also >> does >> > > this. >> > > >> >> > > >> The LICENSE in the sf-hamilton file appears wrong. It mentions >> > > >> > >> > contrib/hamilton/contrib/user/skrawcz/customize_embeddings/__init__.py >> > > >> > but I can't find that file in the tar.gz. >> > > >> >> > > >> ✅ Moving to LICENSE in `contrib` package. We'll release that >> package >> > > >> later. >> > > >> >> > > >> I would also recommend that the directory inside the tar.gz omits >> the >> > > >> > rc0 bit. You can't modify the file after the release vote so you >> > won't >> > > >> > be able to remove the rc0 bit when you do release. >> > > >> >> > > >> ✅ corrected >> > > >> >> > > >> >> > > >> ./CONTRIBUTING.md >> > > >> > >> > > >> ✅Removed >> > > >> >> > > >> > ./MANIFEST.in >> > > >> > >> > > >> ✅Updated >> > > >> ./PKG-INFO >> > > >> ✅ this is autogenerated by the build process -- airflow-core does >> not >> > > >> have >> > > >> the header on this file. >> > > >> >> > > >> > ./README.md >> > > >> > >> > > >> ✅Removed >> > > >> >> > > >> > ./pyproject.toml >> > > >> > >> > > >> ✅Updated >> > > >> >> > > >> > ./rat.txt >> > > >> > >> > > >> ❓ I don't have this? >> > > >> >> > > >> > ./setup.cfg >> > > >> > >> > > >> This is autogenerated via the build process (python -m build >> > --sdist). I >> > > >> can post-process to remove it / add a header manually. Not sure >> what >> > > >> airflow-core does because they don't include this file. >> > > >> >> > > >> >> > > >> > ./hamilton/experimental/databackend.py (this is ok because it >> is >> > > >> > mentioned in the LICENSE file) >> > > >> >> > > >> Not touching. >> > > >> >> > > >> ./sf_hamilton.egg-info/PKG-INFO >> > > >> > ./sf_hamilton.egg-info/SOURCES.txt >> > > >> > ./sf_hamilton.egg-info/dependency_links.txt >> > > >> > ./sf_hamilton.egg-info/entry_points.txt >> > > >> > ./sf_hamilton.egg-info/requires.txt >> > > >> > ./sf_hamilton.egg-info/top_level.txt >> > > >> >> > > >> ❓ These are all generated as part of the source distribution >> process >> > > >> (python -m build --sdist). It's metadata that some python tools use >> > IIUC >> > > >> (but, Airflow core does not have this). Not sure whether they do >> some >> > > post >> > > >> processing to remove it. *I'm guessing that we should too?* >> > > >> >> > > >> Will put up RC1 in a few days. >> > > >> >> > > >> Cheers, >> > > >> >> > > >> Stefan >> > > >> >> > > >> On Sun, Sep 14, 2025 at 10:38 AM PJ Fanning <[email protected]> >> > > wrote: >> > > >> >> > > >> > These files are missing Apache License headers. >> > > >> > >> > > >> > ./CONTRIBUTING.md >> > > >> > ./MANIFEST.in >> > > >> > ./PKG-INFO >> > > >> > ./README.md >> > > >> > ./pyproject.toml >> > > >> > ./rat.txt >> > > >> > ./setup.cfg >> > > >> > ./hamilton/experimental/databackend.py (this is ok because it >> is >> > > >> > mentioned in the LICENSE file) >> > > >> > ./sf_hamilton.egg-info/PKG-INFO >> > > >> > ./sf_hamilton.egg-info/SOURCES.txt >> > > >> > ./sf_hamilton.egg-info/dependency_links.txt >> > > >> > ./sf_hamilton.egg-info/entry_points.txt >> > > >> > ./sf_hamilton.egg-info/requires.txt >> > > >> > ./sf_hamilton.egg-info/top_level.txt >> > > >> > >> > > >> > On Sun, 14 Sept 2025 at 18:35, PJ Fanning <[email protected]> >> > > wrote: >> > > >> > > >> > > >> > > The LICENSE in the sf-hamilton file appears wrong. It mentions >> > > >> > > >> > > contrib/hamilton/contrib/user/skrawcz/customize_embeddings/__init__.py >> > > >> > > but I can't find that file in the tar.gz. >> > > >> > > >> > > >> > > I would also recommend that the directory inside the tar.gz >> omits >> > > the >> > > >> > > rc0 bit. You can't modify the file after the release vote so >> you >> > > won't >> > > >> > > be able to remove the rc0 bit when you do release. >> > > >> > > >> > > >> > > The next vote should be for rc1. And if that fails, rc2, etc. >> > > >> > > I would recommend that the next release starts with rc1. This >> is >> > > more >> > > >> > > usual for ASF release votes. >> > > >> > > >> > > >> > > On Sun, 14 Sept 2025 at 18:28, PJ Fanning < >> [email protected]> >> > > >> wrote: >> > > >> > > > >> > > >> > > > The rule about incubating in the name is at: >> > > >> > > > https://incubator.apache.org/guides/releasemanagement.html >> > > >> > > > >> > > >> > > > <snip> >> > > >> > > > Here is a minimal set of requirements, when using the work in >> > > >> progress >> > > >> > > > disclaimer, a podlings release must abide by: >> > > >> > > > >> > > >> > > > Include the word incubating in the release file name. >> > > >> > > > >> > > >> > > > Include an ASF LICENSE and NOTICE file. >> > > >> > > > >> > > >> > > > Have valid checksums or signatures. >> > > >> > > > >> > > >> > > > Be placed in the correct place on the ASF’s infrastructure. >> > > >> > > > >> > > >> > > > Have a KEYS file to validate the release. >> > > >> > > > >> > > >> > > > Other issues, such as: >> > > >> > > > >> > > >> > > > Missing ASF headers. >> > > >> > > > >> > > >> > > > Missing license information. >> > > >> > > > >> > > >> > > > Included unexpected binary code. >> > > >> > > > >> > > >> > > > Including code of unknown origin. >> > > >> > > > </snip> >> > > >> > > > >> > > >> > > > On Sun, 14 Sept 2025 at 18:25, PJ Fanning < >> [email protected] >> > > >> > > >> > wrote: >> > > >> > > > > >> > > >> > > > > I'm not sure what process is being followed. Many podlings >> > have >> > > a >> > > >> > > > > document describing the release process that they follow. >> > > >> > > > > I want to highlight one important item for podlings. The >> > release >> > > >> must >> > > >> > > > > be approved by a vote in the podling mailing list. >> Requires at >> > > >> least >> > > >> > > > > 72 hours of voting and a min of 3 +1s from PPMC members. >> But >> > you >> > > >> also >> > > >> > > > > need a 2nd vote after this on the Incubator general mailing >> > list >> > > >> > where >> > > >> > > > > the IPMC members vote separately. Requires at least 72 >> hours >> > of >> > > >> > voting >> > > >> > > > > and a min of 3 +1s from IPMC members. >> > > >> > > > > >> > > >> > > > > On Sun, 14 Sept 2025 at 18:19, PJ Fanning < >> > [email protected] >> > > > >> > > >> > wrote: >> > > >> > > > > > >> > > >> > > > > > The source release which is required must have >> 'incubating' >> > in >> > > >> the >> > > >> > name. >> > > >> > > > > > If you want to also release an sf-hamilton - then you can >> > > have 2 >> > > >> > tar gz files. >> > > >> > > > > > >> > > >> > > > > > Here is an actual vote thread to look at. >> > > >> > > > > > >> > > >> https://lists.apache.org/thread/b14519zt9v2s9gvpg6jo0wzhblo7drhx >> > > >> > > > > > >> > > >> > > > > > It has a description of the voting rules and the checks >> that >> > > are >> > > >> > > > > > required (or suggested) for voters to perform. >> > > >> > > > > > >> > > >> > > > > > On Sun, 14 Sept 2025 at 18:16, PJ Fanning < >> > > [email protected] >> > > >> > >> > > >> > wrote: >> > > >> > > > > > > >> > > >> > > > > > > Votes for releases are usually proceeded by discussion >> > > >> threads. >> > > >> > > > > > > I would suggest that you look at other podling mailing >> > lists >> > > >> to >> > > >> > see how podlings are administered. >> > > >> > > > > > > List of podlings >> > > >> > > > > > > https://incubator.apache.org/clutch/ >> > > >> > > > > > > Mailing list reading >> > > >> > > > > > > https://lists.apache.org/ >> > > >> > > > > > > >> > > >> > > > > > > On 2025/09/14 16:38:18 Stefan Krawczyk wrote: >> > > >> > > > > > > > Okay I think I find one issue. So -1. >> > > >> > > > > > > > >> > > >> > > > > > > > The code literally has RC in version.py. if we just >> > > >> literally >> > > >> > move this >> > > >> > > > > > > > source to releases then this won't fly. I'll need to >> > > change >> > > >> > this. Please >> > > >> > > > > > > > continue to verify. I am assuming there might be one >> or >> > > two >> > > >> > more things to >> > > >> > > > > > > > correct. Will create another RC candidate and vote >> after >> > > >> this >> > > >> > one >> > > >> > > > > > > > concludes. >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > On Sat, Sep 13, 2025, 10:26 PM Stefan Krawczyk < >> > > >> > [email protected]> >> > > >> > > > > > > > wrote: >> > > >> > > > > > > > >> > > >> > > > > > > > > Hi all, >> > > >> > > > > > > > > >> > > >> > > > > > > > > This is a call for a vote on releasing Apache >> hamilton >> > > >> > 1.89.0-incubating, >> > > >> > > > > > > > > release candidate 0. >> > > >> > > > > > > > > >> > > >> > > > > > > > > This release includes the following changes: >> > > >> > > > > > > > > - Too many to list. Will build tooling to help >> > automate >> > > >> this. >> > > >> > > > > > > > > >> > > >> > > > > > > > > The artifacts for this release candidate can be >> found >> > > at: >> > > >> > > > > > > > > >> > > >> > >> > https://dist.apache.org/repos/dist/dev/incubator/hamilton/1.89.0-RC0 >> > > >> > > > > > > > > >> > > >> > > > > > > > > The Git tag to be voted upon is: >> > > >> > > > > > > > > v1.89.0 (based off of this PR >> > > >> > > > > > > > > <https://github.com/apache/hamilton/pull/1378>) >> > > >> > > > > > > > > >> > > >> > > > > > > > > The release hash (can use git tag) will be provided >> > once >> > > >> we >> > > >> > merge a commit >> > > >> > > > > > > > > with version 1.89.0 to main. >> > > >> > > > > > > > > >> > > >> > > > > > > > > Release artifacts are signed with the following >> key: >> > > >> > > > > > > > > >> > > >> > > > > > > > > 25E1C6FA3B71D486DC46BD3630C8F2B2CC329C0B >> > > >> > > > > > > > > >> > > >> > > > > > > > > The KEYS file is available at: >> > > >> > > > > > > > > >> https://downloads.apache.org/incubator/hamilton/KEYS >> > > >> > > > > > > > > >> > > >> > > > > > > > > Please download, verify, and test the release >> > candidate. >> > > >> > > > > > > > > >> > > >> > > > > > > > > The vote will run for a minimum of 72 hours. >> > > >> > > > > > > > > Please vote: >> > > >> > > > > > > > > >> > > >> > > > > > > > > [ ] +1 Release this package as Apache hamilton >> > > >> > 1.89.0-incubating >> > > >> > > > > > > > > [ ] +0 No opinion >> > > >> > > > > > > > > [ ] -1 Do not release this package because... >> (Please >> > > >> > provide a reason) >> > > >> > > > > > > > > >> > > >> > > > > > > > > On behalf of the Apache Hamilton PPMC, >> > > >> > > > > > > > > >> > > >> > > > > > > > > Stefan Krawczyk >> > > >> > > > > > > > > >> > > >> > > > > > > > > P.S. @Mentors with our project, we literally commit >> > the >> > > >> > version into the >> > > >> > > > > > > > > source code. This means that we will build releases >> > off >> > > of >> > > >> > branches and >> > > >> > > > > > > > > then if all good merge into main. Is that kosher? >> or >> > > not? >> > > >> > > > > > > > > >> > > >> > > > > > > > >> > > >> > >> > > >> >> > > > >> > > >> > >> >
