Yep - to clarify, what constitutes a source release? We don't compile anything for python for our needs. We literally just package up the source code and ship it; that is, what gets published to pypi is the subset of python code from the repository that constitutes that "package".
The only "package" that does have something "compiled" is the sf-hamilton-ui one that has javascript in it, and therefore also the docker container. So I was suggesting we focus on `sf-hamilton` first, before tackling this one -- no need to publish all at once, they are largely decoupled from one another. On Tue, Jun 17, 2025 at 12:57 PM PJ Fanning <[email protected]> wrote: > We need to concentrate on the source release. We can't release > convenience binaries until we have source releases. The source > releases need to be in a very specific format. Have the right format > nams. Have ASF source headers in all the source files. Lots more. The > Incubator team is very strict about this. > > On Tue, 17 Jun 2025 at 20:53, Stefan Krawczyk <[email protected]> > wrote: > > > > Awesome. Yeah I can now go ahead and now update all the link references > > in the docs. Should have that done sometime this week. > > > > We can then after this go through validating / verifying *just the* > > `sf-hamilton` package for our first apache release - I think, right? > > > > On Tue, Jun 17, 2025 at 12:42 PM PJ Fanning <[email protected]> wrote: > > > > > Just as I sent the last mail, it appears that the staged site is > > > working now too. Just a delay in the deployment, I guess. > > > > > > On Tue, 17 Jun 2025 at 20:40, PJ Fanning <[email protected]> wrote: > > > > > > > > https://hamilton.staged.apache.org/ is not working yet but maybe we > > > > can concentrate on https://hamilton.apache.org/ (asf-site branch). > We > > > > can get back to Infra team if we need the staged site and we can't > get > > > > it working ourselves. I don't want to monopolise their time but we > can > > > > raise another issue in the coming days if needs be. > > > > > > > > On Tue, 17 Jun 2025 at 20:33, Stefan Krawczyk < > [email protected]> > > > wrote: > > > > > > > > > > awesome thanks! > > > > > > > > > > On Tue, Jun 17, 2025 at 12:33 PM PJ Fanning <[email protected]> > > > wrote: > > > > > > > > > > > Already answered by Daniel Gruno. > > > > > > I have https://hamilton.apache.org/ working now. > > > > > > > > > > > > On Tue, 17 Jun 2025 at 20:29, PJ Fanning <[email protected]> > > > wrote: > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/INFRA-26920 is the JIRA > > > > > > > > > > > > > > On Tue, 17 Jun 2025 at 20:27, Stefan Krawczyk < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > 👍 sounds good. > > > > > > > > > > > > > > > > On Tue, Jun 17, 2025 at 12:25 PM PJ Fanning < > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks for merging my .asf.yaml. > > > > > > > > > > > > > > > > > > Still no luck though. I'll raise an ASF Infra JIRA to see > if > > > that > > > > > > team > > > > > > > > > can help us debug what is happening. > > > > > > > > > > > > > > > > > > On Tue, 17 Jun 2025 at 19:35, Stefan Krawczyk < > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Okay I rebased -- should > https://hamilton.staged.apache.org/ > > > be > > > > > > being > > > > > > > > > > populated now as I have `asf-staging > > > > > > > > > > <https://github.com/apache/hamilton/tree/asf-staging>` > with > > > all > > > > > > the HTML > > > > > > > > > > under content? or? > > > > > > > > > > > > > > > > > > > > On Mon, Jun 16, 2025 at 11:13 PM Stefan Krawczyk < > > > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Cool I created this PR for that piece then > > > > > > > > > > > <https://github.com/apache/hamilton/pull/1344>. > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 16, 2025 at 2:29 AM PJ Fanning < > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > >> The .asf.yaml will need to be merged but we can use a > > > different > > > > > > PR for > > > > > > > > > > >> this. > > > > > > > > > > >> > > > > > > > > > > >> On Mon, 16 Jun 2025 at 04:48, Stefan Krawczyk < > > > > > > > > > [email protected]> > > > > > > > > > > >> wrote: > > > > > > > > > > >> > > > > > > > > > > > >> > Cool. @PJ I have it building < > > > > > > > > > > >> https://github.com/apache/hamilton/pull/1339> > > > > > > > > > > >> > and pushing to the asf-staging branch. I assume we > > > need to > > > > > > merge > > > > > > > > > > >> .asf.yaml > > > > > > > > > > >> > to main before it takes effect? Should we do a > separate > > > PR > > > > > > for that? > > > > > > > > > > >> Then I > > > > > > > > > > >> > can test this PR more thoroughly by having the > staging > > > branch > > > > > > > > > deployed > > > > > > > > > > >> (I > > > > > > > > > > >> > know I will need to change some links)? > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > On Sat, Jun 14, 2025 at 7:42 AM PJ Fanning < > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > >> > > The asf yaml docs are at > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > https://github.com/apache/infrastructure-asfyaml/blob/main/README.md#deploy > > > > > > > > > > >> > > > > > > > > > > > > >> > > We probably need a section like > > > > > > > > > > >> > > > > > > > > > > > > >> > > publish: > > > > > > > > > > >> > > whoami: asf-site > > > > > > > > > > >> > > subdir: content > > > > > > > > > > >> > > > > > > > > > > > > >> > > where asf-site is the branch and content is the > dir > > > in that > > > > > > > > > branch. > > > > > > > > > > >> > > These 2 values are good ones as they are the most > > > common > > > > > > ones > > > > > > > > > used in > > > > > > > > > > >> > > other ASF projects. > > > > > > > > > > >> > > These settings should mean that > hamilton.apache.org > > > is > > > > > > redeployed > > > > > > > > > > >> > > every time the asf-site branch is updated. > > > > > > > > > > >> > > > > > > > > > > > > >> > > If you read the asf yaml docs, there is also the > > > concept of > > > > > > a > > > > > > > > > staging > > > > > > > > > > >> > > web site - a preview one. > > > > > > > > > > >> > > > > > > > > > > > > >> > > staging: > > > > > > > > > > >> > > whoami: asf-staging > > > > > > > > > > >> > > subdir: content > > > > > > > > > > >> > > > > > > > > > > > > >> > > asf-staging is the most common branch used for > this. > > > > > > > > > > >> > > These settings should mean that > > > hamilton.staged.apache.org > > > > > > is > > > > > > > > > > >> > > redeployed every time the asf-staging branch is > > > updated. > > > > > > > > > > >> > > We don't necessarily need to use this staging web > > > site but > > > > > > the > > > > > > > > > support > > > > > > > > > > >> > > for it is there. It can be a nice way to try out > > > changes > > > > > > without > > > > > > > > > > >> > > affecting the main site. > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > On Sat, 14 Jun 2025 at 15:30, Stefan Krawczyk < > > > > > > > > > > >> [email protected]> > > > > > > > > > > >> > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > okay yeah I have a githhub workflow (see this PR > > > > > > > > > > >> > > > <https://github.com/apache/hamilton/pull/1339>) > > > that > > > > > > builds the > > > > > > > > > > >> docs. I > > > > > > > > > > >> > > can > > > > > > > > > > >> > > > push it to a different branch other than main > > > placing it > > > > > > under > > > > > > > > > > >> /content > > > > > > > > > > >> > > > (currently it publishes the artifact to > github). I > > > think > > > > > > I saw > > > > > > > > > the > > > > > > > > > > >> > > > .asf.yaml support that option. That should work > > > right? > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > On Sat, Jun 14, 2025 at 7:23 AM PJ Fanning < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > >> wrote: > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > If the website code can't be readily removed > from > > > > > > > > > apache/hamilton > > > > > > > > > > >> repo > > > > > > > > > > >> > > > > - then we can just leave it as is. We can set > up > > > > > > .asf.yaml in > > > > > > > > > the > > > > > > > > > > >> > > > > apache/hamilton repo to publish the website. > > > > > > > > > > >> > > > > I think the default is to have the website > static > > > > > > content in > > > > > > > > > the > > > > > > > > > > >> > > > > 'content' directory. > > > > > > > > > > >> > > > > Stefan - can you build the static content for > the > > > > > > website and > > > > > > > > > put > > > > > > > > > > >> it > > > > > > > > > > >> > > > > in the 'content' directory. > > > > > > > > > > >> > > > > I can then try to set up .asf.yaml to deploy > it. > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > On Sat, 14 Jun 2025 at 15:09, Stefan Krawczyk > < > > > > > > > > > > >> > > [email protected]> > > > > > > > > > > >> > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > I can see a separate hamilton-site repo that > > > contains > > > > > > the > > > > > > > > > > >> landing > > > > > > > > > > >> > > page, > > > > > > > > > > >> > > > > but > > > > > > > > > > >> > > > > > what about documentation that relies on > source > > > code > > > > > > (which > > > > > > > > > is > > > > > > > > > > >> what > > > > > > > > > > >> > > > > > hamilton.dagworks.io currently is)? We'd > still > > > need a > > > > > > > > > process > > > > > > > > > > >> to > > > > > > > > > > >> > > > > generate > > > > > > > > > > >> > > > > > that and push it somewhere to be published? > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > On Sat, Jun 14, 2025 at 2:43 AM PJ Fanning < > > > > > > > > > > >> [email protected]> > > > > > > > > > > >> > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > It would be my preference to separate out > the > > > > > > website docs > > > > > > > > > > >> from > > > > > > > > > > >> > > > > > > https://github.com/apache/hamilton/ and > put > > > them > > > > > > in a > > > > > > > > > > >> separate > > > > > > > > > > >> > > > > > > https://github.com/apache/hamilton-site/ > > > repo. > > > > > > > > > > >> > > > > > > It simplifies the release of Apache > Hamilton > > > if we > > > > > > don't > > > > > > > > > need > > > > > > > > > > >> to > > > > > > > > > > >> > > worry > > > > > > > > > > >> > > > > > > about reviewers having to check the source > > > headers > > > > > > and > > > > > > > > > > >> licensing of > > > > > > > > > > >> > > > > > > everything needed for the website build > too. > > > > > > > > > > >> > > > > > > The simplest initial set up is to have > > > > > > > > > apache/hamilton-site > > > > > > > > > > >> git > > > > > > > > > > >> > > repo > > > > > > > > > > >> > > > > > > set up so that a Hamilton team member can > > > checkout > > > > > > the > > > > > > > > > > >> > > > > > > apache/hamilton-site git repo and run the > > > build on > > > > > > their > > > > > > > > > own > > > > > > > > > > >> > > machine. > > > > > > > > > > >> > > > > > > The HTML etc for the website gets > generated > > > to a > > > > > > directory > > > > > > > > > > >> like > > > > > > > > > > >> > > > > > > `content` or `publish`. The Hamilton team > > > member > > > > > > can then > > > > > > > > > > >> commit > > > > > > > > > > >> > > the > > > > > > > > > > >> > > > > > > content or publish directory into git. We > can > > > > > > configure > > > > > > > > > the > > > > > > > > > > >> > > .asf.yaml > > > > > > > > > > >> > > > > > > file so that the website is redeployed > based > > > on git > > > > > > > > > commits > > > > > > > > > > >> that > > > > > > > > > > >> > > > > > > include changes to the `content` or > `publish` > > > dir. > > > > > > > > > > >> > > > > > > We can look into trying to automate this > > > later. It > > > > > > will > > > > > > > > > need > > > > > > > > > > >> time > > > > > > > > > > >> > > to > > > > > > > > > > >> > > > > > > interact with ASF Infra team to be > allowed to > > > have > > > > > > > > > automated > > > > > > > > > > >> jobs > > > > > > > > > > >> > > > > > > commit to the git repo. > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > On Sat, 14 Jun 2025 at 07:24, Stefan > Krawczyk > > > < > > > > > > > > > > >> > > > > [email protected]> > > > > > > > > > > >> > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > Okay just so I mentally I understand. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > 1. We can have documentation source live > > > under > > > > > > /docs > > > > > > > > > > >> > > > > > > > 2. When a PR that changes docs is > > > created/merged > > > > > > we can > > > > > > > > > > >> kick off > > > > > > > > > > >> > > a > > > > > > > > > > >> > > > > github > > > > > > > > > > >> > > > > > > > workflow that builds these docs > > > > > > > > > > >> > > > > > > > 3. Is it correct that we could then push > > > this > > > > > > built > > > > > > > > > HTML to > > > > > > > > > > >> this > > > > > > > > > > >> > > > > other > > > > > > > > > > >> > > > > > > > hamilton-site repo? Is that correct? > > > > > > > > > > >> > > > > > > > 4. Then using the `.asf.yaml` in the > > > hamilton-site > > > > > > > > > repo, we > > > > > > > > > > >> > > could use > > > > > > > > > > >> > > > > > > > the .asf.yaml > > > > > > > > > > >> > > > > > > > directives > > > > > > > > > > >> > > > > > > > < > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/INFRA/git+-+.asf.yaml+features#Git.asf.yamlfeatures-WebsitedeploymentserviceforGitrepositories > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > to then publish which would update > > > > > > hamilton.apache.org. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > Is that right? > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > Cheers, > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > Stefan > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > On Mon, Jun 9, 2025 at 5:02 AM PJ > Fanning < > > > > > > > > > > >> [email protected]> > > > > > > > > > > >> > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> > > > > > > > > Redirecting to dev mailing list. > Private > > > > > > mailing list > > > > > > > > > is > > > > > > > > > > >> for 2 > > > > > > > > > > >> > > > > things > > > > > > > > > > >> > > > > > > > > - discussing and voting on new > committers > > > and > > > > > > > > > discussing > > > > > > > > > > >> > > security > > > > > > > > > > >> > > > > > > > > issues. Everything else is meant to be > > > > > > discussed in > > > > > > > > > > >> public. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > The main doc for web site publishing > is: > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > https://github.com/apache/infrastructure-asfyaml/blob/main/README.md > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > We already have a DNS entry set up for > > > > > > > > > > >> hamilton.apache.org. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > Most ASF projects push the static > content > > > for > > > > > > their > > > > > > > > > web > > > > > > > > > > >> site > > > > > > > > > > >> > > to a > > > > > > > > > > >> > > > > > > > > directory in a git repo. We can > create an > > > > > > > > > > >> apache/hamilton-site > > > > > > > > > > >> > > repo > > > > > > > > > > >> > > > > > > > > for this. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > There are other approaches that might > be > > > > > > feasible. One > > > > > > > > > > >> example > > > > > > > > > > >> > > is > > > > > > > > > > >> > > > > > > > > pekko.apache.org content - most of > it is > > > > > > rsynced to > > > > > > > > > > >> > > > > > > > > nightlies.apache.org and we use > > > .htaccess files > > > > > > > > > deployed > > > > > > > > > > >> to > > > > > > > > > > >> > > > > > > > > pekko.apache.org to allow the > > > > > > nightlies.apache.org > > > > > > > > > > >> content to > > > > > > > > > > >> > > be > > > > > > > > > > >> > > > > > > > > accessed as if it was deployed > directly to > > > > > > > > > > >> pekko.apache.org. > > > > > > > > > > >> > > This > > > > > > > > > > >> > > > > > > > > avoids having to git commit the > generated > > > > > > content > > > > > > > > > (only > > > > > > > > > > >> the > > > > > > > > > > >> > > > > markdown > > > > > > > > > > >> > > > > > > > > files from which the HTML is generated > > > are in > > > > > > git). > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > On Mon, 9 Jun 2025 at 04:57, Stefan > > > Krawczyk < > > > > > > > > > > >> > > > > [email protected]> > > > > > > > > > > >> > > > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > Hi Mentors, > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > I assume hamilton.apache.org is the > > > front > > > > > > page for > > > > > > > > > the > > > > > > > > > > >> > > project? > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > I think it would currently make > sense > > > for the > > > > > > > > > current > > > > > > > > > > >> docs > > > > > > > > > > >> > > page > > > > > > > > > > >> > > > > to be > > > > > > > > > > >> > > > > > > > > under that domain. > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > So to enable that, what do we need > to > > > do? > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > 1. The current docs are hosted on > > > > > > readthedocs.org > > > > > > > > > -- > > > > > > > > > > >> can we > > > > > > > > > > >> > > > > > > continue to > > > > > > > > > > >> > > > > > > > > use that? or? > > > > > > > > > > >> > > > > > > > > > 2. If so, then we can add a new > domain - > > > > > > > > > > >> hamilton.apache.org > > > > > > > > > > >> > > > > which > > > > > > > > > > >> > > > > > > > > would require a CNAME target to be > added > > > .. > > > > > > > > > > >> > > > > > > > > > 3. If not, the current docs are > sphinx > > > docs, > > > > > > how > > > > > > > > > could > > > > > > > > > > >> we > > > > > > > > > > >> > > migrate > > > > > > > > > > >> > > > > > > them > > > > > > > > > > >> > > > > > > > > to the apache approved place? > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > Cheers, > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > >> > > > > > > > > > Stefan > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
