This sounds as a good proposal to me (at least at the moment where we have separate docs and main site). I agree that documentation should indeed stay with the code, as you want to update those together in PRs. But the website is something you can typically update separately and also might want to update independently from code releases. And certainly if this proposal makes it easier to work on the site, all the better.
Joris Op ma 5 aug. 2019 20:30 schreef Wes McKinney <wesmck...@gmail.com>: > Let's wait a little while to collect any additional opinions about this. > > There's pretty good evidence from other Apache projects that this > isn't too bad of an idea > > Apache Calcite: https://github.com/apache/calcite-site > Apache Kafka: https://github.com/apache/kafka-site > Apache Spark: https://github.com/apache/spark-website > > The Apache projects I've seen where the same repository is used for > $FOO.apache.org tend to be ones where the documentation _is_ the > website. I think we would need to commission a significant web design > overhaul to be able to make our documentation page adequate as the > landing point for visitors to https://arrow.apache.org. > > On Sat, Aug 3, 2019 at 3:46 PM Neal Richardson > <neal.p.richard...@gmail.com> wrote: > > > > Given the status quo, it would be difficult for this to make the Arrow > > website less maintained. In fact, arrow-site is currently missing the > > most recent two patches that modified the site directory in > > apache/arrow. Having multiple manual deploy steps increases the > > likelihood that the website stays stale. > > > > As someone who has been working on the arrow site lately, this > > proposal makes it easier for me to make changes to the website because > > I can automatically deploy my changes to a test site, and that lets > > others in the community, who perhaps don't touch the website much, > > verify that they're good. > > > > I agree that the documentation situation needs attention, but as I > > said initially, that's orthogonal to this static site generation. I'd > > like to work on that next, and I think these changes will make it > > easier to do. I would not propose moving doc generation out of > > apache/arrow--that belongs with the code. > > > > Neal > > > > On Sat, Aug 3, 2019 at 9:49 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > > > > I think that the project website and the project documentation are > > > currently distinct entities. The current Jekyll website is independent > > > from the Sphinx documentation project aside from a link to the > > > documentation from the website. > > > > > > I am guessing that we would want to maintain some amount of separation > > > between the main site at arrow.apache.org and the code / format > > > documentation, at minimum because we may want to make documentation > > > available for multiple versions of the project (this has already been > > > cited as an issue -- when we release, we're overwriting the previous > > > version of the docs) > > > > > > On Sat, Aug 3, 2019 at 11:33 AM Antoine Pitrou <anto...@python.org> > wrote: > > > > > > > > > > > > I am concerned with this. What happens if we happen to move part of > the > > > > current site to e.g. the Sphinx docs in the Arrow repository (we > already > > > > did that, so it's not theoretical)? > > > > > > > > More generally, I also think that any move towards separating website > > > > and code repo more will lead to an even less maintained website. > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > > > > Le 02/08/2019 à 22:39, Wes McKinney a écrit : > > > > > hi Neal, > > > > > > > > > > In general the improvements to the site sound good, and I agree > with > > > > > moving the site into the apache/arrow-site repository. > > > > > > > > > > It sounds like a committer will have to volunteer a PAT for the > Travis > > > > > CI settings in > > > > > > > > > > https://travis-ci.org/apache/arrow-site/settings > > > > > > > > > > Even though you can't get at such an environment variable there > after > > > > > it's set, it could still technically be compromised. Personally I > > > > > wouldn't be comfortable having a token with "repo" scope out > there. We > > > > > might need to think about this some more -- the general idea of > making > > > > > it easier to deploy the website I'm totally on board with > > > > > > > > > > - Wes > > > > > > > > > > > > > > > On Fri, Aug 2, 2019 at 1:35 PM Neal Richardson > > > > > <neal.p.richard...@gmail.com> wrote: > > > > >> > > > > >> Hi all, > > > > >> https://issues.apache.org/jira/browse/ARROW-5746 requested to > move the > > > > >> source for https://arrow.apache.org out of `apache/arrow` due to > the > > > > >> growing number of binary files (mostly images) there. > > > > >> > > > > >> https://issues.apache.org/jira/browse/ARROW-4473 requested > > > > >> improvements to the ability to make a test deploy of the website > and > > > > >> noted challenges/bugs in trying to do this when the site > `baseurl` is > > > > >> a subdirectory. > > > > >> > > > > >> On my fork of `arrow-site` [1] I have a solution to both. I > created a > > > > >> `master` branch and copied the contents of the `site/` directory > in > > > > >> `apache/arrow` to that, using `git filter-branch --prune-empty > > > > >> --subdirectory-filter site master` to preserve the commit history > [2]. > > > > >> Then I added a build script [3] that gets executed by Travis-CI > [4]. > > > > >> > > > > >> The script builds the Jekyll site and pushes it to a branch that > gets > > > > >> published. On `apache/arrow-site`, commits to the `master` branch > > > > >> trigger a build of the Jekyll site and push the result to the > > > > >> `asf-site` branch. On forks, commits to `master` build the site > and > > > > >> publish to the `gh-pages` branch, which can deploy to GitHub > Pages. > > > > >> > > > > >> ## Features > > > > >> > > > > >> * Automatic building of the arrow.apache.org site whenever > changes are > > > > >> made to the Jekyll source--no manual build step required. > > > > >> * Automatic building of a test site from your fork, which will > enable > > > > >> reviewers to verify your changes without having to build and serve > > > > >> locally and trust that what works locally will work when deployed. > > > > >> * Relative URL problems are fixed: links work regardless of > whether > > > > >> the "base URL" is top level or a subdirectory. > > > > >> * Reduced size of the core `apache/arrow` repository > > > > >> * Documentation publishing is not affected. Updating the contents > of > > > > >> the `docs/` directory in the published `asf-site` branch can > continue > > > > >> to happen by whatever other process. The automatic building and > > > > >> publishing of the Jekyll site does not overwrite the `docs/` > > > > >> directory. > > > > >> > > > > >> ## Usage > > > > >> > > > > >> Local development and serving of the Jekyll site is not affected > by > > > > >> this build process--it works exactly the same as before, just > located > > > > >> in the `arrow-site` repository instead of the `site/` directory of > > > > >> `apache/arrow`. > > > > >> > > > > >> To enable the automatic building on your fork, there are a couple > of > > > > >> quick setup steps to enable GitHub Pages and Travis-CI, described > here > > > > >> [5]. > > > > >> > > > > >> In order set up the automatic deploy on `apache/arrow-site`, a > > > > >> committer will need to set a GITHUB_PAT there. I imagine there > could > > > > >> be some hesitation to doing this, but it is safe because > > > > >> > > > > >> 1. Builds only happen on the master branch, and only committers > can > > > > >> modify the master branch, so by accepting a patch to `master`, > they're > > > > >> implicitly accepting a patch to `asf-site` > > > > >> 2. Malicious actors can't modify the build script in a pull > request > > > > >> and use the token because Travis does "not provide > [repository-setting > > > > >> environment variables] to untrusted builds, triggered by pull > requests > > > > >> from another repository" [6] > > > > >> 3. Non-committers cannot access the Travis-CI settings to alter > the > > > > >> GITHUB_PAT (and even committers cannot view the value of the token > > > > >> once it is set) > > > > >> 4. IIUC there is still a manual action required to get the ASF to > > > > >> update arrow.apache.org with the contents of the `asf-site` > branch > > > > >> > > > > >> While it would be useful, it is not required that we enable > automatic > > > > >> deploy on `apache/arrow-site` in order to get benefit from this > > > > >> proposal because this enables contributors to opt-in to deploying > test > > > > >> sites from their forks, and those tests sites will actually work. > > > > >> > > > > >> Let me know if you have any questions or concerns. If there are no > > > > >> objections, then to proceed I'll need a committer to create an > orphan > > > > >> `master` branch on `apache/arrow-site`, and then I can make a pull > > > > >> request to that, which we'd want to merge without squashing in > order > > > > >> to preserve the git history of the site from `apache/arrow`. > > > > >> > > > > >> Thanks, > > > > >> Neal > > > > >> > > > > >> [1] https://github.com/nealrichardson/arrow-site/ > > > > >> [2] https://github.com/nealrichardson/arrow-site/commits/master > > > > >> [3] > https://github.com/nealrichardson/arrow-site/blob/master/build-and-deploy.sh > > > > >> [4] > https://github.com/nealrichardson/arrow-site/blob/master/.travis.yml > > > > >> [5] > https://github.com/nealrichardson/arrow-site/tree/master#previewing-the-site > > > > >> [6] > https://docs.travis-ci.com/user/environment-variables/#defining-variables-in-repository-settings >
