On Sun, 17 Apr 2022 at 23:31, Christopher <ctubb...@apache.org> wrote: > > I don't think this is proposing to have links to the > checksums/signatures on the CDN, but rather to have the closer.lua > template, which is still controlled by INFRA, include links to those > files at the recommended location at downloads.apache.org. closer.lua > can easily link to $x at dlcdn.apache.org, but $x.sha512 and $x.asc > and KEYS at downloads.apache.org.
Yes, the idea is to extend the existing page to create a minimal valid download page for projects to use. The links to KEYS, sigs and hashes would use the same URLs as a regular download page FTR, I have got a working Docker image which I will upload shortly. > On Sun, Apr 17, 2022 at 5:46 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > Sebb, > > > > I think it would be great to see some result analysis on > > security/integrity of the proposal. I wonder, did you make any > > analysis on the integrity aspect? What's the result of it? > > > > I believe originally the requirement for separating file downloads and > > sha/checksum were to make sure that the checksum/sha files are always > > served directly from the fully apache-controlled delivery chain. > > Previously, when there were mirrors, this was the only way to make > > sure you could validate the provenience of the files - the .sha and > > .sum files were always served from "https://download.a.o"; so - unless > > your local certificate authority source were compromised, you could be > > sure that the downloaded files were not tampered with. It's all too > > easy to modify both - the file and their checksum/sha to "pretend it > > is valid". > > > > I understand the main reason why this was done this way was that > > directly the Apache-owned and fully controlled servers were able to > > serve all those files - because they are very small and can be served > > directly by the Apache server under the infra control, while using > > mirrors/cdn is absolutely necessary to serve the binary content which > > is huge, and we cannot really route all the users to hit directly the > > apache download server. This is at least when I tried to understand > > the context and reasoning why it was implemented this way. > > > > I am not sure how dlcdn works and how "trustful" it is for the Apache > > to be 100% sure the content has not been tampered with by the CDN > > provider. > > Do you know what is the consequence of your proposal on the integrity > > check in case they are part of close.lua? Does it mean that it will be > > served from a "fully owned" apache page ? Is it something that ASF can > > "delegate" to the CDN provider? Is it something that closer.lua can > > handle in the "provenience sure way" ? Is the content served by > > closer.lua fully served by the ASF-owned server and cannot be tampered > > with by the CDN provider? > > > > I do not know the answer to those questions (I do not know too much > > about the closer.lua scripts and what's the level of trust between the > > ASF and CDN) - but I am sure that when you proposed the changes, you > > considered the context and reasoning why it was separated - so can you > > tell us all if all the integrity and "provenience" aspects are handled > > well by your proposal? > > > > J. > > > > On Sun, Apr 17, 2022 at 1:06 PM sebb <seb...@gmail.com> wrote: > > > > > > On Sun, 17 Apr 2022 at 11:56, sebb <seb...@gmail.com> wrote: > > > > > > > > On Sun, 17 Apr 2022 at 10:36, Greg Stein <gst...@gmail.com> wrote: > > > > > > > > > > Hi Craig, > > > > > > > > > > The closer.lua, its prior name closer.cgi. and all predecessors have > > > > > been fine with the Foundation's consumers liking its functionality. > > > > > This has supported 200+ projects (or more, depending on how you'd > > > > > like to count). Changes to that system are (thus) demand-based, which > > > > > we have not seen. Please provide some PRs with the changes you would > > > > > like to make. > > > > > > > > > > I do believe that others agree with your ideas on "automated hashes", > > > > > though I really have no idea on the follow-on security details. Nor a > > > > > mechanism for broad-based security chains of GPG keys and hashes for > > > > > the artifacts. > > > > > > > > My idea here was just to link to the sig and hash by adding the > > > > appropriate suffix to the artifact path name, and checking that it > > > > exists. > > > > e.g. look for db/jdo/3.2/jdo-3.2-source-release.tar.gz + .asc also > > > > .sha256 or .sha512 etc. > > > > The code already checks for the presence of the artifact. > > > > > > > > There aren't that many valid hash types to check; if the code cannot > > > > find the required files, it could direct users back to the project. > > > > > > > > > Figure it out, discuss on the mailing list, file some Pull-Requests, > > > > > and we can move forward. > > > > > > It looks like the code is currently at: > > > > > > https://github.com/apache/infrastructure-p6/blob/production/modules/closer_cgi/files/closer.lua > > > (requires GitHub login linked to ASF login with appropriate karma) > > > > > > > > Cheers, > > > > > Greg > > > > > InfraAdmin, ASF > > > > > > > > > > > > > > > On Sat, Apr 16, 2022 at 6:23 PM Craig Russell <apache....@gmail.com> > > > > > wrote: > > > > >> > > > > >> I'd like to ask on behalf of the DB PMC for the infra tool > > > > >> closer.lua to be enhanced a bit for use by projects. > > > > >> > > > > >> It will be much easier for us if we can simply link to e.g. > > > > >> https://www.apache.org/dyn/closer.lua/db/jdo/3.2/jdo-3.2-source-release.tar.gz > > > > >> and have that page contain the .asc and .sha512 as well. > > > > >> > > > > >> Even better if closer.lua can also find the relevant KEYS file and > > > > >> link to it when describing how to check the release. > > > > >> > > > > >> With this approach, it is easier for the project and less > > > > >> error-prone to maintain the download page(s). We can include the > > > > >> current release as well as archived releases on the same download > > > > >> page and omit the links to the .asc and .sha512 and KEYS files. > > > > >> > > > > >> Please let us know how we can help. > > > > >> > > > > >> Regards, > > > > >> Craig > > > > >> > > > > >> > > > > >> On Apr 16, 2022, at 2:01 AM, Greg Stein <gst...@gmail.com> wrote: > > > > >> > > > > >> On Thu, Apr 14, 2022 at 4:43 PM sebb <seb...@gmail.com> wrote: > > > > >>> > > > > >>> On Thu, 14 Apr 2022 at 20:52, Christopher <ctubb...@apache.org> > > > > >>> wrote: > > > > >>> > > > > > >>> > Doing this would probably restrict INFRA's ability to adapt it to > > > > >>> > their changing needs, as projects would become dependent on it. > > > > >>> > > > > >>> Projects are already dependent on the page. > > > > >> > > > > >> > > > > >> Correct. We *want* projects to use closer.lua. It provides a control > > > > >> point to direct users to the appropriate location to download. > > > > >> Today, it primarily sends them to our CDN and to an EU download > > > > >> location. > > > > >> > > > > >> Should those decisions ever change, we want projects to be using > > > > >> closer.lua to effect those changes. > > > > >> > > > > >>> > > > > >>> I think it would make it easier for Infra to make changes, as they > > > > >>> would be able to adjust it. > > > > >> > > > > >> > > > > >> Yes. > > > > >> > > > > >>> > > > > >>> > Probably best to leave the mirror-management page independent > > > > >>> > from the > > > > >>> > project download pages, since they serve the needs of separate > > > > >>> > groups. > > > > >>> > > > > >>> I was not suggesting replacing these, for those projects that want > > > > >>> to > > > > >>> customise the pages. > > > > >> > > > > >> > > > > >> Today, we no longer have a mirror system. But requiring projects to > > > > >> use closer.lua ensures that we can swap in that future option. > > > > >> > > > > >> All that said, as I recall: closer.lua provides support for .ezt > > > > >> pages so that projects can provide custom download pages. Maybe > > > > >> there is a way to provide the needed variables to projects' custom > > > > >> download page templates. > > > > >> > > > > >> Or, they can just keep using their pages. > > > > >> > > > > >> The download pages are owned by the TLPs. Infra isn't gonna > > > > >> interfere with them. Should any TLPs want more features, then they > > > > >> can ask. We haven't seen any requests in years from the TLPs. > > > > >> > > > > >> Cheers, > > > > >> Greg > > > > >> InfraAdmin, ASF > > > > >> > > > > >> > > > > >> > > > > >> Craig L Russell > > > > >> c...@apache.org > > > > >>
