I don't think this is proposing to have links to the checksums/signatures on the CDN, but rather to have the closer.lua template, which is still controlled by INFRA, include links to those files at the recommended location at downloads.apache.org. closer.lua can easily link to $x at dlcdn.apache.org, but $x.sha512 and $x.asc and KEYS at downloads.apache.org.
On Sun, Apr 17, 2022 at 5:46 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > Sebb, > > I think it would be great to see some result analysis on > security/integrity of the proposal. I wonder, did you make any > analysis on the integrity aspect? What's the result of it? > > I believe originally the requirement for separating file downloads and > sha/checksum were to make sure that the checksum/sha files are always > served directly from the fully apache-controlled delivery chain. > Previously, when there were mirrors, this was the only way to make > sure you could validate the provenience of the files - the .sha and > .sum files were always served from "https://download.a.o"; so - unless > your local certificate authority source were compromised, you could be > sure that the downloaded files were not tampered with. It's all too > easy to modify both - the file and their checksum/sha to "pretend it > is valid". > > I understand the main reason why this was done this way was that > directly the Apache-owned and fully controlled servers were able to > serve all those files - because they are very small and can be served > directly by the Apache server under the infra control, while using > mirrors/cdn is absolutely necessary to serve the binary content which > is huge, and we cannot really route all the users to hit directly the > apache download server. This is at least when I tried to understand > the context and reasoning why it was implemented this way. > > I am not sure how dlcdn works and how "trustful" it is for the Apache > to be 100% sure the content has not been tampered with by the CDN > provider. > Do you know what is the consequence of your proposal on the integrity > check in case they are part of close.lua? Does it mean that it will be > served from a "fully owned" apache page ? Is it something that ASF can > "delegate" to the CDN provider? Is it something that closer.lua can > handle in the "provenience sure way" ? Is the content served by > closer.lua fully served by the ASF-owned server and cannot be tampered > with by the CDN provider? > > I do not know the answer to those questions (I do not know too much > about the closer.lua scripts and what's the level of trust between the > ASF and CDN) - but I am sure that when you proposed the changes, you > considered the context and reasoning why it was separated - so can you > tell us all if all the integrity and "provenience" aspects are handled > well by your proposal? > > J. > > On Sun, Apr 17, 2022 at 1:06 PM sebb <seb...@gmail.com> wrote: > > > > On Sun, 17 Apr 2022 at 11:56, sebb <seb...@gmail.com> wrote: > > > > > > On Sun, 17 Apr 2022 at 10:36, Greg Stein <gst...@gmail.com> wrote: > > > > > > > > Hi Craig, > > > > > > > > The closer.lua, its prior name closer.cgi. and all predecessors have > > > > been fine with the Foundation's consumers liking its functionality. > > > > This has supported 200+ projects (or more, depending on how you'd like > > > > to count). Changes to that system are (thus) demand-based, which we > > > > have not seen. Please provide some PRs with the changes you would like > > > > to make. > > > > > > > > I do believe that others agree with your ideas on "automated hashes", > > > > though I really have no idea on the follow-on security details. Nor a > > > > mechanism for broad-based security chains of GPG keys and hashes for > > > > the artifacts. > > > > > > My idea here was just to link to the sig and hash by adding the > > > appropriate suffix to the artifact path name, and checking that it > > > exists. > > > e.g. look for db/jdo/3.2/jdo-3.2-source-release.tar.gz + .asc also > > > .sha256 or .sha512 etc. > > > The code already checks for the presence of the artifact. > > > > > > There aren't that many valid hash types to check; if the code cannot > > > find the required files, it could direct users back to the project. > > > > > > > Figure it out, discuss on the mailing list, file some Pull-Requests, > > > > and we can move forward. > > > > It looks like the code is currently at: > > > > https://github.com/apache/infrastructure-p6/blob/production/modules/closer_cgi/files/closer.lua > > (requires GitHub login linked to ASF login with appropriate karma) > > > > > > Cheers, > > > > Greg > > > > InfraAdmin, ASF > > > > > > > > > > > > On Sat, Apr 16, 2022 at 6:23 PM Craig Russell <apache....@gmail.com> > > > > wrote: > > > >> > > > >> I'd like to ask on behalf of the DB PMC for the infra tool closer.lua > > > >> to be enhanced a bit for use by projects. > > > >> > > > >> It will be much easier for us if we can simply link to e.g. > > > >> https://www.apache.org/dyn/closer.lua/db/jdo/3.2/jdo-3.2-source-release.tar.gz > > > >> and have that page contain the .asc and .sha512 as well. > > > >> > > > >> Even better if closer.lua can also find the relevant KEYS file and > > > >> link to it when describing how to check the release. > > > >> > > > >> With this approach, it is easier for the project and less error-prone > > > >> to maintain the download page(s). We can include the current release > > > >> as well as archived releases on the same download page and omit the > > > >> links to the .asc and .sha512 and KEYS files. > > > >> > > > >> Please let us know how we can help. > > > >> > > > >> Regards, > > > >> Craig > > > >> > > > >> > > > >> On Apr 16, 2022, at 2:01 AM, Greg Stein <gst...@gmail.com> wrote: > > > >> > > > >> On Thu, Apr 14, 2022 at 4:43 PM sebb <seb...@gmail.com> wrote: > > > >>> > > > >>> On Thu, 14 Apr 2022 at 20:52, Christopher <ctubb...@apache.org> wrote: > > > >>> > > > > >>> > Doing this would probably restrict INFRA's ability to adapt it to > > > >>> > their changing needs, as projects would become dependent on it. > > > >>> > > > >>> Projects are already dependent on the page. > > > >> > > > >> > > > >> Correct. We *want* projects to use closer.lua. It provides a control > > > >> point to direct users to the appropriate location to download. Today, > > > >> it primarily sends them to our CDN and to an EU download location. > > > >> > > > >> Should those decisions ever change, we want projects to be using > > > >> closer.lua to effect those changes. > > > >> > > > >>> > > > >>> I think it would make it easier for Infra to make changes, as they > > > >>> would be able to adjust it. > > > >> > > > >> > > > >> Yes. > > > >> > > > >>> > > > >>> > Probably best to leave the mirror-management page independent from > > > >>> > the > > > >>> > project download pages, since they serve the needs of separate > > > >>> > groups. > > > >>> > > > >>> I was not suggesting replacing these, for those projects that want to > > > >>> customise the pages. > > > >> > > > >> > > > >> Today, we no longer have a mirror system. But requiring projects to > > > >> use closer.lua ensures that we can swap in that future option. > > > >> > > > >> All that said, as I recall: closer.lua provides support for .ezt pages > > > >> so that projects can provide custom download pages. Maybe there is a > > > >> way to provide the needed variables to projects' custom download page > > > >> templates. > > > >> > > > >> Or, they can just keep using their pages. > > > >> > > > >> The download pages are owned by the TLPs. Infra isn't gonna interfere > > > >> with them. Should any TLPs want more features, then they can ask. We > > > >> haven't seen any requests in years from the TLPs. > > > >> > > > >> Cheers, > > > >> Greg > > > >> InfraAdmin, ASF > > > >> > > > >> > > > >> > > > >> Craig L Russell > > > >> c...@apache.org > > > >>
