r1514680 Does that, coupled with the knowledge that our source release bundles are *supposed to* include the tag details that they were built from resolve your issue?
(Obviously there is more tooling we can add... for instance I suspect that for GIT we are not including the git hash that release:perform is running from... which is something I think we can address) On 16 August 2013 13:17, sebb <seb...@gmail.com> wrote: > On 16 August 2013 13:08, Fred Cooke <fred.co...@gmail.com> wrote: > > They're deployed as a set, so what I want is the SHA1 or even MD5 of any > > one of the set of uploaded files, such that I can confirm that the set is > > the set that I am supposed to be looking at. I don't see importance in > > which, but I've not thought about it much. I think *all* would be huge > > overkill. > > It's only really needed for source archives, which are the ones being > officially voted on. > > This is something I've long thought is necessary to be able to tie the > vote mail to the artifacts. > > And it could be very useful to have the hash in the mail archive. > > > On Sat, Aug 17, 2013 at 12:00 AM, Stephen Connolly < > > stephen.alan.conno...@gmail.com> wrote: > > > >> That sounds like you are looking for the SHA1 sum of the source bundle > to > >> be included in the vote email. Which would seem perfectly reasonable to > me. > > Yes please! > > >> > >> > >> > >> > >> On 16 August 2013 12:31, Fred Cooke <fred.co...@gmail.com> wrote: > >> > >> > Dennis, of course source bundles will contain URLs and hashes and > >> revisions > >> > and so forth, and the chance of those being mismatched is > approximately > >> > zero. That's not the point. > >> > > >> > The point (for me, at least) is what did you INTEND to release, and > does > >> > THAT match what is actually found in the bundle (including URLs and > >> hashes > >> > etc matching). > >> > > >> > Releasing is a fundamentally human process that consists of "is this > >> > ready?" and "pull trigger". Some binaries and bundles end up on > server of > >> > some type somewhere. I want to know the checksum of one of the set of > >> items > >> > (so I KNOW (not guess) that I'm looking at what you want me to), and I > >> want > >> > to know what SCM or tarball+patchset you think you released it from. > This > >> > is human information that can't be automated. The bundle someone goes > and > >> > finds could have anything in it, and they won't know if it's what you > >> > wanted it to have in it, or not. > >> > > >> > Fred. > >> > > >> > On Fri, Aug 16, 2013 at 11:25 PM, Dennis Lundberg <denn...@apache.org > >> > >wrote: > >> > > >> > > On Fri, Aug 16, 2013 at 11:24 AM, sebb <seb...@gmail.com> wrote: > >> > > > >> > > > On 16 August 2013 09:32, Dennis Lundberg <denn...@apache.org> > wrote: > >> > > > > On Fri, Aug 16, 2013 at 9:52 AM, sebb <seb...@gmail.com> wrote: > >> > > > > > >> > > > >> On 16 August 2013 08:10, Dennis Lundberg <denn...@apache.org> > >> > wrote: > >> > > > >> > On Fri, Aug 16, 2013 at 1:20 AM, sebb <seb...@gmail.com> > wrote: > >> > > > >> > > >> > > > >> >> On 15 August 2013 20:57, Dennis Lundberg < > denn...@apache.org> > >> > > wrote: > >> > > > >> >> > On Thu, Aug 15, 2013 at 9:27 PM, sebb <seb...@gmail.com> > >> > wrote: > >> > > > >> >> > > >> > > > >> >> >> On 15 August 2013 14:16, Jason van Zyl <ja...@tesla.io> > >> > wrote: > >> > > > >> >> >> > What Sebb is doing is perfectly reasonable. > >> > > > >> >> >> > >> > > > >> >> > > >> > > > >> >> > I agree. Checking that the source bundle is correct is > good > >> > > release > >> > > > >> >> review > >> > > > >> >> > practice. > >> > > > >> >> > > >> > > > >> >> > Thank you! > >> > > > >> >> >> > >> > > > >> >> >> > He's trying to assert that everything in the source > ball > >> > > > actually > >> > > > >> >> comes > >> > > > >> >> >> from source control and that no errant files have made > their > >> > way > >> > > > into > >> > > > >> >> the > >> > > > >> >> >> distribution. Right now we cannot assert that the > assembly > >> > > plugin > >> > > > has > >> > > > >> >> not > >> > > > >> >> >> wandered outside the checkout and pulled something else > in, > >> or > >> > > > that > >> > > > >> >> someone > >> > > > >> >> >> didn't accidentally put something else in the > distribution. > >> I > >> > > > think > >> > > > >> this > >> > > > >> >> >> unlikely but we can't assert otherwise right now which I > >> > believe > >> > > > is > >> > > > >> >> Sebb's > >> > > > >> >> >> point. > >> > > > >> >> >> > >> > > > >> >> >> It *has* already happened several times that I am aware > of. > >> > > > >> >> >> > >> > > > >> >> >> The last few releases of the War plugin (various RMs & > >> voters) > >> > > > >> >> >> included at least one spurious file. > >> > > > >> >> >> So it was not just a one-off packaging - and review - > >> failure. > >> > > > >> >> >> [See separate thread(s) for all the details; they are not > >> > > germane > >> > > > >> here.] > >> > > > >> >> >> > >> > > > >> >> >> The message is that mistakes happen even in automated > >> > processes. > >> > > > >> >> >> Which is why independent comparison of input and output > is > >> > > > valuable. > >> > > > >> >> >> > >> > > > >> >> >> > If we can embed the revision from which the assembly > was > >> > made > >> > > in > >> > > > >> the > >> > > > >> >> >> assembly itself (and maybe the build number plugin is > doing > >> > this > >> > > > >> >> already), > >> > > > >> >> >> then a tool can be made to unpack the assembly, checkout > the > >> > > > revision > >> > > > >> >> and > >> > > > >> >> >> assert that everything in the source distribution comes > from > >> > > > source > >> > > > >> >> control. > >> > > > >> >> >> > > >> > > > >> >> >> > If we can also assert that as part of each build all > the > >> > > license > >> > > > >> files > >> > > > >> >> >> are intact and headers are in place then I believe we're > >> done > >> > > with > >> > > > >> >> >> provenance. > >> > > > >> >> >> > Licenses are present, all files have valid license > >> headers, > >> > > all > >> > > > >> files > >> > > > >> >> >> present in the source distribution come from source > control, > >> > all > >> > > > >> >> >> contributions to source control are from bonafide CLA > >> carrying > >> > > > Apache > >> > > > >> >> >> committers because you don't get access to commit until > the > >> > CLA > >> > > > is on > >> > > > >> >> file. > >> > > > >> >> >> > > >> > > > >> >> >> > Sebb, reasonably accurate? > >> > > > >> >> >> > >> > > > >> >> >> Yes. > >> > > > >> >> >> > >> > > > >> >> >> One other point I made already is that I think the vote > >> e-mail > >> > > > needs > >> > > > >> >> >> to be transparent to all, not just those on the PMC. > >> > > > >> >> >> Links to the output from the release process are > obviously > >> > > > already in > >> > > > >> >> >> the mail; what is missing is the input to the process, > e.g.. > >> > SCM > >> > > > >> >> >> coords. > >> > > > >> >> >> Yes, it may be possible to dig out the details from the > >> > archive, > >> > > > but > >> > > > >> >> >> that's not trivial. > >> > > > >> >> >> > >> > > > >> >> > > >> > > > >> >> > I disagree. > >> > > > >> >> > > >> > > > >> >> > If we focus first on a "normal" release, one that > succeeds on > >> > the > >> > > > >> first > >> > > > >> >> > attempt, without a respin or deleting of tags. > >> > > > >> >> > To check provenance you would do this: > >> > > > >> >> > > >> > > > >> >> > 1. download the source bundle > >> > > > >> >> > 2. unpack the source bundle > >> > > > >> >> > 3. checkout the corresponding source code from the SCM > >> > > > >> >> > 4. compare the two trees > >> > > > >> >> > > >> > > > >> >> > Right so far? > >> > > > >> >> > > >> > > > >> >> > What you want, if I understand you correctly, is to have > the > >> > SCM > >> > > > URL > >> > > > >> in > >> > > > >> >> the > >> > > > >> >> > vote email. So that you can give that to your SCM client > in > >> > step > >> > > 3. > >> > > > >> >> > >> > > > >> >> Yes. > >> > > > >> >> > >> > > > >> >> > With the process we use here at the Maven project, the SCM > >> URL > >> > is > >> > > > in > >> > > > >> the > >> > > > >> >> > pom.xml file that sits in the root directory of the > unpacked > >> > > source > >> > > > >> >> bundle. > >> > > > >> >> > All you need to do is open the file and copy the URL from > >> > there. > >> > > I > >> > > > >> still > >> > > > >> >> > fail to see how that is so much harder than to copy the > URL > >> > from > >> > > an > >> > > > >> >> email. > >> > > > >> >> > > >> > > > >> >> > That is if you don't know the conventions that we use, by > way > >> > of > >> > > > the > >> > > > >> >> > Release Plugin. The tag will always be in the format > >> > > > >> >> > ${project.artifactId}-${project.version} > >> > > > >> >> > > >> > > > >> >> > >> > > > >> >> My point is that it should be completely transparent, even > to > >> > > outside > >> > > > >> >> reviewers. > >> > > > >> >> > >> > > > >> > > >> > > > >> > I guess that this is the point that we'll have to agree to > >> > disagree > >> > > > on. > >> > > > >> My > >> > > > >> > view is that if someone wants to to review a release from the > >> > Maven > >> > > > >> > project, they'd have to have a basic understanding of how > Maven > >> > > works > >> > > > and > >> > > > >> > how we do releases in the Maven project. That includes what > the > >> > > > Release > >> > > > >> > Plugin is and how it works. > >> > > > >> > >> > > > >> Why does the release tool matter? > >> > > > >> > >> > > > > > >> > > > > It is not the tool itself that matters, but rather what the > tools > >> > does. > >> > > > > Since the Release plugin automates a lot of the tasks involved > when > >> > > > doing a > >> > > > > release, it can appear to make things less transparent. > Therefor a > >> > > > > knowledge about what it does is needed to make it transparent. > >> > > > > >> > > > That really is not the point. > >> > > > > >> > > > It's possible to create the same outputs from the same inputs > using > >> > > > all sorts of methods, manual or automated. > >> > > > > >> > > > The fact that it is a Maven release should not matter as far as > the > >> > > > source checking is concerned, except insofar as the staging area > may > >> > > > be different. > >> > > > > >> > > > Only the source release archives matter. > >> > > > > >> > > > >> > > All of what you say above is true, but that is not what I'm talking > >> > about. > >> > > My focus is still on the 2.5 bullet in the process of *reviewing* a > >> > source > >> > > release, not it's creation. > >> > > > >> > > You want the SCM URL in the vote email. I'm arguing that that piece > of > >> > > information is already available to you in the source bundle, so > there > >> is > >> > > not point in including it in the vote email. Knowing *what* the > Release > >> > > Plugin does makes you aware of this fact. > >> > > > >> > > > >> > > > > >> > > > > It's still an ASF release. > >> > > > >> > >> > > > > > >> > > > > Yes, and we must adhere to all the requirements of an ASF > release. > >> > > There > >> > > > > are however a lot of different ways to make an ASF compliant > >> release. > >> > > > Each > >> > > > > PMC decides on a process for that, that works best for them. > >> > > > > >> > > > That's fine, the tool and process does not matter so long as the > >> > > > output is correct. > >> > > > > >> > > > > It should not matter how the source release artifacts are > >> generated. > >> > > > >> The process of checking them is still the same. > >> > > > >> > >> > > > >> Of course if things go wrong, then it's important to know how > the > >> > SCM > >> > > > >> input was assembled into the source archives in order to fix > the > >> > > > >> process. > >> > > > >> But the release tool is completely irrelevant to the process of > >> > > > >> checking the contents of the source archives. > >> > > > >> > >> > > > > > >> > > > > See above. > >> > > > > > >> > > > > > >> > > > >> > >> > > > >> > Most people have their own checklist of stuff they do when > >> > > reviewing a > >> > > > >> > release. Would it be a good idea if we aggregate all those > >> points > >> > > on a > >> > > > >> page > >> > > > >> > on the Maven web site, under the development section? That > would > >> > > also > >> > > > >> serve > >> > > > >> > as a guide for "outside reviewers". > >> > > > >> > >> > > > >> Yes, though that is a separate issue. > >> > > > >> > >> > > > > > >> > > > > Agreed. > >> > > > > > >> > > > >> > >> > > > >> >> > Now, for a respun release thing are trickier. Here it > might > >> be > >> > a > >> > > > good > >> > > > >> >> idea > >> > > > >> >> > to include the revision number or hash, or whatever is > unique > >> > in > >> > > > the > >> > > > >> SCM > >> > > > >> >> > being used. > >> > > > >> >> > >> > > > >> >> And how do you know from a vote e-mail that it is respun? > >> > > > >> >> > >> > > > >> > > >> > > > >> > It should always say so in the subject of the vote email. Not > >> sure > >> > > if > >> > > > >> that > >> > > > >> > is written down somewhere, but that's how we have always done > >> it. > >> > If > >> > > > it's > >> > > > >> > missing I'll add it to our release process document. > >> > > > >> > > >> > > > >> > > >> > > > >> >> > >> > > > >> >> > Even though the code under review will always be under the > >> > > > >> >> > latest tag in the above format (at least for Subversion). > >> > > > >> >> > >> > > > >> >> Until the next respin. > >> > > > >> >> > >> > > > >> >> If there is a respin, and reviewers are not following the > >> e-mails > >> > > > very > >> > > > >> >> carefully, it would be quite easy to overlook an updated > tag. > >> > > > >> >> > >> > > > >> >> This is all about making sure that it is really obvious what > >> the > >> > > > vote is > >> > > > >> >> about. > >> > > > >> >> > >> > > > >> > > >> > > > >> > Yes, that's why I agreed that it's a good idea to add the > >> > > > revision/hash > >> > > > >> > when doing a respin. > >> > > > >> > > >> > > > >> > > >> > > > >> >> > >> > > > >> >> > > >> > > > >> >> >> Publishing the SCM coordinates in the mail is trivial to > do, > >> > and > >> > > > >> shows > >> > > > >> >> >> that the input is an important part of the review > process. > >> > > > >> >> >> Having the information in the mail thread is also useful > for > >> > the > >> > > > >> >> archives. > >> > > > >> >> >> > >> > > > >> >> > > >> > > > >> >> > As others have said before, we aim to automate the release > >> > > process > >> > > > as > >> > > > >> >> much > >> > > > >> >> > as possible. Therefor even a seemingly minor addition > will be > >> > > > >> questioned, > >> > > > >> >> > as you have noticed, before it is included in our process. > >> > > > >> >> > > >> > > > >> >> > Can you explain why the information is useful for the > >> archives? > >> > > > I've > >> > > > >> seen > >> > > > >> >> > you mentioned that before. Isn't that moot once the > release > >> is > >> > > > >> approved? > >> > > > >> >> > The tag will be in Subversion for the forseable future and > >> > noone > >> > > > will > >> > > > >> >> touch > >> > > > >> >> > it. What am I missing? > >> > > > >> >> > >> > > > >> >> Why would a release need to be revisited? > >> > > > >> >> Perhaps someone is complaining that one of our releases > >> contains > >> > > code > >> > > > >> >> it should not. > >> > > > >> >> If that is the case, it helps to have the evidence of the > >> release > >> > > > vote > >> > > > >> >> in plain sight. > >> > > > >> >> > >> > > > >> > > >> > > > >> > Sure it helps in those cases. But the evidence in our case > is in > >> > the > >> > > > the > >> > > > >> > source bundle itself, which is even better in my opinion. > There > >> > you > >> > > > have > >> > > > >> > the tag in the POM file. > >> > > > >> > >> > > > >> However, the link to the staging repo is temporary. > >> > > > >> Once the repo is published, the staging repo is deleted and > there > >> is > >> > > > >> no direct route to the pom from the vote e-mail. > >> > > > >> > >> > > > > > >> > > > > Once the staging repo is promoted, its content is moved to the > real > >> > > > > repository. So the source bundle is available there for anyone > who > >> > > wants > >> > > > to > >> > > > > track down the complete contents of a previous release. > >> > > > > >> > > > However that is not the canonical location for source releases; > >> source > >> > > > releases MUST be released via the ASF mirror system. > >> > > > > >> > > > >> > > So, go download it from the mirror system then. I fail to see how > that > >> is > >> > > relevant to this discussion. > >> > > > >> > > > >> > > > > Again this is where a bit of knowledge of what the Release > Plugin > >> > does > >> > > > > helps you. A release done with the Release Plugin will always be > >> > > > available > >> > > > > at > >> > > > > > >> > > > > >> > > > >> > > >> > https://repository.apache.org/content/repositories/releases/${project.groupId}/${project.artifactId}/${project.version}/${project.artifactId}-${project.version}-source-release.zip > >> > > > > where project.groupId has had '.' replaced by '/'. > >> > > > > >> > > > AIUI the name is not controlled by the Release Plugin. > >> > > > > >> > > > The first part of the name is determined by Maven Convention and > ASF > >> > > > hosting. > >> > > > The -source-release.zip suffix depends on settings outside the > >> Release > >> > > > plugin. > >> > > > > >> > > > Apache Commons sometimes uses the Release Plugin, but our archives > >> are > >> > > > called -src.zip and -src.tar.gz > >> > > > > >> > > > >> > > Okay, I'll rephrase that: > >> > > > >> > > Again this is where a bit of knowledge of what the Release Plugin > does > >> > > *during a release from the Apache Maven project* does help you. > >> > > > >> > > Is that clear enough? > >> > > > >> > > > >> > > > > >> > > > > > >> > > > >> > Note that I'm not talking about respins here, that's another > >> > story. > >> > > > >> > >> > > > >> > > >> > > > >> >> > >> > > > >> >> > > >> > > > >> >> >> > >> > > > >> >> >> > On Aug 15, 2013, at 9:01 AM, Chris Graham < > >> > > chrisgw...@gmail.com > >> > > > > > >> > > > >> >> wrote: > >> > > > >> >> >> > > >> > > > >> >> >> >> > >> > > > >> >> >> >> > >> > > > >> >> >> >> Sent from my iPhone > >> > > > >> >> >> >> > >> > > > >> >> >> >> On 15/08/2013, at 10:05 PM, sebb <seb...@gmail.com> > >> wrote: > >> > > > >> >> >> >> > >> > > > >> >> >> >>> On 15 August 2013 10:08, Chris Graham < > >> > chrisgw...@gmail.com > >> > > > > >> > > > >> wrote: > >> > > > >> >> >> >>>> What sebb does not appear to have understood or > >> accepted, > >> > > as > >> > > > >> >> Stephen > >> > > > >> >> >> has > >> > > > >> >> >> >>>> endlessly pointed out, is that we vote on the source > >> > > bundle, > >> > > > >> not a > >> > > > >> >> scm > >> > > > >> >> >> >>>> revision, and that, strictly speaking a SCM is not > even > >> > > > required > >> > > > >> >> >> (however > >> > > > >> >> >> >>>> sensible it is to use one). > >> > > > >> >> >> >>>> > >> > > > >> >> >> >>>> He wants a tree and a revision so that we can > compare > >> > > between > >> > > > >> >> >> releases, > >> > > > >> >> >> >>>> where what he should be doing, strictly speaking, is > >> > > > comparing > >> > > > >> >> source > >> > > > >> >> >> tar > >> > > > >> >> >> >>>> balls, as that is what we really are voting on. > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> I agree that what is released (and voted on) are the > >> > source > >> > > > >> >> tarballs. > >> > > > >> >> >> >>> And any such tarballs should be identical (barring > >> > possibly > >> > > > >> >> different > >> > > > >> >> >> >>> EOL settings for text files). > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> However, that is only one of the checks that need to > be > >> > > made. > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> The PMC also needs to ensure that the files are being > >> > > released > >> > > > >> under > >> > > > >> >> >> >>> the correct license. > >> > > > >> >> >> >> > >> > > > >> >> >> >> Are not the licenses in the source that is in the > source > >> > > > tarball? > >> > > > >> >> >> >> > >> > > > >> >> >> >> If so, can not the rat plugin or similar be used to > check > >> > the > >> > > > >> >> >> compliance? > >> > > > >> >> >> >> > >> > > > >> >> >> >>> I contend that the only practical way to check the > >> > licences > >> > > > is to > >> > > > >> >> >> >>> compare the source tarball(s) with the files in SCM. > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> [The files should only be added to SCM if the > license is > >> > OK, > >> > > > so > >> > > > >> the > >> > > > >> >> >> >>> SCM tag acts as a database of validated source > files.] > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> The SVN revision / Git hash are needed to ensure > >> > uniqueness. > >> > > > >> >> >> >>> > >> > > > >> >> >> >>> > >> > > > >> >> > >> > > > --------------------------------------------------------------------- > >> > > > >> >> >> >>> To unsubscribe, e-mail: > >> dev-unsubscr...@maven.apache.org > >> > > > >> >> >> >>> For additional commands, e-mail: > >> > dev-h...@maven.apache.org > >> > > > >> >> >> >>> > >> > > > >> >> >> >> > >> > > > >> >> >> >> > >> > > > >> > >> > --------------------------------------------------------------------- > >> > > > >> >> >> >> To unsubscribe, e-mail: > dev-unsubscr...@maven.apache.org > >> > > > >> >> >> >> For additional commands, e-mail: > >> dev-h...@maven.apache.org > >> > > > >> >> >> >> > >> > > > >> >> >> > > >> > > > >> >> >> > Thanks, > >> > > > >> >> >> > > >> > > > >> >> >> > Jason > >> > > > >> >> >> > > >> > > > >> >> >> > > ---------------------------------------------------------- > >> > > > >> >> >> > Jason van Zyl > >> > > > >> >> >> > Founder, Apache Maven > >> > > > >> >> >> > http://twitter.com/jvanzyl > >> > > > >> >> >> > > --------------------------------------------------------- > >> > > > >> >> >> > > >> > > > >> >> >> > There's no sense in being precise when you don't even > know > >> > > what > >> > > > >> you're > >> > > > >> >> >> talking about. > >> > > > >> >> >> > > >> > > > >> >> >> > -- John von Neumann > >> > > > >> >> >> > > >> > > > >> >> >> > > >> > > > >> >> >> > > >> > > > >> >> >> > > >> > > > >> >> >> > > >> > > > >> >> >> > > >> > > > >> >> >> > >> > > > >> >> >> > >> > > > > --------------------------------------------------------------------- > >> > > > >> >> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> > > > >> >> >> For additional commands, e-mail: > dev-h...@maven.apache.org > >> > > > >> >> >> > >> > > > >> >> >> -- > >> > > > >> >> >> Dennis Lundberg <dev-h...@maven.apache.org> > >> > > > >> >> >> > >> > > > >> >> > >> > > > >> >> > >> > > > --------------------------------------------------------------------- > >> > > > >> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> > > > >> >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > > > >> >> > >> > > > >> >> -- > >> > > > >> >> Dennis Lundberg <dev-h...@maven.apache.org> > >> > > > >> >> > >> > > > >> > >> > > > >> > >> > --------------------------------------------------------------------- > >> > > > >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> > > > >> For additional commands, e-mail: dev-h...@maven.apache.org > >> > > > >> > >> > > > >> -- > >> > > > >> Dennis Lundberg <dev-h...@maven.apache.org> > >> > > > >> > >> > > > > >> > > > > --------------------------------------------------------------------- > >> > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >> > > > For additional commands, e-mail: dev-h...@maven.apache.org > >> > > > > >> > > > -- > >> > > > Dennis Lundberg <dev-h...@maven.apache.org> > >> > > > > >> > > > >> > > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >