Kenn,
I have been studying the structure and content of the repositories:
- DIST:
- DEV: dist.apache.org/repos/dist/dev
- RELEASE: dist.apache.org/repos/dist/release (Raw) ==
www.apache.org/dist (html, autogenerated)
- http://archive.apache.org/dist/ (html, autogenerated, Release only)
- GitBox: https://gitbox.apache.org/repos/asf
- Nexus: https://repository.apache.org
- Repositories / Releases:
https://repository.apache.org/content/repositories/releases/
- Repositories / Snapshots:
https://repository.apache.org/content/repositories/snapshots/
Observations: (please correct me if I am wrong :) )
DIST sites:
The only way I know how to upload content to these sites is using SVN.
(Correct?)
1) It appears that DIST/DEV is pretty much whatever we want to put there.
There is large variance and inconsistent content across different
projects. Its only purpose that I can see would be to test the ability to
write content to DIST prior to putting it into DIST/RELEASE. (Correct?)
2) The content placed in DIST/RELEASE appears to be Zips of site GitHub
branches and signing files. Relatively recent releases only. Whatever is
uploaded here is automatically copied to archive. So I presume only
approved releases go here. Incubation Releases are OK, but not SNAPSHOTS.
I'm not sure I see the usefulness of this repo except for the historical
record. If I want a download of a specific release I would go to the
GitHub site and select the branch or tag I want and then fork the site.
Nonetheless, this is the official record of releases for Apache.
3) The GitBox site is automatically created and updated from the
http://GitHub.com/apache/ repositories. It appears to be just a summary of
commits. Are these mirrored? Where?
4) Nexus: This is optional and the internal project structures are more
complex. All the individual, Maven-addressable dependency elements are
archived here. Apparently whether a Maven generated "release" ends up in
Snapshots or in Releases depends strictly on how the release version tag
contains "SNAPSHOT" or not. (Is this correct?).
****
I have been studying your Apache Beam Release Guide as of Mar 17, 2018
<https://github.com/apache/beam/blob/16106263a1c10afae7b44e5abb8be1ebddd799f0/website/src/contribute/release-guide.md>,
which (I think) just before you started transitioning to Gradle.
1) The point I am stuck on is "Create a new version in JIRA". I can't find
out where to do that. I have filed a ticket with INFRA for this.
Your sharing these Release Guides are *far far* more helpful than what I
can get from the Apache documentation! Thank you!
Nonetheless, I could still use some 1:1 help from a release engineer
experienced with Apache/Maven release process. Is there someone on your
team that could give me a hand? I don't think it would take more than an
hour. Our project is not nearly as complex as Beam.
Cheers,
Lee.
On Thu, Jun 20, 2019 at 10:17 PM Kenneth Knowles <[email protected]> wrote:
> I think a lot of these questions are good for general@incubator. I expect
> that podlings put their releases under some incubator-specific directory on
> dist.apache.org and that is why it looks different than it does for Beam.
>
> As for release manager, typically a thread to discuss releasing is on dev@
> and that establishes the release manager. Officially, I think whoever calls
> a [VOTE] with a release candidate is the manager. Or maybe there's no
> official definition at all.
>
> Kenn
>
> On Thu, Jun 20, 2019 at 5:29 PM leerho <[email protected]> wrote:
>
>> Kenn,
>>
>> I know you are super busy with your new house and move, so I feel bad
>> about bugging you. So I am wondering if we could set up a video conf
>> whenever you are free to go over this stuff.
>>
>> I can't make much progress until, I can get access to some of the key
>> resources: dist.apache.org, and setting the right stuff in JIRA.
>>
>> Would designating me as "Release Manager" help ??
>>
>> Cheers,
>>
>> Lee.
>>
>> On Wed, Jun 19, 2019 at 6:56 PM Kenneth Knowles <[email protected]> wrote:
>>
>>> TBH I did not personally do any incubating releases. So there might be
>>> something funky there, since the only "real" PMC is Incubator PMC.
>>>
>>> 1) I will check the logs to see how and when our release managers got
>>> added.
>>>
>>> 2) You only need an account on pypi to release Python "convenience
>>> binaries" to pypi.
>>>
>>> 3) How you manage versioning in Jira is your business. I'm not sure how
>>> podling administration of Jira works. All the initial PMC members should
>>> probably be able to manipulate this. It is just a bucket for bugs.
>>>
>>> 3a) About the coordinates:
>>> https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-core.
>>> You can see that the main package is the same for pre- and post-incubating
>>> releases. You don't "release" RCs; they can be accessed via the staging
>>> repository before finalization.
>>>
>>> 4) We used to use maven and the maven release plugin. It is designed in
>>> a world without branches. All of those steps are useless and not necessary
>>> in a git world. You just make a branch and edit the version in your pom on
>>> that branch.
>>>
>>> Kenn
>>>
>>> On Wed, Jun 19, 2019 at 6:50 PM leerho <[email protected]> wrote:
>>>
>>>> Kenn,
>>>>
>>>> Following your Beam Release Guide:
>>>>
>>>> OK, I got new GPG keys generated,
>>>>
>>>> 1) but now I need to add them to dist.apache.org.
>>>>
>>>> There is no entry for datasketches in the list, and I have no write
>>>> access to that site.
>>>>
>>>> Apparently only PMC members can do this, so I think that means you :)
>>>>
>>>> Please add the --global option.
>>>>
>>>> my GPG Key ID is 05F2FA8F8CD4A902
>>>> and the fingerprint is : 82DF 4C90 F72D 57E3 6CB8 355B 05F2 FA8F 8CD4
>>>> A902
>>>> I will need it entered into both dev/ and release/ KEYS files.
>>>>
>>>> 2) Do I need an account on PyPl ?? Is "Release Manager" is Beam
>>>> specific?? and Gradle specific ? (It seems to be so)
>>>>
>>>> 3) Create a new version in JIRA : Only PMC members can do this ...
>>>> I will need to create a release Item for the upcoming release ...
>>>> I need to release *incubator-sketches-memory* The new version will be
>>>> "1.0.0-incubating-memory-rc1" (I think I add the rc1 here ? )
>>>> This will be Release Candidate 1.
>>>>
>>>> Note: When doing the Maven release and publish process, Maven would ask
>>>> for the version # and update the POM version automatically.
>>>> It is not clear if I need to update the POM with the correct artifact
>>>> version at this point or whether something else will.
>>>>
>>>> Also, I have tried my best to anticipate what the POM should look like,
>>>> but I would like to have someone look at it.
>>>> Specifically, I not sure if the <scm>, <repositories> and the
>>>> <nexus-staging-maven-plugin> are configured correctly.
>>>>
>>>> 4) Create a release branch and onwards: This is a bit confusing as it
>>>> is Gradle driven. I will need to do the equivalent in Maven.
>>>>
>>>> Do you have someone I could video conference with or sit with to finish
>>>> this process? I could come to your office or whatever :) :)
>>>>
>>>> Lee.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 19, 2019 at 11:09 AM Kenneth Knowles <[email protected]>
>>>> wrote:
>>>>
>>>>> Oh, ha, oops. If you flip the branch a bit there will be a pom.xml and
>>>>> we used maven. We had very poor / nonexistant automation.
>>>>>
>>>>> https://github.com/apache/beam/tree/release-2.1.0 (no gradle)
>>>>> https://github.com/apache/beam/tree/release-2.5.0 (gradle is there
>>>>> but we were using the poms)
>>>>>
>>>>> And here's a release guide from back then:
>>>>> https://github.com/apache/beam/blob/e969aefa857aa07156fb9606596db9e7234e54c4/website/src/contribute/release-guide.md
>>>>>
>>>>> On Wed, Jun 19, 2019 at 9:56 AM leerho <[email protected]> wrote:
>>>>>
>>>>>> Kenn,
>>>>>>
>>>>>> This is very helpful as it at least outlines the steps via the
>>>>>> scripts. Unfortunately, we don't use Gradle, we use Maven. I wonder if
>>>>>> you know of a project that has done a similar process description &
>>>>>> scripts
>>>>>> using Maven?
>>>>>>
>>>>>> At least this will give me something to study :) :)
>>>>>>
>>>>>> Thanks!
>>>>>> Lee.
>>>>>>
>>>>>> On Tue, Jun 18, 2019 at 8:19 PM Kenneth Knowles <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, definitely I would love to help. I am a bit time constrained
>>>>>>> right now as my partner is traveling for work and we are closing on a
>>>>>>> new
>>>>>>> home and moving this week and next...
>>>>>>>
>>>>>>> TBH a lot of it is the same as
>>>>>>> https://beam.apache.org/contribute/release-guide/
>>>>>>>
>>>>>>> And a lot of that is in scripts in
>>>>>>> https://github.com/apache/beam/tree/master/release/src/main/scripts
>>>>>>>
>>>>>>> 1. Set up signing keys
>>>>>>> 2. Build a release candidate (up to you how to do the build) - check
>>>>>>> it in to the right SVN place and stage it in sonatype; I imagine you've
>>>>>>> done that bit before
>>>>>>> 3. Start a [VOTE] thread
>>>>>>> 4. Input the release into reporter.apache.org is helpful; not sure
>>>>>>> if this applies to podlings
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>> On Tue, Jun 18, 2019 at 2:52 PM leerho <[email protected]> wrote:
>>>>>>>
>>>>>>>> Kenn,
>>>>>>>>
>>>>>>>> I am stuck.
>>>>>>>>
>>>>>>>> I have completed (I think) the modifications to
>>>>>>>> incubator-datasketches-memory so that it can be released. Before I can
>>>>>>>> move ahead and do the mods on the other repos I need to have the memory
>>>>>>>> repo available as an artifact.
>>>>>>>>
>>>>>>>> So I guess I have to go through the release process. I have tried
>>>>>>>> to read through the documentation, but it is really terrible! It must
>>>>>>>> have been written by lawyers as it is very very verbose and difficult
>>>>>>>> to
>>>>>>>> read with the key information scattered all over the place.
>>>>>>>>
>>>>>>>> What I could use is someone that can walk me through the process,
>>>>>>>> step by step. Once I have gone through that, I'm sure I can reproduce
>>>>>>>> it
>>>>>>>> for all the other repos.
>>>>>>>>
>>>>>>>> Is this something that you can help with?
>>>>>>>>
>>>>>>>> BTW, I contacted INFRA and they don't do this.
>>>>>>>> I also have sent email to [email protected] and have not heard
>>>>>>>> back.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Lee.
>>>>>>>>
>>>>>>>
