Rocky 8 should be an equivalent target for building RPMs that carries similar license implications to CentOS. I agree that more detail for the packaging step would be nice, and I'd like to see a job on jenkins.impala.io to simplify these steps.
On Tue, Aug 27, 2024 at 6:12 AM Jim Apple <jbap...@apache.org> wrote: > First of all, let me say that I appreciate the way the community has > continued to update wiki pages to help new developers onboard. We're coming > up on the 7th anniversary of Impala's graduation from the ASF incubator, > and I think we can be grateful and proud of the steps the community has > made to increase openness over that time. Thank you to everyone that > contributed! > > I have two concerns about this recent edit which requires release managers > to build ten different Linux release packages. > > 1. The instructions are substantially less detailed than the other > instructions in the wiki pages on building and releasing. > > 2. I counted the pages in the different legal documents one must agree to > to download a free-as-in-beer developer copy of Redhat. It's 42, and I'm > not even confident I got all of the documents, since a number of them can > be found only by Googling references deeply embedded in the legalese in one > document mentioning another binding agreement. (There's also an HTML page > that doesn't have page numbers, the length of which depends on your legal > jurisdiction.) Additionally, this is the set of documents for where I > live, and RH points to some different documents depending on your legal > jurisdiction, even within different provinces/states in some countries > (California, Quebec). And some jurisdictions don't seem to have the > documents available, even though other documents require the downloader to > agree to be bound by them. Most of us aren't lawyers, and these are not > standard OSS legal documents like the Apache License or the GPL. The > initial documents don't seem, to me, to address whether building software > for distribution is covered by the free license (developing software is. Is > building and uploading release packages "development"? I dunno. Ask Eben > Moglen, or the equivalent person for your country.) Oh, and for the > documents without explicitly different links for different jurisdictions, I > guess I'm not even sure I have the same documents as others, self-defense > by Redhat.com based on the IP address of the country my web traffic > originates from. Furthermore, these terms change seemingly without notice: > some of the documents have changelogs indicating multiple updates in the > past year. And who knows if RH/IBM will continue to allow free downloads - > quite a few enterprise software companies have a history of withdrawing or > limiting free access to their builds of OSS or source-available software, > including Oracle, Microsoft, AT&T, SCO, Intel, etc. (I suppose if that > happens at a later date, we could just remove RH releases from the > instructions at that time.) > > I'm sure there are other Apache projects that require developers to use > macOS or Windows to even do development, but that feels different to me, > since RH is not required for developing Impala. > > Let me note that I don't feel like this is a concern in terms of Impala > following ASF policy, but more of a comment about practicality and keeping > the community open by enabling is too have release managers who do not > work for a company with an Enterprise RH8 license. > > 3. The next edit on this wiki page says that pushing images to Docker is > optional. I think Impala users will might find it frustrating if images and > releases are inconsistently available, especially when security patches are > available. > > In light of #3, I'd suggest we > > A. Make all release package creation (source, Docker, RPM, deb) a required > part of releasing > > B. Remove RH releases from this wiki page describing the release process > > C. Add substantially more explicit instructions for how to build with VMs > for different Linux distributions and ISAs (x86-64, ARM) > > D. Add a link on impala.apache.org to the base domain name of the > organizations that produce release packages produced outside the official > Apache process, along with a disclaimer: something like "Along with the > official releases, some organizations produce their own software packages > that include or are based on Apache Impala. This list includes: > https://cloudarea.biz, https://bigdoop.aol, ..." (Of course, before doing > this, we'll want to check that it's compatible with ASF requirements, but I > figure we should discuss my proposition that official RH8 builds are a > problem first before we dig into solutions, since perhaps the community > doesn't see the same problems I do with the RH8 OS availability.) > > Thanks for taking the time to read and consider this! :-) > > Jim > > ---------- Forwarded message --------- > From: <no-re...@apache.org> > Date: Mon, Aug 19, 2024 at 11:00 PM > Subject: [CONF] Impala > How to Release > > > There's *1 new edit* on your page > > [image: page icon] > <https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view> > How > to Release > <https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view> > > Here's what changed: > > ... > > > > 1. If you have not done so yet, set up your apache.org email address > <https://reference.apache.org/committer/email>. > 2. Make a code signing key > <https://www.apache.org/dev/openpgp.html#generate-key> if you don't > have one yet. This must be done on "hardware owned and controlled by > the committer" > > <https://www.apache.org/legal/release-policy.html#owned-controlled-hardware> > . > 1. Publish your key > <https://www.apache.org/dev/openpgp.html#publish-in-web-space> if > you haven't done so yet > 3. Optional but reccomended: if you haven't done so yet, add yourself > to the Apache Web-of-Trust by meeting face-to-face with a person so they > can sign your key <https://www.apache.org/dev/openpgp.html#wot> > 1. Publish your signed key > 4. If you haven't done so before, add your key to the KEYS file in the > release staging area and the release distribution area using SVN. > 1. The staging area is > https://dist.apache.org/repos/dist/dev/impala/ > 2. The release area is > https://dist.apache.org/repos/dist/release/impala/. If you are not > a PMC member, you will need to ask a PMC member to edit the file for > you. > 5. Pick a commit in the "master" branch you want to release from and > test it. > 6. Test the licencing using Apache RAT; follow the instructions in > bin/check-rat-report.py. > 1. You might have to download Apache RAT if you haven't done so > previously: https://creadur.apache.org/rat/download_rat.cgi > 7. Propose a release on the dev@ list. It should start with the string > "[DISCUSS]". Explain that this is not a "[VOTE]", and that anyone may > participate. > 8. Make a new branch off of your commit called "branch-x.y.z", where > x.y.z is the version you want to release. If you are not on the PMC, ask a > PMC member to do this: > 1. > > > Code Block > language bash > > git checkout $COMMIT_HASH_YOU_CHOSE > git checkout -b branch-x.y.z > git push apache branch-x.y.z > > > 2. > > In this branch (but not in master), update IMPALA_VERSION in > bin/impala-config.sh to x.y.z-RELEASE. For versions older than 4.1.0, > update the version number in bin/save-version.sh instead. > 3. > > (Only for 4.1+) Update the version numbers for Java pom.xml files, > use this maven command: > Code Block > language bash > > # From $IMPALA_HOME > cd java > mvn versions:set -DnewVersion=x.y.z-RELEASE > > > 4. Also in this branch (but not in master), update the GIT_HASH in > bin/save-version.sh to the git hash of the top commit in the branch. > Please > bear in mind to update this on every RC. (The git hash will of course > not > reflect that of the commit that includes this change as well, but that's > okay) > 9. Continue testing. If you find bugs, file them. When they are fixed, > cherry-pick the fixes from master to your branch that you want to include > in the release > 1. > > > Code Block > language bash > > git fetch apache # to make sure you have the latest master > git log apache/master # to find the commits you want to cherry-pick > git checkout apache/branch-x.y.z > git checkout -b x.y.z-patch-foo # the name doesn't matter - it will > only be in your local workstation > git cherry-pick b4d1a3... # or whatever git hashes > # then resolve conflicts, if there are any > # cherry-pick some more commits: > git cherry-pick .... > git cherry-pick ..... > git push apache HEAD:branch-x.y.z > > > 2. > > At that time, tag the tree at the release candidate > Code Block > language bash > > git fetch apache > git checkout apache/branch-x.y.z > git tag -a x.y.z-rcw -m "x.y.z release candidate w" # when making > release candidate w of version x.y.z > git push apache x.y.z-rcw > > > 10. Make a release tarball: > 1. > > Make the tarball using git archive. Name it > apache-impala-x.y.z.tar.gz. > Code Block > language bash > > git archive --prefix=apache-impala-x.y.z/ -o > /tmp/apache-impala-x.y.z.tar.gz x.y.z-rcw > > > 2. > > Make signature, as well as sha512 checksums. This must be done on > "hardware > owned and controlled by the committer" > > <http://web.archive.org/web/20170109170347/http://www.apache.org/dev/release#owned-controlled-hardware> > . > Code Block > language bash > > cd /tmp > > # Make the signature: > gpg -u your_apache_user_n...@apache.org --armor --output > apache-impala-x.y.z.tar.gz.asc --detach-sign apache-impala-x.y.z.tar.gz > > # Make sure it worked: > gpg --verify apache-impala-x.y.z.tar.gz.asc apache-impala-x.y.z.tar.gz > > # Make checksums: > sha512sum apache-impala-x.y.z.tar.gz > apache-impala-x.y.z.tar.gz.sha512 > > # Make sure they worked: > sha512sum --check apache-impala-x.y.z.tar.gz.sha512 > > > 11. > > Before uploading your release candidate, follow the procedure in the > section below on how to vote on a release. Don't upload until you could > vote +1. > 12. Get commitments from at least three PMC members to vote on the > artifact in the time frame for the upcoming vote. The list of PMC members > is available at > http://people.apache.org/committers-by-project.html#impala-pmc. > 13. > > Upload the artifacts. The location is > https://dist.apache.org/repos/dist/dev/impala/. Upload all three > artifacts (tarball, .asc and .sha512). Do not overwrite old release > candidates. > Code Block > language bash > > svn checkout https://dist.apache.org/repos/dist/dev/impala/ > cd impala/ > # If you already have a local checked out repo (e.g. if you did release > before), run "svn update" to pull the latest updates. > # The directory layout is x.y.z/RCw, where w is the release candidate > number - RC1 is the first candidate, RC2 the second, and so on. > mkdir -p x.y.z/RCw > cd x.y.z/RCw > cp /tmp/apache-impala-x.y.z.tar.gz* ./ > cd .. > svn add . > svn commit --username=YOUR_APACHE_USER_NAME -m "Impala x.y.z release > candidate w" > > > 14. Prepare a patch to the downloads.html page (on the asf-site branch > of the git repo) to point to the latest downloads and to remove the links > to any downloads that are subsumed by this release. Do not submit the patch > yet. > > > 1. > > > 1. Prepare a patch to include the changelog for the latest release on > the docs site. More help on this here > > <https://confluence.atlassian.com/adminjiraserver071/creating-release-notes-802592502.html>. > > > > 1. > > > 1. > 1. Access the IMPALA Versions > > <https://issues.apache.org/jira/browse/IMPALA/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel> > page > 2. Select the proper release version. They are not all in a sanely > sorted order. > 3. On the top, select "Release Notes". > 4. Select "Configure Release Notes". Ensure HTML and the proper > version are set. > 5. Click Create > > > 1. > 1. Scroll down to find a TEXTAREA containing the HTML release > notes. Copy the contents and save it to a file, e.g. > docs/changelog-4.4.1.html > > > 1. > 1. > > Add HTML tags to make this a standalone page. For example: > Code Block > language xml > > <!DOCTYPE html> > <html> > <head> > <title>Impala 2.8 Change Log</title> > </head> > <body> > <h1>Impala 2.8 Change Log</h1> > <!-- contents of Jira-generated release notes go here --> > </body> > </html> > > > 2. Reorder the sections so that the issueType groupings show the > more interesting groups, like New Feature, Epic, and Improvement, before > less interesting groups like Sub-tasks. > 3. Provide a link to the changelog on the docs main page. > 4. Full example of this is here > <https://gerrit.cloudera.org/#/c/6919/>. > 5. If the full docs are not yet ready for the release, it's OK to > leave the link to the previous release's HTML documentation so that a > link > to HTML documentation always exists. But please coordinate with the > people > who are working on the docs before you announce the release. > > > 1. > > > 1. Take a vote on dev@. Your vote email should: > 1. Have a subject that starts with "[VOTE]" > 2. Explain what the vote is about > 3. Explain how to find the artifacts for testing, and include the > tag and git tree hash (not release hash!) they correspond to. The tree > hash > can be viewed with git log --pretty="%T %s". > 4. Include apache.org user name of the release manager > 5. Include the key that the artifacts are signed with, e.g. > "Artifacts were signed with the 3BA2A72B key which can be found in > https://downloads.apache.org/impala/KEYS"; > 6. Explain what each type of vote means > 7. Explain the conditions for the vote passing, including how long > the vote will remain open for. > 8. Include a link to this wiki page so voters can read the > instructions in it on how to test, verify, and vote. > 9. Explain how you tested it. > 10. At your discretion, discuss what dependencies or tools you used > to compile or run it, like gcc, hadoop, impala-lzo, and so on. > 11. Be consistent with the Impala bylaws > <https://impala.apache.org/bylaws.html>. For instance, at the > moment I am writing this wiki page, votes stay open for 72 hours (not > counting weekends), and only PMC members have binding votes. > 2. > > When the vote closes, tally up the votes (who votes what) in a thread > with the same title as the vote thread, but with "[RESULT] " prepended. > Link to the vote thread form > https://lists.apache.org/list.html?dev@impala.apache.org. The email > should include a list of every vote, and reasons for the -1s ala: > No Format > nopanel true > > +1 (binding): > > Alice Bobopolis > Carol Davestein > > -1 (non-binding): > > Emily Frankfurt (Build failed) > > > 3. > > If that vote passes, tag the git tree at the release: > Code Block > language bash > > git fetch apache > git checkout apache/branch-x.y.z > git tag -a x.y.z -m "x.y.z release" > git push apache x.y.z > > > 4. > > Publish the release. The location is > https://dist.apache.org/repos/dist/release/impala/. Upload all three > artifacts. If you are not a PMC member, you will need a PMC member to do > the upload for you. > Code Block > language bash > > svn checkout https://dist.apache.org/repos/dist/release/impala/ > cd impala/ > # If you already have a local checked out repo (e.g. if you did release > before), run "svn update" to pull the latest updates. > mkdir x.y.z > cp /tmp/apache-impala-x.y.z.tar.gz* x.y.z/ > svn add x.y.z > svn commit --username=YOUR_APACHE_USER_NAME > # You will be prompted to write a commit message. Make it look like > (without the '#'s): > # > # Impala release x.y.z > # > # Vote thread: ... > # > # Vote results thread: ... > > > > > 1. Prepare a patch for the DOAP file. It's in the asf-site branch: > doap_Impala.rdf. Add the release date in it. > 2. Wait 24 hours for mirrors to catch up. > 3. Push your downloads.html and , changelog, DOAP file patches (Don't > forget to run bin/push_to_asf.py after submitting your changes in gerrit). > > > 1. Optional but recommended: Add the release to the Apache Reporter > Service: https://reporter.apache.org/addrelease.html?impala. If you > are not a PMC member, you will need a PMC member to do it for you. > 2. Announce the release to dev@impala.apache.org, > u...@impala.apache.org, and annou...@apache.org. The email must come > from your apache.org email address. > 1. If you are not already subscribed from your @apache.org address, > subscribe to dev@ and user@ by mailing dev-subscribe@ and > user-subscribe@. > 2. > > Give your email a subject like "[ANNOUNCE] Apache Impala x.y.z > release", and include in the body: > No Format > nopanel true > > The Apache Impala team is pleased to announce the release of Impala > x.y.z. > > Impala is a high-performance distributed SQL engine. > > The release is available at: https://impala.apache.org/downloads.html > > Thanks, > > The Apache Impala team > > > 3. To avoid the email get rejected by annou...@apache.org, use > "Plain text mode". The default in gmail has MIME Content-Type: > 'text/html' > which will be rejected. > 3. Announce the release on the community links listed on > https://impala.apache.org/community.html. > 1. You might need to reach out to the PMC, as e.g. they have access > to ApacheImpala Twitter/X account, etc. > 4. Send a patch review to the master branch to update its version > number to "p.q.r-SNAPSHOT" (where p.q.r is greater than x.y.z) > 1. Update the location in bin/impala-config.sh > 2. > > To update the version numbers for Java pom.xml files, use this > maven command: > Code Block > language bash > > # From $IMPALA_HOME > cd java > mvn versions:set -DnewVersion=p.q.r-SNAPSHOT > > > 5. In the issue tracker, change the target version for open tickets > targeted at x.y.z to p.q.r. You may need to create a version p.q.r if it > does not already exist. > 6. In the issue tracker, "Release" the version on this screen: > > https://issues.apache.org/jira/plugins/servlet/project-config/IMPALA/versions. > If > you are not a PMC member, you will need a PMC member to do it for you. > 7. > > In the SVN repo where you put the final release artifacts, delete the > previous release, assuming that branch is no longer being actively worked > on. If there is more than one active branch, leave the latest release from > each branch up. If you are not a PMC member, you will need a PMC member do > that for you. E.g. for removing the 4.1.1 release: > Code Block > > # In the folder where you checkout > https://dist.apache.org/repos/dist/release/impala/ > svn update > svn delete 4.1.1 > svn commit -m "Remove 4.1.1 release" > > > > > 1. > > Build the RPM/DEB packages and upload them to github > 1. > > Build the packages on CentOS 7, RedHat 8, Ubuntu 18.04, 20.04 and > 22.04. Example commands: > Code Block > > ./buildall.sh -noclean -notests -release -package svn update > rm -rf fe/target > export USE_APACHE_HIVE=true > ./buildall.sh -noclean -notests -release -package > > RPM/DEB packages are built under package/build. Please also build > on ARM instances. > 2. Create a release in github and upload these packages. > 1. Go to the Github page: https://github.com/apache/impala. > Click "Tags". > 2. Find the one of the current release, e.g. 4.4.1. Click the 3 > dots button at the right side. Click "Create release". > 3. Use a title of "Impala x.y.z" and add the link of the > changelog in the content. E.g. > 4. Upload all the RPM/DEB packages. > > How to Vote on a Release Candidate > > ... > Go to page history > <https://cwiki.apache.org/confluence/pages/viewpreviousversions.action?pageId=65869538&src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003> > View page > <https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view> > > Stop watching space > <https://cwiki.apache.org/confluence/users/removespacenotification.action?spaceKey=IMPALA&src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=stop-watching&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ4c3JmOjhhYTk4MDg3NTRjYTkxNzgwMTU0Y2IxNDMwMDAwMDAzIiwicXNoIjoiZGY0MDY2Njc0Mjk2OTc4ZGM0MjMyNDYxMjM3ZGJmMzJhMmJjYjQ2ZTJkNmYxZGY0YTBkZThjZmVlNDYxY2E3ZiIsImlzcyI6ImNvbmZsdWVuY2Vfbm90aWZpY2F0aW9uc0FSRUgtWFVEMS1QT1FHLUNTQU8iLCJleHAiOjE3MjQ3MzEyMDEsImlhdCI6MTcyNDEyNjQwMX0.ynFl4NII22JspCBBlZPOn1fz1yPppC8hrLlZRXSlsyg> > • > Manage notifications > <https://cwiki.apache.org/confluence/users/editmyemailsettings.action?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=manage> > [image: Confluence logo big] > This message was sent by Atlassian Confluence 7.19.25 >
