This section in README.md needs an update as well. https://github.com/apache/impala/?tab=readme-ov-file#supported-os-distributions
On Tue, Aug 27, 2024 at 9:08 AM Michael Smith <michael.sm...@cloudera.com> wrote: > Rocky 8 should be an equivalent target for building RPMs that carries > similar license implications to CentOS. I agree that more detail for the > packaging step would be nice, and I'd like to see a job on > jenkins.impala.io > to simplify these steps. > > On Tue, Aug 27, 2024 at 6:12 AM Jim Apple <jbap...@apache.org> wrote: > > > First of all, let me say that I appreciate the way the community has > > continued to update wiki pages to help new developers onboard. We're > coming > > up on the 7th anniversary of Impala's graduation from the ASF incubator, > > and I think we can be grateful and proud of the steps the community has > > made to increase openness over that time. Thank you to everyone that > > contributed! > > > > I have two concerns about this recent edit which requires release > managers > > to build ten different Linux release packages. > > > > 1. The instructions are substantially less detailed than the other > > instructions in the wiki pages on building and releasing. > > > > 2. I counted the pages in the different legal documents one must agree to > > to download a free-as-in-beer developer copy of Redhat. It's 42, and I'm > > not even confident I got all of the documents, since a number of them can > > be found only by Googling references deeply embedded in the legalese in > one > > document mentioning another binding agreement. (There's also an HTML page > > that doesn't have page numbers, the length of which depends on your legal > > jurisdiction.) Additionally, this is the set of documents for where I > > live, and RH points to some different documents depending on your legal > > jurisdiction, even within different provinces/states in some countries > > (California, Quebec). And some jurisdictions don't seem to have the > > documents available, even though other documents require the downloader > to > > agree to be bound by them. Most of us aren't lawyers, and these are not > > standard OSS legal documents like the Apache License or the GPL. The > > initial documents don't seem, to me, to address whether building software > > for distribution is covered by the free license (developing software is. > Is > > building and uploading release packages "development"? I dunno. Ask Eben > > Moglen, or the equivalent person for your country.) Oh, and for the > > documents without explicitly different links for different > jurisdictions, I > > guess I'm not even sure I have the same documents as others, self-defense > > by Redhat.com based on the IP address of the country my web traffic > > originates from. Furthermore, these terms change seemingly without > notice: > > some of the documents have changelogs indicating multiple updates in the > > past year. And who knows if RH/IBM will continue to allow free downloads > - > > quite a few enterprise software companies have a history of withdrawing > or > > limiting free access to their builds of OSS or source-available software, > > including Oracle, Microsoft, AT&T, SCO, Intel, etc. (I suppose if that > > happens at a later date, we could just remove RH releases from the > > instructions at that time.) > > > > I'm sure there are other Apache projects that require developers to use > > macOS or Windows to even do development, but that feels different to me, > > since RH is not required for developing Impala. > > > > Let me note that I don't feel like this is a concern in terms of Impala > > following ASF policy, but more of a comment about practicality and > keeping > > the community open by enabling is too have release managers who do not > > work for a company with an Enterprise RH8 license. > > > > 3. The next edit on this wiki page says that pushing images to Docker is > > optional. I think Impala users will might find it frustrating if images > and > > releases are inconsistently available, especially when security patches > are > > available. > > > > In light of #3, I'd suggest we > > > > A. Make all release package creation (source, Docker, RPM, deb) a > required > > part of releasing > > > > B. Remove RH releases from this wiki page describing the release process > > > > C. Add substantially more explicit instructions for how to build with VMs > > for different Linux distributions and ISAs (x86-64, ARM) > > > > D. Add a link on impala.apache.org to the base domain name of the > > organizations that produce release packages produced outside the official > > Apache process, along with a disclaimer: something like "Along with the > > official releases, some organizations produce their own software packages > > that include or are based on Apache Impala. This list includes: > > https://cloudarea.biz, https://bigdoop.aol, ..." (Of course, before > doing > > this, we'll want to check that it's compatible with ASF requirements, > but I > > figure we should discuss my proposition that official RH8 builds are a > > problem first before we dig into solutions, since perhaps the community > > doesn't see the same problems I do with the RH8 OS availability.) > > > > Thanks for taking the time to read and consider this! :-) > > > > Jim > > > > ---------- Forwarded message --------- > > From: <no-re...@apache.org> > > Date: Mon, Aug 19, 2024 at 11:00 PM > > Subject: [CONF] Impala > How to Release > > > > > > There's *1 new edit* on your page > > > > [image: page icon] > > < > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view> > How > > to Release > > < > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view > > > > > > Here's what changed: > > > > ... > > > > > > > > 1. If you have not done so yet, set up your apache.org email address > > <https://reference.apache.org/committer/email>. > > 2. Make a code signing key > > <https://www.apache.org/dev/openpgp.html#generate-key> if you don't > > have one yet. This must be done on "hardware owned and controlled by > > the committer" > > < > https://www.apache.org/legal/release-policy.html#owned-controlled-hardware > > > > . > > 1. Publish your key > > <https://www.apache.org/dev/openpgp.html#publish-in-web-space> if > > you haven't done so yet > > 3. Optional but reccomended: if you haven't done so yet, add yourself > > to the Apache Web-of-Trust by meeting face-to-face with a person so > they > > can sign your key <https://www.apache.org/dev/openpgp.html#wot> > > 1. Publish your signed key > > 4. If you haven't done so before, add your key to the KEYS file in the > > release staging area and the release distribution area using SVN. > > 1. The staging area is > > https://dist.apache.org/repos/dist/dev/impala/ > > 2. The release area is > > https://dist.apache.org/repos/dist/release/impala/. If you are not > > a PMC member, you will need to ask a PMC member to edit the file > for you. > > 5. Pick a commit in the "master" branch you want to release from and > > test it. > > 6. Test the licencing using Apache RAT; follow the instructions in > > bin/check-rat-report.py. > > 1. You might have to download Apache RAT if you haven't done so > > previously: https://creadur.apache.org/rat/download_rat.cgi > > 7. Propose a release on the dev@ list. It should start with the > string > > "[DISCUSS]". Explain that this is not a "[VOTE]", and that anyone may > > participate. > > 8. Make a new branch off of your commit called "branch-x.y.z", where > > x.y.z is the version you want to release. If you are not on the PMC, > ask a > > PMC member to do this: > > 1. > > > > > > Code Block > > language bash > > > > git checkout $COMMIT_HASH_YOU_CHOSE > > git checkout -b branch-x.y.z > > git push apache branch-x.y.z > > > > > > 2. > > > > In this branch (but not in master), update IMPALA_VERSION in > > bin/impala-config.sh to x.y.z-RELEASE. For versions older than > 4.1.0, > > update the version number in bin/save-version.sh instead. > > 3. > > > > (Only for 4.1+) Update the version numbers for Java pom.xml files, > > use this maven command: > > Code Block > > language bash > > > > # From $IMPALA_HOME > > cd java > > mvn versions:set -DnewVersion=x.y.z-RELEASE > > > > > > 4. Also in this branch (but not in master), update the GIT_HASH in > > bin/save-version.sh to the git hash of the top commit in the > branch. Please > > bear in mind to update this on every RC. (The git hash will of > course not > > reflect that of the commit that includes this change as well, but > that's > > okay) > > 9. Continue testing. If you find bugs, file them. When they are fixed, > > cherry-pick the fixes from master to your branch that you want to > include > > in the release > > 1. > > > > > > Code Block > > language bash > > > > git fetch apache # to make sure you have the latest master > > git log apache/master # to find the commits you want to cherry-pick > > git checkout apache/branch-x.y.z > > git checkout -b x.y.z-patch-foo # the name doesn't matter - it > will only be in your local workstation > > git cherry-pick b4d1a3... # or whatever git hashes > > # then resolve conflicts, if there are any > > # cherry-pick some more commits: > > git cherry-pick .... > > git cherry-pick ..... > > git push apache HEAD:branch-x.y.z > > > > > > 2. > > > > At that time, tag the tree at the release candidate > > Code Block > > language bash > > > > git fetch apache > > git checkout apache/branch-x.y.z > > git tag -a x.y.z-rcw -m "x.y.z release candidate w" # when making > release candidate w of version x.y.z > > git push apache x.y.z-rcw > > > > > > 10. Make a release tarball: > > 1. > > > > Make the tarball using git archive. Name it > > apache-impala-x.y.z.tar.gz. > > Code Block > > language bash > > > > git archive --prefix=apache-impala-x.y.z/ -o > /tmp/apache-impala-x.y.z.tar.gz x.y.z-rcw > > > > > > 2. > > > > Make signature, as well as sha512 checksums. This must be done on > "hardware > > owned and controlled by the committer" > > < > http://web.archive.org/web/20170109170347/http://www.apache.org/dev/release#owned-controlled-hardware > > > > . > > Code Block > > language bash > > > > cd /tmp > > > > # Make the signature: > > gpg -u your_apache_user_n...@apache.org --armor --output > apache-impala-x.y.z.tar.gz.asc --detach-sign apache-impala-x.y.z.tar.gz > > > > # Make sure it worked: > > gpg --verify apache-impala-x.y.z.tar.gz.asc > apache-impala-x.y.z.tar.gz > > > > # Make checksums: > > sha512sum apache-impala-x.y.z.tar.gz > > apache-impala-x.y.z.tar.gz.sha512 > > > > # Make sure they worked: > > sha512sum --check apache-impala-x.y.z.tar.gz.sha512 > > > > > > 11. > > > > Before uploading your release candidate, follow the procedure in the > > section below on how to vote on a release. Don't upload until you > could > > vote +1. > > 12. Get commitments from at least three PMC members to vote on the > > artifact in the time frame for the upcoming vote. The list of PMC > members > > is available at > > http://people.apache.org/committers-by-project.html#impala-pmc. > > 13. > > > > Upload the artifacts. The location is > > https://dist.apache.org/repos/dist/dev/impala/. Upload all three > > artifacts (tarball, .asc and .sha512). Do not overwrite old release > > candidates. > > Code Block > > language bash > > > > svn checkout https://dist.apache.org/repos/dist/dev/impala/ > > cd impala/ > > # If you already have a local checked out repo (e.g. if you did > release before), run "svn update" to pull the latest updates. > > # The directory layout is x.y.z/RCw, where w is the release candidate > number - RC1 is the first candidate, RC2 the second, and so on. > > mkdir -p x.y.z/RCw > > cd x.y.z/RCw > > cp /tmp/apache-impala-x.y.z.tar.gz* ./ > > cd .. > > svn add . > > svn commit --username=YOUR_APACHE_USER_NAME -m "Impala x.y.z release > candidate w" > > > > > > 14. Prepare a patch to the downloads.html page (on the asf-site branch > > of the git repo) to point to the latest downloads and to remove the > links > > to any downloads that are subsumed by this release. Do not submit the > patch > > yet. > > > > > > 1. > > > > > > 1. Prepare a patch to include the changelog for the latest release on > > the docs site. More help on this here > > < > https://confluence.atlassian.com/adminjiraserver071/creating-release-notes-802592502.html > >. > > > > > > > > 1. > > > > > > 1. > > 1. Access the IMPALA Versions > > < > https://issues.apache.org/jira/browse/IMPALA/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel > > > > page > > 2. Select the proper release version. They are not all in a sanely > > sorted order. > > 3. On the top, select "Release Notes". > > 4. Select "Configure Release Notes". Ensure HTML and the proper > > version are set. > > 5. Click Create > > > > > > 1. > > 1. Scroll down to find a TEXTAREA containing the HTML release > > notes. Copy the contents and save it to a file, e.g. > > docs/changelog-4.4.1.html > > > > > > 1. > > 1. > > > > Add HTML tags to make this a standalone page. For example: > > Code Block > > language xml > > > > <!DOCTYPE html> > > <html> > > <head> > > <title>Impala 2.8 Change Log</title> > > </head> > > <body> > > <h1>Impala 2.8 Change Log</h1> > > <!-- contents of Jira-generated release notes go here --> > > </body> > > </html> > > > > > > 2. Reorder the sections so that the issueType groupings show the > > more interesting groups, like New Feature, Epic, and Improvement, > before > > less interesting groups like Sub-tasks. > > 3. Provide a link to the changelog on the docs main page. > > 4. Full example of this is here > > <https://gerrit.cloudera.org/#/c/6919/>. > > 5. If the full docs are not yet ready for the release, it's OK to > > leave the link to the previous release's HTML documentation so > that a link > > to HTML documentation always exists. But please coordinate with > the people > > who are working on the docs before you announce the release. > > > > > > 1. > > > > > > 1. Take a vote on dev@. Your vote email should: > > 1. Have a subject that starts with "[VOTE]" > > 2. Explain what the vote is about > > 3. Explain how to find the artifacts for testing, and include the > > tag and git tree hash (not release hash!) they correspond to. The > tree hash > > can be viewed with git log --pretty="%T %s". > > 4. Include apache.org user name of the release manager > > 5. Include the key that the artifacts are signed with, e.g. > > "Artifacts were signed with the 3BA2A72B key which can be found in > > https://downloads.apache.org/impala/KEYS"; > > 6. Explain what each type of vote means > > 7. Explain the conditions for the vote passing, including how long > > the vote will remain open for. > > 8. Include a link to this wiki page so voters can read the > > instructions in it on how to test, verify, and vote. > > 9. Explain how you tested it. > > 10. At your discretion, discuss what dependencies or tools you used > > to compile or run it, like gcc, hadoop, impala-lzo, and so on. > > 11. Be consistent with the Impala bylaws > > <https://impala.apache.org/bylaws.html>. For instance, at the > > moment I am writing this wiki page, votes stay open for 72 hours > (not > > counting weekends), and only PMC members have binding votes. > > 2. > > > > When the vote closes, tally up the votes (who votes what) in a thread > > with the same title as the vote thread, but with "[RESULT] " > prepended. > > Link to the vote thread form > > https://lists.apache.org/list.html?dev@impala.apache.org. The email > > should include a list of every vote, and reasons for the -1s ala: > > No Format > > nopanel true > > > > +1 (binding): > > > > Alice Bobopolis > > Carol Davestein > > > > -1 (non-binding): > > > > Emily Frankfurt (Build failed) > > > > > > 3. > > > > If that vote passes, tag the git tree at the release: > > Code Block > > language bash > > > > git fetch apache > > git checkout apache/branch-x.y.z > > git tag -a x.y.z -m "x.y.z release" > > git push apache x.y.z > > > > > > 4. > > > > Publish the release. The location is > > https://dist.apache.org/repos/dist/release/impala/. Upload all three > > artifacts. If you are not a PMC member, you will need a PMC member to > do > > the upload for you. > > Code Block > > language bash > > > > svn checkout https://dist.apache.org/repos/dist/release/impala/ > > cd impala/ > > # If you already have a local checked out repo (e.g. if you did > release before), run "svn update" to pull the latest updates. > > mkdir x.y.z > > cp /tmp/apache-impala-x.y.z.tar.gz* x.y.z/ > > svn add x.y.z > > svn commit --username=YOUR_APACHE_USER_NAME > > # You will be prompted to write a commit message. Make it look like > (without the '#'s): > > # > > # Impala release x.y.z > > # > > # Vote thread: ... > > # > > # Vote results thread: ... > > > > > > > > > > 1. Prepare a patch for the DOAP file. It's in the asf-site branch: > > doap_Impala.rdf. Add the release date in it. > > 2. Wait 24 hours for mirrors to catch up. > > 3. Push your downloads.html and , changelog, DOAP file patches (Don't > > forget to run bin/push_to_asf.py after submitting your changes in > gerrit). > > > > > > 1. Optional but recommended: Add the release to the Apache Reporter > > Service: https://reporter.apache.org/addrelease.html?impala. If you > > are not a PMC member, you will need a PMC member to do it for you. > > 2. Announce the release to dev@impala.apache.org, > > u...@impala.apache.org, and annou...@apache.org. The email must come > > from your apache.org email address. > > 1. If you are not already subscribed from your @apache.org address, > > subscribe to dev@ and user@ by mailing dev-subscribe@ and > > user-subscribe@. > > 2. > > > > Give your email a subject like "[ANNOUNCE] Apache Impala x.y.z > > release", and include in the body: > > No Format > > nopanel true > > > > The Apache Impala team is pleased to announce the release of > Impala x.y.z. > > > > Impala is a high-performance distributed SQL engine. > > > > The release is available at: > https://impala.apache.org/downloads.html > > > > Thanks, > > > > The Apache Impala team > > > > > > 3. To avoid the email get rejected by annou...@apache.org, use > > "Plain text mode". The default in gmail has MIME Content-Type: > 'text/html' > > which will be rejected. > > 3. Announce the release on the community links listed on > > https://impala.apache.org/community.html. > > 1. You might need to reach out to the PMC, as e.g. they have access > > to ApacheImpala Twitter/X account, etc. > > 4. Send a patch review to the master branch to update its version > > number to "p.q.r-SNAPSHOT" (where p.q.r is greater than x.y.z) > > 1. Update the location in bin/impala-config.sh > > 2. > > > > To update the version numbers for Java pom.xml files, use this > > maven command: > > Code Block > > language bash > > > > # From $IMPALA_HOME > > cd java > > mvn versions:set -DnewVersion=p.q.r-SNAPSHOT > > > > > > 5. In the issue tracker, change the target version for open tickets > > targeted at x.y.z to p.q.r. You may need to create a version p.q.r if > it > > does not already exist. > > 6. In the issue tracker, "Release" the version on this screen: > > > https://issues.apache.org/jira/plugins/servlet/project-config/IMPALA/versions. > If > > you are not a PMC member, you will need a PMC member to do it for you. > > 7. > > > > In the SVN repo where you put the final release artifacts, delete the > > previous release, assuming that branch is no longer being actively > worked > > on. If there is more than one active branch, leave the latest release > from > > each branch up. If you are not a PMC member, you will need a PMC > member do > > that for you. E.g. for removing the 4.1.1 release: > > Code Block > > > > # In the folder where you checkout > https://dist.apache.org/repos/dist/release/impala/ > > svn update > > svn delete 4.1.1 > > svn commit -m "Remove 4.1.1 release" > > > > > > > > > > 1. > > > > Build the RPM/DEB packages and upload them to github > > 1. > > > > Build the packages on CentOS 7, RedHat 8, Ubuntu 18.04, 20.04 and > > 22.04. Example commands: > > Code Block > > > > ./buildall.sh -noclean -notests -release -package svn update > > rm -rf fe/target > > export USE_APACHE_HIVE=true > > ./buildall.sh -noclean -notests -release -package > > > > RPM/DEB packages are built under package/build. Please also build > > on ARM instances. > > 2. Create a release in github and upload these packages. > > 1. Go to the Github page: https://github.com/apache/impala. > > Click "Tags". > > 2. Find the one of the current release, e.g. 4.4.1. Click the 3 > > dots button at the right side. Click "Create release". > > 3. Use a title of "Impala x.y.z" and add the link of the > > changelog in the content. E.g. > > 4. Upload all the RPM/DEB packages. > > > > How to Vote on a Release Candidate > > > > ... > > Go to page history > > < > https://cwiki.apache.org/confluence/pages/viewpreviousversions.action?pageId=65869538&src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003 > > > > View page > > < > https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=view > > > > > > Stop watching space > > < > https://cwiki.apache.org/confluence/users/removespacenotification.action?spaceKey=IMPALA&src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=stop-watching&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ4c3JmOjhhYTk4MDg3NTRjYTkxNzgwMTU0Y2IxNDMwMDAwMDAzIiwicXNoIjoiZGY0MDY2Njc0Mjk2OTc4ZGM0MjMyNDYxMjM3ZGJmMzJhMmJjYjQ2ZTJkNmYxZGY0YTBkZThjZmVlNDYxY2E3ZiIsImlzcyI6ImNvbmZsdWVuY2Vfbm90aWZpY2F0aW9uc0FSRUgtWFVEMS1QT1FHLUNTQU8iLCJleHAiOjE3MjQ3MzEyMDEsImlhdCI6MTcyNDEyNjQwMX0.ynFl4NII22JspCBBlZPOn1fz1yPppC8hrLlZRXSlsyg > > > > • > > Manage notifications > > < > https://cwiki.apache.org/confluence/users/editmyemailsettings.action?src=mail&src.mail.product=confluence-server&src.mail.timestamp=1724126401442&src.mail.notification=com.atlassian.confluence.plugins.confluence-notifications-batch-plugin%3Abatching-notification&src.mail.recipient=8aa9808754ca91780154cb1430000003&src.mail.action=manage > > > > [image: Confluence logo big] > > This message was sent by Atlassian Confluence 7.19.25 > > >
