[
https://issues.apache.org/jira/browse/LUCENE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15867824#comment-15867824
]
Jan Høydahl commented on LUCENE-5143:
-------------------------------------
So I did some more research. I downloaded all the {{.asc}} signatures
{noformat}
wget -r -np -k -m -A asc,KEYS https://archive.apache.org/dist/lucene/
{noformat}
It's a big tree, including tika, nutch etc as well. Top of the tree:
{noformat}
├── archive.apache.org
│ └── dist
│ └── lucene
│ ├── KEYS
│ ├── hadoop
│ │ ├── KEYS
│ │ ├── hadoop-0.14.4
│ │ │ └── hadoop-0.14.4.tar.gz.asc
│ │ ├── hadoop-0.15.2
│ │ │ └── hadoop-0.15.2.tar.gz.asc
│ │ └── stable
│ │ └── hadoop-0.14.4.tar.gz.asc
│ ├── java
│ │ ├── 2.9.4
│ │ │ ├── lucene-2.9.4-src.tar.gz.asc
│ │ │ ├── lucene-2.9.4-src.zip.asc
│ │ │ ├── lucene-2.9.4.tar.gz.asc
│ │ │ └── lucene-2.9.4.zip.asc
│ │ ├── 3.0.3
│ │ │ ├── lucene-3.0.3-src.tar.gz.asc
│ │ │ ├── lucene-3.0.3-src.zip.asc
│ │ │ ├── lucene-3.0.3.tar.gz.asc
│ │ │ └── lucene-3.0.3.zip.asc
{noformat}
Next, I found all PGP keys ever used to sign a Lucene / Solr release, abusing
the {{--verify}} flag without the tarball file:
{code}
for f in $(find archive.apache.org/dist/lucene/java/ -name *.asc); do gpg
--verify $f /dev/null 2>> java.txt; done
cat java.txt |grep "key ID" | awk -e '{print $14;}' | sort | uniq > java-ids.txt
for f in $(find archive.apache.org/dist/lucene/solr/ -name *.asc); do echo $f;
gpg --verify $f /dev/null 2>> solr.txt; done
cat solr.txt |grep "key ID" | awk -e '{print $14;}' | sort | uniq > solr-ids.txt
{code}
Now the {{java-ids.txt}} file contains all unique PGP key IDs used to sign
Lucene releases, and the same for {{solr-ids.txt}}.
Then I like to import all these keys to my keychain and list who have ever
signed a release:
{code}
for proj in java solr; do echo "$proj-keys.txt"; for k in $(cat $proj.txt |grep
"key ID" | awk -e '{print $14;}' | sort|uniq); do gpg --recv-keys 0x$k 2>&1
>/dev/null | grep "gpg: key " >> $proj-keys.txt ; done ; done
$ cat java-keys.txt
gpg: key 0AFCEE7C: "Yonik Seeley <[email protected]>" not changed
gpg: key 1EFAFD39: "Ryan Ernst <[email protected]>" not changed
gpg: key 13E57FFC: "Upayavira <[email protected]>" not changed
gpg: key 177050F6: "Shalin Shekhar Mangar (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key 1ED139E7: "Steven Alan Rowe (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key 322D7ECA: "Robert Muir (Code Signing Key) <[email protected]>" not
changed
gpg: key 3A95B305: "Christoph Goller (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key 3FCFDB3E: "Noble Paul (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 6BD872A0: "Michael Busch (Lucene Committer) <[email protected]>" not
changed
gpg: key 6BD872A0: "Michael Busch (Lucene Committer) <[email protected]>" not
changed
gpg: key 6E68DA61: "Michael McCandless (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key 6FDB8105: "Jim Ferenczi (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 76BC6507: "Adrien Grand (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 78796AC8: "Mark Robert Miller (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key A3A13A7F: "Anshum Gupta <[email protected]>" not changed
gpg: key A7239D59: "Doug Cutting (Lucene guy) <[email protected]>" not changed
gpg: key A7239D59: "Doug Cutting (Lucene guy) <[email protected]>" not changed
gpg: key E1EE085F: "Uwe Schindler (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key EB0199F8: "Mark Robert Miller (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key ECA39416: "Simon Willnauer (Code Signing Key) <[email protected]>" not
changed
gpg: key F625308A: "Nicholas Walter Knize (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key F6ED44FC: "Timothy Potter <[email protected]>" not changed
gpg: key FE045966: "Grant Ingersoll (CODE SIGNING KEY) <[email protected]>"
not changed
$ cat solr-keys.txt
gpg: key 0AFCEE7C: "Yonik Seeley <[email protected]>" not changed
gpg: key 1EFAFD39: "Ryan Ernst <[email protected]>" not changed
gpg: key 13E57FFC: "Upayavira <[email protected]>" not changed
gpg: key 177050F6: "Shalin Shekhar Mangar (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key 1ED139E7: "Steven Alan Rowe (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key 322D7ECA: "Robert Muir (Code Signing Key) <[email protected]>" not
changed
gpg: key 3FCFDB3E: "Noble Paul (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 6E68DA61: "Michael McCandless (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key 6FDB8105: "Jim Ferenczi (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 76BC6507: "Adrien Grand (CODE SIGNING KEY) <[email protected]>" not
changed
gpg: key 78796AC8: "Mark Robert Miller (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key 521A0277: "Cassandra Targett <[email protected]>" not changed
gpg: key A3A13A7F: "Anshum Gupta <[email protected]>" not changed
gpg: key A867E8B1: "Grant Ingersoll (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key E1EE085F: "Uwe Schindler (CODE SIGNING KEY) <[email protected]>"
not changed
gpg: key ECA39416: "Simon Willnauer (Code Signing Key) <[email protected]>" not
changed
gpg: key F625308A: "Nicholas Walter Knize (CODE SIGNING KEY)
<[email protected]>" not changed
gpg: key F6ED44FC: "Timothy Potter <[email protected]>" not changed
gpg: key F8F58E19: "Chris "Hoss" Hostetter <[email protected]>" not changed
gpg: key FE045966: "Grant Ingersoll (CODE SIGNING KEY) <[email protected]>"
not changed
{code}
The overlap is not surprising, and you can see some differences related to
pre-3.0 era.
We will now test if all the keys for historic RMs are present in the various
KEYS files in the mirrors:
{code}
cp archive.apache.org/dist/lucene/KEYS KEYS-lucene.txt && cp
archive.apache.org/dist/lucene/java/KEYS KEYS-java.txt && cp
archive.apache.org/dist/lucene/solr/KEYS KEYS-solr.txt && cp
archive.apache.org/dist/lucene/solr/6.4.1/KEYS KEYS-641.txt
for keys in KEYS-*.txt; do echo match_$keys; for id in $(cat
*-ids.txt|sort|uniq); do echo "Looking for id $(cat *-keys.txt |sort|uniq| grep
$id)"; grep -E "$id|$(echo $id|echo $id|awk -e '{print substr($_,0,4)"
"substr($_,5,4);}')" $keys ; done >match_$keys ; done
{code}
The result is depressing. {{lucene/KEYS}} lack 14 keys, {{lucene/java/KEYS}}
lack 12 keys, {{lucene/solr/KEYS}} lack 11 keys and {{lucene/*/6.4.1/KEYS}}
lack 6 keys.
The missing keys in the latest 6.4.1 KEYS file (assuming it is auto generated
from http://people.apache.org/keys/group/lucene.asc) are :
{noformat}
0AFCEE7C: "Yonik Seeley <[email protected]>" (OLD KEY from 2007, probably not
listed in LDAP anymore)
13E57FFC: "Upayavira <[email protected]>" (INVALID KEY IN LDAP, see
https://people.apache.org/keys/committer/)
3A95B305: "Christoph Goller (CODE SIGNING KEY) <[email protected]>" (Not
committer? - key not in LDAP)
521A0277: "Cassandra Targett <[email protected]>" (INVALID KEY IN LDAP, see
https://people.apache.org/keys/committer/)
A867E8B1: "Grant Ingersoll (CODE SIGNING KEY) <[email protected]>" (OLD KEY
from 2008)
EB0199F8: "Mark Robert Miller (CODE SIGNING KEY) <[email protected]>" (OLD
KEY from 2009)
{noformat}
So if the goal is to have a KEYS file available from which one can verify any
release, then we could simply generate one from IDs we know have been used by
RMs:
{code}
for id in $(cat solr-ids.txt java-ids.txt|sort|uniq); do echo "Exporting id $id
- $(grep $id lucene-solr-keys.txt)"; gpg --fingerprint $id >> KEYS_NEW; gpg
--export -a $id >> KEYS ; done
{code}
And then one of the responsibilities of new RMs is to add their key to this one
KEYS file.
> rm or formalize dealing with "general" KEYS files in our dist dir
> -----------------------------------------------------------------
>
> Key: LUCENE-5143
> URL: https://issues.apache.org/jira/browse/LUCENE-5143
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Hoss Man
>
> At some point in the past, we started creating a snapshots of KEYS (taken
> from the auto-generated data from id.apache.org) in the release dir of each
> release...
> http://www.apache.org/dist/lucene/solr/4.4.0/KEYS
> http://www.apache.org/dist/lucene/java/4.4.0/KEYS
> http://archive.apache.org/dist/lucene/java/4.3.0/KEYS
> http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS
> etc...
> But we also still have some "general" KEYS files...
> https://www.apache.org/dist/lucene/KEYS
> https://www.apache.org/dist/lucene/java/KEYS
> https://www.apache.org/dist/lucene/solr/KEYS
> ...which (as i discovered when i went to add my key to them today) are stale
> and don't seem to be getting updated.
> I vaguely remember someone (rmuir?) explaining to me at one point the reason
> we started creating a fresh copy of KEYS in each release dir, but i no longer
> remember what they said, and i can't find any mention of a reason in any of
> the release docs, or in any sort of comment in buildAndPushRelease.py
> we probably do one of the following:
> * remove these "general" KEYS files
> * add a disclaimer to the top of these files that they are legacy files for
> verifying old releases and are no longer used for new releases
> * ensure these files are up to date stop generating per-release KEYS file
> copies
> * update our release process to ensure that the general files get updated on
> each release as well
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
