And it is done now - fresh copy of all centos packages signed by the correct key (as validated locally). Thanks for keep finding those, Evans!
Cos On Sun, Sep 06, 2015 at 02:19AM, Konstantin Boudnik wrote: > It is pretty nuts because the packages I have locally are all signed with the > correct key. But when I download the package in question I see that the local > one is different from the downloaded. And the latter is signed with the wrong > key indeed. > > Considering that I had synced everything after re-signing, there're only two > possibilities that I see: > - S3 eventual consistency bites us in the rear. Which might be possible for > in the short run, but I don't see how it couldn't be updated after all that > time > - s3cmd has screwed up and didn't updated some of the packages. I am going to > wipe out _all_ rpm distros and resync it right away. This might cause a > short interruption in the packages availability, but at least we'll have > all correct stuff up there. > > Should be done in about 30 minutes. Stay tuned > Cos > > On Sun, Sep 06, 2015 at 12:10AM, Evans Ye wrote: > > Sorry guys. I'm back with the issue again. ;) > > > > Turns out that some of the rpms are good, some are not. Look at my tests > > below: > > > > > > ### Centos 6 repo ### > > > > $ docker run -ti --rm bigtop/puppet:centos-6 bash -l > > > > $ wget > > https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo > > -O /etc/yum.repos.d/bigtop.repo > > > > $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat > > zookeeper # Successfully installed > > > > $ yum -y install hadoop hadoop-hdfs > > > > ... > > > > Error Downloading Packages: > > > > hadoop-hdfs-2.6.0-1.el6.x86_64: failure: > > hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] > > No more mirrors to try. > > > > hadoop-2.6.0-1.el6.x86_64: failure: > > hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No > > more mirrors to try. > > > > > > I find the same set of packages(groovy, utils, jsvc, tomcat, zookeeper) can > > be successfully installed across centos 6, 7 and fedora repos and the other > > same set of packages failed to install across the platforms. Therefore, I > > think there might be an issue happening during some sort of automation > > steps. > > > > In addition, I suspect that those packages failed to install are still > > signed by old key, hence the subkey issue found by Cos blocks the packages > > to be installed. > > > > > > [root@34696969ce7d /]# rpm --checksig hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm > > > > hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK > > (MISSING KEYS: (MD5) PGP#d0c3824f) > > > > [root@34696969ce7d /]# rpm --checksig bigtop-groovy-2.3.8-1.fc20.noarch.rpm > > > > bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK > > > > > > Cos can you first check that the hadoop* packages has been successfully > > resigned by your newly generated code signing key? Thanks! > > > > > > Evans > > 2015年9月4日 上午2:23於 "Konstantin Boudnik" <[email protected]>寫道： > > > > > Appreciate the sentiment guys and thanks for kind words! > > > The irony here is that I don't even like this type of packaging and not > > > using > > > it if I can help it ;) Oh well... > > > > > > To close this thread - I will try to put together a blog about 1.0 later > > > today. Thanks everyone for the testing, patience, and - kudos to Evans - > > > detailed instructions on how to reproduce the issue! > > > > > > Cos > > > > > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote: > > > > Yes thanks cos for getting this centos stuff figured out.! > > > > > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <[email protected]> > > > wrote: > > > > > > > > > > Thanks for sticking with it Cos. That's an annoying bug. > > > > > > > > > > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <[email protected]> > > > wrote: > > > > >> > > > > >> Ok, as I suspected there's a long standing (at least from 2006) bug > > > in RPM > > > > >> that doesn't allow to validate RPM signature if a subkey has been > > > used for > > > > >> signing. > > > > >> > > > > >> I ended up generating a new key pair (just for this purpose) and > > > resigning > > > > >> all > > > > >> binaries with it; then resyncing everything with s3. I also have > > > updated > > > > >> KEYS > > > > >> file with the new one. I have quickly ran a test on centos7 by > > > installing > > > > >> bigtop-utils on an empty container and everything worked, including > > > > >> automatic > > > > >> import of the keys and the validation/installation of the package. > > > Looks > > > > >> like > > > > >> we are in the clear. > > > > >> > > > > >> Please shout if you see otherwise. Thanks everyone for your patience! > > > > >> Cos > > > > >> > > > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote: > > > > >>> I think there's a difference between how you've signed the pkgs and > > > how > > > > >> I did > > > > >>> it. I signed with sub-key (as I mentioned before) and yum doesn't > > > > >> recognize > > > > >>> it. Seemingly, it expects that the master key was used for signing. > > > > >>> > > > > >>> Also, in your repo file below > > > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS > > > > >>> points to the old keys. The location should be > > > > >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS > > > > >>> > > > > >>> I am pretty sure I have exported my key with --armor option back in > > > the > > > > >> day. > > > > >>> But I will repeat it and see if I can fix the situation, which I > > > > >>> also > > > > >> observer > > > > >>> following your steps. If that's the only issue I will update the > > > > >>> KEYS > > > > >> and we > > > > >>> should be completed by tonight ;) > > > > >>> > > > > >>> Thanks for your help! > > > > >>> Cos > > > > >>> > > > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote: > > > > >>>> This is the same issue we're trying to solve in the mailing thread > > > > >>>> "convenience artifacts are signed and uploaded". I've built a > > > > >>>> sample > > > > >> repo > > > > >>>> which works properly by using my own key "Evans Ye" to sign and to > > > > >> export > > > > >>>> GPG KEY. So I believe the following steps should be the right way > > > > >>>> to > > > > >> sign > > > > >>>> packages and export the gpgkey: > > > > >>>> > > > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" > > > --addsign > > > > >>>> > > > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye' > > > > >>>> I've verified that the hash is matched now in our official repo. > > > > >>>> So I guess the main issue left is using non-armored gpg key, if we > > > > >> manually > > > > >>>> import the gpgkey in the repo file: > > > > >>>> > > > > >>>> [bigtop] > > > > >>>> name=Bigtop > > > > >>>> enabled=1 > > > > >>>> gpgcheck=1 > > > > >>>> type=NONE > > > > >>>> baseurl= > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64 > > > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS > > > > >>>> > > > > >>>> [root@48723d98dc1b ~]# rpm --import > > > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS > > > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key > > > 2 > > > > >> not an > > > > >>>> armored public key. > > > > >>>> > > > > >>>> It gets error. > > > > >>>> However, my own exported armored key can be imported without an > > > error. > > > > >>>> That's the different. > > > > >>>> > > > > >>>> Can you confirm that the gpgkey( > > > > >> http://archive.apache.org/dist/bigtop/KEYS) > > > > >>>> is exported with --armor flag? > > > > >>>> > > > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <[email protected]>: > > > > >>>> > > > > >>>>> Looks like I have figured out what's wrong with my key. And it is > > > > >>>>> _nothing_. > > > > >>>>> However, it seems that I can not sign RPMs with subkey as YUM can > > > > >> not find > > > > >>>>> the > > > > >>>>> key while importing. Can anyone confirm or disprove my train of > > > > >> thoughts? > > > > >>>>> > > > > >>>>> Thanks! > > > > >>>>> Cos > > > > >>>>> > > > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote: > > > > >>>>>> I've resynced the repodata once again and I don't see this issue > > > > >> on the > > > > >>>>>> centos7 anymore. However, yum still complains about the key being > > > > >> no > > > > >>>>>> available, but there's a workaround by setting gpgcheck=0 And I > > > > >>>>>> am > > > > >> going > > > > >>>>> to > > > > >>>>>> figure out what to do with it and why my key isn't working as > > > > >> expected. > > > > >>>>>> > > > > >>>>>> I also have discovered that the gpgkey file URL is using the old > > > > >>>>> incubation > > > > >>>>>> KEYS. Fixed that as well. > > > > >>>>>> > > > > >>>>>> Please let me know if you still see the issue with checksums > > > > >> mismatch. > > > > >>>>>> Thanks, > > > > >>>>>> Cos > > > > >>>>>> > > > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote: > > > > >>>>>>> I think this is the consequences of me fighting with the package > > > > >>>>> signing... ;( > > > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the > > > > >> RPM-based > > > > >>>>> distros > > > > >>>>>>> and uploaded new repo files to the release. Not sure why the > > > > >> checksums > > > > >>>>> differ > > > > >>>>>>> now... > > > > >>>>>>> > > > > >>>>>>> I will take a look into this again tonight. > > > > >>>>>>> Cos > > > > >>>>>>> > > > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote: > > > > >>>>>>>> I can second it: > > > > >>>>>>>> > > > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo > > > > >>>>>>>> > > > > >>>>>>>> [meins] > > > > >>>>>>>> name=Bigtop epo > > > > >>>>>>>> baseurl= > > > > >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/ > > > > >>>>>>>> enabled=1 > > > > >>>>>>>> gpgcheck=0 > > > > >>>>>>>> priority=1 > > > > >>>>>>>> > > > > >>>>>>>> and got > > > > >>>>>>>> ............ > > > > >>>>>>>> Downloading packages: > > > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED > > > > >>>>> =============================================-] 849 kB/s > > > > >> | 62 > > > > >>>>> MB 00:00:00 ETA > > > > >> > > > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm > > > > >> : > > > > >>>>> [Errno -1] Package does not match intended download. Suggestion: > > > run > > > > >> yum > > > > >>>>> --enablerepo=meins clean metadata > > > > >>>>>>>> Trying other mirror. > > > > >>>>>>>> ............. > > > > >>>>>>>> > > > > >>>>>>>> Olaf > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > > > > > > - Andy > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > > > Hein > > > > > (via Tom White) > > >
signature.asc
Description: Digital signature
