Sorry guys. I'm back with the issue again. ;) Turns out that some of the rpms are good, some are not. Look at my tests below:
### Centos 6 repo ### $ docker run -ti --rm bigtop/puppet:centos-6 bash -l $ wget https://dist.apache.org/repos/dist/release/bigtop/bigtop-1.0.0/repos/centos6/bigtop.repo -O /etc/yum.repos.d/bigtop.repo $ yum -y install bigtop-utils bigtop-groovy bigtop-jsvc bigtop-tomcat zookeeper # Successfully installed $ yum -y install hadoop hadoop-hdfs ... Error Downloading Packages: hadoop-hdfs-2.6.0-1.el6.x86_64: failure: hadoop/x86_64/hadoop-hdfs-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No more mirrors to try. hadoop-2.6.0-1.el6.x86_64: failure: hadoop/x86_64/hadoop-2.6.0-1.el6.x86_64.rpm from bigtop: [Errno 256] No more mirrors to try. I find the same set of packages(groovy, utils, jsvc, tomcat, zookeeper) can be successfully installed across centos 6, 7 and fedora repos and the other same set of packages failed to install across the platforms. Therefore, I think there might be an issue happening during some sort of automation steps. In addition, I suspect that those packages failed to install are still signed by old key, hence the subkey issue found by Cos blocks the packages to be installed. [root@34696969ce7d /]# rpm --checksig hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm hadoop-hdfs-2.6.0-1.fc20.x86_64.rpm: RSA sha1 ((MD5) PGP) md5 NOT OK (MISSING KEYS: (MD5) PGP#d0c3824f) [root@34696969ce7d /]# rpm --checksig bigtop-groovy-2.3.8-1.fc20.noarch.rpm bigtop-groovy-2.3.8-1.fc20.noarch.rpm: rsa sha1 (md5) pgp md5 OK Cos can you first check that the hadoop* packages has been successfully resigned by your newly generated code signing key? Thanks! Evans 2015年9月4日 上午2:23於 "Konstantin Boudnik" <[email protected]>寫道： > Appreciate the sentiment guys and thanks for kind words! > The irony here is that I don't even like this type of packaging and not > using > it if I can help it ;) Oh well... > > To close this thread - I will try to put together a blog about 1.0 later > today. Thanks everyone for the testing, patience, and - kudos to Evans - > detailed instructions on how to reproduce the issue! > > Cos > > On Thu, Sep 03, 2015 at 01:48PM, Jay Vyas wrote: > > Yes thanks cos for getting this centos stuff figured out.! > > > > > On Sep 3, 2015, at 12:35 PM, Andrew Purtell <[email protected]> > wrote: > > > > > > Thanks for sticking with it Cos. That's an annoying bug. > > > > > > > > >> On Wed, Sep 2, 2015 at 9:31 PM, Konstantin Boudnik <[email protected]> > wrote: > > >> > > >> Ok, as I suspected there's a long standing (at least from 2006) bug > in RPM > > >> that doesn't allow to validate RPM signature if a subkey has been > used for > > >> signing. > > >> > > >> I ended up generating a new key pair (just for this purpose) and > resigning > > >> all > > >> binaries with it; then resyncing everything with s3. I also have > updated > > >> KEYS > > >> file with the new one. I have quickly ran a test on centos7 by > installing > > >> bigtop-utils on an empty container and everything worked, including > > >> automatic > > >> import of the keys and the validation/installation of the package. > Looks > > >> like > > >> we are in the clear. > > >> > > >> Please shout if you see otherwise. Thanks everyone for your patience! > > >> Cos > > >> > > >>> On Wed, Sep 02, 2015 at 02:27PM, Konstantin Boudnik wrote: > > >>> I think there's a difference between how you've signed the pkgs and > how > > >> I did > > >>> it. I signed with sub-key (as I mentioned before) and yum doesn't > > >> recognize > > >>> it. Seemingly, it expects that the master key was used for signing. > > >>> > > >>> Also, in your repo file below > > >>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS > > >>> points to the old keys. The location should be > > >>> gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS > > >>> > > >>> I am pretty sure I have exported my key with --armor option back in > the > > >> day. > > >>> But I will repeat it and see if I can fix the situation, which I also > > >> observer > > >>> following your steps. If that's the only issue I will update the KEYS > > >> and we > > >>> should be completed by tonight ;) > > >>> > > >>> Thanks for your help! > > >>> Cos > > >>> > > >>>> On Wed, Sep 02, 2015 at 03:11PM, Evans Ye wrote: > > >>>> This is the same issue we're trying to solve in the mailing thread > > >>>> "convenience artifacts are signed and uploaded". I've built a sample > > >> repo > > >>>> which works properly by using my own key "Evans Ye" to sign and to > > >> export > > >>>> GPG KEY. So I believe the following steps should be the right way to > > >> sign > > >>>> packages and export the gpgkey: > > >>>> > > >>>> $ find -name *.rpm | xargs rpm --define="%_gpg_name Evans Ye" > --addsign > > >>>> > > >>>> $ gpg --armor --output KEYS --export 'Evans Ye' > > >>>> I've verified that the hash is matched now in our official repo. > > >>>> So I guess the main issue left is using non-armored gpg key, if we > > >> manually > > >>>> import the gpgkey in the repo file: > > >>>> > > >>>> [bigtop] > > >>>> name=Bigtop > > >>>> enabled=1 > > >>>> gpgcheck=1 > > >>>> type=NONE > > >>>> baseurl= > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/6/x86_64 > > >>>> gpgkey=http://archive.apache.org/dist/bigtop/KEYS > > >>>> > > >>>> [root@48723d98dc1b ~]# rpm --import > > >>>> https://dist.apache.org/repos/dist/release/bigtop/KEYS > > >>>> error: https://dist.apache.org/repos/dist/release/bigtop/KEYS: key > 2 > > >> not an > > >>>> armored public key. > > >>>> > > >>>> It gets error. > > >>>> However, my own exported armored key can be imported without an > error. > > >>>> That's the different. > > >>>> > > >>>> Can you confirm that the gpgkey( > > >> http://archive.apache.org/dist/bigtop/KEYS) > > >>>> is exported with --armor flag? > > >>>> > > >>>> 2015-09-02 13:25 GMT+08:00 Konstantin Boudnik <[email protected]>: > > >>>> > > >>>>> Looks like I have figured out what's wrong with my key. And it is > > >>>>> _nothing_. > > >>>>> However, it seems that I can not sign RPMs with subkey as YUM can > > >> not find > > >>>>> the > > >>>>> key while importing. Can anyone confirm or disprove my train of > > >> thoughts? > > >>>>> > > >>>>> Thanks! > > >>>>> Cos > > >>>>> > > >>>>>> On Wed, Sep 02, 2015 at 07:42AM, Konstantin Boudnik wrote: > > >>>>>> I've resynced the repodata once again and I don't see this issue > > >> on the > > >>>>>> centos7 anymore. However, yum still complains about the key being > > >> no > > >>>>>> available, but there's a workaround by setting gpgcheck=0 And I am > > >> going > > >>>>> to > > >>>>>> figure out what to do with it and why my key isn't working as > > >> expected. > > >>>>>> > > >>>>>> I also have discovered that the gpgkey file URL is using the old > > >>>>> incubation > > >>>>>> KEYS. Fixed that as well. > > >>>>>> > > >>>>>> Please let me know if you still see the issue with checksums > > >> mismatch. > > >>>>>> Thanks, > > >>>>>> Cos > > >>>>>> > > >>>>>>> On Tue, Sep 01, 2015 at 12:44PM, Konstantin Boudnik wrote: > > >>>>>>> I think this is the consequences of me fighting with the package > > >>>>> signing... ;( > > >>>>>>> A couple of days ago I have re-ran 'createrepo' for all the > > >> RPM-based > > >>>>> distros > > >>>>>>> and uploaded new repo files to the release. Not sure why the > > >> checksums > > >>>>> differ > > >>>>>>> now... > > >>>>>>> > > >>>>>>> I will take a look into this again tonight. > > >>>>>>> Cos > > >>>>>>> > > >>>>>>>> On Tue, Sep 01, 2015 at 09:39PM, Olaf Flebbe wrote: > > >>>>>>>> I can second it: > > >>>>>>>> > > >>>>>>>> I added to /etc/yum.repo.d/meins.repo > > >>>>>>>> > > >>>>>>>> [meins] > > >>>>>>>> name=Bigtop epo > > >>>>>>>> baseurl= > > >>>>> http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/ > > >>>>>>>> enabled=1 > > >>>>>>>> gpgcheck=0 > > >>>>>>>> priority=1 > > >>>>>>>> > > >>>>>>>> and got > > >>>>>>>> ............ > > >>>>>>>> Downloading packages: > > >>>>>>>> hbase-0.98.12-1.el7.centos.noa FAILED > > >>>>> =============================================-] 849 kB/s > > >> | 62 > > >>>>> MB 00:00:00 ETA > > >> > http://bigtop.s3.amazonaws.com/releases/1.0.0/centos/7/x86_64/hbase/noarch/hbase-0.98.12-1.el7.centos.noarch.rpm > > >> : > > >>>>> [Errno -1] Package does not match intended download. Suggestion: > run > > >> yum > > >>>>> --enablerepo=meins clean metadata > > >>>>>>>> Trying other mirror. > > >>>>>>>> ............. > > >>>>>>>> > > >>>>>>>> Olaf > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > (via Tom White) >
