Re: [Spacewalk-list] centos-errata.py and announcement list format changes
1) Allow Errata to be updated on the server. The workflow for this is complex as I think we can only call publish() once for each errata and it is difficult to tell when we are done with a particular errata. A good stopgap approach might be simply to save up all the publishing until the end and use spacecmd to publish all unpublished errata or something. As a bit of a test I'm going to give this a go later next week Before I do - have you put any more thought into dealing with this edge case of c5 and c6 systems - with updates for the same CESA potentially arriving at different times dependant on how the centos build servers go? James ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Thu, Feb 02, 2012 at 06:30:17PM +, Lopez, Abel wrote: I was able to process the 2012-January file using the 0.2 version of the errata script. Few steps required. I changed my channel type to sha256 for both my base and update channel, I did a simple `sed -ie 's/CentOS 6/CentOS 6 x86_64' 2012-January.txt` So there isn't a valid reason why spacewalk search can't work. That is good to know. I will try changing my own channel types and if successful will activate both the affected search strategies. Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
Dne 1.2.2012 19:02, David Nutter napsal(a): Done. I'll move the documentation across to the github Wiki when I get a minute. Regards, David, There seems to be problem with the latest errata. There seem to be erratas with the same ID: [CentOS-announce] CESA-2012:0093 Critical CentOS 4 php Update [CentOS-announce] CESA-2012:0093 Critical CentOS 5 php Update [CentOS-announce] CESA-2012:0093 Critical CentOS 6 php Update The script is not able to handle the same arrata ID on multiple channels as it seems to me. Thanks, DH ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
I was able to process the 2012-January file using the 0.2 version of the errata script. Few steps required. I changed my channel type to sha256 for both my base and update channel, I did a simple `sed -ie 's/CentOS 6/CentOS 6 x86_64' 2012-January.txt` So there isn't a valid reason why spacewalk search can't work. On 1/18/12 10:54 AM, David Nutter dav...@bioss.ac.uk wrote: On Wed, Jan 04, 2012 at 08:54:16PM +, David Nutter wrote: Hi folks, As some of you may have noticed the format of the centos-announce messages has changed, thus breaking my script that creates errata for them (http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/) I am working on a fix but unfortunately the changes required are fairly involved so it may be a day or two before I have a release ready and tested. Apologies for any inconvenience you may experience in the meantime. Hi folks, A new release is now available (finally): http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Due to format changes in the centos-announce mailing lists, the parsing logic has been completely rewritten. It's still horrible though :) Also, you will now have to use the dir package search strategy as the other two strategies spacewalk and satellitedir both rely on obtaining the md5sum from the centos-announce mailings. Since upstream now sends sha256 checksums these techniques can't work. I note with interest various discussions on the CentOS lists about making errata available in a more useful form. Hopefully my script won't be required for too much longer. Regards, -- David Nutter Tel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Tue, Jan 31, 2012 at 08:15:27PM +0100, David Hrbáč wrote: Dne 31.1.2012 16:38, David Nutter napsal(a): OK, a nascent version 0.8 is in the master branch at https://github.com/davidnutter/Centos-Errata Thanks, just tag it wit 0.8, so we can download the package easily. Done. I'll move the documentation across to the github Wiki when I get a minute. Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Fri, Jan 27, 2012 at 09:34:45PM +0100, David Hrbáč wrote: Dne 23.1.2012 12:24, David Nutter napsal(a): OK, a new release (0.7) is out which should correct this problem. Thanks to Stefan F??rster for reporting this issue and testing the fix. http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Regards, David, Thanks, it works now for me. A few points: - I'm running it on C5, had to install C4 and C6 signing keys. Did not need to with previous version. Odd. I don't think I check the signature anywhere, at least not directly. Best advice is to run all pushes on Centos6 I guess. We're still on 5 here so I must admit this aspect of the script is somewhat untested. - Did not helped for C6, since C5 rpm is not capable to work with C6 rpms. :o( So it imports the errata but keeps complaining: .snip error: rpmts_HdrFromFdno: Header V4 RSA/SHA1 signature: BAD, key ID c105b9de process_pkg_file failed with exception error reading package header. Traceback (most recent call last): File ./centos-errata.py, line 1104, in processRPMFile header = rpmQuery.hdrFromFdno(fd) error: error reading package header Warning: package /var/satellite/redhat/strace-4.5.19-1.11.el6_2.1.x86_64.rpm does not exist or cannot be read. Searching for package strace-4.5.19-1.11.el6_2.1.x86_64.rpm failed Skipping errata CEBA-2012:0028 ..snip - I'm attaching the patch to work with mail-archive. Great, thanks. I will apply that with a slight tweak - first change should be to the constant MessageMailArchive.MAILARCHIVE_BASE rather than the string literal. - Are you willing to go to GitHub with script development? Yes indeed! I should have done this months ago. I shall try and sort out an account later on today and will post a further reply once I've got everything working. Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
Dne 31.1.2012 12:12, David Nutter napsal(a): Yes indeed! I should have done this months ago. I shall try and sort out an account later on today and will post a further reply once I've got everything working. Great to read it. DH ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Tue, Jan 31, 2012 at 11:12:49AM +, David Nutter wrote: *snip* - Are you willing to go to GitHub with script development? Yes indeed! I should have done this months ago. I shall try and sort out an account later on today and will post a further reply once I've got everything working. OK, a nascent version 0.8 is in the master branch at https://github.com/davidnutter/Centos-Errata This includes: + fixes to the compile errors in error messages + patch to fix mailarchive.com parsing; my apologies for this error - I only tested with static testdata due to the SOPA blackout of mailarchive.com on the day I released version 0.7 and hence didn't exercise the problem code path. + Changed paths to testdata + Added various boilerplate text files. I'll get the docs on http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ updated just as soon as I may and port those documents into the git repo as well. At the moment I drive git like a granny so progress may be a little slow :) Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
Dne 31.1.2012 16:38, David Nutter napsal(a): OK, a nascent version 0.8 is in the master branch at https://github.com/davidnutter/Centos-Errata Thanks, just tag it wit 0.8, so we can download the package easily. This includes: + fixes to the compile errors in error messages + patch to fix mailarchive.com parsing; my apologies for this error - I only tested with static testdata due to the SOPA blackout of mailarchive.com on the day I released version 0.7 and hence didn't exercise the problem code path. + Changed paths to testdata + Added various boilerplate text files. Thanks for your work. DH ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
Dne 23.1.2012 12:24, David Nutter napsal(a): OK, a new release (0.7) is out which should correct this problem. Thanks to Stefan F??rster for reporting this issue and testing the fix. http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Regards, David, Thanks, it works now for me. A few points: - I'm running it on C5, had to install C4 and C6 signing keys. Did not need to with previous version. - Did not helped for C6, since C5 rpm is not capable to work with C6 rpms. :o( So it imports the errata but keeps complaining: .snip error: rpmts_HdrFromFdno: Header V4 RSA/SHA1 signature: BAD, key ID c105b9de process_pkg_file failed with exception error reading package header. Traceback (most recent call last): File ./centos-errata.py, line 1104, in processRPMFile header = rpmQuery.hdrFromFdno(fd) error: error reading package header Warning: package /var/satellite/redhat/strace-4.5.19-1.11.el6_2.1.x86_64.rpm does not exist or cannot be read. Searching for package strace-4.5.19-1.11.el6_2.1.x86_64.rpm failed Skipping errata CEBA-2012:0028 ..snip - I'm attaching the patch to work with mail-archive. - Are you willing to go to GitHub with script development? Thanks David Hrbáč --- centos-errata.py.txt2012-01-23 12:07:08.0 +0100 +++ centos-errata.py2012-01-27 20:19:35.0 +0100 @@ -1043,7 +1043,7 @@ mailarchive_url=file://%s/mailarchive-test/ % (os.path.abspath( os.path.dirname(sys.argv[0])+/ )) print Using test data at %s % mailarchive_url else: -mailarchive_url=MAILARCHIVE_BASE +mailarchive_url=http://www.mail-archive.com/centos-announce@centos.org/; try: messages_list_f = urllib.urlopen(mailarchive_url+maillist.html) @@ -1060,7 +1060,7 @@ subjects_match = MessageMailArchive.subjects_re.match(line) if len(self.parsedMessages) = self.options.max_errata: -print Max errata count %d exceeded. Processing no more errata % options.max_errata +print Max errata count %d exceeded. Processing no more errata % self.options.max_errata return self.parsedMessages if subjects_match is None: ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Thu, Jan 19, 2012 at 12:23:58PM +, David Nutter wrote: On Wed, Jan 18, 2012 at 06:54:47PM +, David Nutter wrote: *snip* I've been informed by a couple of people that there's a bug which occurs when using multiple architecture flavours of CentOS (e.g. x86_64 and i386) together. Essentially you only get errata published for the first one listed. Unfortunately I can't test this scenario as we're x86_64 only here so a fix will take a little while as any changes will have to be tested by an affected user. Working on it though :) OK, a new release (0.7) is out which should correct this problem. Thanks to Stefan F??rster for reporting this issue and testing the fix. http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Thu, Jan 19, 2012 at 8:30 AM, David Nutter david.nut...@bioss.ac.uk wrote: On Thu, Jan 19, 2012 at 11:49:43AM +, richard rigby wrote: hi david, excellent work - have just downloaded and updated, and all seems to be working well again, thanks for your efforts. Thanks :) i noticed there is a new requirement for python-lxml, which we didn't have installed, but all is now back up and running. Ah yes, that's just to aid the screen scraping of stuff downloaded from the RedHat site. I will mention it in the docs for version 0.7. Release is hopefully imminent, just awaiting test results now. Question - given that it focuses on the centos-announce mailing list, when the CentOS project gets caught in a version squeeze (like when RHEL 6.2 was released and CentOS didn't have 6.1 out the door yet), they activate the CR repository. Those announcements are made to the centos-cr-announce mailing list. Are there any plans to have an option to utilize those archives? Currently, I just make a copy of the centos-errata.py script and edit it by hand, but it's something I have to do for each version upgrade. Of course, if this script is eventually going away this may not be necessary, but... -I ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On 18/01/12 18:54, David Nutter wrote: On Wed, Jan 04, 2012 at 08:54:16PM +, David Nutter wrote: Hi folks, As some of you may have noticed the format of the centos-announce messages has changed, thus breaking my script that creates errata for them (http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/) I am working on a fix but unfortunately the changes required are fairly involved so it may be a day or two before I have a release ready and tested. Apologies for any inconvenience you may experience in the meantime. Hi folks, A new release is now available (finally): http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Due to format changes in the centos-announce mailing lists, the parsing logic has been completely rewritten. It's still horrible though :) Also, you will now have to use the dir package search strategy as the other two strategies spacewalk and satellitedir both rely on obtaining the md5sum from the centos-announce mailings. Since upstream now sends sha256 checksums these techniques can't work. I note with interest various discussions on the CentOS lists about making errata available in a more useful form. Hopefully my script won't be required for too much longer. Regards, hi david, excellent work - have just downloaded and updated, and all seems to be working well again, thanks for your efforts. i noticed there is a new requirement for python-lxml, which we didn't have installed, but all is now back up and running. thanks again, richard ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Wed, Jan 18, 2012 at 06:54:47PM +, David Nutter wrote: *snip* A new release is now available (finally): http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Due to format changes in the centos-announce mailing lists, the parsing logic has been completely rewritten. It's still horrible though :) Also, you will now have to use the dir package search strategy as the other two strategies spacewalk and satellitedir both rely on obtaining the md5sum from the centos-announce mailings. Since upstream now sends sha256 checksums these techniques can't work. I note with interest various discussions on the CentOS lists about making errata available in a more useful form. Hopefully my script won't be required for too much longer. Hi folks, I've been informed by a couple of people that there's a bug which occurs when using multiple architecture flavours of CentOS (e.g. x86_64 and i386) together. Essentially you only get errata published for the first one listed. Unfortunately I can't test this scenario as we're x86_64 only here so a fix will take a little while as any changes will have to be tested by an affected user. Working on it though :) Thanks regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Thu, Jan 19, 2012 at 11:49:43AM +, richard rigby wrote: hi david, excellent work - have just downloaded and updated, and all seems to be working well again, thanks for your efforts. Thanks :) i noticed there is a new requirement for python-lxml, which we didn't have installed, but all is now back up and running. Ah yes, that's just to aid the screen scraping of stuff downloaded from the RedHat site. I will mention it in the docs for version 0.7. Release is hopefully imminent, just awaiting test results now. Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
Re: [Spacewalk-list] centos-errata.py and announcement list format changes
On Wed, Jan 04, 2012 at 08:54:16PM +, David Nutter wrote: Hi folks, As some of you may have noticed the format of the centos-announce messages has changed, thus breaking my script that creates errata for them (http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/) I am working on a fix but unfortunately the changes required are fairly involved so it may be a day or two before I have a release ready and tested. Apologies for any inconvenience you may experience in the meantime. Hi folks, A new release is now available (finally): http://www.bioss.ac.uk/staff/davidn/spacewalk-stuff/ Due to format changes in the centos-announce mailing lists, the parsing logic has been completely rewritten. It's still horrible though :) Also, you will now have to use the dir package search strategy as the other two strategies spacewalk and satellitedir both rely on obtaining the md5sum from the centos-announce mailings. Since upstream now sends sha256 checksums these techniques can't work. I note with interest various discussions on the CentOS lists about making errata available in a more useful form. Hopefully my script won't be required for too much longer. Regards, -- David NutterTel: +44 (0)131 650 4888 BioSS, JCMB, King's Buildings, Mayfield Rd, EH9 3JZ. Scotland, UK Biomathematics and Statistics Scotland (BioSS) is formally part of The James Hutton Institute (JHI), a registered Scottish charity No. SC041796 and a company limited by guarantee No. SC374831 ___ Spacewalk-list mailing list Spacewalk-list@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-list
