Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On 04/21/2011 09:40 AM, Miroslav Suchý wrote: > TypeError: object does not support item assignment I fixed that in commit 94c68cff702bdcfebe9d8d0b5f7874dff5159758 -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On 04/08/2011 04:19 PM, Jan Pazdziora wrote: > On Wed, Mar 23, 2011 at 03:25:43PM +0100, Michael Calmer wrote: >> >> Here is a new patchset. I had to do some more changes. To not loose the >> overview I have combined them into 4 patches. >> >> 0001: The database changes >> 0002: changes in the python code; mainly reposync and yum repo_plugin >> 0003: changes in java web pages >> 0004: changes in perl web pages >> >> I hope I included everything what was mentioned in this thread. >> I also did a short test with the nightly build (2 days ago) with a fedora 14 >> repository and a openSUSE 11.4 repository. > > Michael, > > I've now applied your patches to Spacewalk master. Packages for the > nightly repo are being built while I'm writing this so people will > have chance to see the feature in Spacewalk nightly later today, and > we will have chance to flesh out any issues before Spacewalk 1.4 is > out. > > Thank you for your contribution and I'm sorry that it took me so long > to get to your post. Guys, the patch 0002 - implement updateinfo => Errata import for spacewalk-repo-sync which was commited by Jan as 307d2c5606ab10561acbfa3fda89b5c0a2671233 contains bug. # spacewalk-repo-sync -c fedora14-i386-updates ... Traceback (most recent call last): File "/usr/bin/spacewalk-repo-sync", line 69, in ? sys.exit(abs(main() or 0)) File "/usr/bin/spacewalk-repo-sync", line 63, in main sync.main() File "/usr/lib/python2.4/site-packages/spacewalk/satellite_tools/reposync.py", line 110, in main self.import_updates(plugin, url) File "/usr/lib/python2.4/site-packages/spacewalk/satellite_tools/reposync.py", line 145, in import_updates self.upload_updates(notices) File "/usr/lib/python2.4/site-packages/spacewalk/satellite_tools/reposync.py", line 159, in upload_updates notice = self.fix_notice(notice) File "/usr/lib/python2.4/site-packages/spacewalk/satellite_tools/reposync.py", line 453, in fix_notice notice['version'] = new_version / 100 TypeError: object does not support item assignment Can you please provide fix for SPACEWALK-1.4 branch as soon as possible as this is blocking 1.4 release..? Thanks. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On Wed, Mar 23, 2011 at 03:25:43PM +0100, Michael Calmer wrote: > > Here is a new patchset. I had to do some more changes. To not loose the > overview I have combined them into 4 patches. > > 0001: The database changes > 0002: changes in the python code; mainly reposync and yum repo_plugin > 0003: changes in java web pages > 0004: changes in perl web pages > > I hope I included everything what was mentioned in this thread. > I also did a short test with the nightly build (2 days ago) with a fedora 14 > repository and a openSUSE 11.4 repository. Michael, I've now applied your patches to Spacewalk master. Packages for the nightly repo are being built while I'm writing this so people will have chance to see the feature in Spacewalk nightly later today, and we will have chance to flesh out any issues before Spacewalk 1.4 is out. Thank you for your contribution and I'm sorry that it took me so long to get to your post. Yours, -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
Hi, Am Freitag, 11. März 2011, 11:23:09 schrieb Jan Pazdziora: > On Fri, Mar 11, 2011 at 10:31:45AM +0100, Michael Calmer wrote: > > > > > > is at fault -- at least in that Fedora 7 updateinfo, the version > > > is not integer. When I've changed the line to > > > > > > e['advisory_rel'] = 1 > > > > > > things went much better. > > > > Yes, this is the version appended to advisory_name. Here is an example > > from openSUSE which shows the unique problem > > > > > > > > > version="1596"> > > flash-player > > > > [...] > > > version="1707"> > > flash-player > > > > > > Same id used but with a different version. I think we need to find a way > > to deal with all these differences in how updateinfo is used. > > Right. I've checked > > http://download.englab.brq.redhat.com/pub/fedora/linux/updates/14/i386/repodata/updateinfo.xml.gz > > and it still has version="1.4" there, so it looks like Fedora uses > that attribute to mean the version of the data structure (it's 1.4 > both on Fedora 7 and Fedora 14), not version of the errata. > > Can you come up with some proposal? From top of my head, we could add > a "type" of updatesinfo to the repository definition in the database > and use that, or use some heuristics, either checking that the id > does not end with /-\d+-\d+$/ or checking something else specific to > Fedora / *SuSE. yes. I will try to look into it next week. > > About the channel label added to the advisory_name. > > > > I think the correct way would be to search in DB if the errata already > > exist. > > If so, create the errata object from the DB and add the new packages and > > channels to it. > > Than update the errata in the database. > > Exactly. Fine. > > Doing this, we do not need to append the channel label to advisory_name. > > > > Is there already a method to create an Erratum object from the database? > > There has to be, as satellite-sync does exactly that. Unfortunately, > I won't be able to dig into it for the next week as I will be away > -- if you could investigate, that would be great. I think I will find this. Thanks for the hint. -- Regards, Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
On Fri, Mar 11, 2011 at 10:31:45AM +0100, Michael Calmer wrote: > > > > is at fault -- at least in that Fedora 7 updateinfo, the version > > is not integer. When I've changed the line to > > > > e['advisory_rel'] = 1 > > > > things went much better. > > Yes, this is the version appended to advisory_name. Here is an example from > openSUSE which shows the unique problem > > > > version="1596"> > flash-player > > [...] > version="1707"> > flash-player > > > Same id used but with a different version. I think we need to find a way to > deal with all these differences in how updateinfo is used. Right. I've checked http://download.englab.brq.redhat.com/pub/fedora/linux/updates/14/i386/repodata/updateinfo.xml.gz and it still has version="1.4" there, so it looks like Fedora uses that attribute to mean the version of the data structure (it's 1.4 both on Fedora 7 and Fedora 14), not version of the errata. Can you come up with some proposal? From top of my head, we could add a "type" of updatesinfo to the repository definition in the database and use that, or use some heuristics, either checking that the id does not end with /-\d+-\d+$/ or checking something else specific to Fedora / *SuSE. > About the channel label added to the advisory_name. > > I think the correct way would be to search in DB if the errata already exist. > If so, create the errata object from the DB and add the new packages and > channels to it. > Than update the errata in the database. Exactly. > Doing this, we do not need to append the channel label to advisory_name. > > Is there already a method to create an Erratum object from the database? There has to be, as satellite-sync does exactly that. Unfortunately, I won't be able to dig into it for the next week as I will be away -- if you could investigate, that would be great. -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
Hi, Am Freitag, 11. März 2011, 09:58:39 schrieb Jan Pazdziora: > On Fri, Mar 11, 2011 at 09:40:25AM +0100, Jan Pazdziora wrote: > > [...] > > _buildExternalValue(extObject, object, parentTableObj) > > File "/usr/lib/python2.6/site- > > packages/spacewalk/server/importlib/backend.py", line 1982, in > > _buildExternalValue > > dict[f] = sanitizeValue(entry[attr], datatype) > > File "/usr/lib/python2.6/site- > > packages/spacewalk/server/importlib/backendLib.py", line 494, in > > sanitizeValue > > return int(value) > > ValueError: invalid literal for int() with base 10: '1.4' > > > > I assume the 1.4 value is the version you've put to the errata > > name as the updateinfo.xml.gz has 1.4 in version -- it starts like > > > > > > > >> type="bugfix" version="1.4"> > > FEDORA-2007-2609 > > filezilla-3.0.2.1-1.fc7 > > Fedora 7 > > > > > > > > I further assume that that value somehow gets massaged into epoch > > column somewhere, or some other integer field. > > It turns out, the > > e['advisory_rel'] = notice['version'] > > is at fault -- at least in that Fedora 7 updateinfo, the version > is not integer. When I've changed the line to > > e['advisory_rel'] = 1 > > things went much better. Yes, this is the version appended to advisory_name. Here is an example from openSUSE which shows the unique problem flash-player [...] flash-player Same id used but with a different version. I think we need to find a way to deal with all these differences in how updateinfo is used. About the channel label added to the advisory_name. I think the correct way would be to search in DB if the errata already exist. If so, create the errata object from the DB and add the new packages and channels to it. Than update the errata in the database. Doing this, we do not need to append the channel label to advisory_name. Is there already a method to create an Erratum object from the database? -- Regards, Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) signature.asc Description: This is a digitally signed message part. ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
On Fri, Mar 11, 2011 at 09:40:25AM +0100, Jan Pazdziora wrote: > > By the way, when I've removed some safechecks to actually see some > erratas being created from > > > http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/repodata/updateinfo.xml.gz > > I get traceback > > No cheksum found for hydrogen-0.9.3-9.fc7.1.src. Skipping Patch > FEDORA-2007-3554-1.4-channel-ia32 > Traceback (most recent call last): > File "/usr/bin/spacewalk-repo-sync", line 69, in > sys.exit(abs(main() or 0)) > File "/usr/bin/spacewalk-repo-sync", line 63, in main > sync.main() > File > "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", > line 111, in main > self.import_updates(plugin, url) > File > "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", > line 146, in import_updates > self.upload_updates(notices) > File > "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", > line 297, in upload_updates > importer.run() > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/importLib.py", > line 622, in run > self.submit() > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/errataImport.py", > line 196, in submit > dml = self.backend.processErrata(self.batch) > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", > line 682, in processErrata > transactional=1) > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", > line 1364, in __processObjectCollection > return self.__processObjectCollection__(objColl, parentTable, > childDict, **kwargs) > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", > line 1450, in __processObjectCollection__ > _buildExternalValue(extObject, object, parentTableObj) > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", > line 1982, in _buildExternalValue > dict[f] = sanitizeValue(entry[attr], datatype) > File > "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backendLib.py", > line 494, in sanitizeValue > return int(value) > ValueError: invalid literal for int() with base 10: '1.4' > > I assume the 1.4 value is the version you've put to the errata > name as the updateinfo.xml.gz has 1.4 in version -- it starts like > > > > type="bugfix" version="1.4"> > FEDORA-2007-2609 > filezilla-3.0.2.1-1.fc7 > Fedora 7 > > > > I further assume that that value somehow gets massaged into epoch > column somewhere, or some other integer field. It turns out, the e['advisory_rel'] = notice['version'] is at fault -- at least in that Fedora 7 updateinfo, the version is not integer. When I've changed the line to e['advisory_rel'] = 1 things went much better. -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
On Tue, Mar 08, 2011 at 05:57:05PM +0100, Michael Calmer wrote: > Hi, > > Am Montag, 7. März 2011, 15:49:59 schrieb Miroslav Suchý: > > On 03/07/2011 02:28 PM, Michael Calmer wrote: > > > Hello, > > [...] > > > Regarding patch 0001-implement-updateinfo-Errata-import-5.patch... > > It seems to me that it is merge of 4 smaller commits and therefore the > > commit description is quite misleading... > > I would expectd something like: > > Decide if package is errata based on updateinfo > > Well, this was the initial patch for this feature. So I think > "implement updateinfo to Errata import" is a good headline. > The 2 additional patches merged into this first commit were smaller bugfixes > for the initial patch. > > > and something more how the detection works and which all information you > > extract from updateinfo. > > * fix errata override bug. Create a unique advisory_name > This was about the element in the updateinfo which is not unique. > To get it unique we append notice['version'] to it. The next hit was, By the way, when I've removed some safechecks to actually see some erratas being created from http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/repodata/updateinfo.xml.gz I get traceback No cheksum found for hydrogen-0.9.3-9.fc7.1.src. Skipping Patch FEDORA-2007-3554-1.4-channel-ia32 Traceback (most recent call last): File "/usr/bin/spacewalk-repo-sync", line 69, in sys.exit(abs(main() or 0)) File "/usr/bin/spacewalk-repo-sync", line 63, in main sync.main() File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 111, in main self.import_updates(plugin, url) File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 146, in import_updates self.upload_updates(notices) File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 297, in upload_updates importer.run() File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/importLib.py", line 622, in run self.submit() File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/errataImport.py", line 196, in submit dml = self.backend.processErrata(self.batch) File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line 682, in processErrata transactional=1) File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line 1364, in __processObjectCollection return self.__processObjectCollection__(objColl, parentTable, childDict, **kwargs) File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line 1450, in __processObjectCollection__ _buildExternalValue(extObject, object, parentTableObj) File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line 1982, in _buildExternalValue dict[f] = sanitizeValue(entry[attr], datatype) File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backendLib.py", line 494, in sanitizeValue return int(value) ValueError: invalid literal for int() with base 10: '1.4' I assume the 1.4 value is the version you've put to the errata name as the updateinfo.xml.gz has 1.4 in version -- it starts like FEDORA-2007-2609 filezilla-3.0.2.1-1.fc7 Fedora 7 I further assume that that value somehow gets massaged into epoch column somewhere, or some other integer field. It looks like another reason why we really should try to find alternative approaches to the uniqueness problem, if there indeed is one. Perhaps some command line option? -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On Mon, Mar 07, 2011 at 02:28:48PM +0100, Michael Calmer wrote:
> Hello,
>
> the attached patch set implements updateinfo to errata import in spacewalk-
> repo-sync.
[...]
> diff --git a/backend/satellite_tools/reposync.py
> b/backend/satellite_tools/reposync.py
> index 2fc46a0..ee3be6e 100644
> --- a/backend/satellite_tools/reposync.py
> +++ b/backend/satellite_tools/reposync.py
[...]
> -from spacewalk.server.importlib.importLib import IncompletePackage
> +from spacewalk.common.rhn_mpm import InvalidPackageError
> +from spacewalk.server.importlib.importLib import IncompletePackage, Erratum,
> Checksum, Bug, Keyword
> +from spacewalk.server.importlib.backendOracle import OracleBackend
[...]
> +
> + backend = OracleBackend()
> + backend.init()
> + importer = ErrataImport(batch, backend)
> + importer.run()
Michael,
this fails on PostgreSQL database backend with traceback
Traceback (most recent call last):
File "/usr/bin/spacewalk-repo-sync", line 69, in
sys.exit(abs(main() or 0))
File "/usr/bin/spacewalk-repo-sync", line 63, in main
sync.main()
File
"/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line
111, in main
self.import_updates(plugin, url)
File
"/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line
146, in import_updates
self.upload_updates(notices)
File
"/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line
296, in upload_updates
backend.init()
File
"/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backendOracle.py",
line 573, in init
self.setDateFormat("-MM-DD HH24:MI:SS")
File
"/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line
80, in setDateFormat
sth.execute()
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py",
line 163, in execute
return apply(self._execute_wrapper, (self._execute, ) + p, kw)
File
"/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py",
line 268, in _execute_wrapper
raise rhnSQL.SQLStatementPrepareError(0, str(e), self.sql)
spacewalk.server.rhnSQL.sql_base.SQLStatementPrepareError: ('syntax error at or
near "session"\nLINE 1: alter session set nls_date_format =\'-MM-DD
HH24:MI:SS\'\n ^\n', 0, "alter session set nls_date_format
='-MM-DD HH24:MI:SS'")
You shouldn't need to explicitly use OracleBackend. Instead, we
now have the SQLBackend, see commit
7d5ed504e7e809fdcc7c82f8b5e82c37279200dd
where it was introduced. So I assume something like (untested)
--- satellite_tools/reposync.py.orig2011-03-09 05:36:51.0 -0500
+++ satellite_tools/reposync.py 2011-03-10 06:05:32.213461147 -0500
@@ -288,8 +288,7 @@
if not error:
batch.append(e)
- backend = OracleBackend()
- backend.init()
+ backend = SQLBackend()
importer = ErrataImport(batch, backend)
importer.run()
self.regen = True
could do the trick.
--
Jan Pazdziora
Principal Software Engineer, Satellite Engineering, Red Hat
___
Spacewalk-devel mailing list
Spacewalk-devel@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in?spacewalk-repo-sync
On Tue, Mar 08, 2011 at 05:57:05PM +0100, Michael Calmer wrote: > > * fix errata override bug. Create a unique advisory_name > This was about the element in the updateinfo which is not unique. Not unique where, across what domain? When I take http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/repodata/updateinfo.xml.gz and see errata FEDORA-2007-2609 there I expect that errata to land in Spacewalk as FEDORA-2007-2609, and not as FEDORA-2007-2609-1.4 or FEDORA-2007-2609-1.4-channel-ia32. When does this get non-unique? > To get it unique we append notice['version'] to it. The next hit was, > that patches may be created for mulitarch repositories, but in spacewalk we > have single arch channels. This can lead into duplicate updates where only > the > package list differ because of the architectures. So we also append > self.channel['arch'] to advisory and advisory_name This is not how erratas work in Spacewalk/Satellite. Take this scenario: You have Satellite with i386, x86_64, and s390x channels synced. You make a dump with rhn-satellite-exporter with all the channels. Erratas in that dump will reference packages from all arches. Then on another Satellite/Spacewalk you satellite-sync that dump, but only channel i386. This will create the erratas and link them to 32bit packages (because you have no x86_64 and s390x channels / packages synced). And then you sync the s390x channel. And the s390x packages have to end up added to the same erratas that have been created when syncing the i386 channels. You can see it even if you have one dump with one arch, and second dump with another arch, for example if you use channel dumps from RHN. > * we need to search for the package in the correct channel. > > It may happen, that the same nevra exists multiple times in rhnPackage. > The only difference is the checksum. They are assigned to different > channels. So we need to take care during errata import to get the > right package added to the errata. How do you do that? What if I run the spacewalk-repo-sync with multiple -c options? -- Jan Pazdziora Principal Software Engineer, Satellite Engineering, Red Hat ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On Mon, Mar 07, 2011 at 02:28:48PM +0100, Michael Calmer wrote: > Hello, > > the attached patch set implements updateinfo to errata import in spacewalk- > repo-sync. > This patch also requires two small changes to yum. Patches are also attached. > > This patch include also two enhancements. It adds errata_from to store the > "from" attribute of the updateinfo into the database. > Seconds it adds a URL attribute to bugs which make it possible to show links > in the web page also for 3rd party bugs linked to other bugzilla instances > than bugzilla.redhat.com. Michael, I've scratch built packages with your patches. Then I've run spacewalk-repo-sync -c fedora-7-i386-updates where channel fedora-7-i386-updates has repository http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/ configured to it. After syncing the packages, I get the following traceback: 4752/4754 : xscreensaver-gl-extras-gss-5.05-3.fc7-1.i386 4753/4754 : jack-audio-connection-kit-devel-0.103.0-1.fc7-0.i386 4754/4754 : python-logilab-astng-0.17.1-1.fc7-0.noarch Traceback (most recent call last): File "/usr/bin/spacewalk-repo-sync", line 69, in sys.exit(abs(main() or 0)) File "/usr/bin/spacewalk-repo-sync", line 63, in main sync.main() File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 111, in main self.import_updates(plugin, url) File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 143, in import_updates notices = plug.get_updates() File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/repo_plugins/yum_src.py", line 107, in get_updates um.add(self.repo, all=True) TypeError: add() got an unexpected keyword argument 'all' which I assume stems from the fact that I don't have the patch yum-3.2.20-all-updateinfos.patch applied. My yum is # rpm -qf /usr/lib/python2.6/site-packages/yum/update_md.py yum-3.2.29-7.el6.noarch What is the current plan of getting the patches needed to yum? Maybe we could have a subclass of UpdateMetadata which would override the add method? When I remove the all=True in the call to add, I get Repo http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/ has 2320 patches. No cheksum found for nfs-utils-1.1.0-4.fc7.ppc64. Skipping Patch FEDORA-2007-2206-1.4-channel-ia32 What exactly does this mean? I assume the issue is that I've been syncing i386 channel and this talks about ppc64. However, that shouldn't be a reason for skipping the whole FEDORA-2007-2206, should it? It contains http://download.fedoraproject.org/pub/fedora/linux/updates/7/i386/nfs-utils-debuginfo-1.1.0-4.fc7.i386.rpm"; version="1.1.0"> nfs-utils-debuginfo-1.1.0-4.fc7.i386.rpm in its pkglist/collection. By the way, we will want the cheksum typo addressed as well. I also can see that you are using "patch" where Red Hat and Fedora traditionally use the term "errata". I would probably prefer to have the terminology in the Spacewalk consistent. Can we make it erratas, and possibly come up with some additional product branding approaches which would make it easy to flip erratas to patches easily, without patching the sources. Anyway, the spacewalk-repo-sync without the all=True then continues and fails with No cheksum found for apr-util-debuginfo-1.2.10-1.fc7.ppc64. Skipping Patch FEDORA-2007-2207-1.4-channel-ia32 Traceback (most recent call last): File "/usr/bin/spacewalk-repo-sync", line 69, in sys.exit(abs(main() or 0)) File "/usr/bin/spacewalk-repo-sync", line 63, in main sync.main() File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 111, in main self.import_updates(plugin, url) File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 146, in import_updates self.upload_updates(notices) File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/reposync.py", line 270, in upload_updates bzs = filter(lambda r: r['type'] == 'bugzilla', notice['references']) TypeError: 'NoneType' object is not iterable I can see in that http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/7/i386/repodata/updateinfo.xml.gz file that the references element can be empty. Can you please address this issue? My impulse fix would be along the lines of --- satellite_tools/reposync.py.orig2011-03-09 05:36:51.0 -0500 +++ satellite_tools/reposync.py 2011-03-10 05:09:14.537381370 -0500 @@ -269,3 +269,5 @@ e['bugs'] = [] -bzs = filter(lambda r: r['type'] == 'bugzilla', notice['references']) +bzs = (
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
Hi, Am Mittwoch, 9. März 2011, 11:29:45 schrieb Jan Pazdziora: > On Mon, Mar 07, 2011 at 02:28:48PM +0100, Michael Calmer wrote: [...] > when running git am, I get the following whitespace-related warnings. > Could you please polish the patch to get rid of them? Sure, I will work on all the issues as soon as time permits. May take some days. -- Regards, Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On Mon, Mar 07, 2011 at 02:28:48PM +0100, Michael Calmer wrote:
> Hello,
>
> the attached patch set implements updateinfo to errata import in spacewalk-
> repo-sync.
> This patch also requires two small changes to yum. Patches are also attached.
Michael,
when running git am, I get the following whitespace-related warnings.
Could you please polish the patch to get rid of them?
Applying: implement updateinfo => Errata import
/home/adelton/project/spacewalk/.git/rebase-apply/patch:29: trailing
whitespace.
/home/adelton/project/spacewalk/.git/rebase-apply/patch:81: trailing
whitespace.
/home/adelton/project/spacewalk/.git/rebase-apply/patch:84: trailing
whitespace.
typemap = {
/home/adelton/project/spacewalk/.git/rebase-apply/patch:112: trailing
whitespace.
/home/adelton/project/spacewalk/.git/rebase-apply/patch:127: trailing
whitespace.
warning: squelched 6 whitespace errors
warning: 11 lines add whitespace errors.
Applying: rhnErrate table changed
/home/adelton/project/spacewalk/.git/rebase-apply/patch:65: trailing
whitespace.
--
/home/adelton/project/spacewalk/.git/rebase-apply/patch:87: trailing
whitespace.
--
warning: 2 lines add whitespace errors.
Applying: Add 'from' and 'href' to errata
/home/adelton/project/spacewalk/.git/rebase-apply/patch:199: trailing
whitespace.
--
warning: 1 line adds whitespace errors.
Applying: fixes and use new errata fields for updateinfo creation
/home/adelton/project/spacewalk/.git/rebase-apply/patch:112: trailing
whitespace.
/home/adelton/project/spacewalk/.git/rebase-apply/patch:117: trailing
whitespace.
warning: 2 lines add whitespace errors.
Thank you,
--
Jan Pazdziora
Principal Software Engineer, Satellite Engineering, Red Hat
___
Spacewalk-devel mailing list
Spacewalk-devel@redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
Dne 8.3.2011 17:57, Michael Calmer napsal(a): Hi, Am Montag, 7. März 2011, 15:49:59 schrieb Miroslav Suchý: On 03/07/2011 02:28 PM, Michael Calmer wrote: Hello, [...] Regarding patch 0001-implement-updateinfo-Errata-import-5.patch... It seems to me that it is merge of 4 smaller commits and therefore the commit description is quite misleading... I would expectd something like: Decide if package is errata based on updateinfo Well, this was the initial patch for this feature. So I think "implement updateinfo to Errata import" is a good headline. The 2 additional patches merged into this first commit were smaller bugfixes for the initial patch. and something more how the detection works and which all information you extract from updateinfo. * fix errata override bug. Create a unique advisory_name This was about the element in the updateinfo which is not unique. To get it unique we append notice['version'] to it. The next hit was, that patches may be created for mulitarch repositories, but in spacewalk we have single arch channels. This can lead into duplicate updates where only the package list differ because of the architectures. So we also append self.channel['arch'] to advisory and advisory_name * we need to search for the package in the correct channel. It may happen, that the same nevra exists multiple times in rhnPackage. The only difference is the checksum. They are assigned to different channels. So we need to take care during errata import to get the right package added to the errata. This was a fix for the big select statement which searches for the checksum of a package. The updateinfo only contains the package nevra and and errata wants the package checksum. So we need to search for the package in the DB. But if we do not include the channel as search criteria, we may find the wrong package which exists in a different channel. The complete patchset uses the following updateinfo elements: id: used in "advisory" and "advisory_name" version: used in "advisory" and "advisory_name" and set to "advisory_rel" type: is translated into spacewalk errata types and put into "advisory_type" title: set to "synopsis" release: set to "product" issued: converted to db date and put into "issue_date" references: bugzilla and CVE numbers are converted to the errata tables description: set to "description" The package list is evaluated and the packages are linked to the errata reboot_suggested: this flag is added as keyword to errata restart_suggested: this flag is added as keyword to errata from: we enhanced the errata table and add the column errata_from to have a place where the author/vendor could be stored. The errata fields "topic" and "solution" are set to " "(empty) because there is no equivalent in updateinfo (yet). Can you amend that commit? Or create BZ where you put this explanation and refer it from that commit? Mirek ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
Hi, Am Montag, 7. März 2011, 15:49:59 schrieb Miroslav Suchý: > On 03/07/2011 02:28 PM, Michael Calmer wrote: > > Hello, [...] > Regarding patch 0001-implement-updateinfo-Errata-import-5.patch... > It seems to me that it is merge of 4 smaller commits and therefore the > commit description is quite misleading... > I would expectd something like: > Decide if package is errata based on updateinfo Well, this was the initial patch for this feature. So I think "implement updateinfo to Errata import" is a good headline. The 2 additional patches merged into this first commit were smaller bugfixes for the initial patch. > and something more how the detection works and which all information you > extract from updateinfo. * fix errata override bug. Create a unique advisory_name This was about the element in the updateinfo which is not unique. To get it unique we append notice['version'] to it. The next hit was, that patches may be created for mulitarch repositories, but in spacewalk we have single arch channels. This can lead into duplicate updates where only the package list differ because of the architectures. So we also append self.channel['arch'] to advisory and advisory_name * we need to search for the package in the correct channel. It may happen, that the same nevra exists multiple times in rhnPackage. The only difference is the checksum. They are assigned to different channels. So we need to take care during errata import to get the right package added to the errata. This was a fix for the big select statement which searches for the checksum of a package. The updateinfo only contains the package nevra and and errata wants the package checksum. So we need to search for the package in the DB. But if we do not include the channel as search criteria, we may find the wrong package which exists in a different channel. The complete patchset uses the following updateinfo elements: id: used in "advisory" and "advisory_name" version: used in "advisory" and "advisory_name" and set to "advisory_rel" type: is translated into spacewalk errata types and put into "advisory_type" title: set to "synopsis" release: set to "product" issued: converted to db date and put into "issue_date" references: bugzilla and CVE numbers are converted to the errata tables description: set to "description" The package list is evaluated and the packages are linked to the errata reboot_suggested: this flag is added as keyword to errata restart_suggested: this flag is added as keyword to errata from: we enhanced the errata table and add the column errata_from to have a place where the author/vendor could be stored. The errata fields "topic" and "solution" are set to " "(empty) because there is no equivalent in updateinfo (yet). -- MFG Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
Hi, Am Montag, 7. März 2011, 15:05:36 schrieb Miroslav Suchý: > On 03/07/2011 02:28 PM, Michael Calmer wrote: > > Hello, > > > > the attached patch set implements updateinfo to errata import in > > spacewalk-repo-sync. > > This patch also requires two small changes to yum. Patches are also > > attached. > > > > This patch include also two enhancements. It adds errata_from to store the > > "from" attribute of the updateinfo into the database. > > Seconds it adds a URL attribute to bugs which make it possible to show > > links in the web page also for 3rd party bugs linked to other bugzilla > > instances than bugzilla.redhat.com. > > Unfortunately Spacewalk is not upstream for yum. We are just upstream > for yum-rhn-plugin. > But if I'm reading those patches correctly it should not have affect all > other patches, right? If you do not apply patch 0008, you may get only the newest patch for a package. This also depends on how the in updateinfo is created. SUSE uses often the same id for the same package (e.g. kernel patch) and increase the version number. The idea of how id should be used seems to be different :-) yum create a hash using id as a key. This has the consequence, that only the last patch (normally highest version number) is part of the returned notices. Older are ignored. The "restart_suggested" may need small changes to patch 0001. I am not a python guru, but I think if you call if notice['restart_suggested']: and the key "restart_suggested" does not exist, you will get a KeyError. So maybe something like if "restart_suggested" in notice and notice['restart_suggested']: would workaround this. > -- > Miroslav Suchy > Red Hat Satellite Engineering > > ___ > Spacewalk-devel mailing list > Spacewalk-devel@redhat.com > https://www.redhat.com/mailman/listinfo/spacewalk-devel > -- MFG Michael Calmer -- Michael Calmer SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg T: +49 (0) 911 74053 0 F: +49 (0) 911 74053575 - e-mail: michael.cal...@suse.com -- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) signature.asc Description: This is a digitally signed message part. ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On 03/07/2011 02:28 PM, Michael Calmer wrote: > Hello, > > the attached patch set implements updateinfo to errata import in spacewalk- > repo-sync. Regarding patch 0001-implement-updateinfo-Errata-import-5.patch... It seems to me that it is merge of 4 smaller commits and therefore the commit description is quite misleading... I would expectd something like: Decide if package is errata based on updateinfo and something more how the detection works and which all information you extract from updateinfo. -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
Re: [Spacewalk-devel] [PATCH] updateinfo to Errata import in spacewalk-repo-sync
On 03/07/2011 02:28 PM, Michael Calmer wrote: > Hello, > > the attached patch set implements updateinfo to errata import in spacewalk- > repo-sync. > This patch also requires two small changes to yum. Patches are also attached. > > This patch include also two enhancements. It adds errata_from to store the > "from" attribute of the updateinfo into the database. > Seconds it adds a URL attribute to bugs which make it possible to show links > in the web page also for 3rd party bugs linked to other bugzilla instances > than bugzilla.redhat.com. Unfortunately Spacewalk is not upstream for yum. We are just upstream for yum-rhn-plugin. But if I'm reading those patches correctly it should not have affect all other patches, right? -- Miroslav Suchy Red Hat Satellite Engineering ___ Spacewalk-devel mailing list Spacewalk-devel@redhat.com https://www.redhat.com/mailman/listinfo/spacewalk-devel
