I think I spotted an error.
In “errataImport.py” an errata may be ignored because a newer version has
already been found.
for errata in self.batch:
advisory = errata['advisory_name']
release = errata['advisory_rel']
if advisory in advisories:
if release < advisories[advisory]:
# Seen a newer one already
errata.ignored = 1
continue
The method
def _fix_erratum_packages_lookup
is the one that creates a “dict” and assigns it to “erratum[‘packages’]” later.
This fails for already “ignored” erratas
So adding this
--- errataImport.py.orig 2017-08-31 17:28:03.000000000 +0200
+++ errataImport.py 2017-09-29 10:53:24.520024239 +0200
@@ -132,6 +132,9 @@
self.backend.lookupPackages(list(self.packages.values()),
self.checksums, self.ignoreMissing)
for erratum in self.batch:
+ if erratum.ignored:
+ # Skip it
+ continue
self._fix_erratum_packages(erratum)
self._fix_erratum_file_channels(erratum)
at least fixes the “ERROR: ‘list’ object has no attribute ‘keys’” error that I
received when executing
self._fix_erratum_packages(erratum)
with a “erratum”, that has “ignored” set to ‘1’
I thought, I fixed it but then got a “constraint error” on
"rhn_errata_adv_org_uq" UNIQUE, btree (advisory, org_id) WHERE org_id IS NOT
NULL
I then erased *all* errata (still on test system 😉 ) and tried again, now
receiving a “constraint error” on
"rhn_errata_advname_org_uq" UNIQUE, btree (advisory_name, org_id) WHERE org_id
IS NOT NULL
And this is true for that old SUSE repository, because ‘advisory_name’ is
really pretty short.
Here is a “uniq -c” output from the erratas that the script tried to push into
the database
1 sdksp1-atk
2 sdksp1-audit
2 sdksp1-augeas
1 sdksp1-bash
5 sdksp1-bind
1 sdksp1-bluez
1 sdksp1-boost
3 sdksp1-build
1 sdksp1-bytefx-data-mysql
1 sdksp1-bzip2
2 sdksp1-cifs-mount
1 sdksp1-collectd-494
3 sdksp1-compat-dapl
2 sdksp1-compat-libldap-2_3-0
As you can see, this is really not unique.
Right now I’m trying to re-import the EPEL 7 repository and see, if this error
might only be triggered by “old” repositories.
Will update as soon as have more information.
Robert
Von: [email protected]
[mailto:[email protected]] Im Auftrag von Robert Paschedag
Gesendet: Donnerstag, 28. September 2017 16:08
An: [email protected]
Cc: [email protected]
Betreff: Re: [Spacewalk-list] Importing errata from RPM channel failes with
ERROR: 'list' object has no attribute 'keys'
Hmm...still don't know, why this errata is parsed different right now. Within
the updateinfo.xml.gz, this one (for examle) is imported correctly (as it looks)
<update from="[email protected]<mailto:[email protected]>"
status="stable" type="security" version="2740">
<id>sdksp1-tomcat6</id>
<title>Security update for tomcat6</title>
<release>SUSE Linux Enterprise Software Development Kit 11 SP1</release>
<issued date="1279521920" />
<references>
<reference
href="https://bugzilla.novell.com/show_bug.cgi?id=622188<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.novell.com%2Fshow_bug.cgi%3Fid%3D622188&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=VZ6eZfYKDKyTj%2FmBaZnOlkO5x7Xtv5WRKmNIBTm7fB4%3D&reserved=0>"
id="622188" title="bug number 622188" type="bugzilla" />
<reference
href="https://bugzilla.novell.com/show_bug.cgi?id=599554<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.novell.com%2Fshow_bug.cgi%3Fid%3D599554&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=4%2F0%2BiUfsLpzpG0hDuoqUy2YMqsEo1MhW4VvjbwYObkw%3D&reserved=0>"
id="599554" title="bug number 599554" type="bugzilla" />
<reference
href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-2227<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcve.mitre.org%2Fcgi-bin%2Fcvename.cgi%3Fname%3DCVE-2010-2227&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=2E97IUUlY8zjdColQ7vJYMs1huHpGL5wAAUj1%2BKMU80%3D&reserved=0>"
id="CVE-2010-2227" title="CVE-2010-2227" type="cve" />
<reference
href="http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1157<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcve.mitre.org%2Fcgi-bin%2Fcvename.cgi%3Fname%3DCVE-2010-1157&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=CZUHWNkXKukJhLHh6ip%2BQRQoryvAetMSzYSh9hAHVdg%3D&reserved=0>"
id="CVE-2010-1157" title="CVE-2010-1157" type="cve" />
</references>
<description>
This update of tomcat fixes denial of service and information disclosure
vulnerabilities which could potentially be exploited by remote attackers to
crash tomcat or to obtain sensitive information (CVE-2010-2227,
CVE-2010-1157).
Security Issues:
* CVE-2010-2227
<http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-2227>
* CVE-2010-1157
<http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-1157>
</description>
<pkglist>
<collection>
<package name="tomcat6" version="6.0.18" release="20.5.1" arch="noarch">
<filename>tomcat6-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-admin-webapps" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-admin-webapps-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-docs-webapp" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-docs-webapp-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-javadoc" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-javadoc-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-jsp-2_1-api" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-jsp-2_1-api-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-lib" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-lib-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-servlet-2_5-api" version="6.0.18"
release="20.5.1" arch="noarch">
<filename>tomcat6-servlet-2_5-api-6.0.18-20.5.1.noarch.rpm</filename>
</package>
<package name="tomcat6-webapps" version="6.0.18" release="20.5.1"
arch="noarch">
<filename>tomcat6-webapps-6.0.18-20.5.1.noarch.rpm</filename>
</package>
</collection>
</pkglist>
</update>
But this one not. In the errata above (and all others?), "erratum['packages']"
is a "dict()". And in the one below, it is a "list()", causing the error.
<update from="[email protected]<mailto:[email protected]>"
status="stable" type="recommended" version="4361">
<id>sdksp1-tomcat6</id>
<title>Recommended update for tomcat6</title>
<release>SUSE Linux Enterprise Software Development Kit 11 SP1</release>
<issued date="1302612141" />
<references>
<reference
href="https://bugzilla.novell.com/show_bug.cgi?id=681914<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.novell.com%2Fshow_bug.cgi%3Fid%3D681914&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=i112XGOP%2Brk%2BQZ0a76l0PAZuS1P7d5HaPnpvV%2FdOHFY%3D&reserved=0>"
id="681914" title="bug number 681914" type="bugzilla" />
</references>
<description>
This update fixes the Expression Language parser.
The parser was previously unable parse expressions without additional
whitespaces - like '${not(true)}'. This is addressed in apache bug #45511.
And the last character of namespace is lost during a parsing, which makes
this feature unusable. This is apache bug #45648. Both issues are fixed by
this update.
</description>
<pkglist>
<collection>
<package name="tomcat6" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-admin-webapps" version="6.0.18"
release="20.23.1" arch="noarch">
<filename>tomcat6-admin-webapps-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-docs-webapp" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-docs-webapp-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-javadoc" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-javadoc-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-jsp-2_1-api" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-jsp-2_1-api-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-lib" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-lib-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-servlet-2_5-api" version="6.0.18"
release="20.23.1" arch="noarch">
<filename>tomcat6-servlet-2_5-api-6.0.18-20.23.1.noarch.rpm</filename>
</package>
<package name="tomcat6-webapps" version="6.0.18" release="20.23.1"
arch="noarch">
<filename>tomcat6-webapps-6.0.18-20.23.1.noarch.rpm</filename>
</package>
</collection>
</pkglist>
</update>
Gesendet: Donnerstag, 28. September 2017 um 15:06 Uhr
Von: "Robert Paschedag"
<[email protected]<mailto:[email protected]>>
An: "Paul Robert Marino" <[email protected]<mailto:[email protected]>>
Cc: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Betreff: Re: [Spacewalk-list] Importing errata from RPM channel failes with
ERROR: 'list' object has no attribute 'keys'
this is exactly what I have done. This is a test system and I ran nightly
because of PR500 (debian and ubuntu package version handling) and that is what
I have tested. Now that 2.7 has been released, I wanted to upgrade to the
"release" and test all other stuff.
No....had started debugging and located the error in some "old" erratas.
Sorry for long output. But this is the line in "errataImport.py", where the
error occures. As you can see in this "one" errata I debugged now, "packages"
is truely a "list" of "IncompletePackage" instances and not a "dict".
263 # 'package in erratum['packages'].values()' here. But for (to
me) unknown
264 # reason it sometimes has package.id == None which makes whole
import fail.
265 # And self.packages[nevrao].id contains always right value.
266 for nevrao in erratum['packages'].keys():
(Pdb) p erratum
[<spacewalk.server.importlib.importLib.Erratum instance;
attributes={'advisory_name': 'sdksp1-tomcat6', 'advisory':
'sdksp1-tomcat6-4361', 'product': 'SUSE Linux Enterprise Software Development
Kit 11 SP1', 'description': "\nThis update fixes the Expression Language
parser.\n\nThe parser was previously unable parse expressions without
additional \nwhitespaces - like '${not(true)}'. This is addressed in apache bug
#45511. \nAnd the last character of namespace is lost during a parsing, which
makes \nthis feature unusable. This is apache bug #45648. Both issues are fixed
by \nthis update.\n\n", 'errata_from':
'[email protected]'<mailto:'[email protected]'>, 'locally_modified': None,
'refers_to': None, 'solution': ' ', 'topic': ' ', 'last_modified': None,
'keywords': [],
list starts...
'packages': [[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-6.0.18-20.23.1.noarch.rpm', 'channels': None, 'package_id': 21326,
'last_modified': None, 'version': '6.0.18', 'checksum_type': 'md5', 'release':
'20.23.1', 'checksums': {'md5': '5b887c3e6b5d060f6a1875fa50e897db'},
'checksum': '5b887c3e6b5d060f6a1875fa50e897db', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-admin-webapps', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-admin-webapps-6.0.18-20.23.1.noarch.rpm', 'channels': None,
'package_id': 21328, 'last_modified': None, 'version': '6.0.18',
'checksum_type': 'md5', 'release': '20.23.1', 'checksums': {'md5':
'8a68846141b10f46e3003ca74baa793b'}, 'checksum':
'8a68846141b10f46e3003ca74baa793b', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-docs-webapp', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-docs-webapp-6.0.18-20.23.1.noarch.rpm', 'channels': None,
'package_id': 21323, 'last_modified': None, 'version': '6.0.18',
'checksum_type': 'md5', 'release': '20.23.1', 'checksums': {'md5':
'58e759e5fccad6f265096341f323307d'}, 'checksum':
'58e759e5fccad6f265096341f323307d', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-javadoc', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-javadoc-6.0.18-20.23.1.noarch.rpm', 'channels': None, 'package_id':
21327, 'last_modified': None, 'version': '6.0.18', 'checksum_type': 'md5',
'release': '20.23.1', 'checksums': {'md5': '53dbb12db41537a1d8adb949b52b018e'},
'checksum': '53dbb12db41537a1d8adb949b52b018e', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-jsp-2_1-api', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-jsp-2_1-api-6.0.18-20.23.1.noarch.rpm', 'channels': None,
'package_id': 21324, 'last_modified': None, 'version': '6.0.18',
'checksum_type': 'md5', 'release': '20.23.1', 'checksums': {'md5':
'a72147cfba8d1e0f1ac9c1922d393d85'}, 'checksum':
'a72147cfba8d1e0f1ac9c1922d393d85', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-lib', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-lib-6.0.18-20.23.1.noarch.rpm', 'channels': None, 'package_id': 21330,
'last_modified': None, 'version': '6.0.18', 'checksum_type': 'md5', 'release':
'20.23.1', 'checksums': {'md5': 'f5b1e10d6d78331cb07526ebf8dee6bf'},
'checksum': 'f5b1e10d6d78331cb07526ebf8dee6bf', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-servlet-2_5-api', 'src':
None, 'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None,
'filename': 'tomcat6-servlet-2_5-api-6.0.18-20.23.1.noarch.rpm', 'channels':
None, 'package_id': 21329, 'last_modified': None, 'version': '6.0.18',
'checksum_type': 'md5', 'release': '20.23.1', 'checksums': {'md5':
'ea0410c4628b7e570412d319dac4f466'}, 'checksum':
'ea0410c4628b7e570412d319dac4f466', 'arch': 'noarch'}],
[<spacewalk.server.importlib.importLib.IncompletePackage instance;
attributes={'package_size': None, 'name': 'tomcat6-webapps', 'src': None,
'checksum_list': None, 'md5sum': None, 'org_id': 1, 'epoch': None, 'filename':
'tomcat6-webapps-6.0.18-20.23.1.noarch.rpm', 'channels': None, 'package_id':
21325, 'last_modified': None, 'version': '6.0.18', 'checksum_type': 'md5',
'release': '20.23.1', 'checksums': {'md5': '69436a9c144a06e0d7e47d71610e24e3'},
'checksum': '69436a9c144a06e0d7e47d71610e24e3', 'arch': 'noarch'}]],
list ended...
'files': [], 'advisory_type': 'Bug Fix Advisory', 'issue_date': '2011-04-12
14:42:21', 'notes': '', 'org_id': 1, 'bugs':
[[<spacewalk.server.importlib.importLib.Bug instance; attributes={'bug_id':
681914, 'href':
'https://bugzilla.novell.com/show_bug.cgi?id=681914'<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.novell.com%2Fshow_bug.cgi%3Fid%3D681914%27&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=8TMNd%2FCpCRWioAay0TV2sAw4wIDOmxv0BBVDNNqA9qU%3D&reserved=0>,
'summary': 'bug number 681914'}]], 'advisory_rel': '4361', 'synopsis':
'Recommended update for tomcat6', 'cve': [], 'update_date': '2011-04-12
14:42:21', 'channels': [{'label': 'sle11-sdk-sp1-updates-x86_64'}]}]
...
I wil try to remove that errata (and following, that break the import) and see,
if this is "reimported" correctly.
Robert
Gesendet: Donnerstag, 28. September 2017 um 14:12 Uhr
Von: "Paul Robert Marino" <[email protected]<mailto:[email protected]>>
An: "Robert Paschedag" <[email protected]<mailto:[email protected]>>
Cc: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Betreff: Re: Re: [Spacewalk-list] Importing errata from RPM channel failes with
ERROR: 'list' object has no attribute 'keys'
well and that may be the root of your problem. you should never run the nightly
version in production! Only run the nightly version in a QA/Dev environment
here you are specifically testing for the next release of spacewalk. if you run
the nightly version in production you are asking for things to break, while in
theory you should be able to upgrade fom a nightly to a new release that
doesn't mean that damage done by a broken nightly version will corrected.
the only time i ever run a nightly build is when I'm trying to test prior to a
new release or if i wrote a patch and want to test it against the latest
version of the code, and when i do it is usually an instance on a VM that
manages a few other VM's just to test functionality.
On Thu, Sep 28, 2017 at 6:49 AM, Robert Paschedag
<[email protected]<mailto:[email protected]>> wrote:
Damn....looks like I got this error before upgrading to 2.7 release. Went back
to snapshot (with 2.7 nightly) and this error is present. Will start to debug
this.
> Gesendet: Donnerstag, 28. September 2017 um 08:31 Uhr
> Von: "Robert Paschedag"
> <[email protected]<mailto:[email protected]>>
> An: [email protected]<mailto:[email protected]>, "Paul Robert
> Marino" <[email protected]<mailto:[email protected]>>, "Spacewalk
> Userlist" <[email protected]<mailto:[email protected]>>
> Betreff: Re: [Spacewalk-list] Importing errata from RPM channel failes with
> ERROR: 'list' object has no attribute 'keys'
>
> Am 27. September 2017 23:58:29 MESZ schrieb Paul Robert Marino
> <[email protected]<mailto:[email protected]>>:
> >_______________________________________________
> >Spacewalk-list mailing list
> >[email protected]<mailto:[email protected]>
> >https://www.redhat.com/mailman/listinfo/spacewalk-list<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=lDdfucY6OtLEwtnVmpkqlDHEfyM1oy5opLmOfaLr6fE%3D&reserved=0>
>
> This is 2.7. Just upgraded from nightly. But I'm not sure, if I had this
> error within nightly as I was testing other stuff all time.
>
>
> _______________________________________________
> Spacewalk-list mailing list
> [email protected]<mailto:[email protected]>
> https://www.redhat.com/mailman/listinfo/spacewalk-list<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=lDdfucY6OtLEwtnVmpkqlDHEfyM1oy5opLmOfaLr6fE%3D&reserved=0>
>
_______________________________________________ Spacewalk-list mailing list
[email protected]<mailto:[email protected]>
https://www.redhat.com/mailman/listinfo/spacewalk-list<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redhat.com%2Fmailman%2Flistinfo%2Fspacewalk-list&data=02%7C01%7CPaschedag.Netlution%40swr.de%7Cb9ad1a9f49cd4f82f55f08d5067dfa80%7Cbcca095d88d442f88260cc216b81f62d%7C0%7C0%7C636422060518083543&sdata=lDdfucY6OtLEwtnVmpkqlDHEfyM1oy5opLmOfaLr6fE%3D&reserved=0>
_______________________________________________
Spacewalk-list mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/spacewalk-list
