Andreas, Thanks for being in touch. We're beyond the point of accepting code changes for 2.1, but we would be happy to receive documentation improvements, including workarounds such as yours. Please just make sure your pull request is against the pulp-2.1 branch.
Thanks, Michael ----- Original Message ----- From: "Andreas Piesk" <[email protected]> To: [email protected] Sent: Sunday, March 16, 2014 5:27:16 PM Subject: [Pulp-list] Upgrade from v1 to v2 failed Hello list, i tried to upgrade from v1 (1.1.15) to v2 (2.1.3), unfortunately it failed upgrading the database. these are the issue i encountered: 1. missing package attributes some packages miss some attributes which pulp depends on. # pulp-v1-upgrade = Upgrading Database = .. Upgrading: Packages, Errata, and Distributions Traceback (most recent call last): File "/usr/bin/pulp-v1-upgrade", line 50, in <module> upgrader.upgrade() File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py", line 187, in upgrade self._upgrade_database() File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py", line 228, in _upgrade_database report = db_call(v1_database, tmp_database) File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line 121, in upgrade rpms_success = _rpms(v1_database, v2_database, report) File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line 217, in _rpms return _packages(v1_database, v2_database, rpm_coll, all_rpms, 'rpm', report) File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line 247, in _packages 'description' : v1_rpm['description'], KeyError: 'description' this is one of the the culprits: SON([(u'repoids', [u'oracle-jdk-x86_64']), (u'checksum', SON([(u'sha256', u'338a454a36b034bc1373dbdda00d99a3eeba184fb2b690d3a81adb3825812727')])), (u'filename', u'jdk-1.6.0_33-fcs.x86_64.rpm'), (u'epoch', u'2000'), (u'version', u'1.6.0_33'), (u'_ns', u'packages'), (u'release', u'fcs'), (u'_id', u'f516b651-f958-4d22-bd1d-e7d8ee66bcc4'), (u'arch', u'x86_64'), (u'id', u'f516b651-f958-4d22-bd1d-e7d8ee66bcc4'), (u'name', u'jdk')]) i fixed it by assigning defaults: # diff -Purp /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py --- /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org 2013-08-18 21:42:25.723204539 +0200 +++ /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py 2013-08-18 22:02:27.581580546 +0200 @@ -244,13 +244,13 @@ def _packages(v1_database, v2_database, 'version' : v1_rpm['version'], 'release' : v1_rpm['release'], 'arch' : v1_rpm['arch'], - 'description' : v1_rpm['description'], - 'vendor' : v1_rpm['vendor'], + 'description' : v1_rpm.get('description',None), + 'vendor' : v1_rpm.get('vendor', None), 'filename' : v1_rpm['filename'], - 'requires' : v1_rpm['requires'], - 'provides' : v1_rpm['provides'], - 'buildhost' : v1_rpm['buildhost'], - 'license' : v1_rpm['license'], + 'requires' : v1_rpm.get('requires', None), + 'provides' : v1_rpm.get('provides', None), + 'buildhost' : v1_rpm.get('buildhost', None), + 'license' : v1_rpm.get('license', None), '_id' : new_rpm_id, '_content_type_id' : unit_type_id 2. package names with '.' # pulp-v1-upgrade .. Upgrading: Packages, Errata, and Distributions Traceback (most recent call last): File "/usr/bin/pulp-v1-upgrade", line 50, in <module> upgrader.upgrade() File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py", line 187, in upgrade self._upgrade_database() File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/main.py", line 228, in _upgrade_database report = db_call(v1_database, tmp_database) File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line 125, in upgrade groups_success = _package_groups(v1_database, v2_database, report) File "/usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py", line 622, in _package_groups v2_coll.insert(new_group, safe=True) File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 312, in insert continue_on_error, self.__uuid_subtype), safe) bson.errors.InvalidDocument: key 'openoffice.org-langpack-pt_PT' must not contain '.' I believe, this was introduced in pymongo >2.0.1 and pulp v2 requires 2.1. RHEL5 contains these openoffice packages and i believe (but i might be wrong), rpm allows dots. Funny thing, this issue was mentioned in http://www.pulpproject.org/ug/UGFAQ.html. workaround for this is to downgrade. The issue has been discovered some time ago, see http://osdir.com/ml/pulp-list/2013-06/msg00014.html. To sucessfully upgrade i have to set check_keys=False (w=1 is the same as safe=True on recent versions of pymongo): --- /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py.org 2013-08-18 21:42:25.723204539 +0200 +++ /usr/lib/python2.6/site-packages/pulp/server/upgrade/db/units.py 2013-08-20 21:57:16.201263427 +0200 @@ -619,7 +619,7 @@ def _package_groups(v1_database, v2_data 'translated_name' : v1_group['translated_name'], 'user_visible' : v1_group['user_visible'], } - v2_coll.insert(new_group, safe=True) + v2_coll.insert(new_group, check_keys=False, w=1) new_association = { '_id' : ObjectId(), Unfortunately "check_keys=False" must be set on every insert, save, update, upsert. 3. upgrade documentation the upgrade documentation https://pulp-user-guide.readthedocs.org/en/pulp-2.1/v1_upgrade.html is missing a important step: running pulp-manage-db after db upgrade and updating pulp to the last stable version. Do you want pull requests for issue 1 and/or issue 2? Regards, -ap _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list _______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
