Hi Bin, It's not a migration issue, it's an issue with the bindings that cropped up after a schema change. The fix will be pushed some time this week.
https://pulp.plan.io/issues/9056#note-7 https://github.com/tfranzel/drf-spectacular/blob/812d3ffd39347241ea9ac38793d6801cc6b17136/docs/client_generation.rst#enum-issues On Wed, Aug 18, 2021 at 3:16 PM Bin Li (BLOOMBERG/ 120 PARK) < [email protected]> wrote: > I was able to migrate to the 3.14.4 with updated pulp settings. However, I > got this error when tried to use the pulp_rpm api: > AttributeError: module 'pulpcore.client.pulp_rpm.models' has no attribute > 'OneOf MetadataChecksumTypeEnumNullEnum' > > The error went away if I erase and reset the db completely. I think this > error is still somehow related to the migration from 3.7. Please advise. > > The below is the entire error messages: > # ./test.py > Traceback (most recent call last): > File "./test.py", line 17, in <module> > RepositoriesRpmApi(api_client=rpm_client).list().results > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api/repositories_rpm_api.py", line 305, in list > return self.list_with_http_info(**kwargs) # noqa: E501 > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api/repositories_rpm_api.py", line 417, in list_with_http_info > return self.api_client.call_api( > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 365, in call_api > return self.__call_api(resource_path, method, > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 209, in __call_api > return_data = self.deserialize(response_data, response_type) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 281, in deserialize > return self.__deserialize(data, response_type) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 320, in __deserialize > return self.__deserialize_model(data, klass) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 659, in __deserialize_model > kwargs[attr] = self.__deserialize(value, attr_type) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 297, in __deserialize > return [self.__deserialize(sub_data, sub_kls) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 297, in <listcomp> > return [self.__deserialize(sub_data, sub_kls) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 320, in __deserialize > return self.__deserialize_model(data, klass) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 659, in __deserialize_model > kwargs[attr] = self.__deserialize(value, attr_type) > File > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/client > /pulp_rpm/api_client.py", line 309, in __deserialize > klass = getattr(pulpcore.client.pulp_rpm.models, klass) > AttributeError: module 'pulpcore.client.pulp_rpm.models' has no attribute > 'OneOf MetadataChecksumTypeEnumNullEnum' > > The test.py is as following: > #!/opt/utils/venv/pulp/3.8.8/bin/python > from pulpcore.client import pulpcore > from pulpcore.client import pulp_rpm > from pulpcore.client.pulp_rpm import ( > RepositoriesRpmApi > ) > > > # Configure HTTP basic authorization: basic > configuration = pulpcore.Configuration() > configuration.host = 'http://localhost' > configuration.username = 'pulp' > configuration.password = 'pulp' > configuration.safe_chars_for_path_param = '/' > > rpm_client = pulp_rpm.ApiClient(configuration) > #RepositoriesRpmApi <#m_-3250670613409565681_RepositoriesRpmApi> > (api_client=rpm_client).list().results > RepositoriesRpmApi(api_client=rpm_client).list() > > I am running: > { > "versions": [ > { > "component": "core", > "version": "3.14.4" > }, > { > "component": "rpm", > "version": "3.14.1" > }, > { > "component": "file", > "version": "1.8.2" > } > > > # pip list |grep pulp > pulp-file 1.8.2 > pulp-rpm 3.14.1 > pulp-rpm-client 3.14.1 > pulpcore 3.14.4 > pulpcore-client 3.14.4 > > From: [email protected] At: 08/06/21 02:44:17 UTC-4:00 > To: [email protected] > Subject: Pulp-list Digest, Vol 141, Issue 4 > > Send Pulp-list mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > https://listman.redhat.com/mailman/listinfo/pulp-list > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pulp-list digest..." > > > Today's Topics: > > 1. Re: pulp 3.14 migration (Daniel Alley) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 5 Aug 2021 17:38:56 -0400 > From: Daniel Alley <[email protected]> > To: Fabricio Aguiar <[email protected]> > Cc: pulp-list <[email protected]>, Bin Li <[email protected]> > Subject: Re: [Pulp-list] pulp 3.14 migration > Message-ID: > <CAK1D4mAcs7Fa_wM=zhhmkzxh6+gzf0_3wfnasaxowy+1_ye...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Here's a hypothesis for what happened. > > * System is at pulpcore 3.7, it has artifacts saved with checksums that are > disabled on later versions of Pulp > * 3.14 is installed, migration to 3.14 database schema fails due to the > disallowed checksums (IIRC we have a special case to avoid this happening, > but maybe we missed a scenario. There is a question mark here.) > * Bin attempts to run "handle-artifact-checksums" but is unable to do so > because of the incompleted migrations > > I think the correct workaround is to update the settings file to allow all > checksum types prior to the upgrade. And presumably if you're still > managing content with such checksums, you want to keep them enabled, > anyway. > > @Bin Li, On your test system please edit /etc/pulp/settings.py to include > the following line > > ALLOWED_CONTENT_CHECKSUMS = ["md5", "sha1", "sha224", "sha256", "sha384", > > "sha512"] > > > > And then restart the upgrade process / database migrations. > > On Thu, Aug 5, 2021 at 5:26 PM Daniel Alley <[email protected]> wrote: > > > Bin is using Katello's RPM builds, I'm not sure if Katello has RPMs built > > for 3.10. > > > > On Thu, Aug 5, 2021 at 4:22 PM Fabricio Aguiar <[email protected]> > > wrote: > > > >> I tried but I couldn't reproduce the error, > >> I know pulpcore 3.11 introduced some breaking changes [1] > >> I would try to migrate from 3.7 to 3.10, and then from 3.10 to 3.14 > >> > >> [1] https://docs.pulpproject.org/pulpcore/changes.html#id198 > >> > >> Best regards, > >> Fabricio Aguiar > >> Software Engineer, Pulp Project > >> Red Hat Brazil - Latam <https://www.redhat.com/> > >> +55 22 999000595 > >> > >> > >> > >> On Thu, Jul 29, 2021 at 1:17 PM Bin Li (BLOOMBERG/ 120 PARK) < > >> [email protected]> wrote: > >> > >>> Thanks Faguiard. We are using pulp_installer. The error originally from > >>> pulpcore 3.14 and pulp_rpm 3.13. > >>> > >>> I just tried the latest pulp_installer. I still see the same errors. > >>> pulp-file 1.8.2 > >>> pulp-rpm 3.14.0 > >>> pulpcore 3.14.3 > >>> > >>> I noticed a interesting thing. The migration from 3.7.3 works fine if I > >>> migrate a fresh installed 3.7.3 to 3.14.3. The error only happened > after I > >>> syncd a few repos to the 3.7.3 instance and then migrate to 3.14.3. > >>> > >>> > >>> From: [email protected] At: 07/27/21 10:37:36 UTC-4:00 > >>> To: Bin Li (BLOOMBERG/ 120 PARK ) <[email protected]> > >>> Cc: [email protected] > >>> Subject: Re: [Pulp-list] pulp 3.14 migration > >>> > >>> Hi Bin, > >>> > >>> We currently test migrating from 3.11 to 3.14, so probably something is > >>> happening from 3.7 to 3.11 > >>> I'll be trying to reproduce the error. Are you using pulp_installer? If > >>> so, which version? Which plugins do you use? > >>> > >>> Best regards, > >>> Fabricio Aguiar > >>> Software Engineer, Pulp Project > >>> Red Hat Brazil - Latam <https://www.redhat.com/> > >>> +55 22 999000595 > >>> > >>> > >>> > >>> On Fri, Jul 23, 2021 at 11:29 AM Bin Li (BLOOMBERG/ 120 PARK) < > >>> [email protected]> wrote: > >>> > >>>> The installation didn't pass the health check because the following > >>>> errors: > >>>> pulpcore-worker: django.core.exceptions.ImproperlyConfigured: There > >>>> have been identified artifacts with forbidden checksum 'sha1'. Run > >>>> 'pulpcore-manager handle-artifact-checksums' first to unset forbidden > >>>> checksums. > >>>> > >>>> Tried to run pulpcore-manager. It also failed. This is a migration > from > >>>> existing 3.7.2 to 3.14. Please advise. > >>>> > >>>> $ pulpcore-manager handle-artifact-checksums > >>>> System check identified some issues: > >>>> > >>>> WARNINGS: > >>>> ?: (guardian.W001) Guardian authentication backend is not hooked. You > >>>> can add this in settings as eg: `AUTHENTICATION_BACKENDS = > >>>> ('django.contrib.auth.backends.ModelBackend', > >>>> 'guardian.backends.ObjectPermissionBackend')`. > >>>> Traceback (most recent call last): > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py", > >>>> line 84, in _execute > >>>> return self.cursor.execute(sql, params) > >>>> psycopg2.errors.UndefinedColumn: column > >>>> core_artifact.timestamp_of_interest does not exist > >>>> LINE 1: ...ore_artifact"."sha384", "core_artifact"."sha512", > >>>> "core_arti... > >>>> ^ > >>>> > >>>> > >>>> The above exception was the direct cause of the following exception: > >>>> > >>>> Traceback (most recent call last): > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/sql/compiler.py", > >>>> line 1142, in execute_sql > >>>> cursor.execute(sql, params) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py", > >>>> line 67, in execute > >>>> return self._execute_with_wrappers(sql, params, many=False, > >>>> executor=self._execute) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py", > >>>> line 76, in _execute_with_wrappers > >>>> return executor(sql, params, many, context) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py", > >>>> line 84, in _execute > >>>> return self.cursor.execute(sql, params) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/utils.py", > >>>> line 89, in __exit__ > >>>> raise dj_exc_value.with_traceback(traceback) from exc_value > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/backends/utils.py", > >>>> line 84, in _execute > >>>> return self.cursor.execute(sql, params) > >>>> django.db.utils.ProgrammingError: column > >>>> core_artifact.timestamp_of_interest does not exist > >>>> LINE 1: ...ore_artifact"."sha384", "core_artifact"."sha512", > >>>> "core_arti... > >>>> ^ > >>>> > >>>> > >>>> During handling of the above exception, another exception occurred: > >>>> > >>>> Traceback (most recent call last): > >>>> File "/opt/utils/venv/pulp/3.8.8/bin/pulpcore-manager", line 8, in > >>>> <module> > >>>> sys.exit(manage()) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/app/manage.py", > >>>> line 11, in manage > >>>> execute_from_command_line(sys.argv) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/__init__.py", > >>>> line 381, in execute_from_command_line > >>>> utility.execute() > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/__init__.py", > >>>> line 375, in execute > >>>> self.fetch_command(subcommand).run_from_argv(self.argv) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/base.py", > >>>> line 323, in run_from_argv > >>>> self.execute(*args, **cmd_options) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/core/management/base.py", > >>>> line 364, in execute > >>>> output = self.handle(*args, **options) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/pulpcore/app/management/commands/handle-artifact-checksums.py", > >>>> line 177, in handle > >>>> for a in artifacts_qs.iterator(): > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/query.py", > >>>> line 341, in _iterator > >>>> yield from self._iterable_class(self, chunked_fetch=use_chunked_fetch, > >>>> chunk_size=chunk_size) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/query.py", > >>>> line 55, in __iter__ > >>>> results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, > >>>> chunk_size=self.chunk_size) > >>>> File > >>>> > > "/opt/utils/venv/pulp/3.8.8/lib64/python3.8/site-packages/django/db/models/sql/compiler.py", > >>>> line 1145, in execute_sql > >>>> cursor.close() > >>>> psycopg2.errors.InvalidCursorName: cursor > >>>> "_django_curs_139896608020288_1" does not exist > >>>> > >>>> > >>>> _______________________________________________ > >>>> Pulp-list mailing list > >>>> [email protected] > >>>> https://listman.redhat.com/mailman/listinfo/pulp-list > >>> > >>> > >>> _______________________________________________ > >> Pulp-list mailing list > >> [email protected] > >> https://listman.redhat.com/mailman/listinfo/pulp-list > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > < > https://listman.redhat.com/archives/pulp-list/attachments/20210805/41d782be/ > attachment.html> > > ------------------------------ > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list > > End of Pulp-list Digest, Vol 141, Issue 4 > ***************************************** > > > _______________________________________________ > Pulp-list mailing list > [email protected] > https://listman.redhat.com/mailman/listinfo/pulp-list
_______________________________________________ Pulp-list mailing list [email protected] https://listman.redhat.com/mailman/listinfo/pulp-list
