As for the teardown method, it was written for use with our tests and is not exposed as a command in any way. If it's helpful, here is the list of tables which are NOT SAFE to remove data from https://github.com/pulp/pulp-2to3-migration/blob/master/pulp_2to3_migration/tests/functional/constants.py#L1-L14 .
I hope we can provide the reset option soon, and you'll no longer need any workarounds. Tanya On Thu, Oct 15, 2020 at 2:02 PM Tatiana Tereshchenko <[email protected]> wrote: > Good to know that it's not for the upgrade. > > We've planned to have a way to restart migration from scratch but never > got to it and no one asked before. > Here is a story I filed, feel free to provide any feedback there > https://pulp.plan.io/issues/7714. > > It would be helpful for us to understand your use case. > Could you share why you wanted to start completely from scratch and not > just re-run the existing migration plan or run a new one? > > Thanks, > Tanya > > On Wed, Oct 14, 2020 at 7:07 PM Winberg Adam <[email protected]> wrote: > >> > probably because we never ask to wipe out the database for upgrades. >> >> >> My reason for doing a 'flush' was to rerun my pulp2 migration from >> scratch, so not really because of the upgrade. But with the background >> provided by you, issue https://pulp.plan.io/issues/6963 now makes more >> sense to me. I guess I should preferably use the 'teardown' util mentioned >> there instead? It is however unclear to me how to use that. >> >> >> The 'reset_db' won't work for me since we have a centralized postgres >> infrastructure where I don't have permissions to drop/create db's (I had to >> get the help of a DBMS to drop my pulp-db). >> >> >> @Brian Bouterse - thanks for creating the issue! >> >> >> //Adam >> >> >> >> >> >> ------------------------------ >> *From:* Brian Bouterse <[email protected]> >> *Sent:* 14 October 2020 18:17 >> *To:* Tatiana Tereshchenko >> *Cc:* Winberg Adam; [email protected] >> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query >> does not exist >> >> I've filed this issue tracking the improvement which would allow users to >> run `flush` and not experience this problem. >> >> https://pulp.plan.io/issues/7710 >> >> On Wed, Oct 14, 2020 at 12:14 PM Tatiana Tereshchenko < >> [email protected]> wrote: >> >>> Adam, >>> I agree we lack documentation on resetting the environment and the >>> database, probably because we never ask to wipe out the database for >>> upgrades. >>> The instructions are usually provided with release and ask you basically >>> to run the latest pulp_installer. >>> >>> There is some data which is provided with migrations and which has to be >>> present in the database, that's why dropping the database and applying >>> migrations work and `flush` does not. >>> `flush` just removes data and keeps all the tables in place, so >>> migrations are not re-applied. >>> So for now, please do not use `flush` if you want to start using pulp 3 >>> db from scratch. We potentially can provide a separate command or find some >>> other way to fill in the essential data back. >>> Please file an issue in our tracker if such feature/command is helpful >>> for you. https://pulp.plan.io/projects/pulp/issues/new >>> >>> As a side note, if you are interested, reset_db command drops database. >>> It's provided as a part of django-extensions package >>> https://django-extensions.readthedocs.io/en/latest/command_extensions.html >>> >>> I'm glad that the migration runs for you now, >>> Tanya >>> >>> >>> On Wed, Oct 14, 2020 at 3:52 PM Winberg Adam <[email protected]> >>> wrote: >>> >>>> Thanks for the reply! >>>> >>>> >>>> I use 'flush' to clear the db, I don't have a 'reset_db' command. >>>> Otherwise that's pretty much my process. >>>> >>>> >>>> The migrate command returns 'No migrations to apply'. >>>> >>>> Running 'pulpcore-manager show-migrations' shows that all migrations, >>>> including 'guardian.0001/guardian.0002' has checkmarks ([X]). >>>> >>>> guardian >>>> [X] 0001_initial >>>> [X] 0002_generic_permissions_index >>>> >>>> The creation of the migration plan works so I assume my admin user is >>>> ok. >>>> >>>> >>>> I am using an rpm based installation on RHEL8, with >>>> >>>> python3-pulp-2to3-migration-0.4.0-1.el8.noarch >>>> python3-pulp-rpm-3.7.0-1.el8.noarch >>>> python3-pulpcore-3.7.1-3.el8.noarch >>>> >>>> >>>> I don't know what went wrong, but I surrendered and dropped my DB and >>>> redid the migrations from scratch - and now it works. >>>> Is there a documented instruction on upgrading existing installations? >>>> >>>> //Adam >>>> >>>> >>>> >>>> ------------------------------ >>>> *From:* Tatiana Tereshchenko <[email protected]> >>>> *Sent:* 14 October 2020 14:15 >>>> *To:* Winberg Adam >>>> *Cc:* [email protected] >>>> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching >>>> query does not exist >>>> >>>> >>>> >>>> On Wed, Oct 14, 2020 at 2:12 PM Tatiana Tereshchenko < >>>> [email protected]> wrote: >>>> >>>>> Hi Adam, >>>>> >>>>> My understanding is that you did the following: >>>>> * stop pulp services >>>>> * pulpcore-manager (or django-admin) reset_db >>>>> * pulpcore-manager migrate >>>>> * pulpcore-manager reset-admin-password --password password >>>>> * start services >>>>> * http POST :/pulp/api/v3/migration-plans/ < your_migraiton_plan.json >>>>> * http POST >>>>> :/pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/ >>>>> >>>> >>>> Sent too early :) >>>> I can't reproduce it so far, so any hints about what can be special >>>> about your environment or installation would be appreciated. >>>> Make sure that you have at least one user which has admin >>>> privileges and that the guardian migrations ran indeed. >>>> Applying guardian.0001_initial... OK >>>> Applying guardian.0002_generic_permissions_index... OK >>>> >>>> Tanya >>>> >>>> >>>> >>>> >>>> >>>>> >>>>> On Wed, Oct 14, 2020 at 8:02 AM Winberg Adam <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> >>>>>> so I updated my pulp3 installation from 3.4 to 3.7 and tried to rerun >>>>>> my pulp2 migration - but it errors out with "AccessPolicy matching >>>>>> query does not exist". Anyone know why? >>>>>> >>>>>> >>>>>> I flushed my db, reran the 'migrate' job, created a pulp2migration >>>>>> plan (which worked fine) and then tried to run it. Here's the complete >>>>>> error: >>>>>> >>>>>> >>>>>> Oct 14 05:43:26 gunicorn[2150852]: pulp: django.request:ERROR: >>>>>> Internal Server Error: >>>>>> /pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/ >>>>>> Oct 14 05:43:26 gunicorn[2150852]: Traceback (most recent call last): >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", >>>>>> line >>>>>> 34, in inner >>>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>>> get_response(request) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line >>>>>> 115, >>>>>> in _get_response >>>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>>> self.process_exception_by_middleware(e, request) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line >>>>>> 113, >>>>>> in _get_response >>>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>>> wrapped_callback(request, *callback_args, **callback_kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py", line >>>>>> 54, in wrapped_view >>>>>> Oct 14 05:43:26 gunicorn[2150852]: return view_func(*args, >>>>>> **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/rest_framework/viewsets.py", line 114, >>>>>> in >>>>>> view >>>>>> Oct 14 05:43:26 gunicorn[2150852]: return self.dispatch(request, >>>>>> *args, **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 505, in >>>>>> dispatch >>>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>>> self.handle_exception(exc) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 465, in >>>>>> handle_exception >>>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>>> self.raise_uncaught_exception(exc) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 476, in >>>>>> raise_uncaught_exception >>>>>> Oct 14 05:43:26 gunicorn[2150852]: raise exc >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 502, in >>>>>> dispatch >>>>>> Oct 14 05:43:26 gunicorn[2150852]: response = handler(request, >>>>>> *args, **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py", >>>>>> line 85, in run >>>>>> Oct 14 05:43:26 gunicorn[2150852]: 'dry_run': dry_run >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/pulpcore/tasking/tasks.py", line 236, >>>>>> in >>>>>> enqueue_with_reservation >>>>>> Oct 14 05:43:26 gunicorn[2150852]: **parent_kwarg, >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, >>>>>> in >>>>>> manager_method >>>>>> Oct 14 05:43:26 gunicorn[2150852]: return >>>>>> getattr(self.get_queryset(), name)(*args, **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 422, >>>>>> in >>>>>> create >>>>>> Oct 14 05:43:26 gunicorn[2150852]: obj.save(force_insert=True, >>>>>> using=self.db) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 132, >>>>>> in >>>>>> save >>>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>>> self._run_hooked_methods(AFTER_CREATE) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 207, >>>>>> in >>>>>> _run_hooked_methods >>>>>> Oct 14 05:43:26 gunicorn[2150852]: method() >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/decorators.py", line >>>>>> 69, >>>>>> in func >>>>>> Oct 14 05:43:26 gunicorn[2150852]: hooked_method(*args, **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/pulpcore/app/models/access_policy.py", >>>>>> line 60, in add_perms >>>>>> Oct 14 05:43:26 gunicorn[2150852]: access_policy = >>>>>> AccessPolicy.objects.get(viewset_name=self.ACCESS_POLICY_VIEWSET_NAME) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, >>>>>> in >>>>>> manager_method >>>>>> Oct 14 05:43:26 gunicorn[2150852]: return >>>>>> getattr(self.get_queryset(), name)(*args, **kwargs) >>>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 408, >>>>>> in >>>>>> get >>>>>> Oct 14 05:43:26 gunicorn[2150852]: self.model._meta.object_name >>>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>>> pulpcore.app.models.access_policy.AccessPolicy.DoesNotExist: AccessPolicy >>>>>> matching query does not exist. >>>>>> >>>>>> >>>>>> Regards >>>>>> >>>>>> //Adam >>>>>> _______________________________________________ >>>>>> Pulp-list mailing list >>>>>> [email protected] >>>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>>> >>>>> _______________________________________________ >>> Pulp-list mailing list >>> [email protected] >>> https://www.redhat.com/mailman/listinfo/pulp-list >> >>
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
