Good to know that it's not for the upgrade. We've planned to have a way to restart migration from scratch but never got to it and no one asked before. Here is a story I filed, feel free to provide any feedback there https://pulp.plan.io/issues/7714.
It would be helpful for us to understand your use case. Could you share why you wanted to start completely from scratch and not just re-run the existing migration plan or run a new one? Thanks, Tanya On Wed, Oct 14, 2020 at 7:07 PM Winberg Adam <[email protected]> wrote: > > probably because we never ask to wipe out the database for upgrades. > > > My reason for doing a 'flush' was to rerun my pulp2 migration from > scratch, so not really because of the upgrade. But with the background > provided by you, issue https://pulp.plan.io/issues/6963 now makes more > sense to me. I guess I should preferably use the 'teardown' util mentioned > there instead? It is however unclear to me how to use that. > > > The 'reset_db' won't work for me since we have a centralized postgres > infrastructure where I don't have permissions to drop/create db's (I had to > get the help of a DBMS to drop my pulp-db). > > > @Brian Bouterse - thanks for creating the issue! > > > //Adam > > > > > > ------------------------------ > *From:* Brian Bouterse <[email protected]> > *Sent:* 14 October 2020 18:17 > *To:* Tatiana Tereshchenko > *Cc:* Winberg Adam; [email protected] > *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query > does not exist > > I've filed this issue tracking the improvement which would allow users to > run `flush` and not experience this problem. > > https://pulp.plan.io/issues/7710 > > On Wed, Oct 14, 2020 at 12:14 PM Tatiana Tereshchenko <[email protected]> > wrote: > >> Adam, >> I agree we lack documentation on resetting the environment and the >> database, probably because we never ask to wipe out the database for >> upgrades. >> The instructions are usually provided with release and ask you basically >> to run the latest pulp_installer. >> >> There is some data which is provided with migrations and which has to be >> present in the database, that's why dropping the database and applying >> migrations work and `flush` does not. >> `flush` just removes data and keeps all the tables in place, so >> migrations are not re-applied. >> So for now, please do not use `flush` if you want to start using pulp 3 >> db from scratch. We potentially can provide a separate command or find some >> other way to fill in the essential data back. >> Please file an issue in our tracker if such feature/command is helpful >> for you. https://pulp.plan.io/projects/pulp/issues/new >> >> As a side note, if you are interested, reset_db command drops database. >> It's provided as a part of django-extensions package >> https://django-extensions.readthedocs.io/en/latest/command_extensions.html >> >> I'm glad that the migration runs for you now, >> Tanya >> >> >> On Wed, Oct 14, 2020 at 3:52 PM Winberg Adam <[email protected]> >> wrote: >> >>> Thanks for the reply! >>> >>> >>> I use 'flush' to clear the db, I don't have a 'reset_db' command. >>> Otherwise that's pretty much my process. >>> >>> >>> The migrate command returns 'No migrations to apply'. >>> >>> Running 'pulpcore-manager show-migrations' shows that all migrations, >>> including 'guardian.0001/guardian.0002' has checkmarks ([X]). >>> >>> guardian >>> [X] 0001_initial >>> [X] 0002_generic_permissions_index >>> >>> The creation of the migration plan works so I assume my admin user is >>> ok. >>> >>> >>> I am using an rpm based installation on RHEL8, with >>> >>> python3-pulp-2to3-migration-0.4.0-1.el8.noarch >>> python3-pulp-rpm-3.7.0-1.el8.noarch >>> python3-pulpcore-3.7.1-3.el8.noarch >>> >>> >>> I don't know what went wrong, but I surrendered and dropped my DB and >>> redid the migrations from scratch - and now it works. >>> Is there a documented instruction on upgrading existing installations? >>> >>> //Adam >>> >>> >>> >>> ------------------------------ >>> *From:* Tatiana Tereshchenko <[email protected]> >>> *Sent:* 14 October 2020 14:15 >>> *To:* Winberg Adam >>> *Cc:* [email protected] >>> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query >>> does not exist >>> >>> >>> >>> On Wed, Oct 14, 2020 at 2:12 PM Tatiana Tereshchenko < >>> [email protected]> wrote: >>> >>>> Hi Adam, >>>> >>>> My understanding is that you did the following: >>>> * stop pulp services >>>> * pulpcore-manager (or django-admin) reset_db >>>> * pulpcore-manager migrate >>>> * pulpcore-manager reset-admin-password --password password >>>> * start services >>>> * http POST :/pulp/api/v3/migration-plans/ < your_migraiton_plan.json >>>> * http POST >>>> :/pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/ >>>> >>> >>> Sent too early :) >>> I can't reproduce it so far, so any hints about what can be special >>> about your environment or installation would be appreciated. >>> Make sure that you have at least one user which has admin privileges and >>> that the guardian migrations ran indeed. >>> Applying guardian.0001_initial... OK >>> Applying guardian.0002_generic_permissions_index... OK >>> >>> Tanya >>> >>> >>> >>> >>> >>>> >>>> On Wed, Oct 14, 2020 at 8:02 AM Winberg Adam <[email protected]> >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> >>>>> so I updated my pulp3 installation from 3.4 to 3.7 and tried to rerun >>>>> my pulp2 migration - but it errors out with "AccessPolicy matching >>>>> query does not exist". Anyone know why? >>>>> >>>>> >>>>> I flushed my db, reran the 'migrate' job, created a pulp2migration >>>>> plan (which worked fine) and then tried to run it. Here's the complete >>>>> error: >>>>> >>>>> >>>>> Oct 14 05:43:26 gunicorn[2150852]: pulp: django.request:ERROR: >>>>> Internal Server Error: >>>>> /pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/ >>>>> Oct 14 05:43:26 gunicorn[2150852]: Traceback (most recent call last): >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line >>>>> 34, in inner >>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>> get_response(request) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, >>>>> in _get_response >>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>> self.process_exception_by_middleware(e, request) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, >>>>> in _get_response >>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>> wrapped_callback(request, *callback_args, **callback_kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py", line >>>>> 54, in wrapped_view >>>>> Oct 14 05:43:26 gunicorn[2150852]: return view_func(*args, >>>>> **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/rest_framework/viewsets.py", line 114, >>>>> in >>>>> view >>>>> Oct 14 05:43:26 gunicorn[2150852]: return self.dispatch(request, >>>>> *args, **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 505, in >>>>> dispatch >>>>> Oct 14 05:43:26 gunicorn[2150852]: response = >>>>> self.handle_exception(exc) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 465, in >>>>> handle_exception >>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>> self.raise_uncaught_exception(exc) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 476, in >>>>> raise_uncaught_exception >>>>> Oct 14 05:43:26 gunicorn[2150852]: raise exc >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 502, in >>>>> dispatch >>>>> Oct 14 05:43:26 gunicorn[2150852]: response = handler(request, >>>>> *args, **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py", >>>>> line 85, in run >>>>> Oct 14 05:43:26 gunicorn[2150852]: 'dry_run': dry_run >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/pulpcore/tasking/tasks.py", line 236, in >>>>> enqueue_with_reservation >>>>> Oct 14 05:43:26 gunicorn[2150852]: **parent_kwarg, >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, >>>>> in >>>>> manager_method >>>>> Oct 14 05:43:26 gunicorn[2150852]: return >>>>> getattr(self.get_queryset(), name)(*args, **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 422, in >>>>> create >>>>> Oct 14 05:43:26 gunicorn[2150852]: obj.save(force_insert=True, >>>>> using=self.db) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 132, >>>>> in >>>>> save >>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>> self._run_hooked_methods(AFTER_CREATE) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 207, >>>>> in >>>>> _run_hooked_methods >>>>> Oct 14 05:43:26 gunicorn[2150852]: method() >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/decorators.py", line >>>>> 69, >>>>> in func >>>>> Oct 14 05:43:26 gunicorn[2150852]: hooked_method(*args, **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/pulpcore/app/models/access_policy.py", >>>>> line 60, in add_perms >>>>> Oct 14 05:43:26 gunicorn[2150852]: access_policy = >>>>> AccessPolicy.objects.get(viewset_name=self.ACCESS_POLICY_VIEWSET_NAME) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, >>>>> in >>>>> manager_method >>>>> Oct 14 05:43:26 gunicorn[2150852]: return >>>>> getattr(self.get_queryset(), name)(*args, **kwargs) >>>>> Oct 14 05:43:26 gunicorn[2150852]: File >>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 408, in >>>>> get >>>>> Oct 14 05:43:26 gunicorn[2150852]: self.model._meta.object_name >>>>> Oct 14 05:43:26 gunicorn[2150852]: >>>>> pulpcore.app.models.access_policy.AccessPolicy.DoesNotExist: AccessPolicy >>>>> matching query does not exist. >>>>> >>>>> >>>>> Regards >>>>> >>>>> //Adam >>>>> _______________________________________________ >>>>> Pulp-list mailing list >>>>> [email protected] >>>>> https://www.redhat.com/mailman/listinfo/pulp-list >>>> >>>> _______________________________________________ >> Pulp-list mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/pulp-list > >
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
