On Fri, May 15, 2020 at 12:57 PM Brian Dickson < [email protected]> wrote:
> The local database is empty. I'll make sure that's the case just to be > safe. > > # peeringdb --version > > peeringdb 1.0.0 > > > # django-admin --version > > 2.2.12 > > > # pip freeze > > asgiref==3.2.7 > > certifi==2020.4.5.1 > > cfu==1.5.0 > > chardet==3.0.4 > > click==7.1.2 > > Django==2.2.12 > > django-countries==6.1.2 > > django-handleref==0.5.0 > > django-inet==0.4.0 > > django-peeringdb==2.0.0 > > future==0.18.2 > > idna==2.9 > > munge==1.0.0 > > mysqlclient==1.4.4 > > peeringdb==1.0.0 > > PyMySQL==0.9.3 > > pytz==2020.1 > > PyYAML==5.3.1 > > requests==2.23.0 > > six==1.14.0 > > sqlparse==0.3.1 > > twentyc.rpc==0.4.0 > > urllib3==1.25.9 > Here is the result when running the sync on a clean empty local database: bash-3.2# peeringdb drop-tables bash-3.2# peeringdb sync Syncing to https://www.peeringdb.com/api Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc Fetching & updating all: org Updates to be processed: 18439 Ignoring object updated after sync began: (org-6864) Ignoring object updated after sync began: (org-13687) Ignoring object updated after sync began: (org-23582) Ignoring object updated after sync began: (org-26092) Ignoring object updated after sync began: (org-26094) Ignoring object updated after sync began: (org-26095) Ignoring object updated after sync began: (org-26096) Ignoring object updated after sync began: (org-26097) Ignoring object updated after sync began: (org-26098) Fetching & updating all: fac Updates to be processed: 3714 Ignoring object updated after sync began: (fac-3634) Ignoring object updated after sync began: (fac-8262) Ignoring object updated after sync began: (fac-8582) Ignoring object updated after sync began: (fac-8583) Ignoring object updated after sync began: (fac-8584) Ignoring object updated after sync began: (fac-8585) Ignoring object updated after sync began: (fac-8586) Ignoring object updated after sync began: (fac-8587) Ignoring object updated after sync began: (fac-8588) Ignoring object updated after sync began: (fac-8591) Ignoring object updated after sync began: (fac-8592) Ignoring object updated after sync began: (fac-8593) Ignoring object updated after sync began: (fac-8594) Ignoring object updated after sync began: (fac-8595) Ignoring object updated after sync began: (fac-8597) Ignoring object updated after sync began: (fac-8598) Ignoring object updated after sync began: (fac-8599) Ignoring object updated after sync began: (fac-8600) Ignoring object updated after sync began: (fac-8601) Ignoring object updated after sync began: (fac-8602) Ignoring object updated after sync began: (fac-8603) Fetching & updating all: net Updates to be processed: 18845 Ignoring object updated after sync began: (net-2913) Ignoring object updated after sync began: (net-3087) Ignoring object updated after sync began: (net-4823) Ignoring object updated after sync began: (net-5442) Ignoring object updated after sync began: (net-7482) Ignoring object updated after sync began: (net-9435) Ignoring object updated after sync began: (net-13681) Ignoring object updated after sync began: (net-14816) Ignoring object updated after sync began: (net-19500) Ignoring object updated after sync began: (net-20575) Ignoring object updated after sync began: (net-21292) Ignoring object updated after sync began: (net-21916) Ignoring object updated after sync began: (net-23222) Ignoring object updated after sync began: (net-23266) Ignoring object updated after sync began: (net-23280) Ignoring object updated after sync began: (net-23283) Ignoring object updated after sync began: (net-23284) Ignoring object updated after sync began: (net-23285) Traceback (most recent call last): File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/bin/peeringdb", line 8, in <module> sys.exit(main()) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/cli.py", line 62, in main return handler(config=cfg, **vars(options)) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/commands.py", line 20, in _wrapped r = func(*a, **k) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/commands.py", line 219, in handle client.update_all(rs) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", line 66, in update_all self._atomic_update(lambda: ctx.sync_resource(r, since=since)) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", line 78, in _atomic_update sync_func() File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", line 66, in <lambda> self._atomic_update(lambda: ctx.sync_resource(r, since=since)) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_tasks_async.py", line 65, in _wrapped return loop.run_until_complete(func(*a, **k)) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 488, in run_until_complete return future.result() File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_tasks_async.py", line 41, in _wrapped item = gen.send(r) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", line 287, in sync_row B.clean(obj) File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django_peeringdb/client_adaptor/backend.py", line 145, in clean obj.full_clean() File "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/base.py", line 1203, in full_clean raise ValidationError(errors) django.core.exceptions.ValidationError: {'org': ['organization instance with id 13687 does not exist.']} > > On Fri, May 15, 2020 at 10:55 AM Chris Caputo <[email protected]> wrote: > >> Hi. Does restarting with a fresh/empty local database help at all? >> >> Also, what does the following show?: >> >> peeringdb --version >> django-admin --version >> pip freeze >> >> Thanks, >> Chris >> >> On Fri, 15 May 2020, Brian Dickson wrote: >> > Arnold Nipper said to send to this list. >> > >> > (This is kind of urgent for me, my goal is to get a snapshot of the >> data, so any alternative workaround would be helpful in the >> > meantime.) >> > >> > I'm in the process of setting up a local copy, and have run into a >> problem after getting the peeringdb-py stuff set up. >> > >> > When I do peeringdb sync (with the correct mysql database UTF8 stuff >> and everything,), I am currently getting a persistent error >> > which aborts the sync with no data in the tables: >> > >> > (Apologies in advance for the long error output text.) >> > Syncing to https://www.peeringdb.com/api >> > Updating resources: org fac net ix ixfac ixlan ixpfx netfac netixlan poc >> > Fetching & updating all: org >> > Updates to be processed: 5 >> > Ignoring object updated after sync began: (org-26015) >> > Ignoring object updated after sync began: (org-26052) >> > Ignoring object updated after sync began: (org-17918) >> > Ignoring object updated after sync began: (org-26053) >> > Fetching & updating all: fac >> > Updates to be processed: 0 >> > Fetching & updating all: net >> > Updates to be processed: 18800 >> > Ignoring object updated after sync began: (net-1356) >> > Ignoring object updated after sync began: (net-3684) >> > Ignoring object updated after sync began: (net-7924) >> > Ignoring object updated after sync began: (net-10733) >> > Ignoring object updated after sync began: (net-13084) >> > Ignoring object updated after sync began: (net-14581) >> > Ignoring object updated after sync began: (net-15702) >> > Traceback (most recent call last): >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/utils.py", >> > line 84, in _execute >> > return self.cursor.execute(sql, params) >> > File >> > >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/mysql/base.py", >> line >> > 71, in execute >> > return self.cursor.execute(query, args) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/cursors.py", >> line 209, in >> > execute >> > res = self._query(query) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/cursors.py", >> line 315, in >> > _query >> > db.query(q) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/connections.py", >> line >> > 226, in query >> > _mysql.connection.query(self, query) >> > MySQLdb._exceptions.IntegrityError: (1452, 'Cannot add or update a >> child row: a foreign key constraint fails >> > (`peeringdb`.`peeringdb_network`, CONSTRAINT >> `peeringdb_network_org_id_404d6106_fk_peeringdb_organization_id` FOREIGN KEY >> > (`org_id`) REFERENCES `peeringdb_organization` (`id`))') >> > >> > The above exception was the direct cause of the following exception: >> > >> > Traceback (most recent call last): >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/bin/peeringdb", >> line 8, in <module> >> > sys.exit(main()) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/cli.py", >> line 62, in >> > main >> > return handler(config=cfg, **vars(options)) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/commands.py", >> line 20, >> > in _wrapped >> > r = func(*a, **k) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/commands.py", >> line 219, >> > in handle >> > client.update_all(rs) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", >> line 66, >> > in update_all >> > self._atomic_update(lambda: ctx.sync_resource(r, since=since)) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", >> line 78, >> > in _atomic_update >> > sync_func() >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", >> line 66, >> > in <lambda> >> > self._atomic_update(lambda: ctx.sync_resource(r, since=since)) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_tasks_async.py", >> line >> > 65, in _wrapped >> > return loop.run_until_complete(func(*a, **k)) >> > File >> "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", >> line 488, in >> > run_until_complete >> > return future.result() >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_tasks_async.py", >> line >> > 41, in _wrapped >> > item = gen.send(r) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/peeringdb/_update.py", >> line 274, >> > in sync_row >> > B.save(obj) >> > >> File"/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django_peeringdb/client_adaptor/backend >> > .py", line 148, in save >> > obj.save() >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/base.py", >> line >> > 741, in save >> > force_update=force_update, update_fields=update_fields) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/base.py", >> line >> > 779, in save_base >> > force_update, using, update_fields, >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/base.py", >> line >> > 870, in _save_table >> > result = self._do_insert(cls._base_manager, using, fields, >> update_pk, raw) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/base.py", >> line >> > 908, in _do_insert >> > using=using, raw=raw) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/manager.py", >> > line 82, in manager_method >> > return getattr(self.get_queryset(), name)(*args, **kwargs) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/query.py", >> line >> > 1186, in _insert >> > return query.get_compiler(using=using).execute_sql(return_id) >> > File >> > >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", >> line >> > 1375, in execute_sql >> > cursor.execute(sql, params) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/utils.py", >> > line 67, in execute >> > return self._execute_with_wrappers(sql, params, many=False, >> executor=self._execute) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/utils.py", >> > line 76, in _execute_with_wrappers >> > return executor(sql, params, many, context) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/utils.py", >> > line 84, in _execute >> > return self.cursor.execute(sql, params) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/utils.py", >> line 89, in >> > __exit__ >> > raise dj_exc_value.with_traceback(traceback) from exc_value >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/utils.py", >> > line 84, in _execute >> > return self.cursor.execute(sql, params) >> > File >> > >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/django/db/backends/mysql/base.py", >> line >> > 71, in execute >> > return self.cursor.execute(query, args) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/cursors.py", >> line 209, in >> > execute >> > res = self._query(query) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/cursors.py", >> line 315, in >> > _query >> > db.query(q) >> > File >> "/Users/bdickson1/Documents/projects/site-selection/pdbvenv/lib/python3.6/site-packages/MySQLdb/connections.py", >> line >> > 226, in query >> > _mysql.connection.query(self, query) >> > django.db.utils.IntegrityError: (1452, 'Cannot add or update a child >> row: a foreign key constraint fails >> > (`peeringdb`.`peeringdb_network`, CONSTRAINT >> `peeringdb_network_org_id_404d6106_fk_peeringdb_organization_id` FOREIGN KEY >> > (`org_id`) REFERENCES `peeringdb_organization` (`id`))') >> > >> > Please let me know if this is a problem on my end, or a problem with >> the data in the master db itself... >> > >> > If there is something I should do to fix this on my end, any advice >> would be greatly appreciated. >> > (If there is any other way to grab a copy of the tables in mysql >> format, that's all I'm really trying to get, just a snapshot of >> > the current peeringdb database to process with some perl scripts I >> have.) >> > >> > Thanks in advance, >> > >> > Brian Dickson >> > >> > >> > > >
_______________________________________________ Pdb-tech mailing list [email protected] https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech
