Re: [PDB Tech] Problem doing initial 'peeringdb sync', foreign key constraint fails?

2020-05-15 Thread Asbjørn Sloth Tønnesen

Hi Brian,

On 5/15/20 5:43 PM, Brian Dickson wrote:
(This is kind of urgent for me, my goal is to get a snapshot of the data, so any alternative workaround would be helpful 
in the meantime.)


Alternatively you could use:
https://git.2e8.dk/peeringdb-simplesync/about/

--
Best regards
Asbjørn Sloth Tønnesen
___
Pdb-tech mailing list
Pdb-tech@lists.peeringdb.com
https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech


Re: [PDB Tech] IX-F JSON to PeeringDB importer (temporarily) disabled

2019-09-07 Thread Asbjørn Sloth Tønnesen

Hi,

On Fri, 30 Aug 2019 at 17:30, Job Snijders  wrote:
> A significant challenge here is that sometimes records in PeeringDB
> are out of date (which hinders new IXP participants), and sometimes
> IX-F data doesn't (yet) contain information or this information is
> out-of-date (either case hinders a different group). I think the path
> forward is to express the relations and automated actions between the
> information sources as a state machine which (unlike the current
> version) does account for the full lifecycle from provisioning to
> decommissioning of circuits to IXPs.

Since the thing that's unwanted is stale incorrect netixlan entries,
it should be sufficient to just limit auto-deletion to netixlan's which
hasn't been created/updated in X time. X time should be long enough
to not cause problems with provisioning, a good value might be
3 months.

A problem there might be that "updated", could have changed based
on IX-F or a PDB admin change. Would it make sense to have a separate
"updated-by-owner" timestamp, to indicate if and when the object
owner last touched the object? Could also be "updated-by-net" and
"updated-by-ix" columns.

On Fri, 30 Aug 2019 at 20:12, Lukas Tribus  wrote:

A state machine handling the full life-cycle seems overly complex,
with lots of moving parts. I fear that would also result in a number
of disputes that is similarly unsatisfactory. I mean, you don't really
know how the IX constructs this data - a ASN may be removed from that
list for a short period of time to due to a temporary payment issue.
In fact it looks like the disputes are mainly caused by
bad/imprecise/outdated information from IXes, so building more
complexity around it seems like a bad idea to me.


The temporary missing from the IX-F feed cases, could be handled by a
"marked_for_deletion" column with current timestamp.

In the new/updated netixlan case covered above the flag, will first be
put on the object after the X time since created/updated has expired
and it is still not in the IX-F feed.

A daily job deletes objects which was "marked_for_deletion" more
than eg. 2 weeks ago. If "updated-by-owner" is newer than
"marked_for_deletion" the flag is removed.

If it reappears in IX-F data the flag is also removed.

As an alternative for the "updated-by-owner" column, the website
code could automatically remove the "marked_for_deletion" flag,
if the object is updated by the network.

This boils down to long leash if the network is active, shorter if not.

The "updated-by-owner" could also just be on an org/net level,
rather than individual netixlan.

To sum-up reformulated as simple rules:

An netixlan SHALL NOT be automatically deleted if:
- the owner has touched it less than X time ago
- if it has been missing from IX-F feed for less than Y time
where Y time < X time

--
Best regards
Asbjørn Sloth Tønnesen
___
Pdb-tech mailing list
Pdb-tech@lists.peeringdb.com
https://lists.peeringdb.com/cgi-bin/mailman/listinfo/pdb-tech