Re: [OSM-dev] Synchronous mode

[Tom Hughes]

On 20/02/18 21:16, Paul Norman wrote:

The data loss potential is very minimal, particularly since anything 
that brings down the DB server is likely to cause much larger 
interruptions to mapping than the loss of 5 seconds of PostgreSQL 
replication.


I'd also point out that the data loss risk that page talks about is
only if the primary is a total loss and you decide to flip to the
secondary.

If the primary crashes and restarts then the secondary will replay
the logs in the same way the primary does and both will have the
same view of the data.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Synchronous mode

[Paul Norman]

On 2/20/2018 12:07 PM, Darafei "Komяpa" Praliaskouski wrote:
This suggests that currently apidb replicas aren't in synchronous 
mode. Can it be switched to synchronous?


https://www.postgresql.org/docs/9.5/static/warm-standby.html#SYNCHRONOUS-REPLICATION 



Given the absolutely miserable performance of waiting for the data to 
hit the disk on replicas, this is a bad idea and not normally 
recommended, although we could use remote_apply to reduce some of the 
problems.


The data loss potential is very minimal, particularly since anything 
that brings down the DB server is likely to cause much larger 
interruptions to mapping than the loss of 5 seconds of PostgreSQL 
replication.


It's also not an issue for an editor. The API provides a diff response, 
so the editor doesn't need to make another request.


In fact, this is the first API call I've seen where it is a potential 
issue, and we don't even know if it is one for sure. If MapRoulette is 
doing this as a batch process, it's probably easier to use the changeset 
diffs anyways.


___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Read-only API calls, fair usage

[Komяpa]

>
> There's no guarantee that the read-only changeset call will go to the same
> backend server as the write call.
> https://github.com/openstreetmap/iD/issues/1646 shows the problem caused
> by iD not parsing the diff response and relying on redownloading.
>

This suggests that currently apidb replicas aren't in synchronous mode. Can
it be switched to synchronous?

https://www.postgresql.org/docs/9.5/static/warm-standby.html#SYNCHRONOUS-REPLICATION
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Read-only API calls, fair usage

[Paul Norman]

On 2/20/2018 2:56 AM, Frederik Ramm wrote:

Hi,

On 20.02.2018 10:51, Michał Brzozowski wrote:

You can use changeset replication with ChangesetMD to query your own DB.

I was tempted to respond the same, but I imagine the idea is that as
soon as the user clicks on the "I have fixed this" button, MapRoulette
checks what has been committed... and replication could mean up to one
minute delay.


There's no guarantee that the read-only changeset call will go to the 
same backend server as the write call. 
https://github.com/openstreetmap/iD/issues/1646 shows the problem caused 
by iD not parsing the diff response and relying on redownloading.


In the normal editing workflow a 5s (or even 60s) delay is not an issue. 
Someone is likely to spend longer than that in their editor changing data.


I would say this is a reasonable use of the OSM API, as it is part of 
editing, but is not the best for technical reasons. If you don't need up 
to the second changeset data, you can consume the diff stream every 
minute. If you do need up to the second data, then the API doesn't 
guarantee it.
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Read-only API calls, fair usage

[Simon Poole]

IMHO it boils down to one extra API call per edit made in a maproulette
context(on top of a minimum of two API calls (if the editor is doing
things properly that is actually three), so likely it is simply a bit of
added noise, depending on how expensive
https://wiki.openstreetmap.org/wiki/API_v0.6#Query:_GET_.2Fapi.2F0.6.2Fchangesets
is (which depends on which indices exist on the table etc).

I suppose the timing might be a tiny bit tricky due to replication
delays and against which DB server the query runs in the end.

Simon


Am 20.02.2018 um 01:44 schrieb Martijn van Exel:
> Hi, 
>
> We’re thinking about adding task completion validation to MapRoulette. The 
> basic premise is this: a user completes a task in MapRoulette, and marks it 
> as ‘Fixed'. MapRoulette would then make an OSM API call for recent changesets 
> by that same user, and make a best effort at matching a recent changeset to 
> the task they say they completed. This would mean one call to 
> https://www.openstreetmap.org/api/0.6/changesets?user= for every stated Fix 
> in MapRoulette. Currently the number of fixes is in the thousands per week, 
> but that may grow as we roll out the new version of MapRoulette.
>
> Would you consider this a reasonable use of the OSM API?
>
> Martijn
>
>
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev




signature.asc
Description: OpenPGP digital signature
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Read-only API calls, fair usage

[Frederik Ramm]

Hi,

On 20.02.2018 10:51, Michał Brzozowski wrote:
> You can use changeset replication with ChangesetMD to query your own DB.

I was tempted to respond the same, but I imagine the idea is that as
soon as the user clicks on the "I have fixed this" button, MapRoulette
checks what has been committed... and replication could mean up to one
minute delay. Picture the eager MapRouletteer staring at the "please
wait" cursor for a minute every time they fix something...

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Read-only API calls, fair usage

[Michał Brzozowski]

You can use changeset replication with ChangesetMD to query your own DB.

Michał

20.02.2018 01:45 "Martijn van Exel"  napisał(a):

> Hi,
>
> We’re thinking about adding task completion validation to MapRoulette. The
> basic premise is this: a user completes a task in MapRoulette, and marks it
> as ‘Fixed'. MapRoulette would then make an OSM API call for recent
> changesets by that same user, and make a best effort at matching a recent
> changeset to the task they say they completed. This would mean one call to
> https://www.openstreetmap.org/api/0.6/changesets?user= for every stated
> Fix in MapRoulette. Currently the number of fixes is in the thousands per
> week, but that may grow as we roll out the new version of MapRoulette.
>
> Would you consider this a reasonable use of the OSM API?
>
> Martijn
>
>
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev