Re: [OSM-dev] planet.osm.org is verry slow

[marc marc]

Le 25.01.20 à 21:43, Simon Poole a écrit :
> 
> Am 25.01.2020 um 21:12 schrieb marc marc:
>> is there a convenient way to update a pbf to exactly the same diff as
>> osm.org? and/or add the state.txt of the pbf produced on osm.org?
>> this would allow to have the same pbf mirrored on other sites without
>> having to download and/or encode the same thing several times
> 
> The answer is likely no, because I don't believe there is a guarantee
> that the planet dump contains exactly the commits of a certain set of
> diffs,

the planet dump is done from the db itselft and not using by a cascade
of diffs (min->hourly>daily>weekly) ?
or maybe a special diff between planet dump.

> But that doesn't matter in a practical sense

it avoids having different files between mirrors, which is quite
annoying if a bug occurs on one variant and not on another depending
on the "mirror".

> In any case there is no argument to be made for the ~1'000 downloads 
> that start shortly every week after the plant dump has been produced, 

I agree.
maybe a rate limit "1st full speed" or "full speed for mirror"

> and of which for weird reasons 25% are actually
> downloading the compressed XML file (which is 30% larger).

there are probably some tools that don't support pbf (for example
osmfilter (I don't mind too much, I do an osmconvert before, but I guess
this extra step is not done and/or not available and/or not known by
other people).

> 
> Simon
> 
>>
>> Le 25.01.20 à 18:34, marc tobias a écrit :
>>> Wolfram, who asked about the bandwidth, is already running a mirror
>>> https://download.bbbike.org/osm/planet/
>>> (listed on https://wiki.openstreetmap.org/wiki/Planet.osm#Downloading)
>>> so downloading it every week is likely trying to update his mirror.
>>>
>>> marc tobias
>>>
>>>> Message: 2
>>>> Date: Sat, 25 Jan 2020 11:46:20 +0100
>>>> From: Simon Poole 
>>>> To: dev@openstreetmap.org
>>>> Subject: Re: [OSM-dev] planet.osm.org is verry slow
>>>> Message-ID: <4f5c3f2c-f3c9-0545-0dd2-98df390e4...@poole.ch>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>> Yes, there are now bandwidth restrictions on connections to
>>>> planet.openstreetmap.org because the usage had skyrocketed and outgoing
>>>> bandwidth was completely saturated.
>>>>
>>>> But the obvious question is: why are you even trying to download the
>>>> full planet twice within a fortnight? Particularly given the file can
>>>> easily be updated in situ.
>>>>
>>>> Simon
>>> ___
>>> dev mailing list
>>> dev@openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/dev
>>>
>> ___
>> dev mailing list
>> dev@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/dev
> 

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] planet.osm.org is verry slow

[marc marc]

is there a convenient way to update a pbf to exactly the same diff as
osm.org? and/or add the state.txt of the pbf produced on osm.org?
this would allow to have the same pbf mirrored on other sites without
having to download and/or encode the same thing several times

Le 25.01.20 à 18:34, marc tobias a écrit :
> Wolfram, who asked about the bandwidth, is already running a mirror
> https://download.bbbike.org/osm/planet/
> (listed on https://wiki.openstreetmap.org/wiki/Planet.osm#Downloading)
> so downloading it every week is likely trying to update his mirror.
> 
> marc tobias
> 
>> Message: 2
>> Date: Sat, 25 Jan 2020 11:46:20 +0100
>> From: Simon Poole 
>> To: dev@openstreetmap.org
>> Subject: Re: [OSM-dev] planet.osm.org is verry slow
>> Message-ID: <4f5c3f2c-f3c9-0545-0dd2-98df390e4...@poole.ch>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Yes, there are now bandwidth restrictions on connections to
>> planet.openstreetmap.org because the usage had skyrocketed and outgoing
>> bandwidth was completely saturated.
>>
>> But the obvious question is: why are you even trying to download the
>> full planet twice within a fortnight? Particularly given the file can
>> easily be updated in situ.
>>
>> Simon
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
> 

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] status/prognosis of Merkaator?

[marc marc]

Le 28.10.19 à 15:29, Greg Troxel a écrit :
>Is Merkaator a reasonable choice for OSM editing, it terms of working
>with current APIs and preset notions?

I never use it myself but accord to the stat, it's still in use
in the top 10 of end-user editor.
https://wiki.openstreetmap.org/wiki/Editor_usage_stats

>Is there any reason why someone would choose to use merkaator rather
>than josm (other than wanting to use it on a platform where Java is
>unavailable)?

tastes and colours are an irrational argument.

But seeing that even the screenshot seems to have disappeared from the 
site, it gives the impression of abandonment due to lack of time
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Export: (limit is 50000).

[marc marc]

Le 06.08.19 à 17:11, Dave F via dev a écrit :
>> - all that export does is redirect
>> to the same API that JOSM etc use.
> Yes, I've noticed the limitations of JOSM

josm allow to download data from overpass,
which allows a larger limit.

osm-fr had also an api proxy that redirect
some request to overpass to reduce api load
Unfortunately too few mapper was using it and it is currently offline
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Workflow for bulk upload of GPX files

[marc marc]

Le 20.05.19 à 19:03, Ajay Singh1 a écrit :
> I can just upload and someone else will make roads by looking at that trace.

of course.
but since there is no toolsosm.org api for a contributor to download
the unprocessed traces, it is like hoping that someone else will do
all the work again that you have done of detecting that these traces are 
missing ways in osm
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Workflow for bulk upload of GPX files

[marc marc]

if you want to add yourself all these missing ways in osm,
a simpler workflow :
give a part to each one
load the gpx trace with the editor of your choice
simplify and connect it to the concerned ways, add tags
upload the new ways into osm.
some will say you may send the trace too.
once finished, delete the trace of your computer
or move it to a "done" directory

and so on with the next trace until you finish your part :)

Le 20.05.19 à 15:12, Ajay Singh1 via dev a écrit :
> Hi Joseph,
> 
> Thanks for your response. I've extensively cleaned the GPX data and I'm 
> marking only those roads as 'missing' ones where I'm getting trace more 
> than a threshold number of times, say 5 times.
> 
> Few questions -
> 1. Should I use something like 
> https://github.com/hotosm/tasking-manager to manage the entire process?
> 2. As you mentioned - "/If possible, the mappers should look at more 
> than one GPS trace on the/
> /road: /", do the mapper has to look at GPS in one tab and then make the 
> corresponding change in openstreetmap.org ? 
> I've checked various ways to upload GPX data on OSM but I'm not getting 
> it right.
> 
> Is there any documentation/tutorial for this process?
> 
> 
> On Mon, May 20, 2019 at 5:56 PM Joseph Eisenberg 
> mailto:joseph.eisenb...@gmail.com>> wrote:
> 
> Ajay,
> 
> It's great that you are adding more GPS traces for mappers to use.
> These are really helpful for checking the offset of aerial imagery and
> finding roads and paths to map.
> 
> I think you first 2 steps look fine. As far as tags for GPS traces,
> you can look at some other traces in your area to see what mappers
> usually add. It's common to mention that the trace was obtained by
> motorcycle, for example
> 
> For the 3rd step, make sure you follow the best practice for adding
> highways to OSM. Don't import the GPS traces directly.
> 
> A single GPS trace along a road will be offset from the centerline,
> since the motorcyclist has to travel on one side of the road. Also,
> GPS traces may have errors due to problems with poor signal or
> reflections in some areas. And GPS traces have too many nodes on
> strait sections, but sometimes too few around curves.
> 
> If possible, the mappers should look at more than one GPS trace on the
> road: at least one from each direction, so that the center of the road
> is clear. If possible, also check aerial imagery and compare to the
> GPS trace. The mappers can use the GPS traces and the aerial imagery
> together to find the centerline of the highway and map that, using the
> right number of nodes for straight sections and curves.
> 
> Because of this, it's not possible to find out exactly how many of
> your GPX files have become roads. But if you are adding GPS traces on
> roads that are currently missing in OSM, it's very likely that your
> work will be a main reason that new roads are added, and it will also
> make them more accurate.
> 
> On 5/17/19, Ajay Singh1 via dev  > wrote:
>  > Hi Team,
>  >
>  > I have got a few thousand cleaned GPX files of motorbikes. These
> trace
>  > files majorly represent missing roads in OSM (which I have
> derived by using
>  > months of bikers data and identifying tracepoints which are not
> near any of
>  > the road segments in OSM).
>  > I want to contribute to OSM on a regular basis and want to take
> an opinion
>  > on my workflow for submission of these trace files to OSM.
>  >
>  > Workflow:
>  >  1. Upload all GPX files in bulk using OSM UI(
>  > https://wiki.openstreetmap.org/wiki/Upload_GPS_tracks) or using API(
>  > https://wiki.openstreetmap.org/wiki/API_v0.6).
>  >  2. Pass some GPS-tags, e.g. name, id, status, etc. So, that I
> can query
>  > data that I have uploaded.
>  >  3. Have a team of 2-3 people who can then use JOSM or other
> editors to
>  > make ways/roads from this GPX data to OSM
>  >
>  > Questions:
>  >  1. Is this a correct workflow for uploading and making
> ways/roads from GPX
>  > files?
>  >  2. Can I use GPS-tags to track of how many gpx files have been
> converted
>  > into ways/roads?
>  >
>  > Also, Is there any alternative to have this capability?
>  >
>  > Awaiting a quick response.
>  >
>  > --
>  > Regards,
>  > Ajay Singh
>  >
> 
> 
> 
> -- 
> Regards,
> Ajay Singh
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
> 

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Question Answering over OSM and searching by tags

[marc marc]

Hello,

overpass turbo assistant do that
I'm maybe a good idea to improve it

Regards,
Marc

Le 29. 10. 18 à 14:06, Dennis Diefenbach a écrit :
> Hi Bryan,
> 
> thank you for your answer! I gave a rapid look at the API. I agree that 
> it gives back point of interests in a give region, but what I was 
> speaking about is something more “end-user” friendly. So an end-user 
> cannot simply search for “tram stops in rome”. The only way is to 
> construct a formal query like:
> 
> node
>    [amenity=tram_stop]
>    ({{bbox}});
> out;
> 
> which is not trivial.
> 
> Salut
> D063520
> 
> 
>    
> Dennis Diefenbach
> Doctorand
> dennis.diefenb...@univ-st-etienne.fr 
> 
> 
>   
> Laboratoire Hubert Curien
> Campus Manufacture
> 18 RUE Pr Benoît Lauras
> 42000 SAINT-ETIENNE
> 04 77 91 57 80
> http://laboratoirehubertcurien.fr
> 
>   
> 
>  
>  
>  
> 
>  
>   
> 
>   
> 
> 
>> On 29. Oct 2018, at 13:28, Bryan Housel > > wrote:
>>
>>> On Oct 29, 2018, at 5:58 AM, Dennis Diefenbach 
>>> >> > wrote:
>>>
>>> Especially I found that currently it is not possible in OSM to search 
>>> for nodes, ways, relation, with a specific tag (like tram stop, 
>>> restaurants, post offices and so on), i.e. restaurants in saint 
>>> etienne . Would that be interesting for the community?
>>
>> Most people are currently using Overpass for queries like this..
>>
>> about: https://wiki.openstreetmap.org/wiki/Overpass_API
>> live tool: https://overpass-turbo.eu/
>>
>> Thanks, Bryan
>>
> 
> 
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
> 

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Fatal Error

[marc marc]

Le 31. 07. 18 à 13:01, Leon Boehmer a écrit :
> Currently we are running our own version of overpass. But we are getting 
> this error when we run the apply_to_osc file. Within the reboot.sh file.
> Reading XML file ... finished reading nodes. Version 2 has a later or 
> equal timestamp (2018-07-24T17:12:04Z) than version 3 
> (2018-07-24T17:12:04Z) of Node 2180084868

you may ignore this (it happend when a editor send 2 versions
of the same object with the same timestamp)

> compute_geometry: Node 21157203 used in way 4009269 not found.

this problem can have 2 reasons:
- you do not have a worldwide base and in this case it is normal that a 
capital of a node outside your geographical zone causes this warning 
message.
or
- your base is corrupt or has missed a diff.
You can try to replay diffs from this timestamp but the most realistic 
is to purge the base and redo the import. this can also happen if you 
mis-initialized the sequence number after the import

dédicated overpass mailinglist :
https://listes.openstreetmap.fr/wws/info/overpass

Regards,
Marc

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Dockerfile for overpass api

[marc marc]

Le 19. 07. 18 à 17:45, mmd a écrit :
> Am 19.07.2018 um 16:32 schrieb Stadtherr, Eric - US:
> 
>> I've had success with this one while doing some prototype testing, and
>> it worked well for our test case, but I haven't used it in a production
>> environment:
>> https://hub.docker.com/r/wiktorn/overpass-api/

> As none of the public Overpass instances run by Roland are based on
> Docker images

Rodolphe @ osm-fr make an ansible role used for the osm-fr instance
on a debian vm.
you may run this role in a docker but we didn't test it on other distro.

Regards,
Marc
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev