Re: [OSM-talk] need advice for clever query or script
On Sat, Aug 30, 2014 at 02:09:34AM +0200, Imre Samu wrote: I have got one problem, some of the queries with most bridges do not load into JOSM. Josm says contacting the server and thats all. strange ... :( maybe timeout when generate JOSM data? my impression was that the problem was on the overpass website as it also generated a very strange shorturl for sharing. *#2. You can split the big query-s - to smaller ones ..* yes, thats what I was doing. Is actually nicer to edit in smaller batches and the problem is triggered only for the 3 or 4 most frequent contributors. *Other solution -Filter in JOSM* * JOSM File / Open location : http://www.overpass-api.de/api/xapi_meta?way[bridge=swing] * and after open the Filter menu ( Windows→Filter from the menu, or Alt+Shift+F from the keyboard) https://wiki.openstreetmap.org/wiki/JOSM/Search_function https://www.mapbox.com/blog/2012-08-15-using-filters-josm/ * and copy the query ( without the global keyword ( ..maybe... ) ) but I am not perfect in JOSM ... so if you find th correct JOSM filtering please share with me ... :) yes that also works, I actually use the Download from Overpass API with [timeout:600];way[bridge=swing];(._;;);out meta; and in JOSM use the plain search to select the subset that I want. bridge=swing and ( id:50483896 or id:51513287 or id:51831532 or id:52209220 or id:59430361 or id:88300946 or id:88863770 or id:99211716 ) Never looked at filters yet, it might be that they would hide the nodes? Richard ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
Hi Richard,
clever query or script
maybe you can use OSM OPL format this ad-hoc query,
and process the data with sed, awk, grep .
my draft ubuntu script - with comments.
https://gist.github.com/ImreSamu/be49fd1ce511975325d2#file-bridge_swing-sh
result ( Overpass-Wizard Query - but you can export the data to JOSM )
https://gist.github.com/ImreSamu/a2dd0a8c25f0fea5284c#file-bridge_swing_overpass_wizard-md
the example result - Overpass-Wizard Query :
type:way and ( id:28134411 or id:29295367 or id:30178341 or id:30178382 or
id:33132931 or id:33132936 or id:33132949 ) global
( http://wiki.openstreetmap.org/wiki/Overpass_turbo/Wizard - see
Meta-Data Filters )
*How to check the query *
** go http://overpass-turbo.eu/ http://overpass-turbo.eu/ *
** select Wizard*
* * copy /paste the generated query : type:way and ( id:27001207 or
id:72584563 ) global*
* * Build and run query - and check the result.*
* * if you got timeout error, then check the generated script timeout (
osm-script output=json timeout=25 ) and set to 200 - and rerun *
** you can load data into an OSM editor: JOSM, - see Export menu *
From the OSM OPL history - very easy to grep the first contributor
# convert the osm history file to OPL ( *osmium cat w11323607.osh -f opl* )
# sample OSM History OPL file:
# w100646626 v1 dV c7343540 t2011-02-20T15:08:57Z i37137
uDerick%0020Rethans Tbridge=yes,highway=footway
Nn309461645,n1163494643
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
# w100646626 v3 dV c18538313 t2013-10-25T16:16:57Z i24119 uMauls
Tbridge=swing,highway=footway Nn309461645,n2508499967
# w100646626 v4 dV c25024407 t2014-08-26T10:51:45Z i66391 ugeozeisig
Tbridge=movable,bridge:movable=swing,highway=footway
Nn309461645,n2508499967
# w100646626 v5 dV c25050883 t2014-08-27T12:42:02Z i66391 ugeozeisig
Tbridge=swing,highway=footway Nn309461645,n2508499967
#
# filter the results by the first contributor who added bridge=swing to the way
# ( *egrep -m 1 '( T|,)bridge=swing'* )
#result: v2
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
#
The OPL format - from the OSMIUM manual:
v - Version
d - Deleted flag ('V' - visible or 'D' - deleted)
c - Changeset ID
t - Timestamp (ISO Format)
i - User ID
u - Username
T - Tags
x - Longitude (nodes only)
y - Latitude (nodes only)
N - Nodes (ways only)
M - Members (relations only)
you can find other interesting examples in the OSMIUM manual
Find all users who have created post boxes:
egrep ' v1 ' data.osm.opl | egrep 'amenity=post_box' | cut -d' ' -f7 |
cut -c2- | sort -u
OSMIUM tool - and more examples :
http://osmcode.org/libosmium/manual/libosmium-manual.html#output-formats
https://www.sotm-eu.org/en/slots/36
https://www.sotm-eu.org/slides/44.pdf
Imre
2014-08-28 12:39 GMT+02:00 Richard Z. ricoz@gmail.com:
Hi,
trying to clean up bridge=swing as far as possible. There was at least
user in the past who used the combination systematically wrong, so I want
to split the result by user who introduced the bridge=swing.
To make things complicated - a few days ago one contributor did a well
meant effort to convert all
bridge=swing - bridge=movable+bridge:movable=swing
and reverted that edit because there were too many errors in it. Hence
doing a naive search for user doesn't work.
So I want to :
* find all bridge=swing
* split results by the first contributor who added bridge=swing
to the way
* get the results into JOSM for examination and editing
Tia for any hints,
Richard
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
On 28/08/2014 11:39, Richard Z. wrote: To make things complicated - a few days ago one contributor did a well meant effort to convert all bridge=swing - bridge=movable+bridge:movable=swing and reverted that edit because there were too many errors in it. Hence doing a naive search for user doesn't work. In the meantime I guess that people could try and search out bridges from that changeset that they've mapped themselves or are familiar with, and change them if the new tagging applies. I've done one (originally added by me 5 years ago) near me... Cheers, Andy ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
On Fri, Aug 29, 2014 at 02:22:10PM +0200, Imre Samu wrote:
Hi Imre,
thousands thanks - everything seems to work perfectly as described.
one question - if I would want to compile osmium myself the README says
that no files need to be build and doesn't say where osmium comes from?
Richard
clever query or script
maybe you can use OSM OPL format this ad-hoc query,
and process the data with sed, awk, grep .
my draft ubuntu script - with comments.
https://gist.github.com/ImreSamu/be49fd1ce511975325d2#file-bridge_swing-sh
result ( Overpass-Wizard Query - but you can export the data to JOSM )
https://gist.github.com/ImreSamu/a2dd0a8c25f0fea5284c#file-bridge_swing_overpass_wizard-md
the example result - Overpass-Wizard Query :
type:way and ( id:28134411 or id:29295367 or id:30178341 or id:30178382 or
id:33132931 or id:33132936 or id:33132949 ) global
( http://wiki.openstreetmap.org/wiki/Overpass_turbo/Wizard - see
Meta-Data Filters )
*How to check the query *
** go http://overpass-turbo.eu/ http://overpass-turbo.eu/ *
** select Wizard*
* * copy /paste the generated query : type:way and ( id:27001207 or
id:72584563 ) global*
* * Build and run query - and check the result.*
* * if you got timeout error, then check the generated script timeout (
osm-script output=json timeout=25 ) and set to 200 - and rerun *
** you can load data into an OSM editor: JOSM, - see Export menu *
From the OSM OPL history - very easy to grep the first contributor
# convert the osm history file to OPL ( *osmium cat w11323607.osh -f opl* )
# sample OSM History OPL file:
# w100646626 v1 dV c7343540 t2011-02-20T15:08:57Z i37137
uDerick%0020Rethans Tbridge=yes,highway=footway
Nn309461645,n1163494643
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
# w100646626 v3 dV c18538313 t2013-10-25T16:16:57Z i24119 uMauls
Tbridge=swing,highway=footway Nn309461645,n2508499967
# w100646626 v4 dV c25024407 t2014-08-26T10:51:45Z i66391 ugeozeisig
Tbridge=movable,bridge:movable=swing,highway=footway
Nn309461645,n2508499967
# w100646626 v5 dV c25050883 t2014-08-27T12:42:02Z i66391 ugeozeisig
Tbridge=swing,highway=footway Nn309461645,n2508499967
#
# filter the results by the first contributor who added bridge=swing to the
way
# ( *egrep -m 1 '( T|,)bridge=swing'* )
#result: v2
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
#
The OPL format - from the OSMIUM manual:
v - Version
d - Deleted flag ('V' - visible or 'D' - deleted)
c - Changeset ID
t - Timestamp (ISO Format)
i - User ID
u - Username
T - Tags
x - Longitude (nodes only)
y - Latitude (nodes only)
N - Nodes (ways only)
M - Members (relations only)
you can find other interesting examples in the OSMIUM manual
Find all users who have created post boxes:
egrep ' v1 ' data.osm.opl | egrep 'amenity=post_box' | cut -d' ' -f7 |
cut -c2- | sort -u
OSMIUM tool - and more examples :
http://osmcode.org/libosmium/manual/libosmium-manual.html#output-formats
https://www.sotm-eu.org/en/slots/36
https://www.sotm-eu.org/slides/44.pdf
Imre
2014-08-28 12:39 GMT+02:00 Richard Z. ricoz@gmail.com:
Hi,
trying to clean up bridge=swing as far as possible. There was at least
user in the past who used the combination systematically wrong, so I want
to split the result by user who introduced the bridge=swing.
To make things complicated - a few days ago one contributor did a well
meant effort to convert all
bridge=swing - bridge=movable+bridge:movable=swing
and reverted that edit because there were too many errors in it. Hence
doing a naive search for user doesn't work.
So I want to :
* find all bridge=swing
* split results by the first contributor who added bridge=swing
to the way
* get the results into JOSM for examination and editing
Tia for any hints,
Richard
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
if I would want to compile osmium myself the README says
that no files need to be build and doesn't say where osmium comes from?
I have used the code and instructions from this 2 repo
https://github.com/osmcode/libosmium ( osmium library )
https://github.com/osmcode/osmium-tool ( osmium command line tool )
if you have a some problems with the installation,
then I can create a Docker Container ( www.docker.com ) for you.
Imre
2014-08-29 15:55 GMT+02:00 Richard Z. ricoz@gmail.com:
On Fri, Aug 29, 2014 at 02:22:10PM +0200, Imre Samu wrote:
Hi Imre,
thousands thanks - everything seems to work perfectly as described.
one question - if I would want to compile osmium myself the README says
that no files need to be build and doesn't say where osmium comes from?
Richard
clever query or script
maybe you can use OSM OPL format this ad-hoc query,
and process the data with sed, awk, grep .
my draft ubuntu script - with comments.
https://gist.github.com/ImreSamu/be49fd1ce511975325d2#file-bridge_swing-sh
result ( Overpass-Wizard Query - but you can export the data to JOSM )
https://gist.github.com/ImreSamu/a2dd0a8c25f0fea5284c#file-bridge_swing_overpass_wizard-md
the example result - Overpass-Wizard Query :
type:way and ( id:28134411 or id:29295367 or id:30178341 or id:30178382
or
id:33132931 or id:33132936 or id:33132949 ) global
( http://wiki.openstreetmap.org/wiki/Overpass_turbo/Wizard - see
Meta-Data Filters )
*How to check the query *
** go http://overpass-turbo.eu/ http://overpass-turbo.eu/ *
** select Wizard*
* * copy /paste the generated query : type:way and ( id:27001207 or
id:72584563 ) global*
* * Build and run query - and check the result.*
* * if you got timeout error, then check the generated script timeout (
osm-script output=json timeout=25 ) and set to 200 - and rerun
*
** you can load data into an OSM editor: JOSM, - see Export menu *
From the OSM OPL history - very easy to grep the first contributor
# convert the osm history file to OPL ( *osmium cat w11323607.osh -f
opl* )
# sample OSM History OPL file:
# w100646626 v1 dV c7343540 t2011-02-20T15:08:57Z i37137
uDerick%0020Rethans Tbridge=yes,highway=footway
Nn309461645,n1163494643
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
# w100646626 v3 dV c18538313 t2013-10-25T16:16:57Z i24119 uMauls
Tbridge=swing,highway=footway Nn309461645,n2508499967
# w100646626 v4 dV c25024407 t2014-08-26T10:51:45Z i66391 ugeozeisig
Tbridge=movable,bridge:movable=swing,highway=footway
Nn309461645,n2508499967
# w100646626 v5 dV c25050883 t2014-08-27T12:42:02Z i66391 ugeozeisig
Tbridge=swing,highway=footway Nn309461645,n2508499967
#
# filter the results by the first contributor who added bridge=swing to
the way
# ( *egrep -m 1 '( T|,)bridge=swing'* )
#result: v2
# w100646626 v2 dV c12447614 t2012-07-23T09:52:38Z i404175 urickogden
Tbridge=swing,highway=footway Nn309461645,n1163494643
#
The OPL format - from the OSMIUM manual:
v - Version
d - Deleted flag ('V' - visible or 'D' - deleted)
c - Changeset ID
t - Timestamp (ISO Format)
i - User ID
u - Username
T - Tags
x - Longitude (nodes only)
y - Latitude (nodes only)
N - Nodes (ways only)
M - Members (relations only)
you can find other interesting examples in the OSMIUM manual
Find all users who have created post boxes:
egrep ' v1 ' data.osm.opl | egrep 'amenity=post_box' | cut -d' ' -f7 |
cut -c2- | sort -u
OSMIUM tool - and more examples :
http://osmcode.org/libosmium/manual/libosmium-manual.html#output-formats
https://www.sotm-eu.org/en/slots/36
https://www.sotm-eu.org/slides/44.pdf
Imre
2014-08-28 12:39 GMT+02:00 Richard Z. ricoz@gmail.com:
Hi,
trying to clean up bridge=swing as far as possible. There was at least
user in the past who used the combination systematically wrong, so I
want
to split the result by user who introduced the bridge=swing.
To make things complicated - a few days ago one contributor did a well
meant effort to convert all
bridge=swing - bridge=movable+bridge:movable=swing
and reverted that edit because there were too many errors in it. Hence
doing a naive search for user doesn't work.
So I want to :
* find all bridge=swing
* split results by the first contributor who added bridge=swing
to the way
* get the results into JOSM for examination and editing
Tia for any hints,
Richard
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
On Fri, Aug 29, 2014 at 04:12:08PM +0200, Imre Samu wrote: if I would want to compile osmium myself the README says that no files need to be build and doesn't say where osmium comes from? I have used the code and instructions from this 2 repo https://github.com/osmcode/libosmium ( osmium library ) https://github.com/osmcode/osmium-tool ( osmium command line tool ) ok, the https://github.com/osmcode/osmium-tool was the missing link. if you have a some problems with the installation, then I can create a Docker Container ( www.docker.com ) for you. hopefully not necessary.. I do not need it quickly and if I will do it will try to create all the RPM specfiles. Richard ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
I have cleaned my Dockerfile - maybe you can use. ( if you replace RUN to sudo - you can install line - by line ) https://gist.github.com/ImreSamu/5447252a4e750fd8c61e#file-dockerfile_for_install_osmium_tool good luck :) Imre 2014-08-29 16:37 GMT+02:00 Richard Z. ricoz@gmail.com: On Fri, Aug 29, 2014 at 04:12:08PM +0200, Imre Samu wrote: if I would want to compile osmium myself the README says that no files need to be build and doesn't say where osmium comes from? I have used the code and instructions from this 2 repo https://github.com/osmcode/libosmium ( osmium library ) https://github.com/osmcode/osmium-tool ( osmium command line tool ) ok, the https://github.com/osmcode/osmium-tool was the missing link. if you have a some problems with the installation, then I can create a Docker Container ( www.docker.com ) for you. hopefully not necessary.. I do not need it quickly and if I will do it will try to create all the RPM specfiles. Richard ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] need advice for clever query or script
On Fri, Aug 29, 2014 at 02:22:10PM +0200, Imre Samu wrote: Hi Imre, I have got one problem, some of the queries with most bridges do not load into JOSM. Josm says contacting the server and thats all. When I share the query the shorturl looks like the one bellow so it seems to be somehow an issue at http://overpass-turbo.eu/ ?
Re: [OSM-talk] need advice for clever query or script
I have got one problem, some of the queries with most bridges do not load into JOSM. Josm says contacting the server and thats all. strange ... :( maybe timeout when generate JOSM data? but you can optimize the query some tips: *#1. if you know wich country - replace the global keyword in the query to the precise territory filter ..* ( see Location Filters in http://wiki.openstreetmap.org/wiki/Overpass_turbo/Wizard ) global -in USA global -in Vienna global- in Netherlands *#2. You can split the big query-s - to smaller ones ..* from this big query type:way and ( id:105807680 or id:128093288 or id:135527140 or id:140757611 or id:158593665 or id:158608739 or id:158608747 or id:162817769 or id:162988896 or id:163435617 or id:163451937 or id:163640424 or id:186555649 or id:186667441 or id:190721707 or id:193652513 or id:196184968 or id:199818438 or id:242264939 or id:245514123 or id:268020828 or id:300530708 or id:42712814 or id:50483896 or id:51513287 or id:51831532 or id:52209220 or id:59430361 or id:88300946 or id:88863770 or id:99211716 ) global you can manually create 4 sub - query and you have to run 4 times. -1) type:way and ( id:105807680 or id:128093288 or id:135527140 or id:140757611 or id:158593665 or id:158608739 or id:158608747 ) global -2) type:way and ( id:162817769 or id:162988896 or id:163435617 or id:163451937 or id:163640424 or id:186555649 or id:186667441 or id:190721707 ) global -3) type:way and ( id:193652513 or id:196184968 or id:199818438 or id:242264939 or id:245514123 or id:268020828 or id:300530708 or id:42712814 ) global -4) type:way and ( id:50483896 or id:51513287 or id:51831532 or id:52209220 or id:59430361 or id:88300946 or id:88863770 or id:99211716 ) global *Other solution -Filter in JOSM* * JOSM File / Open location : http://www.overpass-api.de/api/xapi_meta?way[bridge=swing] * and after open the Filter menu ( Windows→Filter from the menu, or Alt+Shift+F from the keyboard) https://wiki.openstreetmap.org/wiki/JOSM/Search_function https://www.mapbox.com/blog/2012-08-15-using-filters-josm/ * and copy the query ( without the global keyword ( ..maybe... ) ) but I am not perfect in JOSM ... so if you find th correct JOSM filtering please share with me ... :) Imre 2014-08-29 23:24 GMT+02:00 Richard Z. ricoz@gmail.com: On Fri, Aug 29, 2014 at 02:22:10PM +0200, Imre Samu wrote: Hi Imre, I have got one problem, some of the queries with most bridges do not load into JOSM. Josm says contacting the server and thats all. When I share the query the shorturl looks like the one bellow so it seems to be somehow an issue at http://overpass-turbo.eu/ ?
[OSM-talk] need advice for clever query or script
Hi, trying to clean up bridge=swing as far as possible. There was at least user in the past who used the combination systematically wrong, so I want to split the result by user who introduced the bridge=swing. To make things complicated - a few days ago one contributor did a well meant effort to convert all bridge=swing - bridge=movable+bridge:movable=swing and reverted that edit because there were too many errors in it. Hence doing a naive search for user doesn't work. So I want to : * find all bridge=swing * split results by the first contributor who added bridge=swing to the way * get the results into JOSM for examination and editing Tia for any hints, Richard ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
