Re: [OSM-dev] Missing records from osmium-tool tags-filter

2019-11-21 Thread Jochen Topf
Hi!

The Osmium version 1.11.1 released yesterday has the fix.

Jochen

On Tue, Nov 12, 2019 at 06:31:14PM +0100, Jochen Topf wrote:
> Date: Tue, 12 Nov 2019 18:31:14 +0100
> From: Jochen Topf 
> To: Craig Kochis via dev 
> Subject: Re: [OSM-dev] Missing records from osmium-tool tags-filter
> 
> Hi!
> 
> Okay, turns out there is a bug in Osmium. It removes the tags from too
> many objects (in this case way 24801630). This only happens if a
> matching way is also referenced from a matching relation. I'll make a
> fix soon.
> 
> Jochen
> 
> On Tue, Nov 12, 2019 at 04:22:24PM +0100, Jochen Topf wrote:
> > Date: Tue, 12 Nov 2019 16:22:24 +0100
> > From: Jochen Topf 
> > To: Craig Kochis via dev 
> > Subject: Re: [OSM-dev] Missing records from osmium-tool tags-filter
> > 
> > Hi!
> > 
> > When I try to run https://www.openstreetmap.org/way/24801630 through the
> > commands you provided, it will come out fine. In fact it comes out
> > twice, once as LineString, once as MultiPolygon.
> > 
> > Tested with:
> > 
> > wget -O w24801630.osm 
> > https://www.openstreetmap.org/api/0.6/way/24801630/full
> > osmium tags-filter w24801630.osm amenity building craft cuisine diet 
> > leisure office public_transport railway shop sport aeroway --remove-tags -v 
> > -o poi.osm.pbf
> > osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs 
> > --add-unique-id=type_id -v -o poi.json
> > 
> > This shouldn't be different if it is in the planet file. Where did your
> > planet file come from? Can you provide a minimal example instead of one
> > that needs an (unspecified) planet file?
> > 
> > Jochen
> > 
> > On Tue, Nov 12, 2019 at 09:45:32AM -0500, Craig Kochis via dev wrote:
> > > Date: Tue, 12 Nov 2019 09:45:32 -0500
> > > From: Craig Kochis via dev 
> > > To: dev@openstreetmap.org
> > > Subject: [OSM-dev] Missing records from osmium-tool tags-filter
> > > 
> > > First off, just want to say this tool has been amazing, it's blazing fast
> > > for parsing through huge amounts of data.
> > > 
> > > There's probably something I'm not understanding about how the tags-filter
> > > command is working, but there are some records missing from the output 
> > > that
> > > I'd expect to be there based on the commands. For instance, I'm running 
> > > the
> > > following command to extract some POIs I'm interested in:
> > > 
> > > osmium tags-filter planet.osm.pbf amenity building craft cuisine diet
> > > leisure office public_transport railway shop sport aeroway --remove-tags 
> > > -v
> > > -o poi.osm.pbf
> > > 
> > > Then exporting to JSON
> > > 
> > > osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs
> > > --add-unique-id=type_id -v -o poi.json
> > > 
> > > And some records are being omitted that I would expect to be in the 
> > > output.
> > > For instance, Yankee Stadium:
> > > https://www.openstreetmap.org/way/24801630 matches
> > > the "leisure" and "sport" tags, yet that way isn't in the result set. If I
> > > run tags-filter with just "leisure" it will show up, so I'm wondering if
> > > there's something happening by specifying multiple tags that I'm not aware
> > > of.
> > > 
> > > Any guidance on this would be greatly appreciated. Thanks!
> > 
> > > ___
> > > dev mailing list
> > > dev@openstreetmap.org
> > > https://lists.openstreetmap.org/listinfo/dev
> > 
> > 
> > -- 
> > Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  
> > +49-351-31778688
> > 
> > ___
> > dev mailing list
> > dev@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/dev
> 
> -- 
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev

-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Missing records from osmium-tool tags-filter

2019-11-12 Thread Jochen Topf
Hi!

Okay, turns out there is a bug in Osmium. It removes the tags from too
many objects (in this case way 24801630). This only happens if a
matching way is also referenced from a matching relation. I'll make a
fix soon.

Jochen

On Tue, Nov 12, 2019 at 04:22:24PM +0100, Jochen Topf wrote:
> Date: Tue, 12 Nov 2019 16:22:24 +0100
> From: Jochen Topf 
> To: Craig Kochis via dev 
> Subject: Re: [OSM-dev] Missing records from osmium-tool tags-filter
> 
> Hi!
> 
> When I try to run https://www.openstreetmap.org/way/24801630 through the
> commands you provided, it will come out fine. In fact it comes out
> twice, once as LineString, once as MultiPolygon.
> 
> Tested with:
> 
> wget -O w24801630.osm https://www.openstreetmap.org/api/0.6/way/24801630/full
> osmium tags-filter w24801630.osm amenity building craft cuisine diet leisure 
> office public_transport railway shop sport aeroway --remove-tags -v -o 
> poi.osm.pbf
> osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs 
> --add-unique-id=type_id -v -o poi.json
> 
> This shouldn't be different if it is in the planet file. Where did your
> planet file come from? Can you provide a minimal example instead of one
> that needs an (unspecified) planet file?
> 
> Jochen
> 
> On Tue, Nov 12, 2019 at 09:45:32AM -0500, Craig Kochis via dev wrote:
> > Date: Tue, 12 Nov 2019 09:45:32 -0500
> > From: Craig Kochis via dev 
> > To: dev@openstreetmap.org
> > Subject: [OSM-dev] Missing records from osmium-tool tags-filter
> > 
> > First off, just want to say this tool has been amazing, it's blazing fast
> > for parsing through huge amounts of data.
> > 
> > There's probably something I'm not understanding about how the tags-filter
> > command is working, but there are some records missing from the output that
> > I'd expect to be there based on the commands. For instance, I'm running the
> > following command to extract some POIs I'm interested in:
> > 
> > osmium tags-filter planet.osm.pbf amenity building craft cuisine diet
> > leisure office public_transport railway shop sport aeroway --remove-tags -v
> > -o poi.osm.pbf
> > 
> > Then exporting to JSON
> > 
> > osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs
> > --add-unique-id=type_id -v -o poi.json
> > 
> > And some records are being omitted that I would expect to be in the output.
> > For instance, Yankee Stadium:
> > https://www.openstreetmap.org/way/24801630 matches
> > the "leisure" and "sport" tags, yet that way isn't in the result set. If I
> > run tags-filter with just "leisure" it will show up, so I'm wondering if
> > there's something happening by specifying multiple tags that I'm not aware
> > of.
> > 
> > Any guidance on this would be greatly appreciated. Thanks!
> 
> > ___
> > dev mailing list
> > dev@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/dev
> 
> 
> -- 
> Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688
> 
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev

-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Missing records from osmium-tool tags-filter

2019-11-12 Thread Michael Reichert
Hi Craig,

Am 12.11.19 um 15:45 schrieb Craig Kochis via dev:
> Then exporting to JSON
> 
> osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs
> --add-unique-id=type_id -v -o poi.json
> 
> And some records are being omitted that I would expect to be in the output.
> For instance, Yankee Stadium:
> https://www.openstreetmap.org/way/24801630 matches
> the "leisure" and "sport" tags, yet that way isn't in the result set. If I
> run tags-filter with just "leisure" it will show up, so I'm wondering if
> there's something happening by specifying multiple tags that I'm not aware
> of.

If the way does not appear in the final JSON, it could miss because the
OSM file misses one or multiple nodes referenced by the way. Therefore,
building the geometry is not possible. Use "osmium check-refs" to check
the referential integrity of an OSM file. Please mind that the input
file to check-refs needs to be sorted (can be done using the Osmium tool
as well; usually OSM files are sorted).

https://docs.osmcode.org/osmium/latest/osmium-check-refs.html
https://docs.osmcode.org/osmium/latest/osmium-sort.html

Best regards

Michael

-- 
Michael Reichert  www.geofabrik.de
Geofabrik GmbHHandelsregister: HRB Mannheim 703657
Amalienstr. 44Geschaeftsfuehrung: C. Karch, F. Ramm
76133 Karlsruhe   Tel: 0721-1803560-3
reich...@geofabrik.de Fax: 0721-1803560-9



signature.asc
Description: OpenPGP digital signature
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Missing records from osmium-tool tags-filter

2019-11-12 Thread Jochen Topf
Hi!

When I try to run https://www.openstreetmap.org/way/24801630 through the
commands you provided, it will come out fine. In fact it comes out
twice, once as LineString, once as MultiPolygon.

Tested with:

wget -O w24801630.osm https://www.openstreetmap.org/api/0.6/way/24801630/full
osmium tags-filter w24801630.osm amenity building craft cuisine diet leisure 
office public_transport railway shop sport aeroway --remove-tags -v -o 
poi.osm.pbf
osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs 
--add-unique-id=type_id -v -o poi.json

This shouldn't be different if it is in the planet file. Where did your
planet file come from? Can you provide a minimal example instead of one
that needs an (unspecified) planet file?

Jochen

On Tue, Nov 12, 2019 at 09:45:32AM -0500, Craig Kochis via dev wrote:
> Date: Tue, 12 Nov 2019 09:45:32 -0500
> From: Craig Kochis via dev 
> To: dev@openstreetmap.org
> Subject: [OSM-dev] Missing records from osmium-tool tags-filter
> 
> First off, just want to say this tool has been amazing, it's blazing fast
> for parsing through huge amounts of data.
> 
> There's probably something I'm not understanding about how the tags-filter
> command is working, but there are some records missing from the output that
> I'd expect to be there based on the commands. For instance, I'm running the
> following command to extract some POIs I'm interested in:
> 
> osmium tags-filter planet.osm.pbf amenity building craft cuisine diet
> leisure office public_transport railway shop sport aeroway --remove-tags -v
> -o poi.osm.pbf
> 
> Then exporting to JSON
> 
> osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs
> --add-unique-id=type_id -v -o poi.json
> 
> And some records are being omitted that I would expect to be in the output.
> For instance, Yankee Stadium:
> https://www.openstreetmap.org/way/24801630 matches
> the "leisure" and "sport" tags, yet that way isn't in the result set. If I
> run tags-filter with just "leisure" it will show up, so I'm wondering if
> there's something happening by specifying multiple tags that I'm not aware
> of.
> 
> Any guidance on this would be greatly appreciated. Thanks!

> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev


-- 
Jochen Topf  joc...@remote.org  https://www.jochentopf.com/  +49-351-31778688

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Missing records from osmium-tool tags-filter

2019-11-12 Thread Craig Kochis via dev
First off, just want to say this tool has been amazing, it's blazing fast
for parsing through huge amounts of data.

There's probably something I'm not understanding about how the tags-filter
command is working, but there are some records missing from the output that
I'd expect to be there based on the commands. For instance, I'm running the
following command to extract some POIs I'm interested in:

osmium tags-filter planet.osm.pbf amenity building craft cuisine diet
leisure office public_transport railway shop sport aeroway --remove-tags -v
-o poi.osm.pbf

Then exporting to JSON

osmium export poi.osm.pbf --output-format=geojsonseq --omit-rs
--add-unique-id=type_id -v -o poi.json

And some records are being omitted that I would expect to be in the output.
For instance, Yankee Stadium:
https://www.openstreetmap.org/way/24801630 matches
the "leisure" and "sport" tags, yet that way isn't in the result set. If I
run tags-filter with just "leisure" it will show up, so I'm wondering if
there's something happening by specifying multiple tags that I'm not aware
of.

Any guidance on this would be greatly appreciated. Thanks!
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev