Re: [OSM-dev] Restarting the EWG
Christoph, On 11/19/20 20:41, Christoph Hormann wrote: > * why i as a pure hobby OSM contributor with experience in the field of > development should volunteer my time to manage the paid development work of > others on my own unpaid time. You make it sound like this was something new, but the OSMF which largely consists of unpaid hobbyists is already managing paid contributions in various forms. That doesn't make your point an invalid one - from the very first time the OSMF was paying for development this question was on everyone's mind, and many other Open Source projects who use bounties or Summer of Code or other means of compensating developers in addition to attracting volunteer contributions are faced with the same problem. Speaking as a member of an existing OSM working group, the DWG, I can say that I could imagine a couple of projects where I would like to put some of my DWG volunteer time into managing paid development work that would in the end make my life in the DWG easier, and it would not diminish my DWG engagement at all - on the contrary, probably. So yes, if not handled well then using money to pay for stuff can be a turn-off, but it certainly doesn't have to be! > * how i as someone with a business or professional career interest in the OSM > context would be able to contribute to this work without universally having a > massive conflict of interest with every decision of substance that is being > made. That remains to be seen. Obviously we wouldn't want someone to bring in their spouse and kids, or even their friends, as contractors. Then again, "I have worked with this guy in the past and I am confident he can do what we need here" could be a very valuable piece of information. As always, the thing about conflicts of interest is that they need to be properly declared and managed (instead of covering everything in sticky "we all want the best for OSM so where's the conflict" sauce), but if that is done well then they can be handled. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Standardized feature/tag database idea & proposal
Seth, On 11/15/20 19:14, Seth Deegan wrote: > Has anyone ever thought of creating an official database that stores all > of the approved and in-use tags/features in OSM? Yes, of course, this idea has been around in a variety of flavours. I remember a conference talk by David Earl in 2010 (https://wiki.openstreetmap.org/wiki/SotM_2010_session:_Tag_Central:_a_Schema_for_OSM) though even then it was not revolutionary. I think that going from a wiki to a plain database doesn't actually solve many problems. Many things that are difficult for people to deal with - like tag definitions leaving room for interpretation, different editors having different presets, different language versions describing tags differently, a lot of vagueness about the proposal process and its importance, etc.etc. - would not be solved by your idea. You'd only replace the backend but that would not automatically fix the messy bits (and for some of the messy bits, "fixing" them might also mean breaking a part of OSM that many cherish). For example, one point you mention is that with a database, things could be somehow "locked", but you're completely ignoring the question of who would get to lock stuff, who would decide which changes are allowed, and how this relates to translations, what the appeals process would be etc.etc. - and *those* are the difficult questions, not the question of whether I could technically have a mechanism that would restrict editing to certain people. And of course, as has been pointed out, the wiki stuff can already be accessed in a database-like fashion, not only through the data endpoint but also through the taginfo-wiki SQLite database from https://taginfo.openstreetmap.org/download. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Latest osc change file returns empty xml.
Hi, On 11/14/20 21:19, Dave F via osmosis-dev wrote: > I'm using osmosis --rri workingDirectory=. --wxc GB.osc.gz > I amended configuration.txt to point to the GB folder: > baseUrl=http://download.geofabrik.de/europe/great-britain-updates > > Running the command it completed it's task, but the osc file contained > an empty xml file: Is it possible that you have accidentally set your replication state to the current state? If you download the Great Britain file now, and set the appropriate state.txt value, then the expected result of running osmosis is an empty XML file because there are no changes available. As of this writing, the /europe/great-britain-updates/state.txt says sequenceNumber=2791 and running osmosis with that will yield an empty file, until such time as the state.txt on the server points to a sequenceNumber of 2792. > On other Q: > As the update interval for Geofabrik is 24 hours, should I amend this > value in the configuration.txt file to suit: > maxInterval = 3600 It does not make a difference; any maxInterval between 1 and 86400 should behave the same, as each would pick exactly one update per run. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
[OSM-dev] Nominatim updates stuck with "tuple already modified"
Hi,
a couple of our (Geofabrik) Nominatim servers stopped updating properly
about 4 days ago. Closer inspection has shown that this was due to a
self-referencing place=locality relation (Chernobyl-2). I have fixed
that in the OSM data just now.
Affected systems would log something like
ERROR: tuple to be updated was already modified by an operation
triggered by the current command
HINT: Consider using an AFTER trigger instead of a BEFORE trigger to
propagate changes to other rows.
when running update.php. This would also be in your postgres log file. A
tell-tale sign would also be if your import_osmosis_log table ends with
something like
2020-09-07 17:23:01 | 4185363 |670328 | 2020-09-07 17:23:16 |
2020-09-07 17:23:20 | import
without a matching "index" line.
Fixing the problem seems to require this SQL command:
update planet_osm_rels set
parts='{437725919}',members='{w437725919,outer}' where id=8184246;
After this, if you run the update.php script again, it will complete
indexing and resume downloading new data from OSM. It is possible that -
depending how your updating is configured - you run into the same issue
a second time, because there are two broken versions of the relation on
OSM and if the 2nd hasn't been downloaded yet, then it will overwrite
your fix. You will then have to enter the same "update" command again.
But after that, it should work.
I've filed an issue with Nominatim about this
https://github.com/osm-search/Nominatim/issues/1941 - just writing here
in case someone encounters the same issue.
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Accessing third-party repositories
Hi, upon starting JOSM I was greeted by, among other things, messages that loaded content from "wikidata.org" and "sophox.org". I have not actively enabled something that would make these queries, nor have I been asked for my consent to transmit the fact that someone is using JOSM at this IP number to wikidata.org or sophox.org. I can understand that if I load Ilya's geochat plugin it will phone home to Ilya's server, or if I enable certain imagery layers they will load data from the imagery server. Also it is clear that JOSM will access the OSM and JOSM servers. But I think that we should not add random third party web sites that are under control of neither OSMF nor the JOSM team to the mix without explaining this to the user and asking for their consent. Would it perhaps make sense to build a generic "consent to access server X" feature into the JOSM core, and anyone - whether core or plugin - would then have to acquire user consent once before accessing a remote resource? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
Re: [OSM-dev] OSM Database schema
Hi, On 10.01.20 15:08, Lorenzo Stucchi wrote: > What does it mean the “timestamp” present in the table (“node", “way" and > “relation")? When the object was last changed. > What does it mean the “tile” element present in the “node" table? An integer derived from the lat/lon of the node using a mathematical formula. > What does it mean the “sequence_id” in the “way_nodes” table and > “relation_member” table? An integer used to store the ordering of the nodes/members (which node is the first, the second, ... in the way). Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Experimental regional taginfo sites
Martijn, On 05.01.20 21:17, Martijn van Exel wrote: > Am I misinterpreting your instructions or did you not add these regions? The US has always been a special case on the download server, where because of its size there never was an extract for the country, just for the continent (North America) and then the census regions and individual US states. This leads to endless special-casing server-side, and I forgot that here, hence the bug. Fixed now: https://taginfo.geofabrik.de/north-america/us/utah Bye Frederik PS: Maybe some day I'll do a proper US extract and then the US will be a normal country like all the others ;) -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Experimental regional taginfo sites
Happy new year! Following in Imre Samu's (https://github.com/ImreSamu/dockerized-taginfo) footsteps, I have used the past holiday season to set up a taginfo server that is supposed to serve daily updated taginfo data for all regional extracts routinely offered on the Geofabrik download server. It's still being tinkered with hence I'm not announcing it widely - I'd hope that a few of you here might want to give it a spin and tell me how it is working for them before it is properly "launched". The site is https://taginfo.geofabrik.de/ and you have to append the path of the region you're interested in as known from the download server, e.g. https://taginfo.geofabrik.de/europe/germany/berlin/ or something. If you do anything fancy with the URL e.g. leave off the trailing slash or add one where it doesn't belong, you'll get an internal server error ;) The same is true for situations in which I should accidentally have neglected to fix a hyperlink and it still points to /something instead of /continent/country/something. The map images are auto-generated from the data extent and will probably require some tweaking in some cases, e.g. the Australia-Oceania image essentially spans the globe. The way this works internally is that it simply runs Jochen's taginfo data analysis on every extract, separately, and then uses a slightly modified web application that is capable of handling multiple databases at the same time. For the non-extract-specific sources like the wiki extract, a shared copy is used by all regions. The regional databases are downloadable (e.g. http://taginfo.geofabrik.de/europe/germany/berlin/download/taginfo-db.db.gz), but in contrast to Jochen's global taginfo site, these downloads are compressed on demand, and you can easily overload the server by trying to download all databases. If you want all regional databases, talk to me and we'll set something up. This is still missing a couple features, most of all some form of navigation between regions (currently only by manual URL manipulation). It also has a few issues that Imre has already encountered and fixed in his approach, most notably the fact that the Geofabrik extracts are not very precise, leading to strange artifacts like a "source=cadastre-dgi-fr" being prominent in Luxembourg and so on. Most of the changes I have made to the taginfo web site are on https://github.com/geofabrik/taginfo/tree/multi-config, some bits and pieces are still missing but will ultimately all end up there. Let me hear of the problems you encounter so I can fix them before announcing this further! Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Google-free phones
Hi, On 26.11.19 08:57, Andrew Hain wrote: > How well do OSM apps work on Google-free Android phones? Don't see what the issue should be... I've personally run OsmAnd, Vespucci, and OsmTracker on a non-Gapps Lineage phone and they all work without a glitch. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Osmosis java.lang.IndexOutOfBoundsException
Hi, I think this was a problem of the Geofabrik download server. We accidentally generated our "cleaned" files (the GDPR conformant file without any user data) in a different way last night, completely dropping the UID/Username fields, instead of replacing them with 0 and empty strings as we did before. This tripped up osmosis and also older versions of osm2pgsql (<= 0.94). The pbf files are replaced with old-style, compatible files now. Nightly updates and bz2 files were not affected since they don't use the PBF format. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Slow osmosis import
Hi, the osm2city software should be changed to use an osm2pgsql database instead of an osmosis database. Not only can a planet be imported in less than a day with osm2pgsql (if you have SSDs), but also the osm2pgsql database already has correctly built geometries for all objects, whereas osm2city has to make an effort to build these geometries from raw OSM data,thereby re-inventing the wheel when it comes to the interpretation of multipolygon relations, the treatment of way-based vs. relation-based polygons, etc. osm2city does not seem to use anything that could *not* be found in an osm2pgsql import. If you insist on continuing down your current path then you must either equip your computer with fast SSDs, or temporarily rent a large-SSD Amazon instance on which you can do your import and then copy over the resulting database (if you choose a setup where the importing instance has the same CPU architecture, as well as exactly the same OS and PostgreSQL/PostGIS versions, then you can copy over the raw database directory). But even this is likely to take at least a week if not several for the import - osmosis imports are just not something people do normally on a planet scale. I have only cursorily looked at the osm2city source code and it seems that it uses most of OSM's data (buildings, roads, landuse). If you should be in a situation where you only need some of OSM's data then a speedup could be gained by first running "osmium tags-filter" to extract the data you really need from the planet file. But if the list of "data you need" contains roads and buildings and landuse then you might as well not filter, since those categories make up the bulk of OSM data. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Slow osmosis import
Hi, first question: are you absolutely sure you need an Osmosis import - does your use case not work with an osm2pgsql import? Best Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] QA, check for tag change
Hi,
On 10/2/19 20:36, yves wrote:
> What would be a cheap way to find ways that once were highway=* and are
> now a piste:type=nordic but no more a highway=* ?
Define "cheap" ;)
I usually tackle issue like this with a contraption of history file,
osmium, and a script in a programming language of choice (for easy cases
even shell script works).
The first step is to have osmium convert the history file into "OPL"
format which gives you one ascii text line for every version of an
object, neatly sorted by type, ID, and version:
osmium cat somefile.osh.pbf -tway -fopl
From there, the following Perl script would for example solve your problem:
#!/usr/bin/perl
use strict;
my $lastid;
my $was_highway;
my $lost;
while(<>)
{
/^w(\d+) .*T(\S+)/ or next;
my ($id, $tags) = ($1, $2);
my $piste = ($tags =~ /piste:type=nordic/);
if ($id eq $lastid)
{
$lost = ($piste && $was_highway && $tags !~ /highway=/);
}
else
{
print "way $lastid lost its highway tag\n" if ($lost);
$lastid = $id;
undef $was_highway;
undef $lost;
}
$was_highway = $was_highway || ($tags =~ /highway=/);
}
save that in foo.pl and simply do a
osmium cat somefile.osh.pbf -tway -fopl | perl foo.pl
You will find that it is a bit rough around the edges, e.g. it doesn't
correctly handle deleted objects and disregards the very last way in the
file, but it works *in general*.
Bye
Frederik
PS: Slightly more sophisticated Perl snippets from some of my scripts
lying around include
while(<>)
{
my $part;
my @parts = split(/ /, $_);
my $obj = shift(@parts);
foreach (@parts)
{
$part->{substr($_,0,1)}=substr($_,1);
}
(this splits the OPL line into a hash that you can then access with
$part->{'v'} for the version, $part->{'T'} for the tags etc.etc.) and
my @tags = split(/,/, $part->{'T'});
my $tag;
foreach (@tags)
{
/(.*)=(.*)/;
$tag->{$1}=$2;
}
which further splits the "tags" part into a hash with tags so that you
can then write code like "if defined $tag->{'highway'}" etc.
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] PostgreSQL 11 - osm2pgsql performance problems during --append?
Hi, if you are using the flatnodes option (which you should for a world-wide import) then the node import step will mainly hit the flatnodes file and only have relatively limited PostgreSQL interaction. It therefore sounds unlikely that the PostgreSQL upgrade could be at fault. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Rendering OSM carto from epsg:4326 database
Hi,
On 11.09.19 17:39, Sven Geggus wrote:
> This does not seem to work.
One thing that is likely to break is any queries that are based on the
content of way_area ("WHERE way_area > somevalue"). You'll either have
to make all these limits much smaller with a script, or use the
--reproject-area osm2pgsql flag on import.
(Or potentially run an update on your existing database the
re-calculates all the way_area values based on a reprojected polygon.)
There might be other issues - how exactly does it not work?
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Feedback on project idea
Michael, On 01.05.19 22:22, HWANG, MICHAEL (MICHAEL) wrote: > I’m looking for feedback/interest in a new open source software project > idea. This project is intended to address the problem of systematically > combining private geo-spatial datasets with OSM data. There can be > overlap (in terms of objects) between the private geo-spatial datasets > and with OSM and the project’s goal is to de-duplicate and to merge all > objects together to produce a new single, consistent, more complete > dataset. The reason why this is needed is that organizations have > private datasets that will take time/never be pushed into OSM. It is important to take a moment to look at the reasons here: * Some datasets might be theoretically suitable for inclusion in OSM, but the person or organisation using them doesn't have the patience or resources to commit to a proper import process. * Some datasets might be of insufficient quality for inclusion in OSM, but the quality might be sufficient for a particular use case. * Some datasets might be outright wrong - e.g. a politically defined "official" set of boundaries that needs to be used to publish maps in a country but is useless for all other purposes. * Some datasets might be confidential or copyrighted and therefore not suitable for contributing to OSM. If there was a platform that allowed people to mix such data with OSM at on the data user side, that would be a huge relief for OSM because it would stop people from pushing low-quality data into OSM "just to make nice maps" or "just to have the hospitals on their Garmin maps" or so - they could mix-in questionable data with your toolchain. Even better if the platform were public in a way that would allow people to mix-in data provided by others, e.g. someone could at the push of a button choose to have built-up areas from naturalearthdata.com in their maps or so. Lots of technical challenges, of course. In some of the cases above there might also be license challenges; mixing your own restaurant data with OSM's and de-duplicating and generating a "new and better" data set would likely, if you publicly use that, lead to it having to be published under ODbL so the "confidential/copyrighted" use cases would have to be carefully checked. > Would the OSM community be receptive to this sort of project and be open > for collaboration? If this is not the proper forum to ask, please let > me know where else I can go to ask. It is hard to ask "the OSM community" for anything but I guess here's as good as anywhere. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Fwd: Request for a Map as JPEG export
Hi, "bigmap" essentially downloads lots of tiles from OSM and stitches them together. This can give you a map with lots of pixels but not really a high resolution map, it will always have "screenshot quality". Plus, the user interface of my bigmap (at openstreetmap.gryph.de) is a bit sucky. Zverik has made an improved version here http://bigmap.osmz.ru/ Bye Frederik On 16.04.19 08:08, PanierAvide wrote: > Hello, > > I'm not sure this covers all the functionalities you're looking for, but > are you aware of "BigMap" ? This website allows to export an area with > more or less high resolution. This avoid to stitch manually the tiles, > and maybe the tool can be customised to add functionalities you're > looking for. > > http://openstreetmap.gryph.de/bigmap.html > > Regards, > > Adrien P. > > Le 16/04/2019 à 00:23, Flaviu S a écrit : >> Dear Dev Community, >> >> I searched for a way or a tool which can do following: >> >> I need to create a JPEG with the Size of A0 (841 x 1189 mm) at least >> or of 2A0 (1189 x 1682 mm) if possible. >> >> The corners of the map have following coordinates: >> 48.03731, 15.85933 >> 47.88788, 15.85933 >> 47.88742, 16.33449 >> 48.03601, 16.33476 >> image.png >> >> The Zoom Level should be as high as possible, so that the names of the >> streets are still readable. >> For example like this >> Screenshot 2019-04-15 20.20.49.png >> >> And then I need also this polygons (a KMZ File) to be on the map >> image.png >> >> Do you have any possibility to make a JPEG like this in that high Zoom >> Level? >> Because without programming skills, the only way I found is to make >> more screenshots and then stitch it together to one big picture and >> then resize it to A0, but this would mean, I can't change anything >> afterwards. >> >> If you can help me, I would be very thankful ☺ >> >> P.s.: >> If something is unclear or you need more information, please feel free >> to contact me. >> >> Best regards >> Flavius Sarca >> >> >> ___ >> dev mailing list >> dev@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/dev > > ___ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev > -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Fwd: Request for a Map as JPEG export
Hi, On 16.04.19 00:23, Flaviu S wrote: > I searched for a way or a tool which can do following: Here's a couple ways/methods you could try. Method 1: The following method does not need *programming* skills but you'll have to know your way around a Linux machine: * install osm2pgsql, Mapnik, and PostGIS * clone openstreetmap-carto from github and follow instructions for downloading shape files there * install nik4.py * get an .osm.pbf data file that covers the region you are interested in, perhaps from extract.bbbike.org * import into database with osm2pgsql * run nik4 to generate large output image To add custom KML data, EITHER (variant 1a) * convert KMZ to shape file, add a new layer to your openstreetmap-carto style to draw this shape file with the appropriate styling OR (variant 1b) * instruct nik4 to output a "world file" in addition to output image, then open output image in QGIS, load your KMZ in QGIS, style as desired. Note you may have to tell QGIS that your image is in EPSG:3857 and possibly enable "on the fly transformation" to load the KMZ which likely is in EPSG:4326 Method 2: If all this sounds too complicated, you might be able to use https://maposmatic.osm-baustelle.de/ to generate a couple of large images that you can then stitch together (better than screenshots). But adding your KMZ would then be difficult - perhaps it can be done by uploading your KMZ tu UMAP and then using the maposmatic UMAP functionality. Method 3: Another option might be to use only QGIS, and start by adding an OSM layer from a free WMS server https://wiki.openstreetmap.org/wiki/WMS and then add your KMZ on top. The "print composer" in QGIS should then allow you to create a large image. The output quality of this approach depends on whether or not the WMS supports high(er) resolution output; worst case, it will be only screenshot quality. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend 23/24 Feb
Hi, the next Karslruhe Hack Weekend is on the 23/24 February. Details: https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_February_2019 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] iD news - 2.12.0 released
Hi, On 10.12.2018 03:14, Bryan Housel wrote: > iD now displays linked data if a feature has a wikidata tag, and will > protect fields like name and brand from direct editing. Can you elaborate on the logic behind this a bit more? On first reading it sounds like if I set a wikidata tag on something, iD will load name and brand from wikidata and not allow editing of those fields but that certainly can't be right. Which fields exactly are protected, what does protection mean, and by what is this protection triggered? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Indexing of PBF files
Hi, On 10/15/2018 11:32 PM, Nick Stallman wrote: > In doing this I noticed that all the tools handling PBF files are > horribly inefficient. With PBF files being in essentially raw > chronological order it means extracting any kind of meaningful data out > of them requires scanning the entire file (even if you are only > interested in a small region) and vast quantities of random reads. I don't think your analysis is correct. I am not aware of any file that processes PBFs and does random reads - they're all streaming, and worst case they're reading the file in full three times. But no seeking. And reading "only a region" from a PBF file is kind of a niche use case; most people get the file that covers the area they need, and load it into a database, where derived data structures will be built for whatever the use case is. The osmium command line tool is relatively good and efficient at cutting out regions from a planet file if needed. Indexing a planet file would only make sense if your use case involves repeatedly cutting out small areas from a planet file. > Judging from the PBF wiki page, all the work was done ~8 years ago and > included the foresight to have fields for indexing but from what I can > find nothing has been done about that since. Adding an index seems like > a logical step which would reduce processing times for many common > operations drastically. As I said, most people take a PBF and load it into a database, and I don't see how that processing would benefit from an index. What are the "many common operations" you are thinking of? > Some tools do make their own index or cache but > it needs to be done for each tool and is sub optimal. I'm only aware of Overpass which is essentially a database implementation of its own, which not only does regional cuts but also filtering by tags, and would certainly not be able to simply replace its own database with an indexed PBF. > I'm a little tempted to find the time to create an indexed format myself > if needed and submit patches to the relevant tools so they can benefit > from it. Again, I struggle to understand which operations and tools would benefit; I don't think the general OSM data user struggles with the issues an index would solve. I could imagine if you ran a custom extract server like extract.bbbike.org then having random, regionally indexed access to a raw data file could be beneficial but that's about the only case I can think of. > With this scheme, if you needed to make a country extract it would be > too easy, Blobs could simply be copied as-is selected by their geohash. > A later step could then filter out by polygon or bounding box if > required over the subsequent significantly smaller file. If the entire > planet file was being imported in to PostGIS then it could be done in a > single pass since everything would be easily locatable. The planet is imported into PostGIS in a single pass even now, at least if you use osm2pgsql. I am running a nightly job that splits the planet into tons of country and smaller extracts on download.geofabrik.de. It takes a couple of hours every night. Having an indexed planet file could save a little time in the process but I'm not sure if it would be worth it. The reason many people download country extracts from download.geofabrik.de is probably not that the planet file isn't indexed and therefor extracting a region is hard - it's that the planet file is huge and they don't want to download that much. An indexed planet file would not help these users. Not saying you shouldn't try it but I haven't yet understood the benefits. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [OSM-talk] Tool update from HOT: MapCampaigner
Hi, On 01.10.2018 03:28, Nate Smith wrote: > Last week we > released a new version of a data quality monitoring tool I would like to recommend that you don't use the term "quality monitoring tool" for this since you're measuring quantity not quality. At best, I'd call it a tool that monitors "richness" or "completeness". Simply counting how many features there are and how many of a pre-defined list of tags each one has shouldn't be called "quality monitoring", because there will be situations where the OpenStreetMap community requests of project managers (who your web site claims to be targeted at) that they implement some form of quality assurance; calling your statistics tool a "quality monitoring" tool runs the risk of making these people believe that quality requirements can be fulfilled by ensuring that enough tags are set, which is definitely not what the wider community would regard as a suitable quality assurance for a humanitarian data entry project. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Various types and means of account blocks
Hi, On 26.09.2018 10:43, Frederik Ramm wrote: > https://www.openstreetmap.org/changeset/62773816 Uh, the "discussion" there is currently taking a xenophobic turn that will lead to more removals maybe it wasn't a good example. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Various types and means of account blocks
Hi, On 24.09.2018 21:05, tigerfell-...@tuta.io wrote: > Well, as outlined in GitHub, these concepts seem to exist already. > Number 1 would be called "block" (you could still use the forum, I > guess) and (2) "suspension" (everything hidden). Yes. The "everything hidden" bit is very confusing though. One example from the recent past: https://www.openstreetmap.org/changeset/62773816 There used to be a changeset comment there. The user, and his comment, were then kicked out. The response to his comment remains, though, without any apparent motivation. It would be better to have "(a comment by user X has been removed)" or so. I guess if someone sets up an account just for adding a spam profile, it would be ok to remove them without a trace, but once they've participated in any way (and potentially provoked a response, like here), you can't really act as if they never existed. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Various types and means of account blocks
Hi, I would like to start a discussion/brainstorming about the technical aspects and means of blocking OSM user accounts. First of all, the wider OSM community uses a wealth of communications channels, most of which are not even controlled by us; here I just want to discuss the actual OSM account and the means of communication associated with that. Currently an account can be blocked (by the DWG for a limited time, or by admins for arbitrary timespans). There's a UI for that (even though I think the long-term blocks need manual database fiddling). A blocked user cannot edit OSM data. They can, however, still use the various communication functions: write personal messages, write or comment on diary entries, comment on changesets, and open, close, and comment on notes. And they can modify their user page, change their account name, and "befriend" other users. Currently, if we wanted to keep someone from using these functions, we'd have to "suspend" the account altogether, which is almost the same as deleting it: The account will not be visible any more, at all, and nobody can log in to it (cf. discussion in https://github.com/openstreetmap/openstreetmap-website/issues/1946). OSM has largely been spared from obnoxious nutcases that you find online elsewhere, but our increasing popularity will certainly send a couple of them our way in the future. Some examples of borderline behaviour that we have seen in the past: * user creating tons of playful/funny notes, and modifying his user name several times a day * user closing 100s of notes without actually doing something about them * user "stalking" another user in changeset comments, writing rant-y comments in response to everything the other user writes * user writing longish, rant-y, unwanted, and off-topic diary comments to third party's diary entries * user sending legal threats to other users in personal messages * user adding a "shit list" to his profile page listing the account names of other mappers they don't like I wonder what the best way would be to deal with issues like that. The ticket I quoted above is from a DWG member suggesting that normal user blocks should simply be extended to block all the "communication" functions as well. In the discussion it was suggested that someone blocked for, say, participating in an edit war, should not necessarily be prevented from writing and receiving messages. Is the opposite true as well - would/should someone given a cool-off period for being a dick in a discussion still be allowed to do mapping? Should a normal user block perhaps simply come in two flavours, "block mapping only" and "block all"? It has been suggested that blocking *all* communication functions might be problematic because one thing you might expect from someone you have blocked is that they apologise, or set something right, which they might not be able to do without the ability to write messages. Do we need a full array of permissions - "can change user name", "can edit own user page", "can write personal messages", etc. - and the ability to short-time suspend any and all of them? Thoughts are welcome. This also ties in somewhat with a separate discussion, on how a prerequisite for allowing children on the platform might be that we can disable the "social" functions of an account. In that case it would not be a short-term block, but a whole class of accounts that can edit, but not participate in discussions (for their own protection). I'm not sure that can work at all (given that the ability to contact a mapper is very important to us). Maybe such accounts would have to be linked to a "responsible" parent account who then gets the messages... Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to find the way with the most relation memberships
Hi, On 27.08.2018 13:23, Maarten Deen wrote: > I have seen duplicates of (bus) relations also in Germany, No way, the Germans would never allow that kind of relation breakage... https://www.openstreetmap.org/way/368506221 ... err, ehm ;) Bye Frederik PS: top 100 planet-wide are in http://www.remote.org/frederik/tmp/relationmembers.txt, done with exactly the script from the beginning of this thread. -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to find the way with the most relation memberships
Hi, On 27.08.2018 11:41, Simon Poole wrote: > Come on, you're leaving us out on a limb. Now we want to know the world > leader in relation membership :-). https://www.openstreetmap.org/way/496296681 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] How to find the way with the most relation memberships
Hi,
in the course of a discussion over on talk-gb, I wanted to find out
which ways have the highest number of relation memberships. In case
someone is interested, here's how to do it with Osmium and Perl.
1. create this Perl script which reads "opl" ascii format
#!/usr/bin/perl
while(<>)
{
next unless /boundary/;
s/.* M//g;
foreach (split(/,/))
{
my ($a,$b)=split(/@/);
$mem{$a}++;
}
}
$i=0;
foreach (sort { $mem{$b}<=>$mem{$a} } keys(%mem))
{
printf "%d %s\n", $mem{$_}, $_;
last if ($i++>100);
}
2. feed an .osm.pbf file into it:
osmium cat some-file.osm.pbf -trelation -fopl | perl myscript.pl
I ran this for England and found a small number of ways that were
actually in over 100 different (bus route) relations ;)
If you like Python, you could of course do the whole thing in one go
using the PyOsmium library
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
Roland, the changes that I proposed mean that your Overpass API will, if it wants to continue downloading user data from OSM, at some point in the future have to identify itself to OSM with an OSM account as proof of your acceptance of the Terms of use. This is the *technical* requirement for having access to OSM user data in the future, and it is easy to do. I'm happy to provide the necessary script for that when the time comes. Overpass already differentiates between output with and output without meta data. The output without meta data, which IMHO is totally sufficient for the overwhelming amount of Overpass use cases, would continue unchanged. So these use cases are all covered without any of us investing any work, without a "development backlog of more than a year" or killing the project entirely. Let's look at those use cases where Overpass users would like to download user data. You seem to assume that this not only requires the overpass user to have an OSM account but also that the overpass user somehow goes through an OAuth process with OSM every time they want to access Overpass. This is *not* intended to be a requirement. The ToU will require - in wording that is yet to be defined - that you take care to only distribute OSM user data for purposes that the OSMF considers legitimate. Now it is clear that you cannot actually *control* what users do with data - but you will be expected to inform them that they have to conform to the OSMF's rules when they process this data. One *possible* way of doing that would be to simply have them prove that they have an OSM account, because if they have an account, then they also have accepted the ToU, and then you don't have to explain anything to them. This *could* be done with OAuth, either with every request they send, or you could have your own database of Overpass API keys where people have to prove they have an OSM account when they register. But you could also run a scheme completely independent of OSM, where anyone can register for an "Overpass account" and you show them some text that says "By signing up for an Overpass account you promise to always stick to OSM's terms of use" or so. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
Hi, On 06/20/18 11:38, Jochen Topf wrote: > And if you actually want to make sure that redacted data (because the > user wanted it to be deleted) is deleted downstream also, We will not try to "make sure" that this happens, but we plan to offer help for downstream data processors, likely by publishing some sort of feed or list of user deletions and redactions. This hasn't been specced out yet, and doesn't have to be at this point in time. We sure as hell don't intend to track and record who accesses OSM user data. > It might be "the least disruptive", but if it doesn't make any sense, > that doesn't make it better. Any judge will laugh at you if you tell > them: Well, we trust the million users we already have and the other 6 > billion who can sign on to OSM anonymously more than we trust the > general public. I think that setting out clear terms for the users we already have and those who might sign up in the future *is* a step in the right direction. It conveys the message that personal data isn't handed out willy-nilly, and that you have a certain responsibility when dealing with it. > It is a step towards making the project more closed and burying it in > burocracy. I don't see that. > You are ceding ground This isn't a war between us and the EU in which we "cede ground". I don't even think that being able to bandy around personal information about OSM users is a goal worth fighting for. If I had to decide whether the fact that everyone can stalk our mappers using OSM data is a necessary side effect of our work, or counter to our interests, I would probably lean towards the latter. > instead of arguing that this data needs to be public for everyone. Any judge will laugh at you if you say that the information that user John Smith has mapped something at 4:23 on the 3rd of January needs to be public for everyone. Why would it, outside of a very narrow number of QA related use cases? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
Hi, On 20.06.2018 08:32, Christoph Hormann wrote: > Such agreement would not be an agreement to process your own data given > by individuals to the OSMF (which is the kind of agreement you would > normally expect in the GDPR context). You probably mean some kind of > contractual agreement about what can be done with the data. Yes. This also requires the delicate distinction that not everything in a .osm file is necessarily under ODbL. > But to be > honest i don't really see the point in that. People who download the > data can easily create an ad hoc account every time they download data. Yes. There would still be a natural person in front of the monitor who clicks "I agree to be bound by these rules" though. > The OSMF does not verify the identity of who is behind a user account > created. And doesn't intend to. > So what do you expect to gain from such an agreement? Is > there any reason to assume that in a case of such data being released > in a way that is not according to the legal requirements by a third > party the agreement can be used to avoid legal responsibility for the > OSMF it would otherwise need to face? I think the idea is more: If someone releases, or abuses, personal OSM data, it is clear that * this violates OSMF policy and * someone somewhere in the transport chain from OSM server to rule-violating use has agreed to rules that they then broke. In my view, this is not "cargo cult". If someone comes to us, today, and complains that their OSM contributions are being used to stalk them, then we cannot even point to a rule that says you cannot do this. The stalker is, as far as OSMF is concerned, 100% within their rightful use of the data. This is something that needs to stop - even if, in the future, it only becomes marginally more difficult for the stalker to use OSM data, at least we clearly say that (a) this use is not allowed, and (b) the stalker knows it. > What i can understand is giving people a simple selection option between > > [ ] i want to be safe w.r.t. personal data and not being provided > potentially sensitive information when logged in. > [ ] i want to have the possibility to access potentially sensitive data > when logged in. > > which would mainly be a service to the user - kind of like the sensitive > content switch on youtube. This is essentially the login. If you are not logged in to OSM then you will not have access to personal data. If you are logged in, then you will. We are not currently planning to offer a third way (logged in with the capability to edit but unable to see personal data). Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] GDPR implementation on planet.osm.org
Hi, On 20.06.2018 07:58, Jochen Topf wrote: >> 3a. issue guidelines about what you are allowed to do with the user data >> files, >> 3b. ensure that everyone who has an OSM account agrees to these >> guidelines one way or the other, > This is the part that's not easy and where there is a lot of important > detail missing. You have to get everybody to agree, which is not going > to happen. I was thinking of simply blocking accounts from logging in until they have agreed. Or more precisely, they would be able to log in, but only to see the message telling them they need to "click here". > You probably have to > change rules in the future so you have to make this generic, keeping > information about who clicked through which version of the rules. Unsure how useful that would be; would I not want to have everyone "on the same page" at all times, i.e. having agreed to the same rules? > So you > are generating more information you are tracking with each user, more > personal information for which you need consent from the user. As I said, I would simply block all accounts until they have agreed to the rule. This is not just about being allowed to download data; someone who edits OSM will also have access to the full user data through the API and hence agreeing to the rules is a prerequisite for editing too. > All of > this needs to be tied in the OAuth stuff and it has to be done in a way > that 3rd party services using OSM data can ask *their* downstream users > to identify in the same way which allows OSM to track everybody who uses > the full OSM data everywhere adding more personal data to keep and to > explain to users and get permissions from users for. No, there's a mistake in your reasoning here. It is true that downstream data distributors like Overpass or the Geofabrik downloads need to be able to verify whether someone has an OSM account or not. Pascal has been doing that for ages on his HDYC site, for example. But downstream data distibutors do not need to know or store anything more than that; the Geofabrik download server for example will not even store the user name of the person who has logged in, just that "whoever is here has just proven they have an OSM account". So the downstream distributor can deal with this without processing any personal data. (It would be possible to extend our OAuth system by a call that would not even return the user's identity to the caller - currently the identity is returned to the caller and the caller must then decide whether to process it or not.) On the OSM server side it is true that the server can know that "user X has just gone through the OAuth process at ". But there's no reason why we would have to keep, store, or process this information in any way. If we don't process the data then we don't have to explain, and we don't have to get permission either. (I don't see why it would be useful to store who has downloaded a full planet file when.) > Please stop this nonsense now! The only alternatives I can see would be: 1. Stop distributing who-did-what-when information to everyone, period. This is possible but it would create a privileged class inside OSM that has access to this information, and it would harm the ability of the community to do QA. 2. Take the view that distributing the data is what we do and tough luck, you've signed up to it. The LWG has advised against that course of action. Even if we were to get away with it, we would still have to stop distributing someone's data once they protest (or at least restrict the distribution of data), at which point you either have to implement the whole stuff I outlined in my original post and ensure that some user data is not publicly available, or (point 1 above) stop distributing that one person's data altogether. Given these alternatives, I think the course currently followed by the OSMF is the least disruptive. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] GDPR implementation on planet.osm.org
Hi, as you probably know, the EU data protection rules compel us to be a bit less open in handing out personal data to everyone. Following LWG's analyses and recommendations, the OSMF has decided to implement restrictions on publishing user names and changeset IDs. The general plan is to allow everyone "in OSM" (i.e. with an OSM account) to fully access all data as before (and have a policy that says you must only use the personal data for OSM purposes), while removing user names, user IDs, and changeset IDs from the publicly availalbe data (i.e. what you can get without an OSM account). This requires changes to the API which I've started to sketch here: https://wiki.openstreetmap.org/wiki/GDPR/Affected_Services but this message is about changes to the downloads on planet.openstreetmap.org. Here's a three phase plan for changing the way we run planet.openstreetmap.org, and I would like to hear feedback about the feasibility from users and those familiar with running the site alike. I haven't run this by the sysadmins so if there are any bloopers I hope they will be pointed out. (I will put this up on https://wiki.openstreetmap.org/wiki/GDPR/Planet.osm_Migration and try to work in any results from discussion here but if you're more comfortable to edit directly on the Wiki that's fine too.) Cheers Frederik Phase 1 - Introduction of no-userdata files --- This does not require software development and could start immediately, but some scripting is required. 1a. set up a new domain for OSM internal data downloads, e.g. "osm-internal.planet.openstreetmap.org", initially duplicating all data. Issue: name of domain? Issue: ironbelly disk usage is at 70%, possible to add space? 1b. modify the planetdump.erb in the planet chef cookbook to generate versions without user information of all the weekly dumps, in addition to the versions with user information; have the versions without user information stored in the old "planet.openstreetmap.org" tree, and the versions with user information in the new "osm-internal" tree. Issue: should files have the same names on internal and public site, or should they be called "planet-with-userdata" and "planet" or something? 1c. modify the replication.cron.erb as follows: * have osmosis write minutely replication files to the new "internal" tree * run a shell script after generating the replication files that will find the newly generated file, pipe it through osmium stripping user information, and write the result to the old "planet" tree, copying the state.txt files as needed * run the osmosis "merge-diff" tasks separately on both trees OR run on internal tree only and pipe result through osmium as above * write changeset replication XMLs to the new "internal" tree only For step 1c, it might make sense to announce a maintenance window beforehand during which the changes will be made, so that consumers who rely on user data can stop their replication for a few hours and then make the switch. 1d. modify planet.openstreetmap.org index pages to point to the internal page in case people wish to download stuff with user data; place marker on internal page that these files are with user data. At the end of phase 1, we will have this situation: * new changeset diffs only on the "internal" tree * regular diffs come in two flavours, with and without user data * planet dumps etc. also come in two flavours * old files are unchanged * consumers will automatically get the stuff without user data * consumers who need user data will have to change their URLs Phase 2 - Cleaning out old files that contain user data --- This can be done slowly in the background over the course of however long it takes: 2a. remove all changeset dumps and changeset diffs from the public tree. 2b. run all .osc, .osm.pbf, and .osm.bz2 files on the public tree through osmium, scrubbing user data (retain file timestamp if possible) and re-creating .md5 files where necessary Phase 3 - Controlling access to files with user data Once the parallel systems are up and running, we will want to 3a. issue guidelines about what you are allowed to do with the user data files, 3b. ensure that everyone who has an OSM account agrees to these guidelines one way or the other, 3c. start requiring an OSM login for all downloads from the internal, "with userdata" tree. One possible technical solution for 3c is https://github.com/geofabrik/sendfile_osm_oauth_protector which also comes with a guide for users on how to run it in a scripted setup. -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Loc2Vec
Hi, On 06/18/2018 07:30 PM, Christoph Hormann wrote: > I think it is probably possible to create scenarios where all three > viewpoints (the two above and the 'no copyright issues involved' case) > are plausible interpretations. I can certainly construct a primitive "machine learning" machine that can be "trained" with OSM data and that will nicely respond with that very OSM data very time I set it to work on the same geographical area ;) I have a hunch that in order to disprove that your machine learning installation and all it produces is a derived work, you would at least have to lay open the algorithms and the training data used. If someone has a proprietary machine and trains it with OSM data, then my default assumption would be that everything the machine ever outputs is derived from OSM, because how will I know that the machine doesn't simply store everything it sees during learning? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changes on planet.osm.org
Hi, On 24.05.2018 07:22, Roland Olbricht wrote: > given that the GDPR is going into effect tomorrow and there have been > plans announced to restrict the minute diffs: It is very likely that access to user data will be restricted at some point in the future, but this will be announced well in advance. There will not be a sudden change tomorrow. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] issue with geofabrik europe update
Hi, On 05/16/18 14:43, Julien Fastré wrote: > We had a strange issue with a europe diff update from geofabrik: the > diff file is not a valid xml. And that's entirely my fault for using "sed" to modify a couple of .osc files around the beginning of May. Sorry for that! It was after we got rid of uid/user fields in .osc files, and it turned out that some people has issues with the reduced files, so we decided to put dummy uid/user fields back in, and for the old files I quickly did that with a too broad search-and-replace command ;) Fixed the file now (and another one in the "georgia-updates" dir). > I wonder if we were the only one affected and, if not, how did you cope > to pass this diff without error ? Since the bug was introduced a day or two after the diff was published, it is possible that other consumers of the diff who loaded it immediately didn't run into the issue. Sorry for that - I'm sure it took you some time to figure out what was wrong! Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] GDPR related changes to web site and API
Hi, (this has been on osmf-talk already but I think it should go to a wider dev audience) The LWG has made some recommendations about what we need to change on the web site and API to comply with future European data protection rules. On the whole this boils down to "logged-in users get the same stuff they get today, but guests who are not logged in will not see details about users". The detailed LWG recommendations are here https://wiki.openstreetmap.org/w/images/8/88/GDPR_Position_Paper.pdf. I have made a list of what I believe needs to be changed on the API and web site, here: https://wiki.openstreetmap.org/wiki/GDPR/Affected_Services It would be good if some more people with a solid knowledge of the Rails code, or people who use it a lot, could cross-check that and point out potential issues. We'll also be needing people who are willing and able to implement the changes once we agree on what's necessary ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Generalisation
Hi, On 04/14/2018 11:18 AM, Tomas Straupis wrote: > As OSM is mature enough to start generalisation (more than > "selection" operator), maybe there is some place where such topics (in > OSM context) are discussed in English? The most likely location for this to be discussed is probably within the openstreetmap-carto developer community as they would benefit most from such approaches. I don't follow their work closely though so couldn't say if the issues have been discussed in the past. I'm sure Arne himself would be happy to participate but I hear he's gone on holiday after completing his thesis ;) > Also, maybe there are ideas to translate the thesis mentioned above > to English? I'm not aware of any plans, but I do know that Arne has quoted similar work done by others, and there were many English-language works among that, so perhaps if you skim through his literature list at the end of the PDF you'll find interesting articles. Best Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: The (dark) future of Java on desktop
Hi, On 08.03.2018 00:06, Vincent Privat wrote: > I'm not sure what it implies for the long-term development of JOSM, but > nothing good I fear. I wouldn't be too concerned. With all due respect for your coding work, I don't think that the actual program code is the essential thing about JOSM. It's the functionality and user interface, the decade-long (!) evolution that has given us the powerful tool we have today. You could sit down today and re-implement everything in, say, C++, and it would be relatively straightforward, and while the result would not share any of JOSM's codebase, it would still encapsulate all the experience and brainpower that has flown into JOSM development over the years. I think what is essential about JOSM will live on even if Java should die. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
Re: [OSM-dev] Read-only API calls, fair usage
Hi, On 20.02.2018 10:51, Michał Brzozowski wrote: > You can use changeset replication with ChangesetMD to query your own DB. I was tempted to respond the same, but I imagine the idea is that as soon as the user clicks on the "I have fixed this" button, MapRoulette checks what has been committed... and replication could mean up to one minute delay. Picture the eager MapRouletteer staring at the "please wait" cursor for a minute every time they fix something... Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Working with OSM data with less or no metadata
Hi, On 14.02.2018 15:23, Martin Koppenhoefer wrote: > it seems Brexit could become effective March next year. Maybe we just wait? We would still have to apply EU regulations to processing the data of EU citizens. > I really hope we will not obfuscate or remove meta data because of some > EU privacy regulation, please do not overreact. The LWG is, or has been, discussing this with lawyers so I hope they will come up with sensible recommendations. I don't think the new regulations will be without consequences though. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Scale of downloaded images seems to vary.
Hi, On 01/11/2018 12:44 PM, Martin Koppenhoefer wrote: > note that this will always only be approximate as it will vary across > your sheet of paper (getting smaller towards the equator and bigger > towards the poles in the mercator projection we use), so you would want > to say something like "the scale in the centre of your sheet will be 1 : > 25000") Also, "this applies to the horizontal scale, the vertical scale is another matter altogether" ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend 17/18 Feb
Gah, got the date in the subject wrong on my first attempt. Fixed ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend 18/19 Feb
Hi, a new year - a new hack weekend! Everyone's welcome in Karlsruhe on the 17th/18th February (and in the pub on the evening before) if they like hacking on something together, or at least not totally on their own ;) Details on the Wiki https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_February_2018 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Automatically triggering export as PDF from openstreetmap.org -> share?
Bjoern, On 12/21/2017 09:40 PM, Bjoern Hassler wrote: > The idea would be to produce a PDF in the same way as manually going to > export, selecting format PDF, scale 1:2500 and dimensions 800x1000? The current usage policies in https://operations.osmfoundation.org/policies/tiles/ say that "Calls to /cgi-bin/export may only be triggered by direct end-user action. (For example: “click here to export”.) The export call is an expensive (CPU+RAM) function to run and will frequently reject when server is under high load." so using a script to produce these PDFs would violate the policy except in rare circumstances where running the script is triggered by a user request. What you could do instead: * download and stitch tiles, convert to PDF; search for "OSM bigmap" for different implementations. * use the "staticmap" script on openstreetmap.de like this: http://staticmap.openstreetmap.de/staticmap.php?center=40,-50=2=500x350 Both will only give you standard resolution raster images. You could also try * https://maposmatic.osm-baustelle.de/ (a working fork of the discontinued MapOsMatic project, does PDFs) or the somewhat idiosyncratic but powerful * http://printmaps-osm.de:8080/ (Europe only, quarterly data updates, does PDFs in theory but currently only PNG works) or if you're on Windows or willing to use Mono, Maperitive can also generate PDFs for any region using data from Overpass, and it's scriptable (even headless). Of course, the canonical solution is "install your own postgres/mapnik/nik4.py and run it locally" ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Live User Statistics
Hi, I've long wanted to have something a bit like "show me the way", where I can have a "live" display of what's happening in OSM. I started building something at the last hack weekend in Karlsruhe but only now had the time to make it into something that is not *totally* embarrassing. A live demo is here: http://www.gryph.de:8080/ and the source is here: https://github.com/woodpeck/livestats I'm sure it has a couple of bugs, and the user interface you see here (with the bar chart) is by far not the only possible thing; it should be equally possible to have a line chart with a time axis for the most active users or so. The whole enterprise is of course hampered a bit by the minutely granularity of the diffs; maybe some time we can have experimental streaming replication (it has been built into Osmosis since forever but I don't think anybody ever tried it: http://wiki.openstreetmap.org/wiki/Osmosis/Replication). Perhaps this gives someone an idea for their own projects. Fork away - the server doesn't really require any resources so you can easily run it yourself. (Or build your own frontend and use my API if you want.) It would also be possible to drop the server altogether and have the browser request and process the diffs directly. But I thought that with the millions of people reading this and clicking on the link above, that could temporarily bring down OSM ;) Bye Frederik PS: The server is in Perl. Sorry. I read it's the most hated programming language. Hope it doesn't say anything about me. -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Hack weekend in Karlsruhe on 21/22 Oct
Hi, there's another OSM hack weekend in Karlsruhe on the 21/22 Oct (pre-hack drinks on the 20th). As always, everyone is welcome. Details here: https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_October_2017 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Andy Allan joining web site maintainers
Hi, On 10.07.2017 11:14, Andy Allan wrote: > My intentions for the next few months are to continue to > do whatever I can to encourage new contributors. I think it would be helpful for new contributors if the following two points could be explained: * What kinds of changes to the API are acceptable while still being "API 0.6"? Is anything that adds new API calls automatically on hold until we do "0.7", or should we just refrain from breaking existing API calls? One reason I'm asking this is that there's a bunch of things that have an API call but are not accessible through the web site (e.g. "show me a specific version of this object" - website has just full history or latest), and vice versa (web site can show all changesets by a given user, but no API call exists for that). It could be a low-hanging fruit to create feature parity between website and API at least in some areas. Maybe this is even on some mental roadmap somewhere - I heard people talking about making the web site actually *use* API calls, rather than accessing the database directly. * What is the relationship between "cgimap" and the web site, and in how far are contributions that touch an area handled by "cgimap" required to cover both the C++ and the Rails implementation? One reason why this is of interest to me is that I'm still very much interested in being able to access deleted objects through the API and through the web site. I've made a few half-baked attemtps at implementing that in the past (https://github.com/woodpeck/openstreetmap-cgimap/tree/deleted_call, https://github.com/woodpeck/openstreetmap-website/tree/browse-deleted-objects) and would appreciate guidance of how to "do this right", if at all possible within the constraints of "0.6". Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] stable labels and resolution for main administrative areas, proposal
Hi, On 06/25/2017 01:32 PM, Peter Krauss wrote: > I am looking for a OSM's webservice where we can request a canonical > place name, and it returns the polygon associated with the place? I think it would be possible to write something like this, either perusing existing hierarchies in Wikipedia, or evaluating OSM data yourself in a suitable server setup. Have a look at https://osm.wno-edv-service.de/boundaries/ which already does a lot of boundary processing. We're unlikely to add a service like that to the core of OSM but the beauty of OSM is that with all the data being available openly, such a service does not *need* to be in the core (and make the core ever more complex to run) - such a service can be developed and operated by a third party. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to use transifex for osm ?
Hi, On 22.06.2017 03:33, Shrinivasan T wrote: > I am looking for translating street names and poi in tamil language. > How to use transifex for this? If an object really has a name in the Tamil language, then we would add a "name:ta" tag to the object (and not use transifex). Of course this name will not automatically show up on any map - you'll have to configure your own map rendering engine to use the name:ta tags. (But it will automatically be considered in th standard search engine.) Please only use this name:xx mechanism in situations where the object does have a name in that language. Never use it to "translate" names. For example, if there's a street in a small town in France called "rue de la gare", it would be wrong to add "name:en=Station Street" just because that is the correct translation of the name! You would only do this in an area that is bi-lingual or where the name "Station Street" is at least locally understood. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
Hi, On 29.03.2017 09:10, Alex K wrote: > * For one, this type of information is already part of > OSM: http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dpostal_code Generally we don't like data that is impossible (or difficult e.g. "knocking on doors") to verify on the ground, but we do make exceptions for admin boundaries and, usually, postal code boundaries. But that exception would certainly not apply to derived data; if it is desired to use derived data in geocoding, then the code to run the derivation must be in Nominatim (so that any derived geometries automatically update when base data is modified). > * The current postal code tagging of admin levels in > Austria/Switzerland is not only of bad quality but also wrong from a > logical aspect. The boundaries of the two have no connection in real > life. We should get rid of THAT information because it produces > false results in Nominatim. You are welcome to suggest the deletion of this information on the mailing lists/forums in Austria and Switzerland, and remove them if the community agrees. > So the information has high practicle value and can help push OSM into > new areas of usage. That's why I believe it is very important to add it > for more countries... You can add code that does sophisticated post code guessing to Nominatim but you cannot add the result of a sophisticated guessing algorithm as a base geometry in OSM - even less so if the algorithm you used for guessing isn't available for inspection. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Programatic reconstruction of postal code areas
Hi, interesting work. Maybe there's a way you can automate that and offer it as a module for Nominatim so people who would like to use "guesswork postcodes" as a better-than-nothing alternative could activate that in Nominatim. Similar things have been done before e.g. for the UK https://github.com/chfw/pc2shape and http://random.dev.openstreetmap.org/postcodes/. Importing your "guesswork postcodes" to OSM is not possible I'm afraid. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] full history files on Geofabrik server
Hi,
"full history" files are kind of a niche interest, but they can be
useful to analyze some things like "how many different people have added
a certain tag", or "how has an object evolved over time", and so on.
They can also be used to reconstruct the complete data set for any given
timestamp in the past (minus, of course, redactions).
I'm happy to announce that the Geofabrik download server now has full
history files for all regions served. These files (called ".osh.pbf")
can for example be processed with the Osmium library and the osmium
command line tool. If you want to run analyses on them without C++
coding, Osmium can convert them to a text-based format ("OPL") that is
then accessible for grep et al.
The full history files will be updated weekly.
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Make Nominatim more dev friendly
Hi, I'm not a Nominatim developer but I've followed Nominatim development and issues for a while. One thing that contributes to the impression that "pull requests/issues are ignored" is that Nominatim aims to be a good, or at least a functioning, geocoder for the whole planet. Contributors (understandably - that's how Open Source works) often scratch their own itch, they find a problem with Spanish addresses and submit a fix - but they don't notice (or care) that it breaks geocoding elsewhere (for example https://trac.openstreetmap.org/ticket/4895 where someone adds stop words). It is then the role of the Nominatim developers to think about the effects the contributor might have been missing, and tell him or her "sorry, but that doesn't work for us". This is actually good and important - it may look unfriendly to you (albeit there's nothing unfriendly in the ticket I quoted) but in fact it ensures that Nominatim doesn't break for some country once a week. > Unfortunately there's not much I can do about it apart > from pointing the problems to wider audience. You said you're a developer, have you actually tried to participate in the Nominatim devlopment? > [4] - https://github.com/twain47/Nominatim/issues/467 Are you the user "sanitas2" from this issue? I've read through it and I must say that I find the reaction of the developers absolutely understandable. I don't think you have been helpful, respectful, or polite in that issue. > Anyway, I think the solutions to the problems are quite obvious. How can I > convince someone to make the project open and friendly to new collaborators? I think this public claim that the current developers ignore "obvious solutions" won't do much good to improve their enthusiasm. What is your suggestion? Chuck out the "unfriendly" developers and replace them with whom? Or force the developers to spend more of their spare time trying to understand your issue? Can you point me to a good pull request that you have submitted and that was ignored/rejected even though it didn't break anything? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Geofabrik downloads to change multipolygon handling
Hi, this is just a small heads-up. The Geofabrik downloads (regional .osm.pbf extracts for various continents, countries etc) will switch to a different software stack later this month. We currently use osm-history-splitter which is based on an older version of Osmium, and will in the future use a new feature offered by the osmium command line tool and the newest Osmium version instead. This will enable us to change the handling of relations in the extracts. Currently, relations are included in the extracts if one of their members lies (partially) inside the area; but members of the relation that lie fully outside the area are not considered. This has adverse effects when large multipolygons cross the border, for example Lake Constance not be fully contained in any of the Germany, Austria, or Switzerland extracts. In the future we will be able to include all members of multipolygon relations that have at least one member (partially) inside the area. This will only be done for multipolygons and not for other relations (like boundaries or routes). Therefore, water bodies, forests etc. that cross the boundary should in the future be fully contained in the extracts. This comes at a price of a slightly larger extract but the difference is negligible. On the day we make the switch, all regional .osc files will have a slight size bump becasue all the missing bits and pieces required to complete the multipolygons will be contained in them. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] osmosis wrongly claims to see UTF8 problem
Hi, On 01/11/2017 10:30 AM, Frederik Ramm wrote: > SEVERE: Thread for task 1-read-xml-change failed I was a bit over-eager in shortening the stack trace. Full detail: org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse xml file x.osc. publicId=(null), systemId=(null), lineNumber=583379, columnNumber=90. at org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:114) at java.lang.Thread.run(Thread.java:745) Caused by: org.xml.sax.SAXParseException; lineNumber: 583379; columnNumber: 90; Invalid byte 2 of 4-byte UTF-8 sequence. at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source) at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at javax.xml.parsers.SAXParser.parse(SAXParser.java:195) at org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:109) ... 1 more Caused by: org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 2 of 4-byte UTF-8 sequence. at org.apache.xerces.impl.io.UTF8Reader.invalidByte(Unknown Source) at org.apache.xerces.impl.io.UTF8Reader.read(Unknown Source) at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source) at org.apache.xerces.impl.XMLEntityScanner.scanLiteral(Unknown Source) at org.apache.xerces.impl.XMLScanner.scanAttributeValue(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanAttribute(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source) ... 11 more -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] osmosis wrongly claims to see UTF8 problem
Hi, I have recently had osmosis in --rri mode refuse to apply an update it had downloaded from OSM, claiming there was an UTF8 error in the file. I looked and looked but the file was fine, passed UTF8 and XML validity checks. I tried to isolate the line that gave me the "error" but isolating it made the problem go away. Only including the 583379 previous lines makes the error occur. So I now have two .osc files, one with 583380 lines and one with 583379 lines: $ wc -l x.osc y.osc 583380 x.osc 583379 y.osc their only difference is one line at the beginning of the longer file: $ diff x.osc y.osc 2d1 < But the longer one fails to process in osmosis, and the shorter one works: $ osmosis --read-xml-change x.osc --write-null-change Jan 11, 2017 10:19:41 AM org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.43.1 ... SEVERE: Thread for task 1-read-xml-change failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse xml file x.osc. publicId=(null), systemId=(null), lineNumber=583379, columnNumber=90. at org.openstreetmap.osmosis.xml.v0_6.XmlChangeReader.run(XmlChangeReader.java:114) $ osmosis --read-xml-change y.osc --write-null-change Jan 11, 2017 10:20:34 AM org.openstreetmap.osmosis.core.Osmosis run INFO: Osmosis Version 0.43.1 ... Jan 11, 2017 10:20:35 AM org.openstreetmap.osmosis.core.Osmosis run INFO: Total execution time: 1448 milliseconds. Since the line which supposedly contains the "error" is identical in both files, it can't really be an error (and the line does not contain any non-ASCII characters). Re-formatting the XML file with "xmlstarlet fo" or "xmlstarlet c14n" makes the problem go away. I've reproduced this bug on different machines with different Osmosis versions. I've tried these java versions with identical results: $ java -showversion java version "1.7.0_121" OpenJDK Runtime Environment (IcedTea 2.6.8) (7u121-2.6.8-1ubuntu0.14.04.1) $ java -showversion openjdk version "1.8.0_111" OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14) OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode) I have uploaded the two .osc files here: http://www.remote.org/frederik/tmp/osmosis-bug-try-read-xml-change-write-null-change-on-these-files-which-differ-only-by-one-line.zip I'd be interested in any insights anyone has to share. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
[OSM-dev] Karlsruhe Hack Weekend 18/19 Feb
Hi, a new year - a new hack weekend! Everyone's welcome in Karlsruhe on the 18th/19th February if they prefer hacking on something together, or at least not totally on their own ;) Details on the Wiki https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_February_2017 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: De-Multipolygonize
Hi, On 11/06/2016 10:21 AM, Darafei "Komяpa" Praliaskouski wrote: > RelToolbox has "reconstruct polygon" button. Superb. That does almost what I wanted. Manual work is still required when a polygon has inner rings but that's a minor issue. Thank you! Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
[OSM-dev] Looking for Hackers/Admins to build OSM sandbox
Hi,
many of you will be familiar with the test APIs available from
*.dev.openstreetmap.org - that's where every now and then a new release
of the OSM web site code is made available for test driving before stuff
is incorporated into the central setup.
These test sites are, alas, rather limited. They are fit for the purpose
of testing API features, but they only have the API and a mostly empty
database behind them. These test sites lack, for example
* processing of GPS traces
* publication of planet files and incremental updates
* tile rendering (you always see the real OSM tiles on the test API
sites, not something drawn from the actual test data)
* coastline processing
* Nominatim, Overpass, Routing, Taginfo...
Not all of that is super important. But we do have regular questions
from programmers who would like to test something with diffs, or from
mappers who would like to construct a complex junction layout and see
how it looks on the map, or maybe even how the router processes it.
Doubtlessly, it would be useful for many people to have a somewhat
complete OSM sandbox.
FOSSGIS e.V. could - subject to a funding decision by the FOSSGIS
members which I can champion when the time comes - make available the
server(s) needed for such a project. But we need more than servers, we need
1. A concept: which services would we like to run, how will they
interact, is one instance enough or do we need several, what amount of
data will we store by default - and deriving from that: What kind of
server(s) do we need?
2. People to install all this
3. People to to maintenance and administration on the running system
I briefly raised the idea with OSMF OWG ("would certainly be useful, we
don't have concrete plans for something like that right now, and if
something came out of this that we can re-use in our setup, all the
better") and with the FOSSGIS board ("sounds good, and if something
comes out of this that others can re-use when setting up a sandbox, all
the better"). So at least we don't have anyone saying it's a stupid idea.
I'm now looking for a small team of people who have the time and engery
to plan and install such a sandbox together. You'd have to spec the
system so that I can go back to FOSSGIS and request the funding, and
then FOSSGIS would provide the server(s) and you could start working.
I'll send the same message to talk-de. Of course I hope that all
interested people will work on this together, and that there aren't
three competing proposals in the end.
I've tentatively set up a wiki page to coordinate discussio/efforts but
feel free to ignore the Wiki and discuss elsewhere if deemed more suitable:
https://wiki.openstreetmap.org/wiki/Full_Stack_Sandbox
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Multipolygon relation way member without role
Hi, On 09/19/2016 05:50 AM, patrick keshishian wrote: > It seems that this association is a mistake. If so, is there a > better place to report these sort of things as I come across them? As pointed out by Michael, using "notes" is an option. > As I am still learning my way through the data, I don't wish to > do wholesale edits. I hope you won't ever do wholesale edits, even after you have learned your way ;) but editing individual objects is totally ok. > Question: Is there a general rule used when processing role=""? > OSM wiki for Relation [3] indicates that roles are optional. It > looks that OSM is rendering way=428362225 as an area [4]. Certainly depends on which kind of relation it is. For polygons/ boundaries, software processing them is advised not to look at the role tag at all, and instead use geometric reasoning to determine how the polygon needs to be constructed. There is usually only one valid interpretation. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM API lookups to complement minutely diffs?
Hi, On 09/15/2016 10:53 PM, Stefan Keller wrote: > AFAIK augmented diffs are rather an experimental feature and I'd like > to avoid the latency time and blackouts of overpass which runs in same > server. So I'm concentrating on the main OSM API. This reads to me like: "There is a third-party service that solves the problem I have, but because it is experimental, I would like to build my own experimental third-party service instead" ;) The augmented diffs were made to solve exactly the problem you have. The underlying idea that Overpass implements - load updates from OSM into its own full history database and generate augmented diffs using that database - is by far the best solution in terms of load caused on the central servers. It also scales nicely; if a thousand people decide to run Overpass instances to produce their own augmented diffs then the API won't suffer. If you cannot work with overpass but must implement your own version of that, then you should at least copy the principle. The central OSM API is meant to support editing activity and doesn't scale to support use cases like yours (especially if others come after you and decide they need their own system because yours is experimental ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend October
Hi, I just made the obligtory wiki page for our next Karlsruhe Hack Weekend: http://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_October_2016 As always, the event is open to all and has no fixed agenda, but if you'd like to use it for something specific, be our guest ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Tile usage without proper identification
Hi, On 08/23/2016 04:45 PM, Manuel Reimer wrote: > For privacy reasons all my browsers don't send a Referrer. It is a valuable information for us who's using our tiles (not who as in you-the-individual, but who as in the site you are visiting that points your browser to our server). We need this information for proper rate limiting and knowing when we might have to contact a site operator and tell them to find another tile source. I'm in favour of privacy too but please spend a moment to think what would happen if everyone did what you do - we'd never know who uses our tiles, at all, and hence we'd be unable to allocate our resources for the maximum benefit of all. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Polygon inner/outer relation in osm file
Hi, On 08/23/2016 09:39 AM, patrick keshishian wrote: > What is the mechanism used, while processing "relation"-s, in > matching "inner" and "outer"-s of polygons? There's no guarantee that role="inner" and role="outer" are even set. I strongly advise against trying to write your own algorithm that combines polygons; you will get 80% right quickly and fight with the other 20% for weeks. Rings can be nested (valid), self-intersecting (invalid) or missing bits (invalid), they can have missing or wrong inner/outer tags (usually considered valid)... the C++/Python library "osmium" is probably the most advanced in building proper polygons, or check out osm2pgsql, imposm, or ogr2ogr which also have code to deal with that. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Simple app for "making contributions" (not to display maps)
Bjoern, not need to stress the inaccuracies of GPS - GPS traces is how this whole OpenStreetMap started, long before we even had aerial imagery. To this day some of the older hands in OSM consider using aerial imagery a not quite sportsmanlike way of adding data a.k.a. "cheating" ;) On 06/29/16 14:04, Bjoern Hassler wrote: > (There are of course issues with GPS traces, so perhaps automated > capture of GPS traces is not that useful.) Automatically recorded tracks, even without any extra information, can be useful but of course it always requires someone with local knowledge to translate a raw GPS track into something that can be added into the database proper. Your user's contributions would be immensely more valuable if there was a way for them to at least record the information whether they're currently on a path or travelling cross-terrain. (Sometimes this can be guessed from the speed of movement but not always.) When mapping on my own with GPS I always used the waypoint marker button on the GPS to mark intersections; this makes it easier to work witht the data later. However this information will not find its way into the standard GPS trace background in editors; it is only available when you work with the track directly. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [osmosis-dev] Reading file over http
Hi, On 06/19/2016 09:09 PM, Stephen Knox wrote: > I should have said, I am using Osmosis as a Java library and don't have > access to the command line as I am using Platform as a service, so I guess > the same applies so long as I can find a decent Bzip converter to stream to > Osmosis? Osmosis includes bzip2 support so it could decode the .bz2 itself but since Java bzip2 is much slower than the standalone commandline program, the version with bzip2 in the pipe is usually recommended. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [josm-dev] Auto center new node
Hi, On 03/22/2016 08:34 AM, Florian Lohoff wrote: > Hadnt hit it for like 8 years and it came up directly after the ugrade > of josm to the latest version - I cant rule out i hit ctrl-f > accidentally. I had never seen that mode in the past - does anyone > actually use it? I added that mode a long time ago to scratch a personal itch - at the time I was tracing very long boundary fences in Turkmenistan and after a while suffered from mild RSI from the constant mouse movements: click to set node - move mouse - click - move - click - move and then when I reached the edge of the screen, click - drag map over - and then repeat for hours on end. The new mode allowed me to keep the mouse pointer roughly somewhere right of centre, and between clicks I would only need a minimal adjustment to move the mouse pointer onto the fence line. But maybe this use case is much rarer today, and/or other JOSM input modes are meanwhile available to cater to that use case. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" signature.asc Description: OpenPGP digital signature ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] SSL-Certificate for osm.org
Hi, On 03/21/2016 09:22 AM, Tom Hughes wrote: > The horribly backward Java certificate root authority list is the main > obstacle to most of our attempts to improve https support in fact. Perhaps we could just ignore that? I'm a JOSM user myself but I don't think that the rest of the world should suffer just because Java is unhappy with our SSL. I'm sure that JOSM users who desperately need SSL can find a workaround (could one not e.g. have JOSM connect insecurely to localhost and then reverse-proxy https://openstreetmap.org/ from there?) Or perhaps there are alternative SSL stacks for Java that can be employed? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release candidate for new osm2pgsql 0.88.0 stable release
Hi, On 02/27/2016 05:57 PM, Paul Norman wrote: > We ended up removing nodecachefilereader > (https://github.com/openstreetmap/osm2pgsql/pull/545), as it is no > longer useful. It was developed for the initial testing of flat nodes, > and hasn't been properly maintained in years. I routinely use nodecachefilereader to find out the highest node ID in order to bootstrap replication. But I guess just --adding an empty .osc file with osm2pgsql will also produce console output that includes this ID so I could use that instead... Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend in February
Hi, there will be another OSM hack weekend in Karlsruhe on the 27/28 February: http://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_February_2016 Everyone is welcome. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Which is best for this task, osmosis, osmconvert osmupdate or...
Hi, On 12/30/2015 01:02 AM, Dave F. wrote: > A couple of problems with that, I'm afraid. Firstly they don't provide > my area & secondly I'm trying to avoid large downloads like the 670mb > England file & amendment files for data I'm never going to need. Is > there not a way to work just within a user specified polygon? There's no service I know that would deliver updates for an user defined area (only). It is however possible to apply updates for a larger area to your extract if you cut away the extra stuff afterwards again, i.e. the workflow being: (once) download UK, or Europe, or world (once) cut out your area of interest (regularly) apply UK, or Europe, or world updates to your area of interest yielding a file with your area of interest plus bits and bobs around the world (regularly) cut out your area of interest from the "area of interest plus bits and bobs around the world" file Of course if the area you are interested in is *so* small that even downloading the too-large diffs each day will seem like a waste, then I'd recommend to simply download the full area from Overpass regularly. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Using native social SDK for signing in to OSM on mobile
Hi, I have a rather non-technical remark about this recurring "we need to make sign-up easier" topic. My question is: Do we want to encourage casual editing? And my answer is "not 100% sure but perhaps rather not". There are some benefits to casual editing; if people could just fire off a quick edit to something without even signing in then surely we could get more people to do just that - upload a quick OCR'd photo of shop opening times or whatever. Point, shoot, upload, bam! - OSM improved in 5 seconds. I see that benefit and I would like to have it. But I also view us OSM contributors as a community. We share something. We care for this project together. We participate in various communication channels. We watch our backyard. We chat up new users and invite them to meetings. I think it is important for people to make a decision to join this community. This decision is not just a quick "I agree" screen where you put your work under a certain license; it also means you should know that you're signing up for something here; that you take responsibility; that you have to be contactable, and will be contacted, about your contributions. Making it too easy to breeze through the signup process, on a mobile device, using your stored credentials from elsewhere - how can we expect anyone who signs up this way to understand what this project is about, what he's signing up to? "Making signup easier" is certainly a good goal to have, but signup includes getting people to understand OSM. A workflow that lets people sign up in 5 seconds but lands us with users who don't even know what the consequences of their actions are is not a step forward, it is detrimental to the project in my opinion. This is not saying you shouldn't write an easy to use mobile editor, or you shouldn't attempt to reduce the mobile signup workflow to a few clicks, but from anyone who ships an app that does quick signups I'd like to see a concept of how they intend to make sure that users understand what they are signing up for (legally and socially). Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] bug in tirex mapnik backend?
Hi, On 12/16/2015 01:41 PM, Stefano Salvador wrote: > I'm trying to set up a tile server with tirex but using the latest svn > code the mapnik backend fails with a buffer overflow. I've wanted to move tirex from SVN to git, and created https://github.com/geofabrik/tirex where the bug you describe has been fixed a while ago. I then had second thoughts about hosting it on "geofabrik" and wondered if it should rather be on "openstreetmap" and then procrastinated, failing to actually remove the SVN repo and replace it with a "moved to github" message as I had originally intended. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Geofabrik OSM extracts breakage
Hi, on the night of Saturday 05 December, a couple of European Geofabrik country extracts (*not* the full Europe extract) failed to compute correctly, and the auto-generated diff files for these region then indicated the deletion of a very large number of objects (sometimes *all* objects). This mistake was detected and fixed on Sunday noon, by dropping the broken data and reverting to the previous version. However, anyone who downloaded and applied the diff between Sunday ~ 4am UTC and ~ noon UTC will probably have ruined their database and will have to start anew from a full extract. Simply carrying on downloading diffs will *not* suffice because when reverting to the previous version we completely erased the broken diff, and hence the diff generated the next night will *not* simply make everything good again if you consumed one of the broken diffs. A list of affected extracts is in this forum post: http://forum.openstreetmap.org/viewtopic.php?pid=564747#p564747 I'm sorry if this has caused a hiccup in your downstream services and I'm implementing a few extra sanity checks to avoid something similar in the future. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] creating OSM-based tactile maps
Hi, On 12/02/2015 04:22 AM, Rich Morin wrote: > I'm hoping for some suggestions on how to get from OpenStreetMap > data to tactile maps for use by blind or visually impaired users. > See below for a bit of background information and some qustions. There was a project once to create these maps but I don't know the status, it may be dead/frozen. Here are two wiko pages (furGerman-langauge wiki pages http://wiki.openstreetmap.org/wiki/HaptoRender (German version of the page may be more current - try auto-translating) http://wiki.openstreetmap.org/wiki/OSM_for_the_blind There's a German-language mailing list https://lists.openstreetmap.de/mailman/listinfo/haptischekarten The main people behind these efforts were OSM users bahnpirat and Lulu-Ann and if all else fails, try to contact them directly for info. Best Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Truncating geometries
Hi, On 11/09/2015 10:00 AM, Malcolm Herring wrote: > Are there any tools out there that will properly truncate ways & > multipolygons at bounding box boundaries, keeping closed ways & > mutipolygons as properly formed areas? Not to my knowledge. You'd have to convert to shape files and then use ogr2ogr (or similar) to achieve that. Of course nothing keeps you from running something like shp2osm or polyshp2osm on the resulting shape file, generating a pseudo-OSM file again... Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New Map Style feedback
Hi,
On 11/02/2015 09:40 AM, Maarten Deen wrote:
> I agree that the abandonging of the blue for motorways is a bad choice.
> It is not only a british color, motorways are signalled in blue also in
> lots of other countries in europe.
I find it strange to see many people arguing that there should be a
similarity between street signs and the colour used on the map. Germany,
for example, uses blue motorway signs exclusively, but the first thing
that went out of the window when the German OSM style
(http://www.openstreetmap.de/karte.html) was created in 2011 was the
blue motorways ("nobody apart from the Brits likes that") ;)
Bye
Frederik
--
Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33"
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] proposal for mechanical edit reg. power_source + generator:source
Hi, On 10/13/2015 08:10 PM, Gerd Petermann wrote: > my motivation is simple: > JOSM claims that the old tag should not be used, > the wiki is also clear: > "Use generator:source > <http://wiki.openstreetmap.org/wiki/Key:generator:source>=*instead." Yes, but JOSM presets and Wiki pages are not made by the community as a whole, and can never serve as a reason for running a mechanical edit. It requires only a very small number of people to devise a silly preset or "vote" a tag to be deprecated; this absolutely must not lead to others mass-editing OSM according to the wishes of that minority. > I assume that I am not the only JOSM user who is wasting > time looking at these obsolete tags, In that case, a change in JOSM would be the best solution. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Announcing the launch of OSM Maps for Wikipedia
Hi, On 09/18/2015 02:53 PM, Yuri Astrakhan wrote: > Sadly not yet -- mapnik needs to be substantially changed to pass > through the hstore. Looking for a kind hardcore c++ soul )) Make a view against the hstore table and let that view have as many columns as there are name:xx tags ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Nect Karlsruhe Hack Weekend 17/18 Oct
Hi, our next hack weekend in Karlsruhe is on the 17/18 October. Details on the Wiki: https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_October_2015 Best Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Common presets for OSM editors?
Hi, On 06/25/15 11:16, Dirk Stöcker wrote: Actually ATM I see not a single advantage of a we replace you stuff with a github project proposal, but many many disadvantages. I can see that this discussion is not going to go anywhere, and instead simply irritate lots of people. Can we not just have a plugin that replaces all of JOSM's presets with those taken from some common-preset-github project in whatever language and fashion supported by the project (and add code to JOSM core that would allow the overwriting/removing of core presets by a plugin if required)? Then people could decide for themselves which set of presets they prefer, and we wouldn't have to battle it out for them here. In the long run, either everyone will use the github project set of presets and we can kiss the JOSM ones with all their hand-curated translation goodness goodbye, or everyone will keep using the JOSM presets becasue they are simply better. Who knows what will happen. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] *** GMX Spamverdacht *** Re: get node data from flatfile
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 05/17/2015 09:42 PM, Bernhard R. Fischer wrote: But there is no existing plugin for flatfiles, right? It always reads flat files. (Option -i) But probably I misunderstood your problem. Yes, Walter is talking about the node coordinate index file created by osm2pgsql's --flatnodes option. It is a binary file that contains (only) the coordinates of every node. Bye Frederik - -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEcBAEBAgAGBQJVWPCaAAoJEOx/uhGAJu9HC6EH/0oA4QJgiUkl4emWkjSJ8UCw 6DKB4V1PgGegkQmCKWbYuyAcebQlLgwewR2S1LI/cbjUA2KHJ7leVAqd/Uis1jrA Xa6j3xR3/Y9w2ehoPtTdMrncvRWmGj+FTy+ZHjguToYtSjoZhVo3BpKd1cp9Wykd jukdF7FD3miWDxIz5ZzE3FZ4DKxZHz3JAd/hGNd30l4+YkzoHavNEF+TKd+KsFqz 9mz66QfEgiUh0GfYAN5VjRCfafa8SB0yS8rEnyndzIhMUK5f6k0PPk832S65ALZO k2wSEqIOTJYjWJZqyylr31CV5VM8CEc4mddTkQKudtdh8fXlmbTmsLmH9MIDUXw= =ssu4 -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[josm-dev] Payment-Preset-Spam?
Hi, https://www.openstreetmap.org/node/2539752897 Hat sich da der User dumm angestellt (hallo Simon!) oder ist das Payment-Preset kaputt? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Payment-Preset-Spam?
Hi, On 05/03/2015 12:07 AM, Frederik Ramm wrote: Hat sich da der User dumm angestellt (hallo Simon!) oder ist das Payment-Preset kaputt? Shit, ich hätte schwören können, da stand JOSM als ich geguckt hab. War wohl schon zu spät. Sorry. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
[OSM-dev] Cloud Foundry
Hi, has anyone set up, or attempted to set up, a tile server in a Cloud Foundry environment and is willing to share what they learned? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Google Summer of Code announcement + Mentor sign up
Hi, On 03/03/15 01:13, Cristiano Giovando wrote: In the mean time we would like to add one or two ideas to your list since they are OSM related: https://github.com/hotosm/HOT-Project-Ideas/issues/15 https://github.com/hotosm/HOT-Project-Ideas/issues/7 I think that /7 is, at least in the form it is currently presented, very difficult as a GSoc project. It appears that it requires someone who is not only up to speed with CartoCSS but also knows enough about HOT work to make hard what's in and what's out decisions - quoting from the comments: rendering ... it's an editorial choice that serve a purpose, so having someone that would take the issues one by one an adding the requested feature on the map would not help ... the challenge about the rendering is a lot in making arbitration in what to render and what not. some of these improvements need skills not just technical but also a lot of communication... We need to ensure that any new tags we are selecting to render are agreed upon by many of the actors in HOT ecosystem including actors out in the field and those involved in activations and OSM communities HOT-francophone communities... To me this sounds much too broad and too difficult for a GSoc student project. As it currently stands it would only be suitable for a student who is already a long-time HOT insider. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[josm-dev] List Admin Role
Hi, I am currently the list admin for josm-dev. This means I can theoretically kick people out (which I'd never considered doing), and that I receive moderator requests when someone who is not registered tries to post (I ignore them and then they expire). I also have a couple of config options with mailman which I am not using. Since I'm not exactly a regular here, I was wondering if someone from the group of more active participants would like to have that role transferred to them? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] Is there or should there be an OSM approved work exchange forum? (Jo Walsh)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 02/13/2015 01:19 PM, Richard Welty wrote: the premise is that it would be for osm related work only (including projects that are using osm tools not necessarily with osm data.) job advertisements for non-osm geo work would not be welcome. Please also make (and enforce) a decision about whether, and in what form, OSM professional services firms are allowed to participate. I am one of the directors of Geofabrik. I never advertise our services directly in the OSM lists/forums, and when someone asks on help.osm I always point them to the list of businesses on the wiki (which, as has been correctly remarked, cries out for some editing but understandably I am keeping out of that). If such a list were created, I would like to see rules about (a) whether it is ok for me as a business to solicit the help of freelancers in projects, and (b) whether it is ok for me as a business to reply to inquiries about services, and whether such replies ought to be public or private. I don't mind either way and will happily adapt to whatever rules are set, I'm just keen on having a level and clearly defined playing field - - and not one where business opportunities depend on how brazen the PR person is ;) Of course such a list could not be approved in any way like the subject line suggests, because who would be approving? Bye Frederik - -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJU3ggzAAoJEOx/uhGAJu9Ha/8H/RrBr/A/7MFp1c6mg0tu+Sf9 rr1sRaclQJPPqRzMQdgsefsVsKs84jHWDA4gGXyqTU2AVqy7WhLsAuvCzyi29ixV 1AxDpftrqDfJKvC4kiCbgWQ9POk5VqsJq8POIy5BNJC2daFjZiFqiZUvLIo6UPCD QZ5mg7tjeLJp8Ajntegi9SrA5nQA48Alh9QEbLuPgKU/Jh9bF2ZCoA2VE3Y9srqY arscrcqmvXsmaBZwniv/Cp1iCky1uHw9VqmuvQL3NCoo1CypwT0f+c7qcWb2ekYz pwVJSnsKj54Baikq2HFEhhozMNpXsSaB5CRPrecukGzbM7bkTxTnplMMdIQ7BNk= =J367 -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Is there or should there be an OSM approved work exchange forum? (Jo Walsh)
Hi, On 02/13/2015 05:13 PM, Jo Walsh wrote: Of course such a list could not be approved in any way like the subject line suggests, because who would be approving? The community of people for whose benefit decisions are enforced? Ok then let's call it self-approved ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release openstreetmap-carto v2.28.0
Hi, On 02/08/2015 08:38 PM, Matthijs Melissen wrote: You are right that this does not look right, it is indeed a bug. We will try to solve this problem as soon as possible. It also appears that there is a problem with railway lines at z12 (only) - more precisely, a bug with *no* railway lines ;) This z12 tile has no railways: http://a.tile.openstreetmap.org/12/2140/1382.png z12 Same area on z11 http://a.tile.openstreetmap.org/11/1070/691.png and on z13 http://a.tile.openstreetmap.org/13/4280/2764.png both have railways. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM with Hadoop
Hi, On 01/14/2015 11:16 AM, Paweł Paprota wrote: Main central server with a single multi-TB-sized database somehow screams single point of failure to me... ... until you learn that there's actually an active replication, based on technology that has been tried and proven. I'm not saying brush away parallel storage, just don't assume from the outset that it will magically solve problems without incurring others. Also, we're not the only people operating large relational databases, and development on that front is also occurring, with the potential of an efficient multi-master replication system somewhere down the line leading to a different kind of also-parallel infrastructure. Where the failure points of each of these architectures lie and how cost-effective, maintenance intensive, or error prone they are, is indeed a good subject for analysis. All I'm saying that it's worth to apply a scientific approach (i.e. looking at facts) rather than a geek approach (i.e. looking at cool new technology and shiny lights). And from Stephen's paper I had the impression that we wouldn't easily be blinded by those shiny lights which is a good thing. (In our particular case, the amount of editing that occurs rises far slower than the amount of data that we have collected, which means that it is not unlikely that we will be able to work with a centralised writing, distributed reading approach for quite a while still.) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM with Hadoop
Stephen, previous discussions of combining NoSQL *or* massively parallel storage with OSM were often less driven by the approach let's investigate solid future storage models for OSM but rather by hey there's a cool new technology I'd like to play with and I'm sure it can somehow work with OSM. The results were often, if there were any at all, along the lines of hey this particular very specific use case is now 2% faster than before, but looking closer you'd see that the same speedup could have been had with an old-fashioned un-sexy create index statement if the author had known anything about PostgreSQL/PostGIS (*), or maybe that the data import took five weeks unless you had massive hardware, or somesuch. I was therefore a bit skeptical reading your message, but relieved when I found that you're keeping an open mind about the results and plan to thoroughly analyse whether using a massively parallel storage will indeed perform better than plain old PostgreSQL/PostGIS for what are OSM's everyday use cases. (I'd like to see the word cost-effective thrown in somewhere - and for data reading we have a sufficiently well scaling data replication in place already. As far as the central database is concerned, OSM is very interested in making it easy for everyone to run their own local copy.) Bye Frederik (*) It is an often overlooked fact that the amount of actual geo information in the central database is small - just the node coordinates - everything else is plain old relational stuff. Therefore the OSM database doesn't even need or use the PostGIS spatial extensions - but they are often used for analysing OSM data after importing them in a separate database. -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend 21/22 Feb
Hi, I've just set up the wiki page for the next hack weekend in Karlsruhe on the 21/22 February: https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_February_2015 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Release openstreetmap-carto v2.25.0
Hi, ah, an opportunty to hijack a thread and plug my pet feature: On 12/12/2014 07:59 PM, SomeoneElse wrote: Rather than say let everyone contribute to the standard map style why not say let everyone create their own map style - let a thousand flowers bloom! I think it would be nice if we could somehow be more dynamic in what layers we support. Currently only world-wide and high-performance third-party tile layers are eligible for addition to www.openstreetmap.org, but just like our editors can offer different aerial imagery layers depending on where you edit, the osm.org map could also, if you're zoomed in to some place, offer regional tile layers that only cover a country or even smaller region. The layer switcher could get an extra selection called regional layers and if you click that, you'd be offered a selection of what is available. Not that that would solve the question of direction-setting for the openstreetmap-carto style in any way but it would be really nice to have. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Adding Slippy Map to Taginfo's Projects Tab?
Hi, On 11/25/2014 08:27 PM, Paul Norman wrote: Any news about a quick-and-dirty 80/20 solution for adding Slippy Map as a project file? What do you mean by Slippy Map? A slippy map doesn't necessarily use OSM data. I think he meant openstreetmap-carto. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[josm-dev] Undelete
Hi, I have added code to the OSM API that supports a /deleted API call with a syntax equal to the /map call, just returning deleted objects in the bounding box. The aim is to use this for an undelete feature. In order to verify that what I'm doing makes sense, I'd like to code such an undelete feature for JOSM. It would roughly have to do the following: * Make deleted call against API. This call will return a number of deleted objects (e.g. way id=1 visible=false version=2) which will neither have tags nor members or coordinates. In addition, the call will also return the previous, not-deleted version of every object, as well as all linked objects (i.e. nodes for a deleted way) even if those have not been deleted. * Display deleted objects in separate layer. * Allow user to copy stuff from separate layer into a normal data layer, thereby undeleting them. * When undeleting, even though version X is being shown, JOSM will have to use version X+1 on upload since that is the current version on the API. I haven't kept track of JOSM development in the last few years, so you could help me a lot by pointing me in the right directions. Would it make sense to piggyback this on e.g. the changeset reverter plugin or something else that does similiar things? When copying a way from one layer to another layer, will required nodes be copied with it? What if a required node exists in the target layer already? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/josm-dev
Re: [osmosis-dev] Supplying postgres port to osmosis commands
Hi, On 10/10/2014 10:49 PM, Walter Nordmann wrote: just call osmosis with host=localhost:15432 Oh dear, that was probably the reason the port thing was never implemented - because it is so simple to embed it in the host. Should have thought about that sooner ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Documentation Improvement
Hi, On 10/03/2014 12:31 PM, Vanya Jauhal wrote: all beginners currently have to start with the OSM tasking manager Can you explain more? As I understand it, the tasking manager is a very specialist tool that will not usually be the first thing a beginner comes into contact with. Beginners in OSM usually start with mapping things they are familiar with, like that bakery around the corner from where they live. They choose their own field of activity. The tasking manager is a coordination platform that lets *other* people tell you what you should be doing. It is used for unusual purposes like data imports or HOT activations, but it is far from the standard mode of working for beginners. 1. The OSM beginner's page can have an FAQ section, where we can add some common issues faced by users and their solutions. Are you talking of the Wiki or some other platform? We have plenty. 2.Adding user comments to the documentation. This kind of depends on which platform you have in mind. The Wiki, of course, already allows everyone to edit the documentation. If you're thinking of another platform, the mechanism for reader participation might be different. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Karlsruhe Hack Weekend 18-19 Oct
Hi, we'll have another hack weekend in Karlsruhe on 18-19 October. Details, as always, on the Wiki: https://wiki.openstreetmap.org/wiki/Karlsruhe_Hack_Weekend_October_2014 Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
