Re: [OSM-dev] dev Digest, Vol 158, Issue 3

2018-05-16 Thread Tomas Straupis
Hello

2018-05-16 10:01 GMT+03:00 SandorS wrote:
> The statement was not quite wise and in some aspects it is wrong. Blindly
> trusting open source solutions is not the best thing for a newcomer
> especially not for a developer. By experience I know that sometimes a hint,
> a simple warning may help a developer to change his way of thinking.
> Besides, reading someone’s complex and complicated source (like the
> generalisation related source) is not just a simple exercise. If you ever
> wrote a complex basic sw and tried to read it after several months of brake
> then you understand what is my point.

  I think there was some misunderstanding here. I only stated that
using commercial (or to be more precise - closed) software in my
opinion is not the way to go for an open project like OSM. Results of
generalisation using commercial software can be used to refine the
approach (by looking, comparing), but final software will probably be
open (and open gis software is gaining fast anyway). For example
Netherlands example of using commercial software shows that
generalisation is successfully(?) done on PARTS of data, so now we
know that we do not have to think of ways to generalise whole world at
once.

  Open data entered by volunteers does have a higher probability of
errors, but generalisation could be just one of many ways of detecting
such errors (and so fixing them).

  Regarding other points. There are a lot of different operations done
on a lot of different types of objects in different sequences. But in
my opinion it is not "all or nothing". You can start with some
generalisation and progress with time.
  For example doing simple polygon aggregation/amalgamation is doable
now with good results but should of course be improved with proper
polygon conflict resolution methods (already described in numerous
scientific papers, f.e. "Detecting and resolving size and proximity
conflicts in the generalization of polygonal maps", Bader and Weibel
1997).
  Transport network collapsing (multi-lines to one line) can already
be done with standard PostGIS functions (buffer,
approximatemedialaxis).
  Building simplification is already in testing stage.
  GRASS provides functionality for transport network displacement and
simplification/selection.
  And these already give good and noticeable results. Of course a lot
of other things must be done, but we have a greatest luxury - time -
there are no deadlines. This is why some operations could be
implemented with even better results than closed implementations.

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-05-02 Thread Tomas Straupis
2018-05-03 1:05 GMT+03:00 Marco Boeringa wrote:
> You do realize the 1-2 years is well after the 2013 date that the Dutch
> Kadastre started to publish their work?

  Lithuania was given as contra to "the only".
  Savino (Italy) was given as contra to "the first" (as his work was
published in 2011). (It is also a "must read" for anyone interested in
generalisation)

  Anyways, there is no point of talking about who first, last, only
etc. All approaches using closed commercial software are pointless for
OSM - it cannot be reused. Everything can be done with open source so
that all code/algorithms are open and clear and there is no need to
pay piles of money for nothing.

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-05-02 Thread Tomas Straupis
Hello

2018-05-02 19:33 GMT+03:00 Marco Boeringa wrote:
> The generalization I wrote about was just a crude basic generalization of
> vector (building) data of OSM using a default tool of ESRI's ArcGIS. The
> specific tool used (Simplify Polygon), has more advanced settings than
> standard Douglas Peucker, but by itself does nothing really special other
> than weeding out vertices / nodes. I just attempted to use it with different
> tolerances to see what the results would be, and concluded the resulting
> defects in building topology, were not worth the reduction in file size.

  You've probably used this:
  
http://desktop.arcgis.com/en/arcmap/10.3/tools/cartography-toolbox/simplify-polygon.htm

  And I'm talking about this:
  
http://desktop.arcgis.com/en/arcmap/10.3/tools/coverage-toolbox/simplify-building.htm

  But both of these are closed and tied to their proprietary
architecture. And there is even less information on implementation
than in first(?) scientific paper about actual building generalisation
algorithms: Sester M. ( 2000 ) Generalization Based on Least Squares
Adjustment In: ISPRS (ed.)

> However, as to interesting stuff to read: our national Kadaster of the
> Netherlands, is actually the very first and only national mapping
> organization world wide, that has successfully managed to implement a fully
> automated generalization work flow for generating 1:50k maps from 1:10k
> maps, including landuses, waterways, highways, but also generalizing
> build-up areas and buildings. They used a range of cartographic
> generalization tools from ArcGIS (that I didn't use...).

  Congratulations to national Kadaster, but I'm not sure you're
correct about "first and only". Our local (Lithuanian) land agency (or
to be more specific gis-centras) has completed automated
generalisation 1-2 years ago (using esri tools as well). As far as I
know fully automated and done in ~day.

  Most GIS people use a work by Sandro Savino "A solution to the
problem of the generalization of the Italian geographical databases
from large to medium scale: approach definition, process design and
operators implementation". Author claims to have completed automated
generalisation for Italy and it dates to 2011. This work is very
interesting because instead of referring to closed commercial tools it
has a very detailed description of how to actually do this and that.

  Also Swiss Topo is known to be doing a very high quality
generalisation for years(?).

  But thank you for your links, it is interesting to learn how
different countries handle generalisation.

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-04-16 Thread Tomas Straupis
SwissTopo maps are some of the best examples of generalisation (as
well as other cartography principles/techniques)
https://map.geo.admin.ch/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-04-16 Thread Tomas Straupis
2018-04-16 19:34 GMT+03:00 Marco Boeringa wrote:
> No, buildings are not the most interesting. I once generalized all buildings
> in Denmark. It only reduced the storage by maybe 5%, at the high cost of
> heavily distorting a large number of them. Most buildings in OSM are in fact
> already in their most generalized state: just 4 nodes. Unless you think
> triangles is a suitable representation ;-)

  Interesting, what algorithm did you use?

  I'm playing around in Vilnius which has urban houses, big block
houses, industrial zones and old town with lots of connected buildings
of very irregular shapes.
  In Vilnius there are 54267 buildings tagged with 366979 vertexes.
  Clustering them with distance of 5m gets 45810 objects (of course
with the same number of vertexes).
  Removing buildings with area < 100 and having neighbours in < 500
meters I'm left with 28974 buildings with 299224 vertexes.
  Simplification (amalgamating buildings in the cluster and trying to
remove edges < 20m) reduces the number of vertexes to 117108.
  So this is much more than 5%.
  There are still a lot of problems (no triangles:), but I do not
expect number of vertexes to rise considerably.

  Even "dumb" generalisation (st_buffer+- with join=mitter) reduces
vertex count by ~25%.

  Reducing storage/tile size is not the only/main purpose of generalisation.

> Besides, buildings are only shown at high zoom,
> while generalization is most needed and beneficial at
> low zoom. Lastly, most vector generalization algorithms are primarily
> designed and effective for rather smooth and node rich data, like a
> stream-digitized feature, neither of which relates to square buildings.
> Hence  i consider generalizing buildings largely senseless.

  I suspect you're talking about st_simplify(preservetopology) use in
vector tile generators. Which as mentioned earlier is only technically
a "generalisation".
  http://www.gitta.info/Generalisati/en/html/unit_GenProcedure.html

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-04-16 Thread Tomas Straupis
2018-04-16 11:34 GMT+03:00 Martin Koppenhoefer wrote:
> There are some precomputed / extracted data files though, some of which
> contain generalized (simplified) data. These are all "external" sources:
> <...>

  Ok, so this is natural polygon generalisation. Looking at
https://github.com/imagico/coastline_gen the method used is to
rasterise, process and then vectorise back.
  I wonder if that is better/faster than full vector way:
st_clusterwithin, st_union, st_buffer(positiveN),
st_buffer(negativeN+M), st_buffer(positiveM) with a seasoning of
st_simplifypresevetopology according to taste.

> Another aspect is filtering: osm-carto removes features when they would be
> very small (pixels at a given zoom level) and lead to "noise".

  Filtering (selection) is technically also a generalisation.
  But you need to group and probably amalgamate them before deciding
that it is "too small". For example if we have a lot of small patches
of forest close together (say 1000 patches of 10x10m with distance
between patches of 1m) you would want to amalgamate them to one large
forest, not to get rid of them all.

  Ways and especially buildings are the most interesting (difficult) part :-)

P.S. GRASS claims to be doing displacement and way selection
(https://grasswiki.osgeo.org/wiki/V.generalize_tutorial)

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Generalisation

2018-04-15 Thread Tomas Straupis
Hello

2018-04-15 22:54 GMT+03:00 Frederik Ramm wrote:
> The most likely location for this to be discussed is probably within the
> openstreetmap-carto developer community as they would benefit most from
> such approaches. I don't follow their work closely though so couldn't
> say if the issues have been discussed in the past.

  Good point. I'll watch openstreetmap-carto.

>>   Also, maybe there are ideas to translate the thesis mentioned above
>> to English?
> I'm not aware of any plans, but I do know that Arne has quoted similar
> work done by others, and there were many English-language works among
> that, so perhaps if you skim through his literature list at the end of
> the PDF you'll find interesting articles.

  I'm already buried in pdf's from google scholar :-) You start with
one paper, then open references, then references in references and
here you are the "wikipedia effect".

  The point is that Arne's work looks (from what I did understand) at
generalising openstreetmap data, while most papers look at general
data. This is important because OSM tagging does help a lot. Some
generalisation can be done much easier with OSM data because of
additional tags and you can add additional tags describing some real
properties (not designed for the sole purpose of generalisation) of an
objects which could help generalisation. For example with road
generalisation, a lot is written on intersection generalisation.
Having *_link ways or ref tags in OSM helps this effort a lot. Railway
network also has different subtags like main/siding/spur etc. which
also helps a lot. So while generalisation is generalisation anywhere,
but generalisation of OSM data does have some important and useful
differences/advantages.

  Another interesting thing is recalculating/updating generalised
data. "OSM way" is to update very often. So it is important to be able
to recalculate generalised data only on impacted part and it is not as
trivial as calculating "dirty tiles".

  I do not know how much of that is covered in Arne's thesis. Will try
to read/translate German version if there are no plans for
translation.

  Thank you!

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Generalisation

2018-04-14 Thread Tomas Straupis
Hello

  In last weeklyOSM (http://www.weeklyosm.eu/archives/10227) there was
a very interesting point:

  "Arne Johannessen has published his diploma thesis (PDF) with the
title "Algorithms for automated generalization by combining polylines
in OpenStreetMap for specific special cases" and he also released the
corresponding Java source code."

  As OSM is mature enough to start generalisation (more than
"selection" operator), maybe there is some place where such topics (in
OSM context) are discussed in English?

  Also, maybe there are ideas to translate the thesis mentioned above
to English?

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] History feed stopped working

2017-10-27 Thread Tomas Straupis
Hello

  For two days history feed is throwing 500 error:
  http://www.openstreetmap.org/history/feed/?bbox=20.9,53.95,26.83,56.45

  Is this intended? Maybe url has changed?

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] PostGIS query "Crossing ways"

2016-09-12 Thread Tomas Straupis
2016-09-11 18:20 GMT+03:00 Mike N :
> Given a PostGIS database populated from OSM data by osm2pgsql, and 2 sets of
> lines  (such as the selection of all footways and the selection of all
> roads)   what function or series of functions will result in a list of
> locations where footways cross roads without any OSM connecting node?  (the
> equivalent of JOSM's "Crossing Ways" warning)

  The closest I could get to this was by splitting way
(planet_osm_lines) geometry into their segments. I was creating a new
table say „segments“ with identical attributes as planet_osm_lines in
which the same osm_id would be repeated as many times as many segments
that way has. (You could also do it without actually creating a new
table and letting postgresql process everything on the fly).

  If you have a way with three nodes, you would get two new lines (two
line geometries, two records in the new table).

  Then you can use PostGIS functions to find intersecting ways. But
you will not find intersection problems where two ways cross each
other on the same x/y, but they have two different points in OSM (two
different points with the same position one for way1, another for
way2).

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Reverting changesets

2016-01-25 Thread Tomas Straupis
2016-01-25 19:58 GMT+02:00 Andy Townsend:
> If it helps, I saw the same problem yesterday ("does not apply any
> changes").  What fixed it for me was to update JOSM to the latest stable
> version (the monthly plugin check had occurred earlier that morning, but I
> was running the previous stable version of JOSM). After doing that I was
> able to do a normal "full changeset revert" without problems.

  I have a wget of josm-tested.jar in my JOSM launch script, so I am
always using the latest version.

  But I managed to revert changes using perl scripts. I just had to
create a changeset separately and then provide its number to revert.pl
- then it worked ok.

  Thank you for help.

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Reverting changesets

2016-01-25 Thread Tomas Straupis
Hello

  Three changesets were found to be accidentally deleting quite large
amounts of correct data. I have informed the user and asked him to
wait until revert is completed. Changesets in question are: 36781121,
36781218 and 36781956.

  I'm starting with the last one and intend to revert all three.

  But there I have a problem, JOSM reverter plugin does not work (it
fetches info but does not apply any changes and does not complain
about anything) and there was info on weeklyosm that reverter plugin
is broken at this time.

  I then tried reverter scripts (as per
http://wiki.openstreetmap.org/wiki/Revert_scripts). But running:
  ./revert.pl 36781956

  fetches objects, identifies correct delete/modify operations to be
done but then fails with:

  PUT changeset//close
Use of uninitialized value $body in concatenation (.) or string at
OsmApi.pm line 227.

  I'm still running in dry mode. I wonder if I could proceed anyway or
should I ask somebody else (DWG?) for help?

  Thank you

P.S. I'm running scripts on fedora 23.

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] osm.org website translations

2015-04-08 Thread Tomas Straupis
Hello

  How often are translations loaded from translatewiki.org to openstreetmap.org?
  Or should I somehow request that?
  Currently month old translations (Lithuanian ones) are not loaded yet.

  Thank you

-- 
Tomas

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] osm2pgsql invalid polygons

2012-05-07 Thread Tomas Straupis
So what is the general rule?
Inner polygons in multipolygon should not have shared vectors (or in
human readable format: inner polygons should not touch each
other)?

-- 
Tomas Straupis

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Info request on: additional GIS layers

2012-03-07 Thread Tomas Straupis
2012-03-07 Andre Joost wrote:
 Another question is how they keep their basic database in sync with ours?

  That is a different question with a lot of different approaches
ranging from using only OSM data up to simple validation-comparing of
two available datasets for error identification.

 I think
 http://wiki.openstreetmap.org/wiki/The_Rails_Port
 is what you are looking for.

   Yes, that is it! Thank you

 Or they can use the UMN Mapserver to combine Data from several sources.

  Interesting. Thank you for pointing at it.

-- 
Tomas Straupis

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Info request on: additional GIS layers

2012-03-06 Thread Tomas Straupis
Hello

  Some governmental institutions are willing to use OSM software stack
and share their information. For example they could use a base map and
share information they have. Some information they want to collect and
give out to public is perfectly fit for OSM (say max speeds) while
others are purely their thing which should not go to main OSM
database (say information like which community a specific house
belongs to). (So they want to share all information with public, but
some information should be captured on a separate storage/db)

  They would like to have a separate layer to hold such non-OSM
information. This would keep OSM database clean and it would allow
government to take responsibility for quality assurance and updates of
that data.

  How should this be implemented? Separate osm database (with data of
those additional layers only) and then different editors could be used
to fill that database by pointing to api url of this extra server? If
I need more layers, more databases have to be created etc.?
(presentation part of mapnik/openlayers is clear)

  I was looking for information about this on osm wiki but didn't find
anything useful so maybe you can give me some urls/keywords where I
can read about it?

  Thank you

-- 
Tomas Straupis

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Info request on: additional GIS layers

2012-03-06 Thread Tomas Straupis
Hello

2012-03-07 Andre Joost wrote:
 This is a point which breaks the idea:

 If *they* want to be responsible for the data and making updates, no other
 user should have access to it.

  True. But this would be PART of the data they provide. And that part
would barely be interesting to the rest of OSM community (both in
terms of filling/taking care of the data and using it). To rephrase
this: they do not want to HIDE some data (they are actually happy with
adding that data directly to OSM), just some data is NOT interesting
to OSM (f.e. land ownership, self-government community information,
problems with infrastructure, illegal structures etc.).

 So the only way is to add a separate layer with openlayers. Or use a WMS
 layer with OSM data in the governmental Web-GIS.

  Point is to move government GIS to OSM software stack. They would
then partly use OSM main db (where data is usable for OSM) and for
other specific data they could use their own server (DB). This would
be kind of GIS layering.
  That is software/skills required would be the same (one stack, no
zoo). People would have to learn to use osm and then there would
simply be more than one osm :-)

  So both sides win:
  * OSM gets some additional data source (and more publicity)
  * government gets free software stack, provides better/more services
to public etc.

  The part of creating separate layers (say with mapnik) and
presenting them (say with openlayers) is clear. Uncertain part (for
me) is just the database part because I only have a general
understanding of what would be required (I guess database and
something which gives API to access that database from JOSM/Mapnik
etc.).

  Therefore my question is: where can I get more information about
creating own database/api and setting up all infrastructure around
it (f.e. authorisation, editors etc.). Or if there is any other
approach of achieving this goal of GIS layers?

  Thank you

-- 
Tomas Straupis

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev