Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I've just been experimenting and looking at shapes and rounding-to-low
-resolution can easily cause self-intersections.

It isn't just a problem with sea, or even with multipolygon hole
cutting. GPSMapEdit reports over 1000 in my test area. This number
reduces only slightly when I disable DP for polygons. Generally these
are not noticeable - a little bit of wood missing when zoomed out isn't
obvious, but same is not true of sea.

Ticker

On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> no need to guess, I use GpxCreator in various places to visualize
> input and output of filters. I load the gpx files into JOSM and maybe
> convert to Data layer to understand what happens, e.g. where the
> start /end point is.
> Reg. self-intersection:
> Think of a merged sea shape with several islands in it. The rounding
> may result in only three or four different nodes, visitied in a more
> or less random order while the correct shape would be a triangle or
> rectangle.
> Not sure if this is only a problem with sea, probably not.
> 
> Feel free to try other orders of the filters, or maybe different
> algos to set the preserved flag. There is definitely room for
> improvements. I've played with this very often and sometimes one area
> is improved and another is worsened.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Divide number in style file

[Ticker Berkin]

Hi Felix

I don't think there is any way to do maths on tag values.

It should not be too difficult to implement additional variable filters
that do a simple maths op with a constant, eg:
  {set population='${population|divide:"7"}'}

Ticker

On Wed, 2021-05-05 at 22:58 +0800, Felix Hartmann wrote:
> Is it possible to divide a number by a factor in the style file?
> 
> ( mkgmap:country=CHN | mkmgap:countrz=CN | mkgmap:country=IND |

 ^^
> mkgmap:country=IN ) & ( place=city | place=island | place=town |
> place=village ) & population=* {set population='${population}/7'}
> 
> Did not work. I intend to divide the population in cities in China
> and India because otherwise cities clutter up the map there (also in
> India they do not use admin level ... which could be an alternative).
> 
> -- 
> Felix Hartman - Openmtbmap.org & VeloMap.org
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Divide number in style file

[Ticker Berkin]

...|conv:"from=>to" does speed and weight conversions as well as length

This is the table (from mkgmap/osmstyle/eval/UnitConversions.java)

  static {
Map m = LENGTH_FACTORS;
m.put("m", 1.0);
m.put("km", 1000.0);
m.put("ft", 0.3048);
m.put("feet", 0.3048);
m.put("mi", 1_609.344);
CONVERSIONS.put(UnitType.LENGTH, LENGTH_FACTORS);

m = SPEED_FACTORS;
m.put("kmh", 1.0);
m.put("km/h", 1.0);
m.put("kmph", 1.0);
m.put("mph", 1.60934);
m.put("knots", 1.852);
CONVERSIONS.put(UnitType.SPEED, SPEED_FACTORS);

m = WEIGHT_FACTORS;
m.put("t", 1.0);
m.put("kg", 0.001);
m.put("lb", 0.00045359237);
m.put("lbs", 0.00045359237);
CONVERSIONS.put(UnitType.WEIGHT, WEIGHT_FACTORS);
}

I see no reason why chaining conv| shouldn't work, but haven't tried
it.

Ticker

On Fri, 2021-05-07 at 17:56 +0800, Felix Hartmann wrote:
> yes I figured out the only current possible division would be to
> convert ft to mt - that could be run against population number as
> well of course.. Dividing the population number to get maps of India
> or China a bit less cluttered without using a specific style would be
> quite handy. Though yes - besides improving rules for cities not much
> else use of a division factor for values that I can think of. So
> right now only 3.28, 3.28^2, 3.28^3 and so on is possible (i did not
> actually check if it is possible to convert twice, but think so).
> 
> On Fri, 7 May 2021 at 16:56, Ticker Berkin 
> wrote:
> > Hi Felix
> > 
> > I don't think there is any way to do maths on tag values.
> > 
> > It should not be too difficult to implement additional variable
> > filters
> > that do a simple maths op with a constant, eg:
> >   {set population='${population|divide:"7"}'}
> > 
> > Ticker
> > 
> > On Wed, 2021-05-05 at 22:58 +0800, Felix Hartmann wrote:
> > > Is it possible to divide a number by a factor in the style file?
> > > 
> > > ( mkgmap:country=CHN | mkmgap:countrz=CN | mkgmap:country=IND |
> > 
> >  ^^
> > > mkgmap:country=IN ) & ( place=city | place=island | place=town |
> > > place=village ) & population=* {set population='${population}/7'}
> > > 
> > > Did not work. I intend to divide the population in cities in
> > China
> > > and India because otherwise cities clutter up the map there (also
> > in
> > > India they do not use admin level ... which could be an
> > alternative).
> > > 
> > > -- 
> > > Felix Hartman - Openmtbmap.org & VeloMap.org
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> 
> -- 
> Felix Hartman - Openmtbmap.org & VeloMap.org
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Duplicate roads added by default style

[Ticker Berkin]

Hi Gerd

This is incorrect tagging. highway=platform should be used just for bus
or tram, not for railway.

However, no problem with your change to the default style.

I don't think it is worthwhile extending de-duplication to other
elements; just the strange cases that road-merger throws up.

Ticker


On Mon, 2021-05-10 at 10:04 +, Gerd Petermann wrote:
> Hi all,
> 
> in my area a lot platforms have highway=platform and
> railway=platform, and the default  style adds a routable way for
> both.
> 
> The default style could be fixed with
> railway=platform & tmp:stopMopUp!=yes [0x16 road_class=0 road_speed=0
> resolution 23]
> instead of
> railway=platform [0x16 road_class=0 road_speed=0 resolution 23]
> 
> The code to find duplicate roads for the same OSM way is rather
> simple. I could add more code to find this issue for other objects
> (points, shapes, other lines) as well but I guess roads are the most
> important?
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 10. Mai 2021 11:51
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Duplicate roads added by default style
> 
> Hi all
> 
> Maybe railway=platform shouldn't generate a routable line.
> 
> Pure duplication of (parts of) roads with identical Garmin img
> attributes could be suppressed. These could come from duplication by
> the style or distinct OSM ways. In the case of platform=railway, I'd
> have thought it to be better mapping to have a distinct OSM way when
> a
> navigable route uses a platform.
> 
> I don't have strong feelings about warnings or not for this case.
> 
> Ticker
> 
> On Mon, 2021-05-10 at 09:21 +, Gerd Petermann wrote:
> > Hi Felix,
> > 
> > sure, I meant roads which RoadMerger would combine. Roadmerger
> > checks
> > a lot more attributes.
> > Multiple (different) routable ways for one OSM way are quite
> > normal,
> > esp. with the --make-opposite-cycleways option.
> > 
> > I think it's worth to log a warning for each.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Felix Hartmann 
> > Gesendet: Montag, 10. Mai 2021 11:16
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Duplicate roads added by default style
> > 
> > It makes the style very complicated to try to remove any identical
> > copies - I do this by setting loads of temporary keys, that are
> > checked later for existing or not existing and based on that add or
> > not add other lines.  I guess some actual duplicates will still
> > happen.
> > 
> > On Mon, 10 May 2021 at 17:14, Felix Hartmann <
> > extremecar...@gmail.com
> > <mailto:extremecar...@gmail.com>> wrote:
> > hmm no, please no warning. This can be very beneficial. As long as
> > the type OR road_speed OR road_class OR oneway plus direction is
> > different. The above of course makes no sense, as it will set
> > exactly
> > the same road twice.
> > 
> > On Mon, 10 May 2021 at 16:49, Gerd Petermann <
> > gpetermann_muenc...@hotmail.com > gpetermann_muenc...@hotmail.com>> wrote:
> > Hi all,
> > 
> > while double checking patch for the RoadMerger I wondered what
> > might
> > happen if the style adds two or more identical roads for the same
> > OSM
> > way.
> > Of course this makes no sense but to my surprise I found out that
> > this really happens with the default style.
> > A way with e.g. highway=footway and railway=platform is added twice
> > ...
> > highway=footway {set tmp:stopMopUp=yes} [0x16 road_class=0
> > road_speed=0 resolution 23 continue with_actions]
> > ...
> > railway=platform [0x16 road_class=0 road_speed=0 resolution 23]
> > 
> > Of course the default style should be fixed to avoid that but I
> > think
> > mkgmap can detect this easily and should print a warning when the
> > style does that?
> > 
> > Gerd
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk > mkgmap-dev@lists.mkgmap.org.uk>
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > 
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> > 
> > 
> > 
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

Here is the change that always splits polygons into subdivisions by
dividing along the internal boundaries rather than clipping. It is more
efficient and the empty rectangle problem goes away.

Ticker

On Wed, 2021-04-28 at 12:41 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> okay, so I'll wait for your results. The v1 patch always produces
> larger files with normal maps without visual effects, so it seems to
> split more often than needed (or prohibit merging too often)
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 28. April 2021 14:26
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Empty rectangles in map
> 
> Hi Gerd
> 
> Here is patch configured for Method 1, with more clarity about how to
> change.
> 
> I think the best solution will be a super Method 4. For a 5 x 4
> subdivision grid it will reduces the number of polygon cuts from 80
> (5*4*4) to 19 (5-1+5*(4-1)).
> 
> It shouldn't be too difficult to implement and I'll look at it next
> week.
> 
> Ticker
> 
> On Wed, 2021-04-28 at 09:14 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > please create two separate patches for method 1 and 2. It's not
> > obvious to me what changes are needed to get method 1.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 27. April 2021 13:11
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Empty rectangles in map
> > 
> > Hi Gerd and others
> > 
> > The cause for this unrendered area is as follows:
> > 
> > A large multipolygon (larger than the maximum size for a level 1
> > subdiv) exists, and is broken into some similar size pieces and
> > some
> > smaller pieces to expose inners/holes. The same problem can be
> > caused
> > by any polygons having these size properties.
> > 
> > Option --order-by-decreasing-area is not in effect (with this
> > option
> > the problem goes away).
> > 
> > At the outermost level, the large pieces are split into the grid of
> > subdivs necessary to be within size limits. Small pieces that fit
> > within a subdiv are allocated to the correct one. Intermediate
> > pieces
> > that extend out of the nominal subdiv bounds, but don't exceed the
> > actual addressing limits, are allocated to an existing subdiv based
> > on
> > their centre. Larger pieces that fit in a subdiv but would cause
> > size
> > problems if added to an existing subdiv are given a subdiv of their
> > own.
> > 
> > At the next level down, each of the above subdivs will required
> > splitting to fit within the addressing limits. The logic that did
> > this
> > used the subdiv bounds and split this into a grid. Then every
> > element
> > in the outer MapArea is processed as above (ie split, safely
> > placed,
> > placed with overlap or own subdiv)
> > 
> > The problem is that the splitting algorithm simply goes through the
> > child subdiv, clips the polygon to this size and saves the bits
> > that
> > are within the area. Polygon parts that are outside the defined
> > outer
> > level never get noticed!
> > 
> > There are various possible solutions:
> > 
> > Method 1: use the rigorous polygon splitting into subdivs that -
> > -order
> > -by-decreasing-area uses for levels above level 0. This is simple
> > to
> > implement, safe, mergeShapes might recombine more cut fragments as
> > they
> > are all together. Disadvantages are that, for a normal map, more
> > polygons might be split, leading to larger RGN size (but not for
> > bad
> > -map-split.osm.pbf). The overall size is still less that with -
> > -order
> > -by...
> > 
> > Method 2: When the child subdiv grid is created, use any expanded
> > size
> > of the parent subdiv. This stops overhanging bits of polygons being
> > lost. Again, simple to implement. It results in overlapping subdivs
> > at
> > lower levels and I don't know this causes other problems, but
> > probably
> > not; this used to be done and still does for some sizes of polygons
> > and
> > lines. Some shape-merging won't happen because the joining bits are
> > in
> > a different subdiv.
> > 
> > Method 3: Allocated oversize polygons to their own subdiv. Again,
> > simple to implement, but I see no advantage over Method 2. Even
> > more
> > shape-merging might n

Re: [mkgmap-dev] patch: improve RoadMerger: allow to merge roads with reversed points

[Ticker Berkin]

Hi

Is it a particular problem to never merge ferry lines? If derived from
a relation, what elements is the relation using? Probably these
shouldn't be merged either.

Ticker


On Mon, 2021-05-10 at 09:57 +, Gerd Petermann wrote:
> Hi all,
> 
> found another problem: In what situation should RoadMerger merge
> ferry routes?
> See e.g. https://www.openstreetmap.org/node/1302656090
> 
> I first thought that they should never be merged but we also have
> cases where ferry lines are mapped with relations.
> 
> Maybe we need special code to handle route=ferry relations to combine
> ways and then forbid to merge in RoadMerger?
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Duplicate roads added by default style

[Ticker Berkin]

Hi all

Maybe railway=platform shouldn't generate a routable line.

Pure duplication of (parts of) roads with identical Garmin img
attributes could be suppressed. These could come from duplication by
the style or distinct OSM ways. In the case of platform=railway, I'd
have thought it to be better mapping to have a distinct OSM way when a
navigable route uses a platform.

I don't have strong feelings about warnings or not for this case.

Ticker

On Mon, 2021-05-10 at 09:21 +, Gerd Petermann wrote:
> Hi Felix,
> 
> sure, I meant roads which RoadMerger would combine. Roadmerger checks
> a lot more attributes.
> Multiple (different) routable ways for one OSM way are quite normal,
> esp. with the --make-opposite-cycleways option.
> 
> I think it's worth to log a warning for each.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Felix Hartmann 
> Gesendet: Montag, 10. Mai 2021 11:16
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Duplicate roads added by default style
> 
> It makes the style very complicated to try to remove any identical
> copies - I do this by setting loads of temporary keys, that are
> checked later for existing or not existing and based on that add or
> not add other lines.  I guess some actual duplicates will still
> happen.
> 
> On Mon, 10 May 2021 at 17:14, Felix Hartmann  > wrote:
> hmm no, please no warning. This can be very beneficial. As long as
> the type OR road_speed OR road_class OR oneway plus direction is
> different. The above of course makes no sense, as it will set exactly
> the same road twice.
> 
> On Mon, 10 May 2021 at 16:49, Gerd Petermann <
> gpetermann_muenc...@hotmail.com gpetermann_muenc...@hotmail.com>> wrote:
> Hi all,
> 
> while double checking patch for the RoadMerger I wondered what might
> happen if the style adds two or more identical roads for the same OSM
> way.
> Of course this makes no sense but to my surprise I found out that
> this really happens with the default style.
> A way with e.g. highway=footway and railway=platform is added twice
> ...
> highway=footway {set tmp:stopMopUp=yes} [0x16 road_class=0
> road_speed=0 resolution 23 continue with_actions]
> ...
> railway=platform [0x16 road_class=0 road_speed=0 resolution 23]
> 
> Of course the default style should be fixed to avoid that but I think
> mkgmap can detect this easily and should print a warning when the
> style does that?
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
> 
> 
> 
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] patch: improve RoadMerger: allow to merge roads with reversed points

[Ticker Berkin]

Hi Gerd

Maybe I'm not understanding. Apart from suppressing merging when
mkgmap:ferry, does any else need to be done?

Ticker


On Mon, 2021-05-10 at 13:22 +, Gerd Petermann wrote:
> reg. route=ferry relations:
> I fear this will be a bit more complex. Mappers seem to put all kind
> of things in those relations: piers, rivers, highways, you name it.
> Test file is here:
> https://files.mkgmap.org.uk/download/506/ferry_relations.osm.pbf
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Montag, 10. Mai 2021 12:23
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] patch: improve RoadMerger: allow to merge
> roads with reversed points
> 
> Hi Ticker,
> 
> yes, I meant: Create a new way for the combination of ways that make
> up one ferry route in a relation but never merge them in RoadMerger.
> Haven't looked at the details yet. There were some special cases in
> British Columbia with this.
> 
> Gerd
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 10. Mai 2021 12:16
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] patch: improve RoadMerger: allow to merge
> roads with reversed points
> 
> Hi
> 
> Is it a particular problem to never merge ferry lines? If derived
> from
> a relation, what elements is the relation using? Probably these
> shouldn't be merged either.
> 
> Ticker
> 
> 
> On Mon, 2021-05-10 at 09:57 +, Gerd Petermann wrote:
> > Hi all,
> > 
> > found another problem: In what situation should RoadMerger merge
> > ferry routes?
> > See e.g. https://www.openstreetmap.org/node/1302656090
> > 
> > I first thought that they should never be merged but we also have
> > cases where ferry lines are mapped with relations.
> > 
> > Maybe we need special code to handle route=ferry relations to
> > combine
> > ways and then forbid to merge in RoadMerger?
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Line direction flag, merging etc

[Ticker Berkin]

Hi Gerd

The ambition was to be able to turn off hasDirection if [oneway=yes/-1]
(roads only) and/or --lines-types-with-direction would have it on.

As I'd never noticed the compass direction on oneway roads, I thought I
might as well turn it off. It also allowed me to test if this
combination causes other problems. When LineMergeFilter does reversals
it might be able to merge more.
 
This might help some of Felix's problems - has-direction can be turned
off for everything - towards the end of lines: 


...
oneway=* {set mkgmap:has-direction=no}

I presume r4719 was ignore-oneway-for-lines.patch and I took elements
from this, but redid the oneway/road handling slightly so it was all in
the same place.

I've updated the patch to include the unit test change.

Ticker

On Mon, 2021-05-17 at 15:38 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> not sure what you want to achive?
> I agree that the evaluation of the --line-types-with-direction option
> should not depend on the --allow-reverse-merge option.
> 
> Seems like an alternative to the solution in r4719 : 
> https://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4719
> but without the correction for the unit test.
> Reg. overlays:
> Yes, first element just gets a different type, the others are forced
> to be MapLine. First element might also be a non-routable line.
> The OpenFietsmap Styles use them, see 
> https://github.com/ligfietser/mkgmap-style-sheets
> 
> Do you see a need for the possibility to have oneway roads which
> don't have the direction flag set?
> In trunk it will not matter reg. merging, in the branch it might
> allow more merging in the overview map.
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 17. Mai 2021 17:06
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Line direction flag, merging etc
> 
> Hi Gerd
> 
> I've rearranged the direction flag setting logic a bit, taking bits
> from ignore-oneway-for-line.patch
> 
> It ignores oneway except for roads and allows direction to be set
> false
> even for routable oneways. I've tested routing when oneway=true,
> hasDirection=false on eTrex HCx and gpsMapEdit and it respected the
> oneway.
> 
> I couldn't find any references to the {style}/overlays file in any
> documentation, but my reading of the code is that first substitution
> element will get the full MapRoad info, including the hasDirection
> flag
> and the MapLine copies will get also get the hasDirection flag.
> 
> Ticker
> 
> On Mon, 2021-05-17 at 11:29 +0100, Ticker Berkin wrote:
> > Hi Gerd and others
> > 
> > I'm starting a dedicated thread as previous posts are split between
> > various svn commits.
> > 
> > Looking at the current trunk code and ignore-oneway-for
> > -lines.patch,
> > there are bits I don't understand or didn't expect.
> > 
> > This is what I hope for:
> > 
> > --line-types-with-direction lists the lineTypes for which the
> > initial
> > state of the hasDirection flag is considered true. It should be
> > independent of --allow-reverse-merge (see later). The purpose of
> > this
> > is to reduce the need to edit styles and clarity as to which
> > lineTypes
> > are displayed with direction.
> > 
> > In the following, road means line with [road_class, road_speed], so
> > includes ferries.
> > 
> > Style processing, having produced a line/road, sets hasDirection,
> > in
> > order of precedence:
> > 1/ value of mkgmap:has-direction if tag exists
> > 2/ true if oneway road
> > 3/ true if in line-types-with-direction list
> > 
> > RoadMerger merges roads with same lineType, names, class, speed,
> > oneway
> > and hasDirection. It must not reverse oneway or hasDirection=true
> > roads
> > when doing this.
> > 
> > I don't think we need --allow-reverse-merge.
> > 
> > LineMergeFilter operates for level > 0. It merges any line/road
> > with
> > the same lineType, names, hasDirection. It must not reverse
> > hasDirection=true.
> > 
> > Ticker
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/mkgmap/osmstyle/StyledConverter.java
===
--- src/uk/me/parabola/mkgmap/osmstyle/StyledCon

[mkgmap-dev] Line direction flag, merging etc

[Ticker Berkin]

Hi Gerd and others

I'm starting a dedicated thread as previous posts are split between
various svn commits.

Looking at the current trunk code and ignore-oneway-for-lines.patch,
there are bits I don't understand or didn't expect.

This is what I hope for:

--line-types-with-direction lists the lineTypes for which the initial
state of the hasDirection flag is considered true. It should be
independent of --allow-reverse-merge (see later). The purpose of this
is to reduce the need to edit styles and clarity as to which lineTypes
are displayed with direction. 

In the following, road means line with [road_class, road_speed], so
includes ferries.

Style processing, having produced a line/road, sets hasDirection, in
order of precedence:
1/ value of mkgmap:has-direction if tag exists
2/ true if oneway road
3/ true if in line-types-with-direction list

RoadMerger merges roads with same lineType, names, class, speed, oneway
and hasDirection. It must not reverse oneway or hasDirection=true roads
when doing this.

I don't think we need --allow-reverse-merge.

LineMergeFilter operates for level > 0. It merges any line/road with
the same lineType, names, hasDirection. It must not reverse
hasDirection=true.

Ticker

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Line direction flag, merging etc

[Ticker Berkin]

Considering the question of hasDirection and resolution.

If the same lineType looks identical, regardless of hasDirection, then
there is no point in setting hasDirection=true as the only effect will
be to inhibit merging.

To get the look of direction at high-res and no-direction when zoomed
out, different lineTypes have to be used, with the style controlling
this with [resolution-range] and [continue]

If the lineType representation can be changed with hasDirection, again
elements for the different levels should be produced, but can have 
 {set mkgmap:has-direction=yes} for the high-res and =no for the low
-res

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Line direction flag, merging etc

[Ticker Berkin]

Hi Gerd

I've rearranged the direction flag setting logic a bit, taking bits
from ignore-oneway-for-line.patch

It ignores oneway except for roads and allows direction to be set false
even for routable oneways. I've tested routing when oneway=true,
hasDirection=false on eTrex HCx and gpsMapEdit and it respected the
oneway.

I couldn't find any references to the {style}/overlays file in any
documentation, but my reading of the code is that first substitution
element will get the full MapRoad info, including the hasDirection flag
and the MapLine copies will get also get the hasDirection flag.

Ticker

On Mon, 2021-05-17 at 11:29 +0100, Ticker Berkin wrote:
> Hi Gerd and others
> 
> I'm starting a dedicated thread as previous posts are split between
> various svn commits.
> 
> Looking at the current trunk code and ignore-oneway-for-lines.patch,
> there are bits I don't understand or didn't expect.
> 
> This is what I hope for:
> 
> --line-types-with-direction lists the lineTypes for which the initial
> state of the hasDirection flag is considered true. It should be
> independent of --allow-reverse-merge (see later). The purpose of this
> is to reduce the need to edit styles and clarity as to which
> lineTypes
> are displayed with direction. 
> 
> In the following, road means line with [road_class, road_speed], so
> includes ferries.
> 
> Style processing, having produced a line/road, sets hasDirection, in
> order of precedence:
> 1/ value of mkgmap:has-direction if tag exists
> 2/ true if oneway road
> 3/ true if in line-types-with-direction list
> 
> RoadMerger merges roads with same lineType, names, class, speed,
> oneway
> and hasDirection. It must not reverse oneway or hasDirection=true
> roads
> when doing this.
> 
> I don't think we need --allow-reverse-merge.
> 
> LineMergeFilter operates for level > 0. It merges any line/road with
> the same lineType, names, hasDirection. It must not reverse
> hasDirection=true.
> 
> Ticker
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/mkgmap/osmstyle/StyledConverter.java
===
--- src/uk/me/parabola/mkgmap/osmstyle/StyledConverter.java	(revision 4720)
+++ src/uk/me/parabola/mkgmap/osmstyle/StyledConverter.java	(working copy)
@@ -234,16 +234,14 @@
 		// undocumented option - usually used for debugging only
 		mergeRoads = !props.getProperty("no-mergeroads", false);
 		allowReverseMerge = props.getProperty("allow-reverse-merge", false);
-		if (allowReverseMerge) {
-			final String typesOption = "line-types-with-direction";
-			String typeList = props.getProperty(typesOption, "");
-			if (typeList.isEmpty())
-typeList = style.getOption(typesOption);
-			List types = CommandArgs.stringToList(typeList, typesOption);
-			for (String type :types) {
-if (!type.isEmpty()) {
-	lineTypesWithDirection.add(Integer.decode(type));
-}
+		final String typesOption = "line-types-with-direction";
+		String typeList = props.getProperty(typesOption, "");
+		if (typeList.isEmpty())
+			typeList = style.getOption(typesOption);
+		List types = CommandArgs.stringToList(typeList, typesOption);
+		for (String type :types) {
+			if (!type.isEmpty()) {
+lineTypesWithDirection.add(Integer.decode(type));
 			}
 		}
 		routable = props.containsKey("route");
@@ -342,6 +340,7 @@
 			}
 			
 			boolean wasReversed = false;
+			boolean hasDirection = false;
 			String oneWay = way.getTag(TK_ONEWAY);
 			if (oneWay != null){
 if("-1".equals(oneWay)) {
@@ -351,11 +350,13 @@
 	way.reverse();
 	wasReversed = true;
 	way.addTag(TK_ONEWAY, "yes");
+	hasDirection = true; // will get ignored if not road
 }
 if (way.tagIsLikeYes(TK_ONEWAY)) {
 	way.addTag(TK_ONEWAY, "yes");
 	if (foundType.isRoad() && hasSkipDeadEndCheckNode(way))
 		way.addTag("mkgmap:dead-end-check", "false");
+	hasDirection = true; // will get ignored if not road
 } else { 
 	way.deleteTag(TK_ONEWAY);
 }
@@ -363,13 +364,17 @@
 			ConvertedWay cw = new ConvertedWay(way, foundType);
 			cw.setReversed(wasReversed);
 
-			if (cw.isOneway() || way.tagIsLikeYes(TKM_HAS_DIRECTION)
-	|| lineTypesWithDirection.contains(foundType.getType()))
-cw.setHasDirection(true);
-			if (way.tagIsLikeNo(TKM_HAS_DIRECTION)) 
-cw.setHasDirection(false);
-			
-			if (cw.isRoad()){
+			if (way.tagIsLikeYes(TKM_HAS_DIRECTION))
+hasDirection = true;
+			else if (way.tagIsLikeNo(TKM_HAS_DIRECTION))
+hasDirection = false;
+			else if (lineTypesWithDirection.contains(foundType.getType()))
+hasDirection = tru

Re: [mkgmap-dev] Line direction flag, merging etc

[Ticker Berkin]

Hi Gerd

Sorry - in my mind, old versions also tried reversing if allowed.

Ticker

 
On Mon, 2021-05-17 at 11:13 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I do see a need for the --allow-reverse-merge option, at least for
> those styles which rely on the old behaviour that lines are not
> reversed unless oneway=-1 is found.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Hi all

I think lines with and without direction shouldn't be merged. The only
reason, as far as I see, for having direction on lines is that they
have a different visual representation.

Ticker

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Hi Gerd

Probably after:



oneway=* {set mkgmap:has-direction=no}

Ticker


On Sat, 2021-05-15 at 18:18 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> that means that styles which don't render direction have to be
> changed to get the smallest IMG files. Where would you add the
> mkgmap:has-direction=no in the default style?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Samstag, 15. Mai 2021 20:08
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for reverse
> merge
> 
> Hi
> 
> Probably the Garmin meaning has changed over time and it is now a
> display characteristic on some devices.
> 
> oneway should continue to set it. {set mkgmap:has-direction=no}
> should
> end up with it not set (but correct routing).
> 
> My eTrex has always shown this (looking back at an old map). I'd just
> never hovered over a one-way road and noticed. Maybe I'd seen
> something
> like "M3 (N)" and just assumed the carriageway was named as such.
> 
> It is a bit of a rubbish way of showing one-way compared with putting
> an arrow on the road.
> 
> Ticker
> 
> On Sat, 2021-05-15 at 17:24 +, Gerd Petermann wrote:
> > Hi all,
> > 
> > what about routable lines with oneway? Should this force the
> > direction flag in IMG format? I found no oneway road in Garmin maps
> > where the dir-flag was not set, but Tickers finding reg. the Etrex
> > HCx: showing compass directions for those roads seem to say that
> > there is no clear rule about the meaning of this flag.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Gerd Petermann 
> > Gesendet: Samstag, 15. Mai 2021 18:30
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for
> > reverse
> > merge
> > 
> > Hi all,
> > 
> > OK, I'll try to change the code so that oneway=* has no effect on
> > the
> > dir-flag for non-routable lines. It's a bit tricky because of the
> > overlays rule file.
> > At least the OFM style still uses this and I forgot what it is good
> > for.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Samstag, 15. Mai 2021 17:38
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for
> > reverse
> > merge
> > 
> > I agree. The style can test "oneway" and set mkgmap:has-direction
> > if
> > appropriate for non-roads.
> > 
> > Ticker
> > 
> > On Sat, 2021-05-15 at 14:53 +, Gerd Petermann wrote:
> > > I think the important question is if the oneway tag should be
> > > evaluated for non-routable lines.
> > > Felix already said no, I agree with that.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Samstag, 15. Mai 2021 16:04
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for
> > > reverse
> > > merge
> > > 
> > > Assuming that the output device shows the direction on lines with
> > > direction and doesn't on lines without:
> > > 
> > > Another example is, say, [intermittent] water using type [0x26]
> > > 0x1f.
> > > stream/river will have a direction and canal/ditch won't.
> > > 
> > > There will be others, under the control of the Style/TYP, so I
> > > don't
> > > think mkgmap should check/warn of this situation.
> > > 
> > > Ticker
> > > 
> > > On Sat, 2021-05-15 at 12:59 +, Gerd Petermann wrote:
> > > > I only thought about non-routable lines and LineMerger. For
> > > > roads
> > > > it
> > > > is normal to have different flags because of the oneway tag.
> > > > Maybe I make too many assumptions about the possible usage of
> > > > types...
> > > > 
> > > > Gerd
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > ___
> > > mkgmap-dev mailing list
> > > mkg

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Hi

Probably the Garmin meaning has changed over time and it is now a
display characteristic on some devices.

oneway should continue to set it. {set mkgmap:has-direction=no} should
end up with it not set (but correct routing).

My eTrex has always shown this (looking back at an old map). I'd just
never hovered over a one-way road and noticed. Maybe I'd seen something
like "M3 (N)" and just assumed the carriageway was named as such.

It is a bit of a rubbish way of showing one-way compared with putting
an arrow on the road.

Ticker

On Sat, 2021-05-15 at 17:24 +, Gerd Petermann wrote:
> Hi all,
> 
> what about routable lines with oneway? Should this force the
> direction flag in IMG format? I found no oneway road in Garmin maps
> where the dir-flag was not set, but Tickers finding reg. the Etrex
> HCx: showing compass directions for those roads seem to say that
> there is no clear rule about the meaning of this flag.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Samstag, 15. Mai 2021 18:30
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for reverse
> merge
> 
> Hi all,
> 
> OK, I'll try to change the code so that oneway=* has no effect on the
> dir-flag for non-routable lines. It's a bit tricky because of the
> overlays rule file.
> At least the OFM style still uses this and I forgot what it is good
> for.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Samstag, 15. Mai 2021 17:38
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for reverse
> merge
> 
> I agree. The style can test "oneway" and set mkgmap:has-direction if
> appropriate for non-roads.
> 
> Ticker
> 
> On Sat, 2021-05-15 at 14:53 +, Gerd Petermann wrote:
> > I think the important question is if the oneway tag should be
> > evaluated for non-routable lines.
> > Felix already said no, I agree with that.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Samstag, 15. Mai 2021 16:04
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for
> > reverse
> > merge
> > 
> > Assuming that the output device shows the direction on lines with
> > direction and doesn't on lines without:
> > 
> > Another example is, say, [intermittent] water using type [0x26]
> > 0x1f.
> > stream/river will have a direction and canal/ditch won't.
> > 
> > There will be others, under the control of the Style/TYP, so I
> > don't
> > think mkgmap should check/warn of this situation.
> > 
> > Ticker
> > 
> > On Sat, 2021-05-15 at 12:59 +, Gerd Petermann wrote:
> > > I only thought about non-routable lines and LineMerger. For roads
> > > it
> > > is normal to have different flags because of the oneway tag.
> > > Maybe I make too many assumptions about the possible usage of
> > > types...
> > > 
> > > Gerd
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Hi all

I've been testing the effect of the DIR flag (0x40) in various ways:

GPSMapEdit: shows a direction arrow on all standard line types. It
appears not to read this flag for extended line types, but, if set
manually, show an arrow on these as well.

eTrex HCx: For all line types, including extended, when hovering over
the line, appends " (DIR)" to the name, where DIR is N, NE, E etc. It
does this regardless of any TYP representation of the line. eg
"Newburgh Street (W)".

eTrex 30x: Like above, but I don't think it shows (DIR) for extended
types.

Basecamp & Mapsource: Having the flag set makes no visible difference.
There seems to be no way of interrogating road/line properties.

I've not found any ill-effects of setting the DIR flag for extended
types.

I'm currently messing with TYP files and code to see if there is some
combination where Garmin device/program will pick up a different
[_line] definition if the dir flag is set.

Ticker



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

I agree. The style can test "oneway" and set mkgmap:has-direction if
appropriate for non-roads.

Ticker
 
On Sat, 2021-05-15 at 14:53 +, Gerd Petermann wrote:
> I think the important question is if the oneway tag should be
> evaluated for non-routable lines.
> Felix already said no, I agree with that.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Samstag, 15. Mai 2021 16:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for reverse
> merge
> 
> Assuming that the output device shows the direction on lines with
> direction and doesn't on lines without:
> 
> Another example is, say, [intermittent] water using type [0x26] 0x1f.
> stream/river will have a direction and canal/ditch won't.
> 
> There will be others, under the control of the Style/TYP, so I don't
> think mkgmap should check/warn of this situation.
> 
> Ticker
> 
> On Sat, 2021-05-15 at 12:59 +, Gerd Petermann wrote:
> > I only thought about non-routable lines and LineMerger. For roads
> > it
> > is normal to have different flags because of the oneway tag.
> > Maybe I make too many assumptions about the possible usage of
> > types...
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Assuming that the output device shows the direction on lines with
direction and doesn't on lines without:

Another example is, say, [intermittent] water using type [0x26] 0x1f.
stream/river will have a direction and canal/ditch won't.

There will be others, under the control of the Style/TYP, so I don't
think mkgmap should check/warn of this situation.

Ticker

On Sat, 2021-05-15 at 12:59 +, Gerd Petermann wrote:
> I only thought about non-routable lines and LineMerger. For roads it
> is normal to have different flags because of the oneway tag.
> Maybe I make too many assumptions about the possible usage of
> types...
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] MDR building out-of-memory

[Ticker Berkin]

Hi Gerd

I've looked at this, and if Mdr5 space becomes a problem again, I'll
consider converting to it.

A couple of comments:

My map had 2909735 poi so the sort chunk size was ~727433. The cache
sizes after each chunk were 563095, 603595, 597239 & 605718. Is the
cache worth-while for this low hit-rate? Just running the Gmapsupp
combiner on existing tiles (without --route, so no streets), I got a
run time of 1 min 44 secs with the cache and 1 min 30 without! However
most of this time is copying the tiles into gmapsupp.img so not an
accurate statistic.

You could pre-allocate List<> "merged" with the correct size.

Ticker

On Tue, 2021-05-11 at 14:40 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I've committed the patch as is. I've not seen big changes in
> performance, but I've used a different (already existing) set of
> files which was created with my own style. For me,
> Mdr11.preWriteImpl() is the most problematic part reg. OOM errors.
> 
> Maybe look at the code which uses LargeListSorter.
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 11. Mai 2021 13:27
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] MDR building out-of-memory
> 
> Hi Gerd
> 
> Here is updated version of patch.
> 
> Changes from last:
> 
> Uses your cache code for region and country (in 2 places). For
> British
> Isles, there are 190 regions and 7 countries, so I don't think the
> extra memory will be a problem and there should be some performance
> benefit.
> 
> Delays allocating cities until it can use sortKeys.size() for initial
> allocation. For above map this is 0.07% too big, so I don't think
> trimToSize() is worthwhile.
> 
> Shares the Sort object between the 4 methods.
> 
> Ticker
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] MDR building out-of-memory

[Ticker Berkin]

Hi Gerd

I did a bit more experimenting with LargeListSorter and Mdr7/11 and
have some changes - patch attached.

1/ explicit param to LargeListSorted to useCache
2/ do nothing if 0 or 1 records to sort
3/ reduce chunkSize to reduce memory usage, increase maxDepth so same
overall limit
4/ allocate temp "merged" with correct length
5/ useCache parameters and remarks on usage in Mdr7 & 11
6/ prevent a copy of allStreets
7/ share the collator and allow it and "sort" to be garbage collected
8/ A comments that sort of partial streets must use sortkey, even if
only sorting a few record. LargeListSorter is a handy way of doing this
for a simple/single sortKey per record.

Sortkey simply makes a byte array of the string converted into the
target charset and sort does a byte comparison on this.

collator.compare(str1, str2) ignores the shield chars (and possibly the
other prefix/suffix markers) during the comparison and has extra
dynamic processing looking to handle char aliases (PRIMARY strength
etc).

The comment at the top of Sort.java:

 * found that sorting with the sort keys and the collator gave
different results in some
 * cases. This implementation does not.

is not really correct.

An alternative collator string comparison could be provided that
behaves in the same way as sort.createSortKey. This would speed up 'on
-the-fly' sorts, maybe becoming feasible.

Ticker

On Wed, 2021-05-12 at 09:08 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I think the MultiSortKeys were introduced because the on-the-fly solution was 
> far too slow, at least with the normal Java sort. Could well be that the 
> problem was solved with Java 8 or newer releases.
> 
> I think there was a special case with maps containing huge numbers of equally 
> named roads causing extreme run times.
> This depends on the style. Some styles add a name like "tr2" for each unnamed 
> track with tracktype=grade2.
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag von 
> Ticker Berkin 
> Gesendet: Mittwoch, 12. Mai 2021 10:48
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] MDR building out-of-memory
> 
> Hi Gerd
> 
> Certainly no cache. Maybe reduce the chunk size, but this might
> increase copying.
> 
> It could be improved by doing a linear chunk split/sort then a multi
> -way merge. This would avoid lots of copying assuming the following:
> 
> Using the original array to store sorted chunks demands that another
> array of the same full size is needed for the final merge. If each
> sorted key chunk is converted to a object chunk and these merged, then
> although the same total size is needed, it is made of number of smaller
> arrays.
> 
> The most space efficient solution might be have Mdr11 "implements
> Comparable" and generate pairs of sortkeys on the fly and let the java
> sort take care of all the details.
> 
> The other use of LargeListSorter is Mdr7. I get a higher hit-rate
> (~50%) for the first/partialSorter (1046096 allstreets).
> 
> However, for the repeated fullNameSorter on the partial results, most
> of the lists are just 1 long, with very few more than 10. I guess this
> depends on --road-name-config/split-name and use of shields etc, but
> LargeListSorted seems overkill.
> 
> Ticker
> 
> 
> On Tue, 2021-05-11 at 19:15 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think the cache is not meant to improve run time, it is used to
> > deduplicate and thus reduce memory. Maybe it would be better to use a
> > smaller chunk size and no cache.
> > No idea why I didn't use
> > merged = new ArrayList<>(len);
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 11. Mai 2021 19:19
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] MDR building out-of-memory
> > 
> > Hi Gerd
> > 
> > I've looked at this, and if Mdr5 space becomes a problem again, I'll
> > consider converting to it.
> > 
> > A couple of comments:
> > 
> > My map had 2909735 poi so the sort chunk size was ~727433. The cache
> > sizes after each chunk were 563095, 603595, 597239 & 605718. Is the
> > cache worth-while for this low hit-rate? Just running the Gmapsupp
> > combiner on existing tiles (without --route, so no streets), I got a
> > run time of 1 min 44 secs with the cache and 1 min 30 without!
> > However
> > most of this time is copying the tiles into gmapsupp.img so not an
> > accurate statistic.
> > 
> > You could pre-allocate List<> "merged" with the correct size.
> > 
&

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

When loading precomp sea tiles, shouldn't SeaGenerator should replace
all nodes on the edges of all the precomp tiles used with nodes from a
shared coordPool. Maybe it needs to replace even more to cover the cut
lines. 

Ticker

On Thu, 2021-05-20 at 14:09 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> reg. Also thinking that shapeMerge might need to operate at
> highPrecision :
> ShapeMerger doesn't compare positions, it checks for identical
> Coords.
> 
> For the overview map the ovm_* data is read and
> MapBuilder.prepShapesForMerge() is used to make equal Coord instances
> identical. This is not 100% safe, no matter what precision we use,
> when the original data has touching rings with equal, but different
> Coords the OverviewBuilder can see identical ones.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I still haven't deciphered the complicated bits of ShapeMergeFilter.

Could it do the testing of mergeable candidates with rounded-to
-resolution points, then, if merging, convert all the points.

Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use
PredictFilterPoints(). It isn't a major problem if this limit is
exceeded because filtering might take it below again and, if it
doesn't, PolygonSplitter will deal with it.

Regarding ordering, maybe if merge succeeds for some potential shapes,
they should all be done.  

Intersecting polygons from rounding of concave shapes with a long
diagonal is slightly insoluble. Maybe something like WrongAngleFixer is
going to be needed. Multipolygon cuts that are vertical or horizontal
might be less of a problem.
 
I'm considering the effect of different orders of filters, such that if
a small hole becomes a point and cuts to it, after merging, become a
series of spikes going into the shape, removeObsoletePoints removes it.

Ticker

On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I think ShapeMergeFilter could be part of the problem. It sometimes
> finds strange ways to merge shapes, esp. when many small parts can be
> merged. Maybe this can be improved by sorting so that it merges from
> top to bottom.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 19. Mai 2021 14:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> I've just been experimenting and looking at shapes and rounding-to
> -low
> -resolution can easily cause self-intersections.
> 
> It isn't just a problem with sea, or even with multipolygon hole
> cutting. GPSMapEdit reports over 1000 in my test area. This number
> reduces only slightly when I disable DP for polygons. Generally these
> are not noticeable - a little bit of wood missing when zoomed out
> isn't
> obvious, but same is not true of sea.
> 
> Ticker
> 
> On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > no need to guess, I use GpxCreator in various places to visualize
> > input and output of filters. I load the gpx files into JOSM and
> > maybe
> > convert to Data layer to understand what happens, e.g. where the
> > start /end point is.
> > Reg. self-intersection:
> > Think of a merged sea shape with several islands in it. The
> > rounding
> > may result in only three or four different nodes, visitied in a
> > more
> > or less random order while the correct shape would be a triangle or
> > rectangle.
> > Not sure if this is only a problem with sea, probably not.
> > 
> > Feel free to try other orders of the filters, or maybe different
> > algos to set the preserved flag. There is definitely room for
> > improvements. I've played with this very often and sometimes one
> > area
> > is improved and another is worsened.
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I'm getting worried by rounding-to-resolution and how often it creates
self-intersection.

When doing the shapeSplitter.splitShape, we found that, unless
operating with high-precision points (ie there were problems at 24bit
precision), it could fail because of self-intersection.

The polygon points limit filters got around this by tricky
backtracking. This option isn't available when the overview map is
being generated as the problem points have been frozen at very low res.

Also thinking that shapeMerge might need to operate at highPrecision.

I'm just trying do work out if the different levels have their own copy
of shapes pre-filtering and how this effects what shapeMerge does as
makeMapAreas steps through the levels.

Ticker

On Thu, 2021-05-20 at 09:07 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I've attached a test case that shows how the order is important.
> Depending on the order of shapes to be merged the resulting shape
> sometimes is self-intersecting, sometimes it isn't.
> I've not yet understood what the trigger is.
> 
> Gerd
> 
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Mittwoch, 19. Mai 2021 18:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Ticker,
> 
> yes, I think ShapeMerger can always merge. There seems to be no need
> to look at the number of points. Maybe 24 is an exception.
> 
> Sorting has an effect, but not always to the better.
> 
> The merger is already used with rounded coord when the shapes for the
> overview map are merged.
> 
> Attached is a patch that shows what I am playing with. Will continue
> tomorrow...
> 
> Gerd
> 
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 19. Mai 2021 17:41
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> I still haven't deciphered the complicated bits of ShapeMergeFilter.
> 
> Could it do the testing of mergeable candidates with rounded-to
> -resolution points, then, if merging, convert all the points.
> 
> Regardless, the limit checks against MAX_POINT_IN_ELEMENT could use
> PredictFilterPoints(). It isn't a major problem if this limit is
> exceeded because filtering might take it below again and, if it
> doesn't, PolygonSplitter will deal with it.
> 
> Regarding ordering, maybe if merge succeeds for some potential
> shapes,
> they should all be done.
> 
> Intersecting polygons from rounding of concave shapes with a long
> diagonal is slightly insoluble. Maybe something like WrongAngleFixer
> is
> going to be needed. Multipolygon cuts that are vertical or horizontal
> might be less of a problem.
> 
> I'm considering the effect of different orders of filters, such that
> if
> a small hole becomes a point and cuts to it, after merging, become a
> series of spikes going into the shape, removeObsoletePoints removes
> it.
> 
> Ticker
> 
> On Wed, 2021-05-19 at 14:12 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think ShapeMergeFilter could be part of the problem. It sometimes
> > finds strange ways to merge shapes, esp. when many small parts can
> > be
> > merged. Maybe this can be improved by sorting so that it merges
> > from
> > top to bottom.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Mittwoch, 19. Mai 2021 14:14
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> > 
> > Hi Gerd
> > 
> > I've just been experimenting and looking at shapes and rounding-to
> > -low
> > -resolution can easily cause self-intersections.
> > 
> > It isn't just a problem with sea, or even with multipolygon hole
> > cutting. GPSMapEdit reports over 1000 in my test area. This number
> > reduces only slightly when I disable DP for polygons. Generally
> > these
> > are not noticeable - a little bit of wood missing when zoomed out
> > isn't
> > obvious, but same is not true of sea.
> > 
> > Ticker
> > 
> > On Wed, 2021-05-19 at 11:43 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > no need to guess, I use GpxCreator in various places to visualize
> > > input and output of filters. I load the gpx files into JOSM and
> > > maybe
> > > convert to Data layer to understand what happens, e.g. where the
> > > start /end point is.
> > > Reg. self-intersection:
> &

Re: [mkgmap-dev] Commit r4715: rework of options for reverse merge

[Ticker Berkin]

Hi Gerd

I haven't found a problem with having having it set like this for
extended types and I'll run with it like this for the moment. I might
have got my eTrex HCx test confused, so not absolutely sure that it
displayed a difference. I'll re-test.

The other thing I'll test today is routing with NOD/NET ONEWAY correct
and has_direction false.

Ticker

On Sun, 2021-05-16 at 04:28 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> if I got that right you say that the dir-for-extended.patch is the
> right way to set the direction flag in RGN for extended line types?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Samstag, 15. Mai 2021 17:47
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Commit r4715: rework of options for reverse
> merge
> 
> Hi all
> 
> I've been testing the effect of the DIR flag (0x40) in various ways:
> 
> GPSMapEdit: shows a direction arrow on all standard line types. It
> appears not to read this flag for extended line types, but, if set
> manually, show an arrow on these as well.
> 
> eTrex HCx: For all line types, including extended, when hovering over
> the line, appends " (DIR)" to the name, where DIR is N, NE, E etc. It
> does this regardless of any TYP representation of the line. eg
> "Newburgh Street (W)".
> 
> eTrex 30x: Like above, but I don't think it shows (DIR) for extended
> types.
> 
> Basecamp & Mapsource: Having the flag set makes no visible
> difference.
> There seems to be no way of interrogating road/line properties.
> 
> I've not found any ill-effects of setting the DIR flag for extended
> types.
> 
> I'm currently messing with TYP files and code to see if there is some
> combination where Garmin device/program will pick up a different
> [_line] definition if the dir flag is set.
> 
> Ticker
> 
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I hope the only case where my algo still has problems is when 2 lines
intersect the cut line at identical points and they have the same area
to that side of the line (implying they follow exactly the same path or
intersect each other). This should generate the warning:
"Lines hit divider at same points and have same area sign"
I've been thinking of ways to resolve this and should be possible.

If you are not seeing this warning and it is flagging errors, then
either another pathological case that I need to consider or it is self
-intersecting.

My last patch should, I hope, fix the case you mention unless - see
above.

I stress-tested with GB, trunk. shapeMerge with no point limit and got
a few examples of this and some "shape / hole" conflict, which I think
might be real self-intersection problems.

I'm going to add more diags and generate a gpx when
errors/debugEnabled.

I was hoping to find a good self-intersection algo, but the ones I
thought might be ok are good for detecting line segments that cross but
not for when all the difficulties are lines following each other and
crossings happen at nodes. 

Ticker

On Wed, 2021-05-26 at 04:46 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> doesn't help much with the low-res-opt branch. I've attached a patch
> which contains my test code. It writes gpx files, so please modify
> the path.
> I use it with in Eclipse a breakpoint on the line
> log.error("split failed, ratio:", ratio);
> to be able to load the GPX files into JOSM for further analyses.
> I use it with my precomp-sea test-environment, see http://gis.19327.n
> 8.nabble.com/precomp-sea-test-environment-tt5974624.html
> options:
> --output-dir=e:\ld  --gmapi --precomp-sea=f:\osm\sea.zip  --max
> -jobs=1 --style-file=e:\precomp-sea-check\sea-check -c e:\precomp-sea
> -check\tiles\template.args
> 
> or with the files in https://files.mkgmap.org.uk/download/507/xxx.zip
> and options
> --output-dir=e:\ld  --gmapi --generate-sea=multipolygon,floodblocker 
> --max-jobs=1 --preserve-element-order --allow-reverse-merge  -c
> e:\xxx\template.args
> 
> If I got that right your algo has problems when the polygon goes
> through points A and  B twice, once in A-B direction and and again in
> B-A and the cut is between A and B?
> 
> I try to find efficient code to detect and avoid just those.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I have to design a shape that does this and think what it means. With
the extra handling needed for identical opposite lines it will be easy
to detect. Maybe possible to just chuck it.

I'm out for the rest of the day so can't do anything until tomorrow.

Ticker

On Wed, 2021-05-26 at 08:46 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> OK, I think your patch improves some cases. I've also fixed another
> potential cause for these problems with r4744.
> 
> I think I have to find a way to avoid those segments which are
> visited multiple times in the same direction. There's probably always
> an alternative merge order that avoids this.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I think we need to step back and consider the various stages towards a
good overview map which I see as:

0/ MultiPolygons spliting (just added this after I wrote the rest)
1/ subDiv shapeMerge
2/ filter chain into the ovm_ img
3/ loading all the ovm_ img into the combiner
4/ any special global merging (per overview level?)
5/ standard-ish map generation into the overview map
   ie subdivs as necessary, shape allocation into them with possible
   splitting, subdiv shapeMerge, filtering & output

Just considering subDiv shapeMerge, because, while this is producing
self-intersection and/or other errors, looking to solve problems in the
following stages is futile.

For stress-testing, it should be allowed to merge without a point limit
at res24; ShapeSpitter will deal it or report errors. Verification
should be done by looking at the ovm_ img rather than the final
overview map.

It must be wrong that its behaviour depends on the processing order.

You say it just uses identity of coord points to make all decisions; ie
it doesn't try and test if a point in one shape is on a line in another
shape. Assuming this:

To merge 2 shapes, they must have a LineSegment in common (ie each has
2 identical adjacent points in the correct direction (determinable from
the sign of the area). From finding a common LineSegment, adjacent
common LineSegments are included. Then the 2 shapes are joined without
these LineSegments (after any necessary direction correction). Any
other common LineSegments between these polygons must be ignored,
otherwise holes will get closed up.

The process is repeated until all shapes in the candidate pool have
been considered.

Sharing a single point is not good enough to consider a merge - I see
figure-of-8 problems emerging from trying this.

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Sorry - sea.zip!

On Tue, 2021-05-25 at 12:36 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> I'm getting very confused by this. When I build with my trunk (some
> filter order changes + a few others), I see a whole lot more that
> JOSM
> shows. It and your 6324001.img just show a central bit. Looking with
> GPSMapEdit with "Highlight Polygon Contours". I also see areas where
> all islands are inverted.
> 
> I'll undo my trunk changes and look again.
> 
> Does JOSM have self-intersection checking?
> 
> Was the gpx trace hi-res (30bit) and then this kept when generating
> the
> osm. If not, then I'd expect problems.
> 
> Ticker
> 
> 
> On Tue, 2021-05-25 at 11:18 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I didn't investigate the details. I've created the osm data by
> > converting a gpx back to osm. I guess in the original input
> > duplicate
> > points are identical. I found more such cases where parts are
> > removed.
> > 
> > Gerd
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 25. Mai 2021 12:51
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > OK - I've reproduced this. Is split-failed.osm self-intersecting?
> > I'm
> > not sure how to tell from Josm, but it looks like it isn't. I get
> > the
> > messages from shapeSplitter because it thinks it is and then the
> > result
> > is not good - as expected.
> > 
> > I'll investigate more.
> > 
> > Ticker
> > 
> > 
> > On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > while looking at the problems with sea I found this case where
> > > ShapeSplitter removes a small part of an island from a self
> > > -intersecting polygon in res 24.
> > > Compiled with mkgmap from trunk, default style and no options.
> > > 
> > > Look at 67.6742611, 14.6783525
> > > 
> > > Found this with some check code that compares the area of the
> > > original polygon with that of sum of the parts.
> > > 
> > > Gerd
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I'm getting very confused by this. When I build with my trunk (some
filter order changes + a few others), I see a whole lot more that JOSM
shows. It and your 6324001.img just show a central bit. Looking with
GPSMapEdit with "Highlight Polygon Contours". I also see areas where
all islands are inverted.

I'll undo my trunk changes and look again.

Does JOSM have self-intersection checking?

Was the gpx trace hi-res (30bit) and then this kept when generating the
osm. If not, then I'd expect problems.

Ticker


On Tue, 2021-05-25 at 11:18 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I didn't investigate the details. I've created the osm data by
> converting a gpx back to osm. I guess in the original input duplicate
> points are identical. I found more such cases where parts are
> removed.
> 
> Gerd
> 
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 25. Mai 2021 12:51
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> OK - I've reproduced this. Is split-failed.osm self-intersecting? I'm
> not sure how to tell from Josm, but it looks like it isn't. I get the
> messages from shapeSplitter because it thinks it is and then the
> result
> is not good - as expected.
> 
> I'll investigate more.
> 
> Ticker
> 
> 
> On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > while looking at the problems with sea I found this case where
> > ShapeSplitter removes a small part of an island from a self
> > -intersecting polygon in res 24.
> > Compiled with mkgmap from trunk, default style and no options.
> > 
> > Look at 67.6742611, 14.6783525
> > 
> > Found this with some check code that compares the area of the
> > original polygon with that of sum of the parts.
> > 
> > Gerd
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

OK - I've reproduced this. Is split-failed.osm self-intersecting? I'm
not sure how to tell from Josm, but it looks like it isn't. I get the
messages from shapeSplitter because it thinks it is and then the result
is not good - as expected.

I'll investigate more.

Ticker


On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> while looking at the problems with sea I found this case where
> ShapeSplitter removes a small part of an island from a self
> -intersecting polygon in res 24.
> Compiled with mkgmap from trunk, default style and no options.
> 
> Look at 67.6742611, 14.6783525
> 
> Found this with some check code that compares the area of the
> original polygon with that of sum of the parts.
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Sorry - excess stuff in patch, correct one now.

Ticker

On Tue, 2021-05-25 at 17:46 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> Patch attached that I hope fixes the splitting problem.
> 
> I haven't looked at your change yet, but your split-failed.osm test
> had, and needed, lots of points visited twice and I don't see how
> this
> can, or should, be avoided.
> 
> Ticker
> 
> On Tue, 2021-05-25 at 15:56 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > with the attached patch ShapeMerger avoids to produce shapes where
> > more than the start point is visited twice.
> > This produces much larger img files (even larger than trunk) but
> > PolygonSplitter doesn't complain.
> > 
> > I have to find out what GpsMapEdit means with "has a jitter". Maybe
> > only those are problematic.
> > 
> > Gerd
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 25. Mai 2021 14:00
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Sorry - sea.zip!
> > 
> > On Tue, 2021-05-25 at 12:36 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > > 
> > > I'm getting very confused by this. When I build with my trunk
> > > (some
> > > filter order changes + a few others), I see a whole lot more that
> > > JOSM
> > > shows. It and your 6324001.img just show a central bit. Looking
> > > with
> > > GPSMapEdit with "Highlight Polygon Contours". I also see areas
> > > where
> > > all islands are inverted.
> > > 
> > > I'll undo my trunk changes and look again.
> > > 
> > > Does JOSM have self-intersection checking?
> > > 
> > > Was the gpx trace hi-res (30bit) and then this kept when
> > > generating
> > > the
> > > osm. If not, then I'd expect problems.
> > > 
> > > Ticker
> > > 
> > > 
> > > On Tue, 2021-05-25 at 11:18 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > I didn't investigate the details. I've created the osm data by
> > > > converting a gpx back to osm. I guess in the original input
> > > > duplicate
> > > > points are identical. I found more such cases where parts are
> > > > removed.
> > > > 
> > > > Gerd
> > > > 
> > > > 
> > > > 
> > > > Von: mkgmap-dev  im
> > > > Auftrag
> > > > von Ticker Berkin 
> > > > Gesendet: Dienstag, 25. Mai 2021 12:51
> > > > An: Development list for mkgmap
> > > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > > without
> > > > a log message
> > > > 
> > > > Hi Gerd
> > > > 
> > > > OK - I've reproduced this. Is split-failed.osm self
> > > > -intersecting?
> > > > I'm
> > > > not sure how to tell from Josm, but it looks like it isn't. I
> > > > get
> > > > the
> > > > messages from shapeSplitter because it thinks it is and then
> > > > the
> > > > result
> > > > is not good - as expected.
> > > > 
> > > > I'll investigate more.
> > > > 
> > > > Ticker
> > > > 
> > > > 
> > > > On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > > 
> > > > > while looking at the problems with sea I found this case
> > > > > where
> > > > > ShapeSplitter removes a small part of an island from a self
> > > > > -intersecting polygon in res 24.
> > > > > Compiled with mkgmap from trunk, default style and no
> > > > > options.
> > > > > 
> > > > > Look at 67.6742611, 14.6783525
> > > > > 
> > > > > Found this with some check code that compares the area of the
> > > > > original polygon with that of sum of the parts.
> > > > > 
> > > > > Gerd
> > > > > ___
> > > > > mkgmap-dev mailing list
> > > > > mkgmap-dev@lists.mkgmap.org.uk
> > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > 

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

Patch attached that I hope fixes the splitting problem.

I haven't looked at your change yet, but your split-failed.osm test
had, and needed, lots of points visited twice and I don't see how this
can, or should, be avoided.

Ticker

On Tue, 2021-05-25 at 15:56 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> with the attached patch ShapeMerger avoids to produce shapes where
> more than the start point is visited twice.
> This produces much larger img files (even larger than trunk) but
> PolygonSplitter doesn't complain.
> 
> I have to find out what GpsMapEdit means with "has a jitter". Maybe
> only those are problematic.
> 
> Gerd
> 
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 25. Mai 2021 14:00
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Sorry - sea.zip!
> 
> On Tue, 2021-05-25 at 12:36 +0100, Ticker Berkin wrote:
> > Hi Gerd
> > 
> > I'm getting very confused by this. When I build with my trunk (some
> > filter order changes + a few others), I see a whole lot more that
> > JOSM
> > shows. It and your 6324001.img just show a central bit. Looking
> > with
> > GPSMapEdit with "Highlight Polygon Contours". I also see areas
> > where
> > all islands are inverted.
> > 
> > I'll undo my trunk changes and look again.
> > 
> > Does JOSM have self-intersection checking?
> > 
> > Was the gpx trace hi-res (30bit) and then this kept when generating
> > the
> > osm. If not, then I'd expect problems.
> > 
> > Ticker
> > 
> > 
> > On Tue, 2021-05-25 at 11:18 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > I didn't investigate the details. I've created the osm data by
> > > converting a gpx back to osm. I guess in the original input
> > > duplicate
> > > points are identical. I found more such cases where parts are
> > > removed.
> > > 
> > > Gerd
> > > 
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Dienstag, 25. Mai 2021 12:51
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > without
> > > a log message
> > > 
> > > Hi Gerd
> > > 
> > > OK - I've reproduced this. Is split-failed.osm self-intersecting?
> > > I'm
> > > not sure how to tell from Josm, but it looks like it isn't. I get
> > > the
> > > messages from shapeSplitter because it thinks it is and then the
> > > result
> > > is not good - as expected.
> > > 
> > > I'll investigate more.
> > > 
> > > Ticker
> > > 
> > > 
> > > On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > while looking at the problems with sea I found this case where
> > > > ShapeSplitter removes a small part of an island from a self
> > > > -intersecting polygon in res 24.
> > > > Compiled with mkgmap from trunk, default style and no options.
> > > > 
> > > > Look at 67.6742611, 14.6783525
> > > > 
> > > > Found this with some check code that compares the area of the
> > > > original polygon with that of sum of the parts.
> > > > 
> > > > Gerd
> > > > ___
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev@lists.mkgmap.org.uk
> > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> m

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Sorry - excess stuff in patch, correct one now.

Ticker

On Tue, 2021-05-25 at 17:46 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> Patch attached that I hope fixes the splitting problem.
> 
> I haven't looked at your change yet, but your split-failed.osm test
> had, and needed, lots of points visited twice and I don't see how
> this
> can, or should, be avoided.
> 
> Ticker
> 
> On Tue, 2021-05-25 at 15:56 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > with the attached patch ShapeMerger avoids to produce shapes where
> > more than the start point is visited twice.
> > This produces much larger img files (even larger than trunk) but
> > PolygonSplitter doesn't complain.
> > 
> > I have to find out what GpsMapEdit means with "has a jitter". Maybe
> > only those are problematic.
> > 
> > Gerd
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 25. Mai 2021 14:00
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Sorry - sea.zip!
> > 
> > On Tue, 2021-05-25 at 12:36 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > > 
> > > I'm getting very confused by this. When I build with my trunk
> > > (some
> > > filter order changes + a few others), I see a whole lot more that
> > > JOSM
> > > shows. It and your 6324001.img just show a central bit. Looking
> > > with
> > > GPSMapEdit with "Highlight Polygon Contours". I also see areas
> > > where
> > > all islands are inverted.
> > > 
> > > I'll undo my trunk changes and look again.
> > > 
> > > Does JOSM have self-intersection checking?
> > > 
> > > Was the gpx trace hi-res (30bit) and then this kept when
> > > generating
> > > the
> > > osm. If not, then I'd expect problems.
> > > 
> > > Ticker
> > > 
> > > 
> > > On Tue, 2021-05-25 at 11:18 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > I didn't investigate the details. I've created the osm data by
> > > > converting a gpx back to osm. I guess in the original input
> > > > duplicate
> > > > points are identical. I found more such cases where parts are
> > > > removed.
> > > > 
> > > > Gerd
> > > > 
> > > > 
> > > > 
> > > > Von: mkgmap-dev  im
> > > > Auftrag
> > > > von Ticker Berkin 
> > > > Gesendet: Dienstag, 25. Mai 2021 12:51
> > > > An: Development list for mkgmap
> > > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > > without
> > > > a log message
> > > > 
> > > > Hi Gerd
> > > > 
> > > > OK - I've reproduced this. Is split-failed.osm self
> > > > -intersecting?
> > > > I'm
> > > > not sure how to tell from Josm, but it looks like it isn't. I
> > > > get
> > > > the
> > > > messages from shapeSplitter because it thinks it is and then
> > > > the
> > > > result
> > > > is not good - as expected.
> > > > 
> > > > I'll investigate more.
> > > > 
> > > > Ticker
> > > > 
> > > > 
> > > > On Tue, 2021-05-25 at 10:04 +, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > > 
> > > > > while looking at the problems with sea I found this case
> > > > > where
> > > > > ShapeSplitter removes a small part of an island from a self
> > > > > -intersecting polygon in res 24.
> > > > > Compiled with mkgmap from trunk, default style and no
> > > > > options.
> > > > > 
> > > > > Look at 67.6742611, 14.6783525
> > > > > 
> > > > > Found this with some check code that compares the area of the
> > > > > original polygon with that of sum of the parts.
> > > > > 
> > > > > Gerd
> > > > > ___
> > > > > mkgmap-dev mailing list
> > > > > mkgmap-dev@lists.mkgmap.org.uk
> > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > 

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

shapeSplitter will have problems (ie get it wrong some of the time)
where there are in/out lines to a hole that share the same cut point as
a line that is the boundary between a shape and hole; could be many
holes (or shapes) and many lines. The simple sort/dedupe I was doing
isn't adequate. I'll come up with something better tomorrow.

Ticker 

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I see what you mean. Is this with splitShaeFix_5_lowRes.patch?
I'll investigate. Is there an OSM file I can run with?

Ticker

On Wed, 2021-06-02 at 06:32 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> please check https://files.mkgmap.org.uk/download/509/special-v5.zip
> 
> Can you tell me why ShapeSplitter drops some points, e.g the one near
> 68.2669706, 15.1206053 without complaining?
> None of the points in the original data is visited more than twice
> and all highprec equal points are unique.
> 
> I'm working on an improvement to make the connecting lines shorter,
> but I don't see yet how I can avoid to have connecting lines like
> that.
> 
> Gerd
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 1. Juni 2021 17:18
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> I've added code to deal with some variants of the case as described -
> I
> hope this will be enough to cope with more complex shapes generated
> by
> ShapeMerger.
> 
> There might be a bit more I can do without having to resort to more
> complex geometry analysis if it still gives problems.
> 
> I've also restructured it a bit.
> 
> Patch attached based on low-res-opt. Trunk version will be the same
> (but the patch would be different)
> 
> Ticker
> 
> On Mon, 2021-05-31 at 17:12 +0100, Ticker Berkin wrote:
> > Hi Gerd
> > 
> > shapeSplitter will have problems (ie get it wrong some of the time)
> > where there are in/out lines to a hole that share the same cut
> > point
> > as
> > a line that is the boundary between a shape and hole; could be many
> > holes (or shapes) and many lines. The simple sort/dedupe I was
> > doing
> > isn't adequate. I'll come up with something better tomorrow.
> > 
> > Ticker
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I'm thinking this is a problem with high-precision points and the
coordPool

Ticker

On Wed, 2021-06-02 at 10:23 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> I see what you mean. Is this with splitShaeFix_5_lowRes.patch?
> I'll investigate. Is there an OSM file I can run with?
> 
> Ticker
> 
> On Wed, 2021-06-02 at 06:32 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > please check 
> > https://files.mkgmap.org.uk/download/509/special-v5.zip
> > 
> > Can you tell me why ShapeSplitter drops some points, e.g the one
> > near
> > 68.2669706, 15.1206053 without complaining?
> > None of the points in the original data is visited more than twice
> > and all highprec equal points are unique.
> > 
> > I'm working on an improvement to make the connecting lines shorter,
> > but I don't see yet how I can avoid to have connecting lines like
> > that.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 1. Juni 2021 17:18
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > I've added code to deal with some variants of the case as described
> > -
> > I
> > hope this will be enough to cope with more complex shapes generated
> > by
> > ShapeMerger.
> > 
> > There might be a bit more I can do without having to resort to more
> > complex geometry analysis if it still gives problems.
> > 
> > I've also restructured it a bit.
> > 
> > Patch attached based on low-res-opt. Trunk version will be the same
> > (but the patch would be different)
> > 
> > Ticker
> > 
> > On Mon, 2021-05-31 at 17:12 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > > 
> > > shapeSplitter will have problems (ie get it wrong some of the
> > > time)
> > > where there are in/out lines to a hole that share the same cut
> > > point
> > > as
> > > a line that is the boundary between a shape and hole; could be
> > > many
> > > holes (or shapes) and many lines. The simple sort/dedupe I was
> > > doing
> > > isn't adequate. I'll come up with something better tomorrow.
> > > 
> > > Ticker
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter default resolution 13 and overview map from default style

[Ticker Berkin]

Hi all

Outermost level should be as Gerd suggests
5 levels of overview seems excessive - what about:

overview-levels = 4:17, 5:15, 6:13

Ticker


On Tue, 2021-06-01 at 13:29 +, Gerd Petermann wrote:
> Hi all,
> 
> I am not sure but I think it is a bad idea to have 
> overview-levels = 4:17, 5:16, 6:15, 7:14, 8:12
> in the default style.
> The lowest resolution is 12 while splitter uses the default 13 for
> its resolution. This also means that tiles from splitter are aligned
> to a resolution of 13. The lowest level in the overview map is
> therefore sometimes much more distorted at tile boundaries than the
> next higher. I think it should be changed to 
> overview-levels = 4:17, 5:16, 6:15, 7:14, 8:13
> 
> I've never thought about this relationship between splitter and the
> overview map. Just wondered why my test data looks so ugly at
> resolution 12.
> An alternative would be to remove the lowest level completely.
> Maybe mkgmap should print a warning when tiles are not aligned for
> the lowest resolution used in the map?
> 
> Any thoughts? 
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I was beginning to suspect this - there were 2 very close cut-lines
which is why I thought coords were not behaving.

I've reproduced with the .osm and am investigating 

Ticker

On Wed, 2021-06-02 at 09:50 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> yes, was with splitShapeFix_5_lowRes.patch
> I've just noticed that s_3 and s_4 were from a different split.
> The shape was the result of various steps with "too small island
> removal" and "cut again after full merge".
> 
> Maybe you can reproduce with the attached file. I've loaded the gpx
> into JOSM, fixed the duplicated points and used Shift+J (join
> overlapping areas)
> 
> Gerd
> 
> 
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 2. Juni 2021 11:23
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> I see what you mean. Is this with splitShaeFix_5_lowRes.patch?
> I'll investigate. Is there an OSM file I can run with?
> 
> Ticker
> 
> On Wed, 2021-06-02 at 06:32 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > please check 
> > https://files.mkgmap.org.uk/download/509/special-v5.zip
> > 
> > Can you tell me why ShapeSplitter drops some points, e.g the one
> > near
> > 68.2669706, 15.1206053 without complaining?
> > None of the points in the original data is visited more than twice
> > and all highprec equal points are unique.
> > 
> > I'm working on an improvement to make the connecting lines shorter,
> > but I don't see yet how I can avoid to have connecting lines like
> > that.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 1. Juni 2021 17:18
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > I've added code to deal with some variants of the case as described
> > -
> > I
> > hope this will be enough to cope with more complex shapes generated
> > by
> > ShapeMerger.
> > 
> > There might be a bit more I can do without having to resort to more
> > complex geometry analysis if it still gives problems.
> > 
> > I've also restructured it a bit.
> > 
> > Patch attached based on low-res-opt. Trunk version will be the same
> > (but the patch would be different)
> > 
> > Ticker
> > 
> > On Mon, 2021-05-31 at 17:12 +0100, Ticker Berkin wrote:
> > > Hi Gerd
> > > 
> > > shapeSplitter will have problems (ie get it wrong some of the
> > > time)
> > > where there are in/out lines to a hole that share the same cut
> > > point
> > > as
> > > a line that is the boundary between a shape and hole; could be
> > > many
> > > holes (or shapes) and many lines. The simple sort/dedupe I was
> > > doing
> > > isn't adequate. I'll come up with something better tomorrow.
> > > 
> > > Ticker
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I've found and fixed some stupid mistakes - sorry for wasting your
time.

Patch attached - works with your test case.

Ticker

Index: src/uk/me/parabola/util/ShapeSplitter.java
===
--- src/uk/me/parabola/util/ShapeSplitter.java	(revision 4753)
+++ src/uk/me/parabola/util/ShapeSplitter.java	(working copy)
@@ -275,11 +275,23 @@
 		return pointsToPath2D(outputList, countVals);
 	}
 
-/* Dec16/Jan17. Ticker Berkin. New implementation for splitting shapes.
+// Dec16/Jan17. Ticker Berkin. New implementation for splitting shapes and clipping
 
-Eventually maybe can be used instead of some of the above and elsewhere
-*/
+	private boolean detectedProblems;
+	private List newLess, newMore;
+	private String gpxDirectory;
+	private long fullArea;
 
+	private List lineInfo; // for the side we are working on
+	private List> origList; // ditto
+
+	private boolean multSameLow; // lineInfo.sort(comparator) might set this
+
+	private void logMsg(Object ... olist) {
+		detectedProblems = true;
+		log.warn(olist);
+	}
+
 	/**
 	 * Service routine for processLineList. Processes a nested list of holes within a shape or
 	 * list of shapes within a hole.
@@ -290,11 +302,8 @@
 	 * @param endEnclosed point where starting line ends on dividing line.
 	 * @param addHolesToThis if not null, then called from a shape and subtract holes from it
 	 * otherwise new shapes within a hole.
-	 * @param lineInfo list of lines.
-	 * @param origList list of shapes to which we append new shapes.
 	 */
-	private static int doLines(int startInx, int endEnclosed, MergeCloseHelper addHolesToThis,
-   List lineInfo, List> origList) {
+	private int doLines(int startInx, int endEnclosed, MergeCloseHelper addHolesToThis) {
 		int inx = startInx;
 		final boolean calledFromHole = addHolesToThis == null;
 		while (inx < lineInfo.size()) {
@@ -304,9 +313,9 @@
 			if (thisLine.lowPoint == endEnclosed && thisLine.highPoint == endEnclosed) // consider carefully
 if (calledFromHole == (thisLine.areaOrHole == -1))
 	break; // stop if same type
-			inx = doLines(inx+1, thisLine.highPoint, calledFromHole ? thisLine : null, lineInfo, origList);
+			inx = doLines(inx+1, thisLine.highPoint, calledFromHole ? thisLine : null);
 			if (calledFromHole) // handling list of shapes
-thisLine.closeAppend(origList, true);
+thisLine.closeAppend(true);
 			else // handling list of holes
 addHolesToThis.addHole(thisLine);
 		}
@@ -317,17 +326,17 @@
 	 * Service routine for splitShape. Takes list of lines and appends distinct shapes
 	 * @param lineInfo list of lines that start and end on the dividing line (or orig startPoint)
 	 * @param origList list of shapes to which we append new shapes formed from above
-	 * @param fullArea of orig polygon. used for sign and handling of last line segment
 	 */
-	private static int processLineList(List lineInfo, List> origList, long fullArea) {
-		int errorCount = 0;
+	private void processLineList(List lineInfo, List> origList) {
+		this.lineInfo = lineInfo;
+		this.origList = origList;
 		if (origList == null) // never wanted this side
-			return errorCount;
+			return;
 		MergeCloseHelper firstLine = lineInfo.get(0);
 		if (lineInfo.size() == 1) { // single shape that never crossed line
 			if (!firstLine.points.isEmpty()) // all on this side
-firstLine.closeAppend(origList, false);
-			return errorCount;
+firstLine.closeAppend(false);
+			return;
 		}
 		// look at last item in list of lines
 		MergeCloseHelper lastLine = lineInfo.get(lineInfo.size()-1);
@@ -335,14 +344,14 @@
 			lineInfo.remove(lineInfo.size()-1);
 		else { // ended up on this side and must have crossed the line
 			// so first element is really the end of the last
-			lastLine.combineFirstIntoLast(firstLine, fullArea);
+			lastLine.combineFirstIntoLast(firstLine);
 			lineInfo.remove(0);
 			firstLine = lineInfo.get(0);
 		}
 		if (lineInfo.size() == 1) { // simple poly that crossed once and back
 			firstLine.setMoreInfo(0);
-			firstLine.closeAppend(origList, true);
-			return errorCount;
+			firstLine.closeAppend(true);
+			return;
 		}
 		// Above were the simple cases - probably 99% of calls.
 
@@ -358,7 +367,6 @@
 		// check and set any missing directions based on signs of full/area
 		boolean someDirectionsNotSet = false;
 		int areaDirection = 0;
-		String diagMsg = "";
 		for (MergeCloseHelper thisLine : lineInfo) {
 			thisLine.setMoreInfo(fullAreaSign);
 			if (thisLine.direction == 0)
@@ -368,78 +376,208 @@
 if (areaDirection == 0)
 	areaDirection = tmpAreaDirection;
 else if (areaDirection != tmpAreaDirection)
-	diagMsg += "Direction/Area conflict.";
+	logMsg("Direction/Area conflict", fullAreaSign, areaDirection, tmpAreaDirection);
 			}
 		}
 		if (someDirectionsNotSet) {
 			if (areaDirection == 0)
-diagMsg += "Cant deduce direction/Area mapping.";
+logMsg

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I'd been doing some investigation of filters ordering (based on trunk).
I'd also done the pre-filtering of lines & shapes by minRes.

My conclusions are:

Shapes:
It is better to run SizeFilter after RemoveObsoleteFilter.
It is more efficient to run DP filter after both of these.

Lines:
LineSplitterFilter should be run after everything that can remove
points.
It is more efficient to run DP after RemoveObsoleteFilter.

I've adapted my changes into a patch for the low-res-opt branch, along
with removal of some resolution tests that are now redundant.

For the contourFilters, I've left DP as the first filter but moved
LineSplitter as for the normalFilters

Ticker

 
Index: src/uk/me/parabola/mkgmap/build/MapBuilder.java
===
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java	(revision 4731)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java	(working copy)
@@ -915,12 +915,15 @@
 		processPoints(map, div, points);
 
 		final int res = z.getResolution();
-		lines = lines.stream().filter(l -> l.getMinResolution() <= res).collect(Collectors.toList());
-		shapes = shapes.stream().filter(s -> s.getMinResolution() <= res).collect(Collectors.toList());
-		
+		final int level = z.getLevel();
+		if (level != 0) {
+			lines = lines.stream().filter(l -> l.getMinResolution() <= res).collect(Collectors.toList());
+			shapes = shapes.stream().filter(s -> s.getMinResolution() <= res).collect(Collectors.toList());
+		}
+
 		if (mergeLines) {
-			LineMergeFilter merger = new LineMergeFilter();
-			lines = merger.merge(lines, res, !hasNet, allowReverseMerge);
+			LineMergeFilter lineMergeFilter = new LineMergeFilter();
+			lines = lineMergeFilter.merge(lines, res, !hasNet, allowReverseMerge);
 		}
 
 		if (mergeShapes) {
@@ -930,7 +933,7 @@
 
 		// recalculate preserved status for all points in lines and shapes
 		shapes.forEach(e -> e.getPoints().forEach(p -> p.preserved(false)));
-		if (z.getLevel() == 0 && hasNet) {
+		if (level == 0 && hasNet) {
 			lines.forEach(e -> e.getPoints().forEach(p -> p.preserved(p.isNumberNode(;	
 		} else {
 			lines.forEach(e -> e.getPoints().forEach(p -> p.preserved(false)));
@@ -1218,12 +1221,14 @@
 	 * Note that the location and resolution of map elements is relative to the
 	 * subdivision that they occur in.
 	 *
-	 * @param map	The map to add points to.
+	 * @param map	The map to add lines to.
 	 * @param div	The subdivision that the lines belong to.
 	 * @param lines The lines to be added.
 	 */
 	private void processLines(Map map, Subdivision div, List lines) {
 		div.startLines();  // Signal that we are beginning to draw the lines.
+		if (lines.size() == 0)
+			return;
 
 		int res = div.getResolution();
 
@@ -1234,6 +1239,7 @@
 
 		LayerFilterChain normalFilters = new LayerFilterChain(config);
 		LayerFilterChain contourFilters = new LayerFilterChain(config);
+		DouglasPeuckerFilter dp = null;
 		if (enableLineCleanFilters && (res < 24)) {
 			MapFilter rounder = new RoundCoordsFilter();
 			MapFilter sizeFilter = new SizeFilter(MIN_SIZE_LINE);
@@ -1240,32 +1246,33 @@
 			normalFilters.addFilter(rounder);
 			normalFilters.addFilter(sizeFilter);
 			double errorForRes = dpFilterLineLevelMap.ceilingEntry(res).getValue();
-			if(errorForRes > 0) {
-DouglasPeuckerFilter dp = new DouglasPeuckerFilter(errorForRes);
-normalFilters.addFilter(dp);
+			if (errorForRes > 0) {
+dp = new DouglasPeuckerFilter(errorForRes);
 contourFilters.addFilter(dp);
+// normalFilter dp is run after removeObsolete. Should be more efficient
 			}
 			contourFilters.addFilter(rounder);
 			contourFilters.addFilter(sizeFilter);
 		}
+		RemoveObsoletePointsFilter removeObsolete = new RemoveObsoletePointsFilter();
+		normalFilters.addFilter(removeObsolete);
+		contourFilters.addFilter(removeObsolete);
+		if (dp != null)
+			normalFilters.addFilter(dp);
 		for (MapFilter filter : Arrays.asList(
-new LineSplitterFilter(), 
 new RemoveEmpty(),
-new RemoveObsoletePointsFilter(), 
-new LinePreparerFilter(div), 
+new LineSplitterFilter(),
+new LinePreparerFilter(div),
 new LineAddFilter(div, map))) {
 			normalFilters.addFilter(filter);
 			contourFilters.addFilter(filter);
 		}
-		
-		for (MapLine line : lines) {
-			if (line.getMinResolution() <= res) {
-if (GType.isContourLine(line)) 
-	contourFilters.startFilter(line);
-else 
-	normalFilters.startFilter(line);
-			}
-		}
+
+		for (MapLine line : lines)
+			if (GType.isContourLine(line))
+contourFilters.startFilter(line);
+			else
+normalFilters.startFilter(line);
 	}
 
 	/**
@@ -1281,6 +1288,8 @@
 	 */
 	private void processShapes(Map map, Subdivision div, List shapes) {
 		div.startShapes();  // Signal that we are beginning to draw the shapes.
+		if (shapes.size() == 0)
+			return;
 
 		int res = div.getResolution();
 
@@ -1288,36 +1297,34 @@
 		config.setResolution(res);
 		config.setLevel(div.getZoom().getLevel());
 	

Re: [mkgmap-dev] Change Coord.toString() to return degrees instead of map units

[Ticker Berkin]

Hi Gerd

This change is fine by me.

Ticker

On Sat, 2021-05-22 at 06:33 +, Gerd Petermann wrote:
> Hi devs,
> 
> I think almost nobody needs the Garmin lat/lon coordinates in log or
> debug messages, the degrees are much easier to use.
> This is esp. true for debugging, so I want to change Coord.toString()
> to return the same result as Coord.toDegreeString().
> 
> The attached patch changes this and simplifies the code where
> possible.
> One unit test requires the garmin units, and the messages about
> boundary nodes in LineClipper are probably better understandable with
> garmin units.
> 
> Unfortunately I have no easy way to find out where Coord.toString()
> is called implicitly when Object.ToString() is called, so maybe there
> are more places where .toGarminString() should be added. 
> 
> Any objections/hints/better ideas? 
> The display tool uses toDegreesString(). I think it never uses
> Coord.toString() implicitly.
> 
> Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] improve sea/coastline rendering

[Ticker Berkin]

Hi

As expected, I vote for commenting out this rule:

# Showing the coastline adds almost nothing to the visuals of the map,
so disable:
#natural=coastline [0x15 resolution 12]

Ticker

On Tue, 2021-05-18 at 07:03 +, Gerd Petermann wrote:
> Hi all,
> 
> see item 15 in http://gis.19327.n8.nabble.com/default-style-lines-enh
> ancements-tc5971280.html
> 
> This is one that I do understand and support.
> I think there are only few situations in which you want to render
> natural=coastline as line. The only one that I can think of is that
> you don't render natural=sea polygons. Is that realistic?
> 
> Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I've been doing experiments on tiles with much more sea and islands
(tests before only had a bit of sea and lots of city).

With my re-ordering I get a very slightly smaller main tile (4096 bytes
in 5614K) and larger osmmap.img (34304 vs. 25088)

In both cases I'm getting quite a few shapeSplitter messages where it
finds inconsistencies in the line directions across the split-line;
normally due to self-intersection. These are happening in both the main
tile generation and the overview combining.

Once shapeSplitter has generated this type of warning it is
indeterminate which is shape and which is hole, so I'd expect gaps in
the rendering. 

I think trunk had eradicated split/self-intersection problems in the
tile processing, including the ovm_ file generation.

I'm not sure about the overview combiner as, in my standard map-build,
I don't merge shapes. 

Ticker

On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I tried this patch with r4734 and I see no improvements. Maps are
> larger, more white rectangles in sea.
> 
> Maybe my recent changes don't work well with yours? Do you have an
> example where it improves something with r4734?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Freitag, 21. Mai 2021 18:10
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> I'd been doing some investigation of filters ordering (based on
> trunk).
> I'd also done the pre-filtering of lines & shapes by minRes.
> 
> My conclusions are:
> 
> Shapes:
> It is better to run SizeFilter after RemoveObsoleteFilter.
> It is more efficient to run DP filter after both of these.
> 
> Lines:
> LineSplitterFilter should be run after everything that can remove
> points.
> It is more efficient to run DP after RemoveObsoleteFilter.
> 
> I've adapted my changes into a patch for the low-res-opt branch,
> along
> with removal of some resolution tests that are now redundant.
> 
> For the contourFilters, I've left DP as the first filter but moved
> LineSplitter as for the normalFilters
> 
> Ticker
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

Trying to use JOSM to decide if a gpx trace represents a self
-intersecting polygon is difficult. Is there a way of forcing it to
consider it closed and then check itself? Failing that, number the
points or segments somehow.

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Thanks Gerd, that works for me

Ticker

On Thu, 2021-05-27 at 12:54 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I first convert the gpx layer to a data layer, then add the tag
> natural=water
> Next execute validator which will complain that the way is not
> closed.
> Right click on that warning allows to "zoom to problem" .
> This tells you where the first point is.
> 
> In the "OSM data" preferences I've enabled "Draw segment order
> numbers on selected way"
> 
> Hope this helps.
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 14:21
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> Trying to use JOSM to decide if a gpx trace represents a self
> -intersecting polygon is difficult. Is there a way of forcing it to
> consider it closed and then check itself? Failing that, number the
> points or segments somehow.
> 
> Ticker
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I'm confused. After mp hole cutting and shape merging, I expect a
single closed way that will have in/out lines, probably on top of each
other, to connect each area that was a hole, eventually, to a point on
the outside of the shape.

However, JOSM does show the water band on the outside of what were the
islands, and on both sides of the joining lines.

Ticker

On Thu, 2021-05-27 at 15:10 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> yes, sure. I've converted a gpx output to OSM. All polygons with
> holes are not valid OSM ways, but they are very normal as MapShape.
> Validator doesn't help much with these polygons.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 17:06
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> JOSM gives a self-crossing way validation error on this.
> 
> Ticker
> 
> On Thu, 2021-05-27 at 14:53 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > no need to hurry ;)
> > Here is one:
> > https://files.mkgmap.org.uk/download/508/water.osm
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 27. Mai 2021 16:47
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > next patch also has a problem - sorry to waste your time.
> > Can I have your test data.
> > 
> > Ticker
> > 
> > On Thu, 2021-05-27 at 14:33 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > with my current test environment the patch doesn't improve the
> > > result.
> > > It reports some errors (the old code didn't always detect them)
> > > and
> > > eithers add or removes parts of the heavily merged shapes.
> > > 
> > > I've inspected one shape and found no obvious problem. No point
> > > is
> > > visited more then twice, but 167 points are visited twice. Let me
> > > know if you need the data.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Donnerstag, 27. Mai 2021 16:09
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > without
> > > a log message
> > > 
> > > Thanks Gerd, that works for me
> > > 
> > > Ticker
> > > 
> > > On Thu, 2021-05-27 at 12:54 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > I first convert the gpx layer to a data layer, then add the tag
> > > > natural=water
> > > > Next execute validator which will complain that the way is not
> > > > closed.
> > > > Right click on that warning allows to "zoom to problem" .
> > > > This tells you where the first point is.
> > > > 
> > > > In the "OSM data" preferences I've enabled "Draw segment order
> > > > numbers on selected way"
> > > > 
> > > > Hope this helps.
> > > > 
> > > > Gerd
> > > > 
> > > > 
> > > > Von: mkgmap-dev  im
> > > > Auftrag
> > > > von Ticker Berkin 
> > > > Gesendet: Donnerstag, 27. Mai 2021 14:21
> > > > An: Development list for mkgmap
> > > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > > without
> > > > a log message
> > > > 
> > > > Hi Gerd
> > > > 
> > > > Trying to use JOSM to decide if a gpx trace represents a self
> > > > -intersecting polygon is difficult. Is there a way of forcing
> > > > it
> > > > to
> > > > consider it closed and then check itself? Failing that, number
> > > > the
> > > > points or segments somehow.
> > > > 
> > > > Ticker
> > > > 
> > > > 
> > > > ___
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev@lists.mkgmap.org.uk
> > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > __

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

JOSM gives a self-crossing way validation error on this.

Ticker

On Thu, 2021-05-27 at 14:53 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> no need to hurry ;)
> Here is one:
> https://files.mkgmap.org.uk/download/508/water.osm
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 16:47
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> next patch also has a problem - sorry to waste your time.
> Can I have your test data.
> 
> Ticker
> 
> On Thu, 2021-05-27 at 14:33 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > with my current test environment the patch doesn't improve the
> > result.
> > It reports some errors (the old code didn't always detect them) and
> > eithers add or removes parts of the heavily merged shapes.
> > 
> > I've inspected one shape and found no obvious problem. No point is
> > visited more then twice, but 167 points are visited twice. Let me
> > know if you need the data.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 27. Mai 2021 16:09
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Thanks Gerd, that works for me
> > 
> > Ticker
> > 
> > On Thu, 2021-05-27 at 12:54 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > I first convert the gpx layer to a data layer, then add the tag
> > > natural=water
> > > Next execute validator which will complain that the way is not
> > > closed.
> > > Right click on that warning allows to "zoom to problem" .
> > > This tells you where the first point is.
> > > 
> > > In the "OSM data" preferences I've enabled "Draw segment order
> > > numbers on selected way"
> > > 
> > > Hope this helps.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Donnerstag, 27. Mai 2021 14:21
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > without
> > > a log message
> > > 
> > > Hi Gerd
> > > 
> > > Trying to use JOSM to decide if a gpx trace represents a self
> > > -intersecting polygon is difficult. Is there a way of forcing it
> > > to
> > > consider it closed and then check itself? Failing that, number
> > > the
> > > points or segments somehow.
> > > 
> > > Ticker
> > > 
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

JOSM gives a self-crossing way validation error on this.

Ticker

On Thu, 2021-05-27 at 14:53 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> no need to hurry ;)
> Here is one:
> https://files.mkgmap.org.uk/download/508/water.osm
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 16:47
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> next patch also has a problem - sorry to waste your time.
> Can I have your test data.
> 
> Ticker
> 
> On Thu, 2021-05-27 at 14:33 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > with my current test environment the patch doesn't improve the
> > result.
> > It reports some errors (the old code didn't always detect them) and
> > eithers add or removes parts of the heavily merged shapes.
> > 
> > I've inspected one shape and found no obvious problem. No point is
> > visited more then twice, but 167 points are visited twice. Let me
> > know if you need the data.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 27. Mai 2021 16:09
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Thanks Gerd, that works for me
> > 
> > Ticker
> > 
> > On Thu, 2021-05-27 at 12:54 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > I first convert the gpx layer to a data layer, then add the tag
> > > natural=water
> > > Next execute validator which will complain that the way is not
> > > closed.
> > > Right click on that warning allows to "zoom to problem" .
> > > This tells you where the first point is.
> > > 
> > > In the "OSM data" preferences I've enabled "Draw segment order
> > > numbers on selected way"
> > > 
> > > Hope this helps.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Donnerstag, 27. Mai 2021 14:21
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > without
> > > a log message
> > > 
> > > Hi Gerd
> > > 
> > > Trying to use JOSM to decide if a gpx trace represents a self
> > > -intersecting polygon is difficult. Is there a way of forcing it
> > > to
> > > consider it closed and then check itself? Failing that, number
> > > the
> > > points or segments somehow.
> > > 
> > > Ticker
> > > 
> > > 
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

Found stupid mistake, updated patch attached.

Also, if debug enabled, always splits both sides of the line, and, if
both sides are collected to the same list, just gives 1 gpx trace for
it.

Ticker

On Thu, 2021-05-27 at 10:54 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> sounds good. I'm making progress with a better ShapeMerger that
> produces less jitter, but so far it still produces some.
> Will try it later..
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 12:49
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> I think this can cope with any number of lines, in either/both
> directions, following the same path. Also it should give better
> output
> if same number of lines in both directions.
> 
> With debug, if it notices a problem, it generates a GPX trace of the
> original and traces of all the resultant bits individually in a
> subdirectory relevant to the split line.
> 
> I've tested it on a few examples, but I'll stress-test GBR again and
> check the shapes where it detects problems.
> 
> I probably need to tidy up some message levels and comments.
> 
> Ticker
> 
> On Wed, 2021-05-26 at 11:17 +0100, Ticker Berkin wrote:
> > Hi Gerd
> > 
> > I have to design a shape that does this and think what it means.
> > With
> > the extra handling needed for identical opposite lines it will be
> > easy
> > to detect. Maybe possible to just chuck it.
> > 
> > I'm out for the rest of the day so can't do anything until
> > tomorrow.
> > 
> > Ticker
> > 
> > On Wed, 2021-05-26 at 08:46 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > OK, I think your patch improves some cases. I've also fixed
> > > another
> > > potential cause for these problems with r4744.
> > > 
> > > I think I have to find a way to avoid those segments which are
> > > visited multiple times in the same direction. There's probably
> > > always
> > > an alternative merge order that avoids this.
> > > 
> > > Gerd
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/util/ShapeSplitter.java
===
--- src/uk/me/parabola/util/ShapeSplitter.java	(revision 4745)
+++ src/uk/me/parabola/util/ShapeSplitter.java	(working copy)
@@ -12,21 +12,20 @@
  */
 package uk.me.parabola.util;
 
+import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
 import java.awt.Shape;
 import java.awt.geom.Path2D;
 import java.awt.geom.PathIterator;
 import java.awt.geom.Rectangle2D;
+import java.util.ArrayList;
 import java.util.Arrays;
+import java.util.List;
 
-// RWB new bits
-import java.util.ArrayList;
-import java.util.List;
-import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
 import uk.me.parabola.imgfmt.Utils;
 import uk.me.parabola.imgfmt.app.Area;
 import uk.me.parabola.imgfmt.app.Coord;
-
 import uk.me.parabola.log.Logger;
+import uk.me.parabola.util.GpxCreator;
 
 public class ShapeSplitter {
 	private static final Logger log = Logger.getLogger(ShapeSplitter.class);
@@ -321,14 +320,15 @@
 	 * @param origList list of shapes to which we append new shapes formed from above
 	 * @param fullArea of orig polygon. used for sign and handling of last line segment
 	 */
-	private static void processLineList(List lineInfo, List> origList, long fullArea) {
+	private static int processLineList(List lineInfo, List> origList, long fullArea) {
+		int errorCount = 0;
 		if (origList == null) // never wanted this side
-			return;
+			return errorCount;
 		MergeCloseHelper firstLine = lineInfo.get(0);
 		if (lineInfo.size() == 1) { // single shape that never crossed line
 			if (!firstLine.points.isEmpty()) // all on this side
 firstLine.closeAppend(origList, false);
-			return;
+			return errorCount;
 		}
 		// look at last item in list of lines
 		MergeCloseHelper lastLine = lineInfo.get(lineInfo.size()-1);
@@ -343,7 +343,7 @@
 		if (lineInfo.size() == 1) { // simple poly that crossed once and back
 			firstLine.setMoreInfo(0);
 			firstLine.closeAppend(origList, true);
-			return;
+			return errorCount;
 		}
 		// Above were the simple cases - probably 99% of calls.
 
@@ -360,7 +360,7 @@
 		boolean someDirectionsNotSet = false;
 		int areaDirection =

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

next patch also has a problem - sorry to waste your time.
Can I have your test data.

Ticker

On Thu, 2021-05-27 at 14:33 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> with my current test environment the patch doesn't improve the
> result.
> It reports some errors (the old code didn't always detect them) and
> eithers add or removes parts of the heavily merged shapes.
> 
> I've inspected one shape and found no obvious problem. No point is
> visited more then twice, but 167 points are visited twice. Let me
> know if you need the data.
> 
> Gerd
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 16:09
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Thanks Gerd, that works for me
> 
> Ticker
> 
> On Thu, 2021-05-27 at 12:54 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I first convert the gpx layer to a data layer, then add the tag
> > natural=water
> > Next execute validator which will complain that the way is not
> > closed.
> > Right click on that warning allows to "zoom to problem" .
> > This tells you where the first point is.
> > 
> > In the "OSM data" preferences I've enabled "Draw segment order
> > numbers on selected way"
> > 
> > Hope this helps.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 27. Mai 2021 14:21
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > Trying to use JOSM to decide if a gpx trace represents a self
> > -intersecting polygon is difficult. Is there a way of forcing it to
> > consider it closed and then check itself? Failing that, number the
> > points or segments somehow.
> > 
> > Ticker
> > 
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I think this can cope with any number of lines, in either/both
directions, following the same path. Also it should give better output
if same number of lines in both directions.

With debug, if it notices a problem, it generates a GPX trace of the
original and traces of all the resultant bits individually in a
subdirectory relevant to the split line.

I've tested it on a few examples, but I'll stress-test GBR again and
check the shapes where it detects problems.

I probably need to tidy up some message levels and comments.

Ticker

On Wed, 2021-05-26 at 11:17 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> I have to design a shape that does this and think what it means. With
> the extra handling needed for identical opposite lines it will be
> easy
> to detect. Maybe possible to just chuck it.
> 
> I'm out for the rest of the day so can't do anything until tomorrow.
> 
> Ticker
> 
> On Wed, 2021-05-26 at 08:46 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > OK, I think your patch improves some cases. I've also fixed another
> > potential cause for these problems with r4744.
> > 
> > I think I have to find a way to avoid those segments which are
> > visited multiple times in the same direction. There's probably
> > always
> > an alternative merge order that avoids this.
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/util/ShapeSplitter.java
===
--- src/uk/me/parabola/util/ShapeSplitter.java	(revision 4745)
+++ src/uk/me/parabola/util/ShapeSplitter.java	(working copy)
@@ -12,21 +12,20 @@
  */
 package uk.me.parabola.util;
 
+import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
 import java.awt.Shape;
 import java.awt.geom.Path2D;
 import java.awt.geom.PathIterator;
 import java.awt.geom.Rectangle2D;
+import java.util.ArrayList;
 import java.util.Arrays;
+import java.util.List;
 
-// RWB new bits
-import java.util.ArrayList;
-import java.util.List;
-import it.unimi.dsi.fastutil.longs.Long2ObjectOpenHashMap;
 import uk.me.parabola.imgfmt.Utils;
 import uk.me.parabola.imgfmt.app.Area;
 import uk.me.parabola.imgfmt.app.Coord;
-
 import uk.me.parabola.log.Logger;
+import uk.me.parabola.util.GpxCreator;
 
 public class ShapeSplitter {
 	private static final Logger log = Logger.getLogger(ShapeSplitter.class);
@@ -321,14 +320,15 @@
 	 * @param origList list of shapes to which we append new shapes formed from above
 	 * @param fullArea of orig polygon. used for sign and handling of last line segment
 	 */
-	private static void processLineList(List lineInfo, List> origList, long fullArea) {
+	private static int processLineList(List lineInfo, List> origList, long fullArea) {
+		int errorCount = 0;
 		if (origList == null) // never wanted this side
-			return;
+			return errorCount;
 		MergeCloseHelper firstLine = lineInfo.get(0);
 		if (lineInfo.size() == 1) { // single shape that never crossed line
 			if (!firstLine.points.isEmpty()) // all on this side
 firstLine.closeAppend(origList, false);
-			return;
+			return errorCount;
 		}
 		// look at last item in list of lines
 		MergeCloseHelper lastLine = lineInfo.get(lineInfo.size()-1);
@@ -343,7 +343,7 @@
 		if (lineInfo.size() == 1) { // simple poly that crossed once and back
 			firstLine.setMoreInfo(0);
 			firstLine.closeAppend(origList, true);
-			return;
+			return errorCount;
 		}
 		// Above were the simple cases - probably 99% of calls.
 
@@ -360,7 +360,7 @@
 		boolean someDirectionsNotSet = false;
 		int areaDirection = 0;
 		String diagMsg = "";
- 		for (MergeCloseHelper thisLine : lineInfo) {
+		for (MergeCloseHelper thisLine : lineInfo) {
 			thisLine.setMoreInfo(fullAreaSign);
 			if (thisLine.direction == 0)
 someDirectionsNotSet = true;
@@ -382,20 +382,58 @@
 		}
 		if (!diagMsg.isEmpty()) {
 			log.warn(diagMsg, "Probably self-intersecting polygon", fullAreaSign, someDirectionsNotSet, areaDirection);
-			for (MergeCloseHelper thisLine : lineInfo) {
-log.warn(thisLine);
-if (log.isDebugEnabled())
-	for (Coord co : thisLine.points)
-		log.debug("line", co, co.getHighPrecLat(), co.getHighPrecLon());
-			}
+			++errorCount;
 		}
 
 		lineInfo.sort(null);
+		errorCount += processDups(lineInfo);
 
 		int dummy = doLines(0, Integer.MAX_VALUE, null, lineInfo, origList);
 		assert dummy == lineInfo.size();
+		for (MergeCloseHelper thisLine : lineInfo)
+			errorCount += thisLine.errorCount;
+		return errorCount;
 	} // processLineList
 
+	private static int processDups(List lineInfo) {
+		// find groups of duplicates, drop equal numbers of different direction (ie keep just 1)
+		int errorCount = 0; // shouldn't be any
+		List newList = new ArrayList

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

There is something about "water.osm" that is breaking the original
logic of ShapeSplitter in a way I'm having trouble understanding. A
whole lot of points get assigned to 2 loops.

It doesn't seem to be related to multiple identical paths around the
cut-line, which should now work correctly with this patch - sorry about
the 2 failed attempts yesterday.

You mentioned slightly adjusting the high-prec values to help
shapeSplitter. I think it best not to do this. You'd have to get it
absolutely correct all the way through and, say, in cases where 3 lines
follow the same path it becomes impossible. Also the algo it makes some
decisions on very fine distinctions in areas which allow it to get rid
of spikes and similar and so this wouldn't happen.

Patch attached. with INFO/DEBUG enabled, it generates lots of gps
traces: original, the various loops either side of the cut-line and all
the resultant shapes.

Ticker
  

On Fri, 2021-05-28 at 05:03 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I've probably not understood the problem before. I've added test code
> to find situations where the ShapeSplitter complains and I found a
> rather simple case where the ShapeMerger produces a self-crossing
> shape (see attached gpx). Now I have an idea what to look for.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Donnerstag, 27. Mai 2021 18:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Ticker,
> 
> I've attached my current state of work as patch for the branch.
> It fixes some bugs and merges shapes in a way that is less likely to
> produce long connections to islands.
> 
> I think the main problem for shape splitter is when such a connection
> has a U-shape and the cut goes through it horizontally. But I'm just
> guessing...
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Donnerstag, 27. Mai 2021 17:59
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Ticker,
> 
> JOSM shows the islands properly when you zoom in. And yes, it shows
> water on both sides of the connecting lines. Very simiar to the
> Garmin renderer.
> Does that help?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 27. Mai 2021 17:48
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> I'm confused. After mp hole cutting and shape merging, I expect a
> single closed way that will have in/out lines, probably on top of
> each
> other, to connect each area that was a hole, eventually, to a point
> on
> the outside of the shape.
> 
> However, JOSM does show the water band on the outside of what were
> the
> islands, and on both sides of the joining lines.
> 
> Ticker
> 
> On Thu, 2021-05-27 at 15:10 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > yes, sure. I've converted a gpx output to OSM. All polygons with
> > holes are not valid OSM ways, but they are very normal as MapShape.
> > Validator doesn't help much with these polygons.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 27. Mai 2021 17:06
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > without
> > a log message
> > 
> > Hi Gerd
> > 
> > JOSM gives a self-crossing way validation error on this.
> > 
> > Ticker
> > 
> > On Thu, 2021-05-27 at 14:53 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > no need to hurry ;)
> > > Here is one:
> > > https://files.mkgmap.org.uk/download/508/water.osm
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Donnerstag, 27. Mai 2021 16:47
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] special case where splitting fails
> > > without
> > > a log message
> > > 
> > > Hi Gerd
> > > 
> > > next patch also has a problem - sorry to waste your time.
> > > Can I have your test data.
> > > 
> >

Re: [mkgmap-dev] low-res-opt branch: error message from ShapeSplitter for self-intersecting multipolygon

[Ticker Berkin]

Hi Gerd

The shape I'm getting in my build has a major problem with a twist
between the 2 parts. I'm having trouble reconciling it with the OSM
map. I've attached the original shape trace.

I agree that some of the messages do need tweaking, but, in this cases
there is a significant problem.

I'll consider the types of messages etc in the new version I'm working
on.

I'm away for a few days and won't be able to do much remotely.

Ticker

On Mon, 2021-06-07 at 07:22 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> please check https://files.mkgmap.org.uk/download/512/63240024.o5m
> with (unpatched) r4756 and these options --generate
> -sea=multipolygon,floodblocker --preserve-element-order --order-by
> -decreasing-area
> --allow-reverse-merge  
> 
> Produces an error message
> SCHWERWIEGEND (ShapeSplitter):
> e:\osm_out_work\norway\20210511_095323\63240024.o5m: Vertical split
> 36691968 failed on shape at 
> http://www.openstreetmap.org/?mlat=65.979056=12.308790=17 P
> ossibly a self-intersecting polygon
> for this invalid multipolygon : 
> https://www.openstreetmap.org/relation/5294624
> 
> The result doesn't look wrong, so I think there should either be no
> error message or we need code to remove those self-intersections or
> ignore invalid MP like that?
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devhttp://www.topografix.com/GPX/1/1; creator="mkgmap" version="1.1" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://www.topografix.com/GPX/1/1 
http://www.topografix.com/GPX/1/1/gpx.xsd;> 
V36691968_3074847/S_hp___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Proof of concept for better sea in overview map

[Ticker Berkin]

Hi Gerd

This is going to take some studying to work out the implications. Can't
do much for the next few days however, but will look carefully at the
end of the week.

Ticker

On Sun, 2021-06-06 at 14:02 +, Gerd Petermann wrote:
> Hi,
> 
> the attached patch improves the overview map, but so far only when
> precomp-sea is NOT used. 
> I tested it with --generate-sea=multipolygon,floodblocker  so that
> mkgmap really has a multipolygon with the natual=sea data.
> 
> For each level in the overview map it uses the original multipolygon
> data to compute the rings which will might visible at the given
> resolution. This requires more time compared to the current code but
> the result is much better and some fine tuning is possible.
> 
> To be able to use this also with --precomp-sea we need some changes
> in the code which generates sea.zip so that one multipolygon relation
> for each tile is stored. 
> 
> I've uploaded the results here: 
> https://files.mkgmap.org.uk/download/511/compare.7z
> What do you think?
> 
> Gerd
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

Thinking about this a bit more, there are still some cases where
adjacent shapes are next to cut-lines to get to holes (ie 3 or more
lines at same cut-point) that might not be ordered correctly. I think
it is possible to fix these by adding another sort structure.

Starting on this, I didn't like some of my new awkward shape handling
code and it has is a bug in the handling of the last element in the
list - fixed in the tidy-up

I've attached low-res-opt patch

Ticker

On Wed, 2021-06-02 at 14:58 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> That seems to work better. I still see some error messages but those
> are probably really invalid shapes produced by ShapeMerger. At least
> I see that one point is visited more than twice and that should not
> happen.
> 
> Maybe you can add unit tests to test the error cases?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 2. Juni 2021 16:06
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] special case where splitting fails without
> a log message
> 
> Hi Gerd
> 
> I've found and fixed some stupid mistakes - sorry for wasting your
> time.
> 
> Patch attached - works with your test case.
> 
> Ticker
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/util/ShapeSplitter.java
===
--- src/uk/me/parabola/util/ShapeSplitter.java	(revision 4755)
+++ src/uk/me/parabola/util/ShapeSplitter.java	(working copy)
@@ -287,6 +287,10 @@
 
 	private boolean multSameLow; // lineInfo.sort(comparator) might set this
 
+	// following are for processAwkward...
+	private MergeCloseHelper forwardLine, backwardLine;
+	private int directionBalance;
+
 	private void logMsg(Object ... olist) {
 		detectedProblems = true;
 		log.warn(olist);
@@ -411,12 +415,14 @@
 		//  Multiple hole balloons from the same point
 		//  Balloon(s) that share the same point as dLoops
 		boolean haveBalloons = false;
-		
+
 		// Duplicate dLoops in same direction can be removed / opposite direction cancel each other out.
 		// Do this before Balloon processing as dLoop removal can make some problems go away.
 		List newList = new ArrayList<>(lineInfo.size());
-		MergeCloseHelper forwardLine = null, backwardLine = null, lastLine = null;
-		int directionBalance = 0;
+		MergeCloseHelper lastLine = null;
+		forwardLine = null;
+		backwardLine = null;
+		directionBalance = 0;
 		boolean grouping = false;
 		for (MergeCloseHelper thisLine : lineInfo) {
 			if (lastLine != null) {
@@ -424,40 +430,14 @@
 	thisLine.lowPoint == lastLine.lowPoint &&
 	thisLine.highPoint == lastLine.highPoint &&
 	Math.abs(thisLine.areaToLine) == Math.abs(lastLine.areaToLine);
-if (grouping || sameAsLast) {
-	grouping = true;
-	if (lastLine.direction > 0) {
-		forwardLine = lastLine;
-		++directionBalance;
-	} else {
-		backwardLine = lastLine;
-		--directionBalance;
-	}
-}
-if (!sameAsLast) { // flush previous
-	if (grouping) {
-		if (directionBalance > 0)
-			newList.add(forwardLine);
-		else if (directionBalance < 0)
-			newList.add(backwardLine);
-		directionBalance = 0;
-		grouping = false;
-	} else
-		newList.add(lastLine);
-}
+fixDups(newList, lastLine, sameAsLast, grouping);
+grouping = sameAsLast;
 			}
 			lastLine = thisLine;
 			if (thisLine.lowPoint == thisLine.highPoint)
 haveBalloons = true;
 		}
-		// flush last
-		if (grouping) {
-			if (directionBalance > 0)
-newList.add(forwardLine);
-			else if (directionBalance < 0)
-newList.add(backwardLine);
-		} else
-			newList.add(lastLine);
+		fixDups(newList, lastLine, false, grouping);
 
 		if (newList.size() < lineInfo.size()) {
 			lineInfo.clear();
@@ -466,7 +446,7 @@
 
 		if (!haveBalloons)
 			return;
-		
+
 		// Balloons will be sorted earlier than dLoops that share the same lowPoint,
 		// but those that form a shape must be within a hole and those that form a hole must be within
 		// a shape and so might need moving.
@@ -486,29 +466,12 @@
 		for (MergeCloseHelper thisLine : lineInfo) {
 			if (lastLine != null) {
 boolean sameAsLast = thisLine.lowPoint == lastLine.lowPoint;
-if (grouping || sameAsLast) {
-	grouping = true;
-	if (lastLine.lowPoint != lastLine.highPoint)
-		dLoops.add(lastLine);
-	else if (lastLine.areaOrHole == 1)
-		shapes.add(lastLine);
-	else
-		holes.add(lastLine);
-}
-if (!sameAsLast) {
-	if (grouping) {
-		reordered |= fixOrder(newList, dLoops, shapes, holes);
-		grouping = false;
-	} else
-		newList.add(lastLine);
-}
+reordere

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I'm coming to the conclusion that water.osm is invalid.

I've attached an example part. This goes through the cut-line following
the same path in and out, and defines both shape and a hole at the same
time, which is impossible without the lines crossing.

Ticker

http://www.topografix.com/GPX/1/1; creator="mkgmap" version="1.1" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://www.topografix.com/GPX/1/1 
http://www.topografix.com/GPX/1/1/gpx.xsd;> 
V45802496_3182268/N4_hp___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

Some of the diags I've added to shapeSplitter might cause exceptions if
log.isDebugEnabled() and one of the sides of the split isn't required.
I didn't notice this for a while because just looking at the first part
and Main doesn't show exception traces.

The extra testing and error tracking are becoming unwieldy because of
the way it is structured so I think I should enclose it in a class and
tidy them up a bit - I'll do this next week.

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

The meaning of the preserved flag + handling of road/line/polygon etc
to different filters seem to be getting a bit mixed up.

I'm currently seeing some problems because of first/last points in line
being preserved (which some filters respect and others don't). I can
understand this for Roads, but not other lines. Is there a reason for
this.

preserveHorizonalAndVertical preserving multiple use points might be
good initially but could hinder removeObsolete spike removal from
removing the cuts to a collapsed hole.

Still haven't got to grips with shapeMergeFilter, but the main tile /
ovm_ generation is producing self-intersecting polygons.

Ticker

On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote:
> Hi Ticker / all,
> 
> it seems I've looked at the wrong test files. I've tested with a few
> more input files and now I see large white triangles with the branch
> version 4736 while trunk 4735 doesn't show them.
> Looks like a problem with preserving points or maybe special cases
> with ShapeSplitter...
> 
> Gerd
> 
> 
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Samstag, 22. Mai 2021 13:30
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> I've been doing experiments on tiles with much more sea and islands
> (tests before only had a bit of sea and lots of city).
> 
> With my re-ordering I get a very slightly smaller main tile (4096
> bytes
> in 5614K) and larger osmmap.img (34304 vs. 25088)
> 
> In both cases I'm getting quite a few shapeSplitter messages where it
> finds inconsistencies in the line directions across the split-line;
> normally due to self-intersection. These are happening in both the
> main
> tile generation and the overview combining.
> 
> Once shapeSplitter has generated this type of warning it is
> indeterminate which is shape and which is hole, so I'd expect gaps in
> the rendering.
> 
> I think trunk had eradicated split/self-intersection problems in the
> tile processing, including the ovm_ file generation.
> 
> I'm not sure about the overview combiner as, in my standard map
> -build,
> I don't merge shapes.
> 
> Ticker
> 
> On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I tried this patch with r4734 and I see no improvements. Maps are
> > larger, more white rectangles in sea.
> > 
> > Maybe my recent changes don't work well with yours? Do you have an
> > example where it improves something with r4734?
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Freitag, 21. Mai 2021 18:10
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> > 
> > Hi Gerd
> > 
> > I'd been doing some investigation of filters ordering (based on
> > trunk).
> > I'd also done the pre-filtering of lines & shapes by minRes.
> > 
> > My conclusions are:
> > 
> > Shapes:
> > It is better to run SizeFilter after RemoveObsoleteFilter.
> > It is more efficient to run DP filter after both of these.
> > 
> > Lines:
> > LineSplitterFilter should be run after everything that can remove
> > points.
> > It is more efficient to run DP after RemoveObsoleteFilter.
> > 
> > I've adapted my changes into a patch for the low-res-opt branch,
> > along
> > with removal of some resolution tests that are now redundant.
> > 
> > For the contourFilters, I've left DP as the first filter but moved
> > LineSplitter as for the normalFilters
> > 
> > Ticker
> > 
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

See other thread about DP / removeObsolete ordering. RemoveObsolete
tidies up the remains after DP has done things it shouldn't

For non-road lines re. first and last - I don't think any filters will
cause there not to be a point at the res-coord of the start or end, so
I don't see that there would be a gap for different types. 

ShapeSplitter will notice if the dividing line has inconsistent (non
-alternating) direction crossings (probably caused by self
-intersection). It can't deal with this, just issues a warning. It
won't notice self-intersection elsewhere and allocates "outside" before
the first line crossing, "inside" next and alternates. I think it is
possible to come up with shapes where this is the opposite what you'd
expect.

Maybe we need a proper intersecting segments checker. I've investigated
some algos. Not sure what to do with self-intersection when we find it.
Lots are simply output in RGN and devices might have some defined way
of rendering. It's only the cutting and the overview merging that don't
really have a solution.

Ticker

On Mon, 2021-05-24 at 12:20 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> the removeObsolete filter sometimes ignores the preserved status, and
> probably it doesn't make sure to keep or copy that status. Maybe that
> explains why a following DP gives other results? I think in the
> current order there is no problem with this but I'll double check
> because I think somewhere mkgmap removes points which should be
> preserved.
> 
> The first/last point of lines is preserved to avoid gaps between
> connected lines of different types, e.g. when a stream gets a river.
> 
> The shape merger produces self intersecting shapes. I think this is
> not a problem as long as ShapeSplitter can handle this. Can it?
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 24. Mai 2021 12:54
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> The meaning of the preserved flag + handling of road/line/polygon etc
> to different filters seem to be getting a bit mixed up.
> 
> I'm currently seeing some problems because of first/last points in
> line
> being preserved (which some filters respect and others don't). I can
> understand this for Roads, but not other lines. Is there a reason for
> this.
> 
> preserveHorizonalAndVertical preserving multiple use points might be
> good initially but could hinder removeObsolete spike removal from
> removing the cuts to a collapsed hole.
> 
> Still haven't got to grips with shapeMergeFilter, but the main tile /
> ovm_ generation is producing self-intersecting polygons.
> 
> Ticker
> 
> On Mon, 2021-05-24 at 06:52 +, Gerd Petermann wrote:
> > Hi Ticker / all,
> > 
> > it seems I've looked at the wrong test files. I've tested with a
> > few
> > more input files and now I see large white triangles with the
> > branch
> > version 4736 while trunk 4735 doesn't show them.
> > Looks like a problem with preserving points or maybe special cases
> > with ShapeSplitter...
> > 
> > Gerd
> > 
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Samstag, 22. Mai 2021 13:30
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> > 
> > Hi Gerd
> > 
> > I've been doing experiments on tiles with much more sea and islands
> > (tests before only had a bit of sea and lots of city).
> > 
> > With my re-ordering I get a very slightly smaller main tile (4096
> > bytes
> > in 5614K) and larger osmmap.img (34304 vs. 25088)
> > 
> > In both cases I'm getting quite a few shapeSplitter messages where
> > it
> > finds inconsistencies in the line directions across the split-line;
> > normally due to self-intersection. These are happening in both the
> > main
> > tile generation and the overview combining.
> > 
> > Once shapeSplitter has generated this type of warning it is
> > indeterminate which is shape and which is hole, so I'd expect gaps
> > in
> > the rendering.
> > 
> > I think trunk had eradicated split/self-intersection problems in
> > the
> > tile processing, including the ovm_ file generation.
> > 
> > I'm not sure about the overview combiner as, in my standard map
> > -build,
> > I don't merge shapes.
> > 
> > Ticker
> > 
> > On Fri, 2021-05-21 at 16:41 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > >

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

Not sure about this. It is a very drastic change and might break a lot
of the element>subdiv code (esp --order-by). Many element well be too
big and have to be split again.

It ends up as a merge attempt at all lines/shapes in the tile, so it
might as well be resolution independent!

Ticker
On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I think I found my mistake. The code to re-calc the preserved points
> was executed for each sub division instead of once per level.
> 
> The attached patch might already solve this.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Douglas Peucker Filter and short lines

[Ticker Berkin]

Hi Gerd / anyone else

Attempting to improve filtering by considering their optimal order, I
got much worse results at very low resolutions. The excess was lots of
1-res-unit lines (these came from the coastlines of lots of small
islands)

These lines start off sufficiently large to make it through the
sizeFilter.

Running the DP filter next, if intermediate points are less than -
-reduce-point-density (default 2.6) res-units from the line between
first and last point, they are dropped. This can result in a single
point if the first and last points are equal, or a line shorter than at
least 2 of the sub-lines that no longer exist.

I'm not sure of the best approach to deal with this. I considered
inhibiting DP if the distance between start (either overall or for
the internal sub-processing) is less than the error distance, but this
is wrong for polygons or lines that loop back towards their starting
point. Maybe the DP error size must be less than sizeFilter size; For
lines this is MIN_LINE_SIZE = 1 and not configurable, For polygons it
is --min-size-polygon (default 8) or --polygon-size-limits.

Running RemoveObsoleteFilter before or after DP effects the final
result. Considering the functions of the filters, I think
RemoveObsolete should be before DP; DP shouldn't produce obsolete
points and there is no point in DP considering obsolete points.

Ticker

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Ah

When I scanned this area a while ago I guessed, without looking
carefully or thinking, that mergeShapesFirst took all the shapes from
all the ovm_ imgs and attempted to merge them. I see now that it
doesn't. I don't know what problem it can solve better than the
standard subdivision merge. It might have been there because, a long
time ago, outer zoom subdivs (esp. overview) were far too small when
there was anything (sea/coastline) because the content #points was
calculated at res24. 

So, yes, I'd try without.

Ticker

On Mon, 2021-05-24 at 14:19 +, Gerd Petermann wrote:
> Oops, I meant
>  should I remove mergeShapesFirst() completely?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Montag, 24. Mai 2021 16:07
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Ticker,
> 
> you mean I should mergeShapesFirst() completely?
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 24. Mai 2021 15:57
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> Not sure about this. It is a very drastic change and might break a
> lot
> of the element>subdiv code (esp --order-by). Many element well be too
> big and have to be split again.
> 
> It ends up as a merge attempt at all lines/shapes in the tile, so it
> might as well be resolution independent!
> 
> Ticker
> On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think I found my mistake. The code to re-calc the preserved
> > points
> > was executed for each sub division instead of once per level.
> > 
> > The attached patch might already solve this.
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

Not quite related, but if merging the shapes from all ovm_ imgs, then
maybe some types of filtering should be turned off when the contents
for these is being generated; esp DP, and probably straigh-line removal
in removeObsolete

Ticker

On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> sorry, I can't follow. The old code only preserved points on the
> boundary of the merged shape, but the outer ring can be far from
> rectangular.
> This way some important points on outer ring were not preserved.
> 
> There's still a possible problem with large white area in sea which
> seems to depend on the order in which shapes are merged. Sill trying
> to understand what goes, but at least I can reproduce it...
> 
> Gerd
> 
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 24. Mai 2021 17:11
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> Re. last commit: preserve all horizontal and vertical lines in
> shapes.
> 
> Do the cut lines go right across the outer polygon or just in from
> one
> side to the hole? If all the way across, might this change inhibit
> the
> merging to one side of the hole, or, if divided for some other reason
> in the other direction, merging across the now unneeded cut line.
> 
> Ticker
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

That would be the conclusion - but I don't think you should do it.

I don't see that making the problem bigger will help!

Ticker

On Mon, 2021-05-24 at 14:07 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> you mean I should mergeShapesFirst() completely?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 24. Mai 2021 15:57
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> Not sure about this. It is a very drastic change and might break a
> lot
> of the element>subdiv code (esp --order-by). Many element well be too
> big and have to be split again.
> 
> It ends up as a merge attempt at all lines/shapes in the tile, so it
> might as well be resolution independent!
> 
> Ticker
> On Mon, 2021-05-24 at 13:33 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think I found my mistake. The code to re-calc the preserved
> > points
> > was executed for each sub division instead of once per level.
> > 
> > The attached patch might already solve this.
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

Re. last commit: preserve all horizontal and vertical lines in shapes.
 
Do the cut lines go right across the outer polygon or just in from one
side to the hole? If all the way across, might this change inhibit the
merging to one side of the hole, or, if divided for some other reason
in the other direction, merging across the now unneeded cut line.

Ticker

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

This seems to be false when combining the ovm_, rather than when
generating them!

It suppresses RoundCoords but maybe shouldn't. Not sure why RoundCoords
should ever be suppressed when res < 24?

It doesn't suppress removeObsolete, which removes points in any
straight line. 

I suppose the simpler way to view this is that unique points per
element on the tile boundary should be kept; making the job of the
overview combiner mergeShape possible.

Ticker

On Mon, 2021-05-24 at 15:35 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> yes, it is. See usage of enableLineCleanFilters
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 24. Mai 2021 17:33
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> 
> Hi Gerd
> 
> Not quite related, but if merging the shapes from all ovm_ imgs, then
> maybe some types of filtering should be turned off when the contents
> for these is being generated; esp DP, and probably straigh-line
> removal
> in removeObsolete
> 
> Ticker
> 
> On Mon, 2021-05-24 at 15:21 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > sorry, I can't follow. The old code only preserved points on the
> > boundary of the merged shape, but the outer ring can be far from
> > rectangular.
> > This way some important points on outer ring were not preserved.
> > 
> > There's still a possible problem with large white area in sea which
> > seems to depend on the order in which shapes are merged. Sill
> > trying
> > to understand what goes, but at least I can reproduce it...
> > 
> > Gerd
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Montag, 24. Mai 2021 17:11
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Problems with sea in overview map
> > 
> > Hi Gerd
> > 
> > Re. last commit: preserve all horizontal and vertical lines in
> > shapes.
> > 
> > Do the cut lines go right across the outer polygon or just in from
> > one
> > side to the hole? If all the way across, might this change inhibit
> > the
> > merging to one side of the hole, or, if divided for some other
> > reason
> > in the other direction, merging across the now unneeded cut line.
> > 
> > Ticker
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] special case where splitting fails without a log message

[Ticker Berkin]

Hi Gerd

I've added code to deal with some variants of the case as described - I
hope this will be enough to cope with more complex shapes generated by
ShapeMerger.

There might be a bit more I can do without having to resort to more
complex geometry analysis if it still gives problems. 

I've also restructured it a bit.

Patch attached based on low-res-opt. Trunk version will be the same
(but the patch would be different)

Ticker

On Mon, 2021-05-31 at 17:12 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> shapeSplitter will have problems (ie get it wrong some of the time)
> where there are in/out lines to a hole that share the same cut point
> as
> a line that is the boundary between a shape and hole; could be many
> holes (or shapes) and many lines. The simple sort/dedupe I was doing
> isn't adequate. I'll come up with something better tomorrow.
> 
> Ticker 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/util/ShapeSplitter.java
===
--- src/uk/me/parabola/util/ShapeSplitter.java	(revision 4752)
+++ src/uk/me/parabola/util/ShapeSplitter.java	(working copy)
@@ -275,11 +275,23 @@
 		return pointsToPath2D(outputList, countVals);
 	}
 
-/* Dec16/Jan17. Ticker Berkin. New implementation for splitting shapes.
+// Dec16/Jan17. Ticker Berkin. New implementation for splitting shapes and clipping
 
-Eventually maybe can be used instead of some of the above and elsewhere
-*/
+	private boolean detectedProblems;
+	private List newLess, newMore;
+	private String gpxDirectory;
+	private long fullArea;
 
+	private List lineInfo; // for the side we are working on
+	private List> origList; // ditto
+
+	private boolean multSameLow; // lineInfo.sort(comparator) might set this
+
+	private void logMsg(Object ... olist) {
+		detectedProblems = true;
+		log.warn(olist);
+	}
+
 	/**
 	 * Service routine for processLineList. Processes a nested list of holes within a shape or
 	 * list of shapes within a hole.
@@ -290,11 +302,8 @@
 	 * @param endEnclosed point where starting line ends on dividing line.
 	 * @param addHolesToThis if not null, then called from a shape and subtract holes from it
 	 * otherwise new shapes within a hole.
-	 * @param lineInfo list of lines.
-	 * @param origList list of shapes to which we append new shapes.
 	 */
-	private static int doLines(int startInx, int endEnclosed, MergeCloseHelper addHolesToThis,
-   List lineInfo, List> origList) {
+	private int doLines(int startInx, int endEnclosed, MergeCloseHelper addHolesToThis) {
 		int inx = startInx;
 		final boolean calledFromHole = addHolesToThis == null;
 		while (inx < lineInfo.size()) {
@@ -304,9 +313,9 @@
 			if (thisLine.lowPoint == endEnclosed && thisLine.highPoint == endEnclosed) // consider carefully
 if (calledFromHole == (thisLine.areaOrHole == -1))
 	break; // stop if same type
-			inx = doLines(inx+1, thisLine.highPoint, calledFromHole ? thisLine : null, lineInfo, origList);
+			inx = doLines(inx+1, thisLine.highPoint, calledFromHole ? thisLine : null);
 			if (calledFromHole) // handling list of shapes
-thisLine.closeAppend(origList, true);
+thisLine.closeAppend(true);
 			else // handling list of holes
 addHolesToThis.addHole(thisLine);
 		}
@@ -317,17 +326,17 @@
 	 * Service routine for splitShape. Takes list of lines and appends distinct shapes
 	 * @param lineInfo list of lines that start and end on the dividing line (or orig startPoint)
 	 * @param origList list of shapes to which we append new shapes formed from above
-	 * @param fullArea of orig polygon. used for sign and handling of last line segment
 	 */
-	private static int processLineList(List lineInfo, List> origList, long fullArea) {
-		int errorCount = 0;
+	private void processLineList(List lineInfoParam, List> origListParam) {
+		lineInfo = lineInfoParam;
+		origList = origListParam;
 		if (origList == null) // never wanted this side
-			return errorCount;
+			return;
 		MergeCloseHelper firstLine = lineInfo.get(0);
 		if (lineInfo.size() == 1) { // single shape that never crossed line
 			if (!firstLine.points.isEmpty()) // all on this side
-firstLine.closeAppend(origList, false);
-			return errorCount;
+firstLine.closeAppend(false);
+			return;
 		}
 		// look at last item in list of lines
 		MergeCloseHelper lastLine = lineInfo.get(lineInfo.size()-1);
@@ -335,14 +344,14 @@
 			lineInfo.remove(lineInfo.size()-1);
 		else { // ended up on this side and must have crossed the line
 			// so first element is really the end of the last
-			lastLine.combineFirstIntoLast(firstLine, fullArea);
+			lastLine.combineFirstIntoLast(firstLine);
 			lineInfo.remove(0);
 			firstLine = lineInfo.get(0);
 		}
 		if (lineInfo.size() == 1) { // simple poly that crossed once and back
 	

Re: [mkgmap-dev] [mkgmap-svn] Commit r4749: remove/disable code to merge shapes to the max for now:

[Ticker Berkin]

Hi Gerd

Do you have an example of this triangle / concave shape? Are you sure
there isn't a rotation through the connection point, resulting in
something like a figure-of-8.

Ticker

On Mon, 2021-05-31 at 07:41 +0100, svn commit wrote:
> Version mkgmap-r4749 was committed by gerd on Mon, 31 May 2021
> BRANCH: low-res-opt
> remove/disable code to merge shapes to the max for now:
> - hole detection was too simple, sometimes removed outer parts of
> shapes 
> - ShapeSplitter doesn't handle some degenarated shapes well where
> parts are connected in a single point, like a triangle connected to a
> concave shape.
> 
> Will try again with a different approach once ShapeSplitter is able
> to handle these cases.
> 
> http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4749
> ___
> mkgmap-svn mailing list
> To unsubscribe send an mail to mkgmap-svn-le...@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-svn
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] new splitter branch solve-parallel

[Ticker Berkin]

Hi Gerd

Is this of any relevance -

https://en.wikipedia.org/wiki/Branch_and_bound

Ticker

On Wed, 2021-06-30 at 16:08 +, Gerd Petermann wrote:
> Hi all,
> 
> I've started this branch to further improve the split results. 
> Splitter has two different algorithms to find good splits. 
> 1) Algo FULL tries first to split in the middle and then continues
> with the next positions (mid+1, mid-1, mid+2, mid-2, ...) 
> The resulting parts are split again recursively. This should find the
> best possible split but can take very long when the only good split
> starts far from the middle. 
> 
> 2) Algo SOME uses some heuristics to test only some of the possible
> splits. This is typically much faster but sometimes doesn't find any
> solution and sometimes the FULL algo finds a better solution. 
> 
> The trunk version tries first the one that is more promising and - if
> that fails to find a good split - tries the other. So, sometimes the
> FULL algo works for 2 minutes and then SOME finds a good result in a
> few seconds.
> 
> This branch executes both algorithms in parallel and uses the better
> result. 
> 
> One of the open problems is to decide under what conditions the
> execution should stop.
> If the SOME algo finds a good solution within 20 secs should we still
> continue the FULL algo to find the best solution? If yes, how long?
> I'm playing with this, any input is welcome.
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] faster-mp r4797 and a question on style for amenity=emergency_phone

[Ticker Berkin]

Hi Karl

polygons near here, eg

https://www.openstreetmap.org/way/215482946
https://www.openstreetmap.org/way/215482937

look to be a bit of a mess - one shows a big spike. If you go to  
40.777236,0.641794 and query features at this point, the hover previews
don't correspond very well. 

Ticker


On Sun, 2021-07-11 at 09:25 +, Gerd Petermann wrote:
> Hi Karl,
> 
> regarding "After running, it leaves a directory called e: with
> subdirectires such as:"
> That was not intended, see 
> https://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4803
> 
> reg. amenity=emergency_phone:
> According to taginfo there is no such tag in OSM.
> Maybe emergency=phone was meant?
> https://wiki.openstreetmap.org/wiki/Tag:emergency%3Dphone
> 
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von 7770 <7...@foskan.eu>
> Gesendet: Sonntag, 11. Juli 2021 11:05
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] faster-mp r4797 and a question on style for
>amenity=emergency_phone
> 
> Hi.
> 
> One observation with faster-mp-r4797.
> After running, it leaves a directory called e: with subdirectires
> such as:
> ld/H121664384_29910
> ld/V-3407872_1992959
> 
> It's ok i presume, these correlate to SEVERE message that are created
> during
> processing:
>  (ShapeSplitter): 77800027.osm.pbf: Horizontal split 121664384
> failed on shape at 40.777236,0.641794 Possibly a self-intersecting
> polygon
> 
> 
> 
> Second, a question.
> In the style for points there is a line for emergacy phone (and later
> phone).
> amenity=emergency_phone [0x2f12 resolution 24 default_name 'Emergency
> Phone']
> 
> I have added a specific image in the TYP file, which compiles well,
> but nothing
> seems to show on the map (even checked with qmapshack).
> Is there something specific to this tag or this number 0x2f12 that
> supresses it
> from being displayed?
> 
> Regards
> Karl
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Problems with sea in overview map

[Ticker Berkin]

Hi Gerd

I often see other polygons becoming self-intersecting at lower
resolutions (or at least GPSMapEdit does). If the DP filter is doing
this then maybe this should be tacked first, with an alternate version
for polygons - I have no idea how this would be implemented.

Should shapeMergeFilter somehow consider each polygon as if its points
are rounded to resolution, with adjacent duplicates removed. This
should allow more merging and will be much less likely to be inhibited
by the 256 point limit. Doing this, small holes/islands are likely to
disappear.

I don't like the idea of trying to delay hole cutting per resolution,
and I don't see quite how it would work for pre-comp-sea

Ticker

On Wed, 2021-05-19 at 07:06 +, Gerd Petermann wrote:
> Hi all,
> 
> I've identified a few problems with the handling of natural=sea in
> areas with lots of islands, e.g. coast of Norway. Thouands of rather
> small polygons are created, esp. when also natual=land is rendered. 
> 1. Typical tiles are rather large, they span several precompiled sea
> tiles if --precomp-sea=sea.zip is used. As of now, the polygons from
> different sea-tiles are never merged. This might be improved to
> reduce file size and possibly also improve rendering
> 2. Even at resolution 12 the curent code distributes the thousands of
> tiles into multiple sub divisions before ShapeMerger is executed for
> each subdiv. Shapes (polygons) are only merged when the number of
> points for one shape stays inside the IMG limit of 256 points, so the
> merger cannot do much.
> 3. Douglas-Peucker (DP) filter is used for the merged shapes which
> contain holes. This gives weird results because the invisible lines
> which connect the holes are causing heavy self-intersections at low
> resolutions. I think this is one reason for white triangles in the
> sea.
> 
> I am experimenting with these ideas:
> - At low resolutions the ShapeMerger could merge more so that shapes
> with more than > 256 points are produced, assuming that DP filter
> will remove many of them. Not sure if this works with shapes that
> have holes.
> - At low resolutions it is likely/wanted that small islands
> disappear, so we should remove those first, together with the lines
> that connect them with the outer ring.
> - The simplified sea polygons without holes should be merged as much
> as possible before using any filter, maybe DP should be executed
> first on the merged shapes so that fever subdivs are created.
> 
> Each method has its own effects and I'm searching for the best
> combination and order. 
> 
> A lot of this is only possible because we now have Tickers code that
> allows multipolygon splitting without using
> java.awt.geom.Area.intersect() . Tickers code in ShapeSplitter seems
> to be able to handle the holes properly. 
> 
> Gerd
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

A few more things I've discovered.

GPSMapEdit will show bands of emptyness for a thin, correct, slice when
you zoom out but with the view zoom level locked at a higher resolution
than it expects. This confused some of my previous findings.

Changing some sizes changes the behaviour and I sometimes get the holes
at level 1 but level 2 is OK.

3 of the edges of the hole correspond to cut-lines to get at the
cluster of 'islands', but the 4th corresponds to the subdivision max
-size splitting, so it looks like the problem might be related to
MapBuilder/MapSplitter/MapArea logic.

With option --x-no-mergeshapes (and current trunk) it appears that
cutting to this island group happens twice from the left, with 1 line
continuing to the right and twice from the top, with 1 line continuing
to the bottom.

Ticker

On Sun, 2021-04-25 at 13:50 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> More more testing with standard style / options etc. As well as -
> -order
> -by, I had some level params that confused.
> 
> With GPSMapEdit at Zoom level 2, there is a wide cut from the LHS
> forest triangle to the holes around 54.9953,-58.2209. It seems to be
> the total height of the holes.
> 
> Similar shows in MapSource, but it much more difficult to find.
> Resolution needs to be <= 7mi before forest kicks-in. I didn't
> investigate how far left the blank area went.
> 
> Looking more closely on Basecamp, the blank area is a rectangle
> approx
> 54.975,-58.302 to 55.0178,-58.6247, ie it doesn't go to the edge of
> the
> forest triangle. Also it only shows when detail-level is set to
> "highest"
> 
> Ticker
> 
> On Sun, 2021-04-25 at 11:03 +0100, Ticker Berkin wrote:
> > Hi Gerd
> > 
> > Just tried this with latest trunk and my standard environment
> > (mainly
> >  --order-by-decreasing-area
> > and get very inconsistent results.
> > 
> > On GPSMapEdit seems reasonable - shows forest triangle and, within,
> > a
> > small area with a number of holes.
> > 
> > MapSource shows a white square, no forest colour and I can't move
> > the
> > map around, also it doesn't seem to zoom out enough. First time I
> > looked I saw 2 white squares.
> > 
> > Basecamp shows the forest triangle, but I can't find the
> > islands/holes
> > 
> > I've had problems before with MapSource or Basecamp caches, so I'll
> > delete these. Also try with simple options etc
> > 
> > Ticker
> > 
> > 
> > On Sun, 2021-04-25 at 07:31 +, Gerd Petermann wrote:
> > > Hi Experts,
> > > 
> > > while working on the faster-mp branch I experimented with the
> > > monster
> > > relation r9488835. I added tag landuse=forest to it so that it is
> > > actually rendered and found huge (wrong) empty rectangles in the
> > > forest.
> > > The original file is very complex but the same problem also
> > > occurs
> > > with a simplified file, see 
> > > http://files.mkgmap.org.uk/download/505/bad-map-split.osm.pbf
> > > Just use e.g.
> > > java -jar mkgmap.jar --gmapi bad-map-split.osm.pbf 
> > > Problem: Basecamp and Mapsource don't show this map, GpsMapEdit
> > > shows
> > > a large white rectangle near 54.97743 -58.30292
> > > 
> > > I assume that we hit some unknown limit , possibly in
> > > LinePreparer
> > > or
> > > MapSplitter. 
> > > I've also tried some older versions, so far they all show the
> > > problem.
> > > Any idea where to look? 
> > > 
> > > RgnDisplay shows this data for the problemat area:
> > > - Level 0, Subdiv 279 ---
> > > --
> > > --
> > > ---
> > >  || | final at 265f, (end
> > > 25e2)
> > >  || | polygon start 2634
> > > to
> > > 265f
> > > 2634 | 00 | 4b  | Polygon 1
> > >  || | Type 0x4b
> > > 2635 | 01 | 00 00 00| Label offset:
> > > 0x00
> > > ()
> > > 2638 | 04 | d0 3b   | Long delta 15312
> > > 263a | 06 | 2a 3a   | Lat delta 14890
> > >  || | Start point
> > > 55.24913,
> > > -61.91139 2574796/-2885280
> > > 263c | 08 | 0c  | Bitstream length 12
> > > 263d | 09 | bb  | Base bb
> > > 263e | 0a | 06 43 0

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

I think I'm getting somewhere with this and will try and have a proper
fix in a day or so. It is related to subdivision splitting.

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd and others

The cause for this unrendered area is as follows:

A large multipolygon (larger than the maximum size for a level 1
subdiv) exists, and is broken into some similar size pieces and some
smaller pieces to expose inners/holes. The same problem can be caused
by any polygons having these size properties.

Option --order-by-decreasing-area is not in effect (with this option
the problem goes away).

At the outermost level, the large pieces are split into the grid of
subdivs necessary to be within size limits. Small pieces that fit
within a subdiv are allocated to the correct one. Intermediate pieces
that extend out of the nominal subdiv bounds, but don't exceed the
actual addressing limits, are allocated to an existing subdiv based on
their centre. Larger pieces that fit in a subdiv but would cause size
problems if added to an existing subdiv are given a subdiv of their
own.

At the next level down, each of the above subdivs will required
splitting to fit within the addressing limits. The logic that did this
used the subdiv bounds and split this into a grid. Then every element
in the outer MapArea is processed as above (ie split, safely placed,
placed with overlap or own subdiv)

The problem is that the splitting algorithm simply goes through the
child subdiv, clips the polygon to this size and saves the bits that
are within the area. Polygon parts that are outside the defined outer
level never get noticed!

There are various possible solutions:

Method 1: use the rigorous polygon splitting into subdivs that --order
-by-decreasing-area uses for levels above level 0. This is simple to
implement, safe, mergeShapes might recombine more cut fragments as they
are all together. Disadvantages are that, for a normal map, more
polygons might be split, leading to larger RGN size (but not for bad
-map-split.osm.pbf). The overall size is still less that with --order
-by...

Method 2: When the child subdiv grid is created, use any expanded size
of the parent subdiv. This stops overhanging bits of polygons being
lost. Again, simple to implement. It results in overlapping subdivs at
lower levels and I don't know this causes other problems, but probably
not; this used to be done and still does for some sizes of polygons and
lines. Some shape-merging won't happen because the joining bits are in
a different subdiv. 

Method 3: Allocated oversize polygons to their own subdiv. Again,
simple to implement, but I see no advantage over Method 2. Even more
shape-merging might not happen.

Method 4: Detect when an object will have bits missed and semi-cut
rather than clip them to continue having an oversize part in the child
subdiv. This is more complicated to implement but, if done, could
replace the clipping code and be more efficient. If a parent subDiv is
divided into 2 children, it already happens like this.

Although I prefer Method 1, I suspect Method 2 is what should be done
in the short-term and so I've attached a patch that is configured for
Method 2, but, with a couple of edits to change some flag handling,
does Method 1.

Ticker

Index: src/uk/me/parabola/mkgmap/build/MapArea.java
===
--- src/uk/me/parabola/mkgmap/build/MapArea.java	(revision 4685)
+++ src/uk/me/parabola/mkgmap/build/MapArea.java	(working copy)
@@ -257,14 +257,16 @@
 	continue;
 }
 int areaIndex = pickArea(mapAreas, e, xbaseHp, ybaseHp, nx, ny, dxHp, dyHp);
-if ((shapeBounds.getHeight() > maxHeight || shapeBounds.getWidth() > maxWidth) &&
-!areas[areaIndex].contains(shapeBounds)) {
+if (areas[areaIndex].contains(shapeBounds)) {
+	mapAreas[areaIndex].addShape(e);
+} else if (shapeBounds.getHeight() <= maxHeight && shapeBounds.getWidth() <= maxWidth) {
+	// this polygon is partially outside the subdivision, but won't exceed the allowable offsets
+	mapAreas[areaIndex].addShape(e);
+} else {
 	MapArea largeObjectArea = new MapArea(shapeBounds, areaResolution, true); // use splitIntoAreas to deal with overflow
 	largeObjectArea.addShape(e);
 	addedAreas.add(largeObjectArea);
-	continue;
 }
-mapAreas[areaIndex].addShape(e);
 			}
 		}
 
Index: src/uk/me/parabola/mkgmap/build/MapBuilder.java
===
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java	(revision 4685)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java	(working copy)
@@ -803,11 +803,15 @@
 			List nextList = new ArrayList<>();
 
 			Zoom zoom = map.createZoom(linfo.getLevel(), linfo.getBits());
+			//%%%mbs see also MapSpltter.split()
+			// method 1: true at levels > 0, depends on --order-by for level 9
+			// method 2: depends on --order-by for all levels
+			boolean splitPolygonsIntoArea = /* linfo.getLevel() != 0 ? true : */ orderByDecreasingArea;
 
 			for (SourceSubdiv srcDivPair : srcList) {
 
 MapSplitter splitter = new MapSplitter(srcDivPair.getSource(), zoom);
-MapArea[] 

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

Here is patch configured for Method 1, with more clarity about how to
change.

I think the best solution will be a super Method 4. For a 5 x 4
subdivision grid it will reduces the number of polygon cuts from 80
(5*4*4) to 19 (5-1+5*(4-1)). 

It shouldn't be too difficult to implement and I'll look at it next
week.

Ticker

On Wed, 2021-04-28 at 09:14 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> please create two separate patches for method 1 and 2. It's not
> obvious to me what changes are needed to get method 1.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 27. April 2021 13:11
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Empty rectangles in map
> 
> Hi Gerd and others
> 
> The cause for this unrendered area is as follows:
> 
> A large multipolygon (larger than the maximum size for a level 1
> subdiv) exists, and is broken into some similar size pieces and some
> smaller pieces to expose inners/holes. The same problem can be caused
> by any polygons having these size properties.
> 
> Option --order-by-decreasing-area is not in effect (with this option
> the problem goes away).
> 
> At the outermost level, the large pieces are split into the grid of
> subdivs necessary to be within size limits. Small pieces that fit
> within a subdiv are allocated to the correct one. Intermediate pieces
> that extend out of the nominal subdiv bounds, but don't exceed the
> actual addressing limits, are allocated to an existing subdiv based
> on
> their centre. Larger pieces that fit in a subdiv but would cause size
> problems if added to an existing subdiv are given a subdiv of their
> own.
> 
> At the next level down, each of the above subdivs will required
> splitting to fit within the addressing limits. The logic that did
> this
> used the subdiv bounds and split this into a grid. Then every element
> in the outer MapArea is processed as above (ie split, safely placed,
> placed with overlap or own subdiv)
> 
> The problem is that the splitting algorithm simply goes through the
> child subdiv, clips the polygon to this size and saves the bits that
> are within the area. Polygon parts that are outside the defined outer
> level never get noticed!
> 
> There are various possible solutions:
> 
> Method 1: use the rigorous polygon splitting into subdivs that -
> -order
> -by-decreasing-area uses for levels above level 0. This is simple to
> implement, safe, mergeShapes might recombine more cut fragments as
> they
> are all together. Disadvantages are that, for a normal map, more
> polygons might be split, leading to larger RGN size (but not for bad
> -map-split.osm.pbf). The overall size is still less that with --order
> -by...
> 
> Method 2: When the child subdiv grid is created, use any expanded
> size
> of the parent subdiv. This stops overhanging bits of polygons being
> lost. Again, simple to implement. It results in overlapping subdivs
> at
> lower levels and I don't know this causes other problems, but
> probably
> not; this used to be done and still does for some sizes of polygons
> and
> lines. Some shape-merging won't happen because the joining bits are
> in
> a different subdiv.
> 
> Method 3: Allocated oversize polygons to their own subdiv. Again,
> simple to implement, but I see no advantage over Method 2. Even more
> shape-merging might not happen.
> 
> Method 4: Detect when an object will have bits missed and semi-cut
> rather than clip them to continue having an oversize part in the
> child
> subdiv. This is more complicated to implement but, if done, could
> replace the clipping code and be more efficient. If a parent subDiv
> is
> divided into 2 children, it already happens like this.
> 
> Although I prefer Method 1, I suspect Method 2 is what should be done
> in the short-term and so I've attached a patch that is configured for
> Method 2, but, with a couple of edits to change some flag handling,
> does Method 1.
> 
> Ticker
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/mkgmap/build/MapArea.java
===
--- src/uk/me/parabola/mkgmap/build/MapArea.java	(revision 4687)
+++ src/uk/me/parabola/mkgmap/build/MapArea.java	(working copy)
@@ -257,14 +257,16 @@
 	continue;
 }
 int areaIndex = pickArea(mapAreas, e, xbaseHp, ybaseHp, nx, ny, dxHp, dyHp);
-if ((shapeBounds.getHeight() > maxHeight || shapeBounds.getWidth() > maxWidth) &&
-!areas[areaIndex].contains(shapeBounds)) {
+

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

I'm away at moment, without full access to sources, so can't follow the
current progress with partitioning, etc.

Regarding finding a point IN (and not ON) polygon:
1/ Try bounds center.
2/ Assumed determined [counter-]clockwise from area calculation.
   Pick 3 sequential points, bisect the angle and go in along this
   line by a small multiple of IsInUtil.EPS[_HP] and try this point.
3/ Move around by one and try again.
4/ If get back to the start, the polygon must be so thin that cannot
   contain another polygon.
   Take any Point and hope that this isn't ON more than possible outer

Ticker

On Thu, 2021-04-29 at 08:31 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I did a few tests with a tile in Zambia that contains some complex
> multipolygons and nestings. No
> 1) disable partitioning
> 2) disable partitioning and skip isLineInShape() test if center of
> geometry of the inner is IN for inner AND outer
> 3) use code in branch as is (enabled partitioning)
> 4) enable partitioning  and skip isLineInShape() test if center of
> geometry of the inner is IN for inner AND outer
> 
> Times for the compilation of the tile are
> 39 secs for 1)
> 32 secs for 2)
> 4 secs for 3) and 4)
> 
> With 3) and 4) I see only ~2 secs for the "processing MP relation"
> part, so it would require more complex relations to see improvements
> with 4)
> 
> If you know a quick and stable and simple method to find a point that
> is IN please let me know and I do some more testing. The getCofG()
> only works well for convex polygons. For around 3 percent of the
> polygons in my tests getCofG() returns a point outside of the
> polygon, and those polygons are likely more complex.
> 
> I've also done some tests with PrecompSeaGenerator and the complex
> tile sea_2752512_851968.osm.pbf
> Results are more or less equal for all 4 methods, the processing of
> the generated MP takes ~20 secs, only ~2 secs are used in
> createContainsMatrix() and ~18 secs are used in MultipolygonCutter
> 
> At the moment I try to decide what to do with incomplete data. I see
> three different situations with incomplete relations:
> 1) BoundaryPreprocessor is used with a country extract or maybe the
> combination of multiple extracts from geofabrik or other sources.
> In this case I tend to think that we can either ignore incomplete
> data completely or we might use the *.poly file(s) that were used to
> cut the data to "close the boundary polygon along the cutting
> polygon". The latter might help to avoid situations where the
> LocationHook returns no info for points outside of the cutting
> polygon.
> 2) mkgmap is used with splitter data and splitter was used with -
> -keep-complete=false
> In this case lots of multipolygons are incomplete, esp. those near
> the tile boundary. We MUST do some guessing about the missing data,
> else the maps are more or less unusable because large wood or lake
> areas wood be missing.
> 3) mkgmap is used with splitter data and splitter was used with -
> -keep-complete=true
> In this case we see incomplete MP when the country extract was
> already incomplete or with those MP that splitter doesn't keep
> complete.
> It's hard to say if we can ignore those incomplete MP or not. Most of
> them are boundaries. The default style doesn't render any boundary as
> polygon, but other styles do, e.g. the OpenFietsMap styles.
> 
> Maybe a simple new option "--ignore-incomplete-data" could improve
> performance. This would be easy to implement. A much more complex
> solution would be a new hook that is called directly before the
> processElements() call. It would use a new rule file similar to
> relations and could set special tags to tell mkgmap if it should
> calculate the areas of the multipolygon or just handle the outlines
> to produce the mkgmap:stylefilter=polyline ways.
> Just an idea so far.
> 
> The current code only tries to use the "tile boundary" to complete
> data. I think it would be better to use the cutting polygon. I
> started to experiment with that but found out that many *.poly files
> from geofabrik intersect with the country borders which are expected
> to be fully inside. I contacted them about this and Frederick Ramm
> confirmed that they want to fix that.
> 
> ciao,
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

More more testing with standard style / options etc. As well as --order
-by, I had some level params that confused.

With GPSMapEdit at Zoom level 2, there is a wide cut from the LHS
forest triangle to the holes around 54.9953,-58.2209. It seems to be
the total height of the holes.

Similar shows in MapSource, but it much more difficult to find.
Resolution needs to be <= 7mi before forest kicks-in. I didn't
investigate how far left the blank area went.

Looking more closely on Basecamp, the blank area is a rectangle approx
54.975,-58.302 to 55.0178,-58.6247, ie it doesn't go to the edge of the
forest triangle. Also it only shows when detail-level is set to
"highest"

Ticker

On Sun, 2021-04-25 at 11:03 +0100, Ticker Berkin wrote:
> Hi Gerd
> 
> Just tried this with latest trunk and my standard environment (mainly
>  --order-by-decreasing-area
> and get very inconsistent results.
> 
> On GPSMapEdit seems reasonable - shows forest triangle and, within, a
> small area with a number of holes.
> 
> MapSource shows a white square, no forest colour and I can't move the
> map around, also it doesn't seem to zoom out enough. First time I
> looked I saw 2 white squares.
> 
> Basecamp shows the forest triangle, but I can't find the
> islands/holes
> 
> I've had problems before with MapSource or Basecamp caches, so I'll
> delete these. Also try with simple options etc
> 
> Ticker
> 
> 
> On Sun, 2021-04-25 at 07:31 +, Gerd Petermann wrote:
> > Hi Experts,
> > 
> > while working on the faster-mp branch I experimented with the
> > monster
> > relation r9488835. I added tag landuse=forest to it so that it is
> > actually rendered and found huge (wrong) empty rectangles in the
> > forest.
> > The original file is very complex but the same problem also occurs
> > with a simplified file, see 
> > http://files.mkgmap.org.uk/download/505/bad-map-split.osm.pbf
> > Just use e.g.
> > java -jar mkgmap.jar --gmapi bad-map-split.osm.pbf 
> > Problem: Basecamp and Mapsource don't show this map, GpsMapEdit
> > shows
> > a large white rectangle near 54.97743 -58.30292
> > 
> > I assume that we hit some unknown limit , possibly in LinePreparer
> > or
> > MapSplitter. 
> > I've also tried some older versions, so far they all show the
> > problem.
> > Any idea where to look? 
> > 
> > RgnDisplay shows this data for the problemat area:
> > - Level 0, Subdiv 279 -
> > --
> > ---
> >  || | final at 265f, (end
> > 25e2)
> >  || | polygon start 2634 to
> > 265f
> > 2634 | 00 | 4b  | Polygon 1
> >  || | Type 0x4b
> > 2635 | 01 | 00 00 00| Label offset:
> > 0x00
> > ()
> > 2638 | 04 | d0 3b   | Long delta 15312
> > 263a | 06 | 2a 3a   | Lat delta 14890
> >  || | Start point 55.24913,
> > -61.91139 2574796/-2885280
> > 263c | 08 | 0c  | Bitstream length 12
> > 263d | 09 | bb  | Base bb
> > 263e | 0a | 06 43 04 00 00 00 4c d1 | Bitstream (len: 12,
> > 0xc)
> > 2646 | 12 | 41 ef 00 00 | 
> >  || | xsame false, xneg
> > false
> >  || | ysame true, yneg true
> >  || | xbase 16, ybase 15
> >  || | Coord 55.24913,
> > -62.56851 2574796/-2915904 [-30624,+0]
> >  || | Coord 54.61014,
> > -62.56851 2545017/-2915904 [+0,-29779]
> >  || | Coord 54.61014,
> > -61.91139 2545017/-2885280 [+30624,+0]
> >  || | 0 bits left, 0
> >  || |  
> > 264a | 16 | 50  | Polygon 2
> >  || | Type 0x50
> > 264b | 17 | 00 00 00| Label offset:
> > 0x00
> > ()
> > 264e | 1a | d0 3b   | Long delta 15312
> > 2650 | 1c | b4 08   | Lat delta 2228
> >  || | Start point 54.97743,
> > -61.91139 2562134/-2885280
> > 2652 | 1e | 0b  | Bitstream length 11
> > 2653 | 000

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

Is there a simple way to look at an .osm.pbf file with JOSM?

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Please try branch check-precomp-sea

[Ticker Berkin]

Hi

I presume this will be most use to people who generate sea-latest.zip.

I use --generate-sea=..., so using the current OSM data for the area
concerned, and, before loading onto a device, look at the overview map
with GPSMapEdit to check the sea has worked correctly.

If the country cut has problems, eg Spain without Portugal, I use sea.z
ip and again check with the overview map. If still problems, try an
older or newer version of sea.zip

Ticker

On Sat, 2021-04-24 at 13:12 +, Gerd Petermann wrote:
> Hi all,
> 
> just noticed that this branch was not merged because I got no
> feedback. A few weeks ago Eric complained about bad precomp-sea data
> and a lot of wasted time to find it. Any reason why nobody is
> interested?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Donnerstag, 17. September 2020 11:34
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Please try branch check-precomp-sea
> 
> Hi all,
> 
> I've implemented some basic checks to detect large flooded areas in
> the precomp-sea data in this new branch.
> A link to the latest binary (r4579 as of now) is a the bottom of the
> normal download page :
> http://www.mkgmap.org.uk/download/mkgmap.html
> 
> - If PrecompSeaGenerator is used to calculate the sea data and the
> index check fails a warning is printed and the program ends with rc 
> -1.
> - If mkgmap is used with a "dubious" sea.zip one or more messages
> about the problematic areas are written and the program stops with an
> ExitException
> unless option --x-check-precomp-sea=0 is given before the first input
> file.
> 
> The index test is normally performed once for the whole planet, so it
> doesn't care about the actual tile boundaries. If e.g. only a part of
> Australia is flooded and you create a map for the UK you'll still see
> the errors.
> 
> BTW: The current code (also in trunk version) doesn't support to use 
> --precomp-sea with different values. Results are rather
> unpredictable.
> 
> Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

Just tried this with latest trunk and my standard environment (mainly
 --order-by-decreasing-area
and get very inconsistent results.

On GPSMapEdit seems reasonable - shows forest triangle and, within, a
small area with a number of holes.

MapSource shows a white square, no forest colour and I can't move the
map around, also it doesn't seem to zoom out enough. First time I
looked I saw 2 white squares.

Basecamp shows the forest triangle, but I can't find the islands/holes

I've had problems before with MapSource or Basecamp caches, so I'll
delete these. Also try with simple options etc

Ticker


On Sun, 2021-04-25 at 07:31 +, Gerd Petermann wrote:
> Hi Experts,
> 
> while working on the faster-mp branch I experimented with the monster
> relation r9488835. I added tag landuse=forest to it so that it is
> actually rendered and found huge (wrong) empty rectangles in the
> forest.
> The original file is very complex but the same problem also occurs
> with a simplified file, see 
> http://files.mkgmap.org.uk/download/505/bad-map-split.osm.pbf
> Just use e.g.
> java -jar mkgmap.jar --gmapi bad-map-split.osm.pbf 
> Problem: Basecamp and Mapsource don't show this map, GpsMapEdit shows
> a large white rectangle near 54.97743 -58.30292
> 
> I assume that we hit some unknown limit , possibly in LinePreparer or
> MapSplitter. 
> I've also tried some older versions, so far they all show the
> problem.
> Any idea where to look? 
> 
> RgnDisplay shows this data for the problemat area:
> - Level 0, Subdiv 279 ---
> ---
>  || | final at 265f, (end
> 25e2)
>  || | polygon start 2634 to
> 265f
> 2634 | 00 | 4b  | Polygon 1
>  || | Type 0x4b
> 2635 | 01 | 00 00 00| Label offset: 0x00
> ()
> 2638 | 04 | d0 3b   | Long delta 15312
> 263a | 06 | 2a 3a   | Lat delta 14890
>  || | Start point 55.24913,
> -61.91139 2574796/-2885280
> 263c | 08 | 0c  | Bitstream length 12
> 263d | 09 | bb  | Base bb
> 263e | 0a | 06 43 04 00 00 00 4c d1 | Bitstream (len: 12,
> 0xc)
> 2646 | 12 | 41 ef 00 00 | 
>  || | xsame false, xneg false
>  || | ysame true, yneg true
>  || | xbase 16, ybase 15
>  || | Coord 55.24913,
> -62.56851 2574796/-2915904 [-30624,+0]
>  || | Coord 54.61014,
> -62.56851 2545017/-2915904 [+0,-29779]
>  || | Coord 54.61014,
> -61.91139 2545017/-2885280 [+30624,+0]
>  || | 0 bits left, 0
>  || |  
> 264a | 16 | 50  | Polygon 2
>  || | Type 0x50
> 264b | 17 | 00 00 00| Label offset: 0x00
> ()
> 264e | 1a | d0 3b   | Long delta 15312
> 2650 | 1c | b4 08   | Lat delta 2228
>  || | Start point 54.97743,
> -61.91139 2562134/-2885280
> 2652 | 1e | 0b  | Bitstream length 11
> 2653 | 1f | 9b  | Base 9b
> 2654 | 20 | f2 8d 04 40 a7 fe d0 4b | Bitstream (len: 11,
> 0xb)
> 265c | 28 | e7 00 00| 
>  || | xsame false, xneg false
>  || | ysame true, yneg false
>  || | xbase 16, ybase 11
>  || | Coord 54.97743,
> -62.51706 2562134/-2913506 [-28226,+0]
>  || | Coord 55.01732,
> -62.54665 2563993/-2914885 [-1379,+1859]
>  || | Coord 55.01732,
> -61.91139 2563993/-2885280 [+29605,+0]
>  || | 4 bits left, 0
>  || |  
> 
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] found multiple points with equal coord.....

[Ticker Berkin]

Hi Gerd & others

This is a quite common physical set-up; a highway with another, more
minor, highway/track/path leading off it with a "barrier" at the
junction, inhibiting access to the minor highway.

Often the only clear access restrictions visible on-the-ground are on
the barrier, so these are what get represented in OSM. A minimal
distance between the barrier and the major highway is needed to map
this on OSM.

mkgmap should handle this situation correctly. I haven't tried to
understand why it didn't before, but if the patch fixes it then it
should be applied.

I've seen other examples of GPSMapEdit flagging errors like this:

WARNING: Unexpected format of input data. Unexpected NodeID=936272 in
turn restriction at (51.013083,-1.389384). The restriction will be
ignored

where there are very short section between the restriction and the
junction node and I think it is just a GPSMapEdit problem.

Ticker


On Sun, 2021-05-02 at 08:46 +, Gerd Petermann wrote:
> Hi all,
> 
> the message
> "Found multiple points with equal coords as CoordPOI at 
> http://www.openstreetmap.org/?mlat=56.835675=37.404949=17;
> is produced because the nodes
> https://www.osm.org/node/2924385337
> https://www.osm.org/node/2924351922
> are so close together that mkgmap could not distinguish the position.
> No idea why someone maps a lift-gate less that close to the end of
> the road, but of course mkgmap should not fail with this.
> 
> The problem doesn't show up with the default style unless I add this
> line in points rules:
> barrier=*{ addaccess no;}
> after
> barrier=* & access=* {addaccess '${access|subst:private=>no}'}
> Without this change the default style doesn't create a route
> restriction for the barrier node 
> https://www.openstreetmap.org/node/2924385337
> 
> I've created a patch to fix this problem for good by adding  a
> hashmap but GpsMapEdit doesn't like the result:
> Unexpected format of input data. Unexpected NodeID=136381 in turn
> restriction at (56.835666 37.404950). The restriction will be
> ignored.
> 
> The old code didn't add the restriction, the new code does.
> Now I am unsure if the "corrected" code causes more trouble than the
> old. Maybe I should somehow detect the special case that a barrier
> is very close to an end of a way which is not connected to other
> ways? If I change the input data so that the end is a bit more away
> GpsMapEdit doesn't complain. Is it just a problem in GpsMapEdit?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Thomas Morgenstern 
> Gesendet: Samstag, 1. Mai 2021 08:56
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] found multiple points with equal coord.
> 
> Hi ,
> yesterday mkgmap r4587 gives me following message : 'found multiple
> points with equal coord as CoordPOI at 56.835467 37.404949...
> The map seems okay. What shell I do with this message ?. Is this
> meassage only a hint for incorrect data ? Or what else ?
> 
> thomas
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] found multiple points with equal coord.....

[Ticker Berkin]

Hi Gerd

I think it should just deal with the (probably) common, significant
case and assume the restriction is needed.

The short distance is, nominally, from the centre-line to the side of
the non-barriered highway. In the examples I've seen this more than
highPrecEqual but less than 2m (apart from the errors where mappers
have put a stile on the junction, forcing car-routing to avoid the
area!)

Ticker

On Tue, 2021-05-04 at 08:16 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> thanks for the info. I didn't see such messages in GpsMapEdit before,
> only for turn-restrictions with multiple via nodes.
> The only special case here is that the barrier node "highPrecEquals"
> the last node.
> 
> A general question would be if mkgmap should detect that a
> restriction shoud better be ignored when it only restricts access to
> the last 5 meters or so.
> I didn't try it but I guess it could lead to routing problems if your
> start is at the end of the road.
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 4. Mai 2021 10:06
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] found multiple points with equal coord.
> 
> Hi Gerd & others
> 
> This is a quite common physical set-up; a highway with another, more
> minor, highway/track/path leading off it with a "barrier" at the
> junction, inhibiting access to the minor highway.
> 
> Often the only clear access restrictions visible on-the-ground are on
> the barrier, so these are what get represented in OSM. A minimal
> distance between the barrier and the major highway is needed to map
> this on OSM.
> 
> mkgmap should handle this situation correctly. I haven't tried to
> understand why it didn't before, but if the patch fixes it then it
> should be applied.
> 
> I've seen other examples of GPSMapEdit flagging errors like this:
> 
> WARNING: Unexpected format of input data. Unexpected NodeID=936272 in
> turn restriction at (51.013083,-1.389384). The restriction will be
> ignored
> 
> where there are very short section between the restriction and the
> junction node and I think it is just a GPSMapEdit problem.
> 
> Ticker
> 
> 
> On Sun, 2021-05-02 at 08:46 +, Gerd Petermann wrote:
> > Hi all,
> > 
> > the message
> > "Found multiple points with equal coords as CoordPOI at
> > http://www.openstreetmap.org/?mlat=56.835675=37.404949=17
> > "
> > is produced because the nodes
> > https://www.osm.org/node/2924385337
> > https://www.osm.org/node/2924351922
> > are so close together that mkgmap could not distinguish the
> > position.
> > No idea why someone maps a lift-gate less that close to the end of
> > the road, but of course mkgmap should not fail with this.
> > 
> > The problem doesn't show up with the default style unless I add
> > this
> > line in points rules:
> > barrier=*{ addaccess no;}
> > after
> > barrier=* & access=* {addaccess '${access|subst:private=>no}'}
> > Without this change the default style doesn't create a route
> > restriction for the barrier node
> > https://www.openstreetmap.org/node/2924385337
> > 
> > I've created a patch to fix this problem for good by adding  a
> > hashmap but GpsMapEdit doesn't like the result:
> > Unexpected format of input data. Unexpected NodeID=136381 in turn
> > restriction at (56.835666 37.404950). The restriction will be
> > ignored.
> > 
> > The old code didn't add the restriction, the new code does.
> > Now I am unsure if the "corrected" code causes more trouble than
> > the
> > old. Maybe I should somehow detect the special case that a barrier
> > is very close to an end of a way which is not connected to other
> > ways? If I change the input data so that the end is a bit more away
> > GpsMapEdit doesn't complain. Is it just a problem in GpsMapEdit?
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Thomas Morgenstern 
> > Gesendet: Samstag, 1. Mai 2021 08:56
> > An: Development list for mkgmap
> > Betreff: [mkgmap-dev] found multiple points with equal coord.
> > 
> > Hi ,
> > yesterday mkgmap r4587 gives me following message : 'found multiple
> > points with equal coord as CoordPOI at 56.835467 37.404949...
> > The map seems okay. What shell I do with this message ?. Is this
> > meassage only a hint for incorrect data ? Or what else ?
> > 
> > thomas
> > 
> > ___
> > mkgmap

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

Point IN and area should be adequate for determining enclosed/enclosing
polygons relationships. Overlapping / intersecting is an error anyway,
but, even with badly/arbitrarily closed boundaries, provided the
closing takes a similar route around the known mapArea, the area shouldsuffice.

Ticker


On Tue, 2021-04-27 at 14:23 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I don't care much about invalid MP so far. It is not allowed that
> inner ways and outer ways are connected, nor is it allowed that outer
> rings share segments.
> So, at the moment that means Garbage in -> Garbage out.
> There are some special cases more than two outer ways are connected
> at the same node and there should be a back-tracking algo to combine
> them in a ways that doesn't produce self intersecting rings. No idea
> how to implement that.
> 
> reg. containsMatrix: I did not find any problems with nested polygons
> so far, but yes, the partitioning is more likely to produce more edge
> cases.
> 
> I don't see how a single call of isPointInShape() can be enough. It
> only helps if it returns OUT, but that is unlikely. If it returns IN
> the polygons may still be overlapping. I've not yet decided what to
> do with those MP because the real MP with complete data might be
> correct and the overlapping could be a result of artifically closed
> rings.
> 
> I've just noticed that the branch is sometimes erronously connecting
> ways which trunk doesn't connect (only with BoundaryRelation).
> 
> I think it will take a one or two more weeks before this branch is
> getting stable.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 27. April 2021 15:37
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] tile takes very long time to generate
> 
> Hi Gerd
> 
> I'd have thought using isPointInShape() in combination with polygon
> areas to be more efficient and robust than isLineInShape() (which
> makes
> multiple calls to isPointInShape). Would need to choose a reference
> point within each polygon, ensuring it is IN but not ON itself.
> 
> Probably splitShape(), isPointInShape() and
> calcAreaSize()/calcAreaSizeTestVal() all have very similar costs;
> linear on number of points.
> 
> I understand some benefits of splitting a massive outer into
> multiples
> polygons, in that a point-IN question might give an quicker answer,
> but
> it must introduce a lot of other difficulties with edge cases on the
> artificial boundaries, etc, etc. Splitting inners shouldn't cause a
> probem in this respect (just a list of inners), but if there is
> another
> outer in the hole, this will become confusing.
> 
> A couple of other things I meant to mention following earlier faster
> -mp
> svn updates:
> 
> I found cases, as both inner and outer, where adjacent rectangles (eg
> buildings) are touched together, with either a duplicate or identical
> member being the dividing wall. I realise that some combinations of
> this violate OSM MP rules, but, keeping the duplicate, allows this
> simple "double-touching" structure to be resolved; where there are 4
> ways at a node, don't join 2 that are identical or have same other
> -end
> and no other points.
> 
> Joining ways if possible in the initial element iteration and
> ignoring
> roles can lead to problems when inner and outer touch. Whereas saving
> them all in a Map of both ends, before attempting to join, will avoid
> the problem for single-touching. For double-touching, any roles can
> be
> used to join the correct parts. For ordering stability if using a
> HashMap, could just keep track of lowest element# in joinedWays and
> always keep this first.
> 
> Ticker
> 
> 
> On Mon, 2021-04-26 at 14:17 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think I am making progress with my divide approach. I see
> > ~20 secs instead of more than 7 minutes to process the monster
> > relation r9488835.
> > The basic approach is to use ShapeSplitter.splitShape() multiple
> > times to divide the multipolygon into smaller pieces where no ring
> > has more than 20.000 nodes and executing the block
> > 
> > analyseRelationRoles()
> > ...
> > doReporting()
> > for each piece.
> > 
> > The work is not yet done, the output file is a bit larger, not yet
> > sure why, and the calculation of the "largestOuterPolygon" neads a
> > bit more work.
> > There are probably edge cases where this doesn't work, esp. when
> > inners are very large.
> > 
> > BTW: I found a bug in IsInUtil.isLineInShape() : It sometimes
> > retu

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

I'd have thought using isPointInShape() in combination with polygon
areas to be more efficient and robust than isLineInShape() (which makes
multiple calls to isPointInShape). Would need to choose a reference
point within each polygon, ensuring it is IN but not ON itself.

Probably splitShape(), isPointInShape() and
calcAreaSize()/calcAreaSizeTestVal() all have very similar costs;
linear on number of points.

I understand some benefits of splitting a massive outer into multiples
polygons, in that a point-IN question might give an quicker answer, but
it must introduce a lot of other difficulties with edge cases on the
artificial boundaries, etc, etc. Splitting inners shouldn't cause a
probem in this respect (just a list of inners), but if there is another
outer in the hole, this will become confusing.

A couple of other things I meant to mention following earlier faster-mp
svn updates:

I found cases, as both inner and outer, where adjacent rectangles (eg
buildings) are touched together, with either a duplicate or identical
member being the dividing wall. I realise that some combinations of
this violate OSM MP rules, but, keeping the duplicate, allows this
simple "double-touching" structure to be resolved; where there are 4
ways at a node, don't join 2 that are identical or have same other-end
and no other points.

Joining ways if possible in the initial element iteration and ignoring
roles can lead to problems when inner and outer touch. Whereas saving
them all in a Map of both ends, before attempting to join, will avoid
the problem for single-touching. For double-touching, any roles can be
used to join the correct parts. For ordering stability if using a
HashMap, could just keep track of lowest element# in joinedWays and
always keep this first.

Ticker


On Mon, 2021-04-26 at 14:17 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I think I am making progress with my divide approach. I see
> ~20 secs instead of more than 7 minutes to process the monster
> relation r9488835.
> The basic approach is to use ShapeSplitter.splitShape() multiple
> times to divide the multipolygon into smaller pieces where no ring
> has more than 20.000 nodes and executing the block
> 
> analyseRelationRoles()
> ...
> doReporting()
> for each piece.
> 
> The work is not yet done, the output file is a bit larger, not yet
> sure why, and the calculation of the "largestOuterPolygon" neads a
> bit more work.
> There are probably edge cases where this doesn't work, esp. when
> inners are very large.
> 
> BTW: I found a bug in IsInUtil.isLineInShape() : It sometimes returns
> ON when it should return IN/ON
> I think it happens when a ring has start /end node ON and also 2nd or
> 2nd-last node ON.
> Probably a special case created by the ShapeSplitter. Attached patch
> fixes the problem in IsInUtil.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Mittwoch, 7. April 2021 13:02
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] tile takes very long time to generate
> 
> Hi Gerd
> 
> Problem is that IdentityHashMap is the ideal solution, but ordering
> behaviour depends on internal java object handles. A solution to this
> stability issue would be to rotate joinedWays.originalWays so, say,
> the
> one with the lowest ID is first, doing this just before the full
> point
> -list is created.
> 
> Maybe not worth it if existing logic is not a problem.
> 
> Some of the other advantages my logic:
> 
>   Clearer and more efficient.
> 
>   Elements are scanned once; cOg and closed polygons are processed
>   immediately. Elements from SeaGenerator are all closed.
> 
>   Single touching polygons are handled without difficulty;
>   just part of the way the algo is expressed.
> 
>   Maintaining jw.inner/outerCount gives clarity to this issue.
>   Can be used for error reporting and behavioral definition.
>   If the roles are not to be trusted, the counts can be adjusted when
>   the geometry is determined.
> 
> Ticker
> 
> 
> On Wed, 2021-04-07 at 08:57 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > reg. mpInitialLogic.patch: I think I tried the same approach to
> > join
> > the ways in the past. Problem is that the patched code produces
> > rings
> > with an unpredictable start/end and thus random content in RGN.
> > This
> > makes comparisons of two mkgmap outputs much more difficult if not
> > impossable.
> > 
> > The existing algo works quite well if the members are sorted and
> > most
> > complex MP are sorted well enough. I think in the worst case (badly
> > shuffled member list) the algo complexity is probably O(n²) , in
> > the
> > b

Re: [mkgmap-dev] Empty rectangles in map

[Ticker Berkin]

Hi Gerd

I had to zoom in to < 10miles, with detail set to "Highest", before the
forest appeared. It was difficult to know where to start from. Then, at
that level, it is very tedious to scroll around trying to finds the
holes.

Ticker

On Tue, 2021-04-27 at 12:48 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> thanks for investigating. I've used the patch with trunk r4680 and
> just --gmapi option.
> I see that GpsMapEdit shows no hole, but Mapsource and Basecamp still
> don't show the map. Any idea why?
> 
> Gerd
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 27. April 2021 13:11
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Empty rectangles in map
> 
> Hi Gerd and others
> 
> The cause for this unrendered area is as follows:
> 
> A large multipolygon (larger than the maximum size for a level 1
> subdiv) exists, and is broken into some similar size pieces and some
> smaller pieces to expose inners/holes. The same problem can be caused
> by any polygons having these size properties.
> 
> Option --order-by-decreasing-area is not in effect (with this option
> the problem goes away).
> 
> At the outermost level, the large pieces are split into the grid of
> subdivs necessary to be within size limits. Small pieces that fit
> within a subdiv are allocated to the correct one. Intermediate pieces
> that extend out of the nominal subdiv bounds, but don't exceed the
> actual addressing limits, are allocated to an existing subdiv based
> on
> their centre. Larger pieces that fit in a subdiv but would cause size
> problems if added to an existing subdiv are given a subdiv of their
> own.
> 
> At the next level down, each of the above subdivs will required
> splitting to fit within the addressing limits. The logic that did
> this
> used the subdiv bounds and split this into a grid. Then every element
> in the outer MapArea is processed as above (ie split, safely placed,
> placed with overlap or own subdiv)
> 
> The problem is that the splitting algorithm simply goes through the
> child subdiv, clips the polygon to this size and saves the bits that
> are within the area. Polygon parts that are outside the defined outer
> level never get noticed!
> 
> There are various possible solutions:
> 
> Method 1: use the rigorous polygon splitting into subdivs that -
> -order
> -by-decreasing-area uses for levels above level 0. This is simple to
> implement, safe, mergeShapes might recombine more cut fragments as
> they
> are all together. Disadvantages are that, for a normal map, more
> polygons might be split, leading to larger RGN size (but not for bad
> -map-split.osm.pbf). The overall size is still less that with --order
> -by...
> 
> Method 2: When the child subdiv grid is created, use any expanded
> size
> of the parent subdiv. This stops overhanging bits of polygons being
> lost. Again, simple to implement. It results in overlapping subdivs
> at
> lower levels and I don't know this causes other problems, but
> probably
> not; this used to be done and still does for some sizes of polygons
> and
> lines. Some shape-merging won't happen because the joining bits are
> in
> a different subdiv.
> 
> Method 3: Allocated oversize polygons to their own subdiv. Again,
> simple to implement, but I see no advantage over Method 2. Even more
> shape-merging might not happen.
> 
> Method 4: Detect when an object will have bits missed and semi-cut
> rather than clip them to continue having an oversize part in the
> child
> subdiv. This is more complicated to implement but, if done, could
> replace the clipping code and be more efficient. If a parent subDiv
> is
> divided into 2 children, it already happens like this.
> 
> Although I prefer Method 1, I suspect Method 2 is what should be done
> in the short-term and so I've attached a patch that is configured for
> Method 2, but, with a couple of edits to change some flag handling,
> does Method 1.
> 
> Ticker
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Fix Sea Patch

[Ticker Berkin]

Hi Mike / Gerd

This patch seems fine to me.

Ticker


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Fix Sea Patch

[Ticker Berkin]

Hi Mike & Gerd

I agree that the existing default partial check is pointless and it is
better for the default to be no checking of land in sea / sea in land.
With checks enabled, much more thorough checks are done.

Re. the wording for the sea options

The documentation for --generate-sea says:
... When this option is defined, natural=coastline tags
  from the input OSM tiles are ignored and the precompiled data is used
  instead. This option can be combined with the generate-sea options
  multipolygon, polygons and land-tag. ...

If this was changed to something like:
... When this option is defined, natural=coastline tags
  from the input OSM tiles are ignored and the precompiled data is used
  to generate the sea. The generate-sea options multipolygon, polygons
  and land-tag, with associated default values, control how this is
  done. ...

Ticker  

On Wed, 2021-02-10 at 09:58 +, Mike Baggaley wrote:
> Hi Gerd,
> 
> In my view it does not make sense to make the option only operate on
> the new
> checks (land within land) leaving sea within sea always checked. I
> think
> that either checks are wanted or they are not, hence the option
> switches on
> and off all the checks. It seems overkill to go for -
> -check=[land|sea].
> 
> To me, because the default behaviour statement is on the multipolygon
> value,
> not the generate-sea option, the docs indicate that multipolygon is
> the
> default when --generate-sea is specified without a value, not that
> --generate-sea=multipolygon is on by default. If --generate
> -sea=multipolygon
> is the default, how is it switched off if sea processing is not
> needed?
> 
> The minor changes look fine to me. Looks like I had got too used to
> editing
> style files with its & and | rather than the Java && and ||!
> 
> Cheers,
> Mike
> 
> -Original Message-
> From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com] 
> Sent: 10 February 2021 08:47
> To: Development list for mkgmap 
> Subject: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike,
> 
> if I got that right there is no impact on the PrecompSeaGenerator as
> that
> doesn't execute this part of SeaGenerator. There is also no impact on
> processing when --precomp-sea is used.
> However, now the patched code only performs any check when new
> parameter
> --check is given. Is that intended? Ticker suggested to add an option
> for
> the additional checks.
> 
> The doc says that --generate-sea=multipolygon is the default, this is
> also
> true with --precomp-sea. Not sure what is confusing you?
> 
> I've made some minor changes to the code to reduce complexity, but
> didn't
> check the details of the new code.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Mike
> Baggaley 
> Gesendet: Montag, 8. Februar 2021 15:58
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] Fix Sea Patch
> 
> HI Gerd, I don't have a build that includes the PrecompSeaGenerator,
> but I
> would expect the impact to be the same as for in-line sea generation:
> an
> increase in execution time if /check is specified with a reduction if
> it is
> not specified.
> 
> I'm not quite clear what happens if you specify --precomp-sea, but
> don't
> specify --generate-sea. The help only says that --precomp-sea can be
> combined with --generate-sea but doesn't say what happens if you
> don't.
> (Does it switch on --generate-sea=multipolygon?)
> 
> I think the order of the options in the documentation is not ideal
> either -
> I would think --generate-sea should come first, followed by -
> -coastline-file
> and that followed by --precomp-sea. Happy to tweak this.
> 
> Cheers,
> Mike
> 
> -Original Message-
> From: Gerd Petermann [mailto:gpetermann_muenc...@hotmail.com]
> Sent: 07 February 2021 17:36
> To: Development list for mkgmap 
> Subject: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike and Ticker,
> 
> do you expect an impact on the PrecompSeaGenerator?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von
> Ticker Berkin 
> Gesendet: Sonntag, 7. Februar 2021 18:16
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Fix Sea Patch
> 
> Hi Mike / Gerd
> 
> This patch seems fine to me.
> 
> Ticker
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Pending changes

[Ticker Berkin]

Hi all

I've re-made this set of changes, along with a few improvements that
I've gathered over the last 6 months. Following list numbering is the
same as original patch, but include some [extra] notes + new items at
the end.

Relevant threads:
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q3/031375.html
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q3/031424.html

1/ Sometimes charges for using a pedestrian highway are expressed as a
fee rather than a toll, so pick this up in mkgmap:toll.

2/ Show bridges using type 0x10107. With the mapnik addition, these
look good for narrow highways, but might not show for wide
representations like motorways.

3/ Where it is likely that bits of highway might not be connected to
the road/path system, use mkgmap:set_unconnected_type and
mkgmap:set_semi_connected_type to stop the NET/NOD rather than relying
on NETnoNOD (now disabled) and --check-routing-island-len=+ve, which is
being suspected of causing problems on some devices and BaseCamp.

[extra] In all cases where unconnected/semi-connected changes are
mentioned, this only applies to lines not derived from an original/OSM
standard highway.

4/ Quote some filter subst strings that contain spaces - no actual
effect but clearer and safer.

5/ Have line for leisure=track even if area.

6/ Change the type for tracks/raceways etc from 0x30, which doesn't
show on BaseCamp or MapSource, to 0x2a.

7/ For piers, if 'unconnecting', use marine type 0x1040c (Obstruction:
Pier or Jetty) instead of footway.

8/ Change type for the various barriers from 0x17, which doesn't show
on BaseCamp to 0x2b, see:
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q1/030348.html

9/ Consider natural=cliff a barrier.

10/ Add motorway[_link] roundabouts (yes, some do exist).

11/ Unquote some numbers - no actual effect.

12/ Tweak some road speeds.
[extra] A few more tweaked.

13/ Use 0x09 (high-speed ramp) for road class 4 links

14/ Add footway around car parks if 'connecting'.
[extra] This change is disabled.

15/ Disable coastline. For a long time the tag was suppressed by the
Sea processing and it adds little to the map.

16/ Improve railway platform names and suppress footway if not
'connecting'.

17/ Show disused:railway in the same way as railway=disused.

18/ Have cable_car, gondola, funicular as routable, by default with a
toll and pedestrian only.

19/ Show "Course of old Railway", unless a highway has taken over the
way (for you Eric, but I like it as well).
[extra] This change is disabled

20/ Speed up car ferries.

21/ A few other layout/space fixes.

Additional changes:

22/ motorroad=yes just sets mkgmap:fast_road, which generally increases
the speed/class of the highway and decreases the resolution

23/ natural=landslide like other barriers (eg cliff)

24/ Don't generate (routable) line for highway=unclassified & area=yes;
there are many instances in OSM where this is used to draw a polygon
around a complex junction.

25/ Change the bridleway from 0x07 (Alley) to 0x16 (Trail)

26/ For ferry/platform/aerialway, blank out address fields to prevent
it getting into the Address index

27/ Add comment about colour pallet to mapnik.txt

Patch attached

Ticker

On Tue, 2021-02-09 at 11:30 +0100, Carlos Dávila wrote:
> On [1] Ticker proposed a set of changes to default style lines file. 
> There was a long discussion about point #14, which ended without a 
> consensus. Other changes didn't get any objection, but they didn't
> get 
> into trunk. I agree with numbers 1, 6, 8, 10, 15, 17, 18. Also with 7
> and 16, but for unconnected ways only, see #3.
> 
> 2: more codes could be added for wider highways and their
> corresponding 
> entries in mapnik.txt
> 
> 3: not sure about this one, specially about semi-connected ways,
> which I 
> think should remain as routable (also applies for 7, 16).
> 
> [1] http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q3/031375.html
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: resources/styles/default/lines
===
--- resources/styles/default/lines	(revision 4600)
+++ resources/styles/default/lines	(working copy)
@@ -19,8 +19,8 @@
 # Assign the street name for house number search
 highway=* & name=* {set mkgmap:street='${name}'}
 
-# Mark highways with the toll flag
-highway=* & (toll=yes | toll=true) {set mkgmap:toll=yes}
+# Mark highways with the toll flag (fee is sometimes used on pedestrian type routes)
+highway=* & (toll=* | fee=*) {set mkgmap:toll='${toll}' | '${fee}'}
 
 # Hide proposed ways
 highway=proposed | highway=proposal | highway=planned | highway~'.*proposed.*' {delete highway; delete junction}
@@ -40,6 +40,14 @@
 highway=piste | highway=ski {delete highway}
 highway=no | highway=none {delete highway}
 
+# Show bridges. Often these also have unconnected highway=track/... If so, we would like to 

[mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

I was trying to diagnose a problem with a repeating points in polylines
as reported by GPSMapEdit and found a problem in
RemoveObsoletePointsFilter where it duplicates a point.

Also in this and/or RoundCoordsFilter I've made some changes:
1/ stop the chain when polygons get too small
2/ keep polygons closed with the same first/last point
3/ slight logic tidy-up
4/ add a couple of debug lines to be consistent

Patch attached.

Actually these didn't make any difference to the repeating points
problem. This happens when there are enough unused bits in the last
byte of the polyline bitsteam to represent an extra point. I can't see
any good way of stopping this.

TickerIndex: src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java
===
--- src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java	(revision 4616)
+++ src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java	(working copy)
@@ -29,8 +29,6 @@
  */
 public class RemoveObsoletePointsFilter implements MapFilter {
 	private static final Logger log = Logger.getLogger(RemoveObsoletePointsFilter.class);
-	
-	final Coord[] areaTest = new Coord[3];
 
 	private boolean checkPreserved;
 	
@@ -48,18 +46,12 @@
 		MapLine line = (MapLine) element;
 		List points = line.getPoints();
 		int numPoints = points.size();
-		if (numPoints <= 1){
-			return;
-		}
-		int requiredPoints = (line instanceof MapShape ) ? 4:2; 
+		int requiredPoints = (line instanceof MapShape) ? 4 : 2;
 		List newPoints = new ArrayList<>(numPoints);
 		while (true){
 			boolean removedSpike = false;
-			numPoints = points.size();
-			
-
-			Coord lastP = points.get(0);
-			newPoints.add(lastP);
+			Coord lastP;
+			newPoints.add(points.get(0));
 			for (int i = 1; i < numPoints; i++) {
 Coord newP = points.get(i);
 int last = newPoints.size()-1;
@@ -72,6 +64,7 @@
 			continue;
 		} else if (!lastP.preserved()){
 			newPoints.set(last, newP); // replace last
+			continue;
 		} 
 	} else {  
 		continue;
@@ -104,36 +97,45 @@
 
 newPoints.add(newP);
 			}
-			if (!removedSpike || newPoints.size() < requiredPoints)
+			numPoints = newPoints.size();
+			if (numPoints < requiredPoints)
+return;
+			if (!removedSpike)
 break;
 			points = newPoints;
-			newPoints = new ArrayList<>(points.size());
+			newPoints = new ArrayList<>(numPoints);
 		}
 		if (line instanceof MapShape){
 			// Check special cases caused by the fact that the first and last point 
 			// in a shape are identical. 
-			while (newPoints.size() > 3){
-int nPoints = newPoints.size();
-switch(Utils.isStraight(newPoints.get(newPoints.size()-2), newPoints.get(0), newPoints.get(1))){
+		isStraightCheck:
+			while (true) {
+switch (Utils.isStraight(newPoints.get(numPoints-2), newPoints.get(0), newPoints.get(1))) {
 case Utils.STRAIGHT_SPIKE:
+	log.debug("removing closing spike");
+	--numPoints;
 	newPoints.remove(0);
-	newPoints.set(newPoints.size()-1, newPoints.get(0));
-	if (newPoints.get(newPoints.size()-2).equals(newPoints.get(newPoints.size()-1)))
-		newPoints.remove(newPoints.size()-1);
+	newPoints.set(numPoints-1, newPoints.get(0));
+	if (newPoints.get(numPoints-2).equals(newPoints.get(numPoints-1))) {
+		newPoints.remove(numPoints-2); // keep the identical closing point
+		--numPoints;
+	}
 	break;
 case Utils.STRICTLY_STRAIGHT:
-	newPoints.remove(newPoints.size()-1);
-	newPoints.set(0, newPoints.get(newPoints.size()-1));
+	log.debug("removing straight line across closing");
+	--numPoints;
+	newPoints.remove(numPoints);
+	newPoints.set(0, newPoints.get(numPoints-1));
 	break;
+default:
+	break isStraightCheck;
 }
-if (nPoints == newPoints.size())
-	break;
+if (numPoints < requiredPoints)
+	return;
 			}
 		}
-		
-		if (newPoints.size() != line.getPoints().size()){
-			if (line instanceof MapShape && newPoints.size() <= 3 || newPoints.size() <= 1)
-return;
+
+		if (numPoints != line.getPoints().size()) {
 			MapLine newLine = line.copy();
 			newLine.setPoints(newPoints);
 			next.doFilter(newLine);
Index: src/uk/me/parabola/mkgmap/filters/RoundCoordsFilter.java
===
--- src/uk/me/parabola/mkgmap/filters/RoundCoordsFilter.java	(revision 4616)
+++ src/uk/me/parabola/mkgmap/filters/RoundCoordsFilter.java	(working copy)
@@ -18,11 +18,14 @@
 
 import uk.me.parabola.imgfmt.app.Coord;
 import uk.me.parabola.imgfmt.app.CoordNode;
+import uk.me.parabola.log.Logger;
 import uk.me.parabola.mkgmap.general.MapElement;
 import uk.me.parabola.mkgmap.general.MapLine;
 import uk.me.parabola.mkgmap.general.MapRoad;
+import uk.me.parabola.mkgmap.general.MapShape;
 
 public class RoundCoordsFilter implements MapFilter {
+	private static final Logger log = Logger.getLogger(RoundCoordsFilter.class);
 
 	private int 

Re: [mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

I don't get any detectable failures, but this might cause difficulties
with adjacent junctions and/or housenumbers.

If there are two adjacent equal points, with the first not preserved
and the second preserved, the existing RemoveObsoleteFilter will
replace the first with the second and leave the second in as well. The
code clearly intended not to duplicate the preserved point, and, if
they were the other way around, doesn't.

Ticker

On Tue, 2021-03-23 at 12:38 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> please add a unit test that shows where the old code fails.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 23. März 2021 13:10
> An: mkgmap development
> Betreff: [mkgmap-dev] line/polygon filters fix
> 
> Hi Gerd
> 
> I was trying to diagnose a problem with a repeating points in
> polylines
> as reported by GPSMapEdit and found a problem in
> RemoveObsoletePointsFilter where it duplicates a point.
> 
> Also in this and/or RoundCoordsFilter I've made some changes:
> 1/ stop the chain when polygons get too small
> 2/ keep polygons closed with the same first/last point
> 3/ slight logic tidy-up
> 4/ add a couple of debug lines to be consistent
> 
> Patch attached.
> 
> Actually these didn't make any difference to the repeating points
> problem. This happens when there are enough unused bits in the last
> byte of the polyline bitsteam to represent an extra point. I can't
> see
> any good way of stopping this.
> 
> Ticker
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

I doubt if forced close is needed - a mix of identical and equal
endPoint polygons has been fed through the filters and RGN polyline
processing for a long time without *noticeable* problems.

But I was trying to diagnose a problem that could have been caused by
the point duplication or the not-properly-closed polygons and it seemed
better to fix and make consistent on my way to finding where the
problem lay.

Ticker

On Thu, 2021-03-25 at 09:59 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> why do you think that this forced closing is needed? Do we have code
> that relies on this?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 25. März 2021 10:47
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] line/polygon filters fix
> 
> Hi Gerd
> 
> Yes, you're right.
> 
> I've changed it so it only keeps polygons correctly closed.
> 
> Ticker
> 
> On Wed, 2021-03-24 at 18:54 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I don't understand the last changes in RoundCoordsFilter.java. What
> > if the replaced 1st point is preserved?
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 23. März 2021 14:17
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] line/polygon filters fix
> > 
> > Hi Gerd
> > 
> > I don't get any detectable failures, but this might cause
> > difficulties
> > with adjacent junctions and/or housenumbers.
> > 
> > If there are two adjacent equal points, with the first not
> > preserved
> > and the second preserved, the existing RemoveObsoleteFilter will
> > replace the first with the second and leave the second in as well.
> > The
> > code clearly intended not to duplicate the preserved point, and, if
> > they were the other way around, doesn't.
> > 
> > Ticker
> > 
> > On Tue, 2021-03-23 at 12:38 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > please add a unit test that shows where the old code fails.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Dienstag, 23. März 2021 13:10
> > > An: mkgmap development
> > > Betreff: [mkgmap-dev] line/polygon filters fix
> > > 
> > > Hi Gerd
> > > 
> > > I was trying to diagnose a problem with a repeating points in
> > > polylines
> > > as reported by GPSMapEdit and found a problem in
> > > RemoveObsoletePointsFilter where it duplicates a point.
> > > 
> > > Also in this and/or RoundCoordsFilter I've made some changes:
> > > 1/ stop the chain when polygons get too small
> > > 2/ keep polygons closed with the same first/last point
> > > 3/ slight logic tidy-up
> > > 4/ add a couple of debug lines to be consistent
> > > 
> > > Patch attached.
> > > 
> > > Actually these didn't make any difference to the repeating points
> > > problem. This happens when there are enough unused bits in the
> > > last
> > > byte of the polyline bitsteam to represent an extra point. I
> > > can't
> > > see
> > > any good way of stopping this.
> > > 
> > > Ticker
> > > ___
> > > mkgmap-dev mailing list
> > > mkgmap-dev@lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

Yes, you're right.

I've changed it so it only keeps polygons correctly closed.

Ticker

On Wed, 2021-03-24 at 18:54 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I don't understand the last changes in RoundCoordsFilter.java. What
> if the replaced 1st point is preserved?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 23. März 2021 14:17
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] line/polygon filters fix
> 
> Hi Gerd
> 
> I don't get any detectable failures, but this might cause
> difficulties
> with adjacent junctions and/or housenumbers.
> 
> If there are two adjacent equal points, with the first not preserved
> and the second preserved, the existing RemoveObsoleteFilter will
> replace the first with the second and leave the second in as well.
> The
> code clearly intended not to duplicate the preserved point, and, if
> they were the other way around, doesn't.
> 
> Ticker
> 
> On Tue, 2021-03-23 at 12:38 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > please add a unit test that shows where the old code fails.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Dienstag, 23. März 2021 13:10
> > An: mkgmap development
> > Betreff: [mkgmap-dev] line/polygon filters fix
> > 
> > Hi Gerd
> > 
> > I was trying to diagnose a problem with a repeating points in
> > polylines
> > as reported by GPSMapEdit and found a problem in
> > RemoveObsoletePointsFilter where it duplicates a point.
> > 
> > Also in this and/or RoundCoordsFilter I've made some changes:
> > 1/ stop the chain when polygons get too small
> > 2/ keep polygons closed with the same first/last point
> > 3/ slight logic tidy-up
> > 4/ add a couple of debug lines to be consistent
> > 
> > Patch attached.
> > 
> > Actually these didn't make any difference to the repeating points
> > problem. This happens when there are enough unused bits in the last
> > byte of the polyline bitsteam to represent an extra point. I can't
> > see
> > any good way of stopping this.
> > 
> > Ticker
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-devIndex: src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java
===
--- src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java	(revision 4618)
+++ src/uk/me/parabola/mkgmap/filters/RemoveObsoletePointsFilter.java	(working copy)
@@ -29,8 +29,6 @@
  */
 public class RemoveObsoletePointsFilter implements MapFilter {
 	private static final Logger log = Logger.getLogger(RemoveObsoletePointsFilter.class);
-	
-	final Coord[] areaTest = new Coord[3];
 
 	private boolean checkPreserved;
 	
@@ -48,18 +46,12 @@
 		MapLine line = (MapLine) element;
 		List points = line.getPoints();
 		int numPoints = points.size();
-		if (numPoints <= 1){
-			return;
-		}
-		int requiredPoints = (line instanceof MapShape ) ? 4:2; 
+		int requiredPoints = (line instanceof MapShape) ? 4 : 2;
 		List newPoints = new ArrayList<>(numPoints);
 		while (true){
 			boolean removedSpike = false;
-			numPoints = points.size();
-			
-
-			Coord lastP = points.get(0);
-			newPoints.add(lastP);
+			Coord lastP;
+			newPoints.add(points.get(0));
 			for (int i = 1; i < numPoints; i++) {
 Coord newP = points.get(i);
 int last = newPoints.size()-1;
@@ -72,6 +64,7 @@
 			continue;
 		} else if (!lastP.preserved()){
 			newPoints.set(last, newP); // replace last
+			continue;
 		} 
 	} else {  
 		continue;
@@ -104,36 +97,45 @@
 
 newPoints.add(newP);
 			}
-			if (!removedSpike || newPoints.size() < requiredPoints)
+			numPoints = newPoints.size();
+			if (numPoints < requiredPoints)
+return;
+			if (!removedSpike)
 break;
 			points = newPoints;
-			newPoints = new ArrayList<>(points.size());
+			newPoints = new ArrayList<>(numPoints);
 		}
 		if (line instanceof MapShape){
 			// Check special cases caused by the fact that the first and last point 
 			// in a shape are identical. 
-			while (newPoints.size() > 3){
-int nPo

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

I'm revising my opinion about considering the geometry to determine
inner and outer.  All the different BitSets, containsMatrix etc logic
is too complex, and if it then conflicts with the explicit definition
of the MP it just gets worse.

I'd simpify it along these lines:

 split the polygon/joinedWay list:
   error: the explicit conflicting inner/outer
   inners: at least one inner element
   outers: all the rest

 for each inner
   find a larger_area outer that contains a part of it
 add to the outer.listOfInners
 error if more than one

 for each outer
   cut out listOfInners
   apply tags
   output to the wayMap

This copes correctly with outer with inner hole that
contains another outer. If, in the above logic, more than one larger
outer is found, then choosing the smallest would allow this nesting to
any level.

Ticker

On Thu, 2021-03-18 at 10:21 +, Gerd Petermann wrote:
> Hi all,
> 
> I still struggle with the unit test because it's hard to say what we
> want to get in special cases.
> The current code doesn't really work in the way that I expected. I
> always thought that roles like "inner" and "outer" are completely
> ignored and that mkgmap calculates and uses the correct roles. This
> is only partly true. See attached file with MP were a forest contains
> a lake that contains a forest.
> For a nested polygon where the innermost ring has wrong role "inner"
> this doesn't work as expected. The forest  in the lake is ignored.
> With the correct role "outer" it is not ignored. No idea if this is
> intended or an error. Fortunately nested MP are very rare.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

On a road, if there is a normal Coord, immediately followed CoordNode
(junction) or HouseNumber node and they are in the same Garmin-square
for level0/maxRes then the CoordNode got duplicated. The isStraight
test won't remove it again because it might not be straight (depends on
another point) and, regardless, both points are 'preserved'.

I haven't followed the logic through the all app/net and app/trergn to
see what happens if road had duplicate CoordNodes or CoordNodes with
duplicate Ids.

Ticker

On Sun, 2021-03-28 at 09:12 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I tried to find a case where the bug in RemoveObsoletePointsFilter
> would really add the same point twice but failed. I tried with --x-no
> -force-end-nodes-routing-nodes so that the 1st point is not a
> CoordNode but still the first (and last) point of a MapLine is always
> preserved.
> So, we always get into the Utils.isStraight() test which contains the
> needed continue.
> So, in the end the bug has no effect on output, it just produced
> useless Utils.isStraight calls and possibly debug messages ""found
> three consecutive points on strictly straight line""
> 
> I commited only parts of your patch, I think the stuff regarding
> closed shapes is too confusing and the changed loop logic really
> doesn't improve robustness or readability.
> See r4619 and r4620.
> 
> Gerd
> 
> ____
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 25. März 2021 11:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] line/polygon filters fix
> 
> Hi Gerd
> 
> I doubt if forced close is needed - a mix of identical and equal
> endPoint polygons has been fed through the filters and RGN polyline
> processing for a long time without *noticeable* problems.
> 
> But I was trying to diagnose a problem that could have been caused by
> the point duplication or the not-properly-closed polygons and it
> seemed
> better to fix and make consistent on my way to finding where the
> problem lay.
> 
> Ticker
> 
> On Thu, 2021-03-25 at 09:59 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > why do you think that this forced closing is needed? Do we have
> > code
> > that relies on this?
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 25. März 2021 10:47
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] line/polygon filters fix
> > 
> > Hi Gerd
> > 
> > Yes, you're right.
> > 
> > I've changed it so it only keeps polygons correctly closed.
> > 
> > Ticker
> > 
> > On Wed, 2021-03-24 at 18:54 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > I don't understand the last changes in RoundCoordsFilter.java.
> > > What
> > > if the replaced 1st point is preserved?
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Dienstag, 23. März 2021 14:17
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] line/polygon filters fix
> > > 
> > > Hi Gerd
> > > 
> > > I don't get any detectable failures, but this might cause
> > > difficulties
> > > with adjacent junctions and/or housenumbers.
> > > 
> > > If there are two adjacent equal points, with the first not
> > > preserved
> > > and the second preserved, the existing RemoveObsoleteFilter will
> > > replace the first with the second and leave the second in as
> > > well.
> > > The
> > > code clearly intended not to duplicate the preserved point, and,
> > > if
> > > they were the other way around, doesn't.
> > > 
> > > Ticker
> > > 
> > > On Tue, 2021-03-23 at 12:38 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > please add a unit test that shows where the old code fails.
> > > > 
> > > > Gerd
> > > > 
> > > > 
> > > > Von: mkgmap-dev  im
> > > > Auftrag
> > > > von Ticker Berkin 
> > > > Gesendet: Dienstag, 23. März 2021 13:10
> > > > An: mkgmap development
> > > > Betreff: [mkgmap-dev] line/polygon filters fix
> > > > 
> > > > Hi Gerd
> > > > 
> > > > I was tryin

Re: [mkgmap-dev] line/polygon filters fix

[Ticker Berkin]

Hi Gerd

In answer to your first question - No; I was thinking that, at a higher
resolution, it wasn't straight, but this is irrelevant.

If all three points are equal they maybe isStraight should return
something else, but first two will have been de-dup already so won't
happen.

Ticker

On Mon, 2021-03-29 at 08:49 +, Gerd Petermann wrote:
> Ah, it may return STRAIGHT_SPIKE if all three points are equal. Have
> to test this again.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Montag, 29. März 2021 10:45
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] line/polygon filters fix
> 
> Hi Ticker,
> 
> can you give me an example where the isStraight test doesn't return
> STRICTLY_STRAIGHT if c2 and c3 are equal?
> 
> Gerd
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 29. März 2021 10:38
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] line/polygon filters fix
> 
> Hi Gerd
> 
> On a road, if there is a normal Coord, immediately followed CoordNode
> (junction) or HouseNumber node and they are in the same Garmin-square
> for level0/maxRes then the CoordNode got duplicated. The isStraight
> test won't remove it again because it might not be straight (depends
> on
> another point) and, regardless, both points are 'preserved'.
> 
> I haven't followed the logic through the all app/net and app/trergn
> to
> see what happens if road had duplicate CoordNodes or CoordNodes with
> duplicate Ids.
> 
> Ticker
> 
> On Sun, 2021-03-28 at 09:12 +, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I tried to find a case where the bug in RemoveObsoletePointsFilter
> > would really add the same point twice but failed. I tried with --x
> > -no
> > -force-end-nodes-routing-nodes so that the 1st point is not a
> > CoordNode but still the first (and last) point of a MapLine is
> > always
> > preserved.
> > So, we always get into the Utils.isStraight() test which contains
> > the
> > needed continue.
> > So, in the end the bug has no effect on output, it just produced
> > useless Utils.isStraight calls and possibly debug messages ""found
> > three consecutive points on strictly straight line""
> > 
> > I commited only parts of your patch, I think the stuff regarding
> > closed shapes is too confusing and the changed loop logic really
> > doesn't improve robustness or readability.
> > See r4619 and r4620.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von Ticker Berkin 
> > Gesendet: Donnerstag, 25. März 2021 11:14
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] line/polygon filters fix
> > 
> > Hi Gerd
> > 
> > I doubt if forced close is needed - a mix of identical and equal
> > endPoint polygons has been fed through the filters and RGN polyline
> > processing for a long time without *noticeable* problems.
> > 
> > But I was trying to diagnose a problem that could have been caused
> > by
> > the point duplication or the not-properly-closed polygons and it
> > seemed
> > better to fix and make consistent on my way to finding where the
> > problem lay.
> > 
> > Ticker
> > 
> > On Thu, 2021-03-25 at 09:59 +, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > why do you think that this forced closing is needed? Do we have
> > > code
> > > that relies on this?
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag
> > > von Ticker Berkin 
> > > Gesendet: Donnerstag, 25. März 2021 10:47
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] line/polygon filters fix
> > > 
> > > Hi Gerd
> > > 
> > > Yes, you're right.
> > > 
> > > I've changed it so it only keeps polygons correctly closed.
> > > 
> > > Ticker
> > > 
> > > On Wed, 2021-03-24 at 18:54 +, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > > 
> > > > I don't understand the last changes in RoundCoordsFilter.java.
> > > > What
> > > > if the replaced 1st point is preserved?
> > > > 
> > > > Gerd
> > > > 
> > > > 
> > > > Von: mkgmap-dev  im
> > > > Auftrag

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

Looking at the Polish format (cGPSmapper-UsrMan-v02.4.pdf), my
understanding is that a polygon definition with multiple Data#
statements at the same level defines a single outer with multiple
inners (Page 20 and 74/5).

Touching inner or outer rings is a slightly different problem and, I
think, the current code has made all its choices about this before the
complexity of geometry determination.

I wasn't suggesting stopping on errors, but if there are only 2 way
elements ending at a Coord, and one is explicitly marked "inner" and
the other "outer" then I think this can be errored and chucked.

Ticker  

On Wed, 2021-03-31 at 07:57 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I don't know what data we see when MultipolygonRelation is used with
> polish input, but I am sure that there is no info about roles.
> 
> I also thought about using the info about the area size as a quick
> indicator before doing complex tests.
> 
> The missing unit tests are my real problem. I've not (yet) found a
> good way to test the result. I'd like to have a test that allows to
> rotate or reverse rings so that we find errors with handling of the
> closing node.
> 
> I think we need the complex and somehow tolerant tests for the
> boundary calculations, we cannot simply stop on error. I know that
> some admin boundaries have very special cases (France), they are not
> exactly correct reg. the rules for touching outer rings.
> 
> 
> So, I started to use already tested code where possible, e.g.
> IsInUtil. See attached patch. Run times seem similar to the existing
> code in trunk.
> Just experimental so far, but at least a lot less code...
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Dienstag, 30. März 2021 15:17
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] tile takes very long time to generate
> 
> Hi Gerd
> 
> I'm revising my opinion about considering the geometry to determine
> inner and outer.  All the different BitSets, containsMatrix etc logic
> is too complex, and if it then conflicts with the explicit definition
> of the MP it just gets worse.
> 
> I'd simpify it along these lines:
> 
>  split the polygon/joinedWay list:
>error: the explicit conflicting inner/outer
>inners: at least one inner element
>outers: all the rest
> 
>  for each inner
>find a larger_area outer that contains a part of it
>  add to the outer.listOfInners
>  error if more than one
> 
>  for each outer
>cut out listOfInners
>apply tags
>output to the wayMap
> 
> This copes correctly with outer with inner hole that
> contains another outer. If, in the above logic, more than one larger
> outer is found, then choosing the smallest would allow this nesting
> to
> any level.
> 
> Ticker
> 
> On Thu, 2021-03-18 at 10:21 +, Gerd Petermann wrote:
> > Hi all,
> > 
> > I still struggle with the unit test because it's hard to say what
> > we
> > want to get in special cases.
> > The current code doesn't really work in the way that I expected. I
> > always thought that roles like "inner" and "outer" are completely
> > ignored and that mkgmap calculates and uses the correct roles. This
> > is only partly true. See attached file with MP were a forest
> > contains
> > a lake that contains a forest.
> > For a nested polygon where the innermost ring has wrong role
> > "inner"
> > this doesn't work as expected. The forest  in the lake is ignored.
> > With the correct role "outer" it is not ignored. No idea if this is
> > intended or an error. Fortunately nested MP are very rare.
> > 
> > Gerd
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

I think this might be the reason why you backed out the Polish
inner/outer definition changes:

http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2019q1/029478.html

Maybe there needs to be an option to say how multiple DATA{sameLevel}
should be handled:
a/ first outer, following inners.
b/ outers clockwise, inners anti-clockwise;
   presume each outer is followed by its inners.
c/ try to work out from geometry.

Ticker

On Wed, 2021-03-31 at 09:39 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> reg. polish input: There was code based on the documentation and I
> removed it again. Don't remember why, see changes for r4269 and r4272
> and
> http://gis.19327.n8.nabble.com/Artifacts-in-final-map-MP-gt-IMG-conve
> rsion-tc5932812.html
> 
> The current code is really too complex. A lot of the code exists for
> reporting purposes only. Another big portion is for the handling of
> incomplete data.
> There are probably hundreds of possible error cases with MPs. One of
> my goals is to find out which ones are tolerated by the current code
> and why they are tolerated.
> 
> If nobody offers an approach regarding unit tests we may try to
> simply write a 2nd implementation for MultipolygonRelation and use an
> option to switch between them.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [mkgmap-svn] Commit r4622: - use IsInUtil to do geometry calculations like insideness or outsideness: Allows to remove a lot of complex code but might be slower in some cases.

[Ticker Berkin]

Hi Gerd

I can't get to see this relation - it times out.

If element roles can be trusted (either OSM or Polish with some
appropriate rule) and there is only 1 outer, then no need to test
anything - all the inners can be assigned to this and proceed onto the
cutting stage.

If multiple outers then I agree, each inner needs to be tested against
all the outers (but not against each other). isPointInShape will be a
lot quicker than isLineInShape or the existing trunk logic.

Ticker

On Thu, 2021-04-01 at 14:49 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I still see no way how any of this will help reg. throughput. I think
> we need something like the BoundaryUtil.rasterArea() method as a
> Divide  approach, at least for monster relations like
> r9488835.
> The outer ring has +413.000 points and there are +15.000 rings. Even
> a single "point in shape" test requires lots of time for such a
> complex outer ring, and a single test will not find intersections.
> 
> Have to think about the data structures but I am sure this will be
> much faster for most of the really complex MP.
> 
> Gerd
> 
> 
> 
> ________
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Donnerstag, 1. April 2021 14:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] [mkgmap-svn] Commit r4622: - use IsInUtil
> to do geometry calculations like insideness or outsideness: Allows to
> remove a lot of complex code but might be slower in some cases.
> 
> Hi Gerd
> 
> 2 more thoughts:
> 
> For elements from Polish input, set a distinct role. This can be
> spotted early and either the order rule or direction rule can be
> applied (they are closed, so the area/direction can be calculated
> immediately). This then allowed OSM data with a blank role to be
> handled as suggested earlier.
> 
> The error reporting is not good - it can generate many lines in the
> log. An idea would be to have:
>   Element lists for each way-like problem.
>   Counts or flags for other problems.
>   etc
> and, for each problem detected, add all the ways in JoinedWays,
> increase appropriate count, set the flag etc. At the end, if any of
> these are populated, output the Relation and then, if full
> diagnostics,
> each populated problem type followed by lists of wayIds on same line
> etc.
> 
> Ticker
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] tile takes very long time to generate

[Ticker Berkin]

Hi Gerd

One of the ideas of the code is that it doesn't keep reversing all the
points as it gets to joining the ways to make a polygon. Once all the
ways have been gathered, then another stage is needed adds the points
from JoinedWay.originalWay to .points the correct order, removing the
intermediate joining points, etc.

Another part that's missing is what to do with unclosed ways, and here
I guess it can take a similar line to the existing logic.

I can do another patch, based on faster_mp branch - for real - if you
think this change is worth while.

Ticker

On Tue, 2021-04-06 at 10:45 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> I tried to use your patch mpInitialLogic.patch.
> It is missing the code to update the field Way.points, means, the
> final polygons only contain the points of one way.
> 
> Not sure how you used that code?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Montag, 15. März 2021 17:15
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] tile takes very long time to generate
> 
> Hi Gerd
> 
> You might consider the some of the ideas here as improvements to the
> initial parts of MP processing.
> 
> This is a patch based on trunk rather than the new branch. It isn't
> structured as for final usage, rather for minimising the spread of
> changes, working in parallel with the existing code so I could see if
> found the same MP problems as the existing code and having clearly
> identifiable diagnostics in the log file.
> 
> Ticker
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev