Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Gerd Petermann]

Hi Ticker,

reg. largeObjectArea:
I found some cases where 5 or more boundary lines with > 1000 points ended up 
in the same small map-area,
e.g. in 63240103 with my split-file of Germany. This happens quite often 
because those lines are added to the map
for different admin-levels.
I can send more details tomorrow.

Gerd


Von: mkgmap-dev  im Auftrag von Ticker 
Berkin 
Gesendet: Dienstag, 17. Januar 2017 17:25:00
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by 
"Too many POIs at location " error message

Hi Gerd

Sorry about the delay in replying.

I think the solution needs to be along the lines of the way
largeObjectArea creates addition subdivisions, but with changed rules,
and replacing largeObjectArea and !useNormalSplit logic. Is there an
existing problem with largeObjectArea?

Some thoughts on how it might work:
1/ much more accurate MapArea.getEstimatedSizes, that takes into
account zoom dependent filtering as best as is feasible.
2/ MapSplitter.addAreasToList stops attempting to split an area in half
once its size is less than quite a few pixels at the zoom level.
Instead it passes a new flag to MapArea.split (or even a new routine)
3/ If this flag is set: have ArrayList of areas and usedSize.
estimate size of item to be added (with 1/). add to first area that it
will fit into. If none, create another.

This will also improve behaviour for low resolution (esp. overview)
maps where excess splitting takes place because it considers the data
usage at res 2. At the actual resolution, the number of points in a
shape/line might be vastly reduced.

Ticker

On Mon, 2017-01-16 at 15:48 +, Gerd Petermann wrote:
> Hi Ticker,
>
> (see 3756)
> the useNormalSplit code is still needed, maybe because the rules for
> the largeObjectAreas are not okay.
> So if you find a better solution please let me know. I am also no
> longer convinced that the code in MapSplitter
> which sets wantSplit to true is a good idea, esp. in combination with
> --order-by-decreasing-area I guess
> it would be better to avoid unnessary splits. Am i right?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Sonntag, 15. Januar 2017 15:10:48
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ...
> followed by "Too many POIs at location " error message
>
> Hi Ticker,
>
> the original intention of the useNormalSplit code  was to solve a
> problem with two huge polygons having the same center point.
> At that time I tried to minimize the possibility that this code is
> executed in other cases.
> However, it seems that this is dead code since r3351 because of the
> handling with largeObjectAreas.
> I think I should remove it.
>
> Gerd
>
>
>
>
>
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Sonntag, 15. Januar 2017 14:41:27
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ...
> followed by "Too many POIs at location " error message
>
> Hi Gerd
>
> Yes; however, regardless of useNormalSplit, with --order-by... the
> shapes need to be split into their correct area.
>
> This bit of code has always troubled me. I wasn't sure if its
> objective
> was to lessen the chance of empty subdivisions or to lessen the
> chance
> of exceeding maximum data sizes in the subdivision.
>
> Maybe splitIntoAreas (and, for that matter, points) shouldn't set
> used[area] to true so that lines can be distributed between the two
> areas.
>
> Also the test ... && (this.lines.size() > 1 || this.shapes.size() >
> 1))
> should take ((lines.size + shapes.size) > 1). But not including
> shapes
> if --order-by...
>
> Another possibility is that the caller of MapArea.split, with
> accurate
> information on the number/size of subdivision data items, knowing
> that
> this is more that will fit into a subDivision and further splits by
> area are not possible, passes a flag that make it send excess items
> into other subdivisions - very much like the way it handles
> largeObjectAreas
>
> Ticker
>
>
> On Sun, 2017-01-15 at 10:00 +, Gerd Petermann wrote:
> > Hi all,
> >
> > attached is a patch to solve this problem.
> >
> > A binary is here:
> > http://files.mkgmap.org.uk/download/327/mkgmap.jar
> >
> > @Ticker: Please review.
> > I've noticed that with --order-by-decreasing-area
> > the code in MapArea to set useNormalSplit may not be used because
> > the condition used[0] != used[1] is never true when there is a
> > shape
> > in that area. I think this can cause problems if we also have some
> > complex
> > lines with (nearly) the same center 

Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Ticker Berkin]

Hi Gerd

Sorry about the delay in replying.

I think the solution needs to be along the lines of the way
largeObjectArea creates addition subdivisions, but with changed rules,
and replacing largeObjectArea and !useNormalSplit logic. Is there an
existing problem with largeObjectArea?

Some thoughts on how it might work:
1/ much more accurate MapArea.getEstimatedSizes, that takes into
account zoom dependent filtering as best as is feasible.
2/ MapSplitter.addAreasToList stops attempting to split an area in half
once its size is less than quite a few pixels at the zoom level.
Instead it passes a new flag to MapArea.split (or even a new routine)
3/ If this flag is set: have ArrayList of areas and usedSize.
estimate size of item to be added (with 1/). add to first area that it
will fit into. If none, create another.

This will also improve behaviour for low resolution (esp. overview)
maps where excess splitting takes place because it considers the data
usage at res 2. At the actual resolution, the number of points in a
shape/line might be vastly reduced.

Ticker
 
On Mon, 2017-01-16 at 15:48 +, Gerd Petermann wrote:
> Hi Ticker,
> 
> (see 3756)
> the useNormalSplit code is still needed, maybe because the rules for
> the largeObjectAreas are not okay.
> So if you find a better solution please let me know. I am also no
> longer convinced that the code in MapSplitter
> which sets wantSplit to true is a good idea, esp. in combination with
> --order-by-decreasing-area I guess
> it would be better to avoid unnessary splits. Am i right?
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Gerd Petermann 
> Gesendet: Sonntag, 15. Januar 2017 15:10:48
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ...
> followed by "Too many POIs at location " error message
> 
> Hi Ticker,
> 
> the original intention of the useNormalSplit code  was to solve a
> problem with two huge polygons having the same center point.
> At that time I tried to minimize the possibility that this code is
> executed in other cases.
> However, it seems that this is dead code since r3351 because of the
> handling with largeObjectAreas.
> I think I should remove it.
> 
> Gerd
> 
> 
> 
> 
> 
> 
> Von: mkgmap-dev  im Auftrag
> von Ticker Berkin 
> Gesendet: Sonntag, 15. Januar 2017 14:41:27
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ...
> followed by "Too many POIs at location " error message
> 
> Hi Gerd
> 
> Yes; however, regardless of useNormalSplit, with --order-by... the
> shapes need to be split into their correct area.
> 
> This bit of code has always troubled me. I wasn't sure if its
> objective
> was to lessen the chance of empty subdivisions or to lessen the
> chance
> of exceeding maximum data sizes in the subdivision.
> 
> Maybe splitIntoAreas (and, for that matter, points) shouldn't set
> used[area] to true so that lines can be distributed between the two
> areas.
> 
> Also the test ... && (this.lines.size() > 1 || this.shapes.size() >
> 1))
> should take ((lines.size + shapes.size) > 1). But not including
> shapes
> if --order-by...
> 
> Another possibility is that the caller of MapArea.split, with
> accurate
> information on the number/size of subdivision data items, knowing
> that
> this is more that will fit into a subDivision and further splits by
> area are not possible, passes a flag that make it send excess items
> into other subdivisions - very much like the way it handles
> largeObjectAreas
> 
> Ticker
> 
> 
> On Sun, 2017-01-15 at 10:00 +, Gerd Petermann wrote:
> > Hi all,
> > 
> > attached is a patch to solve this problem.
> > 
> > A binary is here:
> > http://files.mkgmap.org.uk/download/327/mkgmap.jar
> > 
> > @Ticker: Please review.
> > I've noticed that with --order-by-decreasing-area
> > the code in MapArea to set useNormalSplit may not be used because
> > the condition used[0] != used[1] is never true when there is a
> > shape
> > in that area. I think this can cause problems if we also have some
> > complex
> > lines with (nearly) the same center point in the same area.
> > 
> > Gerd
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> 

Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Gerd Petermann]

Hi Ticker,

(see 3756)
the useNormalSplit code is still needed, maybe because the rules for the 
largeObjectAreas are not okay.
So if you find a better solution please let me know. I am also no longer 
convinced that the code in MapSplitter
which sets wantSplit to true is a good idea, esp. in combination with 
--order-by-decreasing-area I guess
it would be better to avoid unnessary splits. Am i right?

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Sonntag, 15. Januar 2017 15:10:48
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by 
"Too many POIs at location " error message

Hi Ticker,

the original intention of the useNormalSplit code  was to solve a problem with 
two huge polygons having the same center point.
At that time I tried to minimize the possibility that this code is executed in 
other cases.
However, it seems that this is dead code since r3351 because of the handling 
with largeObjectAreas.
I think I should remove it.

Gerd






Von: mkgmap-dev  im Auftrag von Ticker 
Berkin 
Gesendet: Sonntag, 15. Januar 2017 14:41:27
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by 
"Too many POIs at location " error message

Hi Gerd

Yes; however, regardless of useNormalSplit, with --order-by... the
shapes need to be split into their correct area.

This bit of code has always troubled me. I wasn't sure if its objective
was to lessen the chance of empty subdivisions or to lessen the chance
of exceeding maximum data sizes in the subdivision.

Maybe splitIntoAreas (and, for that matter, points) shouldn't set
used[area] to true so that lines can be distributed between the two
areas.

Also the test ... && (this.lines.size() > 1 || this.shapes.size() > 1))
should take ((lines.size + shapes.size) > 1). But not including shapes
if --order-by...

Another possibility is that the caller of MapArea.split, with accurate
information on the number/size of subdivision data items, knowing that
this is more that will fit into a subDivision and further splits by
area are not possible, passes a flag that make it send excess items
into other subdivisions - very much like the way it handles
largeObjectAreas

Ticker


On Sun, 2017-01-15 at 10:00 +, Gerd Petermann wrote:
> Hi all,
>
> attached is a patch to solve this problem.
>
> A binary is here:
> http://files.mkgmap.org.uk/download/327/mkgmap.jar
>
> @Ticker: Please review.
> I've noticed that with --order-by-decreasing-area
> the code in MapArea to set useNormalSplit may not be used because
> the condition used[0] != used[1] is never true when there is a shape
> in that area. I think this can cause problems if we also have some
> complex
> lines with (nearly) the same center point in the same area.
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Gerd Petermann]

Hi Ticker,

the original intention of the useNormalSplit code  was to solve a problem with 
two huge polygons having the same center point.
At that time I tried to minimize the possibility that this code is executed in 
other cases.
However, it seems that this is dead code since r3351 because of the handling 
with largeObjectAreas.
I think I should remove it.

Gerd






Von: mkgmap-dev  im Auftrag von Ticker 
Berkin 
Gesendet: Sonntag, 15. Januar 2017 14:41:27
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by 
"Too many POIs at location " error message

Hi Gerd

Yes; however, regardless of useNormalSplit, with --order-by... the
shapes need to be split into their correct area.

This bit of code has always troubled me. I wasn't sure if its objective
was to lessen the chance of empty subdivisions or to lessen the chance
of exceeding maximum data sizes in the subdivision.

Maybe splitIntoAreas (and, for that matter, points) shouldn't set
used[area] to true so that lines can be distributed between the two
areas.

Also the test ... && (this.lines.size() > 1 || this.shapes.size() > 1))
should take ((lines.size + shapes.size) > 1). But not including shapes
if --order-by...

Another possibility is that the caller of MapArea.split, with accurate
information on the number/size of subdivision data items, knowing that
this is more that will fit into a subDivision and further splits by
area are not possible, passes a flag that make it send excess items
into other subdivisions - very much like the way it handles
largeObjectAreas

Ticker


On Sun, 2017-01-15 at 10:00 +, Gerd Petermann wrote:
> Hi all,
>
> attached is a patch to solve this problem.
>
> A binary is here:
> http://files.mkgmap.org.uk/download/327/mkgmap.jar
>
> @Ticker: Please review.
> I've noticed that with --order-by-decreasing-area
> the code in MapArea to set useNormalSplit may not be used because
> the condition used[0] != used[1] is never true when there is a shape
> in that area. I think this can cause problems if we also have some
> complex
> lines with (nearly) the same center point in the same area.
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Ticker Berkin]

Hi Gerd

Yes; however, regardless of useNormalSplit, with --order-by... the
shapes need to be split into their correct area.

This bit of code has always troubled me. I wasn't sure if its objective
was to lessen the chance of empty subdivisions or to lessen the chance
of exceeding maximum data sizes in the subdivision.

Maybe splitIntoAreas (and, for that matter, points) shouldn't set
used[area] to true so that lines can be distributed between the two
areas.

Also the test ... && (this.lines.size() > 1 || this.shapes.size() > 1))
should take ((lines.size + shapes.size) > 1). But not including shapes
if --order-by...

Another possibility is that the caller of MapArea.split, with accurate
information on the number/size of subdivision data items, knowing that
this is more that will fit into a subDivision and further splits by
area are not possible, passes a flag that make it send excess items
into other subdivisions - very much like the way it handles
largeObjectAreas

Ticker


On Sun, 2017-01-15 at 10:00 +, Gerd Petermann wrote:
> Hi all,
> 
> attached is a patch to solve this problem. 
> 
> A binary is here:
> http://files.mkgmap.org.uk/download/327/mkgmap.jar
> 
> @Ticker: Please review.
> I've noticed that with --order-by-decreasing-area 
> the code in MapArea to set useNormalSplit may not be used because
> the condition used[0] != used[1] is never true when there is a shape
> in that area. I think this can cause problems if we also have some
> complex
> lines with (nearly) the same center point in the same area.
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] [Patch] "Area too small to split at " ... followed by "Too many POIs at location " error message

[Gerd Petermann]

Hi all,

attached is a patch to solve this problem. 

A binary is here:
http://files.mkgmap.org.uk/download/327/mkgmap.jar

@Ticker: Please review.
I've noticed that with --order-by-decreasing-area 
the code in MapArea to set useNormalSplit may not be used because
the condition used[0] != used[1] is never true when there is a shape
in that area. I think this can cause problems if we also have some complex
lines with (nearly) the same center point in the same area.

Gerd

too-many-poi.patch
Description: too-many-poi.patch
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev