Re: [mkgmap-dev] Error processing tile
Hi Ticker, okay, it will probably always be hard to understand that part of the code. So let's see what comes next... Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Montag, 5. Juni 2023 12:39 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd The added comments are helpful but overall I think the readability is worse; it takes careful reading to see where the dp filter is placed. If other tweaks to different orderings between normal and parallel chains are necessary it will become even more confusing. Ticker On Mon, 2023-06-05 at 09:41 +, Gerd Petermann wrote: > Hi Ticker, > > thanks, I've committed the patch with r4909. What do you think about > the attached patch to improve readability? > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error processing tile
Hi Gerd The added comments are helpful but overall I think the readability is worse; it takes careful reading to see where the dp filter is placed. If other tweaks to different orderings between normal and parallel chains are necessary it will become even more confusing. Ticker On Mon, 2023-06-05 at 09:41 +, Gerd Petermann wrote: > Hi Ticker, > > thanks, I've committed the patch with r4909. What do you think about > the attached patch to improve readability? > > Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error processing tile
Hi Ticker,
thanks, I've committed the patch with r4909. What do you think about the
attached patch to improve readability?
Gerd
Von: mkgmap-dev im Auftrag von Ticker
Berkin
Gesendet: Montag, 5. Juni 2023 09:54
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Error processing tile
Hi Gerd
Yes. MapSplitter enforces all the subDivision limits (#items and
#pointsInItems) before the filtering has happened. A Polyline might
need to be split into many lines if it has a large number of points,
but, at a given resolution, many points might be discarded. Using this
fact it can estimate the worst-case number of line splits to use in the
subDivision line count limit. This logic makes the most significant
difference at low resolutions but, in Carlos's data, there must have
been roads with multiple points in res24 squares.
Ticker
On Mon, 2023-06-05 at 06:48 +, Gerd Petermann wrote:
> Hi Ticker,
>
> okay, after some more debugging I think I understand. The problem
> is/was that the code in PredictFilterPoints assumes that the obsolete
> point removal happens before line splitting,
> but the actual order in MapBuilder is/was different (and wrong).
>
> Gerd
>
>
> Von: mkgmap-dev im Auftrag
> von Gerd Petermann
> Gesendet: Montag, 5. Juni 2023 08:14
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Error processing tile
>
> Hi Ticker,
>
> The fix helps with the data from Carlos and it also seems to improve
> the img size.
> So far so good.
> I don't understand if this patch really fixes the error or if just
> changes "something" which prevents the error.
> Or in other words: How does it prevent the overflow of the counter?
>
> Gerd
>
>
> Von: mkgmap-dev im Auftrag
> von Ticker Berkin
> Gesendet: Sonntag, 4. Juni 2023 10:15
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Error processing tile
>
> Hi Gerd
>
> Realising that something is increasing the number of lines in a
> subdivision and I wasn't getting the problem with my build I
> remembered
> I'd noticed that this could happen due to changes made to the line
> filtering in the low-res-opt branch work and supplied
> filterOrderLowRes.patch to restore some of the vital orderings - see
> forwarded email.
>
> The vital part is relating to MapSplitter/Area predicting the maximum
> number of splits to a line after RoundCoords & RemoveObsoletePoints
> have done their work.
>
> Attached is patch of the code I've been using appropriate to trunk.
>
> Ticker
>
> Forwarded Message
> From: Ticker Berkin
> To: Development list for mkgmap
> Subject: Re: [mkgmap-dev] Problems with sea in overview map
> Date: Fri, 21 May 2021 17:10:07 +0100
>
> Hi Gerd
>
> I'd been doing some investigation of filters ordering (based on
> trunk).
> I'd also done the pre-filtering of lines & shapes by minRes.
>
> My conclusions are:
>
> Shapes:
> It is better to run SizeFilter after RemoveObsoleteFilter.
> It is more efficient to run DP filter after both of these.
>
> Lines:
> LineSplitterFilter should be run after everything that can remove
> points.
> It is more efficient to run DP after RemoveObsoleteFilter.
>
> I've adapted my changes into a patch for the low-res-opt branch,
> along
> with removal of some resolution tests that are now redundant.
>
> For the contourFilters, I've left DP as the first filter but moved
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Index: src/uk/me/parabola/mkgmap/build/MapBuilder.java
===
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java (revision 4909)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java (working copy)
@@ -1243,34 +1243,35 @@
config.setLevel(div.getZoom().getLevel());
config.setHasNet(hasNet);
- LayerFilterChain normalFilters = new LayerFilterChain(config);
- LayerFilterChain keepParallelFilters = new
LayerFilterChain(config);
+ List activeFilters = new ArrayList<>();
DouglasPeuckerFilter dp = null;
if (enableLineCleanFilters && (res < 24)) {
- MapFilter rounder = new RoundCoordsFilter();
- MapFilter sizeFilter = new SizeFilter(MIN_SIZE_LINE);
- normalFilters.addFilter(rounder);
- normalFilters.addFilter(sizeFilter);
+ activeFilters.add(new
[mkgmap-dev] Commit r4909: Fix possible AssertionError in RoadIndex.java:45 reported by Carlos Dávila
Version mkgmap-r4909 was committed by gerd on Mon, 05 Jun 2023 Fix possible AssertionError in RoadIndex.java:45 reported by Carlos Dávila filterOrder_v2.patch by Ticker Berkin plus changes in unit tests by Gerd Petermann The patch changes the order of the filters to the order that is expected in the PredictFilterPoints. Expected effects are small changes in file sizes or compile time http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4909 ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] Commit r4908: update some values in unit tests to check that they work on the build system
Version mkgmap-r4908 was committed by gerd on Mon, 05 Jun 2023 update some values in unit tests to check that they work on the build system http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4908 ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error processing tile
Hi Gerd Yes. MapSplitter enforces all the subDivision limits (#items and #pointsInItems) before the filtering has happened. A Polyline might need to be split into many lines if it has a large number of points, but, at a given resolution, many points might be discarded. Using this fact it can estimate the worst-case number of line splits to use in the subDivision line count limit. This logic makes the most significant difference at low resolutions but, in Carlos's data, there must have been roads with multiple points in res24 squares. Ticker On Mon, 2023-06-05 at 06:48 +, Gerd Petermann wrote: > Hi Ticker, > > okay, after some more debugging I think I understand. The problem > is/was that the code in PredictFilterPoints assumes that the obsolete > point removal happens before line splitting, > but the actual order in MapBuilder is/was different (and wrong). > > Gerd > > > Von: mkgmap-dev im Auftrag > von Gerd Petermann > Gesendet: Montag, 5. Juni 2023 08:14 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Error processing tile > > Hi Ticker, > > The fix helps with the data from Carlos and it also seems to improve > the img size. > So far so good. > I don't understand if this patch really fixes the error or if just > changes "something" which prevents the error. > Or in other words: How does it prevent the overflow of the counter? > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Sonntag, 4. Juni 2023 10:15 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] Error processing tile > > Hi Gerd > > Realising that something is increasing the number of lines in a > subdivision and I wasn't getting the problem with my build I > remembered > I'd noticed that this could happen due to changes made to the line > filtering in the low-res-opt branch work and supplied > filterOrderLowRes.patch to restore some of the vital orderings - see > forwarded email. > > The vital part is relating to MapSplitter/Area predicting the maximum > number of splits to a line after RoundCoords & RemoveObsoletePoints > have done their work. > > Attached is patch of the code I've been using appropriate to trunk. > > Ticker > > Forwarded Message > From: Ticker Berkin > To: Development list for mkgmap > Subject: Re: [mkgmap-dev] Problems with sea in overview map > Date: Fri, 21 May 2021 17:10:07 +0100 > > Hi Gerd > > I'd been doing some investigation of filters ordering (based on > trunk). > I'd also done the pre-filtering of lines & shapes by minRes. > > My conclusions are: > > Shapes: > It is better to run SizeFilter after RemoveObsoleteFilter. > It is more efficient to run DP filter after both of these. > > Lines: > LineSplitterFilter should be run after everything that can remove > points. > It is more efficient to run DP after RemoveObsoleteFilter. > > I've adapted my changes into a patch for the low-res-opt branch, > along > with removal of some resolution tests that are now redundant. > > For the contourFilters, I've left DP as the first filter but moved > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error processing tile
Hi Ticker, okay, after some more debugging I think I understand. The problem is/was that the code in PredictFilterPoints assumes that the obsolete point removal happens before line splitting, but the actual order in MapBuilder is/was different (and wrong). Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Montag, 5. Juni 2023 08:14 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Ticker, The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email. The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work. Attached is patch of the code I've been using appropriate to trunk. Ticker Forwarded Message From: Ticker Berkin To: Development list for mkgmap Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100 Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error processing tile
Hi Ticker, The fix helps with the data from Carlos and it also seems to improve the img size. So far so good. I don't understand if this patch really fixes the error or if just changes "something" which prevents the error. Or in other words: How does it prevent the overflow of the counter? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Sonntag, 4. Juni 2023 10:15 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error processing tile Hi Gerd Realising that something is increasing the number of lines in a subdivision and I wasn't getting the problem with my build I remembered I'd noticed that this could happen due to changes made to the line filtering in the low-res-opt branch work and supplied filterOrderLowRes.patch to restore some of the vital orderings - see forwarded email. The vital part is relating to MapSplitter/Area predicting the maximum number of splits to a line after RoundCoords & RemoveObsoletePoints have done their work. Attached is patch of the code I've been using appropriate to trunk. Ticker Forwarded Message From: Ticker Berkin To: Development list for mkgmap Subject: Re: [mkgmap-dev] Problems with sea in overview map Date: Fri, 21 May 2021 17:10:07 +0100 Hi Gerd I'd been doing some investigation of filters ordering (based on trunk). I'd also done the pre-filtering of lines & shapes by minRes. My conclusions are: Shapes: It is better to run SizeFilter after RemoveObsoleteFilter. It is more efficient to run DP filter after both of these. Lines: LineSplitterFilter should be run after everything that can remove points. It is more efficient to run DP after RemoveObsoleteFilter. I've adapted my changes into a patch for the low-res-opt branch, along with removal of some resolution tests that are now redundant. For the contourFilters, I've left DP as the first filter but moved ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
