Hi Gerd
I found another unexpected case: giving the --dem... option caused the
overview map to have NET and NOD sections
To stops surprises, we should either pass in all the args and let the
receiver of them decide what to do with them or pass in only the
presumed significant options.
I think it much better to pass them all in; the logic to decide is then
all in the place where it is significant. I've made some changes to
this effect - attached.
In the case of --order-by-decreasing area, the final overview map needs
to know that this is wanted so it can keep the polygons in the same
order in each detail tile.
The large ovm_ size is because it has a LBL section containing all the
names for all levels. Unused ones don't make it into the final overview
so I don't think this is worth bothering with.
I've removed the scans for 0x4a polygons and do the equivalent when
they are inserted per detail tile. Also I've also removed some
misleading comments
Ticker
On Wed, 2020-12-30 at 11:21 +0000, Gerd Petermann wrote:
> Hi Ticker,
>
> yes, this code needs review, thanks for this. For now I've just
> disabled the --order-by-decreasing-area option for the overview map
> in r4590
>
> I am not sure if it would be better to always pass the args to
> MapBuilder or only those for DEM. I'd prefer to always pass them but
> maybe there are other side effects
> Reg. size 1%: My understanding was that the merging of shapes is
> responsible for all of this, but I might be wrong.
>
> I am working on a routing issue that I found while looking at Carlos'
> problem. It only happens with --x-no-force-end-nodes-routing-nodes.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <[email protected]> im Auftrag
> von Ticker Berkin <[email protected]>
> Gesendet: Mittwoch, 30. Dezember 2020 09:50
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Gerd
>
> I'm going to experiment with the combined overview map and which
> options cause problems. Also look at the effect of 0x4a polygons at
> just 1 level.
>
> I also notices that the combined overview (osmmap.img) is a fraction
> of
> the size (~1%) of the sum of the ovm_ images that are used to build
> it.
>
> Ticker
>
> On Tue, 2020-12-29 at 15:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > thanks for the hints. I agree that the code in OverviewBuilder is
> > not
> > very clear. I'll have a closer look tomorrow, but at least we
> > should
> > remove the -order-by-decreasing-area when copying the options. Or
> > maybe I change the code so that only the hgt/dem options are
> > copied.
> > I guess I was too lazy there.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <[email protected]> im Auftrag
> > von Ticker Berkin <[email protected]>
> > Gesendet: Dienstag, 29. Dezember 2020 10:50
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
> >
> > Hi Gerd & Carlos
> >
> > Reading and trying to understand the code, I'm finding a few
> > strange
> > things with the Overview map generation, DEM, 0x4a etc
> >
> > The most significant is that the MapBuilder invocation for the
> > combined
> > overview map normally runs without any options being passed to it.
> > Only
> > if --overview-dem-dist is supplied are all the other options
> > (including
> > --order-by-decreasing-area) passed in. I'm not sure if would be a
> > good
> > idea to supply all every time or just be more selective and filter
> > out
> > all but the necessary DEM options.
> >
> > I'm still investigating the overview map levels. The ovm_ files are
> > produced with all the overview-levels as specified by options. When
> > this is read back by the overview combiner, a 0x4a polygon is added
> > covering each ovm_ tile, but it looks like it is at all levels, so,
> > for
> > a tile with a large area or lots of detail is very likely to be
> > split
> > (if --order-by) or shunted around into another subdivision and
> > multiple
> > copies might exist.
> >
> > My understanding of the purpose of the 0x4a is that, in the
> > overview
> > map, there should be exactly 1 per detailed tile. It would be
> > sensible
> > to set its maxResolution so it only occurs at one level.
> >
> > After the 0x4a polygon has been added, a couple of bits of code
> > scan
> > for them in all the overview polygons. It might be possible to
> > improve
> > this, given they have just been added in the same processing phase.
> >
> > If --order-by-decreasing-area is used, the overview map combiner
> > shouldn't attempt to respect it because it doesn't have the full
> > size
> > information of polygons that cross a tile boundary. Rather it
> > should
> > respect the polygon ordering in each ovm_ tile. I'm not sure yet
> > this
> > is feasible; Maybe something like the equivalent of --preserve
> > -element
> > order for this phase and look at all the logic paths of polygons to
> > stop any other order changes due to merging etc.
> >
> > Ticker
> >
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > [email protected]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > [email protected]
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [email protected]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> [email protected]
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Index: src/uk/me/parabola/imgfmt/app/map/Map.java
===================================================================
--- src/uk/me/parabola/imgfmt/app/map/Map.java (revision 4591)
+++ src/uk/me/parabola/imgfmt/app/map/Map.java (working copy)
@@ -118,8 +118,10 @@
}
public void config(EnhancedProperties props) {
+ boolean isOverviewComponent = OverviewBuilder.isOverviewImg(mapName);
+ boolean isOverviewCombined = mapName.equals(props.getProperty("overview-mapname", "osmmap"));
// we don't want routing info in the overview map (for now)
- if (!OverviewBuilder.isOverviewImg(mapName)){
+ if (!isOverviewComponent && !isOverviewCombined) {
try {
if (props.containsKey("route") || props.containsKey("net") || props.containsKey("housenumbers")) {
addNet();
@@ -130,6 +132,11 @@
} catch (FileExistsException e) {
log.warn("Could not add NET and/or NOD sections");
}
+ // this sets things like draw-priority/transparent/custom
+ // and not relevant to overview maps
+ treFile.config(props);
+ }
+ if (!isOverviewComponent) { // allow dem on final overview but not in the ovm_
if (props.containsKey("dem")) {
try {
addDem();
@@ -138,7 +145,6 @@
}
}
}
- treFile.config(props);
}
private void addNet() throws FileExistsException {
Index: src/uk/me/parabola/mkgmap/build/MapBuilder.java
===================================================================
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java (revision 4591)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java (working copy)
@@ -127,6 +127,9 @@
private static final int MIN_SIZE_LINE = 1;
+ private final boolean isOverviewComponent;
+ private final boolean isOverviewCombined;
+
private final java.util.Map<MapPoint,POIRecord> poimap = new HashMap<>();
private final java.util.Map<MapPoint,City> cityMap = new HashMap<>();
private List<String> mapInfo = new ArrayList<>();
@@ -175,10 +178,12 @@
private HGTConverter.InterpolationMethod demInterpolationMethod;
- public MapBuilder() {
+ public MapBuilder(boolean overviewComponent, boolean overviewCombined) {
regionName = null;
locationAutofill = Collections.emptySet();
locator = new Locator();
+ this.isOverviewComponent = overviewComponent;
+ this.isOverviewCombined = overviewCombined;
}
public void config(EnhancedProperties props) {
@@ -218,7 +223,7 @@
driveOnLeft = false;
orderByDecreasingArea = props.getProperty("order-by-decreasing-area", false);
pathsToHGT = props.getProperty("dem", null);
- demDists = parseDemDists(props.getProperty("dem-dists", "-1"));
+ String demDistStr = props.getProperty("dem-dists", "-1");
demOutsidePolygonHeight = (short) props.getProperty("dem-outside-polygon", HGTReader.UNDEF);
String demPolygonFile = props.getProperty("dem-poly", null);
if (demPolygonFile != null) {
@@ -239,6 +244,18 @@
throw new IllegalArgumentException("invalid argument for option dem-interpolation: '" + ipm +
"' supported are 'bilinear', 'bicubic', or 'auto'");
}
+
+ if (isOverviewCombined) { // some alternate options, some invalid etc
+ demDistStr = props.getProperty("overview-dem-dists", "-1");
+ mergeLines = true;
+ if (orderByDecreasingArea) {
+ orderByDecreasingArea = false;
+ mergeShapes = false; // shape order in ovm_ imgs must be preserved to have the effect of above
+ } else {
+ mergeShapes = true;
+ }
+ }
+ demDists = parseDemDists(demDistStr);
}
private static List<Integer> parseDemDists(String demDists) {
@@ -339,7 +356,7 @@
demArea = demPolygon;
}
}
- if (demArea == null && src instanceof OverviewMapDataSource) {
+ if (demArea == null && isOverviewCombined) {
Path2D demPoly = ((OverviewMapDataSource) src).getTileAreaPath();
if (demPoly != null) {
demArea = new java.awt.geom.Area(demPoly);
@@ -732,12 +749,11 @@
// The top level has to cover the whole map without subdividing, so
// do a special check to make sure.
LevelInfo[] levels = null;
- if (src instanceof OverviewMapDataSource) {
- mergeLines = true;
- prepShapesForMerge(src.getShapes());
- mergeShapes = true;
+ if (isOverviewCombined) {
+ if (mergeShapes)
+ prepShapesForMerge(src.getShapes());
levels = src.mapLevels();
- } else if (OverviewBuilder.isOverviewImg(map.getFilename())) {
+ } else if (isOverviewComponent) {
levels = src.overviewMapLevels();
} else {
levels = src.mapLevels();
@@ -967,7 +983,7 @@
private void processInfo(Map map, LoadableMapDataSource src) {
// The bounds of the map.
map.setBounds(src.getBounds());
- if (!(src instanceof OverviewMapDataSource))
+ if (!isOverviewCombined)
poiDisplayFlags |= TREHeader.POI_FLAG_DETAIL;
poiDisplayFlags |= src.getPoiDispFlag();
Index: src/uk/me/parabola/mkgmap/combiners/OverviewBuilder.java
===================================================================
--- src/uk/me/parabola/mkgmap/combiners/OverviewBuilder.java (revision 4591)
+++ src/uk/me/parabola/mkgmap/combiners/OverviewBuilder.java (working copy)
@@ -65,8 +65,9 @@
private LevelInfo[] wantedLevels;
private Area bounds;
private boolean hasBackground;
- private EnhancedProperties demProps = new EnhancedProperties();
-
+ private EnhancedProperties overviewProps = new EnhancedProperties();
+ private int maxRes = 16; // we can write a 0x4a polygon for planet in res 16.
+
public OverviewBuilder() {
this.overviewSource = new OverviewMapDataSource();
}
@@ -77,13 +78,7 @@
overviewMapnumber = args.get("overview-mapnumber", "63240000");
outputDir = args.getOutputDir();
- String demDist = args.getProperties().getProperty("overview-dem-dist");
- String hgtPath = args.getProperties().getProperty("dem");
- if (hgtPath != null && demDist != null && !"0".equals(demDist.trim())) {
- demProps = new EnhancedProperties(args.getProperties());
- demProps.setProperty("dem-dists", demDist);
- demProps.setProperty("order-by-decreasing-area", "no");
- }
+ overviewProps = new EnhancedProperties(args.getProperties());
}
public void onMapEnd(FileInfo finfo) {
@@ -99,7 +94,12 @@
public void onFinish() {
if (!hasBackground) {
+ List<MapShape> shapes = overviewSource.getShapes();
+ int inx = shapes.size();
overviewSource.addBackground();
+ // for --order-by-decreasing-area, need the background to be first:
+ if (shapes.size() > inx) // something was added
+ shapes.add(0, shapes.remove(inx));
}
calcLevels();
writeOverviewMap();
@@ -113,31 +113,6 @@
}
private void calcLevels() {
- int maxRes = 16; // we can write a 0x4a polygon for planet in res 16
- if (wantedLevels != null)
- maxRes = wantedLevels[wantedLevels.length-1].getBits();
- int maxSize = 0xffff << (24 - maxRes);
- for (MapShape s : overviewSource.getShapes()){
- if (s.getType() != 0x4a)
- continue;
- int maxDimPoly = s.getBounds().getMaxDimension();
- if (maxDimPoly > maxSize){
- int oldMaxRes = maxRes;
- while (maxDimPoly > maxSize){
- maxRes--;
- maxSize = 0xffff << (24 - maxRes);
- }
- final String[] name = s.getName().split("\u001d");
- final String msg = "Tile selection (0x4a) polygon for";
- final String msg2;
- if (name.length == 2)
- msg2 = "tile " + name[1].trim();
- else
- msg2 = s.getBounds().toString();
- log.error(msg, msg2, "cannot be written in level 0 resolution", oldMaxRes + ", using", maxRes, "instead");
-
- }
- }
if (wantedLevels == null)
setRes(maxRes);
else {
@@ -160,6 +135,25 @@
}
}
+ private void checkFixRes(Area detailTileBounds, String tileName) {
+ int maxSize = 0xffff << (24 - maxRes);
+ int maxDimPoly = detailTileBounds.getMaxDimension();
+ if (maxDimPoly > maxSize) {
+ int oldMaxRes = maxRes;
+ while (maxDimPoly > maxSize) {
+ maxRes--;
+ maxSize = 0xffff << (24 - maxRes);
+ }
+ final String msg = "Tile selection (0x4a) polygon for";
+ final String msg2;
+ if (tileName != null)
+ msg2 = "tile " + tileName;
+ else
+ msg2 = detailTileBounds.toString();
+ log.error(msg, msg2, "cannot be written in level 0 resolution", oldMaxRes + ", using", maxRes, "instead");
+ }
+ }
+
/**
* Write out the overview map.
*/
@@ -167,7 +161,7 @@
if (overviewSource.mapLevels() == null)
return;
- MapBuilder mb = new MapBuilder();
+ MapBuilder mb = new MapBuilder(false, true);
mb.setEnableLineCleanFilters(false);
FileSystemParam params = new FileSystemParam();
@@ -182,11 +176,9 @@
}
Sort sort = SrtTextReader.sortForCodepage(codepage);
Map map = Map.createMap(overviewMapname, outputDir, params, overviewMapnumber, sort);
- if (!demProps.isEmpty()) {
- map.config(demProps);
- mb.config(demProps);
- }
-
+ map.config(overviewProps);
+ mb.config(overviewProps);
+
if (encodingType != null){
map.getLblFile().setEncoder(encodingType, codepage);
}
@@ -205,8 +197,6 @@
* @param finfo Information about an individual map.
*/
private void readFileIntoOverview(FileInfo finfo) throws FileNotFoundException {
- addMapCoverageArea(finfo);
-
MapReader mapReader = null;
String filename = finfo.getFilename();
if (codepage == null){
@@ -262,7 +252,9 @@
mapLevels[0] = new LevelInfo(levels[1].getLevel(), levels[1].getResolution());
}
wantedLevels = mapLevels;
+ maxRes = wantedLevels[wantedLevels.length-1].getBits();
}
+ addMapCoverageArea(finfo);
if (isOverviewImg(filename)){
readPoints(mapReader);
readLines(mapReader);
@@ -277,7 +269,6 @@
/**
* Read the points from the .img file and add them to the overview map.
- * We read from the least detailed level (apart from the empty one).
*
* @param mapReader Map reader on the detailed .img file.
*/
@@ -312,7 +303,6 @@
/**
* Read the lines from the .img file and add them to the overview map.
- * We read from the least detailed level (apart from the empty one).
*
* @param mapReader Map reader on the detailed .img file.
*/
@@ -348,7 +338,6 @@
/**
* Read the polygons from the .img file and add them to the overview map.
- * We read from the least detailed level (apart from the empty one).
*
* @param mapReader Map reader on the detailed .img file.
*/
@@ -389,6 +378,8 @@
* Add an area that shows the area covered by a detailed map. This can
* be an arbitary shape, although at the current time we only support
* rectangles.
+ * Also build up full overview path (not ness. rectangle) for DEM
+ * and check/decrease resolution if necessary
*
* @param finfo Information about a detail map.
*/
@@ -398,6 +389,8 @@
for (Coord co: points){
overviewSource.addToBounds(co);
}
+ overviewSource.addToTileAreaPath(points);
+ checkFixRes(bounds, finfo.getMapname());
// Create the tile coverage rectangle
MapShape bg = new MapShape();
bg.setType(0x4a);
Index: src/uk/me/parabola/mkgmap/main/MapMaker.java
===================================================================
--- src/uk/me/parabola/mkgmap/main/MapMaker.java (revision 4591)
+++ src/uk/me/parabola/mkgmap/main/MapMaker.java (working copy)
@@ -110,7 +110,7 @@
Map map = Map.createMap(mapNamePrefix + args.getMapname(), args.getOutputDir(), params, args.getMapname(), sort);
setOptions(map, args);
- MapBuilder builder = new MapBuilder();
+ MapBuilder builder = new MapBuilder(!mapNamePrefix.isEmpty(), false);
builder.config(args.getProperties());
builder.makeMap(map, src);
Index: src/uk/me/parabola/mkgmap/reader/overview/OverviewMapDataSource.java
===================================================================
--- src/uk/me/parabola/mkgmap/reader/overview/OverviewMapDataSource.java (revision 4591)
+++ src/uk/me/parabola/mkgmap/reader/overview/OverviewMapDataSource.java (working copy)
@@ -55,6 +55,10 @@
return tileAreaPath;
}
+ public void addToTileAreaPath(List<Coord> points) {
+ tileAreaPath.append(Java2DConverter.createPath2D(points), false);
+ }
+
/**
* This is a fake source of data and is not read from a file, so always
* return false here.
@@ -158,10 +162,6 @@
*/
public void addShape(MapShape shape) {
mapper.addShape(shape);
- if (shape.getType() == 0x4a) {
- tileAreaPath.append(Java2DConverter.createPath2D(shape.getPoints()),false);
-
- }
}
public void addRoad(MapRoad road) {
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
