Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-10-11 Thread Carlos Dávila
tivaslibres.org>> Gesendet: Samstag, 9. Oktober 2021 11:17 An: mkgmap-dev@lists.mkgmap.org.uk <mailto:mkgmap-dev@lists.mkgmap.org.uk> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Sorry, not sure what

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-20 Thread Felix Hartmann
emory before >> writing the final result, only data for the global index files which are >> likely to grow very large are written to tempory files. >> >> Gerd >> >> >> >> Von: mkgmap-dev im Auftrag von >> ael

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-20 Thread Gerd Petermann
Auftrag von Gerd Petermann Gesendet: Samstag, 18. September 2021 10:33 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Hi Felix, maybe another speed improvement is possible. I've noticed that each *.img

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-19 Thread Felix Hartmann
mkgmap-dev@lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to > disk if used with --gmapi option > > On Wed, Sep 08, 2021 at 02:10:52PM +0300, Felix Hartmann wrote: > > > > Yes on an nvme disk you barely notice the conversion - it's

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-19 Thread Gerd Petermann
erd Von: mkgmap-dev im Auftrag von ael Gesendet: Mittwoch, 8. September 2021 13:31 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option On Wed, Sep 08, 2021 at 02:10:52PM +0300, Felix Hartmann wr

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-18 Thread Gerd Petermann
list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Hi Felix, maybe another speed improvement is possible. I've noticed that each *.img file is read multiple times, e.g. by the TdbBuilder and the MdrBuilder. I think

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-18 Thread Gerd Petermann
Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Freitag, 17. September 2021 13:47 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Thanks Gerd, I*ll likely not find time to test

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-17 Thread Carlos Dávila
for all? Gerd Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Montag, 13. September 2021 20:44 An: Thomas Morgenstern; Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-17 Thread Gerd Petermann
, see https://files.mkgmap.org.uk/detail/518 Gerd Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Donnerstag, 16. September 2021 15:41 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-16 Thread Felix Hartmann
Felix Hartmann > Gesendet: Donnerstag, 16. September 2021 14:11 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to > disk if used with --gmapi option > > I would prefer if I don't even need to link anything, just the concep

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-16 Thread Gerd Petermann
Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Donnerstag, 16. September 2021 14:11 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option I would prefer if I don't

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-16 Thread Felix Hartmann
esendet: Montag, 13. September 2021 20:44 > An: Thomas Morgenstern; Development list for mkgmap > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to > disk if used with --gmapi option > > .img files are always reused and never rewritten - it's only

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-16 Thread Gerd Petermann
Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Montag, 13. September 2021 20:44 An: Thomas Morgenstern; Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option .img files are always reused and never

Re: [mkgmap-dev] mkgmap doing excessive writing

2021-09-13 Thread Felix Hartmann
encies > between them. > > Gerd > > > Von: mkgmap-dev im Auftrag von > Thomas Morgenstern > Gesendet: Montag, 13. September 2021 18:01 > An: Development list for mkgmap > Betreff: [mkgmap-dev] mkgmap doing excessive writing > &

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
me stage and that you run mkgmap multiple times with different >> combinations of those *.img files as input to produce different zip-files >> with gmapi format or gmapsupp format. >> I think that's the normal way to do it, so I try to support that way. >> >> Gerd &g

Re: [mkgmap-dev] mkgmap doing excessive writing

2021-09-13 Thread Gerd Petermann
. September 2021 18:01 An: Development list for mkgmap Betreff: [mkgmap-dev] mkgmap doing excessive writing I suggest the new option should be named reuse=. exampel reuse=4001.img, 40005*.img. If this option is used, then mkgmap should not overwrite the listed folders in the gmap. Naturally mkgmap

[mkgmap-dev] mkgmap doing excessive writing

2021-09-13 Thread Thomas Morgenstern
I suggest the new option should be named /reuse=files, wildcards enabled>. /exampel /reuse=4001.img, 40005*.img. /If this option is used, then mkgmap should not overwrite the listed folders in the gmap. Naturally mkgmap should writea new tdb, preview.img, mdr and idx. In most cases the

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
ay to do it, so I try to support that way. > > Gerd > > > Von: mkgmap-dev im Auftrag von > Felix Hartmann > Gesendet: Montag, 13. September 2021 12:28 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] mkgmap doing excessiv

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Gerd Petermann
: Montag, 13. September 2021 12:28 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option I move away the info.xml - and use a different name for the mapset files and then just have mkgmap any file that is input

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
idn't try it. Does it make sense? > > Gerd > > > Von: mkgmap-dev im Auftrag von > Felix Hartmann > Gesendet: Montag, 13. September 2021 12:09 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing a

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Gerd Petermann
. September 2021 12:09 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Oh - but data was certainly written - A rename will not show as data written in both Task manager on Windows, as well as in the smart

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
iable it is, but it might be >>> worth trying. >>> >>> Gerd >>> (1) >>> >>> https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/install-enable >>> >>> >>> Von

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
er/storage/data-deduplication/install-enable >> >> ________________ >> Von: mkgmap-dev im Auftrag von >> Carlos Dávila >> Gesendet: Sonntag, 12. September 2021 22:45 >> An: mkgmap-dev@lists.mkgmap.org.uk >> Betreff: Re: [mkgmap-dev

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Felix Hartmann
Sonntag, 12. September 2021 22:45 > An: mkgmap-dev@lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to > disk if used with --gmapi option > > I think it's a good idea if mkgmap checks the required files are present. > > El 12/9/21 a

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-13 Thread Gerd Petermann
: Sonntag, 12. September 2021 22:45 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option I think it's a good idea if mkgmap checks the required files are present. El 12/9/21 a las 21:16, Gerd Petermann

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Carlos Dávila
the content of the exiting file(s)? Gerd Von: mkgmap-dev im Auftrag von Felix Hartmann Gesendet: Sonntag, 12. September 2021 19:10 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Carlos Dávila
mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Well I think the .tmp files are just building up - and the renamed. So they are not causing any actual excessive write. As for

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Gerd Petermann
Hartmann Gesendet: Sonntag, 12. September 2021 19:10 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option And well - it is burning SSD for the contourlines currently even if not calling multiple times. 130GB minimum

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Felix Hartmann
s. So the question is: Why do you do that? >> >> Gerd >> >> ____________ >> Von: mkgmap-dev im Auftrag von >> Felix Hartmann >> Gesendet: Freitag, 10. September 2021 00:02 >> An: Development list for mkgmap >> Betreff: Re:

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Felix Hartmann
ion is: Why do you do that? > > Gerd > > > Von: mkgmap-dev im Auftrag von > Felix Hartmann > Gesendet: Freitag, 10. September 2021 00:02 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-12 Thread Gerd Petermann
im Auftrag von Felix Hartmann Gesendet: Freitag, 10. September 2021 00:02 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option Well I think the .tmp files are just building up - and the renamed. So

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-09 Thread Felix Hartmann
Well I think the .tmp files are just building up - and the renamed. So they are not causing any actual excessive write. As for the gmap - it would be cool if there is a mode to not write them. Actually it would be great if mkgmap could write all in one go. Because the thing that takes so much

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread Felix Hartmann
oh sorry - made a typo at the first number. (there was no error in the SMART data I just somehow wrote 3 instead of 6. Those 30TB of writes did not suddenly disappear). On Wed, 8 Sept 2021 at 22:36, Felix Hartmann wrote: > so continued.. > 33659 GB > After Step 1: > new 33663 GB > now set all

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread Felix Hartmann
so continued.. 33659 GB After Step 1: new 33663 GB now set all files in Product1 folder to read only. Only applies to files not to the actual folder... attrib +r C:\openmtbmap\maps\mtb_%abr%_%date1%%IDT%.gmap\Product1\6* /s /d attrib -r C:\openmtbmap\maps\mtb_%abr%_%date1%%IDT%.gmap\Product1 /d

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread Felix Hartmann
Uncommenting line 102 from /combiners/gmapibuilder.java and then messing by making folder read only works (even though this is really like using crutches while being healthy)! And yeah it's great mkgmap code is so cleanly written that even someone at a loss with java can write a workaround. I

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread ael
On Wed, Sep 08, 2021 at 02:10:52PM +0300, Felix Hartmann wrote: > > Yes on an nvme disk you barely notice the conversion - it's really quick. > BUT it is not needed if you have the files and even more - it burns your > NVME SSD disk. +1. I always use an old spinning rust disk when using mkgmap

Re: [mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread Felix Hartmann
Well I could give it 20 GB ram disk, maybe 32 but then I need to render on less than 12 processes 64GB ram available). But that is not enough for Asia continent map, and I guess super tight for Europe... Mkgmap could definitely keep those .tmp files in memory. But the important bit is the gmap

[mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

2021-09-08 Thread Felix Hartmann
Quite a few people do not compile the contourlines each time they update a map, but reuse the old contourlines. This works fine if you insert them as say 1234*img (and they are named 1234.img 12340001.img and so on). However this does not work when providing the --gmapi option. Actually