Re: [mkgmap-dev] Fix and augment sort definitions
Hi Gerd The same fix was applied to cp1252.txt some time ago and this patch fixes the rest of the sort/cp*.txt in the same way - making them do what the writer intended. I don't think a unit test for this is worth the time or effort. Ticker On Thu, 2023-10-12 at 08:14 +, Gerd Petermann wrote: > > Hi Ticker, > > > > please can you provide a unit test for this? > > > > Gerd > > > > > > Von: mkgmap-dev im Auftrag von > > Ticker > > Berkin > > Gesendet: Donnerstag, 12. Oktober 2023 09:18 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > Hi Gerd > > > > Last year it was reported that string ordering with the '#' character was > > incorrect. This was > > because, in the sort/cp*.txt files, the relevant line with the '#' was taken > > as a comment. > > > > I had a patch that fixed all the files, but it also attempted to do more > > with > > ß/ss and > > dipthongs. > > > > I've done another patch that doesn't have any contentious changes, just > > fixes > > the #, makes the > > layout consistent between the files, increments the version/id2 values and > > slight improvements > > to the documentation. > > > > Ticker > > > > On Tue, 2022-01-11 at 14:00 +, Gerd Petermann wrote: > > > > > > Hi Ticker, > > > > > > > > > > > > if you don't mind I'd like to postpone this patch until the active > > > > > > branches are merged into > > > > > > trunk. > > > > > > > > > > > > Gerd > > > > > > > > > > > > > > > > > > > > > > > > Von: mkgmap-dev im Auftrag > > > > > > von > > > > > > Ticker Berkin > > > > > > > > > > > > Gesendet: Dienstag, 11. Januar 2022 11:25 > > > > > > An: Development list for mkgmap > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > > > > > Hi Gerd > > > > > > > > > > > > Yes - gmapsupp builder gives a warning if id1/id2 are not consistent > > > > > > in > > > > > > all the .img files. It is just a warning and gmapsupp is built > > > > > > anyway > > > > > > and I think the warning can be ignored. gmapi doesn't notice. > > > > > > > > > > > > Almost all of the significant sorting where the Garmin device... > > > > > > needs > > > > > > to know the sort details happens in Mdr, so this isn't a problem. > > > > > > > > > > > > Other uses are mostly for de-duping/efficient processing, so these > > > > > > shouldn't matter either. > > > > > > > > > > > > However the LBL file does hold id1/id2 and many sections (Countries, > > > > > > Regions, Cities, Zips, POIs) are sorted so the effect here is > > > > > > unknown. > > > > > > > > > > > > If using --latin2 / 1252, the only change in ordering is around > > > > > > AE/OE > > > > > > dipthongs. > > > > > > > > > > > > Within the same commit or build as sortResource_v2, the attached > > > > > > sortMashExp.patch should be applied, as it effects the binary SRT > > > > > > file > > > > > > and I don't want to increment all the id2's again. This patch > > > > > > changes > > > > > > the sort.expand TERTIARY mashing from 2 to 3, which is slightly more > > > > > > consistent with the Garmin SRT binaries I've seen and allows > > > > > > SrtDisplay > > > > > > to show expansions with what looks like a meaningful case. > > > > > > > > > > > > Ticker > > > > > > > > > > > > On Tue, 2022-01-11 at 06:31 +, Gerd Petermann wrote: > > > > > > > > > > Hi Ticker, > > > > > > > > > > > > > > > > > > > > didn't try it: Will mkgmap complain when building an indexed > > > > > > > > > > gmapi/gmapsupp > > > > > > > > > > where some tiles where freshly compiled with the new version > > > > > > > > > > and > > > > > > > > > > others with > > > > > > > > > > an older (like Felix and Carlos do)? > > > > > > > > > > > > > > > > > > > > Gerd > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Von: mkgmap-dev im > > > > > > > > > > Auftrag > > > > > > > > > > von Ticker Berkin > > > > > > > > > > Gesendet: Montag, 10. Januar 2022 12:04 > > > > > > > > > > An: Development list for mkgmap > > > > > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > > > > > > > > > > > > > Hi Gerd > > > > > > > > > > > > > > > > > > > > What I meant was that keyboards/devices don't normally have > > > > > > > > > > ways of > > > > > > > > > > entering the single chars "…", "¼", "½", "¾", "™". > > > > > > > > > > > > > > > > > > > > Names with these might be presented by Garmin software after > > > > > > > > > > some > > > > > > > > > > initial chars have been entered and you can then select the > > > > > > > > > > complete > > > > > > > > > > name that contains these chars. > > > > > > > > > > > > > > > > > > > > I didn't see a good reason to remove the expand for these > > > > > > > > > > and find > > > > > > > > > > some > > > > > > > > > > arbitrary sort PRIMARY for them. No one has complained about >
Re: [mkgmap-dev] Fix and augment sort definitions
Hi Ticker, please can you provide a unit test for this? Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Donnerstag, 12. Oktober 2023 09:18 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Fix and augment sort definitions Hi Gerd Last year it was reported that string ordering with the '#' character was incorrect. This was because, in the sort/cp*.txt files, the relevant line with the '#' was taken as a comment. I had a patch that fixed all the files, but it also attempted to do more with ß/ss and dipthongs. I've done another patch that doesn't have any contentious changes, just fixes the #, makes the layout consistent between the files, increments the version/id2 values and slight improvements to the documentation. Ticker On Tue, 2022-01-11 at 14:00 +, Gerd Petermann wrote: > > Hi Ticker, > > > > if you don't mind I'd like to postpone this patch until the active branches > > are merged into > > trunk. > > > > Gerd > > > > > > > > Von: mkgmap-dev im Auftrag von > > Ticker Berkin > > > > Gesendet: Dienstag, 11. Januar 2022 11:25 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > Hi Gerd > > > > Yes - gmapsupp builder gives a warning if id1/id2 are not consistent in > > all the .img files. It is just a warning and gmapsupp is built anyway > > and I think the warning can be ignored. gmapi doesn't notice. > > > > Almost all of the significant sorting where the Garmin device... needs > > to know the sort details happens in Mdr, so this isn't a problem. > > > > Other uses are mostly for de-duping/efficient processing, so these > > shouldn't matter either. > > > > However the LBL file does hold id1/id2 and many sections (Countries, > > Regions, Cities, Zips, POIs) are sorted so the effect here is unknown. > > > > If using --latin2 / 1252, the only change in ordering is around AE/OE > > dipthongs. > > > > Within the same commit or build as sortResource_v2, the attached > > sortMashExp.patch should be applied, as it effects the binary SRT file > > and I don't want to increment all the id2's again. This patch changes > > the sort.expand TERTIARY mashing from 2 to 3, which is slightly more > > consistent with the Garmin SRT binaries I've seen and allows SrtDisplay > > to show expansions with what looks like a meaningful case. > > > > Ticker > > > > On Tue, 2022-01-11 at 06:31 +, Gerd Petermann wrote: > > > > Hi Ticker, > > > > > > > > didn't try it: Will mkgmap complain when building an indexed > > > > gmapi/gmapsupp > > > > where some tiles where freshly compiled with the new version and > > > > others with > > > > an older (like Felix and Carlos do)? > > > > > > > > Gerd > > > > > > > > > > > > Von: mkgmap-dev im Auftrag > > > > von Ticker Berkin > > > > Gesendet: Montag, 10. Januar 2022 12:04 > > > > An: Development list for mkgmap > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > Hi Gerd > > > > > > > > What I meant was that keyboards/devices don't normally have ways of > > > > entering the single chars "…", "¼", "½", "¾", "™". > > > > > > > > Names with these might be presented by Garmin software after some > > > > initial chars have been entered and you can then select the complete > > > > name that contains these chars. > > > > > > > > I didn't see a good reason to remove the expand for these and find > > > > some > > > > arbitrary sort PRIMARY for them. No one has complained about them. > > > > Also > > > > cp65001 had over 1000 expands and I really don't want to start > > > > touching > > > > these. > > > > > > > > Ticker > > > > > > > > > > > > On Mon, 2022-01-10 at 10:29 +, Gerd Petermann wrote: > > > > > > Hi Ticker, > > > > > > > > > > > > I've committed displaySrt_v2.patch . > > > > > > > > > > > > I don't fully understand the comment > > > > > > "Leave the above because no method of inputting them anyway and > > > > > > unlikely at start of names." > > > > > > > > > > > > It is possible to enter these characters in MapSource and I think > > > > > > MapSource uses MDR12 > > > > > > when you type only a few characters for the name of a POI and don't > > > > > > pick up an entry from the list. > > > > > > > > > > > > Gerd > > > > > > > > > > > > > > > > > > Von: mkgmap-dev im Auftrag > > > > > > von > > > > > > Ticker Berkin > > > > > > Gesendet: Montag, 10. Januar 2022 11:20 > > > > > > An: Development list for mkgmap > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > > > > > Hi Gerd > > > > > > > > > > > > I tried various approaches to fixing "Find" when the fixed length > > > > > > Mdr17 > > > > > > (maybe also Mdr12) prefix contains sort.expand chars and couldn't > > > > > > make > > > > > > it work. I could documents these attempts in Sort.java if you feel > >
Re: [mkgmap-dev] Fix and augment sort definitions
Hi Gerd Last year it was reported that string ordering with the '#' character was incorrect. This was because, in the sort/cp*.txt files, the relevant line with the '#' was taken as a comment. I had a patch that fixed all the files, but it also attempted to do more with ß/ss and dipthongs. I've done another patch that doesn't have any contentious changes, just fixes the #, makes the layout consistent between the files, increments the version/id2 values and slight improvements to the documentation. Ticker On Tue, 2022-01-11 at 14:00 +, Gerd Petermann wrote: > > Hi Ticker, > > > > if you don't mind I'd like to postpone this patch until the active branches > > are merged into > > trunk. > > > > Gerd > > > > > > > > Von: mkgmap-dev im Auftrag von > > Ticker Berkin > > > > Gesendet: Dienstag, 11. Januar 2022 11:25 > > An: Development list for mkgmap > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > Hi Gerd > > > > Yes - gmapsupp builder gives a warning if id1/id2 are not consistent in > > all the .img files. It is just a warning and gmapsupp is built anyway > > and I think the warning can be ignored. gmapi doesn't notice. > > > > Almost all of the significant sorting where the Garmin device... needs > > to know the sort details happens in Mdr, so this isn't a problem. > > > > Other uses are mostly for de-duping/efficient processing, so these > > shouldn't matter either. > > > > However the LBL file does hold id1/id2 and many sections (Countries, > > Regions, Cities, Zips, POIs) are sorted so the effect here is unknown. > > > > If using --latin2 / 1252, the only change in ordering is around AE/OE > > dipthongs. > > > > Within the same commit or build as sortResource_v2, the attached > > sortMashExp.patch should be applied, as it effects the binary SRT file > > and I don't want to increment all the id2's again. This patch changes > > the sort.expand TERTIARY mashing from 2 to 3, which is slightly more > > consistent with the Garmin SRT binaries I've seen and allows SrtDisplay > > to show expansions with what looks like a meaningful case. > > > > Ticker > > > > On Tue, 2022-01-11 at 06:31 +, Gerd Petermann wrote: > > > > Hi Ticker, > > > > > > > > didn't try it: Will mkgmap complain when building an indexed > > > > gmapi/gmapsupp > > > > where some tiles where freshly compiled with the new version and > > > > others with > > > > an older (like Felix and Carlos do)? > > > > > > > > Gerd > > > > > > > > > > > > Von: mkgmap-dev im Auftrag > > > > von Ticker Berkin > > > > Gesendet: Montag, 10. Januar 2022 12:04 > > > > An: Development list for mkgmap > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > Hi Gerd > > > > > > > > What I meant was that keyboards/devices don't normally have ways of > > > > entering the single chars "…", "¼", "½", "¾", "™". > > > > > > > > Names with these might be presented by Garmin software after some > > > > initial chars have been entered and you can then select the complete > > > > name that contains these chars. > > > > > > > > I didn't see a good reason to remove the expand for these and find > > > > some > > > > arbitrary sort PRIMARY for them. No one has complained about them. > > > > Also > > > > cp65001 had over 1000 expands and I really don't want to start > > > > touching > > > > these. > > > > > > > > Ticker > > > > > > > > > > > > On Mon, 2022-01-10 at 10:29 +, Gerd Petermann wrote: > > > > > > Hi Ticker, > > > > > > > > > > > > I've committed displaySrt_v2.patch . > > > > > > > > > > > > I don't fully understand the comment > > > > > > "Leave the above because no method of inputting them anyway and > > > > > > unlikely at start of names." > > > > > > > > > > > > It is possible to enter these characters in MapSource and I think > > > > > > MapSource uses MDR12 > > > > > > when you type only a few characters for the name of a POI and don't > > > > > > pick up an entry from the list. > > > > > > > > > > > > Gerd > > > > > > > > > > > > > > > > > > Von: mkgmap-dev im Auftrag > > > > > > von > > > > > > Ticker Berkin > > > > > > Gesendet: Montag, 10. Januar 2022 11:20 > > > > > > An: Development list for mkgmap > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions > > > > > > > > > > > > Hi Gerd > > > > > > > > > > > > I tried various approaches to fixing "Find" when the fixed length > > > > > > Mdr17 > > > > > > (maybe also Mdr12) prefix contains sort.expand chars and couldn't > > > > > > make > > > > > > it work. I could documents these attempts in Sort.java if you feel > > > > > > this > > > > > > is worthwhile. > > > > > > > > > > > > New patch attached that, for cp1252, leaves "ß" as its own PRIMARY > > > > > > after "s". Moved æ,Æ etc to be PRIMARIES on the grounds that their > > > > > > behaviour will be the same as "ß".
