Re: [SyncEvolution] Char encoding in syncevo config
Looking further into it I solved the issue by passing c_str() to parseVCard TDEABC::Addressee addressee = converter.parseVCard(item.c_str()); works OK and when reading an item the same, after converting the TQString into std::string, passing the c_str() to the item object item.assign(data_str.c_str()); works OK so in both directions now encoding is preserved. thanks for the hints and advises, without your help I wouldn't have solved it so fast. ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config [identified]
Hi Patrick, thanks again for the input. It seems to be a bug in the vCard to addressee converter in TDE. regards ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config
Patrick Ohly wrote: > On Thu, 2016-03-24 at 00:13 +0100, deloptes wrote: >> Patrick Ohly wrote: >> >> > On Wed, 2016-03-23 at 08:39 +0100, deloptes wrote: >> > When you receive vCards from those phones and they contain data in >> > ISO-8859-15, does the vCard contain a CHARSET parameter on the property >> > with the non-ASCII content? >> >> My assumption was wrong that it was not UTF (may be). It says quoted >> printable/utf-8. Could be that the UTF there is misleading? > > N;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:H=C3=B6sch seems to decode > fine to "Hösch", so it should be correct. > yes indeed >> > It is possible to override the default charset for specific phones. See >> > "11.36.19 " in >> > libsynthesis/doc/SySync_config_reference.pdf >> > >> > There is an example of that in >> > syncevolution/src/syncevo/configs/remoterules/server/00_sony_ericsson.xml >> > >> > You can add similar .xml fragments for your phones, using manufacturer >> > and model tags to limit the effect to certain phones. >> > >> >> I attach here some snippets from the log. I see the data properly >> converted/mapped to the internal fields, only I don't know at which stage >> it is done. > > In the Synthesis vCard parser. It knows about ENCODING and CHARSET > parameters. I'm not exactly sure where in the source, though. > >> The result however in the Addressbook is not readable (for the >> cyrillic) and mangled for the ö. > > What's mangled about the ö? The name "Hösch" looks okay to me, at least > in the internal representation. I can't read cyrillic, so no idea > whether that example is okay. > What I mean that I see the Cyrillic as ? and the ö as two squares. The first happens according my experience when you convert to ASCII and the latter when you do utf to utf - anyway this is not that important. Important is that it is broken and most probably not in syncevolution. > If it is wrong after storing in your local database, then you need to > dig further in the processing pipeline. The internal representation will > get re-encoded again as vCard for storage, and that string then gets > passed to your backend. > I now also think the problem is somewhere deeper. From what I see the vCard passed to the backend looks properly encoded. I'll try to get it printed out to check how it looks like when we receive it and after we've saved it in the address book. thank you for looking into it regards ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config
On Thu, 2016-03-24 at 00:13 +0100, deloptes wrote: > Patrick Ohly wrote: > > > On Wed, 2016-03-23 at 08:39 +0100, deloptes wrote: > > When you receive vCards from those phones and they contain data in > > ISO-8859-15, does the vCard contain a CHARSET parameter on the property > > with the non-ASCII content? > > My assumption was wrong that it was not UTF (may be). It says quoted > printable/utf-8. Could be that the UTF there is misleading? N;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:H=C3=B6sch seems to decode fine to "Hösch", so it should be correct. > > It is possible to override the default charset for specific phones. See > > "11.36.19 " in > > libsynthesis/doc/SySync_config_reference.pdf > > > > There is an example of that in > > syncevolution/src/syncevo/configs/remoterules/server/00_sony_ericsson.xml > > > > You can add similar .xml fragments for your phones, using manufacturer > > and model tags to limit the effect to certain phones. > > > > I attach here some snippets from the log. I see the data properly > converted/mapped to the internal fields, only I don't know at which stage > it is done. In the Synthesis vCard parser. It knows about ENCODING and CHARSET parameters. I'm not exactly sure where in the source, though. > The result however in the Addressbook is not readable (for the > cyrillic) and mangled for the ö. What's mangled about the ö? The name "Hösch" looks okay to me, at least in the internal representation. I can't read cyrillic, so no idea whether that example is okay. If it is wrong after storing in your local database, then you need to dig further in the processing pipeline. The internal representation will get re-encoded again as vCard for storage, and that string then gets passed to your backend. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config
Patrick Ohly wrote: > On Wed, 2016-03-23 at 08:39 +0100, deloptes wrote: I forgot to attach the portions from syncevo logs [2016-03-23 01:01:44.853] Explicit type 'text/x-vcalendar' specified in command or item meta [2016-03-23 01:01:44.853] Version '1.0' obtained from item data â [2016-03-23 01:01:44.853] 'Item_Parse' - parsing SyncML item, SyncOp=add, format=plain-text, RemoteID=10 [--][++] [->end] [->enclosing] [2016-03-23 01:01:44.853] Created new item of datatype 'vCalendar10', localID='' remoteID='10' [2016-03-23 01:01:44.853] Parsing: [2016-03-23 01:01:44.853] BEGIN:VCALENDAR VERSION:1.0 BEGIN:VEVENT UID;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:23072010191708845500=CE=B3 SUMMARY:rd DTSTART:20041124T00 DTEND:20041124T00 CLASS:PUBLIC SEQUENCE:0 RRULE:YM1 11 21001124T00 AALARM;TYPE=X-EPOCSOUND;ENCODING=QUOTED-PRINTABLE:20041124T093000;;;z:=5Csystem=5CSystemSounds=5Calarm.wav LAST-MODIFIED:20101202T132409Z PRIORITY:0 END:VEVENT END:VCALENDAR [2016-03-23 23:24:05.510] Explicit type 'text/x-vcard' specified in command or item meta [2016-03-23 23:24:05.510] Version '2.1' obtained from item data â [2016-03-23 23:24:05.510] 'Item_Parse' - parsing SyncML item, SyncOp=add, format=plain-text, RemoteID=753 [--][++] [->end] [->enclosing] [2016-03-23 23:24:05.510] Created new item of datatype 'vCard21', localID='' remoteID='753' [2016-03-23 23:24:05.510] Parsing: [2016-03-23 23:24:05.510] BEGIN:VCARD VERSION:2.1 REV:20101002T211708Z N;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:H=C3=B6sch TEL;CELL:00436769226242 END:VCARD [2016-03-23 23:24:05.510] Successfully parsed: [2016-03-23 23:24:05.510] Item LocalID='', RemoteID='753', operation=add, size: [maxlocal,maxremote,actual] [2016-03-23 23:24:05.511] - 0 :integer SYNCLVL [ n/a, 0, 0] : - 1 : timestamp REV [ -1, 0, 0] : 2010-10-02T21:17:08Z (TZ: UTC) - 2 : string UID [ n/a, 0, 0] : - 3 : string GROUP_TAG [ n/a, 0, 0] : -- element0 : - 4 : string N_LAST [ -1, 0, 9] : "Hösch" [2016-03-23 23:23:54.139] Explicit type 'text/x-vcard' specified in command or item meta [2016-03-23 23:23:54.139] Version '2.1' obtained from item data â [2016-03-23 23:23:54.139] 'Item_Parse' - parsing SyncML item, SyncOp=add, format=plain-text, RemoteID=416 [--][++] [->end] [->enclosing] [2016-03-23 23:23:54.139] Created new item of datatype 'vCard21', localID='' remoteID='416' [2016-03-23 23:23:54.140] Parsing: [2016-03-23 23:23:54.140] BEGIN:VCARD VERSION:2.1 REV:20100725T123716Z N;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:= =D0=95=D1=80=D0=B5=D0=B2=D0=B8=D0=BD=D0=BE=D0=B2;=D0=9A=D0=B0=D0=BB=D0=BE= =D1=8F=D0=BD;;; FN;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:= =D0=9A=D0=B0=D0=BB=D0=BE=D1=8F=D0=BD=20=D0=95=D1=80=D0=B5=D0=B2=D0=B8=D0= =BD=D0=BE=D0=B2 ADR;HOME;ENCODING=QUOTED-PRINTABLE;CHARSET=utf-8:= ;;32=20Bellefields=20Road=0D=0ALondon=20SW99UQ=0D=0A=D0=90=D0=BD=D0= =B3=D0=BB=D0=B8=D1=8F BDAY:19750130 NOTE;ENCODING=QUOTED-PRINTABLE:Konto:=2000915098966=0D=0ABLZ=2012000=20BA-CA TEL;CELL:004369911862293 END:VCARD [2016-03-23 23:23:54.140] Successfully parsed: [2016-03-23 23:23:54.140] Item LocalID='', RemoteID='416', operation=add, size: [maxlocal,maxremote,actual] [2016-03-23 23:23:54.140] - 0 :integer SYNCLVL [ n/a, 0, 0] : - 1 : timestamp REV [ -1, 0, 0] : 2010-07-25T12:37:16Z (TZ: UTC) - 2 : string UID [ n/a, 0, 0] : - 3 : string GROUP_TAG [ n/a, 0, 0] : -- element0 : -- element1 : - 4 : string N_LAST [ -1, 0,16] : "ÐÑевинов" - 5 : string N_FIRST [ -1, 0,12] : "ÐалоÑн" - 6 : string N_MIDDLE[ -1, 0, 0] : - 7 : string N_PREFIX[ -1, 0, 0] : - 8 : string N_SUFFIX[ -1, 0, 0] : - 9 : string NICKNAME[ n/a, 0, 0] : - 10 : string TITLE [ -1, 0, 0] : - 11 : string FN [ -1, 0,29] : "ÐалоÑн ÐÑевинов" ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config
Patrick Ohly wrote: > On Wed, 2016-03-23 at 08:39 +0100, deloptes wrote: >> Hi all, >> >> I might be too frustrated recently and might have overseen something, but >> is there a way to set up specific char encoding in SyncEvolution >> configuration? >> >> The reason is that I have few old phones in Latin1/ISO-8859-15 and a >> newer N9 using UTF-8 >> While N9 works fine, I get mangled ö/ä/ü in the contacts from the older >> phones. >> I'm just thinking how can I solve this. > > The older phones presumably use vCard 2.1? 3.0 always uses UTF-8. Yes it is vCard 2.1. > > When you receive vCards from those phones and they contain data in > ISO-8859-15, does the vCard contain a CHARSET parameter on the property > with the non-ASCII content? My assumption was wrong that it was not UTF (may be). It says quoted printable/utf-8. Could be that the UTF there is misleading? > > It is possible to override the default charset for specific phones. See > "11.36.19 " in > libsynthesis/doc/SySync_config_reference.pdf > > There is an example of that in > syncevolution/src/syncevo/configs/remoterules/server/00_sony_ericsson.xml > > You can add similar .xml fragments for your phones, using manufacturer > and model tags to limit the effect to certain phones. > I attach here some snippets from the log. I see the data properly converted/mapped to the internal fields, only I don't know at which stage it is done. The result however in the Addressbook is not readable (for the cyrillic) and mangled for the ö. I'll have a look next week as the time window for private work gets closed now. Thanks in advance ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
Re: [SyncEvolution] Char encoding in syncevo config
On Wed, 2016-03-23 at 08:39 +0100, deloptes wrote: > Hi all, > > I might be too frustrated recently and might have overseen something, but is > there a way to set up specific char encoding in SyncEvolution > configuration? > > The reason is that I have few old phones in Latin1/ISO-8859-15 and a newer > N9 using UTF-8 > While N9 works fine, I get mangled ö/ä/ü in the contacts from the older > phones. > I'm just thinking how can I solve this. The older phones presumably use vCard 2.1? 3.0 always uses UTF-8. When you receive vCards from those phones and they contain data in ISO-8859-15, does the vCard contain a CHARSET parameter on the property with the non-ASCII content? It is possible to override the default charset for specific phones. See "11.36.19 " in libsynthesis/doc/SySync_config_reference.pdf There is an example of that in syncevolution/src/syncevo/configs/remoterules/server/00_sony_ericsson.xml You can add similar .xml fragments for your phones, using manufacturer and model tags to limit the effect to certain phones. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ SyncEvolution mailing list SyncEvolution@syncevolution.org https://lists.syncevolution.org/mailman/listinfo/syncevolution
