Re: [Development] Backwards compatibiltiy break in Qt 5.5
2015-07-28 9:50 GMT+02:00 Robert Iakobashvili corobe...@gmail.com: On Tue, Jul 28, 2015 at 11:48 AM, Marc Mutz marc.m...@kdab.com wrote: On Monday 27 July 2015 20:44:48 Thiago Macieira wrote: On Monday 27 July 2015 21:41:44 NIkolai Marchenko wrote: Yes,but this requires making override visible in every file of the project. We can solve this by having a global #ifdef that can change the setting. Or an environment variable, so you don't have to recompile the world... Yes, it's a good idea to have an environment variable or a pre-processing flag to preserve the existing behavior. In Qt 5.* the default could be the old behavior, and in Qt 6 - the new. Isn't this needless convenience? Yet another environment variable that needs documenting? If you want to inspect the contents of a QString, can't you just add the suitable escape code to your own code? Forcing qPrintable() or qUtf8Printable() on everyone who wants to keep human readable debug output sounds pretty evil, but having e.g. toHtmlEscaped() or a similar function doing the octa/hex thing could be better. -- Andreas Aardal Hanssen ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Tue, Jul 28, 2015 at 11:48 AM, Marc Mutz marc.m...@kdab.com wrote: On Monday 27 July 2015 20:44:48 Thiago Macieira wrote: On Monday 27 July 2015 21:41:44 NIkolai Marchenko wrote: Yes,but this requires making override visible in every file of the project. We can solve this by having a global #ifdef that can change the setting. Or an environment variable, so you don't have to recompile the world... -- Marc Mutz marc.m...@kdab.com | Senior Software Engineer KDAB (Deutschland) GmbH Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts Yes, it's a good idea to have an environment variable or a pre-processing flag to preserve the existing behavior. In Qt 5.* the default could be the old behavior, and in Qt 6 - the new. jm2c to add. Regards, Robert ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 20:44:48 Thiago Macieira wrote: On Monday 27 July 2015 21:41:44 NIkolai Marchenko wrote: Yes,but this requires making override visible in every file of the project. We can solve this by having a global #ifdef that can change the setting. Or an environment variable, so you don't have to recompile the world... -- Marc Mutz marc.m...@kdab.com | Senior Software Engineer KDAB (Deutschland) GmbH Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On 28 Jul 2015, at 10:48, Marc Mutz marc.m...@kdab.com wrote: On Monday 27 July 2015 20:44:48 Thiago Macieira wrote: On Monday 27 July 2015 21:41:44 NIkolai Marchenko wrote: Yes,but this requires making override visible in every file of the project. We can solve this by having a global #ifdef that can change the setting. Or an environment variable, so you don't have to recompile the world… +1 to that. And I think the homograph-debugging should be the non-default one, so we go back to getting readable output by default. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On 2015-07-28 04:29, Andreas Aardal Hanssen wrote: Isn't this needless convenience? Yet another environment variable that needs documenting? If you want to inspect the contents of a QString, can't you just add the suitable escape code to your own code? Forcing qPrintable() or qUtf8Printable() on everyone who wants to keep human readable debug output sounds pretty evil, but having e.g. toHtmlEscaped() or a similar function doing the octa/hex thing could be better. While I'm generally trying to stay out of this debate, I'd like to toss in that a QString → QString or QString → QByteArray function that converts from raw to either C-style escaped or HTML-style escaped (or for bonus points, both), AND BACK AGAIN, would be useful in other contexts. (For instance, I recently needed to write a new file format that is SSV-like but needs to contain user supplied strings. In order to make tokenizing not incredibly painful, it was decided to HTML-escape the strings so that they would not contain spaces or strange unprintable characters. I've also written C-style escaping functions more than once, especially for encoding and decoding quoted strings with escaped embedded quotes and so forth.) -- Matthew ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
I would repeat what I said in JIRA. Everything that Thiago explained makes sense to me as soon as it does not break behavior of existing applications. If global flag to enable escaping of QString contents is going to be introduced then it should be disabled by default. On changes that might break compatibility there should be some API that defaults application behavior to the particular version. For example: QCoreApplication::setDefaultPolicy(Qt::Version_5_4); So when application is running under Qt 5.5.x it default behavior to what is was in Qt 5.4. Something similar to QDataStream::Version. For instance, you can look at CMake which has lot of updates in each version that might change behavior, but they never ever break existing application as soon as cmake_minimum_required() and cmake_policy() have been used in your code. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
People were warned in the changelog. I've already quoted Adams in JIRA, but this is exactly how changelog warnen users of Qt about this change: https://youtu.be/VGLFweev_iI?t=386 Among panicking customers not understanding wtf is happening, unintelligible message, posted in the wrong place and hard to understand, and without any window for discussion. The Master's right again. On Mon, Jul 27, 2015 at 7:05 PM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 01:25:18 NIkolai Marchenko wrote: Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 Yes, I asked that they post here if there's a need to overrule me. I wanted Nikolai to get his story out first, which is why I waited a day before replying. Now, let's start with the subject: it's a change in behaviour, properly documented in the changelog[*], in an area that did not work properly before. Previously, the debugging of QStrings would produce ambiguous output if the source string had any quote characters. More importantly, it could lose data, as the qDebug output was not 8-bit clean. Therefore, the functionality was never officially supported. [* Important: the changelog was not posted for the 5.5 release] The change makes the functionality now consistent across all platforms, through all qDebug backends. It also makes the output unambiguous, since a quote character is escaped to indicate it's still part of the string. It adds two more interesting features: 1) the string can be copied back into the source code and will compile to exactly the contents of the QString or QByteArray that produced it. No more need to escape backslashes, for example. 2) the output has no homographs, no control characters and no zero-width codepoints (including no combining characters), allowing for easier debugging. A homograph is a character whose glyph is identical to another character. The most widely publicised case is that of the Latin letter a and the Cyrillic а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control character could change the way the text gets logged, like for example the carriage return character causing overwrite of what had already been output in a terminal. I'm not saying qDebug's output is a security issue or possible attack vector, but having no ambiguity and no homographs makes for easier debugging. It could save quite a few minutes of staring at a string that *looks* right but whose manipulation generates an apparently unexpected result. That is, after all, the purpose of qDebug: debugging. By the way, the output of QtTest has also changed in the same way. In version 5.5 Thiago Macieira introduced a compatibility break with older qt versions. This break has come in the form of completely changing the way qDebug() prints non-english characters to console. Let's be clear: this is about non-US-ASCII characters and only if we're talking about a QString or QByteArray. If you're passing a character literal or char* directly, the feature does not kick in. People who did not want the quotes around the string before Qt 5.4 would have already used qPrintable() or similar mechanisms. Anyone using the noquote() feature since 5.4 is also similarly not affected. In essense - his fix to qDebug() will make it so that all non-english output from QString variable to qDebug() will be passed as a sequence of unicode escaped characters along the lines of: \u043F\u0440\u043E\u0432\u0435\u0440\u043A\u0430 043A\u0438\u0440\u0438\u043B\u043B\u0438\u0446\u044B Previously, in _some_ cases, but not reliably cross platform it would actually appear in console as readable output in user's language. I have to admit that Thiago is right about it being non-reliable contradicts qt's goal. Still... Right. What Thiago completely disregards as unimportant - qDebug()'s been around for a long time. Even during times of qt4, users, through the use of setCodec functions, achieved readable output from qDebug in the console in their native language. Again, not properly and not everywhere. Moreover, the requirements to continue to make it work have changed along the way. In Qt 5.0, we introduced a message handler that took QString instead of a char*, so the previous solutions that relied in setCodecForLocale may have stopped working. (I actually don't know if it did, as we've never tested, which goes to prove my point of it being an unsupported feature) Along the next few Qt 5.x releases, we introduced different output backends too, some of which took the data in UTF-8 and others didn't. Note that I say qt4. It's been around for almost a decade and there being no
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Tuesday 28 July 2015 09:44:23 Matthew Woehlke wrote: While I'm generally trying to stay out of this debate, I'd like to toss in that a QString → QString or QString → QByteArray function that converts from raw to either C-style escaped or HTML-style escaped (or for bonus points, both), AND BACK AGAIN, would be useful in other contexts. We already have one that does HTML escaping, but we don't have an HTML parser or even a simple unescaper. A QString C-style escaper function would be useful indeed. I'll add it for 5.7. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Tuesday 28 July 2015 18:42:20 NIkolai Marchenko wrote: People were warned in the changelog. I've already quoted Adams in JIRA, but this is exactly how changelog warnen users of Qt about this change: https://youtu.be/VGLFweev_iI?t=386 Among panicking customers not understanding wtf is happening, unintelligible message, posted in the wrong place and hard to understand, and without any window for discussion. The Master's right again. I understand the feeling, but I've done my job and we created the changelog. You've now seen the changelog and you've seen the type of important messages that are there. So the release team and marketing team failed, indeed, by not making the changelog more readily accessible. That does not invalidate the choices we made that are documented in the changelog. This is not the only notice in the Important Behavior Changes section. Moreover, all the deprecations for 5.6 and 5.7 that being warned about are still valid. The fact that the changelog was not linked from the announcement will not prevent us from removing support for non-C++11 builds in 5.7. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27. July 2015 01:25:18 NIkolai Marchenko wrote: Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 [...] I agree with you that by default qDebug() someString; should indeed not escape the printable character, and print them as it. This is what most would expect. QString is meant to contains unicode! (For QByteArray this is different since QByteArray does not specify the encoding) -- Olivier Woboq - Qt services and support - http://woboq.com - http://code.woboq.org ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 01:25:18 NIkolai Marchenko wrote: Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 Yes, I asked that they post here if there's a need to overrule me. I wanted Nikolai to get his story out first, which is why I waited a day before replying. Now, let's start with the subject: it's a change in behaviour, properly documented in the changelog[*], in an area that did not work properly before. Previously, the debugging of QStrings would produce ambiguous output if the source string had any quote characters. More importantly, it could lose data, as the qDebug output was not 8-bit clean. Therefore, the functionality was never officially supported. [* Important: the changelog was not posted for the 5.5 release] The change makes the functionality now consistent across all platforms, through all qDebug backends. It also makes the output unambiguous, since a quote character is escaped to indicate it's still part of the string. It adds two more interesting features: 1) the string can be copied back into the source code and will compile to exactly the contents of the QString or QByteArray that produced it. No more need to escape backslashes, for example. 2) the output has no homographs, no control characters and no zero-width codepoints (including no combining characters), allowing for easier debugging. A homograph is a character whose glyph is identical to another character. The most widely publicised case is that of the Latin letter a and the Cyrillic а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control character could change the way the text gets logged, like for example the carriage return character causing overwrite of what had already been output in a terminal. I'm not saying qDebug's output is a security issue or possible attack vector, but having no ambiguity and no homographs makes for easier debugging. It could save quite a few minutes of staring at a string that *looks* right but whose manipulation generates an apparently unexpected result. That is, after all, the purpose of qDebug: debugging. By the way, the output of QtTest has also changed in the same way. In version 5.5 Thiago Macieira introduced a compatibility break with older qt versions. This break has come in the form of completely changing the way qDebug() prints non-english characters to console. Let's be clear: this is about non-US-ASCII characters and only if we're talking about a QString or QByteArray. If you're passing a character literal or char* directly, the feature does not kick in. People who did not want the quotes around the string before Qt 5.4 would have already used qPrintable() or similar mechanisms. Anyone using the noquote() feature since 5.4 is also similarly not affected. In essense - his fix to qDebug() will make it so that all non-english output from QString variable to qDebug() will be passed as a sequence of unicode escaped characters along the lines of: \u043F\u0440\u043E\u0432\u0435\u0440\u043A\u0430 043A\u0438\u0440\u0438\u043B\u043B\u0438\u0446\u044B Previously, in _some_ cases, but not reliably cross platform it would actually appear in console as readable output in user's language. I have to admit that Thiago is right about it being non-reliable contradicts qt's goal. Still... Right. What Thiago completely disregards as unimportant - qDebug()'s been around for a long time. Even during times of qt4, users, through the use of setCodec functions, achieved readable output from qDebug in the console in their native language. Again, not properly and not everywhere. Moreover, the requirements to continue to make it work have changed along the way. In Qt 5.0, we introduced a message handler that took QString instead of a char*, so the previous solutions that relied in setCodecForLocale may have stopped working. (I actually don't know if it did, as we've never tested, which goes to prove my point of it being an unsupported feature) Along the next few Qt 5.x releases, we introduced different output backends too, some of which took the data in UTF-8 and others didn't. Note that I say qt4. It's been around for almost a decade and there being no warning qDebug ever changing, it's become a standard on a lot of non-english installations, that actually allowed getting readable output, to use qDebug for tracing variables.. After all, opening debugger to inspect every little thing is very counterproductive. Agreed, even if poor practice to debug via qDebug. A debugger is always a superior solution. Systems where a debugger will not run are not systems worth developing for (fix your toolchain instead or stop using that vendor). Note that superior solution does
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 20:52:24 NIkolai Marchenko wrote: Let's make it harder, can you guesss that it even is a link to paypal website with qDebug() as you intend it to work without copypasting? раураl.соm ? Hint: only 2 letters here are in english. With qDebug() as it is in Qt 5.5, it would be quite evident. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
The problem with your approach to qDebug is that it only works well where most of the text is in English and only individual letters are unicode. Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. 2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com: On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
Question, how much do you think this use case happens for non-latin users compared to the one I, and Knoll Lars are arguing about? 2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com: On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
Let's make it harder, can you guesss that it even is a link to paypal website with qDebug() as you intend it to work without copypasting? раураl.соm ? Hint: only 2 letters here are in english. 2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com: On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. And that is the situation, where aforementioned flag to change the behaviour in one place will be VERY useful. Unlike your suggestion of fixing source code everywhere line by line. And users genrally know if they need to consider the case you are arguing about or not. On Mon, Jul 27, 2015 at 8:58 PM, NIkolai Marchenko enmarantis...@gmail.com wrote: The problem with your approach to qDebug is that it only works well where most of the text is in English and only individual letters are unicode. Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. 2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com: On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
27.07.2015, 11:01, "NIkolai Marchenko" enmarantis...@gmail.com: Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. And that is the situation, where aforementioned flag to change the behaviour in one place will be VERY useful. Unlike your suggestion of fixing source code everywhere line by line.And users genrally know if they need to consider the case you are arguing about or not. qDebug is just a macro so you may override it:#define qDebug() qDebug() unescape It's a bit more tricky if you use it as both function and output stream, but it's still solvable by single header include. P.S. As a Russian speaker I prefer Thiago's solution, I've faced numerous times with ambiguous output of qDebug and even 'a' vs 'а' problem (which led to lots of text.toHex()) and I'm happy that it's finally solved. On Mon, Jul 27, 2015 at 8:58 PM, NIkolai Marchenko enmarantis...@gmail.com wrote:The problem with your approach to qDebug is that it only works well where most of the text is in English and only individual letters are unicode. Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance.2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com:On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: "а", which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out?The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ,___Development mailing listDevelopment@qt-project.orghttp://lists.qt-project.org/mailman/listinfo/development --Ruslan Nigmatullin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 21:41:44 NIkolai Marchenko wrote: Yes,but this requires making override visible in every file of the project. We can solve this by having a global #ifdef that can change the setting. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
\u0440\u0430\u0443\u0440\u0430l.\u0441\u043em Can you really tell at a glance what exactly you are looking at here without consulting the oracle or, at least, reencoding it to readable format? On Mon, Jul 27, 2015 at 9:22 PM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 20:52:24 NIkolai Marchenko wrote: Let's make it harder, can you guesss that it even is a link to paypal website with qDebug() as you intend it to work without copypasting? раураl.соm ? Hint: only 2 letters here are in english. With qDebug() as it is in Qt 5.5, it would be quite evident. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 21:27:23 NIkolai Marchenko wrote: \u0440\u0430\u0443\u0440\u0430l.\u0441\u043em Oh, really? Note, I said, without copypasting to uncode converter. It's quite evident it's not a link to paypal.com because I can't see paypal there. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
Oh, but now you KNOW that it is a misspeled link to paypal. In real situation you won't have this forward knowlegde. On Mon, Jul 27, 2015 at 9:43 PM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 21:27:23 NIkolai Marchenko wrote: \u0440\u0430\u0443\u0440\u0430l.\u0441\u043em Oh, really? Note, I said, without copypasting to uncode converter. It's quite evident it's not a link to paypal.com because I can't see paypal there. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On 27 Jul 2015, at 21:14, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 21:50:15 NIkolai Marchenko wrote: Oh, but now you KNOW that it is a misspeled link to paypal. In real situation you won't have this forward knowlegde. The point is that I have all of the information I need. If I see a debug output saying: ClassName::openLink: trying to open \u0440\u0430\u0443\u0440\u0430l. \u0441\u043em Thiago what was the original cause for the change? Shouldn’t a change like this be argued in, rather than out? Andreas ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
Yes,but this requires making override visible in every file of the project. On Mon, Jul 27, 2015 at 9:34 PM, Ruslan Nigmatullin euroeles...@yandex.ru wrote: 27.07.2015, 11:01, NIkolai Marchenko enmarantis...@gmail.com: Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. And that is the situation, where aforementioned flag to change the behaviour in one place will be VERY useful. Unlike your suggestion of fixing source code everywhere line by line. And users genrally know if they need to consider the case you are arguing about or not. qDebug is just a macro so you may override it: #define qDebug() qDebug() unescape It's a bit more tricky if you use it as both function and output stream, but it's still solvable by single header include. P.S. As a Russian speaker I prefer Thiago's solution, I've faced numerous times with ambiguous output of qDebug and even 'a' vs 'а' problem (which led to lots of text.toHex()) and I'm happy that it's finally solved. On Mon, Jul 27, 2015 at 8:58 PM, NIkolai Marchenko enmarantis...@gmail.com wrote: The problem with your approach to qDebug is that it only works well where most of the text is in English and only individual letters are unicode. Once we venture into the territory of debugging 50%+ non-latin, it will be impossible to extract any useful information from debug output at a glance. 2015-07-27 20:25 GMT+03:00 Thiago Macieira thiago.macie...@intel.com: On Monday 27 July 2015 09:05:29 Thiago Macieira wrote: а, which was the attack vector for homograph sites like pаypal.com or pаypаl.com or раураl.com or other permutations of the letters. A control Question: which of the three is the legit website? And what method did you use to find out? The following is an excerpt from the Japanese Wikipedia homepage just so that KMail will encode the email in Base64 instead of quoted-printable: 1988年10月に競走馬としてデビュー。翌1989年にアメリカ三冠のうち二冠（ケンタッキーダービー、プリークネスステークス）、さらにブリーダーズカップ・クラ シックを勝つなどG1を5勝する活躍を見せ、エクリプス賞年度代表馬に選ばれた。1990年に右前脚の靭帯を痛めて競走馬を引退。引退後は日本で種牡馬となり、初年度 産駒がデビューした翌年の1995年から13年連続で日本のリーディングサイアーを獲得。さらに中央競馬における種牡馬にまつわる記録を次々と更新した。サンデーサイ レンスを起点とするサイアーラインは日本競馬界における一大勢力となり、サンデーサイレンス系とも呼ばれる。幼少期は見栄えのしない容貌ゆえに買い手がつかず、生命に かかわる事態に見舞われながら競走馬、さらに種牡馬として成功した生涯は童話『みにくいアヒルの子』に例えられる。 PS: if you've answered that you opened in the web browser, please send me your credit card number so I can credit you for $5. You may also want to talk to this prince who came into a fortune recently -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development , ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development -- Ruslan Nigmatullin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 21:50:15 NIkolai Marchenko wrote: Oh, but now you KNOW that it is a misspeled link to paypal. In real situation you won't have this forward knowlegde. The point is that I have all of the information I need. If I see a debug output saying: ClassName::openLink: trying to open \u0440\u0430\u0443\u0440\u0430l. \u0441\u043em I would know to inspect why that is an unexpected URL. Also note how QUrl would also mangle it and would show: QUrl( xn--l-7sba6dbr.xn--m-0tbi ) -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
\u0440\u0430\u0443\u0440\u0430l.\u0441\u043em Oh, really? Note, I said, without copypasting to uncode converter. On Mon, Jul 27, 2015 at 9:22 PM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 20:52:24 NIkolai Marchenko wrote: Let's make it harder, can you guesss that it even is a link to paypal website with qDebug() as you intend it to work without copypasting? раураl.соm ? Hint: only 2 letters here are in english. With qDebug() as it is in Qt 5.5, it would be quite evident. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 22:01:26 Andreas Aardal Hanssen wrote: On 27 Jul 2015, at 21:14, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 27 July 2015 21:50:15 NIkolai Marchenko wrote: Oh, but now you KNOW that it is a misspeled link to paypal. In real situation you won't have this forward knowlegde. The point is that I have all of the information I need. If I see a debug output saying: ClassName::openLink: trying to open \u0440\u0430\u0443\u0440\u0430l. \u0441\u043em Thiago what was the original cause for the change? Shouldn’t a change like this be argued in, rather than out? It was argued for and approved by the reviewers. It did not seem such a big deal, which is why it wasn't brought up to the list. Now we need to decide whether to go back, continue or change again. The whole story is as follows: For Qt 5.4, I changed QtTest printing of QStrings. See 579526cfec082679241548a0fca1ff9ba2c350a7: Make the printing of complex Unicode in a QString prettier This also has the advantage of not requiring the use of the locale codec. Quite an advantage if you're debugging the locale codec. But it's mostly so that we don't get question marks that hide the difference we were trying to locate. [ChangeLog][QtTest] QtTest now prints an escaped version of QStrings that failed to compare with QCOMPARE. That is, instead of converting non-printable characters to question marks, QtTest will print the Unicode representation of the character in question. This was a net benefit because, like the commit message says, it avoided the conversion from non-ASCII to question marks, which often hid the exact character you were trying to compare. I had a corresponding change for QByteArray, but it did not integrate in time for 5.4 branching, so landed in 5.5 as f5b609cac864e010085d5965b84832b2d85643e6 Make the printing of complex byte arrays prettier Similar to what we've done to QString, only we print each byte that is not ASCII as \OOO (octal representation). [ChangeLog][QtTest] QtTest now prints an escaped version of QByteArrays that failed to compare with QCOMPARE, instead of the hex dump. This was again a net benefit because the common case of QByteArray is to contain an arbitrarily-encoded human text. After I'd done that, we decided that we'd actually do the same for QDebug as it's used in often the same situations as QtTest. Also note that we changed from octal to hex at Ossi's insistence, which made the code a more complex in the process. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
On Monday 27 July 2015 00:25:18 NIkolai Marchenko wrote: (I am not entirely sure why I haven't received at least a notification of why this post has not appeared the first time I tried to send it, so here we go again. Btw, it was Thiago Macieira himself who actually suggested I mail this list) Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 In version 5.5 Thiago Macieira introduced a compatibility break with older qt versions. This break has come in the form of completely changing the way qDebug() prints non-english characters to console. I don't think qDebug() should guarantee any output, but your position might become a relevant one if logging output also changed. That is something that we shoudn't change, and since IIIRC, qDebug and logging use the same stream, you may have a point here. Just my 2ct, Marc -- Marc Mutz marc.m...@kdab.com | Senior Software Engineer KDAB (Deutschland) GmbH Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Backwards compatibiltiy break in Qt 5.5
My first instinct was suggesting MS to support Unicode in their console... but then I tried to reproduce the described issue on OS X and succeeded. qDebug() this is сообщение на русском; which I'd expect to be treated like a byte array, gives me this is сообщение на русском though qDebug() QString::fromUtf8(this is сообщение на русском); gives me this is \u0441\u043E\u043E\u0431\u0449\u0435\u043D\u0438\u0435 \u043D\u0430 \u0440\u0443\u0441\u0441\u043A\u043E\u043C A bit unexpected, isn't it? Regards, Konstantin 2015-07-27 12:20 GMT+03:00 Marc Mutz marc.m...@kdab.com: On Monday 27 July 2015 00:25:18 NIkolai Marchenko wrote: (I am not entirely sure why I haven't received at least a notification of why this post has not appeared the first time I tried to send it, so here we go again. Btw, it was Thiago Macieira himself who actually suggested I mail this list) Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 In version 5.5 Thiago Macieira introduced a compatibility break with older qt versions. This break has come in the form of completely changing the way qDebug() prints non-english characters to console. I don't think qDebug() should guarantee any output, but your position might become a relevant one if logging output also changed. That is something that we shoudn't change, and since IIIRC, qDebug and logging use the same stream, you may have a point here. Just my 2ct, Marc -- Marc Mutz marc.m...@kdab.com | Senior Software Engineer KDAB (Deutschland) GmbH Co.KG, a KDAB Group Company Tel: +49-30-521325470 KDAB - The Qt Experts ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
[Development] Backwards compatibiltiy break in Qt 5.5
(I am not entirely sure why I haven't received at least a notification of why this post has not appeared the first time I tried to send it, so here we go again. Btw, it was Thiago Macieira himself who actually suggested I mail this list) Hi! We (Russian Qt community) have a situation we could not resolve with Thiago Macieira in bug tracker, so have to ask for Chief Maintainer's attention. The bug in question is there: https://bugreports.qt.io/browse/QTBUG-47316 In version 5.5 Thiago Macieira introduced a compatibility break with older qt versions. This break has come in the form of completely changing the way qDebug() prints non-english characters to console. In essense - his fix to qDebug() will make it so that all non-english output from QString variable to qDebug() will be passed as a sequence of unicode escaped characters along the lines of: \u043F\u0440\u043E\u0432\u0435\u0440\u043A\u0430 043A\u0438\u0440\u0438\u043B\u043B\u0438\u0446\u044B Previously, in _some_ cases, but not reliably cross platform it would actually appear in console as readable output in user's language. I have to admit that Thiago is right about it being non-reliable contradicts qt's goal. Still... What Thiago completely disregards as unimportant - qDebug()'s been around for a long time. Even during times of qt4, users, through the use of setCodec functions, achieved readable output from qDebug in the console in their native language. Note that I say qt4. It's been around for almost a decade and there being no warning qDebug ever changing, it's become a standard on a lot of non-english installations, that actually allowed getting readable output, to use qDebug for tracing variables.. After all, opening debugger to inspect every little thing is very counterproductive. So here is where we are now, there are hundreds (probably thousands actually) codebases, where there is code like that: qDebug() someLabel; // skip qDebug() someVariableFromDatabase; // skip qDebug() someTextFromNetworkService; This is kinda normal and completely adhering to Kai Kohne's message he posted on youtube in qt-dev days video of using qDebug() to log program execution to console. Now, we arrive to Qt5.5. From here on now, instead of reading the trace of program execution everyone using the old code will be subjected to strings like \u043F\u0440\u043E\u0432\u0435\u0440\u043A\u0430 043A\u0438\u0440\u0438\u043B\u043B\u0438\u0446\u044B Note, that there are dozens to hundreds of such lines per application. And in SUCH a format, the trace posted above becomes completely useless for the purposes of debugging, because people, generaly, don't have unicode parser plugged into their brains. Thiago argues that his way, user will always get exactly which type of A the symbol is. Our point - now user will first need to pass the output of qDebug to unicode converter to even understand what's he looking at. Now, again, I do not dispute that qDebug in a way it was being used was correct or reliable across all platforms or allowed to check exactly which symbol was contained in a string. But BETTER way to fix that, in my opinion, and all of the Russian users who are already aware of the problem is to introduce a qDebug().escape() or something along those lines, in those RARE cases when instead of just tracing variables, user NEEDS to see unicode. (note I say Russian, because that is the community I come from, but in reality the situation is the same for greek, japanese, whatever non-english) Here we have a conflict of interest: Do we fix qDebug() in a way it is SUPPOSED to work, breaking code for users, or do we make a different flavour of qDebug that actually works as we intended it to, cross platform and escapes everything and let people who need it switch to that? Personally, from what I know of application development, breaking codebase is always bad. breaking a LARGE codebase, is horrible. So, I am at a loss of why Thiago decided to go the first way. We've tried to convince him he underestimates the potential harm to customers of Qt, but he refuses to listen insisting that if ppl used qDebug() incorrect, they are not entitled to their code not breaking at any moment he seems fit to fix it. But the point is - how were we to know we were using qDebug() incorrectly, when nothing along those lines were EVER posted on qDebug()'s help page? People relied on the way it's worked before. No one warned peope, about the change. The change is not even backwards compatible to older qt versions. As Thiago refuses to admit there is a problem, we have to ask the authorities behind Qt to look into this. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
