Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Al-Khanji Louai
It's not a platform bug. It's an application/framework (Qt) bug. Unix paths are just a byte array, where certain bytes have special meaning (mostly just '/'). Passing around those byte arrays from/to platform functions will always work correctly. The problem is that Qt tries to interpret

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Kuba Ober
On Tue, Oct 7, 2014 at 11:41 AM, Thiago Macieira thiago.macie...@intel.com wrote: On Tuesday 07 October 2014 10:38:47 Kuba Ober wrote: Just to be very clear: it is currently impossible to make a truly portable file management utility with Qt’s core APIs. Why? Because it will simply ignore

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Thiago Macieira
On Thursday 09 October 2014 01:57:05 Marc Mutz wrote: The value lies _also_ in being able to iterate over weird filenames (where weird simply means plugging in a USB stick into an otherwise UTF-8-only system). Mind you these two facts: * USB mass-storage devices came into being after Linux

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Thiago Macieira
On Thursday 09 October 2014 02:46:49 Kuba Ober wrote: The problem manifests itself almost anytime you plug in a small USB memory stick that has localized file names on FAT-16 into any Unix system You mean a pre-Windows 95 floppy? Are you sure you have a floppy drive? Everything since Windows

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Julien Blanc
On 09/10/2014 09:27, Thiago Macieira wrote: The only one that poses trouble are ISO-9660 CD-ROMs that have Rock Ridge extensions for Unix attributes and longer file names. Do people still have CD drives? People also have zip files, which unfortunately may have various encoding in them, since

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-09 Thread Thiago Macieira
On Thursday 09 October 2014 09:55:36 Julien Blanc wrote: On 09/10/2014 09:27, Thiago Macieira wrote: The only one that poses trouble are ISO-9660 CD-ROMs that have Rock Ridge extensions for Unix attributes and longer file names. Do people still have CD drives? People also have zip files,

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Thiago Macieira
On Tuesday 07 October 2014 23:31:11 Christoph Feck wrote: On Tuesday 07 October 2014 23:19:23 Tony Van Eerd wrote: The problem is serious enough, indeed, that Python 3 has resorted to a hack where they use a private Unicode range to encode the bytes between 128 and 255 in strings that

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Thiago Macieira
On Wednesday 08 October 2014 09:44:56 Tomasz Siekierda wrote: On Tuesday 07 October 2014 23:31:11 Christoph Feck wrote: We are using the same hack in KDE4Libs, but it relies on QFile::setDecodingFunction. Unfortunately, this function is no longer available in Qt5, so in a few years, we will

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Oswald Buddenhagen
On Mon, Oct 06, 2014 at 05:21:45PM +0200, Thiago Macieira wrote: File names that cannot be decoded using the locale codec are considered filesystem corruption and are silently dropped. They won't appear in directory listings. This was discussed to exhaustion in Qt 5's development process.

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Thiago Macieira
On Wednesday 08 October 2014 12:10:07 Oswald Buddenhagen wrote: On Mon, Oct 06, 2014 at 05:21:45PM +0200, Thiago Macieira wrote: File names that cannot be decoded using the locale codec are considered filesystem corruption and are silently dropped. They won't appear in directory listings.

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Oswald Buddenhagen
On Wed, Oct 08, 2014 at 12:16:55PM +0200, Thiago Macieira wrote: The proposal by Kuba would be to have everything use a QFileName or similar class. So why not just use QUrl? yes, i think kuba's proposal is just as unpractical as the qurl one. i'd have a much more favorable view towards a

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Marc Mutz
Hi Julien, On Tuesday 07 October 2014 14:30:59 Julien Blanc wrote: However, i agree that changing this would : * break a lot of code No, it cannot, if, as I propose, it's added to Qt 5. * permit only to solve really lower level / corner case issues The value lies _also_ in being able to

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-08 Thread Konstantin Ritt
2014-10-09 3:57 GMT+04:00 Marc Mutz marc.m...@kdab.com: Hi Julien, On Tuesday 07 October 2014 14:30:59 Julien Blanc wrote: However, i agree that changing this would : * break a lot of code No, it cannot, if, as I propose, it's added to Qt 5. * permit only to solve really lower level /

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Tomasz Siekierda
This was discussed to exhaustion in Qt 5's development process. The conclusion is to remain at status quo since there is no good, technical solution. I’d think that the solution could be to use a dedicated class for file names, perhaps with a base class for uninterpreted platform strings.

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Marc Mutz
Hi Kuba, Your criticisms are completely valid, and the conclusions you draw from them are, too. The problems Thiago lists make this a daunting task, but mostly not because of complexity, but of sheer volume of code that needs to be modified. I believe it's worth it, but most of us here lack

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Kuba Ober
On Oct 7, 2014, at 8:30 AM, Julien Blanc julien.bl...@nmc-company.com wrote: On 07/10/2014 12:11, Tomasz Siekierda wrote: For file paths, I feel QString is really enough. Changing it to something else because of a few corner cases seems like an overkill to me. We already have a lot of

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Alejandro Exojo
El Tuesday 07 October 2014, Tomasz Siekierda escribió: For file paths, I feel QString is really enough. Changing it to something else because of a few corner cases seems like an overkill to me. Just for the sake of documenting the issue and pointing to this thread if future questions arise:

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Thiago Macieira
On Tuesday 07 October 2014 10:38:47 Kuba Ober wrote: Just to be very clear: it is currently impossible to make a truly portable file management utility with Qt’s core APIs. Why? Because it will simply ignore all file names that it can’t decode when iterating the directory, and it won’t be able

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Tony Van Eerd
The problem is serious enough, indeed, that Python 3 has resorted to a hack where they use a private Unicode range to encode the bytes between 128 and 255 in strings that fail normal decoding. I think that putting this hack into QString is unthinkable, and the concept of a platform string

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-07 Thread Christoph Feck
On Tuesday 07 October 2014 23:19:23 Tony Van Eerd wrote: The problem is serious enough, indeed, that Python 3 has resorted to a hack where they use a private Unicode range to encode the bytes between 128 and 255 in strings that fail normal decoding. I think that putting this hack into

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-06 Thread Thiago Macieira
On Monday 06 October 2014 11:12:57 Kuba Ober wrote: Thus, how does Qt deal with a directory listing with such “invalid” file names? Do they survive the round-trip through a QString and QDirIterator? Would it be worthwhile to tackle this issue in a better fashion (whatever it might be) for Qt

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-06 Thread Kuba Ober
On Oct 6, 2014, at 11:21 AM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 06 October 2014 11:12:57 Kuba Ober wrote: Thus, how does Qt deal with a directory listing with such “invalid” file names? Do they survive the round-trip through a QString and QDirIterator? Would it be

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-06 Thread Thiago Macieira
On Monday 06 October 2014 13:30:29 Kuba Ober wrote: This was discussed to exhaustion in Qt 5's development process. The conclusion is to remain at status quo since there is no good, technical solution. I’d think that the solution could be to use a dedicated class for file names, perhaps

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-06 Thread Kuba Ober
On Oct 6, 2014, at 2:45 PM, Thiago Macieira thiago.macie...@intel.com wrote: On Monday 06 October 2014 13:30:29 Kuba Ober wrote: This was discussed to exhaustion in Qt 5's development process. The conclusion is to remain at status quo since there is no good, technical solution. I’d think

Re: [Development] The life of a file name and other possibly mal-encoded strings on non-Windows systems

2014-10-06 Thread Konstantin Ritt
2014-10-06 23:53 GMT+04:00 Kuba Ober k...@mareimbrium.org: On Oct 6, 2014, at 2:45 PM, Thiago Macieira thiago.macie...@intel.com wrote: How do you pass it on the command-line? Mind you, QProcess takes a QStringList for arguments. It look as if we’d need something like QPlatformString