Re: [fpc-devel] RTL and Unicode filenames operations.
Marco van de Voort het geskryf: For the same reason why the Unix rtl is not a Windows emulation. We are writing a multiplatform compiler and rtl, not an emulation. If only that was true... I can highlight a few places where Windows emulation is used rather that pure multi-platform implementation. :-/ Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Tue, 23 Mar 2010, Graeme Geldenhuys wrote: Marco van de Voort het geskryf: For the same reason why the Unix rtl is not a Windows emulation. We are writing a multiplatform compiler and rtl, not an emulation. If only that was true... I can highlight a few places where Windows emulation is used rather that pure multi-platform implementation. :-/ Most cases are out of Delphi/TP compatibility reasons. We cannot throw that overboard. But nothing stops people from developing an alternate RTL, free of windows-isms, and preferably 100% OOP :-) Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Tue, Mar 21, 2010 at 2:01 PM, Marco van de Voort http://bugs.freepascal.org/view.php?id=15795 wrote: I'll reiterate my opinion that first a decision about what the working stringtype of the RTL will be. IMHO there is no decent solution till there is a real utf-8 type (read cpnewstr) The whole reason for the package, is not to wait for cpnewstr being implemented. That's the point that package's interface uses UnicodeString, rather than ambiguous String type (that's currently system dependent). Note that the recent update does not tackle my main gripe at all, the main units interface is still UTF-16 on Unix, it just translates it to UTF-8 in the backend. Yes, it does so and i see no problems here. The package also converts UTF-16 to ansi encoding for Win9x. Converting string from UTF16 to UTF8 (or any other encoding) is not much time penalty comparing to the time of the file operation itself. Btw, the current RTL implementation also forces a user to convert a string type to utf8, to work correctly on Unixes for unicode names. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: On Tue, Mar 21, 2010 at 2:01 PM, Marco van de Voort http://bugs.freepascal.org/view.php?id=15795 wrote: I'll reiterate my opinion that first a decision about what the working stringtype of the RTL will be. IMHO there is no decent solution till there is a real utf-8 type (read cpnewstr) The whole reason for the package, is not to wait for cpnewstr being implemented. That's the point that package's interface uses UnicodeString, rather than ambiguous String type (that's currently system dependent). Yes. Which is no problem for the package. But it is a reason to not include it. Note that the recent update does not tackle my main gripe at all, the main units interface is still UTF-16 on Unix, it just translates it to UTF-8 in the backend. Yes, it does so and i see no problems here. The package also converts UTF-16 to ansi encoding for Win9x. Converting string from UTF16 to UTF8 (or any other encoding) is not much time penalty comparing to the time of the file operation itself. Sure. But that still doesn't mean UTF-16 is the ideal choice. Btw, the current RTL implementation also forces a user to convert a string type to utf8, to work correctly on Unixes for unicode names. Currently there is no unicode support. The point is rather that I don't want to commit an halve one, and be stuck with it (and all kinds of packages just using it) forever. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Martin Schreiber said: Yes, it does so and i see no problems here. The package also converts UTF-16 to ansi encoding for Win9x. Converting string from UTF16 to UTF8 (or any other encoding) is not much time penalty comparing to the time of the file operation itself. Agreed. The MSEgui file utils do the same with good results. If I understand right, Marco insists that on Linux utf-8 should be used for file operations because it is the native file name encoding on Linux. On Linux and FreeBSD yes, everything is UTF-8. Then using UTF-16 types here and there should not be necessary. Even if it is not a performance killer. don't know how that can be achieved BTW. On Windows and Mac utf-16 is then native filename encoding so I don't understand why the API can't be standardised to utf-16. For the same reason why the Unix rtl is not a Windows emulation. We are writing a multiplatform compiler and rtl, not an emulation. The Delphi stuff causes already enough windowsisms. Let's not add one. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On 21 Mar 2010, at 17:29, Martin Schreiber wrote: On Windows and Mac utf-16 is then native filename encoding On Mac it's utf-8, not utf-16. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Sun, Mar 21, 2010 at 7:53 PM, Jonas Maebe jonas.ma...@elis.ugent.be wrote: On Mac it's utf-8, not utf-16. At least, not as sequence of bytes, see test1.pas of the ufiles package. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Any hints about the package? http://bugs.freepascal.org/view.php?id=15795 thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Hello, The patch has been updated: http://bugs.freepascal.org/view.php?id=15795 (is it possible to allow the reporter to delete its files in FPC bug tracker?) The library is now RTL encoding independent. Actually it's configurable, so, if RTL string encoding changes, only a 2-3 lines in the library should be changed. On Wed, Feb 17, 2010 at 1:10 PM, Mattias Gaertner nc-gaert...@netcologne.de wrote: The implementation is not only about RTL encoding, but system environment as well, since it depends upon Windows *W function to be available. But who cares about 9x these days?! :) Notes: - Comparison is wrong under OS X, see lcl FileUtil.pas - missing a note that files with broken UTF8 chars can not be read under Linux. Maybe add a note where the system encoding file routines are. - ExtractRelativePath works Delphi compatible, but always case insensitive, so not useful under Linux. All 3 issues should be fixed by now. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Tue, Mar 9, 2010 at 1:49 PM, Marco van de Voort mar...@stack.nl wrote: The library is now RTL encoding independent. Actually it's configurable, so, if RTL string encoding changes, only a 2-3 lines in the library should be changed. I don't know, but I can delete anything but the most recent if you wish? yes, please. just keep: ufiles-03032010-clean.zip afaik, in Lazarus it's allowed to delete attached files for the reporter. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Felipe Monteiro de Carvalho schrieb: 2010/2/17 Flávio Etrusco flavio.etru...@gmail.com: I've read somewhere that Windows ANSI functions support utf-8? (despite the name) lol!!! You read from someone deeply missinformed =D Switch the console to a font supporting unicode, execute chcp 65001 and you can output utf-8. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Vincent Snijders said: dmitry boyarintsev schreef: Reported: http://bugs.freepascal.org/view.php?id=15795 It's up to FPC team to accept or reject the package. Thanks, I will monitor the issue to see the outcome. Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. This sets a precedent. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Wed, Feb 17, 2010 at 11:15 AM, Marco van de Voort mar...@stack.nl wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. Vincent, is there a place in CCR for the run-time package? thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
No comment on the idea of converting dynamically typed string constants at the first touch in realtime ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
dmitry boyarintsev schreef: On Wed, Feb 17, 2010 at 11:15 AM, Marco van de Voort mar...@stack.nl wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. Vincent, is there a place in CCR for the run-time package? Yes. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Wed, Feb 17, 2010 at 11:15 AM, Marco van de Voort mar...@stack.nl wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. This sets a precedent. Could you please resolve the issue with won't be accepted (or no changes required)? The implementation is not only about RTL encoding, but system environment as well, since it depends upon Windows *W function to be available. But who cares about 9x these days?! :) thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Wed, 17 Feb 2010 12:34:35 +0300 dmitry boyarintsev skalogryz.li...@gmail.com wrote: On Wed, Feb 17, 2010 at 11:15 AM, Marco van de Voort mar...@stack.nl wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. This sets a precedent. Could you please resolve the issue with won't be accepted (or no changes required)? The implementation is not only about RTL encoding, but system environment as well, since it depends upon Windows *W function to be available. But who cares about 9x these days?! :) Notes: - Comparison is wrong under OS X, see lcl FileUtil.pas - missing a note that files with broken UTF8 chars can not be read under Linux. Maybe add a note where the system encoding file routines are. - ExtractRelativePath works Delphi compatible, but always case insensitive, so not useful under Linux. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Thanks Mattias, i'll do proper changes to file name comparison functions. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Mattias Gaertner said: Could you please resolve the issue with won't be accepted (or no changes required)? The implementation is not only about RTL encoding, but system environment as well, since it depends upon Windows *W function to be available. But who cares about 9x these days?! :) Notes: - Comparison is wrong under OS X, see lcl FileUtil.pas - missing a note that files with broken UTF8 chars can not be read under Linux. Maybe add a note where the system encoding file routines are. - ExtractRelativePath works Delphi compatible, but always case insensitive, so not useful under Linux. On the implementation front I had some doubts about charinset. It doesn't seem to be locale aware and only treat the bottom 128 chars. Is this compatible? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: RTLString. This sets a precedent. Could you please resolve the issue with won't be accepted (or no changes required)? It's not my decision alone. The whole problem with this unicode string is that there is no definitive roadmap, and the utf-8 ansistring is not ready. The implementation is not only about RTL encoding, but system environment as well, since it depends upon Windows *W function to be available. True, but I'm not going to commit anything Windows only till the plans for the system environment are clear. On all relevant platforms. But who cares about 9x these days?! :) I don't. But that is not even a problem; FPC has prepared long for this by having and UNICODE define in most windows headers. (afaik Peter already did when he translated windows unit originally), but specially the older bits are not test and probably full of bugs. In theory, we could have a win32u target that sets UNICODE and switch the default of most functions to unicode, like D2009+. It is probably going to happen that way anyway, sooner or later. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Wednesday 17 February 2010 09:15:53 Marco van de Voort wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. Aren't filenames on Linux an array of bytes without any encoding by definition? Are you sure there are no Linux running with other system encoding than utf-8 today? How are imported filesystems with other encodings (for example Samba shares) handled? Martin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Martin Schreiber said: On Wednesday 17 February 2010 09:15:53 Marco van de Voort wrote: Reject as far as I'm concerned. I'd like to see the RTL being in the system encoding. (IOW UTF-8 on at least FreeBSD/Linux) using an abstracted RTLString. Aren't filenames on Linux an array of bytes without any encoding by definition? I never saw that. The encoding is sometimes a bit opague yes. It will be a practical detail which exact 1-byte encoding to choose for rtlstring (cp_utf8, rawbytestring or default). Whatever works best. Are you sure there are no Linux running with other system encoding than utf-8 today? No. Does it matter? How are imported filesystems with other encodings (for example Samba shares) handled? Don't see a relevance here. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Wed, Feb 17, 2010 at 4:10 AM, dmitry boyarintsev skalogryz.li...@gmail.com wrote: Reported: http://bugs.freepascal.org/view.php?id=15795 It's up to FPC team to accept or reject the package. thanks, dmitry I've read somewhere that Windows ANSI functions support utf-8? (despite the name) -Flávio ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
2010/2/17 Flávio Etrusco flavio.etru...@gmail.com: I've read somewhere that Windows ANSI functions support utf-8? (despite the name) lol!!! You read from someone deeply missinformed =D -- Felipe Monteiro de Carvalho ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
dmitry boyarintsev schreef: Reported: http://bugs.freepascal.org/view.php?id=15795 It's up to FPC team to accept or reject the package. Thanks, I will monitor the issue to see the outcome. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: Any hints on these bugs? http://bugs.freepascal.org/view.php?id=12923 http://bugs.freepascal.org/view.php?id=15642 // patch proposed to workaround RTL limitation in LCL This is mostly concern of Windows OS. Moders Unix doesn't have the problem, usually having UTF8 as their's Ansi codepage. Would it be possible to fix the problem in RTL? by adding UnicodeString (WideString) version of file operation functions? What would you do on Unix? Any solution must be platform-wide. First the UnicodeString (which is not Widestring) should be finished, and some global decisions how to implement unicode system wide should be taken. Ant other simplier way than switching compiler itself to Unicode or other special string type? Temporarily have some helper functions in the LCL. FPC doesn't support unicode for all operations atm, while Lazarus does. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
dmitry boyarintsev wrote: Would it be possible to fix the problem in RTL? by adding UnicodeString (WideString) version of file operation functions? Until the cpstrnew branch in FPC is complete, I guess the answer is No. Alternatively you can introduce helper RTL functions. We have done similar in fpGUI project (fpg_utils.pas unit), having functions like fpgFileExists(), fpgFindFirst(), fpgFindNext() etc. Use them instead of the standard RTL functions, and they will try and handle Unicode file system access correctly on each platform. Regards, - Graeme - -- fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://opensoft.homeip.net/fpgui/ ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Mon, Feb 1, 2010 at 11:01 AM, Marco van de Voort mar...@stack.nl wrote: What would you do on Unix? Any solution must be platform-wide. I'm not talking about removing Ansi functions. I'm talking about the possibility of adding Unicode function equivalents. For Unix unicode functions would simply call Ansi functions, converting Unicode to AnsiString. thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: What would you do on Unix? Any solution must be platform-wide. I'm not talking about removing Ansi functions. I'm talking about the possibility of adding Unicode function equivalents. For Unix unicode functions would simply call Ansi functions, converting Unicode to AnsiString. Which requires libc, and *nix supports UTF-8 directly. The point is more or less to come up with some logical system, not just random additions when something is missing. And IMHO it is still to early, first the unicodestring support must be finished, and then a course has to be laid out which of the thousands of strings operations to convert. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: Now, i see, the technical solution is not easy. The point is more or less to come up with some logical system, not just random additions when something is missing. FPC will be forced to catch-up with D2009, anyway. Yes, but D2009/d2010 are still windows only. This is probably going to change with Codegear's crosscompiling plans (expected in +/- august/september). Otoh they'll probably keep everything UTF-16, since that is what they are used to. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On 02/01/2010 09:46 AM, Marco van de Voort wrote: D2009/d2010 are still windows only. Do you think there is any general problem in using D2009 strings with dynamic encoding with Unix ? On the first view I don't see one. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Michael Schnell said: D2009/d2010 are still windows only. Do you think there is any general problem in using D2009 strings with dynamic encoding with Unix ? On the first view I don't see one. In general I prefer to use each platform using its native encoding. Some commercial unices are also using utf-16 afaik, since they haven't switched their TTY to UTF-8 yet. To be investigated. I lean towards RTLString=utf8string; and RTLString=unicodestring on the respective platforms, and then express the RTL in RTLString. Since both types can be passed to another for the most, aside for a few VAR functions this is transparent, and leaves the decision to the user what his main string type is, and how many conversions he wants to allow or avoid. Moreover, it allows for passing UTF-8 strings to system functions on freebsd/linux in the base system (which could be benificial for the smaller utils and the compiler itself), since then just passing it on does not require cwstring We are not Codegear, for which everything but Windows is an afterthought. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Op Mon, 1 Feb 2010, schreef Marco van de Voort: In our previous episode, Michael Schnell said: D2009/d2010 are still windows only. Do you think there is any general problem in using D2009 strings with dynamic encoding with Unix ? On the first view I don't see one. In general I prefer to use each platform using its native encoding. Some commercial unices are also using utf-16 afaik, since they haven't switched their TTY to UTF-8 yet. To be investigated. I lean towards RTLString=utf8string; and RTLString=unicodestring on the respective platforms, and then express the RTL in RTLString. No, RTLString=ansistring on Linux and most Unix platforms. Unix is encoding agnostic so if you want to use the native string type you must simple use old fashioned ansistring. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Dani?l Mantione said: Some commercial unices are also using utf-16 afaik, since they haven't switched their TTY to UTF-8 yet. To be investigated. I lean towards RTLString=utf8string; and RTLString=unicodestring on the respective platforms, and then express the RTL in RTLString. No, RTLString=ansistring on Linux and most Unix platforms. Unix is encoding agnostic so if you want to use the native string type you must simple use old fashioned ansistring. This means crossplatform unicode support goes out of the window. I don't like that. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On 02/01/2010 10:27 AM, Marco van de Voort wrote: We are not Codegear, for which everything but Windows is an afterthought. Right, But exactly this seems to ask for dynamically encoded strings allowing to do the user code as agnostic of the system encoding as possible while automatic conversions are made whenever necessary, but avoided whenever possible. (Which as we all know can't be provided in an absolutely perfect way: in some cases the agnostic user code will not work as expected and in some other cases performance might be reduced due to multiple automatic conversions). -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Michael Schnell said: Right, But exactly this seems to ask for dynamically encoded strings allowing to do the user code as agnostic of the system encoding as possible while automatic conversions are made whenever necessary, but avoided whenever possible. Both string types will probably be fully supported on unicode supporting platforms. The point is what the RTL uses. It makes sense to take the encoding closest to the system, to at least allow minimal conversion scenarios. (Which as we all know can't be provided in an absolutely perfect way: in some cases the agnostic user code will not work as expected and in some other cases performance might be reduced due to multiple automatic conversions). Yes. Note that what I say goes for the RTL, since obviously, in the core library, speed and reducing dependancies is the most important. I don't mind if some XML lib or so is is always either UTF-16 or UTF-8 (depending on pedigree). ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Op Mon, 1 Feb 2010, schreef Marco van de Voort: In our previous episode, Dani?l Mantione said: Some commercial unices are also using utf-16 afaik, since they haven't switched their TTY to UTF-8 yet. To be investigated. I lean towards RTLString=utf8string; and RTLString=unicodestring on the respective platforms, and then express the RTL in RTLString. No, RTLString=ansistring on Linux and most Unix platforms. Unix is encoding agnostic so if you want to use the native string type you must simple use old fashioned ansistring. This means crossplatform unicode support goes out of the window. I don't like that. It is a consequence if using the native string type. Otherwiser you can just as well use UTF-16 everywhere. Daniël___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On 02/01/2010 10:53 AM, Marco van de Voort wrote: It makes sense to take the encoding closest to the system, to at least allow minimal conversion scenarios. Of course ! If the RTL is done with dynamically encoded strings you would need to take care that anything string value that neither comes from the GUI framework (and thus is encoded in a way the RTL designer might know but should ignore as largely as possible to produce as portable code as possible) nor comes from the user code (the RTL designers can't know which encoding they use) is handled appropriately. I suppose most of this is string constants. If possible, those should be allocated with the system encoding at compile time. I feel that the runtime performance overhead necessary for doing the RTL with dynamically encoded strings is not too big, provided the RTL itself does not force automatic conversions (e.g. by inappropriately encoded string constants) . -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, Michael Schnell said: I feel that the runtime performance overhead necessary for doing the RTL with dynamically encoded strings is not too big, provided the RTL itself does not force automatic conversions (e.g. by inappropriately encoded string constants) . Daniel touches the main problem. The string type is compiletime, while the standard encoding is runtime. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On 02/01/2010 12:26 PM, Marco van de Voort wrote: Daniel touches the main problem. The string type is compiletime, while the standard encoding is runtime. I do see this (that is why I said If possible, even if I have no idea about what impact this has. Maybe it could be possible to find a way to invent a one-time conversion for this class of strings: The first time they are touched they get converted and the next time they are already in place correctly. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Mon, Feb 1, 2010 at 11:09 AM, Graeme Geldenhuys graemeg.li...@gmail.com wrote: Alternatively you can introduce helper RTL functions. We have done similar in fpGUI project (fpg_utils.pas unit), having functions like fpgFileExists(), fpgFindFirst(), fpgFindNext() etc. Use them instead of the standard RTL functions, and they will try and handle Unicode file system access correctly on each platform. Can Unicode file operations become a FCL package? So it would be RTL implementation independent. As well as always available for FPC user, without need of external GUI packages (i.e. fpGUI, LCL or mse) thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
In our previous episode, dmitry boyarintsev said: in fpGUI project (fpg_utils.pas unit), having functions like fpgFileExists(), fpgFindFirst(), fpgFindNext() etc. Use them instead of the standard RTL functions, and they will try and handle Unicode file system access correctly on each platform. Can Unicode file operations become a FCL package? No, since this is part of the RTL, and needs to stay that way out of delphi compat. More importantly, the stuff in the FCL depends on the RTL, so stuffing in the FCL is useless since it would need primitives in the RTL to operate on. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
On Mon, Feb 1, 2010 at 3:20 PM, Marco van de Voort mar...@stack.nl wrote: More importantly, the stuff in the FCL depends on the RTL, so stuffing in the FCL is useless since it would need primitives in the RTL to operate on. Does it mean, if i provide a FCL package with: unit UnicodeFileUtils; function FindFirst(const Name: UnicodeString) ... function IncludeTrailngPathDelimiter(const path: UnicodeString) type TFileStreamUnicode = class(THandleStream) constructor Create(const AFileName: UnicodeString) etc... it would be rejected? thanks, dmitry ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] RTL and Unicode filenames operations.
Marco van de Voort schreef: That is not my call, but I won't commit it. Because all this stuff must be maintained when real solutions come later. if Lazarus needs a quick fix, they should keep it in own SVN. Or use the units from msegui or use the units from tiopf or use the units from (any other group that needed a quick fix). I just do not like that everybody needs to implement this on their own. I would vote to add them in 2.5.1 and you can remove them if you have implemented these things in the RTL. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
