Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 12/29/2013 10:57 AM, Michael Van Canneyt wrote: We're way past that stage. Sorry, but IMHO freezing the current (supposedly DXE compatible) state does not make much sense. To decently support creating fully portable applications for multiple OSes, TStrings and its siblings (e.g.

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 12/29/2013 10:57 AM, Michael Van Canneyt wrote: We're way past that stage. Sorry, but IMHO freezing the current (supposedly DXE compatible) state does not make much sense. To decently support creating fully portable applications for

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 12/29/2013 07:26 PM, Hans-Peter Diettrich wrote: My view on RawByteString: My view on RawByteString originally had been that it can hold strings of any encoding and the software that gets such a variable (or function argument) can detect the actual encoding and behave accordingly. Thus

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 12/30/2013 12:53 PM, Jy V wrote: A quick note: the new LLVM Delphi compiler forbid the use of AnsiString and AnsiChar, Yuck ! Forking the platforms in that incompatible way a really ugly move of Delphi's. Hopefully fpc (and Lazarus) is able to continue it's path to real source code

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 10:41 AM, Michael Van Canneyt wrote: We know this. But that is stage 2. Sounds great ! Could you elaborate those plans ? I fear that releasing stage 1 to the public might introduce another source of incompatibility. Explanation: My horror-scenario when trying to convince

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 01/07/2014 10:41 AM, Michael Van Canneyt wrote: We know this. But that is stage 2. Sounds great ! Could you elaborate those plans ? There is not much to say. There will be 2 sets of units: * Unicode * AnsiString for unicode, there is no

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 11:22 AM, Michael Van Canneyt wrote: There is not much to say. There will be 2 sets of units: * Unicode * AnsiString for unicode, there is no problem, since everything is 2 bytes (ignoring some exotic codepoints here) so string encoding does not apply. for ansistring,

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Jonas Maebe
On 07 Jan 2014, at 11:22, Michael Van Canneyt wrote: There will be 2 sets of units: * Unicode * AnsiString Or they may be integrated to a large extent. This has not yet been decided. In any case, creating two separate sets of units is a good step regardless of what happens eventually,

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 11:28 AM, Jonas Maebe wrote: creating two separate sets of units is a good step regardless of what happens eventually, since we need the functionality of both in any case. This sounds like State 3 is flickering at the horizon t as well :-) :-) -Michael

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 01/07/2014 11:22 AM, Michael Van Canneyt wrote: There is not much to say. There will be 2 sets of units: * Unicode * AnsiString for unicode, there is no problem, since everything is 2 bytes (ignoring some exotic codepoints here) so string

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 11:48 AM, Michael Van Canneyt wrote: TRawByteString is what you need. AFAI was told by DXE users, this is not true. That is why i did not test this yet. But as you state otherwise I will check into that. But anyway this does not help, as long as the RTL (especially TStrings,

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 01/07/2014 11:48 AM, Michael Van Canneyt wrote: TRawByteString is what you need. AFAI was told by DXE users, this is not true. That is why i did not test this yet. But as you state otherwise I will check into that. But anyway this does not

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 12:27 PM, Michael Van Canneyt wrote: if you have use-case 1 (which I doubt, since it is not possible even today) then you're better off using unicodestring anyway. My argumentation is driven by the experience (some myself, a huge lot by my colleagues) with doing embedded

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 01/07/2014 12:27 PM, Michael Van Canneyt wrote: if you have use-case 1 (which I doubt, since it is not possible even today) then you're better off using unicodestring anyway. My argumentation is driven by the experience (some myself, a huge

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 01:05 PM, Michael Van Canneyt wrote: This is a mistake. You should use TByteArray for that. Old pre-unicode Delphi also handles this type. Of course I do know that this sometimes is recommended, but (with pre-Unicode Delphi) I don't see any point in not using String, which is

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Michael Schnell wrote: On 01/07/2014 01:05 PM, Michael Van Canneyt wrote: This is a mistake. You should use TByteArray for that. Old pre-unicode Delphi also handles this type. Of course I do know that this sometimes is recommended, but (with pre-Unicode Delphi) I

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 01:24 PM, Michael Van Canneyt wrote: But like I said: Do not expect us to adapt the RTL to suit any inappropriate use of strings. Like I said: I don't ! (With the requested behavior this comes for free as a side-effect.) But I do expect decent handling of locale based 1 Byte

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Hans-Peter Diettrich
Michael Van Canneyt schrieb: If you want a TStrings that can hold strings which may differ in their encoding (i.e. strings[0] has a different encoding from strings[1]) then you'll be left in the cold. Just an idea: What if FPC adds another encoding, similar to RawByteString ($), but

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Hans-Peter Diettrich
Jy V schrieb: A quick note: the new LLVM Delphi compiler forbid the use of AnsiString and AnsiChar, (declared in the unit AnsiString.pas, you cannot use this unit anyway), The compiler supports AnsiStrings, but these are hidden for *mobile* targets. There exists a hack to enable AnsiString

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 02:24 PM, Hans-Peter Diettrich wrote: The compiler supports AnsiStrings, but these are hidden for *mobile* targets. Any reason for this strangeness ? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 03:35 PM, Hans-Peter Diettrich wrote: What if FPC adds another encoding, similar to RawByteString ($), but without the Delphi quirks? Or simply fix the RawByteString flaws in the *Ansi* compiler and RTL? +1 (in fact I elaborated on that in some older Threads here) 1) In a

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Craig Peterson
On 1/7/2014 8:35 AM, Hans-Peter Diettrich wrote: Sorting (TStringList) eventually should ignore the dynamic encoding, i.e. work on a strictly binary (byte-by-byte) base. Why would that be desirable? If you sort a *string* list you'd expect it to do a string based sort, and more than likely, a

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Craig Peterson
The compiler supports AnsiStrings, but these are hidden for *mobile* targets. Any reason for this strangeness ? They're using the mobile compiler as an opportunity to break backwards compatibility and push the language in the directions they want to go. A single, 0-based string type, automatic

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Jonas Maebe
On 07 Jan 2014, at 15:35, Hans-Peter Diettrich wrote: 1) In a discussion in the Embarcadero groups it turned out that, in an assignment of a RawByteString to another AnsiString type, the Delphi compiler should (but does not) check and eventually convert the string to the static encoding

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Van Canneyt
On Tue, 7 Jan 2014, Craig Peterson wrote: The compiler supports AnsiStrings, but these are hidden for *mobile* targets. Any reason for this strangeness ? They're using the mobile compiler as an opportunity to break backwards compatibility and push the language in the directions they want

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 04:24 PM, Craig Peterson wrote: They're using the mobile compiler as an opportunity to break backwards compatibility and push the language in the directions they want to go. A single, 0-based string type, automatic reference counting, no with, etc. Apparently the developers

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Michael Schnell
On 01/07/2014 04:33 PM, Jonas Maebe wrote: but more regarding descendent classes in existing third party (= user) source code that override methods using AnsiString parameters. Automatic type conversion should be able to handle this (as the new type would preserve and thus dynamically know

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Sven Barth
Am 07.01.2014 17:00 schrieb Michael Schnell mschn...@lumino.de: - there have been decent discussions in multiple forums showing that that classes with automatic reference counting are dangerous I like the idea of ARC classes. In fact I'm trying to develop an implementation that is fully

Re: [fpc-devel] Encoded AnsiString

2014-01-07 Thread Hans-Peter Diettrich
Jonas Maebe schrieb: On 07 Jan 2014, at 15:35, Hans-Peter Diettrich wrote: 2) The stupid conversion to CP_ACP in an assignment *to* an RawByteString should be dropped. This applies in detail to the assignment to *function results*. The conversion does not happen for all assignments, it only

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Hans-Peter Diettrich
Paul Ishenin schrieb: 30.12.2013 9:07, Hans-Peter Diettrich пишет: Do you think that FPC should really reproduce all this inconsistent behaviour? Who would test or even specify the compatible behaviour, when every new variation will result in more unexpected results? IMO it's much easier to

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Michael Van Canneyt
On Mon, 30 Dec 2013, Paul Ishenin wrote: 30.12.2013 9:07, Hans-Peter Diettrich пишет: Do you think that FPC should really reproduce all this inconsistent behaviour? Who would test or even specify the compatible behaviour, when every new variation will result in more unexpected results? IMO

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Jy V
On Sun, Dec 29, 2013 at 7:26 PM, Hans-Peter Diettrich drdiettri...@aol.comwrote: var a: AnsiString; u: UTF8String; function cpy(s: RawByteString):RawByteString; begin Result := s; end; a := cpy(u); //now a has encoding UTF-8! Here the XE compiler omits the conversion of the RawByteString

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Paul Ishenin
30.12.2013 18:33, Michael Van Canneyt пишет: So how one can help at this stage: 1. Check related FPC tests and write new for the missing cases. 2. Compare FPC and Delphi RTL classes which had beed adjusted in Delphi during the unicodestring move and check whether something minor can be added

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Michael Van Canneyt
On Mon, 30 Dec 2013, Paul Ishenin wrote: 30.12.2013 18:33, Michael Van Canneyt пишет: So how one can help at this stage: 1. Check related FPC tests and write new for the missing cases. 2. Compare FPC and Delphi RTL classes which had beed adjusted in Delphi during the unicodestring move

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Paul Ishenin
30.12.2013 20:25, Michael Van Canneyt wrote: But the experiments for ansi/unicode RTL are already in trunk. Do you plan to take them out ? By experiments I ment having 2 (or 1.5) RTL with unicode + ansi classes which we planned to solve using namespaces. What good is having the unicode

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Marco van de Voort
In our previous episode, Paul Ishenin said: I don't think that this is a good idea, it means that e.g. TStrings.SaveToFile() or TFileStream.Create() is still crippled. Better bite the bullet. This is what I wanted to test in feb/march. This will also mean that we will release 2.8 much

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said: But the experiments for ansi/unicode RTL are already in trunk. Do you plan to take them out ? and: What good is having the unicode string support if none of the classes or units make use of it ? Limited. But 2.8.x is not all about

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Michael Van Canneyt
On Mon, 30 Dec 2013, Marco van de Voort wrote: In our previous episode, Michael Van Canneyt said: But the experiments for ansi/unicode RTL are already in trunk. Do you plan to take them out ? and: What good is having the unicode string support if none of the classes or units make use of it

Re: [fpc-devel] Encoded AnsiString

2013-12-30 Thread Paul Ishenin
30.12.2013 20:59, Michael Van Canneyt wrote: That was my idea. If it turns out to be really impossible, we can still do as Paul suggests, but if it works, I would attempt to release 2.8 with dotted units. In each case, the ground work will be done in a branch, so as not to irreversibly

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Michael Van Canneyt
On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: Inspired by the current Lazarus discussion I'd like to learn more about the current state of the implementation of the new AnsiStrings. In case nothing has be done yet, I'd suggest to extend TAnsiRec by the new codePage and elemSize fields

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Hans-Peter Diettrich
Michael Van Canneyt schrieb: On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: Inspired by the current Lazarus discussion I'd like to learn more about the current state of the implementation of the new AnsiStrings. In case nothing has be done yet, I'd suggest to extend TAnsiRec by the new

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Michael Van Canneyt
On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: Michael Van Canneyt schrieb: On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: Inspired by the current Lazarus discussion I'd like to learn more about the current state of the implementation of the new AnsiStrings. In case nothing has be

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Jonas Maebe
On 29 Dec 2013, at 16:25, Hans-Peter Diettrich wrote: I've heard that a mix of encodings converts the (concatenated) output (RawByteString?) to CP_ACP, with possible losses. Is this correct? The code page of ansistrings concatenations is the code page of the result to which this

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Hans-Peter Diettrich
Michael Van Canneyt schrieb: On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: This will be combined with the dotted unit filenames, to be Delphi 2010+ compatible. sigh.sigh How do I create source files for use with both versions? What do you mean by this statement ? I'm not familiar

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Hans-Peter Diettrich
Jonas Maebe schrieb: The code page of ansistrings concatenations is the code page of the result to which this concatenation is assigned/converted. For rawbytestring, this code page is CP_ACP per Delphi compatibility. This does not match my experience with Delphi XE :-( Can you give an Delphi

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Sven Barth
On 29.12.2013 17:53, Hans-Peter Diettrich wrote: Michael Van Canneyt schrieb: On Sun, 29 Dec 2013, Hans-Peter Diettrich wrote: This will be combined with the dotted unit filenames, to be Delphi 2010+ compatible. sigh.sigh How do I create source files for use with both versions? What do

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Sven Barth
On 29.12.2013 19:26, Hans-Peter Diettrich wrote: Jonas Maebe schrieb: The code page of ansistrings concatenations is the code page of the result to which this concatenation is assigned/converted. For rawbytestring, this code page is CP_ACP per Delphi compatibility. This does not match my

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Hans-Peter Diettrich
Sven Barth schrieb: [...] This was tested using Delphi XE (it might not compile though as I've just rewritten the code from memory as the original is on a different computer) Thanks, the code is okay, and it produces the expected results. The compiler also warns about a downcast from

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Hans-Peter Diettrich
Jonas Maebe schrieb: I'm inclined to add a global boolean variable to the system unit that allows changing this behaviour so that it uses CP_UTF8 instead in such cases (defaulting to false, for Delphi compatibility). In practice, setting it to true shouldn't cause problems even with virtually

Re: [fpc-devel] Encoded AnsiString

2013-12-29 Thread Paul Ishenin
30.12.2013 9:07, Hans-Peter Diettrich пишет: Do you think that FPC should really reproduce all this inconsistent behaviour? Who would test or even specify the compatible behaviour, when every new variation will result in more unexpected results? IMO it's much easier to do it right, and fix the

[fpc-devel] Encoded AnsiString

2013-12-28 Thread Hans-Peter Diettrich
Inspired by the current Lazarus discussion I'd like to learn more about the current state of the implementation of the new AnsiStrings. In case nothing has be done yet, I'd suggest to extend TAnsiRec by the new codePage and elemSize fields (words). These can be zero for now, so that the