Re: [fpc-devel] CMem allocator memory alignment experiment

2014-11-23 Thread Florian Klaempfl
Am 19.11.2014 12:32, schrieb Karoly Balogh (Charlie/SGR):
 Hi,
 
 On Wed, 19 Nov 2014, Jonas Maebe wrote:
 
 Since the RTL's allocator is documented to align to 16 bytes

 Where?
 
 Ok, that's actually a very good question. :) I didn't find it anywhere,
 except some earlier ML/forum posts revealed by Google.
 
 However, in practice it still seems to align to 16 bytes, and I asked
 several people (compiler, RTL, Lazarus developers) during the FPC/Lazraus
 conference last weekend in the Netherlands and the consensus was, it's
 known the heap manager aligns to 16 bytes, it's designed to do that, and
 in general it's a feature, which should be documented if it's not.
 
 But yeah, everyone appended but better ask Jonas. :)

From heap.inc:

  { we need to align the user pointers to 8 byte at least for
mmx/sse and doubles on sparc, align to 16 bytes }

Cryptic though :)
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


[fpc-devel] DragonFly BSD x86-64 support --- patch sets ready for commit to trunk

2014-11-23 Thread John Marino
Hello,
I've applied the DragonFly BSD support to the trunk.  The 2.6.4 compiler
can bootstrap it and it builds itself and helloworld.

I've split it into 3 patch sets for the convenience of the committer:

http://leaf.dragonflybsd.org/~marino/df-fpc/dfly-support-existing-most.diff

http://leaf.dragonflybsd.org/~marino/df-fpc/dfly-support-existing-packages.diff

http://leaf.dragonflybsd.org/~marino/df-fpc/dfly-support-rtl-dragonfly.diff

The first diff are modifications to existing files minus anything in
packages.  The second diff are modifications to existing files in
packages.  The third diff is the entire rtl/dragonfly directory minus a
makefile.

After committing,
  1) all makefile needs need regenerated (and rtl/dragonfly/Makefile
generated)
  2) utils/fpcm/fpcmake.inc needs to be regenerated


Notes:
  A) The change to compiler/cfileutl.pas is not intended to be
permanent, but until I figure out why dircache isn't working, it's needed.
  B) rtl/bsd/ostypes: fixed constants S_IRUSR, S_IWUSR, S_IRGRP,
S_IWGRP, S_IXGRP.  These were actually incorrect.
  C) rtl/bsd/ostypes: changed S_IFMT and friends to octal.  They were
correct but octal is a lot easier to check than decimal -- headers are
written in octal.


I would be very elated if this could get committed prior to the next
release as Free Pascal 2.6.4 is already available to via a derivative of
the FreeBSD Ports Collection (called DPorts)

Thanks to all who provided advise to get me here.
Regards,
John
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] DragonFly BSD x86-64 support --- patch sets ready for commit to trunk

2014-11-23 Thread Jonas Maebe
On 23/11/14 12:04, John Marino wrote:

   B) rtl/bsd/ostypes: fixed constants S_IRUSR, S_IWUSR, S_IRGRP,
 S_IWGRP, S_IXGRP.  These were actually incorrect.

Not for Darwin at least:

/* Read, write, execute/search by owner */
#define S_IRWXU 700 /* [XSI] RWX mask for owner */
#define S_IRUSR 400 /* [XSI] R for owner */
#define S_IWUSR 200 /* [XSI] W for owner */
#define S_IXUSR 100 /* [XSI] X for owner */
/* Read, write, execute/search by group */
#define S_IRWXG 070 /* [XSI] RWX mask for group */
#define S_IRGRP 040 /* [XSI] R for group */
#define S_IWGRP 020 /* [XSI] W for group */
#define S_IXGRP 010 /* [XSI] X for group */
/* Read, write, execute/search by others */
#define S_IRWXO 007 /* [XSI] RWX mask for other */
#define S_IROTH 004 /* [XSI] R for other */
#define S_IWOTH 002 /* [XSI] W for other */
#define S_IXOTH 001 /* [XSI] X for other */

Since these are octal numbers, they correspond to the existing (binary
notation) constants in the RTL afaics.


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] DragonFly BSD x86-64 support --- patch sets ready for commit to trunk

2014-11-23 Thread John Marino
On 11/23/2014 13:13, Jonas Maebe wrote:
 On 23/11/14 12:04, John Marino wrote:
 
   B) rtl/bsd/ostypes: fixed constants S_IRUSR, S_IWUSR, S_IRGRP,
 S_IWGRP, S_IXGRP.  These were actually incorrect.
 
 Not for Darwin at least:
 
 Since these are octal numbers, they correspond to the existing (binary
 notation) constants in the RTL afaics.
 

You're right, I got burned by oct/hex.

John

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] DragonFly BSD x86-64 support --- patch sets ready for commit to trunk

2014-11-23 Thread John Marino
On 11/23/2014 13:28, John Marino wrote:
 On 11/23/2014 13:13, Jonas Maebe wrote:
 On 23/11/14 12:04, John Marino wrote:

   B) rtl/bsd/ostypes: fixed constants S_IRUSR, S_IWUSR, S_IRGRP,
 S_IWGRP, S_IXGRP.  These were actually incorrect.

 Not for Darwin at least:

 Since these are octal numbers, they correspond to the existing (binary
 notation) constants in the RTL afaics.

 
 You're right, I got burned by oct/hex.
 
 John
 

I have updated the link in place to remove that change:
http://leaf.dragonflybsd.org/~marino/df-fpc/dfly-support-existing-most.diff


thanks for resolving that issue,
John
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Jonas Maebe
On 18/11/14 19:51, Mattias Gaertner wrote:
 On Tue, 18 Nov 2014 18:17:25 +0100
  Thanks, but there is no UTF-8 RTL.
 That's what I thought too a week ago.
 FPC 2.7 made an old dream come true. :)

Nevertheless, please stop calling it the UTF-8 RTL. It will give people
the wrong impression, because there is no and may well never be an UTF-8
RTL. It is no way whatsoever specific to UTF-8, nor will it
automagically work correctly with UTF-8 encoded data in all scenarios
(e.g. when DefaultSystemCodePageCP_UTF8, or when the file system API
doesn't support all characters that can be encoded using UTF-8, ...) or
on all platforms (some platforms don't even support multiple code pages).


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Mattias Gaertner
On Sun, 23 Nov 2014 14:26:08 +0100
Jonas Maebe jonas.ma...@elis.ugent.be wrote:

 On 18/11/14 19:51, Mattias Gaertner wrote:
  On Tue, 18 Nov 2014 18:17:25 +0100
   Thanks, but there is no UTF-8 RTL.
  That's what I thought too a week ago.
  FPC 2.7 made an old dream come true. :)
 
 Nevertheless, please stop calling it the UTF-8 RTL.
 It will give people the wrong impression, because there is no and may well 
 never be an UTF-8
 RTL. It is no way whatsoever specific to UTF-8, nor will it
 automagically work correctly with UTF-8 encoded data in all scenarios
 (e.g. when DefaultSystemCodePageCP_UTF8, or when the file system API
 doesn't support all characters that can be encoded using UTF-8, ...) or
 on all platforms (some platforms don't even support multiple code pages).

Hmm, maybe you have a point there.
It is similar to normal RTL and RTL with CurrencyString:='Euro'.
When I say Euro RTL it does not work correctly in all
scenarios, but I'm sure some people think it does.

I doubt that I can come up with a name, that everyone understands
correct, but I have to call it something.

What about RTL with UTF8 as default?


Mattias
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Michael Van Canneyt



On Sun, 23 Nov 2014, Mattias Gaertner wrote:


On Sun, 23 Nov 2014 14:26:08 +0100
Jonas Maebe jonas.ma...@elis.ugent.be wrote:


On 18/11/14 19:51, Mattias Gaertner wrote:

On Tue, 18 Nov 2014 18:17:25 +0100

Thanks, but there is no UTF-8 RTL.

That's what I thought too a week ago.
FPC 2.7 made an old dream come true. :)


Nevertheless, please stop calling it the UTF-8 RTL.
It will give people the wrong impression, because there is no and may well 
never be an UTF-8
RTL. It is no way whatsoever specific to UTF-8, nor will it
automagically work correctly with UTF-8 encoded data in all scenarios
(e.g. when DefaultSystemCodePageCP_UTF8, or when the file system API
doesn't support all characters that can be encoded using UTF-8, ...) or
on all platforms (some platforms don't even support multiple code pages).


Hmm, maybe you have a point there.
It is similar to normal RTL and RTL with CurrencyString:='Euro'.
When I say Euro RTL it does not work correctly in all
scenarios, but I'm sure some people think it does.

I doubt that I can come up with a name, that everyone understands
correct, but I have to call it something.

What about RTL with UTF8 as default?


That seems wrong, since UTF-8 is not the default on Windows ?

Let's try to understand first why do you insist on the UTF-8 in the name ?

Maybe UTF-8 aware is better, if you really want the UTF-8 in the name.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


[fpc-devel] Building of FPC trunk broken on Darwin

2014-11-23 Thread Joao Morais


Hello list!

Trunk is broken on Darwin i386 since rev 29085 with the following message:

=
/Applications/Xcode.app/Contents/Developer/usr/bin/make echotime
Start now 13:25:24
/usr/bin/diff ppc3 ppc386
Binary files ppc3 and ppc386 differ
make[2]: *** [cycle] Error 2
make[1]: *** [compiler_cycle] Error 2
make: *** [build-stamp.i386-darwin] Error 2
=

Using both 2.6.2 or 2.6.4 as the starting compiler. Rev 29084 is nice.

Joao Morais

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Building of FPC trunk broken on Darwin

2014-11-23 Thread Jonas Maebe
On 23/11/14 16:31, Joao Morais wrote:
 Trunk is broken on Darwin i386 since rev 29085 with the following message:
 
 =
 /Applications/Xcode.app/Contents/Developer/usr/bin/make echotime
 Start now 13:25:24
 /usr/bin/diff ppc3 ppc386
 Binary files ppc3 and ppc386 differ
 make[2]: *** [cycle] Error 2
 make[1]: *** [compiler_cycle] Error 2
 make: *** [build-stamp.i386-darwin] Error 2
 =
 
 Using both 2.6.2 or 2.6.4 as the starting compiler. Rev 29084 is nice.

When reporting problems, always mention all steps that you took and
extra parameters you specified. In this case, the problem only occurs if
you use a variant of the -g parameter (which I only know because someone
else reported it before): http://bugs.freepascal.org/view.php?id=27051


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Marco van de Voort
In our previous episode, Mattias Gaertner said:
  doesn't support all characters that can be encoded using UTF-8, ...) or
  on all platforms (some platforms don't even support multiple code pages).
 
 Hmm, maybe you have a point there.
 It is similar to normal RTL and RTL with CurrencyString:='Euro'.
 When I say Euro RTL it does not work correctly in all
 scenarios, but I'm sure some people think it does.
 
 I doubt that I can come up with a name, that everyone understands
 correct, but I have to call it something.
 
 What about RTL with UTF8 as default?

It is not, and neither is it aware. It is only so when you set
defaultsystemcodepage to utf8 and ignore the problems that causes.  That is
not intended functionality.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Building of FPC trunk broken on Darwin

2014-11-23 Thread Florian Klaempfl
Am 23.11.2014 16:50, schrieb Jonas Maebe:
 On 23/11/14 16:31, Joao Morais wrote:
 Trunk is broken on Darwin i386 since rev 29085 with the following message:

 =
 /Applications/Xcode.app/Contents/Developer/usr/bin/make echotime
 Start now 13:25:24
 /usr/bin/diff ppc3 ppc386
 Binary files ppc3 and ppc386 differ
 make[2]: *** [cycle] Error 2
 make[1]: *** [compiler_cycle] Error 2
 make: *** [build-stamp.i386-darwin] Error 2
 =

 Using both 2.6.2 or 2.6.4 as the starting compiler. Rev 29084 is nice.
 
 When reporting problems, always mention all steps that you took and
 extra parameters you specified. In this case, the problem only occurs if
 you use a variant of the -g parameter (which I only know because someone
 else reported it before): http://bugs.freepascal.org/view.php?id=27051

What about a OVERRIDECOMPARECHECK parameter for the makefiles?

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Michael Van Canneyt



On Sun, 23 Nov 2014, Marco van de Voort wrote:


In our previous episode, Mattias Gaertner said:

doesn't support all characters that can be encoded using UTF-8, ...) or
on all platforms (some platforms don't even support multiple code pages).


Hmm, maybe you have a point there.
It is similar to normal RTL and RTL with CurrencyString:='Euro'.
When I say Euro RTL it does not work correctly in all
scenarios, but I'm sure some people think it does.

I doubt that I can come up with a name, that everyone understands
correct, but I have to call it something.

What about RTL with UTF8 as default?


It is not, and neither is it aware. It is only so when you set
defaultsystemcodepage to utf8 and ignore the problems that causes.  That is
not intended functionality.


It is UTF-8 aware in the sense that now you can safely define a 1-byte 
string containing UTF-8 and expect the compiler to do the necessary 
conversions more or less automatically when needed, such as when 
passing it to a routine that expects widestrings.


That was simply impossible prior to the work of Jonas and others.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Mattias Gaertner
On Sun, 23 Nov 2014 16:31:10 +0100 (CET)
Michael Van Canneyt mich...@freepascal.org wrote:

[...]
 That seems wrong, since UTF-8 is not the default on Windows ?
 
 Let's try to understand first why do you insist on the UTF-8 in the name ?
 
 Maybe UTF-8 aware is better, if you really want the UTF-8 in the name.

Maybe there is a misunderstanding. At least I can't follow you here.

I started the thread about ParamStr, which only supports the system
codepage. I would like to improve it so that it supports
DefaultSystemCodepage. Or at least add an Unicode version of
ParamStr.
The reason I need this, is that when I set
DefaultSystemCodepage to CP_UTF8, most RTL functions work with UTF8 as
default.
I also want to update the UTF-8 wiki pages. For example If you use the
insert name here then  For this I need a short name.

Some people has called the RTL with UnicodeString the Unicode RTL,
and the Ansistring RTL with system codepage Ansi RTL.
I thought UTF8 RTL is analog, short and unambiguous.
Obviously I was wrong.

(And, yes, I know, that all three names Unicode|Ansi|UTF8 RTL are
not 100% correct from a technical point of view.)

Mattias
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said:
  It is not, and neither is it aware. It is only so when you set
  defaultsystemcodepage to utf8 and ignore the problems that causes.  That is
  not intended functionality.

To make things clear:
I meant in the way Mattias proposed, continueing making the default
string type utf8 on Windows. Utf8string is fine, but limited.

That basically perpetuates the current hack, just slightly more elegant. 

That being said, FPC is not offering a complete solution yet, and 2.8.x will be 
a
transition version. (or one in a series of transitional versions)

 It is UTF-8 aware in the sense that now you can safely define a 1-byte 
 string containing UTF-8 and expect the compiler to do the necessary 
 conversions more or less automatically when needed, such as when 
 passing it to a routine that expects widestrings.

 That was simply impossible prior to the work of Jonas and others.

It is certainly progress, and hopefully a start on at least removing the OS
specific aspects of Lazarus *utf8 routines.

I would at least try to get as far as possible without changing
defaultsystemcodepage (etc) though.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Michael Van Canneyt



On Sun, 23 Nov 2014, Marco van de Voort wrote:


In our previous episode, Michael Van Canneyt said:

It is not, and neither is it aware. It is only so when you set
defaultsystemcodepage to utf8 and ignore the problems that causes.  That is
not intended functionality.


To make things clear:
I meant in the way Mattias proposed, continueing making the default
string type utf8 on Windows. Utf8string is fine, but limited.

That basically perpetuates the current hack, just slightly more elegant.


I don't know about that.

It is a perfectly valid solution to choose UTF-8 as the unicode encoding of 
choice.
Now at least the compiler makes this a more viable decision. 
So from this point on, I would no longer speak of a hack.



That being said, FPC is not offering a complete solution yet, and 2.8.x will be 
a
transition version. (or one in a series of transitional versions)


Correct.


It is UTF-8 aware in the sense that now you can safely define a 1-byte
string containing UTF-8 and expect the compiler to do the necessary
conversions more or less automatically when needed, such as when
passing it to a routine that expects widestrings.



That was simply impossible prior to the work of Jonas and others.


It is certainly progress, and hopefully a start on at least removing the OS
specific aspects of Lazarus *utf8 routines.


Fully agreed.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] Building of FPC trunk broken on Darwin

2014-11-23 Thread Jonas Maebe
On 23/11/14 17:21, Florian Klaempfl wrote:
 Am 23.11.2014 16:50, schrieb Jonas Maebe:
 When reporting problems, always mention all steps that you took and
 extra parameters you specified. In this case, the problem only occurs if
 you use a variant of the -g parameter (which I only know because someone
 else reported it before): http://bugs.freepascal.org/view.php?id=27051
 
 What about a OVERRIDECOMPARECHECK parameter for the makefiles?

Definitely not. I think the only solution at that level would be to
compare the generated object files rather than the entire binaries (like
gcc does), but FPC's build system does not easily lend itself to that.

Another solution would be to not support building the compiler with the
RTL of the previous release, or to introduce a policy that the compiler
itself must not depend on RTL features introduced since the last release
(although this is hard to enforce/test automatically, since make cycle
always uses the new RTL).


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Marco van de Voort
In our previous episode, Mattias Gaertner said:
  Let's try to understand first why do you insist on the UTF-8 in the name ?
  
  Maybe UTF-8 aware is better, if you really want the UTF-8 in the name.
 
 Maybe there is a misunderstanding. At least I can't follow you here.
 
 I started the thread about ParamStr, which only supports the system
 codepage. I would like to improve it so that it supports
 DefaultSystemCodepage. Or at least add an Unicode version of
 ParamStr.

And the 2-byte unicode version exists, in unit uuchar.  (the objpas of
$mode delphiunicode).  For now, simply make a utf8 wrapper that returns an
utf8string.

 Some people has called the RTL with UnicodeString the Unicode RTL,
 and the Ansistring RTL with system codepage Ansi RTL.

Well, that's what Windows calls them that (-W are unicode, -A are ansi).
Delphi follows that terminology (D2007- being ansi, D2009+ being unicode).

 I thought UTF8 RTL is analog, short and unambiguous.
 Obviously I was wrong.

The RTL leans somewhat to the prefered encoding on each target, so 1-byte on
*nix and 2-byte on Windows. That means that there is no real utf8 support on
 Windows other than generic codepage aware string type (that goes for all
1-byte encodings).

Setting defaultsystemcodepage will make all autoconversions to ansistring(0)
return utf8, so also when calling e.g. unit windows functions. I think it
would be very wise to be careful with that, and have an extensive trial
period.

You might want to keep the current -utf8 routines as mere codepage
correcting wrappers.

 (And, yes, I know, that all three names Unicode|Ansi|UTF8 RTL are
 not 100% correct from a technical point of view.)

The filesystem routines are now encoding agnostic. But that assumes you use
a type that the compiler knows the associated encoding.

But filesystem routines are only a small part of the system libraries.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said:
  To make things clear:
  I meant in the way Mattias proposed, continueing making the default
  string type utf8 on Windows. Utf8string is fine, but limited.
 
  That basically perpetuates the current hack, just slightly more elegant.
 
 I don't know about that.
 
 It is a perfectly valid solution to choose UTF-8 as the unicode encoding
 of choice.

So now we support changing defaultsystemcodepage formally by endusers? I was
not made aware of that.

And aside from formal status, that is IMHO a hack, and an incompatible (to
both old and new delphi) to start with.

 Now at least the compiler makes this a more viable decision. 
 So from this point on, I would no longer speak of a hack.

It moves removes the conversions from a few RTL routines, but keeps the
insert conversion here barrier to all other interfacing to the system
firmly intact. And I already mentioned it is incompatible.

I assume this will also make default I/O utf8 etc, which makes no sense.
 
  That was simply impossible prior to the work of Jonas and others.
 
  It is certainly progress, and hopefully a start on at least removing the
  OS specific aspects of Lazarus *utf8 routines.
 
 Fully agreed.

(I would suggest keeping them for now, just make them simple codepage
wrappers in case the defaultsystemcodepage doesnt' work out).
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Michael Van Canneyt



On Sun, 23 Nov 2014, Marco van de Voort wrote:


In our previous episode, Michael Van Canneyt said:

To make things clear:
I meant in the way Mattias proposed, continueing making the default
string type utf8 on Windows. Utf8string is fine, but limited.

That basically perpetuates the current hack, just slightly more elegant.


I don't know about that.

It is a perfectly valid solution to choose UTF-8 as the unicode encoding
of choice.


So now we support changing defaultsystemcodepage formally by endusers? I was
not made aware of that.


Whether or not to change defaultsystemcodepage is entirely irrelevant to the 
discussion: the fact that now lazarus *can* use a real UTF8String type.


One does not necessarily imply the other.

How they choose to use this UTF8 type (i.e. how to set the various variables 
controlling the behaviour of the RTL), is IMHO a second design decision, which 
comes AFTER the decision to use UTF8..


But anyway, it's not my intention to argue about this. There is progress, that 
is what matters.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Jonas Maebe
On 23/11/14 17:46, Marco van de Voort wrote:
 So now we support changing defaultsystemcodepage formally by endusers? I was
 not made aware of that.

There is the SetMultiByteConversionCodePage() routine in the interface
of the system unit that does exactly that. It's kind of strange to have
such a routine if using it is not supported.

 And aside from formal status, that is IMHO a hack, and an incompatible (to
 both old and new delphi) to start with.

That routine comes from Delphi and does the same there.

 It moves removes the conversions from a few RTL routines, but keeps the
 insert conversion here barrier to all other interfacing to the system
 firmly intact. And I already mentioned it is incompatible.
 
 I assume this will also make default I/O utf8 etc, which makes no sense.

It depends on what you mean by default I/O. It won't change standard
input/output to UTF-8 on Windows, as every textrec has its own codepage
settings and those for standard input/output are initialised with
concrete code page numbers on startup (see OpenStdIO in
rtl/inc/textrec.inc, and Win32GetStandardCodePage() in rtl/win/syswin.inc.

On Unix, it will change the code page of stdio, because there the stdio
textrecs are initialised with the value of defaultsystemcodepage
(DefaultGetStandardCodePage() in rtl/inc/ustrings.inc), which is still
CP_ACP at that point. We could fix this by adding code to the
initialisation of cwstring to set the codepage of those textrecs to the
concrete code pages though (which we probably should do anyway).


Jonas


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel


Re: [fpc-devel] UTF8 RTL

2014-11-23 Thread Jonas Maebe
On 23/11/14 17:23, Mattias Gaertner wrote:
 I started the thread about ParamStr, which only supports the system
 codepage. I would like to improve it so that it supports
 DefaultSystemCodepage. Or at least add an Unicode version of
 ParamStr.

We need both, as Marco mentioned. The main issue is that on Unix
platforms, to fix it properly you must interpret the parameters
according to DefaultSystemCodePage at startup, or in case a separate
widestring manager is used then right after that one was initialised.
Maybe we need another Default*CodePage variable which indicates the
real system code page...

 I also want to update the UTF-8 wiki pages. For example If you use the
 insert name here then  For this I need a short name.

If you use FPC 2.7.1 or later then ...


Jonas
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel